Deep learning reading group

A repository to keep track of papers in the reading group at University of Oslo.

Comments to the papers can be found under Issues.

Sessions

#13 09.03.17

Papers: - Asynchronous Methods for Deep Reinforcement Learning: https://arxiv.org/pdf/1602.01783.pdf

#12 23.02.17

Papers: - Human-level control through deep reinforcement learning: https://storage.googleapis.com/deepmind-data/assets/papers/DeepMindNature14236Paper.pdf

Additional: - Blog by Karpathy http://karpathy.github.io/2016/05/31/rl/, though he use policy gradient instead of Q learning. - Blog by DeepMind https://deepmind.com/blog/deep-reinforcement-learning/. Some videos from the paper, and some improvement and achievements.

#11: 24.11.16

Papers: - End-To-End Memory Networks: (https://papers.nips.cc/paper/5846-end-to-end-memory-networks.pdf) - Hybrid computing using a neural network with dynamic external memory: (http://www.nature.com/nature/journal/v538/n7626/pdf/nature20101.pdf)

#10: 13.10.16

Papers: - Neural Turing Machines (https://arxiv.org/pdf/1410.5401v2.pdf)

Additional: - Alex Graves talks about the Neural Turing Machines https://www.youtube.com/watch?v=_H0i0IhEO2g

#9: 29.09.16

Papers: - Generative Adversarial Nets (https://arxiv.org/pdf/1406.2661v1.pdf) - Adversarial Autoencoders (http://arxiv.org/pdf/1511.05644v2.pdf)

#8: 15.09.16

Papers: - Variational Inference: A Review for Statisticians (https://arxiv.org/pdf/1601.00670v3.pdf) - Tutorial on Variational Autoencoders (https://arxiv.org/pdf/1606.05908v2.pdf)

Other things discussed during the group: - Morphing faces : http://vdumoulin.github.io/morphing_faces/ - Wavenet : https://deepmind.com/blog/wavenet-generative-model-raw-audio/

#7: 01.09.16

Papers: - Deep Kalman Filters (https://arxiv.org/pdf/1511.05121v2.pdf)

#6: 11.05.16

Papers: - Distilling the Knowledge in a Neural Network (http://arxiv.org/abs/1503.02531 and https://www.youtube.com/watch?v=EK61htlw8hY) - Do Deep Nets Really Need to be Deep? (https://papers.nips.cc/paper/5484-do-deep-nets-really-need-to-be-deep.pdf)

Suggested readings: - Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding (http://arxiv.org/abs/1510.00149) - SqueezeNet: AlexNet-level Accuracy with 50x Fewer Parameters and <0.5MB Model Size (http://arxiv.org/pdf/1602.07360v3.pdf)

#5: 27.04.16

Papers: - Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning (http://arxiv.org/pdf/1506.02142v4.pdf)

Suggested reading: Dropout as a Bayesian Approximation: Appendix (http://arxiv.org/pdf/1506.02157v4.pdf)

#4: 14.04.16

Papers: - Generating Sequences With Recurrent Neural Networks (http://arxiv.org/pdf/1308.0850v5.pdf)

Suggested reading: Parts of the article is summarised in the lecture (https://www.youtube.com/watch?v=-yX1SYeDHbg)

#3: 16.03.16

Papers: - Detecting Methane Outbreaks from Time Series Data with Deep Neural Networks (http://link.springer.com/chapter/10.1007/978-3-319-25783-9_42#page-1) - Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition (http://www.mdpi.com/1424-8220/16/1/115/html)

#2: 24.02.16

Papers: - Natural Language Understanding with Distributed Representation (http://arxiv.org/pdf/1511.07916v1.pdf)

Suggested reading: I also recommend the following two blog posts, they are both very good introductions to RNN and LSTM models.

http://colah.github.io/posts/2015-08-Understanding-LSTMs/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/

#1: 12.02.16

Papers: - A Primer on Neural Network Models for Natural Language Processing (http://u.cs.biu.ac.il/~yogo/nnlp.pdf)

Suggested reading: Natural Language Understanding with Distributed Representation (http://arxiv.org/pdf/1511.07916v1.pdf) (Ch. 1-3, 5 and perhaps 7).