https://github.com/alexhernandezgarcia/course-content-dl

https://github.com/alexhernandezgarcia/course-content-dl

Repository

https://github.com/alexhernandezgarcia/course-content-dl/blob/main/

# Neuromatch Academy Deep Learning (NMA-DL) syllabus

The content should primarily be accessed from our new ebook: https://deeplearning.neuromatch.io/ [under continuous development]

----

*August 2-20, 2021*

**Objectives**: Gain hands-on, code-first experience with deep learning theories, models, and skills that are useful for applications and for advancing science. We focus on how to decide which problems can be tackled with deep learning, how to determine what model is best, how to best implement a model, how to visualize / justify findings, and how neuroscience can inspire deep learning. And throughout we emphasize the ethical use of DL. 

Please check out [expected prerequisites here](https://github.com/NeuromatchAcademy/precourse-dl)!

**Confirmed speakers**: 

* [Amita Kapoor (U Delhi)](https://www.dramitakapoor.com/)
* [Anima Anandkumar (Caltech)](http://tensorlab.cms.caltech.edu/users/anima/)
* [Aude Oliva (MIT)](http://olivalab.mit.edu/audeoliva.html)
* [Chelsea Finn (Stanford)](https://ai.stanford.edu/~cbfinn/)
* [Emily Denton (Google)](https://cephaloponderer.com/)
* [Geoffrey Hinton (U Toronto)](http://www.cs.toronto.edu/~hinton/)
* [Joao Sedoc (NYU)](https://scholar.google.com/citations?user=vv355NgAAAAJ&hl=en)
* [Kyunghyun Cho (NYU)](https://kyunghyuncho.me/)
* [Melanie Mitchell (Santa Fe Institute)](https://melaniemitchell.me/)
* [Yann LeCun (Facebook)](http://yann.lecun.com/)
* [Yoshua Bengio (MILA)](https://yoshuabengio.org/)

# Course materials

----

Coming soon... stay tuned...

# Course outline

----

## Week 1: The basics

----

### Mon, August 2, 2021:  Intro to DL academy

*coordinated by [Konrad Kording (U Penn)](http://koerding.com/)*

**Description** Welcome, introduction to Google Colab, meet and greet, a bit of DL history, DL basics and introduction to Pytorch

----

### Tue, August 3, 2021: Linear DL

*coordinated by [Andrew Saxe (Oxford)](https://www.saxelab.org/)*

**Description** Gradients, AutoGrad, linear regression, concept of optimization, loss functions, designing deep linear systems and how to train them

----

### Wed, August 4, 2021: Multi-layer Perceptrons (MLPs)

*coordinated by [Surya Ganguli (Stanford)](https://ganguli-gang.stanford.edu/)*

**Description** From neuroscience inspiration, to solving the XOR problem, to function approximation, cross-validation, training, and trade-offs

----

### Thu, August 5, 2021: Optimization

*coordinated by [Ioannis Mitliagkas (MILA)](http://mitliagkas.github.io/)*

**Description** Why optimization is hard and all the tricks to get it to work

----

### Fri, August 6, 2021: Regularization

*coordinated by [Lyle Ungar (U Penn)](https://www.cis.upenn.edu/~ungar/)*

**Description** The problem of overfitting and different ways to solve it

----

## Week 2: Doing more with fewer parameters

----

### Mon, August 9, 2021:  Parameter sharing: Convnets and RNNs

*coordinated by [Alona Fyshe (U Alberta)](https://webdocs.cs.ualberta.ca/~alona/)*

**Description** How the number of parameters affects generalization, and what Convolutional Neural Networks (Convnets) and Recurrent Neural Networks (RNNs) can do for you to help

----

### Tue, August 10, 2021: Modern Convnets 

*coordinated by [Alexander Ecker (U Goettingen)](https://eckerlab.org/)*

**Description** Modern Convolutional Neural Nets and how to use them for Transfer Learning

----

### Wed, August 11, 2021: Modern  RNNs

*coordinated by [James Evans (DeepAI)](https://deepai.org/profile/james-a-evans)*

**Description** Memory, time series, recurrence, vanishing gradients and embeddings 

----

### Thu, August 12, 2021: Attention and Transformers

*coordinated by [He He (NYU)](https://hhexiy.github.io/)*

**Description** How attention helps classification, encoding and decoding

----

### Fri, August 13, 2021: Generative Models (VAEs & GANs)

*coordinated by [Vikash Gilja (UCSD)](https://scholar.google.com/citations?user=EO3cAGQAAAAJ&hl=en) and [Akash Srivastava (MIT-IBM)](http://akashgit.github.io/)*

**Description** Variational Auto-Encoders (VAEs) and Generative Adversarial Networks (GANs) as methods for representing latent data statistics

----

## Week 3: Advanced methods

----

### Mon, August 16, 2021:  Projects day 

*coordinated by Project TAs*

----

### Tue, August 17, 2021:  Unsupervised and Self-supervised Learning 

*coordinated by [Blake Richards (McGill)](https://sites.google.com/mila.quebec/linc-lab/home) and [Tim Lillicrap (Google DeepMind)](https://contrastiveconvergence.net/)*

**Description** Learning without direct supervision

----

### Wed, August 18, 2021: Basic Reinforcement Learning (RL) ideas 

*coordinated by [Jane Wang (Google DeepMind)](http://www.janexwang.com/) and [Feryal Behbahani (Google DeepMind)](https://feryal.github.io/)*

**Description** How RL can help solve DL problems

----

### Thu, August 19, 2021: RL for games

*coordinated by [Tim Lillicrap (Google DeepMind)](https://contrastiveconvergence.net/) and [Blake Richards (McGill)](https://sites.google.com/mila.quebec/linc-lab/home)*

**Description** Get to learn how RL solved the game of Go

----

### Fri, August 20, 2021: Continual Learning / Causality / Future stuff & Finishing Proposals and Wrap-up

*coordinated by [Joshua T. Vogelstein (Johns Hopkins)](https://jovo.me/) and [Vincenzo Lomonaco (U Pisa)](https://www.vincenzolomonaco.com/)*

**Description** How can we get a causality, how to generalize out of sample, what will the future bring?

**Description** After the tutorials the day is dedicated to group projects and celebrating course completion

Owner

GitHub Events