https://github.com/daniel-furman/great-deep-learning-tutorials

A Great Collection of Deep Learning Tutorials and Repositories

https://github.com/daniel-furman/great-deep-learning-tutorials

Repository

A Great Collection of Deep Learning Tutorials and Repositories

https://github.com/daniel-furman/Great-Deep-Learning-Tutorials/blob/master/

# Great-Deep-Learning-Tutorials
A Great Collection of Deep Learning Tutorials and Repositories

## General Deep Learning Tutorials:
- [Browse state-of-the-art Deep Learning based Papers with their associated codes](https://paperswithcode.com/sota) [_Extremely Fantastic_]
- [Deep-Learning-Roadmap](https://github.com/astorfi/Deep-Learning-Roadmap)  
- [DeepLizard](https://deeplizard.com/) [_Good Tutorials for Deep Learning_]  
- [Sebastian Ruder - Blog](https://ruder.io/) [_Great NLP & Deep Learning Posts_]  
- [Jeremy Jordan - Blog](https://www.jeremyjordan.me/author/jeremy/)  
- [Excellent Blog](https://lilianweng.github.io/lil-log/)  
- [Torchvision Release Notes](https://github.com/pytorch/vision/releases)  [_Important_]
- [The 6 most useful Machine Learning projects of the past year (2018)](https://towardsdatascience.com/the-10-most-useful-machine-learning-projects-of-the-past-year-2018-5378bbd4919f)  
- [ResNet Review](https://towardsdatascience.com/review-resnet-winner-of-ilsvrc-2015-image-classification-localization-detection-e39402bfa5d8)  
- [Receptive Field Estimation](https://github.com/fornaxai/receptivefield)  [_Great_]  
- [An overview of gradient descent optimization algorithms](https://ruder.io/optimizing-gradient-descent/) [_Useful_]  
- [How to decide on learning rate](https://towardsdatascience.com/how-to-decide-on-learning-rate-6b6996510c98)  
- [Overview of State-of-the-art Machine Learning Algorithms per Discipline per Task](https://towardsdatascience.com/overview-state-of-the-art-machine-learning-algorithms-per-discipline-per-task-c1a16a66b8bb)  
- [Awesome Machine Learning and AI Courses](https://github.com/luspr/awesome-ml-courses)  
- [PyTorch Book](https://github.com/chenyuntc/pytorch-book)  

## Deep Learning Useful Resources for Computer Vision:  
- [Great Deep Learning Resources for Computer Vision Tasks](https://github.com/ahkarami/Great-Deep-Learning-Tutorials/blob/master/ComputerVision.md) [_Excellent_]  

## Deep Learning Useful Resources for Natural Language Processing (NLP):  
- [Great Deep Learning Resources for NLP Tasks](https://github.com/ahkarami/Great-Deep-Learning-Tutorials/blob/master/NLP.md) [_Excellent_]  

## Deep Learning Useful Resources for Spoken Language Processing (Speech Processing):  
- [Great Deep Learning Resources for Speech Processing Tasks](https://github.com/ahkarami/Great-Deep-Learning-Tutorials/blob/master/Speech.md) [_Excellent_]  

## Quantization & Distillation of Deep Learning Models:
- [Quantization](https://nervanasystems.github.io/distiller/quantization/)  
- [Neural Network Distiller](https://github.com/NervanaSystems/distiller/)  
- [Introduction to Quantization on PyTorch](https://pytorch.org/blog/introduction-to-quantization-on-pytorch/) [_Excellent_]  
- [Dynamic Quantization in PyTorch](https://pytorch.org/tutorials/advanced/dynamic_quantization_tutorial.html)  
- [Static Quantization in PyTorch](https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html)  
- [Intel(R) Math Kernel Library - Intel MKL-DNN](https://github.com/intel/mkl-dnn)  
- [Intel MKL-Dnn](https://01.org/mkl-dnn)  
- [ONNX Float32 to Float16](https://github.com/onnx/onnx-docker/blob/master/onnx-ecosystem/converter_scripts/float32_float16_onnx.ipynb)  
- [Neural Network Quantization Introduction](https://jackwish.net/neural-network-quantization-introduction.html) [_Tutorial_]  
- [Quantization in Deep Learning](https://medium.com/@joel_34050/quantization-in-deep-learning-478417eab72b) [_Tutorial_]  
- [Speeding up Deep Learning with Quantization](https://towardsdatascience.com/speeding-up-deep-learning-with-quantization-3fe3538cbb9) [_Tutorial_]  
- [Knowledge Distillation in Deep Learning](https://medium.com/analytics-vidhya/knowledge-distillation-dark-knowledge-of-neural-network-9c1dfb418e6a)  
- [Model Distillation Techniques for Deep Learning](https://heartbeat.fritz.ai/research-guide-model-distillation-techniques-for-deep-learning-4a100801c0eb)  


## Deep Learning for Data Science:
- [Python Data Science Tutorials](https://realpython.com/tutorials/data-science/)  
- [Data Science Course](https://github.com/amingheibi/Data-Science-Course)  
- [First Steps With PySpark and Big Data Processing](https://realpython.com/pyspark-intro/)  
- [A Brief Introduction to PySpark](https://towardsdatascience.com/a-brief-introduction-to-pyspark-ff4284701873)  
- [Introduction to Anomaly Detection in Python](https://blog.floydhub.com/introduction-to-anomaly-detection-in-python/)  
- [Time Series basics Exploring](https://www.kaggle.com/jagangupta/time-series-basics-exploring-traditional-ts)  
- [Understanding Variational Autoencoders (VAEs)](https://towardsdatascience.com/understanding-variational-autoencoders-vaes-f70510919f73)  
- [Variational Autoencoders](https://www.jeremyjordan.me/variational-autoencoders/)  
- [Benchmarking Performance and Scaling of Python Clustering Algorithms](https://hdbscan.readthedocs.io/en/latest/performance_and_scalability.html) [_Important_]  
- [Comparing Python Clustering Algorithms](https://hdbscan.readthedocs.io/en/latest/comparing_clustering_algorithms.html)  
- [Cracking the Data Science Interview](https://github.com/khanhnamle1994/cracking-the-data-science-interview)  
- [Data Engineer Interview Questions Python](https://realpython.com/data-engineer-interview-questions-python/)  
- [khanacademy statistics course](https://www.khanacademy.org/math/ap-statistics) [_Good_]  
- [NumPy Exercises](https://www.w3resource.com/python-exercises/numpy/index.php)  [_Good_]

### Scikit-learn Algorithms on GPU & for Large-Scale Data Sets:
- [skorch - scikit-learn compatible neural network library that wraps PyTorch](https://github.com/skorch-dev/skorch)  
- [scikit-cuda](https://github.com/lebedov/scikit-cuda)  
- [Hummingbird - trained traditional ML models into tensor computations](https://github.com/microsoft/hummingbird)  

### Anomaly Detection:
- [An Awesome Tutorial to Learn Outlier Detection in Python using PyOD Library](https://www.analyticsvidhya.com/blog/2019/02/outlier-detection-python-pyod/)  
- [PyOD: Python Outlier Detection](https://github.com/yzhao062/pyod) [**Great**]  
- [PyNomaly](https://github.com/vc1492a/PyNomaly)  

## Deep Learning Recommendation Model:
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems - DLRM](https://github.com/facebookresearch/dlrm)  
- [DLRM: An advanced, open source deep learning recommendation model](https://ai.facebook.com/blog/dlrm-an-advanced-open-source-deep-learning-recommendation-model/)  
- [LightFM](https://github.com/lyst/lightfm)  
- [Neural Recommendation Algorithms](https://towardsdatascience.com/recotour-ii-neural-recommendation-algorithms-49733938d56e)  
- [Build a Recommendation Engine With Collaborative Filtering](https://realpython.com/build-recommendation-engine-collaborative-filtering/)  [_**Great**_]  
- [NCF - Neural Collaborative Filtering](https://github.com/NervanaSystems/distiller/tree/master/examples/ncf)  
- [Neural Collaborative Filtering](https://towardsdatascience.com/neural-collaborative-filtering-96cef1009401)  
- [AWS Personalized Recommendation Model](https://aws.amazon.com/personalize/)  

## AutoML:
- [Auto Gluon AI](https://auto.gluon.ai/stable/index.html#)  
- [AWS Auto Gluon](https://github.com/awslabs/autogluon)  

## Deep Reasoning:
- [Whats Next For AI? Enter: Deep Reasoning](https://towardsdatascience.com/whats-next-for-ai-enter-deep-reasoning-fae8b131962a)  
- [Deep Learning approaches to understand Human Reasoning](https://towardsdatascience.com/deep-learning-approaches-to-understand-human-reasoning-46f1805d454d)  

## Deep Reinforcement Learning (Great Courses & Tutorials):
- [A Free course in Deep Reinforcement Learning from beginner to expert](https://simoninithomas.github.io/Deep_reinforcement_learning_Course/) [_Great_] 
- [Deep Reinforcement Learning Algorithms with PyTorch](https://github.com/p-christ/Deep-Reinforcement-Learning-Algorithms-with-PyTorch)  
- [Deep Reinforcement Learning - CS 285 Berkeley Course](rail.eecs.berkeley.edu/deeprlcourse/)  
- [solutions to UC Berkeley CS 285](https://github.com/xuanlinli17/CS285_Fa19_Deep_Reinforcement_Learning)  
- [Reinforcement Learning: An Introduction - main book in this field](http://www.incompleteideas.net/book/the-book-2nd.html)  
- [CS234: Reinforcement Learning Course](https://www.youtube.com/playlist?list=PLoROMvodv4rOSOPzutgyCTapiGlY2Nd8u)  
- [Introduction to Reinforcement Learning Course - by DeepMind](https://www.youtube.com/playlist?list=PLoROMvodv4rOSOPzutgyCTapiGlY2Nd8u)  

## Graph Neural Networks:
- [An Introduction to Graph Neural Networks](https://towardsdatascience.com/an-introduction-to-graph-neural-networks-e23dc7bdfba5)  
- [How to Train Graph Convolutional Network Models in a Graph Database](https://towardsdatascience.com/how-to-train-graph-convolutional-network-models-in-a-graph-database-5c919a2f95d7)  
- [A comprehensive survey on graph neural networks](https://arxiv.org/pdf/1901.00596)  
- [Graph Neural Networks: A Review of Methods and Applications](https://arxiv.org/abs/1812.08434)  

### Graph Neural Networks Frameworks:
- [Spektral](https://github.com/danielegrattarola/spektral)  
- [Deep Graph Library - DGL](https://www.dgl.ai/)  
- [PyTorch Geometric - PyG](https://github.com/rusty1s/pytorch_geometric)  
- [ptgnn: A PyTorch GNN Library](https://github.com/microsoft/ptgnn)  


## Best Practices for Training Deep Models:

### Loss Functions:
- [Loss Functions Explained](https://medium.com/deep-learning-demystified/loss-functions-explained-3098e8ff2b27)  

### Weight Initialization:
- [Deep Learning Best Practices (1) - Weight Initialization](https://medium.com/usf-msds/deep-learning-best-practices-1-weight-initialization-14e5c0295b94)  

### Batch Normalization:
- [Batch Normalization in Neural Networks](https://towardsdatascience.com/batch-normalization-in-neural-networks-1ac91516821c)  
- [Batch Normalization and Dropout in Neural Networks](https://towardsdatascience.com/batch-normalization-and-dropout-in-neural-networks-explained-with-pytorch-47d7a8459bcd)  
- [Difference between Local Response Normalization and Batch Normalization](https://towardsdatascience.com/difference-between-local-response-normalization-and-batch-normalization-272308c034ac)  

### Learning Rate Scheduling & Initialization:
- [Automated Learning Rate Suggester](https://forums.fast.ai/t/automated-learning-rate-suggester/44199)  
- [Learning Rate Finder - fastai](https://fastai1.fast.ai/callbacks.lr_finder.html)  
- [Cyclical Learning Rates for Training Neural Networks](https://arxiv.org/abs/1506.01186)  
- [ignite - Example of FastaiLRFinder](https://github.com/pytorch/ignite/blob/master/examples/notebooks/FastaiLRFinder_MNIST.ipynb)   
- [Find Learning Rate - a gist code](https://gist.github.com/colllin/738cd2a9f0abec9be5e8b9becc23a812)    
- [Learning rate finder - PyTorch Lightning](https://pytorch-lightning.readthedocs.io/en/1.1.3/lr_finder.html)  
- [RAdam - On the Variance of the Adaptive Learning Rate and Beyond](https://github.com/LiyuanLucasLiu/RAdam)  

### Early Stopping:
- [Early Stopping in PyTorch - Bjarten](https://github.com/Bjarten/early-stopping-pytorch)  
- [Catalyst - Early Stopping](https://catalyst-team.github.io/catalyst/faq/early_stopping.html)  
- [ignite - Early Stopping](https://github.com/pytorch/ignite/blob/master/ignite/handlers/early_stopping.py)  
- [PyTorch High-Level Training Sample](https://github.com/ncullen93/torchsample/blob/master/README.md)    
- [PyTorch Discussion about Early Stopping](https://discuss.pytorch.org/t/early-stopping-in-pytorch/18800)  

## Conferences News:
- [Latest Computer Vision Trends from CVPR 2019](https://towardsdatascience.com/latest-computer-vision-trends-from-cvpr-2019-c07806dd570b)  
- [Interesting 2019 CVPR papers](https://medium.com/@mattmiesnieks/interesting-2019-cvpr-papers-865e303db5ca)  
- [Summaries of CVPR papers on ShortScience.org](https://www.shortscience.org/venue?key=conf/cvpr)
- [Summaries of ICCV papers on ShortScience.org](https://www.shortscience.org/venue?key=conf/iccv)
- [Summaries of ECCV papers on ShortScience.org](https://www.shortscience.org/venue?key=conf/eccv)

## Deep Learning Frameworks and Infrustructures:
- [set-up a Paperspace GPU Server](https://towardsdatascience.com/how-to-set-up-a-powerful-and-cost-efficient-gpu-server-for-deep-learning-aa1de0d4ea56)  
- [Distributed ML with OpenMPI](https://clusterone.com/tutorials/openmpi-introduction)  
- [Tensorflow 2.0 vs Mxnet](https://medium.com/@mouryarishik/tensorflow-2-0-vs-mxnet-41edd3b7574f)  
- [TensorFlow is dead, long live TensorFlow!](https://hackernoon.com/tensorflow-is-dead-long-live-tensorflow-49d3e975cf04)  

## Great Libraries:
- [Skorch - A scikit-learn compatible neural network library that wraps PyTorch](https://github.com/skorch-dev/skorch)  
- [Hummingbird - traditional ML models into tensor computations via PyTorch](https://github.com/microsoft/hummingbird)  
- [BoTorch - Bayesian Optimization in PyTorch](https://botorch.org/)  
- [torchvision 0.3: segmentation, detection models, new datasets and more](https://pytorch.org/blog/torchvision03/)  
- [TorchAudio: an audio library for PyTorch](https://github.com/pytorch/audio)  
- [AudTorch](https://github.com/audeering/audtorch)  
- [TorchAudio-Contrib](https://github.com/keunwoochoi/torchaudio-contrib) 
- [fastText - Facebook AI Research (FAIR)](https://fasttext.cc/)  
- [Fairseq - Facebook AI Research (FAIR)](https://github.com/pytorch/fairseq)  
- [ParlAI - dialogue models - Facebook AI Research (FAIR)](https://parl.ai/)  
- [DALI - highly optimized engine for data pre-processing](https://github.com/NVIDIA/DALI)  
- [Netron - GitHub](https://github.com/lutzroeder/netron) [_Visualizer for deep learning Models (Excellent)_]
- [Netron - Web Site](https://www.lutzroeder.com/ai)  
- [JupyterLab GPU Dashboards](https://github.com/rapidsai/jupyterlab-nvdashboard) [_Good_]  
- [PyTorch Hub](https://pytorch.org/hub)  
- [Neural Structured Learning (NSL) in TensorFlow](https://github.com/tensorflow/neural-structured-learning)  
- [Pywick - High-Level Training framework for Pytorch](https://github.com/achaiah/pywick)  
- [torchbearer: A model fitting library for PyTorch](https://github.com/pytorchbearer/torchbearer)  
- [torchlayers - Shape inference for PyTorch (like in Keras)](https://github.com/szymonmaszke/torchlayers)  
- [torchtext - GitHub](https://github.com/pytorch/text)  
- [torchtext - Doc](https://torchtext.readthedocs.io/en/latest/)   
- [Optuna - hyperparameter optimization framework](https://optuna.org/)  
- [PyTorchLightning](https://github.com/PyTorchLightning/pytorch-lightning)  
- [Nvidia - runx - An experiment management tool](https://github.com/NVIDIA/runx)  
- [MLogger: a Machine Learning logger](https://github.com/oval-group/mlogger)  
- [ClearML - ML/DL development and production suite](https://github.com/allegroai/clearml)  
- [NVIDIA NeMo -  toolkit for creating Conversational AI (ASR, TTS, and NLP)](https://github.com/NVIDIA/NeMo)  

## Great Models:
- [ResNext WSL](https://pytorch.org/hub/facebookresearch_WSL-Images_resnext/) [_Great Pretrained Model_]  
- [Semi-Weakly Supervised (SWSL) ImageNet Models](https://pytorch.org/hub/facebookresearch_semi-supervised-ImageNet1K-models_resnext/) [_Great Pretrained Model_]  
- [Deep High-Resolution Representation Learning (HRNet)](https://jingdongwang2017.github.io/Projects/HRNet/)  

## Deep Model Conversion:
- [Convert Full ImageNet Pre-trained Model from MXNet to PyTorch](https://blog.paperspace.com/convert-full-imagenet-pre-trained-model-from-mxnet-to-pytorch/) [_Great_] 
- [ONNX Runtime](https://github.com/microsoft/onnxruntime)  

## Great Deep Learning Repositories (for learning DL-based programming):
- [deeplearning-models - PyTorch & TensorFlow Learning](https://github.com/rasbt/deeplearning-models)  [_Very Excellent Repository_]  
- [PyTorch Image Models](https://github.com/rwightman/pytorch-image-models) [_Great_] 
- [5 Advanced PyTorch Tools to Level up Your Workflow](https://towardsdatascience.com/5-advanced-pytorch-tools-to-level-up-your-workflow-d0bcf0603ad5) [_Interesting_]  

## PyTorch High-Level Libraries:  
- [Catalyst - PyTorch framework for Deep Learning research and development](https://github.com/catalyst-team/catalyst) [_Great_]  
- [PyTorch Lightning - GitHub](https://github.com/PyTorchLightning/pytorch-lightning) [_Great_]    
- [PyTorch Lightning - Web Page](https://pytorchlightning.ai/)  
- [Ignite - GitHub](https://github.com/pytorch/ignite) [_Great_]    
- [Ignite - Web Page](https://pytorch.org/ignite/)  

## Other:
- [Clova AI Research - NAVER & LINE](https://github.com/clovaai)  
- [Exploring Weight Agnostic Neural Networks](https://ai.googleblog.com/2019/08/exploring-weight-agnostic-neural.html)  
- [Weight Agnostic Neural Networks](https://weightagnostic.github.io/)  
- [Weight Agnostic Neural Networks - GitHub](https://github.com/google/brain-tokyo-workshop/tree/master/WANNRelease)  
- [SAM: Sharpness-Aware Minimization for Efficiently Improving Generalization](https://github.com/google-research/sam)  

Owner

GitHub Events