Science Score: 10.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
○codemeta.json file
-
○.zenodo.json file
-
○DOI references
-
✓Academic publication links
Links to: arxiv.org, nature.com -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (2.9%) to scientific vocabulary
Last synced: 10 months ago
·
JSON representation
Repository
深度学习经典、新论文逐段精读
Basic Info
Statistics
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
- Releases: 0
Fork of mli/paper-reading
Created about 3 years ago
· Last pushed over 3 years ago
https://github.com/BeingGod/paper-reading/blob/main/
# ## | | | | | | | --: | -- | -- | --: | -- | | 3/30/23 | [GPT-4](https://openai.com/research/gpt-4) || 1:20:38 | [](https://www.bilibili.com/video/BV1vM4y1U7b5)
[](https://youtu.be/K0SZ9mdygTw) | | 3/23/23 | || 1:06:29 | [](https://www.bilibili.com/video/BV1oX4y1d7X6)
[](https://youtu.be/sh79Z8i15PI) | | 3/10/23 | [Anthropic LLM](https://arxiv.org/pdf/2204.05862.pdf) || 1:01:51 | [](https://www.bilibili.com/video/BV1XY411B7nM)
[](https://youtu.be/iqX0pgNDon0) | | 1/20/23 | [Helm](https://arxiv.org/pdf/2211.09110.pdf) || 1:23:37 | [](https://www.bilibili.com/video/BV1z24y1B7uX)
[](https://youtu.be/WgFEw9U3BXA) | | 1/11/23 | || 1:03:29 | [](https://www.bilibili.com/video/BV1fA411Z772)
[](https://youtu.be/S1le41J76lQ) | | 12/29/22 | [Instruct GPT](https://arxiv.org/pdf/2203.02155.pdf) || 1:07:10 | [](https://www.bilibili.com/video/BV1hd4y187CR)
[](https://youtu.be/zfIGAwD1jOQ) | | 12/19/22 | [Neural Corpus Indexer](https://arxiv.org/pdf/2206.02743.pdf) || 55:47 | [](https://www.bilibili.com/video/BV1Se411w7Sn)
[](https://youtu.be/QRffZMSGJyU) | | 12/12/22 | || 1:12:27 | [](https://www.bilibili.com/video/BV1Vd4y1v77v)
[](https://youtu.be/6pzBOQAXUB8) | | 11/14/22 | [OpenAI Whisper](https://cdn.openai.com/papers/whisper.pdf) || 1:12:16 | [](https://www.bilibili.com/video/BV1VG4y1t74x)
[](https://youtu.be/3eXCJd32UnM) | | 11/07/22 | OpenAI Whisper || 23:39 | [](https://www.bilibili.com/video/BV1Pe4y1t7de)
[](https://youtu.be/PwVlvCPDnrI) | | 10/23/22 | [Chain of Thought](https://arxiv.org/pdf/2201.11903.pdf) || 33:21 | [](https://www.bilibili.com/video/BV1t8411e7Ug)
[](https://youtu.be/H4J59iG3t5o) | | 9/17/22 | CLIP || 1:04:26 | [](https://www.bilibili.com/video/BV1gg411U7n4)
[](https://youtu.be/ugJeBivv65s) | | 9/2/22 | CLIP || 1:14:43 | [](https://www.bilibili.com/video/BV1FV4y1p7Lm)
[](https://youtu.be/x4CDhZz_Dvg) | | 7/29/22 | [ViLT](https://arxiv.org/pdf/2102.03334.pdf) || 1:03:26 | [](https://www.bilibili.com/video/BV14r4y1j74y)
[](https://youtu.be/ug8YvZOjOCE) | | 7/22/22 | [](https://press.uchicago.edu/ucp/books/book/chicago/C/bo23521678.html) || 44:14 | [](https://www.bilibili.com/video/BV1SB4y1a75c) | | 7/15/22 | [](https://press.uchicago.edu/ucp/books/book/chicago/C/bo23521678.html)|
| 43:56 |[](https://www.bilibili.com/video/BV1WB4y1v7ST)| | 7/8/22 | [DALLE 2](https://arxiv.org/pdf/2204.06125.pdf) |
| 1:27:54 |[](https://www.bilibili.com/video/BV17r4y1u77B)
[](https://youtu.be/hO57mntSMl0)| | 7/1/22 | [](https://press.uchicago.edu/ucp/books/book/chicago/C/bo23521678.html)|| 1:03:40 |[](https://www.bilibili.com/video/BV11S4y1v7S2/)| | 6/24/22 | [](https://press.uchicago.edu/ucp/books/book/chicago/C/bo23521678.html) |
| 45:01 | [](https://www.bilibili.com/video/BV1hY411T7vy/) | | 6/17/22 | [Zero](https://arxiv.org/pdf/1910.02054.pdf) |
| 52:21 | [](https://www.bilibili.com/video/BV1tY411g7ZT/) | | 6/10/22 | [DETR](https://arxiv.org/pdf/2005.12872.pdf) |
| 54:22 | [](https://www.bilibili.com/video/BV1GB4y1X72R/) | | 6/3/22 | [Megatron LM](https://arxiv.org/pdf/1909.08053.pdf) |
| 56:07 | [](https://www.bilibili.com/video/BV1nB4y1R7Yz/) | | 5/27/22 | [GPipe](https://proceedings.neurips.cc/paper/2019/file/093f65e080a295f8076b1c5722a46aa2-Paper.pdf) |
| 58:47 | [](https://www.bilibili.com/video/BV1v34y1E7zu/)
[](https://youtu.be/eXjRpS_BTbs) | | 5/5/22 | [Pathways](https://arxiv.org/pdf/2203.12533.pdf) || 1:02:13 | [](https://www.bilibili.com/video/BV1xB4y1m7Xi/)
[](https://youtu.be/8hS1ZtgG0wU) | | 4/28/22 | [](https://arxiv.org/pdf/2012.06567.pdf) || 1:08:32 | [](https://www.bilibili.com/video/BV11Y411P7ep/)
[](https://youtu.be/J2YC0-k57NM) | | 4/21/22 | [Parameter Server](https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-li_mu.pdf) || 1:37:40 | [](https://www.bilibili.com/video/BV1YA4y197G8/)
[](https://youtu.be/xt-AwUrDxQk) | | 4/14/22 | [](https://arxiv.org/pdf/2012.06567.pdf) || 51:15 | [](https://www.bilibili.com/video/BV1fL4y157yA/)
[](https://youtu.be/gK7AGO6okhc) | | 3/31/22 | [I3D](https://arxiv.org/pdf/1705.07750.pdf) || 52:31 | [](https://www.bilibili.com/video/BV1tY4y1p7hq/)
[](https://youtu.be/9lIkKiAn6uE) | | 3/24/22 | 2022 [AI ](https://aiindex.stanford.edu/wp-content/uploads/2022/03/2022-AI-Index-Report_Master.pdf) || 1:19:56 | [](https://www.bilibili.com/video/BV1s44y1N7eu/)
[](https://youtu.be/K8h_xjQ6ufY) | | 3/17/22 | [AlphaCode](https://storage.googleapis.com/deepmind-media/AlphaCode/competition_level_code_generation_with_alphacode.pdf) || 44:00 | [](https://www.bilibili.com/video/BV1ab4y1s7rc/)
[](https://youtu.be/t8Gzkca9pW4) | | 3/10/22 | [OpenAI Codex](https://arxiv.org/pdf/2107.03374.pdf) || 47:58 | [](https://www.bilibili.com/video/BV1iY41137Zi/)
[](https://www.zhihu.com/zvideo/1490959755963666432)
[](https://youtu.be/oZriUGkQSNM) | | 3/3/22 | [GPT](https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf), [GPT-2](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf), [GPT-3](https://arxiv.org/abs/2005.14165) || 1:29:58 | [](https://www.bilibili.com/video/BV1AF411b7xQ/)
[](https://youtu.be/t70Bl3w7bxY) | | 2/24/22 | [Two-Stream](https://proceedings.neurips.cc/paper/2014/file/00ec53c4682d36f5c4359f4ae7bd7ba1-Paper.pdf) || 52:57 | [](https://www.bilibili.com/video/BV1mq4y1x7RU/)
[](https://youtu.be/vuqwKP2iDe0) | | 2/10/22 | [CLIP](https://openai.com/blog/clip/) || 1:38:25 | [](https://www.bilibili.com/video/BV1SL4y1s7LQ/)
[](https://www.zhihu.com/zvideo/1475706654562299904)
[](https://youtu.be/OZF1t_Hieq8) | | 2/6/22 | [ novel](https://perceiving-systems.blog/en/post/novelty-in-science) || 14:11 | [](https://www.bilibili.com/video/BV1ea41127Bq/)
[](https://www.zhihu.com/zvideo/1475719090198876161) | | 1/23/22 | [AlphaFold 2](https://www.nature.com/articles/s41586-021-03819-2.pdf) || 1:15:28 | [](https://www.bilibili.com/video/BV1oR4y1K7Xr/)
[](https://www.zhihu.com/zvideo/1469132410537717760)
[](https://youtu.be/Oy3OCoGUr-w) | | 1/18/22 | || 9:59 | [](https://www.bilibili.com/video/BV1oL411c7Us/)
[](https://www.zhihu.com/zvideo/1475716940051869696) | | 1/15/22 | [Swin Transformer](https://arxiv.org/pdf/2103.14030.pdf) || 1:00:21 | [](https://www.bilibili.com/video/BV13L4y1475U/)
[](https://www.zhihu.com/zvideo/1466282983652691968)
[](https://youtu.be/luP3-Fs0QCo) | | 1/7/22 | [](https://www.nature.com/articles/s41586-021-04086-x.pdf) || 52:51 | [](https://www.bilibili.com/video/BV1YZ4y1S72j/)
[](https://www.zhihu.com/zvideo/1464060386375299072)
[](https://youtu.be/czFGjvhtss8) | | 1/5/22 | AlphaFold 2 || 03:28 | [](https://www.bilibili.com/video/BV1Eu411U7Te/) | | 12/20/21 | [](#contrastive_learning) |
| 1:32:01 | [](https://www.bilibili.com/video/BV19S4y1M7hm/)
[](https://www.zhihu.com/zvideo/1460828005077164032)
[](https://www.youtube.com/watch?v=1pvxufGRuW4) | | 12/15/21 | [MoCo](https://arxiv.org/pdf/1911.05722.pdf) || 1:24:11 | [](https://www.bilibili.com/video/BV1C3411s7t9/)
[](https://www.zhihu.com/zvideo/1454723120678936576)
[](https://www.youtube.com/watch?v=1pvxufGRuW4) | | 12/9/21 | 1 || 5:34 | [](https://www.bilibili.com/video/BV1qq4y1z7F2/) | | 12/8/21 | [MAE](https://arxiv.org/pdf/2111.06377.pdf) |
| 47:04 | [](https://www.bilibili.com/video/BV1sq4y1q77t/)
[](https://www.zhihu.com/zvideo/1452458167968251904)
[](https://youtu.be/mYlX2dpdHHM) | | 11/29/21 | [ViT](https://arxiv.org/pdf/2010.11929.pdf) || 1:11:30 | [](https://www.bilibili.com/video/BV15P4y137jb/)
[](https://www.zhihu.com/zvideo/1449195245754380288)
[](https://youtu.be/FRFt3x0bO94) | | 11/18/21 | [BERT](https://arxiv.org/abs/1810.04805) || 45:49 | [](https://www.bilibili.com/video/BV1PL411M7eQ/)
[](https://www.zhihu.com/zvideo/1445340200976785408)
[](https://youtu.be/ULD3uIb2MHQ) | | 11/9/21 | [GAN](https://papers.nips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf) || 46:16 | [](https://www.bilibili.com/video/BV1rb4y187vD/)
[](https://www.zhihu.com/zvideo/1442091389241159681)
[](https://www.youtube.com/watch?v=g_0HtlrLiDo) | | 11/3/21 | [](https://distill.pub/2021/gnn-intro/)GNN/GCN || 1:06:19 | [](https://www.bilibili.com/video/BV1iT4y1d7zP/)
[](https://www.zhihu.com/zvideo/1439540657619087360)
[](https://youtu.be/sejA2PtCITw) | | 10/27/21 | [Transformer](https://arxiv.org/abs/1706.03762)
[^transformer]) || 1:27:05 | [](https://www.bilibili.com/video/BV1pu411o7BE/)
[](https://www.zhihu.com/zvideo/1437034536677404672)
[](https://youtu.be/nzqlFIcCSWQ) | | 10/22/21 | [ResNet](https://arxiv.org/abs/1512.03385) || 53:46 | [](https://www.bilibili.com/video/BV1P3411y7nn/)
[](https://www.zhihu.com/zvideo/1434795406001180672)
[](https://www.youtube.com/watch?v=pWMnzCX4cwQ) | | 10/21/21 | [ResNet](https://arxiv.org/abs/1512.03385) || 11:50 | [](https://www.bilibili.com/video/BV1Fb4y1h73E/)
[](https://www.zhihu.com/zvideo/1434787226101751808)
[](https://www.youtube.com/watch?v=NnSldWhSqvY) | | 10/15/21 | [AlexNet](https://papers.nips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf) || 55:21 | [](https://www.bilibili.com/video/BV1hq4y157t1/)
[](https://www.zhihu.com/zvideo/1432354207483871232)
[](https://www.youtube.com/watch?v=wYmlILPsLlY) | | 10/14/21 | 9[AlexNet](https://papers.nips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf) || 19:59 | [](https://www.bilibili.com/video/BV1ih411J7Kz/)
[](https://www.zhihu.com/zvideo/1432155856322920448)
[](https://www.youtube.com/watch?v=vdYH0fE6thY) | | 10/06/21 | || 06:39 | [](https://www.bilibili.com/video/BV1H44y1t75x/)
[](https://www.zhihu.com/zvideo/1428973951632969728)
[](https://www.youtube.com/watch?v=txjl_Q4jCyQ&list=PLFXJ6jwg0qW-7UM8iUTj3qKqdhbQULP5I&index=1) | [^transformer]: 1 [100+200+](https://arxiv.org/abs/2108.07258)2 [LayerNorm](https://arxiv.org/pdf/1911.07013.pdf)3 [AttentionTransformer](https://arxiv.org/abs/2103.03404) ## 10 [](https://c.d2l.ai/zh-v2/) [](https://github.com/mli/paper-reading/discussions) 67 32 semanticscholar [API](https://api.semanticscholar.org/api-docs/graph#operation/get_graph_get_paper) ### - CNN | | | | | | | ------ | ---- | ------------------------------------------------------------ | -------------------- | ------------------------------------------------------------ | | | 2012 | [AlexNet](https://papers.nips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf) | | [](https://www.semanticscholar.org/paper/ImageNet-classification-with-deep-convolutional-Krizhevsky-Sutskever/abd1c342495432171beb7ca8fd9551ef13cbd0ff) | | | 2014 | [VGG](https://arxiv.org/pdf/1409.1556.pdf) | 3x3 | [](https://www.semanticscholar.org/paper/Very-Deep-Convolutional-Networks-for-Large-Scale-Simonyan-Zisserman/eb42cf88027de515750f230b23b1a057dc782108) | | | 2014 | [GoogleNet](https://arxiv.org/pdf/1409.4842.pdf) | | [](https://www.semanticscholar.org/paper/Going-deeper-with-convolutions-Szegedy-Liu/e15cf50aa89fee8535703b9f9512fca5bfc43327) | | | 2015 | [ResNet](https://arxiv.org/pdf/1512.03385.pdf) | |[](https://www.semanticscholar.org/paper/Deep-Residual-Learning-for-Image-Recognition-He-Zhang/2c03df8b48bf3fa39054345bafabfeff15bfd11d) | | | 2017 | [MobileNet](https://arxiv.org/pdf/1704.04861.pdf) | CNN |[](https://www.semanticscholar.org/paper/MobileNets%3A-Efficient-Convolutional-Neural-Networks-Howard-Zhu/3647d6d0f151dc05626449ee09cc7bce55be497e) | | | 2019 | [EfficientNet](https://arxiv.org/pdf/1905.11946.pdf) | CNN |[](https://www.semanticscholar.org/paper/EfficientNet%3A-Rethinking-Model-Scaling-for-Neural-Tan-Le/4f2eda8077dc7a69bb2b4e0a1a086cf054adb3f9) | | | 2021 | [Non-deep networks](https://arxiv.org/pdf/2110.07641.pdf) | ImageNetSOTA | [](https://www.semanticscholar.org/paper/Non-deep-Networks-Goyal-Bochkovskiy/0d7f6086772079bc3e243b7b375a9ca1a517ba8b) | ### - Transformer | | | | | | | ------ | ---- | ------------------------------------------------------------ | -------------------- | ------------------------------------------------------------ | | | 2020 | [ViT](https://arxiv.org/pdf/2010.11929.pdf) | TransformerCV |[](https://www.semanticscholar.org/paper/An-Image-is-Worth-16x16-Words%3A-Transformers-for-at-Dosovitskiy-Beyer/7b15fa1b8d413fbe14ef7a97f651f47f5aff3903) | | | 2021 | [Swin Transformer](https://arxiv.org/pdf/2103.14030.pdf) | Vision Transformer | [](https://www.semanticscholar.org/paper/Swin-Transformer%3A-Hierarchical-Vision-Transformer-Liu-Lin/c8b25fab5608c3e033d34b4483ec47e68ba109b7) | | | 2021 | [MLP-Mixer](https://arxiv.org/pdf/2105.01601.pdf) | MLPself-attention |[](https://www.semanticscholar.org/paper/MLP-Mixer%3A-An-all-MLP-Architecture-for-Vision-Tolstikhin-Houlsby/2def61f556f9a5576ace08911496b7c7e4f970a4) | | | 2021 | [MAE](https://arxiv.org/pdf/2111.06377.pdf) | BERTCV |[](https://www.semanticscholar.org/paper/Masked-Autoencoders-Are-Scalable-Vision-Learners-He-Chen/c1962a8cf364595ed2838a097e9aa7cd159d3118) | ### | | | | | | | ------ | ---- | ------------------------------------------------- | ------------ | ------------------------------------------------------------ | | | 2014 | [GAN](https://papers.nips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf) | |[](https://www.semanticscholar.org/paper/Generative-Adversarial-Nets-Goodfellow-Pouget-Abadie/54e325aee6b2d476bbbb88615ac15e251c6e8214) | | | 2015 | [DCGAN](https://arxiv.org/pdf/1511.06434.pdf) | CNNGAN |[](https://www.semanticscholar.org/paper/Unsupervised-Representation-Learning-with-Deep-Radford-Metz/8388f1be26329fa45e5807e968a641ce170ea078) | | | 2016 | [pix2pix](https://arxiv.org/pdf/1611.07004.pdf) | |[](https://www.semanticscholar.org/paper/Image-to-Image-Translation-with-Conditional-Isola-Zhu/8acbe90d5b852dadea7810345451a99608ee54c7) | | | 2016 | [SRGAN](https://arxiv.org/pdf/1609.04802.pdf) | |[](https://www.semanticscholar.org/paper/Photo-Realistic-Single-Image-Super-Resolution-Using-Ledig-Theis/df0c54fe61f0ffb9f0e36a17c2038d9a1964cba3) | | | 2017 | [WGAN](https://arxiv.org/abs/1701.07875) | |[](https://www.semanticscholar.org/paper/Wasserstein-GAN-Arjovsky-Chintala/2f85b7376769473d2bed56f855f115e23d727094) | | | 2017 | [CycleGAN](https://arxiv.org/abs/1703.10593) | |[](https://www.semanticscholar.org/paper/Unpaired-Image-to-Image-Translation-Using-Networks-Zhu-Park/c43d954cf8133e6254499f3d68e45218067e4941) | | | 2018 | [StyleGAN](https://arxiv.org/abs/1812.04948) | |[](https://www.semanticscholar.org/paper/A-Style-Based-Generator-Architecture-for-Generative-Karras-Laine/ceb2ebef0b41e31c1a21b28c2734123900c005e2) | | | 2019 | [StyleGAN2](https://arxiv.org/pdf/1912.04958.pdf) | |[](https://www.semanticscholar.org/paper/Analyzing-and-Improving-the-Image-Quality-of-Karras-Laine/f3e3d1f86a534a3654d0ee263142e44f4e2c61e9) | | | 2020 | [DDPM](https://arxiv.org/pdf/2006.11239.pdf) | Diffusion Models |[](https://www.semanticscholar.org/paper/Denoising-Diffusion-Probabilistic-Models-Ho-Jain/289db3be7bf77e06e75541ba93269de3d604ac72) | | | 2021 | [Improved DDPM](https://arxiv.org/pdf/2102.09672.pdf) | DDPM |[](https://www.semanticscholar.org/paper/Improved-Denoising-Diffusion-Probabilistic-Models-Nichol-Dhariwal/de18baa4964804cf471d85a5a090498242d2e79f) | | | 2021 | [Guided Diffusion Models](https://arxiv.org/pdf/2105.05233.pdf) | GAN |[](https://www.semanticscholar.org/paper/Diffusion-Models-Beat-GANs-on-Image-Synthesis-Dhariwal-Nichol/64ea8f180d0682e6c18d1eb688afdb2027c02794) | | | 2021 | [StyleGAN3](https://arxiv.org/pdf/2106.12423.pdf) | |[](https://www.semanticscholar.org/paper/Alias-Free-Generative-Adversarial-Networks-Karras-Aittala/c1ff08b59f00c44f34dfdde55cd53370733a2c19) | | | 2022 | [DALL.E 2](https://arxiv.org/pdf/2204.06125.pdf) | CLIP + Diffusion models |[](https://www.semanticscholar.org/paper/Hierarchical-Text-Conditional-Image-Generation-with-Ramesh-Dhariwal/c57293882b2561e1ba03017902df9fc2f289dea2) | ### - Object Detection | | | | | | | ------ | ---- | ------------------------------------------------- | ------------ | ------------------------------------------------------------ | | | 2014 | [R-CNN](https://arxiv.org/pdf/1311.2524v5.pdf) | Two-stage |[](https://www.semanticscholar.org/paper/2f4df08d9072fc2ac181b7fced6a245315ce05c8) | | | 2015 | [Fast R-CNN](http://arxiv.org/abs/1504.08083v2) | |[](https://www.semanticscholar.org/paper/7ffdbc358b63378f07311e883dddacc9faeeaf4b) | | | 2015 | [Faster R-CNN](http://arxiv.org/abs/1506.01497v3) | |[](https://www.semanticscholar.org/paper/424561d8585ff8ebce7d5d07de8dbf7aae5e7270) | | | 2016 | [SSD](http://arxiv.org/abs/1512.02325v5) | Single stage |[](https://www.semanticscholar.org/paper/4d7a9197433acbfb24ef0e9d0f33ed1699e4a5b0) | | | 2016 | [YOLO](http://arxiv.org/abs/1506.02640v5) | |[](https://www.semanticscholar.org/paper/f8e79ac0ea341056ef20f2616628b3e964764cfd) | | | 2017 | [Mask R-CNN](http://arxiv.org/abs/1703.06870v3) | |[](https://www.semanticscholar.org/paper/ea99a5535388196d0d44be5b4d7dd02029a43bb2) | | | 2017 | [YOLOv2](http://arxiv.org/abs/1612.08242v1) | |[](https://www.semanticscholar.org/paper/7d39d69b23424446f0400ef603b2e3e22d0309d6) | | | 2018 | [YOLOv3](http://arxiv.org/abs/1804.02767v1) | |[](https://www.semanticscholar.org/paper/e4845fb1e624965d4f036d7fd32e8dcdd2408148) | | | 2019 | [CenterNet](https://arxiv.org/pdf/1904.07850.pdf) | Anchor free |[](https://www.semanticscholar.org/paper/Objects-as-Points-Zhou-Wang/6a2e2fd1b5bb11224daef98b3fb6d029f68a73f2) | | | 2020 | [DETR](https://arxiv.org/pdf/2005.12872.pdf) | Transformer |[](https://www.semanticscholar.org/paper/End-to-End-Object-Detection-with-Transformers-Carion-Massa/962dc29fdc3fbdc5930a10aba114050b82fe5a3e) | ### - | | | | | | | ------ | ---- | ------------------------------------------------------------ | -------------------- | ------------------------------------------------------------ | | | 2018 | [InstDisc](https://arxiv.org/pdf/1805.01978.pdf) | memory bank |[](https://www.semanticscholar.org/paper/Unsupervised-Feature-Learning-via-Non-parametric-Wu-Xiong/155b7782dbd713982a4133df3aee7adfd0b6b304) | | | 2018 | [CPC](https://arxiv.org/pdf/1807.03748.pdf) | | [](https://www.semanticscholar.org/paper/Representation-Learning-with-Contrastive-Predictive-Oord-Li/b227f3e4c0dc96e5ac5426b85485a70f2175a205) | | | 2019 | [InvaSpread](https://arxiv.org/pdf/1904.03436.pdf) | |[](https://www.semanticscholar.org/paper/Unsupervised-Embedding-Learning-via-Invariant-and-Ye-Zhang/e4bde6fe33b6c2cf9d1647ac0b041f7d1ba29c5b) | | | 2019 | [CMC](https://arxiv.org/pdf/1906.05849.pdf) | |[](https://www.semanticscholar.org/paper/Contrastive-Multiview-Coding-Tian-Krishnan/97f4d09175705be4677d675fa27e55defac44800) | | | 2019 | [MoCov1](https://arxiv.org/pdf/1911.05722.pdf) | | [](https://www.semanticscholar.org/paper/Momentum-Contrast-for-Unsupervised-Visual-Learning-He-Fan/ec46830a4b275fd01d4de82bffcabe6da086128f) | | | 2020 | [SimCLRv1](https://arxiv.org/pdf/2002.05709.pdf) | ( + MLP head + batch) |[](https://www.semanticscholar.org/paper/A-Simple-Framework-for-Contrastive-Learning-of-Chen-Kornblith/34733eaf66007516347a40ad5d9bbe1cc9dacb6b) | | | 2020 | [MoCov2](https://arxiv.org/pdf/2003.04297.pdf) | MoCov1 + improvements from SimCLRv1 |[](https://www.semanticscholar.org/paper/Improved-Baselines-with-Momentum-Contrastive-Chen-Fan/a1b8a8df281bbaec148a897927a49ea47ea31515) | | | 2020 | [SimCLRv2](https://arxiv.org/pdf/2006.10029.pdf) | |[](https://www.semanticscholar.org/paper/Big-Self-Supervised-Models-are-Strong-Learners-Chen-Kornblith/3e7f5f4382ac6f9c4fef6197dd21abf74456acd1) | | | 2020 | [BYOL](https://arxiv.org/pdf/2006.07733.pdf) | | [](https://www.semanticscholar.org/paper/Bootstrap-Your-Own-Latent%3A-A-New-Approach-to-Grill-Strub/38f93092ece8eee9771e61c1edaf11b1293cae1b) | | | 2020 | [SWaV](https://arxiv.org/pdf/2006.09882.pdf) | | [](https://www.semanticscholar.org/paper/Unsupervised-Learning-of-Visual-Features-by-Cluster-Caron-Misra/10161d83d29fc968c4612c9e9e2b61a2fc25842e) | | | 2020 | [SimSiam](https://arxiv.org/pdf/2011.10566.pdf) | |[](https://www.semanticscholar.org/paper/Exploring-Simple-Siamese-Representation-Learning-Chen-He/0e23d2f14e7e56e81538f4a63e11689d8ac1eb9d) | | | 2021 | [MoCov3](https://arxiv.org/pdf/2104.02057.pdf) | ViT |[](https://www.semanticscholar.org/paper/An-Empirical-Study-of-Training-Self-Supervised-Chen-Xie/739ceacfafb1c4eaa17509351b647c773270b3ae) | | | 2021 | [DINO](https://arxiv.org/pdf/2104.14294.pdf) | transformer |[](https://www.semanticscholar.org/paper/Emerging-Properties-in-Self-Supervised-Vision-Caron-Touvron/ad4a0938c48e61b7827869e4ac3baffd0aefab35) | ### - | | | | | | | ------ | ---- | ------------------------------------------------------------ | -------------------- | ------------------------------------------------------------ | | | 2014 | [DeepVideo](https://cs.stanford.edu/people/karpathy/deepvideo/) | sports1M |[](https://www.semanticscholar.org/paper/Large-Scale-Video-Classification-with-Convolutional-Karpathy-Toderici/6d4c9c923e9f145d1c01a2de2afc38ec23c44253) | | | 2014 | [Two-stream](https://arxiv.org/pdf/1406.2199.pdf) | |[](https://www.semanticscholar.org/paper/Two-Stream-Convolutional-Networks-for-Action-in-Simonyan-Zisserman/67dccc9a856b60bdc4d058d83657a089b8ad4486) | | | 2014 | [C3D](https://arxiv.org/pdf/1412.0767.pdf) | 3D-CNN |[](https://www.semanticscholar.org/paper/Learning-Spatiotemporal-Features-with-3D-Networks-Tran-Bourdev/d25c65d261ea0e6a458be4c50c40ffe5bc508f77) | | | 2015 | [Beyond-short-snippets](https://arxiv.org/pdf/1503.08909.pdf) | LSTM |[](https://www.semanticscholar.org/paper/Beyond-short-snippets%3A-Deep-networks-for-video-Ng-Hausknecht/5418b2a482720e013d487a385c26fae0f017c6a6) | | | 2016 | [Convolutional fusion](https://arxiv.org/pdf/1604.06573.pdf) | early fusion |[](https://www.semanticscholar.org/paper/Convolutional-Two-Stream-Network-Fusion-for-Video-Feichtenhofer-Pinz/9d9aced120e530484609164c836da64548693484) | | | 2016 | [TSN](https://arxiv.org/pdf/1608.00859.pdf) | bag of tricks in video |[](https://www.semanticscholar.org/paper/Temporal-Segment-Networks%3A-Towards-Good-Practices-Wang-Xiong/ea3d7de6c0880e14455b9acb28f1bc1234321456) | | | 2017 | [I3D](https://arxiv.org/pdf/1705.07750.pdf) | Kinetics2D3D3D-CNN |[](https://www.semanticscholar.org/paper/Quo-Vadis%2C-Action-Recognition-A-New-Model-and-the-Carreira-Zisserman/b61a3f8b80bbd44f24544dc915f52fd30bbdf485) | | | 2017 | [R2+1D](https://arxiv.org/pdf/1711.11248.pdf) | 3D3D |[](https://www.semanticscholar.org/paper/A-Closer-Look-at-Spatiotemporal-Convolutions-for-Tran-Wang/89c3050522a0bb9820c32dc7444e003ef0d3e2e4) | | | 2017 | [Non-local](https://arxiv.org/pdf/1711.07971.pdf) | |[](https://www.semanticscholar.org/paper/Non-local-Neural-Networks-Wang-Girshick/8899094797e82c5c185a0893896320ef77f60e64) | | | 2018 | [SlowFast](https://arxiv.org/pdf/1812.03982.pdf) | |[](https://www.semanticscholar.org/paper/SlowFast-Networks-for-Video-Recognition-Feichtenhofer-Fan/8b47b9c3c35b2b2a78bff7822605b3040f87d699) | | | 2021 | [TimeSformer](https://arxiv.org/pdf/2102.05095.pdf) | transformervideo transformer |[](https://www.semanticscholar.org/paper/Is-Space-Time-Attention-All-You-Need-for-Video-Bertasius-Wang/c143ea9e30b1f2d93a9c060253845423f9e60e1f) | ### | | | | | | | ------ | ---- | ------------------------------------------------------------ | -------------------- | ------------------------------------------------------------ | | | 2021 | [CLIP](https://openai.com/blog/clip/) | |[](https://www.semanticscholar.org/paper/Learning-Transferable-Visual-Models-From-Natural-Radford-Kim/6f870f7f02a8c59c3e23f407f3ef00dd1dcf8fc4) | | | 2021 | [ViLT](https://arxiv.org/pdf/2102.03334.pdf) | |[](https://www.semanticscholar.org/paper/ViLT%3A-Vision-and-Language-Transformer-Without-or-Kim-Son/0839722fb5369c0abaff8515bfc08299efc790a1) | | | 2021 | [ViLD](https://arxiv.org/pdf/2104.13921.pdf) | CLIP |[](https://www.semanticscholar.org/paper/Open-vocabulary-Object-Detection-via-Vision-and-Gu-Lin/cf9b8da26d9b92e75ba49616ed2a1033f59fce14) | | | 2021 | [GLIP](https://arxiv.org/pdf/2112.03857.pdf) | |[](https://www.semanticscholar.org/paper/Grounded-Language-Image-Pre-training-Li-Zhang/5341b412383c43f4a693ad63ec4489e3ec7688c8) | | | 2021 | [CLIP4Clip](https://arxiv.org/pdf/2104.08860.pdf) | CLIPretrieval |[](https://www.semanticscholar.org/paper/CLIP4Clip%3A-An-Empirical-Study-of-CLIP-for-End-to-Luo-Ji/281ad83e06d731d5d686acf07cd701576f1188c4) | | | 2021 | [ActionCLIP](https://arxiv.org/pdf/2109.08472.pdf) | |[](https://www.semanticscholar.org/paper/ActionCLIP%3A-A-New-Paradigm-for-Video-Action-Wang-Xing/dc05240a06326b5b1664f7e8c95c330b08cd0349) | | | 2021 | [PointCLIP](https://arxiv.org/pdf/2112.02413.pdf) | 3D2DCLIP |[](https://www.semanticscholar.org/paper/PointCLIP%3A-Point-Cloud-Understanding-by-CLIP-Zhang-Guo/f3ce9ba3fcec362b70263a7ed63d9404975496a0) | | | 2022 | [LSeg](https://arxiv.org/pdf/2201.03546.pdf) | |[](https://www.semanticscholar.org/paper/Language-driven-Semantic-Segmentation-Li-Weinberger/cc9826c222ac1e81b4b374dd9e0df130f298b1e8) | | | 2022 | [GroupViT](https://arxiv.org/pdf/2202.11094.pdf) | |[](https://www.semanticscholar.org/paper/GroupViT%3A-Semantic-Segmentation-Emerges-from-Text-Xu-Mello/0b5f27a5766c5d1394a6282ad94fec21d620bd6b) | | | 2022 | [CLIPasso](https://arxiv.org/pdf/2202.05822.pdf) | CLIP |[](https://www.semanticscholar.org/paper/CLIPasso%3A-Semantically-Aware-Object-Sketching-Vinker-Pajouheshgar/9dec819778bebae4a468c7813f7638534c826f52) | | | 2022 | [DepthCLIP](https://arxiv.org/pdf/2207.01077.pdf) | |[](https://www.semanticscholar.org/paper/Can-Language-Understand-Depth-Zhang-Zeng/9d0afe58801fe9e5537902e853d6e9e385340a92) | ### - Transformer | | | | | | | ------ | ---- | ------------------------------------------------------------ | -------------------- | ------------------------------------------------------------ | | | 2017 | [Transformer](https://arxiv.org/abs/1706.03762) | MLPCNNRNN |[](https://www.semanticscholar.org/paper/Attention-is-All-you-Need-Vaswani-Shazeer/204e3073870fae3d05bcbc2f6a8e263d9b72e776) | | | 2018 | [GPT](https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf) | Transformer |[](https://www.semanticscholar.org/paper/Improving-Language-Understanding-by-Generative-Radford-Narasimhan/cd18800a0fe0b668a1cc19f2ec95b5003d0a5035) | | | 2018 | [BERT](https://arxiv.org/abs/1810.04805) | TransformerNLP |[](https://www.semanticscholar.org/paper/BERT%3A-Pre-training-of-Deep-Bidirectional-for-Devlin-Chang/df2b0e26d0599ce3e70df8a9da02e51594e0e992) | | | 2019 | [GPT-2](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) | GPT zero-shot learning |[](https://www.semanticscholar.org/paper/Language-Models-are-Unsupervised-Multitask-Learners-Radford-Wu/9405cc0d6169988371b2755e573cc28650d14dfe) | | | 2020 | [GPT-3](https://arxiv.org/abs/2005.14165) | 100 GPT-2few-shot learning |[](https://www.semanticscholar.org/paper/Language-Models-are-Few-Shot-Learners-Brown-Mann/6b85b63579a916f705a8e10a49bd8d849d91b1fc) | ### | | | | | | | ------ | ---- | ------------------------------------------------------------ | -------------------- | ------------------------------------------------------------ | | | 2014 | [](https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-li_mu.pdf) | |[](https://www.semanticscholar.org/paper/Scaling-Distributed-Machine-Learning-with-the-Li-Andersen/0de0c3240bda7972bd0a3c8369ebc4b4f2e4f9c2) | | | 2018 | [GPipe](https://proceedings.neurips.cc/paper/2019/file/093f65e080a295f8076b1c5722a46aa2-Paper.pdf) | Pipeline |[](https://www.semanticscholar.org/paper/GPipe%3A-Efficient-Training-of-Giant-Neural-Networks-Huang-Cheng/c18663fea10c8a303d045fd2c1f33cacf9b73ca3) | | | 2019 | [Megatron-LM](https://arxiv.org/pdf/1909.08053.pdf) | Tensor |[](https://www.semanticscholar.org/paper/Megatron-LM%3A-Training-Multi-Billion-Parameter-Using-Shoeybi-Patwary/8323c591e119eb09b28b29fd6c7bc76bd889df7a) | | | 2019 | [Zero](https://arxiv.org/pdf/1910.02054.pdf) | |[](https://www.semanticscholar.org/paper/ZeRO%3A-Memory-optimizations-Toward-Training-Trillion-Rajbhandari-Rasley/00c957711b12468cb38424caccdf5291bb354033) | | | 2022 | [Pathways](https://arxiv.org/pdf/2203.12533.pdf) | JaxTPU |[](https://www.semanticscholar.org/paper/Pathways%3A-Asynchronous-Distributed-Dataflow-for-ML-Barham-Chowdhery/512e9aa873a1cd7eb61ce1f25ca7df6acb7e2352) | ### | | | | | | | ------ | ---- | ------------------------------------------------------------ | -------------------- | ------------------------------------------------------------ | | | 2021 | [](https://distill.pub/2021/gnn-intro/) | GNN |[](https://www.semanticscholar.org/paper/A-Gentle-Introduction-to-Graph-Neural-Networks-S%C3%A1nchez-Lengeling-Reif/2c0e0440882a42be752268d0b64243243d752a74) | ### | | | | | | | ------ | ---- | ------------------------------------------------------------ | -------------------- | ------------------------------------------------------------ | | | 2014 | [Adam](https://arxiv.org/abs/1412.6980) | |[](https://www.semanticscholar.org/paper/Adam%3A-A-Method-for-Stochastic-Optimization-Kingma-Ba/a6cb366736791bcccc5c8639de5a8f9636bf87e8) | | | 2016 | [](https://arxiv.org/abs/1611.03530) | |[](https://www.semanticscholar.org/paper/Understanding-deep-learning-requires-rethinking-Zhang-Bengio/54ddb00fa691728944fd8becea90a373d21597cf) | | | 2017 | [Momentum](https://distill.pub/2017/momentum/) | Distill |[](https://www.semanticscholar.org/paper/Why-Momentum-Really-Works-Goh/3e8ccf9d3d843c9855c5d76ab66d3e775384da72) | ### | | | | | | | ------ | ---- | ------------------------------------------------------------ | -------------------- | ------------------------------------------------------------ | | | 2016 | [AlphaGo](https://storage.googleapis.com/deepmind-media/alphago/AlphaGoNaturePaper.pdf) | |[](https://www.semanticscholar.org/paper/Mastering-the-game-of-Go-with-deep-neural-networks-Silver-Huang/846aedd869a00c09b40f1f1f35673cb22bc87490) | | | 2020 | [AlphaFold](https://discovery.ucl.ac.uk/id/eprint/10089234/1/343019_3_art_0_py4t4l_convrt.pdf) | 3D |[](https://www.semanticscholar.org/paper/Improved-protein-structure-prediction-using-from-Senior-Evans/3a083d843f891b3574494c385699c21766ce8b7a) | | | 2021 | [AlphaFold 2](https://www.nature.com/articles/s41586-021-03819-2.pdf) | 3D |[](https://www.semanticscholar.org/paper/Highly-accurate-protein-structure-prediction-with-Jumper-Evans/dc32a984b651256a8ec282be52310e6bd33d9815) | | | 2021 | [Codex](https://arxiv.org/pdf/2107.03374.pdf) | |[](https://www.semanticscholar.org/paper/Evaluating-Large-Language-Models-Trained-on-Code-Chen-Tworek/acbdbf49f9bc3f151b93d9ca9a06009f4f6eb269) | | | 2021 | [](https://www.nature.com/articles/s41586-021-04086-x.pdf) | |[](https://www.semanticscholar.org/paper/Advancing-mathematics-by-guiding-human-intuition-AI-Davies-Velickovic/f672b8fb430606fee0bb368f16603531ce1e90c4) | | | 2022 | [AlphaCode](https://storage.googleapis.com/deepmind-media/AlphaCode/competition_level_code_generation_with_alphacode.pdf) | |[](https://www.semanticscholar.org/paper/Competition-Level-Code-Generation-with-AlphaCode-Li-Choi/5cbe278b65a81602a864184bbca37de91448a5f5) |
Owner
- Name: Ruibin Cheung
- Login: BeingGod
- Kind: user
- Location: Pudong, Shanghai
- Repositories: 28
- Profile: https://github.com/BeingGod