├── .gitignore ├── Contributing.md ├── README.md ├── ReadingNotes.md ├── fetch_papers.py ├── get_authors.py └── top100papers.bib /.gitignore: -------------------------------------------------------------------------------- 1 | papers/ 2 | -------------------------------------------------------------------------------- /Contributing.md: -------------------------------------------------------------------------------- 1 | # Contributing Guide 2 | 3 | You can directly edit(add/delete/modify) the [README.md](https://github.com/terryum/awesome-deep-learning-papers/blob/master/README.md) file for contribution. If you don't know how to edit the file, please leave a message to the [Issue board](https://github.com/terryum/awesome-deep-learning-papers/issues). 4 | 5 | ## Awesome list criteria 6 | 7 | - **2016** : +20 citations (:sparkles: +50) 8 | - **2015** : +100 citations (:sparkles: +200) 9 | - **2014** : +200 citations (:sparkles: +400) 10 | - **2013** : +300 citations (:sparkles: +600) 11 | - **2012** : +400 citations (:sparkles: +800) 12 | - **2011** : +500 citations (:sparkles: +1000) 13 | - **2010** : +600 citations (:sparkles: +1200) 14 | 15 | This criteria is not a strict baseline, but a **flexible guideline** for being added to the awesome list. (Since the number of citations is affected by the research area, some papers under the threshold may be added to the list while some over the threshold may not.) If you add papers which are under the above criteria, please provide **enough descriptions** which the papers should be added. 16 | 17 | ## Edit the awesome list 18 | You can **add / delete / modify** the papers on the awesome list. 19 | 20 | #### Addition 21 | As mentioned above, please provide **enough descriptions** when you push a request, especially for the papers which do not meet the criteria or for the papers for the *Papers Worth Reading* section. 22 | 23 | Here is an example of markdown code for a paper on the list. 24 | 25 | `Distilling the knowledge in a neural network (2015), G. Hinton et al. [[pdf]](http://arxiv.org/pdf/1503.02531)` 26 | 27 | The pdf link which can directly download the pdf file rather than links to another webpage is preferred. If the number of authors are more than two, please write the first author's name only. 28 | 29 | #### Deletion 30 | If a paper seems to be out-of-date or be a duplication of another paper, you can suggest deletion. Deletion is also an important contribution to maintain a tractable number of papers on the awesome list. 31 | 32 | #### Modification 33 | Modification (or Updates) that can usually happen is 34 | - addition of :sparkles: marks (as increasing number of citations) 35 | - addition of distinguished researchers 36 | - move papers from a section from another 37 | - correction of typo, grammatical error, etc. 38 | 39 | Note that you are *not* allowed to add or modify the sections. 40 | 41 | ## Acknowledgement 42 | 43 | Thank you for all your contributions. You can also follow my [facebook page](https://www.facebook.com/terryum.io/) or [google plus](https://plus.google.com/+TerryTaeWoongUm/) to get useful information about machine learning and robotics. Thank you! 44 | 45 | Terry 46 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Awesome - Most Cited Deep Learning Papers 2 | 3 | [![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/sindresorhus/awesome) 4 | 5 | [Notice] This list is not being maintained anymore because of the overwhelming amount of deep learning papers published every day since 2017. 6 | 7 | A curated list of the most cited deep learning papers (2012-2016) 8 | 9 | We believe that there exist *classic* deep learning papers which are worth reading regardless of their application domain. Rather than providing overwhelming amount of papers, We would like to provide a *curated list* of the awesome deep learning papers which are considered as *must-reads* in certain research domains. 10 | 11 | ## Background 12 | 13 | Before this list, there exist other *awesome deep learning lists*, for example, [Deep Vision](https://github.com/kjw0612/awesome-deep-vision) and [Awesome Recurrent Neural Networks](https://github.com/kjw0612/awesome-rnn). Also, after this list comes out, another awesome list for deep learning beginners, called [Deep Learning Papers Reading Roadmap](https://github.com/songrotek/Deep-Learning-Papers-Reading-Roadmap), has been created and loved by many deep learning researchers. 14 | 15 | Although the *Roadmap List* includes lots of important deep learning papers, it feels overwhelming for me to read them all. As I mentioned in the introduction, I believe that seminal works can give us lessons regardless of their application domain. Thus, I would like to introduce **top 100 deep learning papers** here as a good starting point of overviewing deep learning researches. 16 | 17 | To get the news for newly released papers everyday, follow my [twitter](https://twitter.com/TerryUm_ML) or [facebook page](https://www.facebook.com/terryum.io/)! 18 | 19 | ## Awesome list criteria 20 | 21 | 1. A list of **top 100 deep learning papers** published from 2012 to 2016 is suggested. 22 | 2. If a paper is added to the list, another paper (usually from *More Papers from 2016" section) should be removed to keep top 100 papers. (Thus, removing papers is also important contributions as well as adding papers) 23 | 3. Papers that are important, but failed to be included in the list, will be listed in *More than Top 100* section. 24 | 4. Please refer to *New Papers* and *Old Papers* sections for the papers published in recent 6 months or before 2012. 25 | 26 | *(Citation criteria)* 27 | - **< 6 months** : *New Papers* (by discussion) 28 | - **2016** : +60 citations or "More Papers from 2016" 29 | - **2015** : +200 citations 30 | - **2014** : +400 citations 31 | - **2013** : +600 citations 32 | - **2012** : +800 citations 33 | - **~2012** : *Old Papers* (by discussion) 34 | 35 | Please note that we prefer seminal deep learning papers that can be applied to various researches rather than application papers. For that reason, some papers that meet the criteria may not be accepted while others can be. It depends on the impact of the paper, applicability to other researches scarcity of the research domain, and so on. 36 | 37 | **We need your contributions!** 38 | 39 | If you have any suggestions (missing papers, new papers, key researchers or typos), please feel free to edit and pull a request. 40 | (Please read the [contributing guide](https://github.com/terryum/awesome-deep-learning-papers/blob/master/Contributing.md) for further instructions, though just letting me know the title of papers can also be a big contribution to us.) 41 | 42 | (Update) You can download all top-100 papers with [this](https://github.com/terryum/awesome-deep-learning-papers/blob/master/fetch_papers.py) and collect all authors' names with [this](https://github.com/terryum/awesome-deep-learning-papers/blob/master/get_authors.py). Also, [bib file](https://github.com/terryum/awesome-deep-learning-papers/blob/master/top100papers.bib) for all top-100 papers are available. Thanks, doodhwala, [Sven](https://github.com/sunshinemyson) and [grepinsight](https://github.com/grepinsight)! 43 | 44 | + Can anyone contribute the code for obtaining the statistics of the authors of Top-100 papers? 45 | 46 | 47 | ## Contents 48 | 49 | * [Understanding / Generalization / Transfer](#understanding--generalization--transfer) 50 | * [Optimization / Training Techniques](#optimization--training-techniques) 51 | * [Unsupervised / Generative Models](#unsupervised--generative-models) 52 | * [Convolutional Network Models](#convolutional-neural-network-models) 53 | * [Image Segmentation / Object Detection](#image-segmentation--object-detection) 54 | * [Image / Video / Etc](#image--video--etc) 55 | * [Natural Language Processing / RNNs](#natural-language-processing--rnns) 56 | * [Speech / Other Domain](#speech--other-domain) 57 | * [Reinforcement Learning / Robotics](#reinforcement-learning--robotics) 58 | * [More Papers from 2016](#more-papers-from-2016) 59 | 60 | *(More than Top 100)* 61 | 62 | * [New Papers](#new-papers) : Less than 6 months 63 | * [Old Papers](#old-papers) : Before 2012 64 | * [HW / SW / Dataset](#hw--sw--dataset) : Technical reports 65 | * [Book / Survey / Review](#book--survey--review) 66 | * [Video Lectures / Tutorials / Blogs](#video-lectures--tutorials--blogs) 67 | * [Appendix: More than Top 100](#appendix-more-than-top-100) : More papers not in the list 68 | 69 | * * * 70 | 71 | ### Understanding / Generalization / Transfer 72 | - **Distilling the knowledge in a neural network** (2015), G. Hinton et al. [[pdf]](http://arxiv.org/pdf/1503.02531) 73 | - **Deep neural networks are easily fooled: High confidence predictions for unrecognizable images** (2015), A. Nguyen et al. [[pdf]](http://arxiv.org/pdf/1412.1897) 74 | - **How transferable are features in deep neural networks?** (2014), J. Yosinski et al. [[pdf]](http://papers.nips.cc/paper/5347-how-transferable-are-features-in-deep-neural-networks.pdf) 75 | - **CNN features off-the-Shelf: An astounding baseline for recognition** (2014), A. Razavian et al. [[pdf]](http://www.cv-foundation.org//openaccess/content_cvpr_workshops_2014/W15/papers/Razavian_CNN_Features_Off-the-Shelf_2014_CVPR_paper.pdf) 76 | - **Learning and transferring mid-Level image representations using convolutional neural networks** (2014), M. Oquab et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2014/papers/Oquab_Learning_and_Transferring_2014_CVPR_paper.pdf) 77 | - **Visualizing and understanding convolutional networks** (2014), M. Zeiler and R. Fergus [[pdf]](http://arxiv.org/pdf/1311.2901) 78 | - **Decaf: A deep convolutional activation feature for generic visual recognition** (2014), J. Donahue et al. [[pdf]](http://arxiv.org/pdf/1310.1531) 79 | 80 | 81 | 82 | ### Optimization / Training Techniques 83 | - **Training very deep networks** (2015), R. Srivastava et al. [[pdf]](http://papers.nips.cc/paper/5850-training-very-deep-networks.pdf) 84 | - **Batch normalization: Accelerating deep network training by reducing internal covariate shift** (2015), S. Loffe and C. Szegedy [[pdf]](http://arxiv.org/pdf/1502.03167) 85 | - **Delving deep into rectifiers: Surpassing human-level performance on imagenet classification** (2015), K. He et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/He_Delving_Deep_into_ICCV_2015_paper.pdf) 86 | - **Dropout: A simple way to prevent neural networks from overfitting** (2014), N. Srivastava et al. [[pdf]](http://jmlr.org/papers/volume15/srivastava14a/srivastava14a.pdf) 87 | - **Adam: A method for stochastic optimization** (2014), D. Kingma and J. Ba [[pdf]](http://arxiv.org/pdf/1412.6980) 88 | - **Improving neural networks by preventing co-adaptation of feature detectors** (2012), G. Hinton et al. [[pdf]](http://arxiv.org/pdf/1207.0580.pdf) 89 | - **Random search for hyper-parameter optimization** (2012) J. Bergstra and Y. Bengio [[pdf]](http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a) 90 | 91 | 92 | 93 | ### Unsupervised / Generative Models 94 | - **Pixel recurrent neural networks** (2016), A. Oord et al. [[pdf]](http://arxiv.org/pdf/1601.06759v2.pdf) 95 | - **Improved techniques for training GANs** (2016), T. Salimans et al. [[pdf]](http://papers.nips.cc/paper/6125-improved-techniques-for-training-gans.pdf) 96 | - **Unsupervised representation learning with deep convolutional generative adversarial networks** (2015), A. Radford et al. [[pdf]](https://arxiv.org/pdf/1511.06434v2) 97 | - **DRAW: A recurrent neural network for image generation** (2015), K. Gregor et al. [[pdf]](http://arxiv.org/pdf/1502.04623) 98 | - **Generative adversarial nets** (2014), I. Goodfellow et al. [[pdf]](http://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf) 99 | - **Auto-encoding variational Bayes** (2013), D. Kingma and M. Welling [[pdf]](http://arxiv.org/pdf/1312.6114) 100 | - **Building high-level features using large scale unsupervised learning** (2013), Q. Le et al. [[pdf]](http://arxiv.org/pdf/1112.6209) 101 | 102 | 103 | ### Convolutional Neural Network Models 104 | - **Rethinking the inception architecture for computer vision** (2016), C. Szegedy et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Szegedy_Rethinking_the_Inception_CVPR_2016_paper.pdf) 105 | - **Inception-v4, inception-resnet and the impact of residual connections on learning** (2016), C. Szegedy et al. [[pdf]](http://arxiv.org/pdf/1602.07261) 106 | - **Identity Mappings in Deep Residual Networks** (2016), K. He et al. [[pdf]](https://arxiv.org/pdf/1603.05027v2.pdf) 107 | - **Deep residual learning for image recognition** (2016), K. He et al. [[pdf]](http://arxiv.org/pdf/1512.03385) 108 | - **Spatial transformer network** (2015), M. Jaderberg et al., [[pdf]](http://papers.nips.cc/paper/5854-spatial-transformer-networks.pdf) 109 | - **Going deeper with convolutions** (2015), C. Szegedy et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Szegedy_Going_Deeper_With_2015_CVPR_paper.pdf) 110 | - **Very deep convolutional networks for large-scale image recognition** (2014), K. Simonyan and A. Zisserman [[pdf]](http://arxiv.org/pdf/1409.1556) 111 | - **Return of the devil in the details: delving deep into convolutional nets** (2014), K. Chatfield et al. [[pdf]](http://arxiv.org/pdf/1405.3531) 112 | - **OverFeat: Integrated recognition, localization and detection using convolutional networks** (2013), P. Sermanet et al. [[pdf]](http://arxiv.org/pdf/1312.6229) 113 | - **Maxout networks** (2013), I. Goodfellow et al. [[pdf]](http://arxiv.org/pdf/1302.4389v4) 114 | - **Network in network** (2013), M. Lin et al. [[pdf]](http://arxiv.org/pdf/1312.4400) 115 | - **ImageNet classification with deep convolutional neural networks** (2012), A. Krizhevsky et al. [[pdf]](http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf) 116 | 117 | 118 | 119 | ### Image: Segmentation / Object Detection 120 | - **You only look once: Unified, real-time object detection** (2016), J. Redmon et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Redmon_You_Only_Look_CVPR_2016_paper.pdf) 121 | - **Fully convolutional networks for semantic segmentation** (2015), J. Long et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Long_Fully_Convolutional_Networks_2015_CVPR_paper.pdf) 122 | - **Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks** (2015), S. Ren et al. [[pdf]](http://papers.nips.cc/paper/5638-faster-r-cnn-towards-real-time-object-detection-with-region-proposal-networks.pdf) 123 | - **Fast R-CNN** (2015), R. Girshick [[pdf]](http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Girshick_Fast_R-CNN_ICCV_2015_paper.pdf) 124 | - **Rich feature hierarchies for accurate object detection and semantic segmentation** (2014), R. Girshick et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2014/papers/Girshick_Rich_Feature_Hierarchies_2014_CVPR_paper.pdf) 125 | - **Spatial pyramid pooling in deep convolutional networks for visual recognition** (2014), K. He et al. [[pdf]](http://arxiv.org/pdf/1406.4729) 126 | - **Semantic image segmentation with deep convolutional nets and fully connected CRFs**, L. Chen et al. [[pdf]](https://arxiv.org/pdf/1412.7062) 127 | - **Learning hierarchical features for scene labeling** (2013), C. Farabet et al. [[pdf]](https://hal-enpc.archives-ouvertes.fr/docs/00/74/20/77/PDF/farabet-pami-13.pdf) 128 | 129 | 130 | 131 | ### Image / Video / Etc 132 | - **Image Super-Resolution Using Deep Convolutional Networks** (2016), C. Dong et al. [[pdf]](https://arxiv.org/pdf/1501.00092v3.pdf) 133 | - **A neural algorithm of artistic style** (2015), L. Gatys et al. [[pdf]](https://arxiv.org/pdf/1508.06576) 134 | - **Deep visual-semantic alignments for generating image descriptions** (2015), A. Karpathy and L. Fei-Fei [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Karpathy_Deep_Visual-Semantic_Alignments_2015_CVPR_paper.pdf) 135 | - **Show, attend and tell: Neural image caption generation with visual attention** (2015), K. Xu et al. [[pdf]](http://arxiv.org/pdf/1502.03044) 136 | - **Show and tell: A neural image caption generator** (2015), O. Vinyals et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Vinyals_Show_and_Tell_2015_CVPR_paper.pdf) 137 | - **Long-term recurrent convolutional networks for visual recognition and description** (2015), J. Donahue et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Donahue_Long-Term_Recurrent_Convolutional_2015_CVPR_paper.pdf) 138 | - **VQA: Visual question answering** (2015), S. Antol et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Antol_VQA_Visual_Question_ICCV_2015_paper.pdf) 139 | - **DeepFace: Closing the gap to human-level performance in face verification** (2014), Y. Taigman et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2014/papers/Taigman_DeepFace_Closing_the_2014_CVPR_paper.pdf): 140 | - **Large-scale video classification with convolutional neural networks** (2014), A. Karpathy et al. [[pdf]](http://vision.stanford.edu/pdf/karpathy14.pdf) 141 | - **Two-stream convolutional networks for action recognition in videos** (2014), K. Simonyan et al. [[pdf]](http://papers.nips.cc/paper/5353-two-stream-convolutional-networks-for-action-recognition-in-videos.pdf) 142 | - **3D convolutional neural networks for human action recognition** (2013), S. Ji et al. [[pdf]](http://machinelearning.wustl.edu/mlpapers/paper_files/icml2010_JiXYY10.pdf) 143 | 144 | 145 | 146 | 147 | 148 | ### Natural Language Processing / RNNs 149 | - **Neural Architectures for Named Entity Recognition** (2016), G. Lample et al. [[pdf]](http://aclweb.org/anthology/N/N16/N16-1030.pdf) 150 | - **Exploring the limits of language modeling** (2016), R. Jozefowicz et al. [[pdf]](http://arxiv.org/pdf/1602.02410) 151 | - **Teaching machines to read and comprehend** (2015), K. Hermann et al. [[pdf]](http://papers.nips.cc/paper/5945-teaching-machines-to-read-and-comprehend.pdf) 152 | - **Effective approaches to attention-based neural machine translation** (2015), M. Luong et al. [[pdf]](https://arxiv.org/pdf/1508.04025) 153 | - **Conditional random fields as recurrent neural networks** (2015), S. Zheng and S. Jayasumana. [[pdf]](http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Zheng_Conditional_Random_Fields_ICCV_2015_paper.pdf) 154 | - **Memory networks** (2014), J. Weston et al. [[pdf]](https://arxiv.org/pdf/1410.3916) 155 | - **Neural turing machines** (2014), A. Graves et al. [[pdf]](https://arxiv.org/pdf/1410.5401) 156 | - **Neural machine translation by jointly learning to align and translate** (2014), D. Bahdanau et al. [[pdf]](http://arxiv.org/pdf/1409.0473) 157 | - **Sequence to sequence learning with neural networks** (2014), I. Sutskever et al. [[pdf]](http://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf) 158 | - **Learning phrase representations using RNN encoder-decoder for statistical machine translation** (2014), K. Cho et al. [[pdf]](http://arxiv.org/pdf/1406.1078) 159 | - **A convolutional neural network for modeling sentences** (2014), N. Kalchbrenner et al. [[pdf]](http://arxiv.org/pdf/1404.2188v1) 160 | - **Convolutional neural networks for sentence classification** (2014), Y. Kim [[pdf]](http://arxiv.org/pdf/1408.5882) 161 | - **Glove: Global vectors for word representation** (2014), J. Pennington et al. [[pdf]](http://anthology.aclweb.org/D/D14/D14-1162.pdf) 162 | - **Distributed representations of sentences and documents** (2014), Q. Le and T. Mikolov [[pdf]](http://arxiv.org/pdf/1405.4053) 163 | - **Distributed representations of words and phrases and their compositionality** (2013), T. Mikolov et al. [[pdf]](http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf) 164 | - **Efficient estimation of word representations in vector space** (2013), T. Mikolov et al. [[pdf]](http://arxiv.org/pdf/1301.3781) 165 | - **Recursive deep models for semantic compositionality over a sentiment treebank** (2013), R. Socher et al. [[pdf]](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.383.1327&rep=rep1&type=pdf) 166 | - **Generating sequences with recurrent neural networks** (2013), A. Graves. [[pdf]](https://arxiv.org/pdf/1308.0850) 167 | 168 | 169 | 170 | ### Speech / Other Domain 171 | - **End-to-end attention-based large vocabulary speech recognition** (2016), D. Bahdanau et al. [[pdf]](https://arxiv.org/pdf/1508.04395) 172 | - **Deep speech 2: End-to-end speech recognition in English and Mandarin** (2015), D. Amodei et al. [[pdf]](https://arxiv.org/pdf/1512.02595) 173 | - **Speech recognition with deep recurrent neural networks** (2013), A. Graves [[pdf]](http://arxiv.org/pdf/1303.5778.pdf) 174 | - **Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups** (2012), G. Hinton et al. [[pdf]](http://www.cs.toronto.edu/~asamir/papers/SPM_DNN_12.pdf) 175 | - **Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition** (2012) G. Dahl et al. [[pdf]](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.337.7548&rep=rep1&type=pdf) 176 | - **Acoustic modeling using deep belief networks** (2012), A. Mohamed et al. [[pdf]](http://www.cs.toronto.edu/~asamir/papers/speechDBN_jrnl.pdf) 177 | 178 | 179 | 180 | ### Reinforcement Learning / Robotics 181 | - **End-to-end training of deep visuomotor policies** (2016), S. Levine et al. [[pdf]](http://www.jmlr.org/papers/volume17/15-522/source/15-522.pdf) 182 | - **Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection** (2016), S. Levine et al. [[pdf]](https://arxiv.org/pdf/1603.02199) 183 | - **Asynchronous methods for deep reinforcement learning** (2016), V. Mnih et al. [[pdf]](http://www.jmlr.org/proceedings/papers/v48/mniha16.pdf) 184 | - **Deep Reinforcement Learning with Double Q-Learning** (2016), H. Hasselt et al. [[pdf]](https://arxiv.org/pdf/1509.06461.pdf ) 185 | - **Mastering the game of Go with deep neural networks and tree search** (2016), D. Silver et al. [[pdf]](http://www.nature.com/nature/journal/v529/n7587/full/nature16961.html) 186 | - **Continuous control with deep reinforcement learning** (2015), T. Lillicrap et al. [[pdf]](https://arxiv.org/pdf/1509.02971) 187 | - **Human-level control through deep reinforcement learning** (2015), V. Mnih et al. [[pdf]](http://www.davidqiu.com:8888/research/nature14236.pdf) 188 | - **Deep learning for detecting robotic grasps** (2015), I. Lenz et al. [[pdf]](http://www.cs.cornell.edu/~asaxena/papers/lenz_lee_saxena_deep_learning_grasping_ijrr2014.pdf) 189 | - **Playing atari with deep reinforcement learning** (2013), V. Mnih et al. [[pdf]](http://arxiv.org/pdf/1312.5602.pdf)) 190 | 191 | 192 | 193 | ### More Papers from 2016 194 | - **Layer Normalization** (2016), J. Ba et al. [[pdf]](https://arxiv.org/pdf/1607.06450v1.pdf) 195 | - **Learning to learn by gradient descent by gradient descent** (2016), M. Andrychowicz et al. [[pdf]](http://arxiv.org/pdf/1606.04474v1) 196 | - **Domain-adversarial training of neural networks** (2016), Y. Ganin et al. [[pdf]](http://www.jmlr.org/papers/volume17/15-239/source/15-239.pdf) 197 | - **WaveNet: A Generative Model for Raw Audio** (2016), A. Oord et al. [[pdf]](https://arxiv.org/pdf/1609.03499v2) [[web]](https://deepmind.com/blog/wavenet-generative-model-raw-audio/) 198 | - **Colorful image colorization** (2016), R. Zhang et al. [[pdf]](https://arxiv.org/pdf/1603.08511) 199 | - **Generative visual manipulation on the natural image manifold** (2016), J. Zhu et al. [[pdf]](https://arxiv.org/pdf/1609.03552) 200 | - **Texture networks: Feed-forward synthesis of textures and stylized images** (2016), D Ulyanov et al. [[pdf]](http://www.jmlr.org/proceedings/papers/v48/ulyanov16.pdf) 201 | - **SSD: Single shot multibox detector** (2016), W. Liu et al. [[pdf]](https://arxiv.org/pdf/1512.02325) 202 | - **SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 1MB model size** (2016), F. Iandola et al. [[pdf]](http://arxiv.org/pdf/1602.07360) 203 | - **Eie: Efficient inference engine on compressed deep neural network** (2016), S. Han et al. [[pdf]](http://arxiv.org/pdf/1602.01528) 204 | - **Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1** (2016), M. Courbariaux et al. [[pdf]](https://arxiv.org/pdf/1602.02830) 205 | - **Dynamic memory networks for visual and textual question answering** (2016), C. Xiong et al. [[pdf]](http://www.jmlr.org/proceedings/papers/v48/xiong16.pdf) 206 | - **Stacked attention networks for image question answering** (2016), Z. Yang et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Yang_Stacked_Attention_Networks_CVPR_2016_paper.pdf) 207 | - **Hybrid computing using a neural network with dynamic external memory** (2016), A. Graves et al. [[pdf]](https://www.gwern.net/docs/2016-graves.pdf) 208 | - **Google's neural machine translation system: Bridging the gap between human and machine translation** (2016), Y. Wu et al. [[pdf]](https://arxiv.org/pdf/1609.08144) 209 | 210 | * * * 211 | 212 | 213 | ### New papers 214 | *Newly published papers (< 6 months) which are worth reading* 215 | - MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications (2017), Andrew G. Howard et al. [[pdf]](https://arxiv.org/pdf/1704.04861.pdf) 216 | - Convolutional Sequence to Sequence Learning (2017), Jonas Gehring et al. [[pdf]](https://arxiv.org/pdf/1705.03122) 217 | - A Knowledge-Grounded Neural Conversation Model (2017), Marjan Ghazvininejad et al. [[pdf]](https://arxiv.org/pdf/1702.01932) 218 | - Accurate, Large Minibatch SGD:Training ImageNet in 1 Hour (2017), Priya Goyal et al. [[pdf]](https://research.fb.com/wp-content/uploads/2017/06/imagenet1kin1h3.pdf) 219 | - TACOTRON: Towards end-to-end speech synthesis (2017), Y. Wang et al. [[pdf]](https://arxiv.org/pdf/1703.10135.pdf) 220 | - Deep Photo Style Transfer (2017), F. Luan et al. [[pdf]](http://arxiv.org/pdf/1703.07511v1.pdf) 221 | - Evolution Strategies as a Scalable Alternative to Reinforcement Learning (2017), T. Salimans et al. [[pdf]](http://arxiv.org/pdf/1703.03864v1.pdf) 222 | - Deformable Convolutional Networks (2017), J. Dai et al. [[pdf]](http://arxiv.org/pdf/1703.06211v2.pdf) 223 | - Mask R-CNN (2017), K. He et al. [[pdf]](https://128.84.21.199/pdf/1703.06870) 224 | - Learning to discover cross-domain relations with generative adversarial networks (2017), T. Kim et al. [[pdf]](http://arxiv.org/pdf/1703.05192v1.pdf) 225 | - Deep voice: Real-time neural text-to-speech (2017), S. Arik et al., [[pdf]](http://arxiv.org/pdf/1702.07825v2.pdf) 226 | - PixelNet: Representation of the pixels, by the pixels, and for the pixels (2017), A. Bansal et al. [[pdf]](http://arxiv.org/pdf/1702.06506v1.pdf) 227 | - Batch renormalization: Towards reducing minibatch dependence in batch-normalized models (2017), S. Ioffe. [[pdf]](https://arxiv.org/abs/1702.03275) 228 | - Wasserstein GAN (2017), M. Arjovsky et al. [[pdf]](https://arxiv.org/pdf/1701.07875v1) 229 | - Understanding deep learning requires rethinking generalization (2017), C. Zhang et al. [[pdf]](https://arxiv.org/pdf/1611.03530) 230 | - Least squares generative adversarial networks (2016), X. Mao et al. [[pdf]](https://arxiv.org/abs/1611.04076v2) 231 | 232 | 233 | ### Old Papers 234 | *Classic papers published before 2012* 235 | - An analysis of single-layer networks in unsupervised feature learning (2011), A. Coates et al. [[pdf]](http://machinelearning.wustl.edu/mlpapers/paper_files/AISTATS2011_CoatesNL11.pdf) 236 | - Deep sparse rectifier neural networks (2011), X. Glorot et al. [[pdf]](http://machinelearning.wustl.edu/mlpapers/paper_files/AISTATS2011_GlorotBB11.pdf) 237 | - Natural language processing (almost) from scratch (2011), R. Collobert et al. [[pdf]](http://arxiv.org/pdf/1103.0398) 238 | - Recurrent neural network based language model (2010), T. Mikolov et al. [[pdf]](http://www.fit.vutbr.cz/research/groups/speech/servite/2010/rnnlm_mikolov.pdf) 239 | - Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion (2010), P. Vincent et al. [[pdf]](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.297.3484&rep=rep1&type=pdf) 240 | - Learning mid-level features for recognition (2010), Y. Boureau [[pdf]](http://ece.duke.edu/~lcarin/boureau-cvpr-10.pdf) 241 | - A practical guide to training restricted boltzmann machines (2010), G. Hinton [[pdf]](http://www.csri.utoronto.ca/~hinton/absps/guideTR.pdf) 242 | - Understanding the difficulty of training deep feedforward neural networks (2010), X. Glorot and Y. Bengio [[pdf]](http://machinelearning.wustl.edu/mlpapers/paper_files/AISTATS2010_GlorotB10.pdf) 243 | - Why does unsupervised pre-training help deep learning (2010), D. Erhan et al. [[pdf]](http://machinelearning.wustl.edu/mlpapers/paper_files/AISTATS2010_ErhanCBV10.pdf) 244 | - Learning deep architectures for AI (2009), Y. Bengio. [[pdf]](http://sanghv.com/download/soft/machine%20learning,%20artificial%20intelligence,%20mathematics%20ebooks/ML/learning%20deep%20architectures%20for%20AI%20(2009).pdf) 245 | - Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations (2009), H. Lee et al. [[pdf]](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.149.802&rep=rep1&type=pdf) 246 | - Greedy layer-wise training of deep networks (2007), Y. Bengio et al. [[pdf]](http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2006_739.pdf) 247 | - Reducing the dimensionality of data with neural networks, G. Hinton and R. Salakhutdinov. [[pdf]](http://homes.mpimf-heidelberg.mpg.de/~mhelmsta/pdf/2006%20Hinton%20Salakhudtkinov%20Science.pdf) 248 | - A fast learning algorithm for deep belief nets (2006), G. Hinton et al. [[pdf]](http://nuyoo.utm.mx/~jjf/rna/A8%20A%20fast%20learning%20algorithm%20for%20deep%20belief%20nets.pdf) 249 | - Gradient-based learning applied to document recognition (1998), Y. LeCun et al. [[pdf]](http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf) 250 | - Long short-term memory (1997), S. Hochreiter and J. Schmidhuber. [[pdf]](http://www.mitpressjournals.org/doi/pdfplus/10.1162/neco.1997.9.8.1735) 251 | 252 | 253 | ### HW / SW / Dataset 254 | - SQuAD: 100,000+ Questions for Machine Comprehension of Text (2016), Rajpurkar et al. [[pdf]](https://arxiv.org/pdf/1606.05250.pdf) 255 | - OpenAI gym (2016), G. Brockman et al. [[pdf]](https://arxiv.org/pdf/1606.01540) 256 | - TensorFlow: Large-scale machine learning on heterogeneous distributed systems (2016), M. Abadi et al. [[pdf]](http://arxiv.org/pdf/1603.04467) 257 | - Theano: A Python framework for fast computation of mathematical expressions, R. Al-Rfou et al. 258 | - Torch7: A matlab-like environment for machine learning, R. Collobert et al. [[pdf]](https://ronan.collobert.com/pub/matos/2011_torch7_nipsw.pdf) 259 | - MatConvNet: Convolutional neural networks for matlab (2015), A. Vedaldi and K. Lenc [[pdf]](http://arxiv.org/pdf/1412.4564) 260 | - Imagenet large scale visual recognition challenge (2015), O. Russakovsky et al. [[pdf]](http://arxiv.org/pdf/1409.0575) 261 | - Caffe: Convolutional architecture for fast feature embedding (2014), Y. Jia et al. [[pdf]](http://arxiv.org/pdf/1408.5093) 262 | 263 | 264 | ### Book / Survey / Review 265 | - On the Origin of Deep Learning (2017), H. Wang and Bhiksha Raj. [[pdf]](https://arxiv.org/pdf/1702.07800) 266 | - Deep Reinforcement Learning: An Overview (2017), Y. Li, [[pdf]](http://arxiv.org/pdf/1701.07274v2.pdf) 267 | - Neural Machine Translation and Sequence-to-sequence Models(2017): A Tutorial, G. Neubig. [[pdf]](http://arxiv.org/pdf/1703.01619v1.pdf) 268 | - Neural Network and Deep Learning (Book, Jan 2017), Michael Nielsen. [[html]](http://neuralnetworksanddeeplearning.com/index.html) 269 | - Deep learning (Book, 2016), Goodfellow et al. [[html]](http://www.deeplearningbook.org/) 270 | - LSTM: A search space odyssey (2016), K. Greff et al. [[pdf]](https://arxiv.org/pdf/1503.04069.pdf?utm_content=buffereddc5&utm_medium=social&utm_source=plus.google.com&utm_campaign=buffer) 271 | - Tutorial on Variational Autoencoders (2016), C. Doersch. [[pdf]](https://arxiv.org/pdf/1606.05908) 272 | - Deep learning (2015), Y. LeCun, Y. Bengio and G. Hinton [[pdf]](https://www.cs.toronto.edu/~hinton/absps/NatureDeepReview.pdf) 273 | - Deep learning in neural networks: An overview (2015), J. Schmidhuber [[pdf]](http://arxiv.org/pdf/1404.7828) 274 | - Representation learning: A review and new perspectives (2013), Y. Bengio et al. [[pdf]](http://arxiv.org/pdf/1206.5538) 275 | 276 | ### Video Lectures / Tutorials / Blogs 277 | 278 | *(Lectures)* 279 | - CS231n, Convolutional Neural Networks for Visual Recognition, Stanford University [[web]](http://cs231n.stanford.edu/) 280 | - CS224d, Deep Learning for Natural Language Processing, Stanford University [[web]](http://cs224d.stanford.edu/) 281 | - Oxford Deep NLP 2017, Deep Learning for Natural Language Processing, University of Oxford [[web]](https://github.com/oxford-cs-deepnlp-2017/lectures) 282 | 283 | *(Tutorials)* 284 | - NIPS 2016 Tutorials, Long Beach [[web]](https://nips.cc/Conferences/2016/Schedule?type=Tutorial) 285 | - ICML 2016 Tutorials, New York City [[web]](http://techtalks.tv/icml/2016/tutorials/) 286 | - ICLR 2016 Videos, San Juan [[web]](http://videolectures.net/iclr2016_san_juan/) 287 | - Deep Learning Summer School 2016, Montreal [[web]](http://videolectures.net/deeplearning2016_montreal/) 288 | - Bay Area Deep Learning School 2016, Stanford [[web]](https://www.bayareadlschool.org/) 289 | 290 | *(Blogs)* 291 | - OpenAI [[web]](https://www.openai.com/) 292 | - Distill [[web]](http://distill.pub/) 293 | - Andrej Karpathy Blog [[web]](http://karpathy.github.io/) 294 | - Colah's Blog [[Web]](http://colah.github.io/) 295 | - WildML [[Web]](http://www.wildml.com/) 296 | - FastML [[web]](http://www.fastml.com/) 297 | - TheMorningPaper [[web]](https://blog.acolyer.org) 298 | 299 | ### Appendix: More than Top 100 300 | *(2016)* 301 | - A character-level decoder without explicit segmentation for neural machine translation (2016), J. Chung et al. [[pdf]](https://arxiv.org/pdf/1603.06147) 302 | - Dermatologist-level classification of skin cancer with deep neural networks (2017), A. Esteva et al. [[html]](http://www.nature.com/nature/journal/v542/n7639/full/nature21056.html) 303 | - Weakly supervised object localization with multi-fold multiple instance learning (2017), R. Gokberk et al. [[pdf]](https://arxiv.org/pdf/1503.00949) 304 | - Brain tumor segmentation with deep neural networks (2017), M. Havaei et al. [[pdf]](https://arxiv.org/pdf/1505.03540) 305 | - Professor Forcing: A New Algorithm for Training Recurrent Networks (2016), A. Lamb et al. [[pdf]](https://arxiv.org/pdf/1610.09038) 306 | - Adversarially learned inference (2016), V. Dumoulin et al. [[web]](https://ishmaelbelghazi.github.io/ALI/)[[pdf]](https://arxiv.org/pdf/1606.00704v1) 307 | - Understanding convolutional neural networks (2016), J. Koushik [[pdf]](https://arxiv.org/pdf/1605.09081v1) 308 | - Taking the human out of the loop: A review of bayesian optimization (2016), B. Shahriari et al. [[pdf]](https://www.cs.ox.ac.uk/people/nando.defreitas/publications/BayesOptLoop.pdf) 309 | - Adaptive computation time for recurrent neural networks (2016), A. Graves [[pdf]](http://arxiv.org/pdf/1603.08983) 310 | - Densely connected convolutional networks (2016), G. Huang et al. [[pdf]](https://arxiv.org/pdf/1608.06993v1) 311 | - Region-based convolutional networks for accurate object detection and segmentation (2016), R. Girshick et al. 312 | - Continuous deep q-learning with model-based acceleration (2016), S. Gu et al. [[pdf]](http://www.jmlr.org/proceedings/papers/v48/gu16.pdf) 313 | - A thorough examination of the cnn/daily mail reading comprehension task (2016), D. Chen et al. [[pdf]](https://arxiv.org/pdf/1606.02858) 314 | - Achieving open vocabulary neural machine translation with hybrid word-character models, M. Luong and C. Manning. [[pdf]](https://arxiv.org/pdf/1604.00788) 315 | - Very Deep Convolutional Networks for Natural Language Processing (2016), A. Conneau et al. [[pdf]](https://arxiv.org/pdf/1606.01781) 316 | - Bag of tricks for efficient text classification (2016), A. Joulin et al. [[pdf]](https://arxiv.org/pdf/1607.01759) 317 | - Efficient piecewise training of deep structured models for semantic segmentation (2016), G. Lin et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Lin_Efficient_Piecewise_Training_CVPR_2016_paper.pdf) 318 | - Learning to compose neural networks for question answering (2016), J. Andreas et al. [[pdf]](https://arxiv.org/pdf/1601.01705) 319 | - Perceptual losses for real-time style transfer and super-resolution (2016), J. Johnson et al. [[pdf]](https://arxiv.org/pdf/1603.08155) 320 | - Reading text in the wild with convolutional neural networks (2016), M. Jaderberg et al. [[pdf]](http://arxiv.org/pdf/1412.1842) 321 | - What makes for effective detection proposals? (2016), J. Hosang et al. [[pdf]](https://arxiv.org/pdf/1502.05082) 322 | - Inside-outside net: Detecting objects in context with skip pooling and recurrent neural networks (2016), S. Bell et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Bell_Inside-Outside_Net_Detecting_CVPR_2016_paper.pdf). 323 | - Instance-aware semantic segmentation via multi-task network cascades (2016), J. Dai et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Dai_Instance-Aware_Semantic_Segmentation_CVPR_2016_paper.pdf) 324 | - Conditional image generation with pixelcnn decoders (2016), A. van den Oord et al. [[pdf]](http://papers.nips.cc/paper/6527-tree-structured-reinforcement-learning-for-sequential-object-localization.pdf) 325 | - Deep networks with stochastic depth (2016), G. Huang et al., [[pdf]](https://arxiv.org/pdf/1603.09382) 326 | - Consistency and Fluctuations For Stochastic Gradient Langevin Dynamics (2016), Yee Whye Teh et al. [[pdf]](http://www.jmlr.org/papers/volume17/teh16a/teh16a.pdf) 327 | 328 | *(2015)* 329 | - Ask your neurons: A neural-based approach to answering questions about images (2015), M. Malinowski et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Malinowski_Ask_Your_Neurons_ICCV_2015_paper.pdf) 330 | - Exploring models and data for image question answering (2015), M. Ren et al. [[pdf]](http://papers.nips.cc/paper/5640-stochastic-variational-inference-for-hidden-markov-models.pdf) 331 | - Are you talking to a machine? dataset and methods for multilingual image question (2015), H. Gao et al. [[pdf]](http://papers.nips.cc/paper/5641-are-you-talking-to-a-machine-dataset-and-methods-for-multilingual-image-question.pdf) 332 | - Mind's eye: A recurrent visual representation for image caption generation (2015), X. Chen and C. Zitnick. [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Chen_Minds_Eye_A_2015_CVPR_paper.pdf) 333 | - From captions to visual concepts and back (2015), H. Fang et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Fang_From_Captions_to_2015_CVPR_paper.pdf). 334 | - Towards AI-complete question answering: A set of prerequisite toy tasks (2015), J. Weston et al. [[pdf]](http://arxiv.org/pdf/1502.05698) 335 | - Ask me anything: Dynamic memory networks for natural language processing (2015), A. Kumar et al. [[pdf]](http://arxiv.org/pdf/1506.07285) 336 | - Unsupervised learning of video representations using LSTMs (2015), N. Srivastava et al. [[pdf]](http://www.jmlr.org/proceedings/papers/v37/srivastava15.pdf) 337 | - Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding (2015), S. Han et al. [[pdf]](https://arxiv.org/pdf/1510.00149) 338 | - Improved semantic representations from tree-structured long short-term memory networks (2015), K. Tai et al. [[pdf]](https://arxiv.org/pdf/1503.00075) 339 | - Character-aware neural language models (2015), Y. Kim et al. [[pdf]](https://arxiv.org/pdf/1508.06615) 340 | - Grammar as a foreign language (2015), O. Vinyals et al. [[pdf]](http://papers.nips.cc/paper/5635-grammar-as-a-foreign-language.pdf) 341 | - Trust Region Policy Optimization (2015), J. Schulman et al. [[pdf]](http://www.jmlr.org/proceedings/papers/v37/schulman15.pdf) 342 | - Beyond short snippents: Deep networks for video classification (2015) [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Ng_Beyond_Short_Snippets_2015_CVPR_paper.pdf) 343 | - Learning Deconvolution Network for Semantic Segmentation (2015), H. Noh et al. [[pdf]](https://arxiv.org/pdf/1505.04366v1) 344 | - Learning spatiotemporal features with 3d convolutional networks (2015), D. Tran et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Tran_Learning_Spatiotemporal_Features_ICCV_2015_paper.pdf) 345 | - Understanding neural networks through deep visualization (2015), J. Yosinski et al. [[pdf]](https://arxiv.org/pdf/1506.06579) 346 | - An Empirical Exploration of Recurrent Network Architectures (2015), R. Jozefowicz et al. [[pdf]](http://www.jmlr.org/proceedings/papers/v37/jozefowicz15.pdf) 347 | - Deep generative image models using a laplacian pyramid of adversarial networks (2015), E.Denton et al. [[pdf]](http://papers.nips.cc/paper/5773-deep-generative-image-models-using-a-laplacian-pyramid-of-adversarial-networks.pdf) 348 | - Gated Feedback Recurrent Neural Networks (2015), J. Chung et al. [[pdf]](http://www.jmlr.org/proceedings/papers/v37/chung15.pdf) 349 | - Fast and accurate deep network learning by exponential linear units (ELUS) (2015), D. Clevert et al. [[pdf]](https://arxiv.org/pdf/1511.07289.pdf%5Cnhttp://arxiv.org/abs/1511.07289%5Cnhttp://arxiv.org/abs/1511.07289) 350 | - Pointer networks (2015), O. Vinyals et al. [[pdf]](http://papers.nips.cc/paper/5866-pointer-networks.pdf) 351 | - Visualizing and Understanding Recurrent Networks (2015), A. Karpathy et al. [[pdf]](https://arxiv.org/pdf/1506.02078) 352 | - Attention-based models for speech recognition (2015), J. Chorowski et al. [[pdf]](http://papers.nips.cc/paper/5847-attention-based-models-for-speech-recognition.pdf) 353 | - End-to-end memory networks (2015), S. Sukbaatar et al. [[pdf]](http://papers.nips.cc/paper/5846-end-to-end-memory-networks.pdf) 354 | - Describing videos by exploiting temporal structure (2015), L. Yao et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Yao_Describing_Videos_by_ICCV_2015_paper.pdf) 355 | - A neural conversational model (2015), O. Vinyals and Q. Le. [[pdf]](https://arxiv.org/pdf/1506.05869.pdf) 356 | - Improving distributional similarity with lessons learned from word embeddings, O. Levy et al. [[pdf]] (https://www.transacl.org/ojs/index.php/tacl/article/download/570/124) 357 | - Transition-Based Dependency Parsing with Stack Long Short-Term Memory (2015), C. Dyer et al. [[pdf]](http://aclweb.org/anthology/P/P15/P15-1033.pdf) 358 | - Improved Transition-Based Parsing by Modeling Characters instead of Words with LSTMs (2015), M. Ballesteros et al. [[pdf]](http://aclweb.org/anthology/D/D15/D15-1041.pdf) 359 | - Finding function in form: Compositional character models for open vocabulary word representation (2015), W. Ling et al. [[pdf]](http://aclweb.org/anthology/D/D15/D15-1176.pdf) 360 | 361 | 362 | *(~2014)* 363 | - DeepPose: Human pose estimation via deep neural networks (2014), A. Toshev and C. Szegedy [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2014/papers/Toshev_DeepPose_Human_Pose_2014_CVPR_paper.pdf) 364 | - Learning a Deep Convolutional Network for Image Super-Resolution (2014, C. Dong et al. [[pdf]](https://www.researchgate.net/profile/Chen_Change_Loy/publication/264552416_Lecture_Notes_in_Computer_Science/links/53e583e50cf25d674e9c280e.pdf) 365 | - Recurrent models of visual attention (2014), V. Mnih et al. [[pdf]](http://arxiv.org/pdf/1406.6247.pdf) 366 | - Empirical evaluation of gated recurrent neural networks on sequence modeling (2014), J. Chung et al. [[pdf]](https://arxiv.org/pdf/1412.3555) 367 | - Addressing the rare word problem in neural machine translation (2014), M. Luong et al. [[pdf]](https://arxiv.org/pdf/1410.8206) 368 | - On the properties of neural machine translation: Encoder-decoder approaches (2014), K. Cho et. al. 369 | - Recurrent neural network regularization (2014), W. Zaremba et al. [[pdf]](http://arxiv.org/pdf/1409.2329) 370 | - Intriguing properties of neural networks (2014), C. Szegedy et al. [[pdf]](https://arxiv.org/pdf/1312.6199.pdf) 371 | - Towards end-to-end speech recognition with recurrent neural networks (2014), A. Graves and N. Jaitly. [[pdf]](http://www.jmlr.org/proceedings/papers/v32/graves14.pdf) 372 | - Scalable object detection using deep neural networks (2014), D. Erhan et al. [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2014/papers/Erhan_Scalable_Object_Detection_2014_CVPR_paper.pdf) 373 | - On the importance of initialization and momentum in deep learning (2013), I. Sutskever et al. [[pdf]](http://machinelearning.wustl.edu/mlpapers/paper_files/icml2013_sutskever13.pdf) 374 | - Regularization of neural networks using dropconnect (2013), L. Wan et al. [[pdf]](http://machinelearning.wustl.edu/mlpapers/paper_files/icml2013_wan13.pdf) 375 | - Learning Hierarchical Features for Scene Labeling (2013), C. Farabet et al. [[pdf]](https://hal-enpc.archives-ouvertes.fr/docs/00/74/20/77/PDF/farabet-pami-13.pdf) 376 | - Linguistic Regularities in Continuous Space Word Representations (2013), T. Mikolov et al. [[pdf]](http://www.aclweb.org/anthology/N13-1#page=784) 377 | - Large scale distributed deep networks (2012), J. Dean et al. [[pdf]](http://papers.nips.cc/paper/4687-large-scale-distributed-deep-networks.pdf) 378 | - A Fast and Accurate Dependency Parser using Neural Networks. Chen and Manning. [[pdf]](http://cs.stanford.edu/people/danqi/papers/emnlp2014.pdf) 379 | 380 | 381 | 382 | ## Acknowledgement 383 | 384 | Thank you for all your contributions. Please make sure to read the [contributing guide](https://github.com/terryum/awesome-deep-learning-papers/blob/master/Contributing.md) before you make a pull request. 385 | 386 | ## License 387 | [![CC0](http://mirrors.creativecommons.org/presskit/buttons/88x31/svg/cc-zero.svg)](https://creativecommons.org/publicdomain/zero/1.0/) 388 | 389 | To the extent possible under law, [Terry T. Um](https://www.facebook.com/terryum.io/) has waived all copyright and related or neighboring rights to this work. 390 | -------------------------------------------------------------------------------- /ReadingNotes.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | ## Robotics 5 | - [Learning Inverse Dynamics Models in O(n) time with LSTM networks](#learning-inverse-dynamics-models-in-on-time-with-lstm-networks) (2017), Elmar Rueckert, Moritz Nakatenus, Samuele Tosatto, and Jan Peters, Humanoid 2017. (2017/10/03 read). 6 | 7 | 8 | 9 | 10 | ### Learning Inverse Dynamics Models in O(n) time with LSTM networks 11 | * Limitation of the current methods 12 | * GPR (![Eq:On3])and LWR is not scalable 13 | * Contribution of this paper 14 | * 15 | 16 | **Bibtex**: to appear 17 | 18 | $$ 19 | o(n^{3}) 20 | $$ 21 | 22 | [Eq:On3]: http://rogercortesi.com/eqn/tempimagedir/eqn8190.png 23 | -------------------------------------------------------------------------------- /fetch_papers.py: -------------------------------------------------------------------------------- 1 | ''' 2 | Author: doodhwala 3 | Python3 script to fetch the top 100 papers 4 | ''' 5 | 6 | import os, re, requests, codecs 7 | 8 | directory = 'papers' 9 | if not os.path.exists(directory): 10 | os.makedirs(directory) 11 | 12 | papers = [] 13 | with codecs.open('README.md', encoding='utf-8', mode='r', buffering=1, errors='strict') as f: 14 | lines = f.read().split('\n') 15 | heading, section_path = '', '' 16 | for line in lines: 17 | if('###' in line): 18 | heading = line.strip().split('###')[1] 19 | win_restricted_chars = re.compile(r'[\^\/\\\:\*\?\"<>\|]') 20 | heading = win_restricted_chars.sub("", heading) 21 | section_path = os.path.join(directory, heading) 22 | if not os.path.exists(section_path): 23 | os.makedirs(section_path) 24 | if('[[pdf]]' in line): 25 | # The stars ensure you pick up only the top 100 papers 26 | # Modify the expression if you want to fetch all other papers as well 27 | result = re.search('\*\*(.*?)\*\*.*?\[\[pdf\]\]\((.*?)\)', line) 28 | if(result): 29 | paper, url = result.groups() 30 | paper = win_restricted_chars.sub("", paper) 31 | # Auto - resume functionality 32 | if(not os.path.exists(os.path.join(section_path, paper + '.pdf'))): 33 | print('Fetching', paper) 34 | try: 35 | response = requests.get(url) 36 | with open(os.path.join(section_path, paper + '.pdf'), 'wb') as f: 37 | f.write(response.content) 38 | except requests.exceptions.RequestException as e: 39 | print("Error: {}".format(e)) 40 | -------------------------------------------------------------------------------- /get_authors.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | import re 5 | import requests 6 | from html.parser import HTMLParser 7 | import codecs 8 | 9 | search_engine="https://www.semanticscholar.org/search?q=" 10 | post_fix = "&sort=relevance&ae=false" 11 | 12 | class AuthorParser( HTMLParser ): 13 | tail_string = "" #contains the last tag's name which point to author field 14 | m_Stop = False 15 | m_authors = [] 16 | def handle_starttag(self, tag, attr): 17 | if self.m_Stop: 18 | return 19 | if tag == 'article': 20 | self.tail_string += tag 21 | return 22 | if self.tail_string != "": 23 | #print("search already kick-off") 24 | self.tail_string = self.tail_string+"."+tag 25 | #print(self.tail_string) 26 | def handle_endtag(self, tag): 27 | if self.m_Stop : 28 | return 29 | if self.tail_string == "article": 30 | # ONLY handle the first article 31 | self.m_Stop = True 32 | if self.tail_string != "": 33 | tags = self.tail_string.split('.') 34 | tags.reverse() 35 | for t in tags: 36 | if t == tag: 37 | tags.remove(t) 38 | break 39 | self.tail_string = "" 40 | tags.reverse() 41 | for i,t in enumerate(tags): 42 | self.tail_string = self.tail_string + "." + t if i > 0 else t 43 | 44 | def handle_data(self, data): 45 | if self.m_Stop: 46 | return 47 | if self.tail_string == "article.header.ul.li.span.span.a.span.span": 48 | #print(data) 49 | self.m_authors.append(data) 50 | 51 | def get_authors(self): 52 | return self.m_authors 53 | 54 | def clean(self): 55 | self.m_authors = [] 56 | self.tail_string= "" 57 | self.m_Stop = False 58 | 59 | 60 | def getPaperNames( readme_file ): 61 | paper_list = [] 62 | with codecs.open( readme_file,encoding='utf-8',mode='r',buffering = 1, errors='strict' ) as f: 63 | lines = f.read().split('\n') 64 | heading, section_path = '', '' 65 | for line in lines: 66 | if('###' in line): 67 | heading = line.strip().split('###')[1] 68 | heading = heading.replace('/', '|') 69 | 70 | if('[[pdf]]' in line): 71 | # The stars ensure you pick up only the top 100 papers 72 | # Modify the expression if you want to fetch all other papers as well 73 | result = re.search('\*\*(.*?)\*\*.*?\[\[pdf\]\]\((.*?)\)', line) 74 | if(result): 75 | paper, url = result.groups() 76 | paper_list.append(paper) 77 | 78 | return paper_list 79 | 80 | all_papers = getPaperNames("README.md") 81 | 82 | author_parser = AuthorParser() 83 | author_dict = {} 84 | for index,paper in enumerate(all_papers): 85 | paper.replace(" ", "%20") 86 | search_result = requests.get(search_engine + paper + post_fix) 87 | author_parser.feed(search_result.text) 88 | #print( paper, '==>', author_parser.get_authors() ) 89 | authors = author_parser.get_authors() 90 | for weight, author in enumerate( authors): 91 | if author not in author_dict.keys(): 92 | author_dict[author] = [] 93 | 94 | author_dict[author].append( (weight+1,paper)) 95 | author_parser.clean() 96 | print("Processed %d |"%(index), paper) 97 | 98 | # example usage of author information 99 | with open( "author.csv",'w') as fcsv: 100 | for (author, papers) in author_dict.items(): 101 | score = 0.0 102 | for (weight, paper) in papers: 103 | score += 1.0/weight 104 | print(author," score: %.2f"%score) 105 | fcsv.write( author+','+"%.2f"%score) 106 | 107 | 108 | 109 | -------------------------------------------------------------------------------- /top100papers.bib: -------------------------------------------------------------------------------- 1 | @article{hinton2015distilling, 2 | title={Distilling the knowledge in a neural network}, 3 | author={Hinton, Geoffrey and Vinyals, Oriol and Dean, Jeff}, 4 | journal={arXiv preprint arXiv:1503.02531}, 5 | year={2015} 6 | } 7 | @inproceedings{nguyen2015deep, 8 | title={Deep neural networks are easily fooled: High confidence predictions for unrecognizable images}, 9 | author={Nguyen, Anh and Yosinski, Jason and Clune, Jeff}, 10 | booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition}, 11 | pages={427--436}, 12 | year={2015} 13 | } 14 | @inproceedings{yosinski2014transferable, 15 | title={How transferable are features in deep neural networks?}, 16 | author={Yosinski, Jason and Clune, Jeff and Bengio, Yoshua and Lipson, Hod}, 17 | booktitle={Advances in neural information processing systems}, 18 | pages={3320--3328}, 19 | year={2014} 20 | } 21 | @inproceedings{sharif2014cnn, 22 | title={CNN features off-the-shelf: an astounding baseline for recognition}, 23 | author={Sharif Razavian, Ali and Azizpour, Hossein and Sullivan, Josephine and Carlsson, Stefan}, 24 | booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops}, 25 | pages={806--813}, 26 | year={2014} 27 | } 28 | @inproceedings{oquab2014learning, 29 | title={Learning and transferring mid-level image representations using convolutional neural networks}, 30 | author={Oquab, Maxime and Bottou, Leon and Laptev, Ivan and Sivic, Josef}, 31 | booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition}, 32 | pages={1717--1724}, 33 | year={2014} 34 | } 35 | @inproceedings{zeiler2014visualizing, 36 | title={Visualizing and understanding convolutional networks}, 37 | author={Zeiler, Matthew D and Fergus, Rob}, 38 | booktitle={European conference on computer vision}, 39 | pages={818--833}, 40 | year={2014}, 41 | organization={Springer} 42 | } 43 | @inproceedings{donahue2014decaf, 44 | title={DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition.}, 45 | author={Donahue, Jeff and Jia, Yangqing and Vinyals, Oriol and Hoffman, Judy and Zhang, Ning and Tzeng, Eric and Darrell, Trevor}, 46 | booktitle={Icml}, 47 | volume={32}, 48 | pages={647--655}, 49 | year={2014} 50 | } 51 | 52 | @inproceedings{srivastava2015training, 53 | title={Training very deep networks}, 54 | author={Srivastava, Rupesh K and Greff, Klaus and Schmidhuber, J{\"u}rgen}, 55 | booktitle={Advances in neural information processing systems}, 56 | pages={2377--2385}, 57 | year={2015} 58 | } 59 | 60 | @article{ioffe2015batch, 61 | title={Batch normalization: Accelerating deep network training by reducing internal covariate shift}, 62 | author={Ioffe, Sergey and Szegedy, Christian}, 63 | journal={arXiv preprint arXiv:1502.03167}, 64 | year={2015} 65 | } 66 | @inproceedings{he2015delving, 67 | title={Delving deep into rectifiers: Surpassing human-level performance on imagenet classification}, 68 | author={He, Kaiming and Zhang, Xiangyu and Ren, Shaoqing and Sun, Jian}, 69 | booktitle={Proceedings of the IEEE international conference on computer vision}, 70 | pages={1026--1034}, 71 | year={2015} 72 | } 73 | @article{srivastava2014dropout, 74 | title={Dropout: a simple way to prevent neural networks from overfitting.}, 75 | author={Srivastava, Nitish and Hinton, Geoffrey E and Krizhevsky, Alex and Sutskever, Ilya and Salakhutdinov, Ruslan}, 76 | journal={Journal of Machine Learning Research}, 77 | volume={15}, 78 | number={1}, 79 | pages={1929--1958}, 80 | year={2014} 81 | } 82 | @article{kingma2014adam, 83 | title={Adam: A method for stochastic optimization}, 84 | author={Kingma, Diederik and Ba, Jimmy}, 85 | journal={arXiv preprint arXiv:1412.6980}, 86 | year={2014} 87 | } 88 | @article{hinton2012improving, 89 | title={Improving neural networks by preventing co-adaptation of feature detectors}, 90 | author={Hinton, Geoffrey E and Srivastava, Nitish and Krizhevsky, Alex and Sutskever, Ilya and Salakhutdinov, Ruslan R}, 91 | journal={arXiv preprint arXiv:1207.0580}, 92 | year={2012} 93 | } 94 | @article{bergstra2012random, 95 | title={Random search for hyper-parameter optimization}, 96 | author={Bergstra, James and Bengio, Yoshua}, 97 | journal={Journal of Machine Learning Research}, 98 | volume={13}, 99 | number={Feb}, 100 | pages={281--305}, 101 | year={2012} 102 | } 103 | @article{oord2016pixel, 104 | title={Pixel recurrent neural networks}, 105 | author={Oord, Aaron van den and Kalchbrenner, Nal and Kavukcuoglu, Koray}, 106 | journal={arXiv preprint arXiv:1601.06759}, 107 | year={2016} 108 | } 109 | @inproceedings{salimans2016improved, 110 | title={Improved techniques for training gans}, 111 | author={Salimans, Tim and Goodfellow, Ian and Zaremba, Wojciech and Cheung, Vicki and Radford, Alec and Chen, Xi}, 112 | booktitle={Advances in Neural Information Processing Systems}, 113 | pages={2226--2234}, 114 | year={2016} 115 | } 116 | @article{radford2015unsupervised, 117 | title={Unsupervised representation learning with deep convolutional generative adversarial networks}, 118 | author={Radford, Alec and Metz, Luke and Chintala, Soumith}, 119 | journal={arXiv preprint arXiv:1511.06434}, 120 | year={2015} 121 | } 122 | @article{gregor2015draw, 123 | title={DRAW: A recurrent neural network for image generation}, 124 | author={Gregor, Karol and Danihelka, Ivo and Graves, Alex and Rezende, Danilo Jimenez and Wierstra, Daan}, 125 | journal={arXiv preprint arXiv:1502.04623}, 126 | year={2015} 127 | } 128 | @inproceedings{goodfellow2014generative, 129 | title={Generative adversarial nets}, 130 | author={Goodfellow, Ian and Pouget-Abadie, Jean and Mirza, Mehdi and Xu, Bing and Warde-Farley, David and Ozair, Sherjil and Courville, Aaron and Bengio, Yoshua}, 131 | booktitle={Advances in neural information processing systems}, 132 | pages={2672--2680}, 133 | year={2014} 134 | } 135 | @article{kingma2013auto, 136 | title={Auto-encoding variational bayes}, 137 | author={Kingma, Diederik P and Welling, Max}, 138 | journal={arXiv preprint arXiv:1312.6114}, 139 | year={2013} 140 | } 141 | @inproceedings{le2013building, 142 | title={Building high-level features using large scale unsupervised learning}, 143 | author={Le, Quoc V}, 144 | booktitle={Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on}, 145 | pages={8595--8598}, 146 | year={2013}, 147 | organization={IEEE} 148 | } 149 | @inproceedings{szegedy2016rethinking, 150 | title={Rethinking the inception architecture for computer vision}, 151 | author={Szegedy, Christian and Vanhoucke, Vincent and Ioffe, Sergey and Shlens, Jon and Wojna, Zbigniew}, 152 | booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition}, 153 | pages={2818--2826}, 154 | year={2016} 155 | } 156 | @article{szegedy2016inception, 157 | title={Inception-v4, inception-resnet and the impact of residual connections on learning}, 158 | author={Szegedy, Christian and Ioffe, Sergey and Vanhoucke, Vincent and Alemi, Alex}, 159 | journal={arXiv preprint arXiv:1602.07261}, 160 | year={2016} 161 | } 162 | @inproceedings{he2016identity, 163 | title={Identity mappings in deep residual networks}, 164 | author={He, Kaiming and Zhang, Xiangyu and Ren, Shaoqing and Sun, Jian}, 165 | booktitle={European Conference on Computer Vision}, 166 | pages={630--645}, 167 | year={2016}, 168 | organization={Springer} 169 | } 170 | @inproceedings{he2016deep, 171 | title={Deep residual learning for image recognition}, 172 | author={He, Kaiming and Zhang, Xiangyu and Ren, Shaoqing and Sun, Jian}, 173 | booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition}, 174 | pages={770--778}, 175 | year={2016} 176 | } 177 | @inproceedings{szegedy2015going, 178 | title={Going deeper with convolutions}, 179 | author={Szegedy, Christian and Liu, Wei and Jia, Yangqing and Sermanet, Pierre and Reed, Scott and Anguelov, Dragomir and Erhan, Dumitru and Vanhoucke, Vincent and Rabinovich, Andrew}, 180 | booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition}, 181 | pages={1--9}, 182 | year={2015} 183 | } 184 | @article{simonyan2014very, 185 | title={Very deep convolutional networks for large-scale image recognition}, 186 | author={Simonyan, Karen and Zisserman, Andrew}, 187 | journal={arXiv preprint arXiv:1409.1556}, 188 | year={2014} 189 | } 190 | @inproceedings{he2014spatial, 191 | title={Spatial pyramid pooling in deep convolutional networks for visual recognition}, 192 | author={He, Kaiming and Zhang, Xiangyu and Ren, Shaoqing and Sun, Jian}, 193 | booktitle={European Conference on Computer Vision}, 194 | pages={346--361}, 195 | year={2014}, 196 | organization={Springer} 197 | } 198 | @article{chatfield2014return, 199 | title={Return of the devil in the details: Delving deep into convolutional nets}, 200 | author={Chatfield, Ken and Simonyan, Karen and Vedaldi, Andrea and Zisserman, Andrew}, 201 | journal={arXiv preprint arXiv:1405.3531}, 202 | year={2014} 203 | } 204 | @article{sermanet2013overfeat, 205 | title={Overfeat: Integrated recognition, localization and detection using convolutional networks}, 206 | author={Sermanet, Pierre and Eigen, David and Zhang, Xiang and Mathieu, Micha{\"e}l and Fergus, Rob and LeCun, Yann}, 207 | journal={arXiv preprint arXiv:1312.6229}, 208 | year={2013} 209 | } 210 | @article{goodfellow2013maxout, 211 | title={Maxout Networks.}, 212 | author={Goodfellow, Ian J and Warde-Farley, David and Mirza, Mehdi and Courville, Aaron C and Bengio, Yoshua}, 213 | journal={ICML (3)}, 214 | volume={28}, 215 | pages={1319--1327}, 216 | year={2013} 217 | } 218 | @article{lin2013network, 219 | title={Network in network}, 220 | author={Lin, Min and Chen, Qiang and Yan, Shuicheng}, 221 | journal={arXiv preprint arXiv:1312.4400}, 222 | year={2013} 223 | } 224 | @inproceedings{krizhevsky2012imagenet, 225 | title={Imagenet classification with deep convolutional neural networks}, 226 | author={Krizhevsky, Alex and Sutskever, Ilya and Hinton, Geoffrey E}, 227 | booktitle={Advances in neural information processing systems}, 228 | pages={1097--1105}, 229 | year={2012} 230 | } 231 | @inproceedings{redmon2016you, 232 | title={You only look once: Unified, real-time object detection}, 233 | author={Redmon, Joseph and Divvala, Santosh and Girshick, Ross and Farhadi, Ali}, 234 | booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition}, 235 | pages={779--788}, 236 | year={2016} 237 | } 238 | @article{girshick2016region, 239 | title={Region-based convolutional networks for accurate object detection and segmentation}, 240 | author={Girshick, Ross and Donahue, Jeff and Darrell, Trevor and Malik, Jitendra}, 241 | journal={IEEE transactions on pattern analysis and machine intelligence}, 242 | volume={38}, 243 | number={1}, 244 | pages={142--158}, 245 | year={2016}, 246 | publisher={IEEE} 247 | } 248 | @inproceedings{long2015fully, 249 | title={Fully convolutional networks for semantic segmentation}, 250 | author={Long, Jonathan and Shelhamer, Evan and Darrell, Trevor}, 251 | booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition}, 252 | pages={3431--3440}, 253 | year={2015} 254 | } 255 | @inproceedings{ren2015faster, 256 | title={Faster r-cnn: Towards real-time object detection with region proposal networks}, 257 | author={Ren, Shaoqing and He, Kaiming and Girshick, Ross and Sun, Jian}, 258 | booktitle={Advances in neural information processing systems}, 259 | pages={91--99}, 260 | year={2015} 261 | } 262 | @inproceedings{girshick2015fast, 263 | title={Fast r-cnn}, 264 | author={Girshick, Ross}, 265 | booktitle={Proceedings of the IEEE International Conference on Computer Vision}, 266 | pages={1440--1448}, 267 | year={2015} 268 | } 269 | @inproceedings{girshick2014rich, 270 | title={Rich feature hierarchies for accurate object detection and semantic segmentation}, 271 | author={Girshick, Ross and Donahue, Jeff and Darrell, Trevor and Malik, Jitendra}, 272 | booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition}, 273 | pages={580--587}, 274 | year={2014} 275 | } 276 | @article{chen2014semantic, 277 | title={Semantic image segmentation with deep convolutional nets and fully connected crfs}, 278 | author={Chen, Liang-Chieh and Papandreou, George and Kokkinos, Iasonas and Murphy, Kevin and Yuille, Alan L}, 279 | journal={arXiv preprint arXiv:1412.7062}, 280 | year={2014} 281 | } 282 | @article{farabet2013learning, 283 | title={Learning hierarchical features for scene labeling}, 284 | author={Farabet, Clement and Couprie, Camille and Najman, Laurent and LeCun, Yann}, 285 | journal={IEEE transactions on pattern analysis and machine intelligence}, 286 | volume={35}, 287 | number={8}, 288 | pages={1915--1929}, 289 | year={2013}, 290 | publisher={IEEE} 291 | } 292 | @article{dong2016image, 293 | title={Image super-resolution using deep convolutional networks}, 294 | author={Dong, Chao and Loy, Chen Change and He, Kaiming and Tang, Xiaoou}, 295 | journal={IEEE transactions on pattern analysis and machine intelligence}, 296 | volume={38}, 297 | number={2}, 298 | pages={295--307}, 299 | year={2016}, 300 | publisher={IEEE} 301 | } 302 | @article{gatys2015neural, 303 | title={A neural algorithm of artistic style}, 304 | author={Gatys, Leon A and Ecker, Alexander S and Bethge, Matthias}, 305 | journal={arXiv preprint arXiv:1508.06576}, 306 | year={2015} 307 | } 308 | @inproceedings{karpathy2015deep, 309 | title={Deep visual-semantic alignments for generating image descriptions}, 310 | author={Karpathy, Andrej and Fei-Fei, Li}, 311 | booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition}, 312 | pages={3128--3137}, 313 | year={2015} 314 | } 315 | @inproceedings{xu2015show, 316 | title={Show, Attend and Tell: Neural Image Caption Generation with Visual Attention.}, 317 | author={Xu, Kelvin and Ba, Jimmy and Kiros, Ryan and Cho, Kyunghyun and Courville, Aaron C and Salakhutdinov, Ruslan and Zemel, Richard S and Bengio, Yoshua}, 318 | booktitle={ICML}, 319 | volume={14}, 320 | pages={77--81}, 321 | year={2015} 322 | } 323 | @inproceedings{vinyals2015show, 324 | title={Show and tell: A neural image caption generator}, 325 | author={Vinyals, Oriol and Toshev, Alexander and Bengio, Samy and Erhan, Dumitru}, 326 | booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition}, 327 | pages={3156--3164}, 328 | year={2015} 329 | } 330 | @inproceedings{donahue2015long, 331 | title={Long-term recurrent convolutional networks for visual recognition and description}, 332 | author={Donahue, Jeffrey and Anne Hendricks, Lisa and Guadarrama, Sergio and Rohrbach, Marcus and Venugopalan, Subhashini and Saenko, Kate and Darrell, Trevor}, 333 | booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition}, 334 | pages={2625--2634}, 335 | year={2015} 336 | } 337 | @inproceedings{antol2015vqa, 338 | title={Vqa: Visual question answering}, 339 | author={Antol, Stanislaw and Agrawal, Aishwarya and Lu, Jiasen and Mitchell, Margaret and Batra, Dhruv and Lawrence Zitnick, C and Parikh, Devi}, 340 | booktitle={Proceedings of the IEEE International Conference on Computer Vision}, 341 | pages={2425--2433}, 342 | year={2015} 343 | } 344 | @inproceedings{taigman2014deepface, 345 | title={Deepface: Closing the gap to human-level performance in face verification}, 346 | author={Taigman, Yaniv and Yang, Ming and Ranzato, Marc'Aurelio and Wolf, Lior}, 347 | booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition}, 348 | pages={1701--1708}, 349 | year={2014} 350 | } 351 | @inproceedings{karpathy2014large, 352 | title={Large-scale video classification with convolutional neural networks}, 353 | author={Karpathy, Andrej and Toderici, George and Shetty, Sanketh and Leung, Thomas and Sukthankar, Rahul and Fei-Fei, Li}, 354 | booktitle={Proceedings of the IEEE conference on Computer Vision and Pattern Recognition}, 355 | pages={1725--1732}, 356 | year={2014} 357 | } 358 | @inproceedings{toshev2014deeppose, 359 | title={Deeppose: Human pose estimation via deep neural networks}, 360 | author={Toshev, Alexander and Szegedy, Christian}, 361 | booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition}, 362 | pages={1653--1660}, 363 | year={2014} 364 | } 365 | @inproceedings{simonyan2014two, 366 | title={Two-stream convolutional networks for action recognition in videos}, 367 | author={Simonyan, Karen and Zisserman, Andrew}, 368 | booktitle={Advances in neural information processing systems}, 369 | pages={568--576}, 370 | year={2014} 371 | } 372 | @article{ji20133d, 373 | title={3D convolutional neural networks for human action recognition}, 374 | author={Ji, Shuiwang and Xu, Wei and Yang, Ming and Yu, Kai}, 375 | journal={IEEE transactions on pattern analysis and machine intelligence}, 376 | volume={35}, 377 | number={1}, 378 | pages={221--231}, 379 | year={2013}, 380 | publisher={IEEE} 381 | } 382 | @inproceedings{zheng2015conditional, 383 | title={Conditional random fields as recurrent neural networks}, 384 | author={Zheng, Shuai and Jayasumana, Sadeep and Romera-Paredes, Bernardino and Vineet, Vibhav and Su, Zhizhong and Du, Dalong and Huang, Chang and Torr, Philip HS}, 385 | booktitle={Proceedings of the IEEE International Conference on Computer Vision}, 386 | pages={1529--1537}, 387 | year={2015} 388 | } 389 | @article{weston2014memory, 390 | title={Memory networks}, 391 | author={Weston, Jason and Chopra, Sumit and Bordes, Antoine}, 392 | journal={arXiv preprint arXiv:1410.3916}, 393 | year={2014} 394 | } 395 | @article{graves2014neural, 396 | title={Neural turing machines}, 397 | author={Graves, Alex and Wayne, Greg and Danihelka, Ivo}, 398 | journal={arXiv preprint arXiv:1410.5401}, 399 | year={2014} 400 | } 401 | @article{graves2013generating, 402 | title={Generating sequences with recurrent neural networks}, 403 | author={Graves, Alex}, 404 | journal={arXiv preprint arXiv:1308.0850}, 405 | year={2013} 406 | } 407 | @article{chung2016character, 408 | title={A character-level decoder without explicit segmentation for neural machine translation}, 409 | author={Chung, Junyoung and Cho, Kyunghyun and Bengio, Yoshua}, 410 | journal={arXiv preprint arXiv:1603.06147}, 411 | year={2016} 412 | } 413 | @article{jozefowicz2016exploring, 414 | title={Exploring the limits of language modeling}, 415 | author={Jozefowicz, Rafal and Vinyals, Oriol and Schuster, Mike and Shazeer, Noam and Wu, Yonghui}, 416 | journal={arXiv preprint arXiv:1602.02410}, 417 | year={2016} 418 | } 419 | @inproceedings{hermann2015teaching, 420 | title={Teaching machines to read and comprehend}, 421 | author={Hermann, Karl Moritz and Kocisky, Tomas and Grefenstette, Edward and Espeholt, Lasse and Kay, Will and Suleyman, Mustafa and Blunsom, Phil}, 422 | booktitle={Advances in Neural Information Processing Systems}, 423 | pages={1693--1701}, 424 | year={2015} 425 | } 426 | @article{luong2015effective, 427 | title={Effective approaches to attention-based neural machine translation}, 428 | author={Luong, Minh-Thang and Pham, Hieu and Manning, Christopher D}, 429 | journal={arXiv preprint arXiv:1508.04025}, 430 | year={2015} 431 | } 432 | @article{bahdanau2014neural, 433 | title={Neural machine translation by jointly learning to align and translate}, 434 | author={Bahdanau, Dzmitry and Cho, Kyunghyun and Bengio, Yoshua}, 435 | journal={arXiv preprint arXiv:1409.0473}, 436 | year={2014} 437 | } 438 | @inproceedings{sutskever2014sequence, 439 | title={Sequence to sequence learning with neural networks}, 440 | author={Sutskever, Ilya and Vinyals, Oriol and Le, Quoc V}, 441 | booktitle={Advances in neural information processing systems}, 442 | pages={3104--3112}, 443 | year={2014} 444 | } 445 | @article{cho2014learning, 446 | title={Learning phrase representations using RNN encoder-decoder for statistical machine translation}, 447 | author={Cho, Kyunghyun and Van Merri{\"e}nboer, Bart and Gulcehre, Caglar and Bahdanau, Dzmitry and Bougares, Fethi and Schwenk, Holger and Bengio, Yoshua}, 448 | journal={arXiv preprint arXiv:1406.1078}, 449 | year={2014} 450 | } 451 | @article{kalchbrenner2014convolutional, 452 | title={A convolutional neural network for modelling sentences}, 453 | author={Kalchbrenner, Nal and Grefenstette, Edward and Blunsom, Phil}, 454 | journal={arXiv preprint arXiv:1404.2188}, 455 | year={2014} 456 | } 457 | @article{kim2014convolutional, 458 | title={Convolutional neural networks for sentence classification}, 459 | author={Kim, Yoon}, 460 | journal={arXiv preprint arXiv:1408.5882}, 461 | year={2014} 462 | } 463 | @inproceedings{pennington2014glove, 464 | title={Glove: Global Vectors for Word Representation.}, 465 | author={Pennington, Jeffrey and Socher, Richard and Manning, Christopher D}, 466 | booktitle={EMNLP}, 467 | volume={14}, 468 | pages={1532--1543}, 469 | year={2014} 470 | } 471 | @inproceedings{le2014distributed, 472 | title={Distributed Representations of Sentences and Documents.}, 473 | author={Le, Quoc V and Mikolov, Tomas}, 474 | booktitle={ICML}, 475 | volume={14}, 476 | pages={1188--1196}, 477 | year={2014} 478 | } 479 | @inproceedings{mikolov2013distributed, 480 | title={Distributed representations of words and phrases and their compositionality}, 481 | author={Mikolov, Tomas and Sutskever, Ilya and Chen, Kai and Corrado, Greg S and Dean, Jeff}, 482 | booktitle={Advances in neural information processing systems}, 483 | pages={3111--3119}, 484 | year={2013} 485 | } 486 | @article{mikolov2013efficient, 487 | title={Efficient estimation of word representations in vector space}, 488 | author={Mikolov, Tomas and Chen, Kai and Corrado, Greg and Dean, Jeffrey}, 489 | journal={arXiv preprint arXiv:1301.3781}, 490 | year={2013} 491 | } 492 | @inproceedings{socher2013recursive, 493 | title={Recursive deep models for semantic compositionality over a sentiment treebank}, 494 | author={Socher, Richard and Perelygin, Alex and Wu, Jean Y and Chuang, Jason and Manning, Christopher D and Ng, Andrew Y and Potts, Christopher and others}, 495 | booktitle={Proceedings of the conference on empirical methods in natural language processing (EMNLP)}, 496 | volume={1631}, 497 | pages={1642}, 498 | year={2013}, 499 | organization={Citeseer} 500 | } 501 | @inproceedings{bahdanau2016end, 502 | title={End-to-end attention-based large vocabulary speech recognition}, 503 | author={Bahdanau, Dzmitry and Chorowski, Jan and Serdyuk, Dmitriy and Brakel, Philemon and Bengio, Yoshua}, 504 | booktitle={Acoustics, Speech and Signal Processing (ICASSP), 2016 IEEE International Conference on}, 505 | pages={4945--4949}, 506 | year={2016}, 507 | organization={IEEE} 508 | } 509 | @article{amodei2015deep, 510 | title={Deep speech 2: End-to-end speech recognition in english and mandarin}, 511 | author={Amodei, Dario and Anubhai, Rishita and Battenberg, Eric and Case, Carl and Casper, Jared and Catanzaro, Bryan and Chen, Jingdong and Chrzanowski, Mike and Coates, Adam and Diamos, Greg and others}, 512 | journal={arXiv preprint arXiv:1512.02595}, 513 | year={2015} 514 | } 515 | @inproceedings{graves2013speech, 516 | title={Speech recognition with deep recurrent neural networks}, 517 | author={Graves, Alex and Mohamed, Abdel-rahman and Hinton, Geoffrey}, 518 | booktitle={Acoustics, speech and signal processing (icassp), 2013 ieee international conference on}, 519 | pages={6645--6649}, 520 | year={2013}, 521 | organization={IEEE} 522 | } 523 | @article{hinton2012deep, 524 | title={Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups}, 525 | author={Hinton, Geoffrey and Deng, Li and Yu, Dong and Dahl, George E and Mohamed, Abdel-rahman and Jaitly, Navdeep and Senior, Andrew and Vanhoucke, Vincent and Nguyen, Patrick and Sainath, Tara N and others}, 526 | journal={IEEE Signal Processing Magazine}, 527 | volume={29}, 528 | number={6}, 529 | pages={82--97}, 530 | year={2012}, 531 | publisher={IEEE} 532 | } 533 | @article{dahl2012context, 534 | title={Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition}, 535 | author={Dahl, George E and Yu, Dong and Deng, Li and Acero, Alex}, 536 | journal={IEEE Transactions on Audio, Speech, and Language Processing}, 537 | volume={20}, 538 | number={1}, 539 | pages={30--42}, 540 | year={2012}, 541 | publisher={IEEE} 542 | } 543 | @article{mohamed2012acoustic, 544 | title={Acoustic modeling using deep belief networks}, 545 | author={Mohamed, Abdel-rahman and Dahl, George E and Hinton, Geoffrey}, 546 | journal={IEEE Transactions on Audio, Speech, and Language Processing}, 547 | volume={20}, 548 | number={1}, 549 | pages={14--22}, 550 | year={2012}, 551 | publisher={IEEE} 552 | } 553 | @article{levine2016end, 554 | title={End-to-end training of deep visuomotor policies}, 555 | author={Levine, Sergey and Finn, Chelsea and Darrell, Trevor and Abbeel, Pieter}, 556 | journal={Journal of Machine Learning Research}, 557 | volume={17}, 558 | number={39}, 559 | pages={1--40}, 560 | year={2016} 561 | } 562 | @article{levine2016learning, 563 | title={Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection}, 564 | author={Levine, Sergey and Pastor, Peter and Krizhevsky, Alex and Quillen, Deirdre}, 565 | journal={arXiv preprint arXiv:1603.02199}, 566 | year={2016} 567 | } 568 | @inproceedings{mnih2016asynchronous, 569 | title={Asynchronous methods for deep reinforcement learning}, 570 | author={Mnih, Volodymyr and Badia, Adria Puigdomenech and Mirza, Mehdi and Graves, Alex and Lillicrap, Timothy P and Harley, Tim and Silver, David and Kavukcuoglu, Koray}, 571 | booktitle={International Conference on Machine Learning}, 572 | year={2016} 573 | } 574 | @inproceedings{van2016deep, 575 | title={Deep Reinforcement Learning with Double Q-Learning.}, 576 | author={Van Hasselt, Hado and Guez, Arthur and Silver, David}, 577 | booktitle={AAAI}, 578 | pages={2094--2100}, 579 | year={2016} 580 | } 581 | @article{silver2016mastering, 582 | title={Mastering the game of Go with deep neural networks and tree search}, 583 | author={Silver, David and Huang, Aja and Maddison, Chris J and Guez, Arthur and Sifre, Laurent and Van Den Driessche, George and Schrittwieser, Julian and Antonoglou, Ioannis and Panneershelvam, Veda and Lanctot, Marc and others}, 584 | journal={Nature}, 585 | volume={529}, 586 | number={7587}, 587 | pages={484--489}, 588 | year={2016}, 589 | publisher={Nature Publishing Group} 590 | } 591 | @article{lillicrap2015continuous, 592 | title={Continuous control with deep reinforcement learning}, 593 | author={Lillicrap, Timothy P and Hunt, Jonathan J and Pritzel, Alexander and Heess, Nicolas and Erez, Tom and Tassa, Yuval and Silver, David and Wierstra, Daan}, 594 | journal={arXiv preprint arXiv:1509.02971}, 595 | year={2015} 596 | } 597 | @article{mnih2015human, 598 | title={Human-level control through deep reinforcement learning}, 599 | author={Mnih, Volodymyr and Kavukcuoglu, Koray and Silver, David and Rusu, Andrei A and Veness, Joel and Bellemare, Marc G and Graves, Alex and Riedmiller, Martin and Fidjeland, Andreas K and Ostrovski, Georg and others}, 600 | journal={Nature}, 601 | volume={518}, 602 | number={7540}, 603 | pages={529--533}, 604 | year={2015}, 605 | publisher={Nature Research} 606 | } 607 | @article{lenz2015deep, 608 | title={Deep learning for detecting robotic grasps}, 609 | author={Lenz, Ian and Lee, Honglak and Saxena, Ashutosh}, 610 | journal={The International Journal of Robotics Research}, 611 | volume={34}, 612 | number={4-5}, 613 | pages={705--724}, 614 | year={2015}, 615 | publisher={SAGE Publications Sage UK: London, England} 616 | } 617 | @article{mnih2013playing, 618 | title={Playing atari with deep reinforcement learning}, 619 | author={Mnih, Volodymyr and Kavukcuoglu, Koray and Silver, David and Graves, Alex and Antonoglou, Ioannis and Wierstra, Daan and Riedmiller, Martin}, 620 | journal={arXiv preprint arXiv:1312.5602}, 621 | year={2013} 622 | } 623 | @article{ba2016layer, 624 | title={Layer normalization}, 625 | author={Ba, Jimmy Lei and Kiros, Jamie Ryan and Hinton, Geoffrey E}, 626 | journal={arXiv preprint arXiv:1607.06450}, 627 | year={2016} 628 | } 629 | @inproceedings{andrychowicz2016learning, 630 | title={Learning to learn by gradient descent by gradient descent}, 631 | author={Andrychowicz, Marcin and Denil, Misha and Gomez, Sergio and Hoffman, Matthew W and Pfau, David and Schaul, Tom and de Freitas, Nando}, 632 | booktitle={Advances in Neural Information Processing Systems}, 633 | pages={3981--3989}, 634 | year={2016} 635 | } 636 | @article{ganin2016domain, 637 | title={Domain-adversarial training of neural networks}, 638 | author={Ganin, Yaroslav and Ustinova, Evgeniya and Ajakan, Hana and Germain, Pascal and Larochelle, Hugo and Laviolette, Fran{\c{c}}ois and Marchand, Mario and Lempitsky, Victor}, 639 | journal={Journal of Machine Learning Research}, 640 | volume={17}, 641 | number={59}, 642 | pages={1--35}, 643 | year={2016} 644 | } 645 | @article{van2016wavenet, 646 | title={Wavenet: A generative model for raw audio}, 647 | author={van den Oord, A{\"a}ron and Dieleman, Sander and Zen, Heiga and Simonyan, Karen and Vinyals, Oriol and Graves, Alex and Kalchbrenner, Nal and Senior, Andrew and Kavukcuoglu, Koray}, 648 | journal={CoRR abs/1609.03499}, 649 | year={2016} 650 | } 651 | @inproceedings{zhang2016colorful, 652 | title={Colorful image colorization}, 653 | author={Zhang, Richard and Isola, Phillip and Efros, Alexei A}, 654 | booktitle={European Conference on Computer Vision}, 655 | pages={649--666}, 656 | year={2016}, 657 | organization={Springer} 658 | } 659 | @inproceedings{zhu2016generative, 660 | title={Generative visual manipulation on the natural image manifold}, 661 | author={Zhu, Jun-Yan and Kr{\"a}henb{\"u}hl, Philipp and Shechtman, Eli and Efros, Alexei A}, 662 | booktitle={European Conference on Computer Vision}, 663 | pages={597--613}, 664 | year={2016}, 665 | organization={Springer} 666 | } 667 | @inproceedings{ulyanov2016texture, 668 | title={Texture networks: Feed-forward synthesis of textures and stylized images}, 669 | author={Ulyanov, Dmitry and Lebedev, Vadim and Vedaldi, Andrea and Lempitsky, Victor}, 670 | booktitle={Int. Conf. on Machine Learning (ICML)}, 671 | year={2016} 672 | } 673 | @inproceedings{liu2016ssd, 674 | title={SSD: Single shot multibox detector}, 675 | author={Liu, Wei and Anguelov, Dragomir and Erhan, Dumitru and Szegedy, Christian and Reed, Scott and Fu, Cheng-Yang and Berg, Alexander C}, 676 | booktitle={European Conference on Computer Vision}, 677 | pages={21--37}, 678 | year={2016}, 679 | organization={Springer} 680 | } 681 | @article{iandola2016squeezenet, 682 | title={SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size}, 683 | author={Iandola, Forrest N and Han, Song and Moskewicz, Matthew W and Ashraf, Khalid and Dally, William J and Keutzer, Kurt}, 684 | journal={arXiv preprint arXiv:1602.07360}, 685 | year={2016} 686 | } 687 | @inproceedings{han2016eie, 688 | title={EIE: efficient inference engine on compressed deep neural network}, 689 | author={Han, Song and Liu, Xingyu and Mao, Huizi and Pu, Jing and Pedram, Ardavan and Horowitz, Mark A and Dally, William J}, 690 | booktitle={Proceedings of the 43rd International Symposium on Computer Architecture}, 691 | pages={243--254}, 692 | year={2016}, 693 | organization={IEEE Press} 694 | } 695 | @article{courbariaux2016binarized, 696 | title={Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1}, 697 | author={Courbariaux, Matthieu and Hubara, Itay and Soudry, Daniel and El-Yaniv, Ran and Bengio, Yoshua}, 698 | journal={arXiv preprint arXiv:1602.02830}, 699 | year={2016} 700 | } 701 | @article{xiong2016dynamic, 702 | title={Dynamic memory networks for visual and textual question answering}, 703 | author={Xiong, Caiming and Merity, Stephen and Socher, Richard}, 704 | journal={arXiv}, 705 | volume={1603}, 706 | year={2016} 707 | } 708 | @inproceedings{yang2016stacked, 709 | title={Stacked attention networks for image question answering}, 710 | author={Yang, Zichao and He, Xiaodong and Gao, Jianfeng and Deng, Li and Smola, Alex}, 711 | booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition}, 712 | pages={21--29}, 713 | year={2016} 714 | } 715 | @article{graves2016hybrid, 716 | title={Hybrid computing using a neural network with dynamic external memory}, 717 | author={Graves, Alex and Wayne, Greg and Reynolds, Malcolm and Harley, Tim and Danihelka, Ivo and Grabska-Barwi{\'n}ska, Agnieszka and Colmenarejo, Sergio G{\'o}mez and Grefenstette, Edward and Ramalho, Tiago and Agapiou, John and others}, 718 | journal={Nature}, 719 | volume={538}, 720 | number={7626}, 721 | pages={471--476}, 722 | year={2016}, 723 | publisher={Nature Research} 724 | } 725 | @article{wu2016google, 726 | title={Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation}, 727 | author={Wu, Yonghui and Schuster, Mike and Chen, Zhifeng and Le, Quoc V and Norouzi, Mohammad and Macherey, Wolfgang and Krikun, Maxim and Cao, Yuan and Gao, Qin and Macherey, Klaus and others}, 728 | journal={arXiv preprint arXiv:1609.08144}, 729 | year={2016} 730 | } 731 | --------------------------------------------------------------------------------