└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # Awesome Meta Learning [![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/sindresorhus/awesome) 2 | 3 | A curated list of Meta Learning papers, code, books, blogs, videos, datasets and other resources. 4 | 5 | 6 | # [Table of Contents]() 7 | 8 | * [Papers and Code](#Papers-and-Code) 9 | * [Books](#Books) 10 | * [Blogs](#Blogs) 11 | * [Lecture Videos]() 12 | * [Datasets](#Datasets) 13 | * [Researchers](#Researchers) 14 | 15 | ## [Papers and Code]() 16 | 17 | A curated set of papers along with code. 18 | 19 | ### [Zero-Shot / One-Shot / Few-Shot / Learning]() 20 | 21 | * __Siamese Neural Networks for One-shot Image Recognition__, (2015), _Gregory Koch, Richard Zemel, Ruslan Salakhutdinov_. [[pdf]](https://www.cs.cmu.edu/~rsalakhu/papers/oneshot1.pdf) [[code]](https://github.com/sudharsan13296/Hands-On-Meta-Learning-With-Python/blob/master/02.%20Face%20and%20Audio%20Recognition%20using%20Siamese%20Networks/2.4%20Face%20Recognition%20Using%20Siamese%20Network.ipynb) 22 | 23 | * __Prototypical Networks for Few-shot Learning__, (2017), _Jake Snell, Kevin Swersky, Richard S. Zemel_. [[pdf]](https://arxiv.org/pdf/1703.05175.pdf) [[code]](https://github.com/sudharsan13296/Hands-On-Meta-Learning-With-Python/blob/master/03.%20Prototypical%20Networks%20and%20its%20Variants/3.3%20Omniglot%20Character%20set%20classification%20using%20Prototypical%20Network.ipynb) 24 | 25 | * __Gaussian Prototypical Networks for Few-Shot Learning on Omniglot__ (2017), _Stanislav Fort_. [[pdf]](https://arxiv.org/pdf/1708.02735.pdf) [[code]](https://github.com/stanislavfort/gaussian-prototypical-networks) 26 | 27 | * __Matching Networks for One Shot Learning__, (2017), _Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, Daan Wierstra_. [[pdf]](https://arxiv.org/pdf/1606.04080.pdf) [[code]](https://github.com/sudharsan13296/Hands-On-Meta-Learning-With-Python/blob/master/04.%20Relation%20and%20Matching%20Networks%20Using%20Tensorflow/4.9%20Matching%20Networks%20Using%20Tensorflow.ipynb) 28 | 29 | * __Learning to Compare: Relation Network for Few-Shot Learning__, (2017), _Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip H.S. Torr, Timothy M. Hospedales_. [[pdf]](https://arxiv.org/pdf/1711.06025.pdf) [[code]](https://github.com/sudharsan13296/Hands-On-Meta-Learning-With-Python/blob/master/04.%20Relation%20and%20Matching%20Networks%20Using%20Tensorflow/4.5%20Building%20Relation%20Network%20Using%20Tensorflow.ipynb) 30 | 31 | * __One-shot Learning with Memory-Augmented Neural Networks__, (2016), _Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, Timothy Lillicrap_. [[pdf]](https://arxiv.org/pdf/1605.06065.pdf) [[code]](https://github.com/vineetjain96/one-shot-mann) 32 | 33 | * __Optimization as a Model for Few-Shot Learning__, (2016), _Sachin Ravi and Hugo Larochelle_. [[pdf]](https://openreview.net/pdf?id=rJY0-Kcll) [[code]](https://github.com/gitabcworld/FewShotLearning) 34 | 35 | * __An embarrassingly simple approach to zero-shot learning__, (2015), _B Romera-Paredes, Philip H. S. Torr_. [[pdf]](http://proceedings.mlr.press/v37/romera-paredes15.pdf) [[code]](https://github.com/bernard24/Embarrassingly-simple-ZSL) 36 | 37 | * __Low-shot Learning by Shrinking and Hallucinating Features__, (2017), _Bharath Hariharan, Ross Girshick_. [[pdf]](https://arxiv.org/pdf/1606.02819.pdf) [[code]](https://github.com/facebookresearch/low-shot-shrink-hallucinate) 38 | 39 | * __Low-shot learning with large-scale diffusion__, (2018), _Matthijs Douze, Arthur Szlam, Bharath Hariharan, Hervé Jégou_. 40 | [[pdf]](https://arxiv.org/pdf/1706.02332v2.pdf) [[code]](https://github.com/facebookresearch/low-shot-with-diffusion) 41 | 42 | * __Low-Shot Learning with Imprinted Weights__, (2018), _Hang Qi, Matthew Brown, David G. Lowe_. [[pdf]](http://openaccess.thecvf.com/content_cvpr_2018/papers/Qi_Low-Shot_Learning_With_CVPR_2018_paper.pdf) [[code]](https://github.com/YU1ut/imprinted-weights) 43 | 44 | * __Dynamic Few-Shot Visual Learning without Forgetting__, (2018), _Spyros Gidaris, Nikos Komodakis_. [[pdf]](https://arxiv.org/pdf/1804.09458.pdf) [[code]](https://github.com/gidariss/FewShotWithoutForgetting) 45 | 46 | * __Feature Generating Networks for Zero-Shot Learning__, (2017), _Yongqin Xian, Tobias Lorenz, Bernt Schiele, Zeynep Akata_. [[pdf]](https://arxiv.org/pdf/1712.00981.pdf) 47 | 48 | 49 | 50 | ## [Model Agnostic Meta Learning]() 51 | 52 | * __Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks__, (2017), _Chelsea Finn, Pieter Abbeel, Sergey Levine_. [[pdf]](https://arxiv.org/pdf/1703.03400.pdf) [[code]](https://github.com/sudharsan13296/Hands-On-Meta-Learning-With-Python/blob/master/06.%20MAML%20and%20it's%20Variants/6.5%20Building%20MAML%20From%20Scratch.ipynb) 53 | 54 | * __Adversarial Meta-Learning__, (2018), _Chengxiang Yin, Jian Tang, Zhiyuan Xu, Yanzhi Wang_. [[pdf]](https://arxiv.org/pdf/1806.03316.pdf) [[code]](https://github.com/sudharsan13296/Hands-On-Meta-Learning-With-Python/blob/master/06.%20MAML%20and%20it's%20Variants/6.7%20Building%20ADML%20From%20Scratch.ipynb) 55 | 56 | * __On First-Order Meta-Learning Algorithms__, (2018), _Alex Nichol, Joshua Achiam, John Schulman_. [[pdf]](https://arxiv.org/pdf/1803.02999.pdf) [[code]](https://github.com/sudharsan13296/Hands-On-Meta-Learning-With-Python/blob/master/07.%20Meta-SGD%20and%20Reptile%20Algorithms/7.7%20Sine%20wave%20Regression%20Using%20Reptile.ipynb) 57 | 58 | * __Meta-SGD: Learning to Learn Quickly for Few-Shot Learning__, (2017), _Zhenguo Li, Fengwei Zhou, Fei Chen, Hang Li_. [[pdf]](https://arxiv.org/pdf/1707.09835.pdf) [[code]](https://github.com/sudharsan13296/Hands-On-Meta-Learning-With-Python/blob/master/07.%20Meta-SGD%20and%20Reptile%20Algorithms/7.4%20Building%20Meta-SGD%20from%20Scratch.ipynb) 59 | 60 | * __Gradient Agreement as an Optimization Objective for Meta-Learning__, (2018), _Amir Erfan Eshratifar, David Eigen, Massoud Pedram_. [[pdf]](https://arxiv.org/pdf/1810.08178.pdf) [[code]](https://github.com/sudharsan13296/Hands-On-Meta-Learning-With-Python/blob/master/08.%20Gradient%20Agreement%20As%20An%20Optimization%20Objective/8.4%20Building%20Gradient%20Agreement%20Algorithm%20with%20MAML.ipynb) 61 | 62 | * __Gradient-Based Meta-Learning with Learned Layerwise Metric and Subspace__, (2018), _Yoonho Lee, Seungjin Choi_. [[pdf]](https://arxiv.org/pdf/1801.05558.pdf) [[code]](https://github.com/yoonholee/MT-net) 63 | 64 | * __A Simple Neural Attentive Meta-Learner__, (2018), _Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, Pieter Abbeel_. [[pdf]](https://arxiv.org/pdf/1707.03141.pdf) [[code]](https://github.com/eambutu/snail-pytorch) 65 | 66 | * __Personalizing Dialogue Agents via Meta-Learning__, (2019), _Zhaojiang Lin, Andrea Madotto, Chien-Sheng Wu, Pascale Fung_. [[pdf]](https://arxiv.org/pdf/1905.10033.pdf) [[code]](https://github.com/HLTCHKUST/PAML) 67 | 68 | * __How to train your MAML__, (2019), _Antreas Antoniou, Harrison Edwards, Amos Storkey_. [[pdf]](https://arxiv.org/pdf/1810.09502.pdf) [[code]](https://github.com/AntreasAntoniou/HowToTrainYourMAMLPytorch) 69 | 70 | * __Learning to learn by gradient descent by gradient descent__, (206), _Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W. Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, Nando de Freitas_. [[pdf]](https://arxiv.org/pdf/1606.04474.pdf) [[code]](https://github.com/deepmind/learning-to-learn) 71 | 72 | * __Unsupervised Learning via Meta-Learning__, (2019), _Kyle Hsu, Sergey Levine, Chelsea Finn_. [[pdf]](https://arxiv.org/pdf/1810.02334.pdf) [[code]](https://github.com/hsukyle/cactus-maml) 73 | 74 | 75 | * __Few-Shot Image Recognition by Predicting Parameters from Activations__, (2018), _Siyuan Qiao, Chenxi Liu, Wei Shen, Alan Yuille_. [[pdf]](https://arxiv.org/pdf/1706.03466.pdf) [[code]](https://github.com/joe-siyuan-qiao/FewShot-CVPR) 76 | 77 | 78 | * __One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning__, (2018), _Tianhe Yu, Chelsea Finn, Annie Xie, Sudeep Dasari, Pieter Abbeel, Sergey Levine_, [[pdf]](https://arxiv.org/pdf/1802.01557.pdf) [[code]](https://github.com/aravind0706/upn) 79 | 80 | * __MetaGAN: An Adversarial Approach to Few-Shot Learning__, (2018), _ZHANG, Ruixiang and Che, Tong and Ghahramani, Zoubin and Bengio, Yoshua and Song, Yangqiu_. [[pdf]](http://papers.nips.cc/paper/7504-metagan-an-adversarial-approach-to-few-shot-learning.pdf) 81 | 82 | * __Fast Parameter Adaptation for Few-shot Image Captioning and Visual Question Answering__,(2018), _Xuanyi Dong, Linchao Zhu, De Zhang, Yi Yang, Fei Wu_. [[pdf]](https://xuanyidong.com/pdf/FPAIT-MM-18.pdf) 83 | 84 | * __CAML: Fast Context Adaptation via Meta-Learning__, (2019), _Luisa M Zintgraf, Kyriacos Shiarlis, Vitaly Kurin, Katja Hofmann, Shimon Whiteson_. [[pdf]](https://arxiv.org/pdf/1810.03642.pdf) 85 | 86 | * __Meta-Learning for Low-resource Natural Language Generation in Task-oriented Dialogue Systems__, (2019), _Fei Mi, Minlie Huang, Jiyong Zhang, Boi Faltings_. [[pdf]](https://arxiv.org/pdf/1905.05644.pdf) 87 | 88 | * __MIND: Model Independent Neural Decoder__, (2019), _Yihan Jiang, Hyeji Kim, Himanshu Asnani, Sreeram Kannan_. [[pdf]](https://arxiv.org/pdf/1903.02268.pdf) 89 | 90 | * __Toward Multimodal Model-Agnostic Meta-Learning__, (2018), _Risto Vuorio, Shao-Hua Sun, Hexiang Hu, Joseph J. Lim_. [[pdf]](https://arxiv.org/pdf/1812.07172.pdf) 91 | 92 | * __Alpha MAML: Adaptive Model-Agnostic Meta-Learning__, (2019), _Harkirat Singh Behl, Atılım Güneş Baydin, Philip H. S. Torr._ [[pdf]](https://arxiv.org/pdf/1905.07435.pdf) 93 | 94 | * __Online Meta-Learning__, (2019), Chelsea Finn, _Aravind Rajeswaran, Sham Kakade, Sergey Levine_. [[pdf]](https://arxiv.org/pdf/1902.08438.pdf) 95 | 96 | 97 | 98 | ### [Meta Reinforcement Learning]() 99 | 100 | * __Generalizing Skills with Semi-Supervised Reinforcement Learning__, (2017), _Chelsea Finn, Tianhe Yu, Justin Fu, Pieter Abbeel, Sergey Levine_. [[pdf]](https://arxiv.org/pdf/1612.00429.pdf) [[code]](https://github.com/cbfinn/gps/tree/ssrl) 101 | 102 | * __Guided Meta-Policy Search__, (2019), _Russell Mendonca, Abhishek Gupta, Rosen Kralev, Pieter Abbeel, Sergey Levine, Chelsea Finn_. [[pdf]](https://arxiv.org/pdf/1904.00956.pdf) [[code]](https://github.com/RussellM2020/GMPS) 103 | 104 | * __End-to-End Robotic Reinforcement Learning without Reward Engineering__, (2019), _Avi Singh, Larry Yang, Kristian Hartikainen, Chelsea Finn, Sergey Levine_. [[pdf]](https://arxiv.org/abs/1904.07854) [[code]](https://github.com/avisingh599/reward-learning-rl) 105 | 106 | * __Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables__, (2019), _Kate Rakelly, Aurick Zhou, Deirdre Quillen, Chelsea Finn, Sergey Levine_. [[pdf]](https://arxiv.org/pdf/1903.08254) [[code]](https://github.com/katerakelly/oyster) 107 | 108 | 109 | 110 | * __Task-Agnostic Dynamics Priors for Deep Reinforcement Learning__, (2019), _Yilun Du, Karthik Narasimhan_. [[pdf]](https://arxiv.org/pdf/1905.04819.pdf) 111 | 112 | * __Meta Reinforcement Learning with Task Embedding and Shared Policy__,(2019), _Lin Lan, Zhenguo Li, Xiaohong Guan, Pinghui Wang_. [[pdf]](https://arxiv.org/pdf/1905.06527.pdf) 113 | 114 | * __NoRML: No-Reward Meta Learning__, (2019), _Yuxiang Yang, Ken Caluwaerts, Atil Iscen, Jie Tan, Chelsea Finn_. [[pdf]](https://arxiv.org/pdf/1903.01063.pdf) 115 | 116 | * __Actor-Critic Algorithms for Constrained Multi-agent Reinforcement Learning__, (2019), _Raghuram Bharadwaj Diddigi, Sai Koti Reddy Danda, Prabuchandran K. J., Shalabh Bhatnagar_. [[pdf]](https://arxiv.org/pdf/1905.02907.pdf) 117 | 118 | * __Adaptive Guidance and Integrated Navigation with Reinforcement Meta-Learning__, (2019), _Brian Gaudet, Richard Linares, Roberto Furfaro_. [[pdf]](https://arxiv.org/pdf/1904.09865.pdf) 119 | 120 | 121 | * __Watch, Try, Learn: Meta-Learning from Demonstrations and Reward__, (2019), _Allan Zhou, Eric Jang, Daniel Kappler, Alex Herzog, Mohi Khansari, Paul Wohlhart, Yunfei Bai, Mrinal Kalakrishnan, Sergey Levine, Chelsea Finn_. [[pdf]](https://arxiv.org/pdf/1906.03352.pdf) 122 | 123 | * __Options as responses: Grounding behavioural hierarchies in multi-agent RL__, (2019), _Alexander Sasha Vezhnevets, Yuhuai Wu, Remi Leblond, Joel Z. Leibo_. [[pdf]](https://arxiv.org/pdf/1906.01470.pdf) 124 | 125 | * __Learning latent state representation for speeding up exploration__, (2019), _Giulia Vezzani, Abhishek Gupta, Lorenzo Natale, Pieter Abbeel_. [[pdf]](https://arxiv.org/pdf/1905.12621.pdf) 126 | 127 | * __Beyond Exponentially Discounted Sum: Automatic Learning of Return Function__, (2019), _Yufei Wang, Qiwei Ye, Tie-Yan Liu_. [[pdf]](https://arxiv.org/pdf/1905.11591.pdf) 128 | 129 | * __Learning Efficient and Effective Exploration Policies with Counterfactual Meta Policy__, (2019), _Ruihan Yang, Qiwei Ye, Tie-Yan Liu_. [[pdf]](https://arxiv.org/pdf/1905.11583.pdf) 130 | 131 | * __Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning__, (2019), _Georgios Papoudakis, Filippos Christianos, Arrasy Rahman, Stefano V. Albrecht_. [[pdf]](https://arxiv.org/pdf/1906.04737.pdf) 132 | 133 | * __Learning to Discretize: Solving 1D Scalar Conservation Laws via Deep Reinforcement Learning__, (2019), _Yufei Wang, Ziju Shen, Zichao Long, Bin Dong_. [[pdf]](https://arxiv.org/pdf/1905.11079.pdf) 134 | 135 | 136 | ## [Books]() 137 | 138 | * __Hands-On Meta Learning with Python: Meta learning using one-shot learning, MAML, Reptile, and Meta-SGD with TensorFlow__, (2019), _Sudharsan Ravichandiran_, [[pdf]](https://www.amazon.com/Hands-Meta-Learning-Python-TensorFlow-ebook/dp/B07KJJHYKF/ref=tmm_kin_swatch_0?_encoding=UTF8&qid=&sr=) [[code]](https://github.com/sudharsan13296/Hands-On-Meta-Learning-With-Python) 139 | 140 | 141 | 142 | 143 | ## Blogs 144 | 145 | * [Berkeley Artificial Intelligence Research blog](https://bair.berkeley.edu/blog/2017/07/18/learning-to-learn/) 146 | 147 | * [Meta-Learning: Learning to Learn Fast](https://lilianweng.github.io/lil-log/2018/11/30/meta-learning.html) 148 | 149 | * [Meta-Reinforcement Learning](https://blog.floydhub.com/meta-rl/) 150 | 151 | * [How to train your MAML: A step by step approach](https://www.bayeswatch.com/2018/11/30/HTYM/) 152 | 153 | * [An Introduction to Meta-Learning](https://medium.com/walmartlabs/an-introduction-to-meta-learning-ced7072b80e7) 154 | 155 | * [From zero to research — An introduction to Meta-learning](https://medium.com/huggingface/from-zero-to-research-an-introduction-to-meta-learning-8e16e677f78a) 156 | * [What’s New in Deep Learning Research: Understanding Meta-Learning](https://towardsdatascience.com/whats-new-in-deep-learning-research-understanding-meta-learning-91fef1295660) 157 | 158 | ## Lecture Videos 159 | 160 | * [Chelsea Finn: Building Unsupervised Versatile Agents with Meta-Learning](https://www.youtube.com/watch?v=i05Fk4ebMY0) 161 | 162 | * [Sam Ritter: Meta-Learning to Make Smart Inferences from Small Data](https://www.youtube.com/watch?v=NpSpHlHpz6k) 163 | 164 | * [Model Agnostic Meta Learning by Siavash Khodadadeh](https://www.youtube.com/watch?v=wT45v8sIMDM) 165 | 166 | * [Meta Learning by Siraj Raval](https://www.youtube.com/watch?v=2z0ofe2lpz4) 167 | 168 | * [Meta Learning by Hugo Larochelle](https://www.youtube.com/watch?v=lz0ekIVfoFs) 169 | 170 | * [Meta Learning and One-Shot Learning](https://www.youtube.com/watch?v=KUWywwvQv8E) 171 | 172 | 173 | 174 | 175 | ## Datasets 176 | 177 | * [Omniglot](https://github.com/brendenlake/omniglot) 178 | * [mini-ImageNet](https://github.com/y2l/mini-imagenet-tools) 179 | * [ILSVRC](http://image-net.org/challenges/LSVRC/) 180 | 181 | 182 | ## Researchers 183 | 184 | * [Chelsea Finn](http://people.eecs.berkeley.edu/~cbfinn/) 185 | * [Pieter Abbeel](https://people.eecs.berkeley.edu/~pabbeel/) 186 | * [David Abel](https://david-abel.github.io/) 187 | * [Erin Grant](https://people.eecs.berkeley.edu/~eringrant/) 188 | * [Brenden Lake](https://cims.nyu.edu/~brenden/) 189 | * [Raia Hadsell](http://raiahadsell.com/index.html) 190 | * [Misha Denil](http://mdenil.com/) 191 | 192 | 193 | 194 | 195 | 196 | 197 | 198 | 199 | 200 | 201 | 202 | 203 | 204 | 205 | 206 | 207 | 208 | 209 | 210 | 211 | 212 | 213 | 214 | 215 | 216 | 217 | 218 | 219 | 220 | 221 | --------------------------------------------------------------------------------