└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # MARL Resources Collection 2 | This is a collection of Multi-Agent Reinforcement Learning (MARL) Resources. The purpose of this repository is to give beginners a better understanding of MARL and accelerate the learning process. Note that some of the resources are written in Chinese and only important papers that have a lot of citations were listed. 3 | 4 | I will continually update this repository and I welcome suggestions. (missing important papers, missing important resources, invalid links, etc.) This is only a first draft so far and I'll add more resources in the next few months. 5 | 6 | This repository is not for commercial purposes. 7 | 8 | My email: chenhao915@mails.ucas.ac.cn 9 | 10 | 11 | ## Overview 12 | * [Courses](https://github.com/TimeBreaker/MARL-resources-collection#courses) 13 | * [Important Conferences](https://github.com/TimeBreaker/MARL-resources-collection#important-conferences) 14 | * [Reviews](https://github.com/TimeBreaker/MARL-resources-collection#reviews) 15 | * [Books](https://github.com/TimeBreaker/MARL-resources-collection#books) 16 | * [Open Source Environments](https://github.com/TimeBreaker/MARL-resources-collection#open-source-environments) 17 | * [Research Groups](https://github.com/TimeBreaker/MARL-resources-collection#research-groups) 18 | * [Companies](https://github.com/TimeBreaker/MARL-resources-collection#companies) 19 | * [Paper List](https://github.com/TimeBreaker/MARL-resources-collection#paper-list) 20 | * [Talks](https://github.com/TimeBreaker/MARL-resources-collection#talks) 21 | * [Useful Resources](https://github.com/TimeBreaker/MARL-resources-collection#useful-links) 22 | * [TODO](https://github.com/TimeBreaker/MARL-resources-collection#todo) 23 | 24 | 25 | ## Courses 26 | * [RLChina](https://rlchina.org/) 27 | * [UCL Multi-agent AI](https://www.bilibili.com/video/BV1fz4y1S72S) 28 | * [SJTU Multi-Agent Reinforcement Learning Tutorial](http://wnzhang.net/tutorials/marl2018/index.html) 29 | * [SJTU Reinforcement Learning](https://hrl.boyuai.com/slides/) 30 | 31 | 32 | ## Important Conferences 33 | * AAMAS, AAAI, IJCAI, ICLR, ICML, NIPS 34 | * Sorted by difficulty (roughly) 35 | 36 | 37 | ## Reviews 38 | ### Recent Reviews (Since 2019) 39 | * [A Survey and Critique of Multiagent Deep Reinforcement Learning](https://arxiv.org/pdf/1810.05587v3) 40 | * [An Overview of Multi-Agent Reinforcement Learning from Game Theoretical Perspective](https://arxiv.org/abs/2011.00583v2) 41 | * [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://arxiv.org/abs/1911.10635v1) 42 | * [A Review of Cooperative Multi-Agent Deep Reinforcement Learning](https://arxiv.org/abs/1908.03963) 43 | * [Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning](https://arxiv.org/abs/1906.04737) 44 | * [A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity](https://arxiv.org/abs/1707.09183v1) 45 | * [Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications](https://arxiv.org/pdf/1812.11794.pdf) 46 | * [A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems](https://www.researchgate.net/publication/330752409_A_Survey_on_Transfer_Learning_for_Multiagent_Reinforcement_Learning_Systems) 47 | 48 | ### Other Reviews (Before 2019) 49 | * [If multi-agent learning is the answer, what is the question?](https://ai.stanford.edu/people/shoham/www%20papers/LearningInMAS.pdf) 50 | * [Multiagent learning is not the answer. It is the question](https://core.ac.uk/download/pdf/82595758.pdf) 51 | * [Is multiagent deep reinforcement learning the answer or the question? A brief survey](https://arxiv.org/abs/1810.05587v1) Note that [A Survey and Critique of Multiagent Deep Reinforcement Learning](https://arxiv.org/pdf/1810.05587v3) is an updated version of this paper with the same authors. 52 | * [Evolutionary Dynamics of Multi-Agent Learning: A Survey](https://www.researchgate.net/publication/280919379_Evolutionary_Dynamics_of_Multi-Agent_Learning_A_Survey) 53 | * (Worth reading although they're not recent reviews.) 54 | 55 | 56 | ## Books 57 | * [Multiagent systems: Algorithmic, game-theoretic, and logical foundations](http://www.masfoundations.org/download.html) 58 | * [Multi‐Agent Machine Learning A Reinforcement Approach](https://www.engineerrefe.com/multi-agent-machine-learning/) 59 | 60 | 61 | ## Open Source Environments 62 | * StarCraft Micromanagement Environment 63 | * [pymarl](https://github.com/oxwhirl/pymarl) is the original environment mentioned in the paper [The StarCraft Multi-Agent Challenge](https://arxiv.org/abs/1902.04043). Note that pymarl is based on [SMAC](https://github.com/oxwhirl/smac). 64 | * [MARL-Algorithms](https://github.com/starry-sky6688/MARL-Algorithms) is a simplified implementation of [pymarl](https://github.com/oxwhirl/pymarl) 65 | * [EPyMARL](https://github.com/uoe-agents/epymarl) is a extended python MARL framework with more environments (Level Based Foraging, Multi-Robot Warehouse, Multi-Agent Particle Environment) and more algorithms. [Paper](https://link.zhihu.com/?target=https%3A//arxiv.org/abs/2006.07869) 66 | * [pymarl2](https://github.com/hijkzzz/pymarl2) added code-level tricks to the original pymarl. [Paper](https://arxiv.org/abs/2102.03479) 67 | * [Multi-Agent Particle Environment](https://github.com/openai/multiagent-particle-envs) [PyTorch Implementation](https://github.com/shariqiqbal2810/maddpg-pytorch) 68 | * [Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents](https://github.com/openai/neural-mmo) 69 | * [OpenSpiel: A Framework for Reinforcement Learning in Games](https://github.com/deepmind/open_spiel) 70 | * [Hanabi-learning-environment](https://github.com/deepmind/hanabi-learning-environment) 71 | * [RoboCup 2D Half Field Offense](https://github.com/LARG/HFO) 72 | * [Pommerman](https://www.pommerman.com/) 73 | * [Multi-agent-emergence-environments](https://github.com/openai/multi-agent-emergence-environments) 74 | * [Google Research Football](https://github.com/google-research/football) 75 | * [MAgent](https://github.com/PettingZoo-Team/MAgent) Note that [the original project](https://github.com/geek-ai/MAgent) is no longer maintained. 76 | * [DI-engine](https://github.com/opendilab/DI-engine) 77 | * [MARLlib](https://github.com/Replicable-MARL/MARLlib) is a MARL Extension for RLlib 78 | * [Multiagent Mujoco](https://github.com/schroederdewitt/multiagent_mujoco) 79 | * [PettingZoo](https://github.com/Farama-Foundation/PettingZoo) [website](https://www.pettingzoo.ml/) 80 | * [Safe Policy Optimization (SafePO)](https://github.com/PKU-MARL/Safe-Policy-Optimization) 81 | * (I personally recommend the first two environments for beginners, especially EPyMARL.) 82 | 83 | 84 | ## Research Groups 85 | Organization|Reaearcher|Lab homepage (if any) 86 | --|:--:|--: 87 | Oxford|[Shimon Whiteson](https://www.cs.ox.ac.uk/people/shimon.whiteson/), [Jakob N. Foerster](https://www.jakobfoerster.com/)|[link](http://whirl.cs.ox.ac.uk/ ) 88 | University College London (UCL)|[Jun Wang](http://www0.cs.ucl.ac.uk/staff/Jun.Wang/)| 89 | Tsinghua University (THU)|[Chongjie Zhang](http://people.iiis.tsinghua.edu.cn/~zhang/)|[link](http://group.iiis.tsinghua.edu.cn/~milab/index.html) 90 | Tsinghua University (THU)|[Yi Wu](http://jxwuyi.weebly.com/)| 91 | Peking University (PKU)|[Zongqing Lu](https://z0ngqing.github.io/)| 92 | HUAWEI|[Hangyu Mao](https://maohangyu.github.io/)| 93 | Nanjing University (NJU)|[Yang Yu](http://www.lamda.nju.edu.cn/yuy/)| 94 | Facebook|[Yuandong Tian](http://yuandong-tian.com/)| 95 | Tianjin University (TJU)|[Jianye Hao](http://faculty.tju.edu.cn/156102/zh_CN/index/24194/list/index.htm)|[link](http://www.icdai.org/) 96 | University of Illinois at Urbana-Champaign (UIUC)|[Kaiqing Zhang](https://kzhang66.github.io/index.html)| 97 | Peking University (PKU)|[Yaodong Yang](https://www.yangyaodong.com)|[Link](https://github.com/PKU-MARL) 98 | Nanyang Technological University (NTU)|[Bo An](https://personal.ntu.edu.sg/boan/index.html)| 99 | Shanghai Jiao Tong University (SJTU)|[Weinan Zhang](http://wnzhang.net/)|[link](http://apex.sjtu.edu.cn/) 100 | University of Chinese Academy of Sciences (UCAS)|[Haifeng Zhang](https://pkuzhf.github.io/)|[link](http://marl.ia.ac.cn/index.html) 101 | University of Edinburgh|[Stefano V. Albrecht](https://www.turing.ac.uk/people/researchers/stefano-albrecht)|[link](https://agents.inf.ed.ac.uk/) [GitHub](https://github.com/uoe-agents) 102 | University College London (UCL)|UCL Deciding, Acting, and Reasoning with Knowledge (DARK) Lab |[Link](https://dark.cs.ucl.ac.uk/) 103 | University of Maryland|[Furong Huang](http://furong-huang.com/)|[Link](http://furong-huang.com/) 104 | 105 | 106 | ## Companies 107 | * [DeepMind](https://deepmind.com/) 108 | * [OpenAI](https://openai.com/) 109 | * [Facebook](https://ai.facebook.com/) 110 | * [Tencent](https://ai.tencent.com/ailab/zh/index) 111 | * [NetEase](https://fuxi.163.com/#/home) 112 | * [Huawei](https://www.noahlab.com.hk/#/home) 113 | * [Parametrix.ai](https://chaocanshu.cn/) 114 | * [Inspir.ai](http://www.inspirai.com/) 115 | 116 | 117 | ## Paper Lists 118 | * https://github.com/TimeBreaker/Multi-Agent-Reinforcement-Learning-papers 119 | * https://github.com/TimeBreaker/MARL-papers-with-code 120 | * https://github.com/LantaoYu/MARL-Papers 121 | 122 | 123 | ## Talks 124 | ### In English 125 | * https://www.youtube.com/watch?v=W_9kcQmaWjo 126 | * https://www.youtube.com/watch?v=TMTT2z8lifA 127 | * https://www.youtube.com/watch?v=Yd6HNZnqjis 128 | * https://www.youtube.com/watch?v=ufFue5_gR4c 129 | 130 | ### In Chinese 131 | * https://www.techbeat.net/talk-info?id=501 132 | * https://www.bilibili.com/video/av457780236/ 133 | * https://space.bilibili.com/551888585/channel/detail?cid=167587 134 | * https://www.bilibili.com/video/BV1ig4y1v7xU 135 | * https://www.bilibili.com/video/BV18z411q7Kc 136 | * https://www.bilibili.com/video/BV1k5411V7ue 137 | 138 | 139 | ## Useful Resources 140 | ### In English 141 | * https://dblp.uni-trier.de/ 142 | * https://paperswithcode.com/ 143 | * https://www.connectedpapers.com 144 | * https://deeplearn.org 145 | * https://spinningup.openai.com/ 146 | * https://github.com/openai/spinningup 147 | * https://github.com/Jinjiarui/hrl-papers 148 | 149 | ### In Chinese 150 | * http://www.neurondance.com/ 151 | * https://www.zhihu.com/question/376068768 152 | * https://www.zhihu.com/question/323584412 153 | * https://zhuanlan.zhihu.com/p/372558232 154 | * https://space.bilibili.com/4801051?spm_id_from=333.788.b_765f7570696e666f.2 155 | * https://www.zhihu.com/people/tian-yuan-dong 156 | * https://www.zhihu.com/people/eyounx 157 | * https://www.zhihu.com/people/wan-shang-zhu-ce-de 158 | * Wechat public account: AIORHHC; RLCN 159 | * https://www.bilibili.com/video/av925922430/ 160 | * https://www.bilibili.com/video/av626777400/ 161 | * https://github.com/NeuronDance/DeepRL 162 | 163 | 164 | ## TODO 165 | * The Research Groups part needs to be completed 166 | * The Companies part needs to be completed 167 | * The Useful Resources part needs to be perfected 168 | 169 | 170 | ## Citation 171 | 172 | If you find this repository useful, please cite our repo: 173 | ``` 174 | @misc{chen2021collection, 175 | author={Chen, Hao}, 176 | title={A Collection of Multi-Agent Reinforcement Learning Resources}, 177 | year={2021} 178 | publisher = {GitHub}, 179 | journal = {GitHub Repository}, 180 | howpublished = {\url{https://github.com/TimeBreaker/MARL-resources-collection}} 181 | } 182 | ``` 183 | --------------------------------------------------------------------------------