├── LICENSE └── README.md /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 Guillaume Jaume 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Deep Learning on Graphs: a roadmap 2 | 3 | This roadmap explores the latest advances made in the field of deep learning on graphs. After listing the main papers that set the foundations of DL on graphs and Graph Neural Networks, we dig in various sub-topics. Sub-topics include graph VAE, generative model of graphs, theoretical studies of the expressiveness power of GNNs, edge-informative graphs etc.. 4 | 5 | I would continue adding papers to this roadmap. Feel free to suggest new papers that are missing in this list. 6 | 7 | --------------------------------------- 8 | 9 | # 1. Impactful Graph Neural Networks (chronological order): 10 | 11 | **[1]** M. Defferrard, X. Bresson, and P. Vandergheynst, **Convolutional Neural Networks on Graphs with Fast 12 | Localized Spectral Filtering,** NeurIPS, 2016 [[pdf]](https://arxiv.org/pdf/1606.09375.pdf) [[code TensorFlow]](https://github.com/mdeff/cnn_graph) 13 | 14 | **[2]** N. Kipf and M. Welling, **Semi supervised classification with graph convolutional networks,** 2017, ICLR [[pdf]](https://arxiv.org/pdf/1609.02907.pdf)[[code TensorFlow]](https://github.com/tkipf/gcn) 15 | 16 | **[3]** A. Santoro, D. Raposo, D. G. T. Barrett, M. Malinowski, R. Pascanu, P. Battaglia, and T. Lillicrap, 17 | **A simple neural network module for relational reasoning,** NeurIPS, 2017 [[pdf]](https://arxiv.org/pdf/1706.01427.pdf) [[code PyTorch]](https://github.com/kimhc6028/relational-networks) [[code TensorFlow]](https://github.com/gitlimlab/Relation-Network-Tensorflow) 18 | 19 | **[4]** J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, **Neural Message Passing for 20 | Quantum Chemistry,** ICML, 2017 [[pdf]](https://arxiv.org/pdf/1704.01212.pdf) 21 | 22 | **[5]** P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, **Graph Attention Networks,** 23 | ICLR, 2018 [[pdf]](https://arxiv.org/pdf/1710.10903.pdf) [[code TensorFlow]](https://github.com/PetarV-/GAT). 24 | 25 | # 2. Literature Reviews 26 | 27 | **[1]** P. W. Battaglia, J. B. Hamrick, V. Bapst, A. Sanchez-Gonzalez, V. Zambaldi, M. Malinowski, A. Tacchetti, D. Raposo, A. Santoro, R. Faulkner, C. Gulcehre, F. Song, A. Ballard, J. Gilmer, G. Dahl, A. Vaswani, K. Allen, C. Nash, V. Langston, C. Dyer, N. Heess, D. Wierstra, P. Kohli, M. Botvinick, O. Vinyals, Y. Li, and R. Pascanu, **Relational 28 | inductive biases, deep learning, and graph networks** [[pdf]](https://arxiv.org/pdf/1806.01261.pdf) [[code]](https://github.com/deepmind/graph_nets) 29 | 30 | **[2]** J. Zhou, G. Cui, Z. Zhang, C. Yang, Z. Liu, and M. Sun, **Graph Neural Networks: A Review of Methods and Applications** [[pdf]](https://arxiv.org/pdf/1812.08434.pdf) 31 | 32 | **[3]** Z. Zhang, P. Cui, and W. Zhu, **Deep Learning on Graphs: A Survey** [[pdf]](https://arxiv.org/pdf/1812.04202v1.pdf) 33 | 34 | **[4]** W. Zonghan, P. Shirui, C. Fengwen, L. Guodong, Z. Chengqi, Y. Philip, **A Comprehensive Survey on Graph Neural Networks** [[pdf]](https://arxiv.org/pdf/1901.00596.pdf) 35 | 36 | # 3. Let's dig by topic: 37 | 38 | ## 3.1 GNN for Edge-Informative graphs: 39 | 40 | **[1]** M. Simonovsky and N. Komodakis. **Dynamic edge-conditioned filters in convolutional neu- 41 | ral networks on graphs**, 2017, CVPR.[[pdf]](https://arxiv.org/pdf/1704.02901.pdf) 42 | 43 | **[2]** M. Schlichtkrull, T. N. Kipf, P. Bloem, R. van den Berg, I. Titov, M. 44 | Welling. **Modeling Relational Data with Graph Convolutional Networks**, 2018, In 45 | Extended Semantic Web Conference. [[pdf]](https://arxiv.org/pdf/1703.06103.pdf) 46 | 47 | **[3]** G. Jaume, A. Nguyen, M. Rodriguez, J-P. Thiran, M. Gabrani, **edGNN: A simple and powerful GNN for directed labeled graphs**, 2019, ICLR workshop on graphs and manifolds. [[pdf]](https://arxiv.org/pdf/1904.08745.pdf) [[code]](https://github.com/guillaumejaume/edGNN) 48 | 49 | ## 3.2 Unsupervised Graph Neural Networks 50 | 51 | **[1]** Petar Veličković, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, R Devon Hjelm, **Deep Graph Infomax** ICLR 2019 [[pdf]](https://arxiv.org/pdf/1809.10341.pdf) [[code PyTorch]](https://github.com/PetarV-/DGI). 52 | 53 | **[2]** Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay S. Pande, Jure Leskovec, **Pre-training Graph Neural Networks** submitted to NeuIPS 2019 [[pdf]](https://arxiv.org/pdf/1905.12265.pdf). 54 | 55 | **[3]** Graph auto-encoder and generative graph modeling. See Section 3.6 & 3.7 56 | 57 | ## 3.3 Characterization of Graph Neural Networks 58 | 59 | **[1]** K. Xu, W. Hu, J. Leskovec, S. Jegelka, **How Powerful are Graph Neural Networks ?,** 60 | ICLR, 2019 [[pdf]](https://arxiv.org/pdf/1810.00826.pdf) [[code PyTorch]](https://github.com/weihua916/powerful-gnns) 61 | 62 | **[2]** C. Morris, M. Ritzert, M. Fey , W. L. Hamilton, J. Lenssen, G. Rattan, M. Grohe, **Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks,** 63 | AAAI, 2018 [[pdf]](https://arxiv.org/pdf/1810.02244.pdf) [[code PyTorch]](https://github.com/chrsmrrs/k-gnn) 64 | 65 | **[3]** F. Wu, T. Zhang, A. Holanda de Souza Jr., C. Fifty, T. Yu, K. Q. Weinberger, **Simplifying Graph Convolutional Networks,** 66 | ICML, 2019 [[pdf]](https://arxiv.org/pdf/1902.07153.pdf) 67 | 68 | **[4]** H. NT, T. Maehara, **Revisiting Graph Neural Networks: 69 | All We Have is Low-Pass Filters,** 70 | submitted to NeurIPS, 2019 [[pdf]](https://arxiv.org/pdf/1905.09550.pdf) 71 | 72 | **[5]** N. Dehmamy,A-L Barabási, R. Yu, **Understanding the Representation Power of GraphNeural Networks in Learning Graph Topology,** NeurIPS, 2019 [[pdf]](https://arxiv.org/pdf/1907.05008.pdf) 73 | 74 | ## 3.4 Pooling on graphs: 75 | 76 | **[1]** M. Zhang, Z. Cui, M. Neumann, Y. Chen, **An End-to-End Deep Learning Architecture for Graph Classification**, AAAI, 2018 [[pdf]](https://www.cse.wustl.edu/~muhan/papers/AAAI_2018_DGCNN.pdf) 77 | 78 | **[2]** R. Ying, J. You, C. Morris, X. Ren, W. L. Hamilton, and J. Leskovec, **Hierarchical Graph Representation Learning with Differentiable Pooling,** 2018, NeurIPS. [[pdf]](https://arxiv.org/pdf/1806.08804.pdf) [[code]](https://github.com/RexYing/diffpool) 79 | 80 | **[3]** Anonymous Review, **GRAPH U-NET,** to be presented at ICLR 2019 [[pdf]](https://openreview.net/pdf?id=HJePRoAct7) 81 | 82 | **[4]** J. Lee, I. Lee, J. Kang, **Self-Attention Graph Pooling,** ICML, 2019 [[pdf]](https://arxiv.org/pdf/1904.08082.pdf) 83 | 84 | ## 3.5 Relational Reinforcement Learning: 85 | 86 | **[1]** V. Zambaldi, D. Raposo, A. Santoro, V. Bapst, Y. Li, I. Babuschkin, K. Tuyls, D. Reichert, T. Lillicrap, 87 | E. Lockhart, M. Shanahan, V. Langston, R. Pascanu, M. Botvinick, O. Vinyals, and P. Battaglia, 88 | **Relational Deep Reinforcement Learning,** 2018 [[pdf]](https://arxiv.org/pdf/1806.01830.pdf) 89 | 90 | ## 3.6 Generative models of graphs: 91 | 92 | **[1]** J. You, R. Ying, X. Ren, W. L. Hamilton, J. Leskovec, **GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models**, ICML, 2018 [[pdf]](https://arxiv.org/pdf/1802.08773.pdf) 93 | 94 | **[2]** Y. Li, O. Vinyals, C. Dyer, R. Pascanu, P. Battaglia, **Learning Deep Generative Models of Graphs**, 2018 [[pdf]](https://arxiv.org/pdf/1803.03324.pdf) 95 | 96 | **[3]** J. You, B. Liu, R. Ying, V. Pande, and J. Leskovec, **Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation**, NeurIPS, 2018 [[pdf]](https://arxiv.org/pdf/1806.02473.pdf) 97 | 98 | **[4]** M. Simonovsky, N. Komodakis **GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoders**, 2018. [[pdf]](https://arxiv.org/pdf/1802.03480.pdf) 99 | 100 | **[5]** W. Jin, R. Barzilay, T. Jaakkola, **Junction Tree Variational Autoencoder for Molecular Graph Generation**, ICML, 2018. [[pdf]](https://arxiv.org/pdf/1802.04364.pdf) [[code PyTorch]](https://github.com/wengong-jin/icml18-jtnn) 101 | 102 | **[6]** N. De Cao and T. Kipf, **MolGAN: An implicit generative model for small molecular graphs,** 2018 [[pdf]](https://arxiv.org/pdf/1805.11973.pdf) 103 | 104 | **[7]** Q. Liu, M. Allamanis, M. Brockschmidt, A. L. Gaunt **Constrained Graph Variational Autoencoders for Molecule Design**, NeuIPS, 2018. [[pdf]](https://arxiv.org/pdf/1802.03480.pdf) 105 | 106 | **[8]** A. Grover, A. Zweig, S. Ermon, **Graphite: Iterative Generative Modeling of Graphs**, ICML, 2019. [[pdf]](https://arxiv.org/pdf/1803.10459.pdf) [[code TensorFlow]](https://github.com/ermongroup/graphite) 107 | 108 | **[9]** C. Yang, P. Zhuang, W. Shi, A. Luu, P. Li, **Conditional Structure Generation through Graph Variational Generative Adversarial Nets**, NeurIPS, 2019. [[pdf]](http://jiyang3.web.engr.illinois.edu/files/condgen.pdf) [[code PyTorch]](https://github.com/KelestZ/CondGen) 109 | 110 | **[10]** R. Liao, Y. Li, Y. Song, S. Wang, C. Nash, W. L. Hamilton, D. Duvenaud, R. Urtasun, R. S. Zemel, **Efficient Graph Generation with Graph Recurrent Attention Networks**, NeurIPS, 2019. [[pdf]](https://arxiv.org/pdf/1910.00760.pdf) [[code PyTorch]](https://github.com/lrjconan/GRAN) 111 | 112 | ## 3.7 Graph Autoencoders 113 | 114 | **[1]** T. N. Kipf and M. Welling., **Variational graph auto-encoders,** 2016, Bayesian DL, NeurIPS [[pdf]](https://arxiv.org/pdf/1611.07308.pdf) 115 | 116 | **[2]** T. N. Kipf, E. Fetaya, K. Wang, M. Welling, R. Zemel, **Neural Relational Inference for Interacting Systems,** ICML, 2018 [[pdf]](https://arxiv.org/pdf/1611.07308.pdf) 117 | 118 | ## 3.8 Scene Understanding (e.g., Scene Graph Generation) 119 | 120 | **[1]** D. Xu, Y. Zhu, C. B. Choy, and L. Fei-Fei. **Scene graph generation by iterative message passing**, 121 | CVPR,2017. [[pdf]](https://arxiv.org/pdf/1701.02426.pdf) 122 | 123 | **[2]** J. Yang, J. Lu, S. Lee, D. Batra, and D. Parikh. **Graph R-CNN for Scene Graph Generation**, 124 | ECCV, 2018. [[pdf]](https://arxiv.org/pdf/1808.00191.pdf) 125 | 126 | **[3]** G. Jaume, B. Bozorgtabar, H. Ekenel, J-P. Thiran, M. Gabrani, **Image-Level Attentional Context Modeling using Nest Graph Neural Networks**, NeuIPS workshop on Relational Representation Learning [[pdf]](https://arxiv.org/abs/1811.03830) 127 | 128 | ## 3.9 Knowledge Graphs 129 | 130 | **[1]** M. Nickel, K. Murphy, V. Tresp, E. Gabrilovich, **A Review of Relational Machine Learning for Knowledge Graphs** 2015 [[pdf]](https://arxiv.org/pdf/1503.00759.pdf) 131 | 132 | ## 3.10 Neural Architecture Search with GNNs 133 | 134 | **[1]** C. Zhang, M. Ren, R. Urtasun, **Graph Hyper Networks for Neural Architecture Search**, ICLR 2019 [[pdf]](https://openreview.net/pdf?id=rkgW0oA9FX) 135 | 136 | ## 3.xx More to come 137 | 138 | # 4. Libraries 139 | 140 | - **GraphNets** by DeepMind (written in TensorFlow/Sonnet) [[code]](https://github.com/deepmind/graph_nets) 141 | - **Deep Graph Library (DGL)** (written in PyTorch/MXNet) [[code]](https://github.com/dmlc/dgl) 142 | - **pytorch-geometric** (written in PyTorch) [[code]](https://github.com/rusty1s/pytorch_geometric). Provides implementation for: 143 | * GCN 144 | * GAT 145 | * Graph U-Net 146 | * Deep Graph InfoMax 147 | * GIN 148 | - **Graph Embedding Methods (GEM)** (written in Keras) [[code]](https://github.com/palash1992/GEM) 149 | * Node2vec 150 | * Laplacian Eigenmaps 151 | - **Spektral** (written in Keras) [[code]](https://github.com/danielegrattarola/spektral/) 152 | - **Chainer Chemistry** (written in Chainer) [[code]](https://github.com/pfnet-research/chainer-chemistry) 153 | - **PyTorch Big-Graph** (written in PyTorch) [[code]](https://github.com/facebookresearch/PyTorch-BigGraph?fbclid=IwAR0QbWESbygic26TW1b3pxHf0Jr2XBvQTI3kVYCIs7iRP-sE1DIubpAFwTo) 154 | 155 | # 5. Presentation/Slides 156 | 157 | * Weekly presentation on **Graph Neural Networks at MILA** [[presentation]](https://github.com/shagunsodhani/Graph-Reading-Group) 158 | * Presentation on **Relational Learning** by Petar Veličković [[presentation]](https://www.cl.cam.ac.uk/~pv273/slides/MILA-RN.pdf) 159 | * Presentation on **Adversarial learning meets graphs** by Petar Veličković [[presentation]](https://www.cl.cam.ac.uk/~pv273/slides/MILA-AdvGraph.pdf) 160 | * Presentation on **How Powerful are Graph Neural Networks?** by Jure 161 | Leskovec [[presentation]](http://i.stanford.edu/~jure/pub/talks2/graphsage_gin-ita-feb19.pdf?fbclid=IwAR1rM7xiVmHpk6PJ1dMntYPz1odn1TQiOGYOZKpaLuuBjVb34LxwupioABw) 162 | - Graph SAGE 163 | - Hierarchical Graph Pooling 164 | - Graph Isomorphism Networks 165 | 166 | 167 | # 6. Videos 168 | 169 | - **Geometric Deep Learning** by Siraj Raval [[video]](https://www.youtube.com/watch?v=D3fnGG7cdjY) 170 | - **Graph neural networks: Variations and applications** by Alexander Gaunt [[video]](https://www.youtube.com/watch?v=cWIeTMklzNg) 171 | - **Large-scale Graph Representation Learning** by Jure Leskovec [[video]](https://www.youtube.com/watch?v=oQL4E1gK3VU) 172 | 173 | # 7. Workshops 174 | 175 | * **Geometry and Learning from Data in 3D and Beyond** at IPAM 2019 [[link]](http://www.ipam.ucla.edu/programs/long-programs/geometry-and-learning-from-data-in-3d-and-beyond/) 176 | - [Geometry and Learning from Data Tutorials](https://www.ipam.ucla.edu/programs/workshops/geometry-and-learning-from-data-tutorials/) 177 | - [Workshop I: Geometric Processing](https://www.ipam.ucla.edu/programs/workshops/workshop-i-geometric-processing/) 178 | - [Workshop II: Shape Analysis](https://www.ipam.ucla.edu/programs/workshops/workshop-ii-shape-analysis/) 179 | - [Workshop III: Geometry of Big Data](https://www.ipam.ucla.edu/programs/workshops/workshop-iii-geometry-of-big-data/) 180 | - [Workshop IV: Deep Geometric Learning of Big Data and Applications](https://www.ipam.ucla.edu/programs/workshops/workshop-iv-deep-geometric-learning-of-big-data-and-applications/) 181 | 182 | * **Geometry meets deep learning** at ICCV 2019 [[link]](http://geometricdeeplearning.com/) 183 | 184 | * **Learning and Reasoning with Graph-Structured Data** at ICML 2019 [[link]](https://graphreason.github.io/) 185 | 186 | * **Representation Learning on Graphs and Manifolds** at ICLR 2019 [[link]](https://rlgm.github.io/) 187 | 188 | * **Relational Representation Learning** at NeurIPS 2018 [[link]](https://r2learning.github.io/) 189 | 190 | * **Representation Learning on Networks** by Jure Leskovec [[link]](http://snap.stanford.edu/proj/embeddings-www/) 191 | - Network/Relational Representation Learning 192 | - Node embeddings 193 | - Graph Neural Networks 194 | - Applications in recommender systems and computational biology 195 | 196 | * **Deep Learning for Network Biology** by Jure Leskovec [[link]](http://snap.stanford.edu/deepnetbio-ismb/) 197 | - Network propagation and node embeddings 198 | - Graph autoencoders and deep representation learning 199 | - Heterogeneous networks 200 | - Tensorflow examples 201 | --------------------------------------------------------------------------------