├── README.md └── assets └── README └── image-20231101100623183.png /README.md: -------------------------------------------------------------------------------- 1 | # Awesome Attention-based GNNs 2 | 3 | - A collection of resources on attention-based graph neural networks([Github](https://github.com/sunxiaobei/awesome-attention-based-gnns)，[Paper](https://doi.org/10.1007/s10462-023-10577-2)). 4 | 5 | - Welcome to submit a pull request to add more awesome papers. 6 | 7 | ```markdown 8 | - [x] [journal] [model] paper_title [[paper]](link) [[code]](link) 9 | ``` 10 | 11 | ## Table of Contents 12 | - [Surveys](#Surveys) 13 | - [GRANs](#GRANs) : (Graph Recurrent Attention Networks) 14 | - [GATs](#GATs) : (Graph Attention Networks：Intra-layer， Inter-layer) 15 | - [Graph Transformers](#GraphTransformers) : (Graph Transformers) 16 | 17 | ![image-20231101100623183](assets/README/image-20231101100623183.png) 18 | 19 | ## Survey 20 | 21 | ### GNN Survey 22 | - [x] [Artif Intell Rev2023] [survey] Attention-based Graph Neural Networks: A Survey [[paper]](https://doi.org/10.1007/s10462-023-10577-2) 23 | - [x] [TKDD2019] [survey] Attention Models in Graphs: A Survey [[paper]](https://doi.org/10.1145/3363574) 24 | - [x] [arXiv2022] [survey] Transformer for Graphs: An Overview from Architecture Perspective [[paper]]() 25 | - [x] [TIST2022] [] Graph neural networks:Taxonomy, advances, and trends [[paper]]() 26 | 27 | - [x] [2020] [survey] A Comprehensive Survey on Graph Neural Networks [[paper]]() 28 | - [x] [2021] [survey] Graph Deep Learning: State of the Art and Challenges [[paper]]() 29 | 30 | ### Application 31 | - [x] [2022] [survey] Graph Neural Networks for Natural Language Processing: A Survey [[paper]]() 32 | - [x] [2022] [] Graph neural networks for recommender system [[paper]]() 33 | - [x] [2021] [survey] Graph Neural Network for Traffic Forecasting: A Survey [[paper]]() 34 | - [x] [2022] [survey] Graph Neural Networks in IoT: A Survey [[paper]]() 35 | - [x] [2023] [] Multimodal learning with graphs [[paper]]() 36 | 37 | ### Attention Survey 38 | - [x] [TKDE2022] [survey] A General Survey on Attention Mechanisms in Deep Learning [[paper]]() 39 | - [x] [2021] [survey] Neural Attention Models in Deep Learning: Survey and Taxonomy [[paper]]() 40 | - [x] [2021] [survey] An Attentive Survey of Attention Models [[paper]]() 41 | - [x] [2021] [survey] Attention in Natural Language Processing [[paper]]() 42 | - [x] [2018] [survey] An Introductory Survey on Attention Mechanisms in NLP Problems [[paper]]() 43 | - [x] [2022] [survey] Visual Attention Methods in Deep Learning: An In-Depth Survey [[paper]]() 44 | - [x] [2022] [survey] Visual Attention Network [[paper]]() 45 | - [x] [2021] [survey] Attention Mechanisms in Computer Vision: A Survey [[paper]]() 46 | - [x] [2021] [survey] Attention mechanisms and deep learning for machine vision: A survey of the state of the art [[paper]]() 47 | - [x] [2016] [survey] Survey on the attention based RNN model and its applications in computer vision [[paper]]() 48 | - [x] [2022] [survey] Efficient Transformers: A Survey [[paper]]() 49 | - [x] [2021] [survey] A Survey of Transformers [[paper]]() 50 | - [x] [TPAMI2022] [survey] A Survey on Vision Transformer [[paper]]() 51 | - [x] [2022] [survey] Transformers in Vision: A Survey [[paper]]() 52 | - [x] [2022] [survey] Transformers Meet Visual Learning Understanding: A Comprehensive Review [[paper]]() 53 | - [x] [2022] [survey] Transformers in computational visual media: A survey [[paper]]() 54 | - [x] [2021] [survey] A Survey of Visual Transformers [[paper]]() 55 | - [x] [2020] [survey] A Survey on Visual Transformer [[paper]]() 56 | - [x] [2022] [survey] Transformers in Time Series: A Survey [[paper]]() 57 | 58 | ## GRANs 59 | 60 | ### GRU Attention 61 | - [x] [ICLR2016] [GGNN] Gated Graph Sequence Neural Networks [[paper]](http://arxiv.org/abs/1511.05493) [[code]](https://github.com/yujiali/ggnn) 62 | - [x] [UAI2018] [GaAN] GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs [[paper]](http://arxiv.org/abs/1803.07294) [[code]](https://github.com/jennyzhang0215/GaAN) 63 | - [x] [ICML2018] [GraphRNN] Graphrnn: Generating realistic graphs with deep auto-regressive models [[paper]](http://arxiv.org/abs/1802.08773) [[code]](https://github.com/snap-stanford/GraphRNN) 64 | - [x] [NeurIPS2019] [GRAN] Efficient Graph Generation with Graph Recurrent Attention Networks [[paper]](http://arxiv.org/abs/1910.00760) [[code]](https://github.com/lrjconan/GRAN) 65 | - [x] [T-SP2020] [GRNN] Gated Graph Recurrent Neural Networks [[paper]](https://arxiv.org/abs/2002.01038) 66 | - [x] [2021] [GR-GAT] Gated Relational Graph Attention Networks [[paper]](https://openreview.net/forum?id=v-9E8egy_i) 67 | 68 | ### LSTM Attention 69 | - [x] [NeurIPS2017] [GraphSage] Inductive representation learning on large graphs [[paper]](https://arxiv.org/abs/1706.02216) [[code]](https://github.com/williamleif/GraphSAGE) 70 | - [x] [ICML2018] [JK-Net] Representation Learning on Graphs with Jumping Knowledge Networks [[paper]](http://proceedings.mlr.press/v80/xu18c.html) 71 | - [x] [KDD2018] [GAM] Graph Classification using Structural Attention [[paper]](https://doi.org/10.1145/3219819.3219980) [[code]](https://github.com/benedekrozemberczki/GAM) 72 | - [x] [AAAI2019] [GeniePath] GeniePath: Graph Neural Networks with Adaptive Receptive Paths [[paper]](http://arxiv.org/abs/1802.00910) 73 | 74 | ## GATs（Intra-layer） 75 | 76 | ### Neighbor Attention 77 | 78 | - [x] [ICLR2018] [GAT] Graph Attention Networks [[paper]](https://openreview.net/forum?id=rJXMpikCZ) [[code]](https://github.com/PetarV-/GAT) 79 | - [x] [NeurIPS2019] [C-GAT] Improving Graph Attention Networks with Large Margin-based Constraints [[paper]](http://arxiv.org/abs/1910.11945) 80 | - [x] [IJCAI2020] [CPA] Improving Attention Mechanism in Graph Neural Networks via Cardinality Preservation [[paper]](https://pubmed.ncbi.nlm.nih.gov/32782421/) [[code]](https://github.com/zetayue/CPA) 81 | - [x] [ICLR2022] [GATv2] How Attentive are Graph Attention Networks? [[paper]](http://arxiv.org/abs/2105.14491) [[code]](https://github.com/tech-srl/how_attentive_are_gats) 82 | - [x] [ICLR2022] [PPRGAT] Personalized PageRank Meets Graph Attention Networks [[paper]](https://openreview.net/forum?id=XNYOJD0QdBD) 83 | - [x] [KDD2021] [Simple-HGN] Are we really making much progress? Revisiting, benchmarking and refining heterogeneous graph neural networks [[paper]](https://doi.org/10.1145/3447548.3467350) [[code]](https://github.com/THUDM/HGB) 84 | - [x] [ICLR2021] [SuperGAT] How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision [[paper]](https://openreview.net/forum?id=Wi5KUNlqWty) [[code]](https://github.com/dongkwan-kim/SuperGAT) 85 | - [x] [NeurIPS2021] [DMP] Diverse Message Passing for Attribute with Heterophily [[paper]](https://openreview.net/forum?id=4jPVcKEYpSZ) 86 | - [x] [KDD2019] [GANet] Graph Representation Learning via Hard and Channel-Wise Attention Networks [[paper]](https://doi.org/10.1145/3292500.3330897) 87 | - [x] [NeurIPS2021] [CAT] Learning Conjoint Attentions for Graph Neural Nets [[paper]](http://arxiv.org/abs/2102.03147) [[code]](https://github.com/he-tiantian/cats) 88 | - [x] [TransNNLS2021] [MSNA] Neighborhood Attention Networks With Adversarial Learning for Link Prediction [[paper]](https://ieeexplore.ieee.org/document/9174790) 89 | - [x] [arXiv2018] [GGAT] Deeply learning molecular structure-property relationships using attention- and gate-augmented graph convolutional network [[paper]](http://arxiv.org/abs/1805.10988) 90 | - [x] [NeurIPS2019] [HGCN] Hyperbolic Graph Convolutional Neural Networks [[paper]](http://arxiv.org/abs/1910.12933) 91 | - [x] [TBD2021] [HAT] Hyperbolic graph attention network [[paper]](https://arxiv.org/abs/1912.03046) 92 | - [x] [IJCAI2020] [Hype-HAN] Hype-HAN: Hyperbolic Hierarchical Attention Network for Semantic Embedding [[paper]](https://www.ijcai.org/proceedings/2020/552) 93 | - [x] [ICLR2019] [] Hyperbolic Attention Networks [[paper]](http://arxiv.org/abs/1805.09786) 94 | - [x] [ICLR2018] [AGNN] Attention-based Graph Neural Network for Semi-supervised Learning [[paper]](http://arxiv.org/abs/1803.03735) 95 | - [x] [KDD2018] [HTNE] Embedding Temporal Network via Neighborhood Formation [[paper]](https://doi.org/10.1145/3219819.3220054) [[code]](https://github.com/hongyuanmei/neurawkes) 96 | - [x] [CVPR2020] [DKGAT] Distilling Knowledge from Graph Convolutional Networks [[paper]](http://arxiv.org/abs/2003.10477) [[code]](https://github.com/ihollywhy/DistillGCN.PyTorch) 97 | - [x] [ICCV2021] [DAGL] Dynamic Attentive Graph Learning for Image Restoration [[paper]](http://arxiv.org/abs/2109.06620) [[code]](https://github.com/jianzhangcs/dagl) 98 | - [x] [AAAI2021] [SCGA] Structured Co-reference Graph Attention for Video-grounded Dialogue [[paper]](http://arxiv.org/abs/2103.13361) 99 | - [x] [AAAI2021] [Co-GAT] Co-GAT: A Co-Interactive Graph Attention Network for Joint Dialog Act Recognition and Sentiment Classification [[paper]](http://arxiv.org/abs/2012.13260) 100 | - [x] [ACL2020] [ED-GAT] Entity-Aware Dependency-Based Deep Graph Attention Network for Comparative Preference Classification [[paper]](https://aclanthology.org/2020.acl-main.512) 101 | - [x] [EMNLP2019] [TD-GAT] Syntax-Aware Aspect Level Sentiment Classification with Graph Attention Networks [[paper]](http://arxiv.org/abs/1909.02606) 102 | - [x] [WWW2020] [GATON] Graph Attention Topic Modeling Network [[paper]](https://doi.org/10.1145/3366423.3380102) 103 | - [x] [KDD2017] [GRAM] GRAM: Graph-based attention model for healthcare representation learning [[paper]](https://doi.org/10.1145/3097983.3098126) [[code]](https://github.com/mp2893/gram) 104 | 105 | ### High-Order Attention 106 | - [x] [IJCAI2019] [SPAGAN] SPAGAN: Shortest Path Graph Attention Network [[paper]](https://www.ijcai.org/proceedings/2019/569) 107 | - [x] [PKDD2021] [PaGNN] Inductive Link Prediction with Interactive Structure Learning on Attributed Graph [[paper]](https://link.springer.com/book/10.1007/978-3-030-86486-6) 108 | - [x] [arXiv2019] [DeepLinker] Link Prediction via Graph Attention Network [[paper]](http://arxiv.org/abs/1910.04807) 109 | - [x] [IJCNN2020] [CGAT] Heterogeneous Information Network Embedding with Convolutional Graph Attention Networks [[paper]](https://ieeexplore.ieee.org/document/9206610) 110 | - [x] [ICLR2020] [ADSF] Adaptive Structural Fingerprints for Graph Attention Networks [[paper]](https://openreview.net/forum?id=BJxWx0NYPr) 111 | - [x] [KDD2021] [T-GAP] Learning to Walk across Time for Interpretable Temporal Knowledge Graph Completion [[paper]](https://doi.org/10.1145/3447548.3467292) [[code]](https://github.com/sharkmir1/T-GAP) 112 | - [x] [NeurIPS2018] [MAF] Modeling Attention Flow on Graphs [[paper]](http://arxiv.org/abs/1811.00497) 113 | - [x] [IJCAI2021] [MAGNA] Multi-hop Attention Graph Neural Network [[paper]](http://arxiv.org/abs/2009.14332) [[code]](https://github.com/xjtuwgt/gnn-magna) 114 | 115 | ### Relation-Aware Attention 116 | 117 | - [x] [AAAI2020] [SNEA] Learning Signed Network Embedding via Graph Attention [[paper]](https://ojs.aaai.org/index.php/AAAI/article/view/5911) 118 | - [x] [ICANN2019] [SiGAT] Signed Graph Attention Networks [[paper]](https://link.springer.com/chapter/10.1007/978-3-030-30493-5_53) 119 | - [x] [ICLR2019] [RGAT] Relational Graph Attention Networks [[paper]](http://arxiv.org/abs/1904.05811) [[code]](https://github.com/babylonhealth/rgat) 120 | - [x] [arXiv2018] [EAGCN] Edge attention-based multi-relational graph convolutional networks [[paper]](http://arxiv.org/abs/1802.04944) [[code]](https://github.com/Luckick/EAGCN) 121 | - [x] [KDD2021] [WRGNN] Breaking the Limit of Graph Neural Networks by Improving the Assortativity of Graphs with Local Mixing Patterns [[paper]](https://doi.org/10.1145/3447548.3467373) [[code]](https://github.com/susheels/gnns-and-local-assortativity) 122 | - [x] [AAAI2020] [HetSANN] An Attention-based Graph Neural Network for Heterogeneous Structural Learning [[paper]](http://arxiv.org/abs/1912.10832) [[code]](https://github.com/didi/hetsann) 123 | - [x] [AAAI2020] [TALP] Type-Aware Anchor Link Prediction across Heterogeneous Networks Based on Graph Attention Network [[paper]](https://ojs.aaai.org/index.php/AAAI/article/view/5345) 124 | - [x] [KDD2019] [KGAT] KGAT: Knowledge Graph Attention Network for Recommendation [[paper]](http://arxiv.org/abs/1905.07854) [[code]](https://github.com/xiangwang1223/knowledge_graph_attention_network) 125 | - [x] [KDD2019] [GATNE] Representation Learning for Attributed Multiplex Heterogeneous Network [[paper]](https://doi.org/10.1145/3292500.3330964) [[code]](https://github.com/THUDM/GATNE) 126 | - [x] [AAAI2021] [RelGNN] Relation-aware Graph Attention Model With Adaptive Self-adversarial Training [[paper]](http://arxiv.org/abs/2102.07186) 127 | - [x] [KDD2020] [CGAT] Graph Attention Networks over Edge Content-Based Channels [[paper]](https://doi.org/10.1145/3394486.3403233) 128 | - [x] [ACL2019] [AFE] Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs [[paper]](http://arxiv.org/abs/1906.01195) [[code]](https://github.com/deepakn97/relationPrediction) 129 | - [x] [CIKM2021] [DisenKGAT] DisenKGAT: Knowledge Graph Embedding with Disentangled Graph Attention Network [[paper]](https://arxiv.org/abs/2108.09628v2) 130 | - [x] [AAAI2021] [GTAN] Graph-Based Tri-Attention Network for Answer Ranking in CQA [[paper]](http://arxiv.org/abs/2103.03583) 131 | - [x] [ACL2020] [R-GAT] Relational Graph Attention Network for Aspect-based Sentiment Analysis [[paper]](http://arxiv.org/abs/2004.12362) [[code]](https://github.com/shenwzh3/RGAT-ABSA) 132 | - [x] [ICCV2019] [ReGAT] Relation-Aware Graph Attention Network for Visual Question Answering [[paper]](https://arxiv.org/abs/1903.12314v3) [[code]](https://github.com/linjieli222/VQA_ReGAT) 133 | - [x] [AAAI2021] [AD-GAT] Modeling the Momentum Spillover Effect for Stock Prediction via Attribute-Driven Graph Attention Networks [[paper]](https://ojs.aaai.org/index.php/AAAI/article/view/16077) 134 | 135 | ### Attention Sampling/Pooling 136 | 137 | - [x] [NeurIPS2018] [GAW] Watch your step: learning node embeddings via graph attention [[paper]](https://arxiv.org/abs/1710.09599v2) 138 | - [x] [TPAMI2021] [NLGAT] Non-Local Graph Neural Networks [[paper]](https://arxiv.org/abs/2005.14612) 139 | - [x] [NeurIPS2019] [ChebyGIN] Understanding Attention and Generalization in Graph Neural Networks [[paper]](http://papers.nips.cc/paper/8673-understanding-attention-and-generalization-in-graph-neural-networks.pdf) [[code]](https://github.com/bknyaz/graph_attention_pool) 140 | - [x] [ICML2019] [SAGPool] Self-Attention Graph Pooling [[paper]](http://proceedings.mlr.press/v97/lee19c.html) [[code]](https://github.com/inyeoplee77/SAGPool) 141 | - [x] [ICCV2019] [Attpool] Attpool: Towards hierarchical feature representation in graph convolutional networks via attention mechanism [[paper]](https://ieeexplore.ieee.org/document/9009471) 142 | 143 | ### Hierarchical Attention 144 | 145 | - [x] [WWW2019] [HAN] Heterogeneous Graph Attention Network [[paper]](https://doi.org/10.1145/3308558.3313562) [[code]]( https://github.com/Jhy1993/HAN) 146 | - [x] [NC2022] [PSHGAN] Heterogeneous graph embedding by aggregating meta-path and meta-structure through attention mechanism [[paper]](https://www.sciencedirect.com/science/article/abs/pii/S092523122101479X) 147 | - [x] [IJCAI2017] [PRML] Link prediction via ranking metric dual-level attention network learning [[paper]](https://www.ijcai.org/proceedings/2017/493) 148 | - [x] [WSDM2022] [GraphHAM] Graph Embedding with Hierarchical Attentive Membership [[paper]](http://arxiv.org/abs/2111.00604) 149 | - [x] [AAAI2020] [RGHAT] Relational Graph Neural Network with Hierarchical Attention for Knowledge Graph Completion [[paper]](https://ojs.aaai.org/index.php/AAAI/article/view/6508) 150 | - [x] [AAAI2019] [LAN] Logic Attention Based Neighborhood Aggregation for Inductive Knowledge Graph Embedding [[paper]](https://doi.org/10.1609/aaai.v33i01.33017152) [[code]](https://github.com/wangpf3/LAN) 151 | - [x] [2022] [EFEGAT] Learning to Solve an Order Fulfillment Problem in Milliseconds with Edge-Feature-Embedded Graph Attention [[paper]](https://openreview.net/forum?id=qPQRIj_Y_EW) 152 | - [x] [WWW2019] [DANSER] Dual Graph Attention Networks for Deep Latent Representation of Multifaceted Social Effects in Recommender Systems [[paper]](https://doi.org/10.1145/3308558.3313442) [[code]](https://github.com/echo740/DANSER-WWW-19) 153 | - [x] [WWW2019] [UVCAN] User-Video Co-Attention Network for Personalized Micro-video Recommendation [[paper]](https://doi.org/10.1145/3308558.3313513) 154 | - [x] [KBS2020] [HAGERec] HAGERec: Hierarchical Attention Graph Convolutional Network Incorporating Knowledge Graph for Explainable Recommendation [[paper]](https://www.sciencedirect.com/science/article/pii/S0950705120304196) 155 | - [x] [Bio2021] [GCATSL] Graph contextualized attention network for predicting synthetic lethality in human cancers [[paper]](https://doi.org/10.1093/bioinformatics/btab110) [[code]](https://github.com/longyahui/GCATSL) 156 | - [x] [ICMR2020] [DAGC] DAGC: Employing Dual Attention and Graph Convolution for Point Cloud based Place Recognition [[paper]](https://doi.org/10.1145/3372278.3390693) [[code]](https://github.com/dawenzi123/DAGCN) 157 | - [x] [AAAI2020] [AGCN] Graph Attention Based Proposal 3D ConvNets for Action Detection [[paper]](https://ojs.aaai.org/index.php/AAAI/article/view/5893) 158 | - [x] [EMNLP2019] [HGAT] HGAT: Heterogeneous Graph Attention Networks for Semi-supervised Short Text Classification [[paper]](https://dl.acm.org/doi/10.1145/3450352) [[code]](https://github.com/ytc272098215/HGAT) 159 | 160 | ### Hyper-Attention 161 | 162 | - [x] [ICLR2020] [Hyper-SAGNN] Hyper-Sagnn: A Self-Attention based Graph Neural Network for Hypergraphs [[paper]](http://arxiv.org/abs/1911.02613) [[code]](https://github.com/ma-compbio/Hyper-SAGNN) 163 | - [x] [CIKM2021] [HHGR] Double-Scale Self-Supervised Hypergraph Learning for Group Recommendation [[paper]](http://arxiv.org/abs/2109.04200) [[code]](https://github.com/0411tony/hhgr) 164 | - [x] [ICDM2021] [HyperTeNet] HyperTeNet: Hypergraph and Transformer-based Neural Network for Personalized List Continuation [[paper]](http://arxiv.org/abs/2110.01467) [[code]](https://github.com/mvijaikumar/hypertenet) 165 | - [x] [PR2020] [Hyper-GAT] Hypergraph Convolution and Hypergraph Attention [[paper]](http://arxiv.org/abs/1901.08150) [[code]](https://github.com/rusty1s/pytorch_geometric) 166 | - [x] [CVPR2020] [] Hypergraph attention networks for multimodal learning [[paper]](https://ieeexplore.ieee.org/document/9157723) 167 | 168 | ## Inter-Layer GATs 169 | ### Multi-Level Attention 170 | 171 | - [x] [KDD2020] [DAGNN] Towards Deeper Graph Neural Networks [[paper]](https://doi.org/10.1145/3394486.3403076) [[code]](https://github.com/divelab/DeeperGNN) 172 | - [x] [T-NNLS2020] [AP-GCN] Adaptive propagation graph convolutional network [[paper]](https://arxiv.org/abs/2002.10306) [[code]](https://github.com/spindro/AP-GCN) 173 | - [x] [CIKM2021] [TDGNN] Tree Decomposed Graph Neural Network [[paper]](https://doi.org/10.1145/3459637.3482487) [[code]](https://github.com/YuWVandy/TDGNN) 174 | - [x] [arXiv2020] [GMLP] GMLP: Building Scalable and Flexible Graph Neural Networks with Feature-Message Passing [[paper]](http://arxiv.org/abs/2104.09880) 175 | - [x] [KDD2022] [GAMLP] Graph Attention Multi-layer Perceptron [[paper]](http://arxiv.org/abs/2108.10097) [[code]](https://github.com/zwt233/GAMLP) 176 | 177 | ### Multi-Channel Attention 178 | 179 | - [x] [AAAI2021] [FAGCN] Beyond Low-frequency Information in Graph Convolutional Networks [[paper]](http://arxiv.org/abs/2101.00797) [[code]](https://github.com/bdy9527/FAGCN) 180 | - [x] [arXiv2021] [ACM] Is Heterophily A Real Nightmare For Graph Neural Networks To Do Node Classification? [[paper]](http://arxiv.org/abs/2109.05641) 181 | 182 | ### Multi-View Attention 183 | 184 | - [x] [KDD2020] [AM-GCN] AM-GCN: Adaptive Multi-channel Graph Convolutional Networks [[paper]](https://doi.org/10.1145/3394486.3403177) [[code]](https://github.com/zhumeiqiBUPT/AM-GCN) 185 | - [x] [AAAI2021] [UAG] Uncertainty-aware Attention Graph Neural Network for Defending Adversarial Attacks [[paper]](http://arxiv.org/abs/2009.10235) 186 | - [x] [CIKM2021] [MV-GNN] Semi-Supervised and Self-Supervised Classification with Multi-View Graph Neural Networks [[paper]](https://doi.org/10.1145/3459637.3482477) 187 | - [x] [KSEM2021] [GENet] Graph Ensemble Networks for Semi-supervised Embedding Learning [[paper]](https://dl.acm.org/doi/abs/10.1007/978-3-030-82136-4_33) 188 | - [x] [NN2020] [MGAT] MGAT: Multi-view Graph Attention Networks [[paper]](https://www.sciencedirect.com/science/article/pii/S0893608020303105) 189 | - [x] [CIKM2017] [MVE] An Attention-based Collaboration Framework for Multi-View Network Representation Learning [[paper]](https://doi.org/10.1145/3132847.3133021) 190 | - [x] [NC2021] [EAGCN] Multi-view spectral graph convolution with consistent edge attention for molecular modeling [[paper]](https://www.sciencedirect.com/science/article/pii/S092523122100271X) 191 | - [x] [SIGIR2020] [GCE-GNN] Global Context Enhanced Graph Neural Networks for Session-based Recommendation [[paper]](http://arxiv.org/abs/2106.05081) 192 | 193 | ### Spatio-Temporal Attention 194 | 195 | - [x] [ICLR2019] [DySAT] Dynamic Graph Representation Learning via Self-Attention Networks [[paper]](https://arxiv.org/abs/1812.09430v2) [[code]](https://github.com/aravindsankar28/DySAT) 196 | - [x] [PAKDD2020] [TemporalGAT] TemporalGAT: Attention-Based Dynamic Graph Representation Learning [[paper]](https://dl.acm.org/doi/10.1007/978-3-030-47426-3_32) 197 | - [x] [IJCAI2021] [GAEN] GAEN: Graph Attention Evolving Networks [[paper]](https://www.ijcai.org/proceedings/2021/213) [[code]](https://github.com/codeshareabc/GAEN) 198 | - [x] [CIKM2019] [MMDNE] Temporal Network Embedding with Micro- and Macro-dynamics [[paper]](https://doi.org/10.1145/3357384.3357943) [[code]](https://github.com/rootlu/MMDNE) 199 | - [x] [ICLR2020] [TGAT] Inductive Representation Learning on Temporal Graphs [[paper]](http://arxiv.org/abs/2002.07962) [[code]](https://github.com/StatsDLMathsRecomSys/Inductive-representation-learning-on-temporal-graphs) 200 | - [x] [2022] [TR-GAT] Time-Aware Relational Graph Attention Network for Temporal Knowledge Graph Embeddings [[paper]](https://openreview.net/forum?id=ShtJLsF7cbb) 201 | - [x] [CVPR2022] [T-GNN] Adaptive Trajectory Prediction via Transferable GNN [[paper]](http://arxiv.org/abs/2203.05046) 202 | - [x] [AAAI2018] [ST-GCN] Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition [[paper]](https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/17135) [[code]](https://github.com/yysijie/st-gcn) 203 | - [x] [AAAI2020] [GMAN] GMAN: A Graph Multi-Attention Network for Traffic Prediction [[paper]](http://arxiv.org/abs/1911.08415) 204 | - [x] [AAAI2019] [ASTGCN] Attention Based Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting [[paper]](https://ojs.aaai.org/index.php/AAAI/article/view/3881) [[code]](https://github.com/Davidham3/ASTGCN) 205 | - [x] [KDD2020] [ConSTGAT] ConSTGAT: Contextual Spatial-Temporal Graph Attention Network for Travel Time Estimation at Baidu Maps [[paper]](https://doi.org/10.1145/3394486.3403320) 206 | 207 | ### Time Series Attention 208 | 209 | - [x] [ICLR2022] [RainDrop] Graph-Guided Network for Irregularly Sampled Multivariate Time Series [[paper]](http://arxiv.org/abs/2110.05357) 210 | - [x] [ICLR2021] [mTAND] Multi-Time Attention Networks for Irregularly Sampled Time Series [[paper]](http://arxiv.org/abs/2101.10318) [[code]](https://github.com/reml-lab/mTAN) 211 | - [x] [ICDM2020] [MTAD-GAT] Multivariate Time-series Anomaly Detection via Graph Attention Network [[paper]](http://arxiv.org/abs/2009.02040) 212 | - [x] [WWW2020] [GACNN] Towards Fine-grained Flow Forecasting: A Graph Attention Approach for Bike Sharing Systems [[paper]](https://dl.acm.org/doi/10.1145/3366423.3380097) 213 | - [x] [IJCAI2018] [GeoMAN] GeoMAN: Multi-level Attention Networks for Geo-sensory Time Series Prediction [[paper]](https://www.ijcai.org/proceedings/2018/476) 214 | 215 | 216 | 217 | ## GraphTransformers 218 | 219 | - [x] [arXiv2019] [GTR] Graph transformer [[paper]](https://openreview.net/forum?id=HJei-2RcK7) 220 | - [x] [arXiv2020] [U2GNN] Universal Self-Attention Network for Graph Classification [[paper]](http://arxiv.org/abs/1909.11855) [[code]](https://github.com/daiquocnguyen/Graph-Transformer) 221 | - [x] [WWW2022] [UGformer] Universal Graph Transformer Self-Attention Networks [[paper]](http://arxiv.org/abs/1909.11855) [[code]](https://github.com/daiquocnguyen/Graph-Transformer) 222 | - [x] [ICLR2021] [GMT] Accurate learning of graph representations with graph multiset pooling [[paper]](http://arxiv.org/abs/2102.11533) [[code]](https://github.com/JinheonBaek/GMT) 223 | - [x] [KDDCup2021] [Graphormer] Do Transformers Really Perform Bad for Graph Representation? [[paper]](http://arxiv.org/abs/2106.08279) [[code]](https://github.com/microsoft/Graphormer) 224 | - [x] [NeurIPS2021] [Graphormer] Do Transformers Really Perform Bad for Graph Representation? [[paper]](http://arxiv.org/abs/2106.05234) [[code]](https://github.com/microsoft/Graphormer) 225 | - [x] [NeurIPS2021] [HOT] Transformers Generalize DeepSets and Can be Extended to Graphs and Hypergraphs [[paper]](http://arxiv.org/abs/2110.14416) [[code]](https://github.com/jw9730/hot) 226 | - [x] [NeurIPS2020] [GROVER] Self-Supervised Graph Transformer on Large-Scale Molecular Data [[paper]](http://arxiv.org/abs/2007.02835) [[code]](https://github.com/tencent-ailab/grover) 227 | - [x] [ICML(Workshop)2019] [PAGAT] Path-augmented graph transformer network [[paper]](https://arxiv.org/abs/1905.12712) [[code]](https://github.com/benatorc/PA-Graph-Transformer) 228 | - [x] [AAAI2021] [GTA] GTA: Graph Truncated Attention for Retrosynthesis [[paper]](https://ojs.aaai.org/index.php/AAAI/article/view/16131) 229 | - [x] [AAAI2021] [GT] A generalization of transformer networks to graphs [[paper]](http://arxiv.org/abs/2012.09699) [[code]](https://github.com/graphdeeplearning/graphtransformer) 230 | - [x] [NeurIPS2021] [SAN] Rethinking graph transformers with spectral attention [[paper]](http://arxiv.org/abs/2106.03893) [[code]](https://github.com/DevinKreuzer/SAN) 231 | - [x] [2020] [GraphBert] Graph-bert: Only attention is needed for learning graph representations [[paper]](http://arxiv.org/abs/2001.05140) [[code]](https://github.com/jwzhanggy/Graph-Bert) 232 | - [x] [ICML2021] [] Lipschitz Normalization for Self-Attention Layers with Application to Graph Neural Networks [[paper]](http://arxiv.org/abs/2103.04886) 233 | - [x] [IJCAI2021] [UniMP] Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification [[paper]](http://arxiv.org/abs/2009.03509) [[code]](https://github.com/PaddlePaddle/PGL/tree/main/ogb_examples/nodeproppred/unimp) 234 | - [x] [NeurIPS2019] [GTN] Graph Transformer Networks [[paper]](http://arxiv.org/abs/1911.06455) [[code]](https://github.com/seongjunyun/Graph_Transformer_Networks) 235 | - [x] [KDD2020] [TagGen] A Data-Driven Graph Generative Model for Temporal Interaction Networks [[paper]](https://doi.org/10.1145/3394486.3403082) 236 | - [x] [NeurIPS2021] [GraphFormers] GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph [[paper]](http://arxiv.org/abs/2105.02605) 237 | - [x] [WWW2020] [HGT] Heterogeneous Graph Transformer [[paper]](https://doi.org/10.1145/3366423.3380027) 238 | - [x] [AAAI2020] [GTOS] Graph transformer for graph-to-sequence learning [[paper]](http://arxiv.org/abs/1911.07470) [[code]](https://github.com/jcyk/gtos) 239 | - [x] [NAACL2019] [GraphWriter] Text Generation from Knowledge Graphs with Graph Transformers [[paper]](http://arxiv.org/abs/1904.02342) [[code]](https://github.com/rikdz/GraphWriter) 240 | - [x] [AAAI2021] [KHGT] Knowledge-Enhanced Hierarchical Graph Transformer Network for Multi-Behavior Recommendation [[paper]](https://ojs.aaai.org/index.php/AAAI/article/view/16576) [[code]](https://github.com/akaxlh/KHGT) 241 | - [x] [AAAI2021] [GATE] GATE: Graph Attention Transformer Encoder for Cross-lingual Relation and Event Extraction [[paper]](http://arxiv.org/abs/2010.03009) 242 | - [x] [NeurIPS2021] [STAGIN] Learning Dynamic Graph Representation of Brain Connectome with Spatio-Temporal Attention [[paper]](http://arxiv.org/abs/2105.13495) [[code]](https://github.com/egyptdj/stagin) 243 | 244 | ## Next 245 | 246 | - [x] [journal] [model] paper_title [[paper]](link) [[code]](link) 247 | 248 | ## Citation 249 | 250 | ``` 251 | Sun, C., Li, C., Lin, X. et al. Attention-based graph neural networks: a survey. Artif Intell Rev (2023). https://doi.org/10.1007/s10462-023-10577-2 252 | ``` 253 | 254 | ### Bib 255 | 256 | ``` 257 | @article{ WOS:001051847200001, 258 | Author = {Sun, Chengcheng and Li, Chenhao and Lin, Xiang and Zheng, Tianji and Meng, Fanrong and Rui, Xiaobin and Wang, Zhixiao}, 259 | Title = {Attention-based graph neural networks: a survey}, 260 | Journal = {ARTIFICIAL INTELLIGENCE REVIEW}, 261 | Year = {2023}, 262 | Month = {2023 AUG 21}, 263 | DOI = {10.1007/s10462-023-10577-2}, 264 | EarlyAccessDate = {AUG 2023}, 265 | ISSN = {0269-2821}, 266 | EISSN = {1573-7462}, 267 | Unique-ID = {WOS:001051847200001}, 268 | } 269 | ``` 270 | 271 | -------------------------------------------------------------------------------- /assets/README/image-20231101100623183.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunxiaobei/awesome-attention-based-gnns/aac4f4025cf6a49a0987ae6e5202ef0c48d3b9a7/assets/README/image-20231101100623183.png --------------------------------------------------------------------------------