├── data └── README.md ├── ours ├── README.md ├── inference_time.sh ├── train_and_test.sh ├── print_dataset_statistics.py ├── utils.py ├── convert_datasets_to_pygDataset.py ├── load_other_datasets.py ├── preprocessing.py ├── train.py └── inference_time_test.py ├── baselines_hypergnn ├── README.md ├── print_dataset_statistics.py ├── utils.py ├── convert_datasets_to_pygDataset.py ├── load_other_datasets.py ├── perturb.sh ├── preprocessing.py ├── inference_time_test.py └── layers.py └── README.md /data/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /ours/README.md: -------------------------------------------------------------------------------- 1 | ## Overview 2 | 3 | For detailed instructions on reproducing the results presented in Table 2 of our paper, please refer to the train_and_test.sh script. 4 | 5 | For detailed instructions on reproducing the results presented in Table 3 of our paper, please refer to the inference_time.sh script. 6 | -------------------------------------------------------------------------------- /baselines_hypergnn/README.md: -------------------------------------------------------------------------------- 1 | ## Overview 2 | 3 | The results for our baseline hypergraph neural networks, as detailed in Table 2 of our paper, are from the experimental results in AllSet ([Paper](https://openreview.net/forum?id=hpBTIv2uy_E); [Github](https://github.com/jianhao2016/AllSet)). 4 | 5 | For detailed instructions on reproducing the results presented in Table 3 of our paper, please refer to the inference_time.sh script. 6 | 7 | For detailed instructions on reproducing the results presented in Figure 2 of our paper, please refer to the perturb.sh script. 8 | -------------------------------------------------------------------------------- /ours/inference_time.sh: -------------------------------------------------------------------------------- 1 | python inference_time_test.py --method MLP --dname 20newsW100 --All_num_layers 3 --feature_noise 0.0 --MLP_hidden 256 --wd 0.0 --runs 60000 --cuda 0 --lr 0.001 --alpha 0.05 2 | python inference_time_test.py --method MLP --dname cora --All_num_layers 5 --feature_noise 0.0 --MLP_hidden 256 --wd 0.0 --runs 60000 --cuda 0 --lr 0.001 --alpha 0.01 3 | python inference_time_test.py --method MLP --dname citeseer --All_num_layers 2 --feature_noise 0.0 --MLP_hidden 512 --wd 0.0 --runs 60000 --cuda 0 --lr 0.001 --alpha 0.5 4 | python inference_time_test.py --method MLP --dname NTU2012 --All_num_layers 4 --feature_noise 0.0 --MLP_hidden 256 --wd 0.0 --runs 60000 --cuda 0 --lr 0.001 --alpha 0.05 5 | python inference_time_test.py --method MLP --dname pubmed --All_num_layers 2 --feature_noise 0.0 --MLP_hidden 256 --wd 0.0 --runs 60000 --cuda 0 --lr 0.001 --alpha 0.05 6 | python inference_time_test.py --method MLP --dname coauthor_dblp --All_num_layers 5 --feature_noise 0.0 --MLP_hidden 512 --wd 0.0 --runs 60000 --cuda 0 --lr 0.001 --alpha 0.02 7 | python inference_time_test.py --method MLP --dname house-committees-100 --All_num_layers 3 --feature_noise 0.6 --MLP_hidden 512 --wd 0.0 --runs 60000 --cuda 0 --lr 0.0001 --alpha 0.1 8 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Hypergraph-MLP 2 | 3 | This is the repo for our ICASSP 2024 paper: [Hypergraph-MLP: Learning on Hypergraphs without Message Passing](https://arxiv.org/pdf/2312.09778). 4 | 5 | ## Overview 6 | 7 | A quick summary of different folders: 8 | 9 | - 'baselines_hypergnn' contains the source code for our baseline hypergraph neural networks. 10 | 11 | - 'ours' contains the source code for our Hypergraph-MLP. 12 | 13 | ## Recommend Environment: 14 | ``` 15 | conda create -n "hgmlp" python=3.7 16 | conda activate hgmlp 17 | ``` 18 | ``` 19 | conda install pytorch==1.4.0 torchvision==0.5.0 cudatoolkit=10.0 -c pytorch 20 | ``` 21 | ``` 22 | pip install torch-scatter==2.0.4 -f https://pytorch-geometric.com/whl/torch-1.4.0+cu100.html 23 | pip install torch-sparse==0.6.0 -f https://pytorch-geometric.com/whl/torch-1.4.0+cu100.html 24 | pip install torch-cluster==1.5.2 -f https://pytorch-geometric.com/whl/torch-1.4.0+cu100.html 25 | pip install torch-geometric==1.6.3 -f https://pytorch-geometric.com/whl/torch-1.4.0+cu100.html 26 | pip install ipdb 27 | pip install tqdm 28 | pip install scipy 29 | pip install matplotlib 30 | ``` 31 | ## Data Preparation: 32 | 33 | To generate a dataset for training using PyG or DGL, please set up the following three directories: 34 | ``` 35 | p2root: './data/pyg_data/hypergraph_dataset_updated/' 36 | p2raw: './data/AllSet_all_raw_data/' 37 | p2dgl_data: './data/dgl_data_raw/' 38 | ``` 39 | 40 | Next, unzip the raw data zip file into `p2raw`. The raw data zip file can be found in this [link](https://github.com/jianhao2016/AllSet/tree/main/data/raw_data). 41 | 42 | ## Acknowledgement 43 | 44 | This code is based on the official code of AllSet ([Paper](https://openreview.net/forum?id=hpBTIv2uy_E); [Github](https://github.com/jianhao2016/AllSet)). Sincere appreciation is extended for their valuable contributions. 45 | 46 | ## Citation 47 | 48 | If you use this code, please cite our paper: 49 | 50 | ``` 51 | @inproceedings{tang2024hypergraph, 52 | title={Hypergraph-MLP: Learning on Hypergraphs without Message Passing}, 53 | author={Tang, Bohan and Chen, Siheng and Dong, Xiaowen}, 54 | booktitle={ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, 55 | year={2024}, 56 | organization={IEEE} 57 | } 58 | ``` 59 | 60 | 61 | -------------------------------------------------------------------------------- /ours/train_and_test.sh: -------------------------------------------------------------------------------- 1 | # Hypergraph-MLP 2 | python train.py --method MLP --dname 20newsW100 --All_num_layers 3 --feature_noise 0.0 --MLP_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --alpha 0.05 3 | python train.py --method MLP --dname cora --All_num_layers 5 --feature_noise 0.0 --MLP_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --alpha 0.01 4 | python train.py --method MLP --dname citeseer --All_num_layers 2 --feature_noise 0.0 --MLP_hidden 512 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --alpha 0.5 5 | python train.py --method MLP --dname NTU2012 --All_num_layers 4 --feature_noise 0.0 --MLP_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --alpha 0.05 6 | python train.py --method MLP --dname pubmed --All_num_layers 2 --feature_noise 0.0 --MLP_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --alpha 0.05 7 | python train.py --method MLP --dname coauthor_dblp --All_num_layers 5 --feature_noise 0.0 --MLP_hidden 512 --wd 0.0 --epochs 1000 --runs 20 --cuda 0 --lr 0.001 --alpha 0.02 8 | python train.py --method MLP --dname house-committees-100 --All_num_layers 3 --feature_noise 0.6 --MLP_hidden 512 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.0001 --alpha 0.1 9 | 10 | # MLP 11 | python train.py --method MLP --dname 20newsW100 --All_num_layers 3 --feature_noise 0.0 --MLP_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --alpha 0 12 | python train.py --method MLP --dname cora --All_num_layers 5 --feature_noise 0.0 --MLP_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --alpha 0 13 | python train.py --method MLP --dname citeseer --All_num_layers 2 --feature_noise 0.0 --MLP_hidden 512 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --alpha 0 14 | python train.py --method MLP --dname NTU2012 --All_num_layers 4 --feature_noise 0.0 --MLP_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --alpha 0 15 | python train.py --method MLP --dname pubmed --All_num_layers 2 --feature_noise 0.0 --MLP_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --alpha 0 16 | python train.py --method MLP --dname coauthor_dblp --All_num_layers 5 --feature_noise 0.0 --MLP_hidden 512 --wd 0.0 --epochs 1000 --runs 20 --cuda 0 --lr 0.001 --alpha 0 17 | python train.py --method MLP --dname house-committees-100 --All_num_layers 3 --feature_noise 0.6 --MLP_hidden 512 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.0001 --alpha 0 18 | -------------------------------------------------------------------------------- /ours/print_dataset_statistics.py: -------------------------------------------------------------------------------- 1 | #! /usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | # vim:fenc=utf-8 4 | # 5 | # Copyright © 2021 6 | # 7 | # Distributed under terms of the MIT license. 8 | 9 | """ 10 | 11 | """ 12 | 13 | import numpy as np 14 | import torch as th 15 | import pandas as pd 16 | from tqdm import tqdm 17 | import ipdb 18 | 19 | from convert_datasets_to_pygDataset import dataset_Hypergraph 20 | from torch_scatter import scatter_add, scatter 21 | 22 | dname_list = ['cora', 'citeseer', 'pubmed', 23 | 'coauthor_cora', 'coauthor_dblp', 24 | 'NTU2012', 'ModelNet40', 25 | 'zoo', 'Mushroom', '20newsW100', 26 | 'yelp', 'house-committees-100', 'walmart-trips-100'] 27 | 28 | idx_list = ['num_node', 'num_he', 'num_feature', 'num_class', 29 | 'max_he_size', 'min_he_size', 'avg_he_size', 'median_he_size', 30 | 'max_node_degree', 'min_node_degree', 'avg_node_degree', 'median_node_degree'] 31 | 32 | stats_df = pd.DataFrame(columns = dname_list, index = idx_list) 33 | feature_noise = 1 34 | 35 | def get_stats(deg_list): 36 | tmp_list = deg_list.numpy() 37 | return [np.max(tmp_list), np.min(tmp_list), np.mean(tmp_list), np.median(tmp_list)] 38 | 39 | for dname in tqdm(dname_list): 40 | if dname not in ['house-committees-100', 'walmart-trips-100']: 41 | ds = dataset_Hypergraph(name = dname) 42 | else: 43 | ds = dataset_Hypergraph(name = dname, feature_noise = feature_noise) 44 | 45 | data = ds.data 46 | 47 | num_nodes = data.x.shape[0] 48 | num_features = data.x.shape[1] 49 | num_classes = len(data.y.unique()) 50 | 51 | c_idx = th.where(data.edge_index[0] == num_nodes)[0].min() 52 | V2E = data.edge_index[:, :c_idx] 53 | 54 | num_edges = len(V2E[1].unique()) 55 | if 'num_hyperedges' in data: 56 | num_he = data.num_hyperedges 57 | if isinstance(num_he, list): 58 | num_he = num_he[0] 59 | 60 | if num_he != num_edges: 61 | ipdb.set_trace() 62 | 63 | 64 | edge_weight = th.ones_like(V2E[0]) 65 | Vdeg = scatter_add(edge_weight, V2E[0], dim=0) 66 | HEdeg = scatter_add(edge_weight, V2E[1] - num_nodes, dim=0) 67 | 68 | V_list = get_stats(Vdeg) 69 | E_list = get_stats(HEdeg) 70 | 71 | 72 | num2str = lambda x: f'{int(x)}' if x == int(x) else f'{x:.2f}' 73 | stat_list = [num_nodes, num_edges, num_features, num_classes] + E_list + V_list 74 | stat_list = [num2str(x) for x in stat_list] 75 | 76 | stats_df[dname] = stat_list 77 | 78 | # stats_df.to_csv('datasets_stats.csv') 79 | print(stats_df) 80 | 81 | -------------------------------------------------------------------------------- /baselines_hypergnn/print_dataset_statistics.py: -------------------------------------------------------------------------------- 1 | #! /usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | # vim:fenc=utf-8 4 | # 5 | # Copyright © 2021 6 | # 7 | # Distributed under terms of the MIT license. 8 | 9 | """ 10 | 11 | """ 12 | 13 | import numpy as np 14 | import torch as th 15 | import pandas as pd 16 | from tqdm import tqdm 17 | import ipdb 18 | 19 | from convert_datasets_to_pygDataset import dataset_Hypergraph 20 | from torch_scatter import scatter_add, scatter 21 | 22 | dname_list = ['cora', 'citeseer', 'pubmed', 23 | 'coauthor_cora', 'coauthor_dblp', 24 | 'NTU2012', 'ModelNet40', 25 | 'zoo', 'Mushroom', '20newsW100', 26 | 'yelp', 'house-committees-100', 'walmart-trips-100'] 27 | 28 | idx_list = ['num_node', 'num_he', 'num_feature', 'num_class', 29 | 'max_he_size', 'min_he_size', 'avg_he_size', 'median_he_size', 30 | 'max_node_degree', 'min_node_degree', 'avg_node_degree', 'median_node_degree'] 31 | 32 | stats_df = pd.DataFrame(columns = dname_list, index = idx_list) 33 | feature_noise = 1 34 | 35 | def get_stats(deg_list): 36 | tmp_list = deg_list.numpy() 37 | return [np.max(tmp_list), np.min(tmp_list), np.mean(tmp_list), np.median(tmp_list)] 38 | 39 | for dname in tqdm(dname_list): 40 | if dname not in ['house-committees-100', 'walmart-trips-100']: 41 | ds = dataset_Hypergraph(name = dname) 42 | else: 43 | ds = dataset_Hypergraph(name = dname, feature_noise = feature_noise) 44 | 45 | data = ds.data 46 | 47 | num_nodes = data.x.shape[0] 48 | num_features = data.x.shape[1] 49 | num_classes = len(data.y.unique()) 50 | 51 | c_idx = th.where(data.edge_index[0] == num_nodes)[0].min() 52 | V2E = data.edge_index[:, :c_idx] 53 | 54 | num_edges = len(V2E[1].unique()) 55 | if 'num_hyperedges' in data: 56 | num_he = data.num_hyperedges 57 | if isinstance(num_he, list): 58 | num_he = num_he[0] 59 | 60 | if num_he != num_edges: 61 | ipdb.set_trace() 62 | 63 | 64 | edge_weight = th.ones_like(V2E[0]) 65 | Vdeg = scatter_add(edge_weight, V2E[0], dim=0) 66 | HEdeg = scatter_add(edge_weight, V2E[1] - num_nodes, dim=0) 67 | 68 | V_list = get_stats(Vdeg) 69 | E_list = get_stats(HEdeg) 70 | 71 | 72 | num2str = lambda x: f'{int(x)}' if x == int(x) else f'{x:.2f}' 73 | stat_list = [num_nodes, num_edges, num_features, num_classes] + E_list + V_list 74 | stat_list = [num2str(x) for x in stat_list] 75 | 76 | stats_df[dname] = stat_list 77 | 78 | # stats_df.to_csv('datasets_stats.csv') 79 | print(stats_df) 80 | 81 | -------------------------------------------------------------------------------- /ours/utils.py: -------------------------------------------------------------------------------- 1 | import torch, math, numpy as np, scipy.sparse as sp 2 | import torch.nn as nn, torch.nn.functional as F, torch.nn.init as init 3 | # import ipdb 4 | 5 | from torch.autograd import Variable 6 | from torch.nn.modules.module import Module 7 | from torch.nn.parameter import Parameter 8 | 9 | 10 | 11 | class HyperGraphConvolution(Module): 12 | """ 13 | Simple GCN layer, similar to https://arxiv.org/abs/1609.02907 14 | """ 15 | 16 | def __init__(self, a, b, reapproximate=True, cuda=None): 17 | super(HyperGraphConvolution, self).__init__() 18 | self.a, self.b = a, b 19 | self.reapproximate, self.device = reapproximate, torch.device('cuda:'+str(cuda)) 20 | 21 | self.W = Parameter(torch.FloatTensor(a, b)) 22 | self.bias = Parameter(torch.FloatTensor(b)) 23 | self.reset_parameters() 24 | 25 | 26 | 27 | def reset_parameters(self): 28 | std = 1. / math.sqrt(self.W.size(1)) 29 | self.W.data.uniform_(-std, std) 30 | self.bias.data.uniform_(-std, std) 31 | 32 | 33 | 34 | def forward(self, structure, H, m=True): 35 | # ipdb.set_trace() 36 | W, b = self.W, self.bias 37 | HW = torch.mm(H, W) 38 | 39 | if self.reapproximate: 40 | n, X = H.shape[0], HW.cpu().detach().numpy() 41 | A = Laplacian(n, structure, X, m) 42 | else: A = structure 43 | 44 | A = A.to(self.device) 45 | A = Variable(A) 46 | 47 | AHW = SparseMM.apply(A, HW) 48 | return AHW + b 49 | 50 | 51 | 52 | def __repr__(self): 53 | return self.__class__.__name__ + ' (' \ 54 | + str(self.a) + ' -> ' \ 55 | + str(self.b) + ')' 56 | 57 | 58 | 59 | class SparseMM(torch.autograd.Function): 60 | """ 61 | Sparse x dense matrix multiplication with autograd support. 62 | Implementation by Soumith Chintala: 63 | https://discuss.pytorch.org/t/ 64 | does-pytorch-support-autograd-on-sparse-matrix/6156/7 65 | """ 66 | @staticmethod 67 | def forward(ctx, M1, M2): 68 | ctx.save_for_backward(M1, M2) 69 | return torch.mm(M1, M2) 70 | 71 | @staticmethod 72 | def backward(ctx, g): 73 | M1, M2 = ctx.saved_tensors 74 | g1 = g2 = None 75 | 76 | if ctx.needs_input_grad[0]: 77 | g1 = torch.mm(g, M2.t()) 78 | 79 | if ctx.needs_input_grad[1]: 80 | g2 = torch.mm(M1.t(), g) 81 | 82 | return g1, g2 83 | 84 | 85 | 86 | def Laplacian(V, E, X, m): 87 | """ 88 | approximates the E defined by the E Laplacian with/without mediators 89 | 90 | arguments: 91 | V: number of vertices 92 | E: dictionary of hyperedges (key: hyperedge, value: list/set of hypernodes) 93 | X: features on the vertices 94 | m: True gives Laplacian with mediators, while False gives without 95 | 96 | A: adjacency matrix of the graph approximation 97 | returns: 98 | updated data with 'graph' as a key and its value the approximated hypergraph 99 | """ 100 | 101 | edges, weights = [], {} 102 | rv = np.random.rand(X.shape[1]) 103 | 104 | for k in E.keys(): 105 | hyperedge = list(E[k]) 106 | 107 | p = np.dot(X[hyperedge], rv) #projection onto a random vector rv 108 | s, i = np.argmax(p), np.argmin(p) 109 | Se, Ie = hyperedge[s], hyperedge[i] 110 | 111 | # two stars with mediators 112 | c = 2*len(hyperedge) - 3 # normalisation constant 113 | if m: 114 | 115 | # connect the supremum (Se) with the infimum (Ie) 116 | edges.extend([[Se, Ie], [Ie, Se]]) 117 | 118 | if (Se,Ie) not in weights: 119 | weights[(Se,Ie)] = 0 120 | weights[(Se,Ie)] += float(1/c) 121 | 122 | if (Ie,Se) not in weights: 123 | weights[(Ie,Se)] = 0 124 | weights[(Ie,Se)] += float(1/c) 125 | 126 | # connect the supremum (Se) and the infimum (Ie) with each mediator 127 | for mediator in hyperedge: 128 | if mediator != Se and mediator != Ie: 129 | edges.extend([[Se,mediator], [Ie,mediator], [mediator,Se], [mediator,Ie]]) 130 | weights = update(Se, Ie, mediator, weights, c) 131 | else: 132 | edges.extend([[Se,Ie], [Ie,Se]]) 133 | e = len(hyperedge) 134 | 135 | if (Se,Ie) not in weights: 136 | weights[(Se,Ie)] = 0 137 | weights[(Se,Ie)] += float(1/e) 138 | 139 | if (Ie,Se) not in weights: 140 | weights[(Ie,Se)] = 0 141 | weights[(Ie,Se)] += float(1/e) 142 | 143 | return adjacency(edges, weights, V) 144 | 145 | 146 | 147 | def update(Se, Ie, mediator, weights, c): 148 | """ 149 | updates the weight on {Se,mediator} and {Ie,mediator} 150 | """ 151 | 152 | if (Se,mediator) not in weights: 153 | weights[(Se,mediator)] = 0 154 | weights[(Se,mediator)] += float(1/c) 155 | 156 | if (Ie,mediator) not in weights: 157 | weights[(Ie,mediator)] = 0 158 | weights[(Ie,mediator)] += float(1/c) 159 | 160 | if (mediator,Se) not in weights: 161 | weights[(mediator,Se)] = 0 162 | weights[(mediator,Se)] += float(1/c) 163 | 164 | if (mediator,Ie) not in weights: 165 | weights[(mediator,Ie)] = 0 166 | weights[(mediator,Ie)] += float(1/c) 167 | 168 | return weights 169 | 170 | 171 | 172 | def adjacency(edges, weights, n): 173 | """ 174 | computes an sparse adjacency matrix 175 | 176 | arguments: 177 | edges: list of pairs 178 | weights: dictionary of edge weights (key: tuple representing edge, value: weight on the edge) 179 | n: number of nodes 180 | 181 | returns: a scipy.sparse adjacency matrix with unit weight self loops for edges with the given weights 182 | """ 183 | 184 | dictionary = {tuple(item): index for index, item in enumerate(edges)} 185 | edges = [list(itm) for itm in dictionary.keys()] 186 | organised = [] 187 | 188 | for e in edges: 189 | i,j = e[0],e[1] 190 | w = weights[(i,j)] 191 | organised.append(w) 192 | 193 | edges, weights = np.array(edges), np.array(organised) 194 | adj = sp.coo_matrix((weights, (edges[:, 0], edges[:, 1])), shape=(n, n), dtype=np.float32) 195 | adj = adj + sp.eye(n) 196 | 197 | A = symnormalise(sp.csr_matrix(adj, dtype=np.float32)) 198 | A = ssm2tst(A) 199 | return A 200 | 201 | 202 | 203 | def symnormalise(M): 204 | """ 205 | symmetrically normalise sparse matrix 206 | 207 | arguments: 208 | M: scipy sparse matrix 209 | 210 | returns: 211 | D^{-1/2} M D^{-1/2} 212 | where D is the diagonal node-degree matrix 213 | """ 214 | 215 | d = np.array(M.sum(1)) 216 | 217 | dhi = np.power(d, -1/2).flatten() 218 | dhi[np.isinf(dhi)] = 0. 219 | DHI = sp.diags(dhi) # D half inverse i.e. D^{-1/2} 220 | 221 | return (DHI.dot(M)).dot(DHI) 222 | 223 | 224 | 225 | def ssm2tst(M): 226 | """ 227 | converts a scipy sparse matrix (ssm) to a torch sparse tensor (tst) 228 | 229 | arguments: 230 | M: scipy sparse matrix 231 | 232 | returns: 233 | a torch sparse tensor of M 234 | """ 235 | 236 | M = M.tocoo().astype(np.float32) 237 | 238 | indices = torch.from_numpy(np.vstack((M.row, M.col))).long() 239 | values = torch.from_numpy(M.data) 240 | shape = torch.Size(M.shape) 241 | 242 | return torch.sparse.FloatTensor(indices, values, shape) 243 | 244 | 245 | def normalise(M): 246 | """ 247 | row-normalise sparse matrix 248 | 249 | arguments: 250 | M: scipy sparse matrix 251 | 252 | returns: 253 | D^{-1} M 254 | where D is the diagonal node-degree matrix 255 | """ 256 | 257 | d = np.array(M.sum(1)) 258 | 259 | di = np.power(d, -1).flatten() 260 | di[np.isinf(di)] = 0. 261 | di = np.nan_to_num(di) 262 | DI = sp.diags(di) # D inverse i.e. D^{-1} 263 | 264 | return DI.dot(M) -------------------------------------------------------------------------------- /baselines_hypergnn/utils.py: -------------------------------------------------------------------------------- 1 | import torch, math, numpy as np, scipy.sparse as sp 2 | import torch.nn as nn, torch.nn.functional as F, torch.nn.init as init 3 | import ipdb 4 | 5 | from torch.autograd import Variable 6 | from torch.nn.modules.module import Module 7 | from torch.nn.parameter import Parameter 8 | 9 | 10 | 11 | class HyperGraphConvolution(Module): 12 | """ 13 | Simple GCN layer, similar to https://arxiv.org/abs/1609.02907 14 | """ 15 | 16 | def __init__(self, a, b, reapproximate=True, cuda=None): 17 | super(HyperGraphConvolution, self).__init__() 18 | self.a, self.b = a, b 19 | self.reapproximate, self.device = reapproximate, torch.device('cuda:'+str(cuda)) 20 | 21 | self.W = Parameter(torch.FloatTensor(a, b)) 22 | self.bias = Parameter(torch.FloatTensor(b)) 23 | self.reset_parameters() 24 | 25 | 26 | 27 | def reset_parameters(self): 28 | std = 1. / math.sqrt(self.W.size(1)) 29 | self.W.data.uniform_(-std, std) 30 | self.bias.data.uniform_(-std, std) 31 | 32 | 33 | 34 | def forward(self, structure, H, m=True): 35 | # ipdb.set_trace() 36 | W, b = self.W, self.bias 37 | HW = torch.mm(H, W) 38 | 39 | if self.reapproximate: 40 | n, X = H.shape[0], HW.cpu().detach().numpy() 41 | A = Laplacian(n, structure, X, m) 42 | else: A = structure 43 | 44 | A = A.to(self.device) 45 | A = Variable(A) 46 | 47 | AHW = SparseMM.apply(A, HW) 48 | return AHW + b 49 | 50 | 51 | 52 | def __repr__(self): 53 | return self.__class__.__name__ + ' (' \ 54 | + str(self.a) + ' -> ' \ 55 | + str(self.b) + ')' 56 | 57 | 58 | 59 | class SparseMM(torch.autograd.Function): 60 | """ 61 | Sparse x dense matrix multiplication with autograd support. 62 | Implementation by Soumith Chintala: 63 | https://discuss.pytorch.org/t/ 64 | does-pytorch-support-autograd-on-sparse-matrix/6156/7 65 | """ 66 | @staticmethod 67 | def forward(ctx, M1, M2): 68 | ctx.save_for_backward(M1, M2) 69 | return torch.mm(M1, M2) 70 | 71 | @staticmethod 72 | def backward(ctx, g): 73 | M1, M2 = ctx.saved_tensors 74 | g1 = g2 = None 75 | 76 | if ctx.needs_input_grad[0]: 77 | g1 = torch.mm(g, M2.t()) 78 | 79 | if ctx.needs_input_grad[1]: 80 | g2 = torch.mm(M1.t(), g) 81 | 82 | return g1, g2 83 | 84 | 85 | 86 | def Laplacian(V, E, X, m): 87 | """ 88 | approximates the E defined by the E Laplacian with/without mediators 89 | 90 | arguments: 91 | V: number of vertices 92 | E: dictionary of hyperedges (key: hyperedge, value: list/set of hypernodes) 93 | X: features on the vertices 94 | m: True gives Laplacian with mediators, while False gives without 95 | 96 | A: adjacency matrix of the graph approximation 97 | returns: 98 | updated data with 'graph' as a key and its value the approximated hypergraph 99 | """ 100 | 101 | edges, weights = [], {} 102 | rv = np.random.rand(X.shape[1]) 103 | 104 | for k in E.keys(): 105 | hyperedge = list(E[k]) 106 | 107 | p = np.dot(X[hyperedge], rv) #projection onto a random vector rv 108 | s, i = np.argmax(p), np.argmin(p) 109 | Se, Ie = hyperedge[s], hyperedge[i] 110 | 111 | # two stars with mediators 112 | c = 2*len(hyperedge) - 3 # normalisation constant 113 | if m: 114 | 115 | # connect the supremum (Se) with the infimum (Ie) 116 | edges.extend([[Se, Ie], [Ie, Se]]) 117 | 118 | if (Se,Ie) not in weights: 119 | weights[(Se,Ie)] = 0 120 | weights[(Se,Ie)] += float(1/c) 121 | 122 | if (Ie,Se) not in weights: 123 | weights[(Ie,Se)] = 0 124 | weights[(Ie,Se)] += float(1/c) 125 | 126 | # connect the supremum (Se) and the infimum (Ie) with each mediator 127 | for mediator in hyperedge: 128 | if mediator != Se and mediator != Ie: 129 | edges.extend([[Se,mediator], [Ie,mediator], [mediator,Se], [mediator,Ie]]) 130 | weights = update(Se, Ie, mediator, weights, c) 131 | else: 132 | edges.extend([[Se,Ie], [Ie,Se]]) 133 | e = len(hyperedge) 134 | 135 | if (Se,Ie) not in weights: 136 | weights[(Se,Ie)] = 0 137 | weights[(Se,Ie)] += float(1/e) 138 | 139 | if (Ie,Se) not in weights: 140 | weights[(Ie,Se)] = 0 141 | weights[(Ie,Se)] += float(1/e) 142 | 143 | return adjacency(edges, weights, V) 144 | 145 | 146 | 147 | def update(Se, Ie, mediator, weights, c): 148 | """ 149 | updates the weight on {Se,mediator} and {Ie,mediator} 150 | """ 151 | 152 | if (Se,mediator) not in weights: 153 | weights[(Se,mediator)] = 0 154 | weights[(Se,mediator)] += float(1/c) 155 | 156 | if (Ie,mediator) not in weights: 157 | weights[(Ie,mediator)] = 0 158 | weights[(Ie,mediator)] += float(1/c) 159 | 160 | if (mediator,Se) not in weights: 161 | weights[(mediator,Se)] = 0 162 | weights[(mediator,Se)] += float(1/c) 163 | 164 | if (mediator,Ie) not in weights: 165 | weights[(mediator,Ie)] = 0 166 | weights[(mediator,Ie)] += float(1/c) 167 | 168 | return weights 169 | 170 | 171 | 172 | def adjacency(edges, weights, n): 173 | """ 174 | computes an sparse adjacency matrix 175 | 176 | arguments: 177 | edges: list of pairs 178 | weights: dictionary of edge weights (key: tuple representing edge, value: weight on the edge) 179 | n: number of nodes 180 | 181 | returns: a scipy.sparse adjacency matrix with unit weight self loops for edges with the given weights 182 | """ 183 | 184 | dictionary = {tuple(item): index for index, item in enumerate(edges)} 185 | edges = [list(itm) for itm in dictionary.keys()] 186 | organised = [] 187 | 188 | for e in edges: 189 | i,j = e[0],e[1] 190 | w = weights[(i,j)] 191 | organised.append(w) 192 | 193 | edges, weights = np.array(edges), np.array(organised) 194 | adj = sp.coo_matrix((weights, (edges[:, 0], edges[:, 1])), shape=(n, n), dtype=np.float32) 195 | adj = adj + sp.eye(n) 196 | 197 | A = symnormalise(sp.csr_matrix(adj, dtype=np.float32)) 198 | A = ssm2tst(A) 199 | return A 200 | 201 | 202 | 203 | def symnormalise(M): 204 | """ 205 | symmetrically normalise sparse matrix 206 | 207 | arguments: 208 | M: scipy sparse matrix 209 | 210 | returns: 211 | D^{-1/2} M D^{-1/2} 212 | where D is the diagonal node-degree matrix 213 | """ 214 | 215 | d = np.array(M.sum(1)) 216 | 217 | dhi = np.power(d, -1/2).flatten() 218 | dhi[np.isinf(dhi)] = 0. 219 | DHI = sp.diags(dhi) # D half inverse i.e. D^{-1/2} 220 | 221 | return (DHI.dot(M)).dot(DHI) 222 | 223 | 224 | 225 | def ssm2tst(M): 226 | """ 227 | converts a scipy sparse matrix (ssm) to a torch sparse tensor (tst) 228 | 229 | arguments: 230 | M: scipy sparse matrix 231 | 232 | returns: 233 | a torch sparse tensor of M 234 | """ 235 | 236 | M = M.tocoo().astype(np.float32) 237 | 238 | indices = torch.from_numpy(np.vstack((M.row, M.col))).long() 239 | values = torch.from_numpy(M.data) 240 | shape = torch.Size(M.shape) 241 | 242 | return torch.sparse.FloatTensor(indices, values, shape) 243 | 244 | 245 | def normalise(M): 246 | """ 247 | row-normalise sparse matrix 248 | 249 | arguments: 250 | M: scipy sparse matrix 251 | 252 | returns: 253 | D^{-1} M 254 | where D is the diagonal node-degree matrix 255 | """ 256 | 257 | d = np.array(M.sum(1)) 258 | 259 | di = np.power(d, -1).flatten() 260 | di[np.isinf(di)] = 0. 261 | di = np.nan_to_num(di) 262 | DI = sp.diags(di) # D inverse i.e. D^{-1} 263 | 264 | return DI.dot(M) -------------------------------------------------------------------------------- /ours/convert_datasets_to_pygDataset.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # coding: utf-8 3 | 4 | # In[45]: 5 | 6 | 7 | import torch 8 | import pickle 9 | import os 10 | #import ipdb 11 | 12 | import os.path as osp 13 | 14 | from torch_geometric.data import Data 15 | from torch_geometric.data import InMemoryDataset 16 | 17 | from load_other_datasets import * 18 | 19 | 20 | def save_data_to_pickle(data, p2root = '../data/', file_name = None): 21 | ''' 22 | if file name not specified, use time stamp. 23 | ''' 24 | # now = datetime.now() 25 | # surfix = now.strftime('%b_%d_%Y-%H:%M') 26 | surfix = 'star_expansion_dataset' 27 | if file_name is None: 28 | tmp_data_name = '_'.join(['Hypergraph', surfix]) 29 | else: 30 | tmp_data_name = file_name 31 | p2he_StarExpan = osp.join(p2root, tmp_data_name) 32 | if not osp.isdir(p2root): 33 | os.makedirs(p2root) 34 | with open(p2he_StarExpan, 'bw') as f: 35 | pickle.dump(data, f) 36 | return p2he_StarExpan 37 | 38 | 39 | class dataset_Hypergraph(InMemoryDataset): 40 | def __init__(self, root = '../data/pyg_data/hypergraph_dataset_updated/', name = None, 41 | p2raw = None, 42 | train_percent = 0.01, 43 | feature_noise = None, 44 | transform=None, pre_transform=None): 45 | 46 | existing_dataset = ['20newsW100', 'ModelNet40', 'zoo', 47 | 'NTU2012', 'Mushroom', 48 | 'coauthor_cora', 'coauthor_dblp', 49 | 'yelp', 'amazon-reviews', 'walmart-trips', 'house-committees', 50 | 'walmart-trips-100', 'house-committees-100', 51 | 'cora', 'citeseer', 'pubmed'] 52 | if name not in existing_dataset: 53 | raise ValueError(f'name of hypergraph dataset must be one of: {existing_dataset}') 54 | else: 55 | self.name = name 56 | 57 | self.feature_noise = feature_noise 58 | 59 | self._train_percent = train_percent 60 | 61 | 62 | if (p2raw is not None) and osp.isdir(p2raw): 63 | self.p2raw = p2raw 64 | elif p2raw is None: 65 | self.p2raw = None 66 | elif not osp.isdir(p2raw): 67 | raise ValueError(f'path to raw hypergraph dataset "{p2raw}" does not exist!') 68 | 69 | if not osp.isdir(root): 70 | os.makedirs(root) 71 | 72 | self.root = root 73 | self.myraw_dir = osp.join(root, self.name, 'raw') 74 | self.myprocessed_dir = osp.join(root, self.name) 75 | 76 | super(dataset_Hypergraph, self).__init__(osp.join(root, name), transform, pre_transform) 77 | 78 | 79 | 80 | self.data, self.slices = torch.load(self.processed_paths[0]) 81 | self.train_percent = self.data.train_percent.item() 82 | 83 | # @property 84 | # def raw_dir(self): 85 | # return osp.join(self.root, self.name, 'raw') 86 | 87 | # @property 88 | # def processed_dir(self): 89 | # return osp.join(self.root, self.name, 'processed') 90 | 91 | 92 | @property 93 | def raw_file_names(self): 94 | if self.feature_noise is not None: 95 | file_names = [f'{self.name}_noise_{self.feature_noise}'] 96 | else: 97 | file_names = [self.name] 98 | return file_names 99 | 100 | @property 101 | def processed_file_names(self): 102 | if self.feature_noise is not None: 103 | file_names = [f'data_noise_{self.feature_noise}.pt'] 104 | else: 105 | file_names = ['data.pt'] 106 | return file_names 107 | 108 | @property 109 | def num_features(self): 110 | return self.data.num_node_features 111 | 112 | 113 | def download(self): 114 | for name in self.raw_file_names: 115 | p2f = osp.join(self.myraw_dir, name) 116 | if not osp.isfile(p2f): 117 | # file not exist, so we create it and save it there. 118 | print(p2f) 119 | print(self.p2raw) 120 | print(self.name) 121 | 122 | if self.name in ['cora', 'citeseer', 'pubmed']: 123 | tmp_data = load_citation_dataset(path = self.p2raw, 124 | dataset = self.name, 125 | train_percent = self._train_percent) 126 | 127 | elif self.name in ['coauthor_cora', 'coauthor_dblp']: 128 | assert 'coauthorship' in self.p2raw 129 | dataset_name = self.name.split('_')[-1] 130 | tmp_data = load_citation_dataset(path = self.p2raw, 131 | dataset = dataset_name, 132 | train_percent = self._train_percent) 133 | 134 | elif self.name in ['amazon-reviews', 'walmart-trips', 'house-committees']: 135 | if self.feature_noise is None: 136 | raise ValueError(f'for cornell datasets, feature noise cannot be {self.feature_noise}') 137 | tmp_data = load_cornell_dataset(path = self.p2raw, 138 | dataset = self.name, 139 | feature_noise = self.feature_noise, 140 | train_percent = self._train_percent) 141 | elif self.name in ['walmart-trips-100', 'house-committees-100']: 142 | if self.feature_noise is None: 143 | raise ValueError(f'for cornell datasets, feature noise cannot be {self.feature_noise}') 144 | feature_dim = int(self.name.split('-')[-1]) 145 | tmp_name = '-'.join(self.name.split('-')[:-1]) 146 | tmp_data = load_cornell_dataset(path = self.p2raw, 147 | dataset = tmp_name, 148 | feature_dim = feature_dim, 149 | feature_noise = self.feature_noise, 150 | train_percent = self._train_percent) 151 | 152 | 153 | elif self.name == 'yelp': 154 | tmp_data = load_yelp_dataset(path = self.p2raw, 155 | dataset = self.name, 156 | train_percent = self._train_percent) 157 | 158 | else: 159 | tmp_data = load_LE_dataset(path = self.p2raw, 160 | dataset = self.name, 161 | train_percent= self._train_percent) 162 | 163 | _ = save_data_to_pickle(tmp_data, 164 | p2root = self.myraw_dir, 165 | file_name = self.raw_file_names[0]) 166 | else: 167 | # file exists already. Do nothing. 168 | pass 169 | 170 | def process(self): 171 | p2f = osp.join(self.myraw_dir, self.raw_file_names[0]) 172 | with open(p2f, 'rb') as f: 173 | data = pickle.load(f) 174 | data = data if self.pre_transform is None else self.pre_transform(data) 175 | torch.save(self.collate([data]), self.processed_paths[0]) 176 | 177 | def __repr__(self): 178 | return '{}()'.format(self.name) 179 | 180 | 181 | if __name__ == '__main__': 182 | 183 | p2root = '../data/pyg_data/hypergraph_dataset_updated/' 184 | p2raw = '../data/AllSet_all_raw_data/' 185 | # dd = dataset_Hypergraph(root = p2root, name = 'walmart-trips-100', feature_noise = 0, 186 | # p2raw = p2raw) 187 | 188 | for f in ['walmart-trips-100', ]:# 'house-committees', 'amazon-reviews']: 189 | for feature_noise in [0.1, 1]: 190 | dd = dataset_Hypergraph(root = p2root, 191 | name = f, 192 | feature_noise = feature_noise, 193 | p2raw = p2raw) 194 | 195 | assert dd.data.num_nodes in dd.data.edge_index[0] 196 | print(dd, dd.data) 197 | 198 | 199 | p2root = '../data/pyg_data/hypergraph_dataset_updated/' 200 | p2raw = '../data/AllSet_all_raw_data/coauthorship/' 201 | for f in ['coauthor_cora', ]: #'coauthor_dblp']: 202 | dd = dataset_Hypergraph(root = p2root, 203 | name = f, 204 | p2raw = p2raw) 205 | assert dd.data.num_nodes in dd.data.edge_index[0] 206 | print(dd, dd.data) 207 | 208 | p2root = '../data/pyg_data/hypergraph_dataset_updated/' 209 | p2raw = '../data/AllSet_all_raw_data/cocitation/' 210 | for f in ['cora', 'citeseer']: 211 | dd = dataset_Hypergraph(root = p2root, 212 | name = f, 213 | p2raw = p2raw) 214 | assert dd.data.num_nodes in dd.data.edge_index[0] 215 | print(dd, dd.data) 216 | -------------------------------------------------------------------------------- /baselines_hypergnn/convert_datasets_to_pygDataset.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # coding: utf-8 3 | 4 | # In[45]: 5 | 6 | 7 | import torch 8 | import pickle 9 | import os 10 | import ipdb 11 | 12 | import os.path as osp 13 | 14 | from torch_geometric.data import Data 15 | from torch_geometric.data import InMemoryDataset 16 | 17 | from load_other_datasets import * 18 | 19 | 20 | def save_data_to_pickle(data, p2root = '../data/', file_name = None): 21 | ''' 22 | if file name not specified, use time stamp. 23 | ''' 24 | # now = datetime.now() 25 | # surfix = now.strftime('%b_%d_%Y-%H:%M') 26 | surfix = 'star_expansion_dataset' 27 | if file_name is None: 28 | tmp_data_name = '_'.join(['Hypergraph', surfix]) 29 | else: 30 | tmp_data_name = file_name 31 | p2he_StarExpan = osp.join(p2root, tmp_data_name) 32 | if not osp.isdir(p2root): 33 | os.makedirs(p2root) 34 | with open(p2he_StarExpan, 'bw') as f: 35 | pickle.dump(data, f) 36 | return p2he_StarExpan 37 | 38 | 39 | class dataset_Hypergraph(InMemoryDataset): 40 | def __init__(self, root = '../data/pyg_data/hypergraph_dataset_updated/', name = None, 41 | p2raw = None, 42 | train_percent = 0.01, 43 | feature_noise = None, 44 | transform=None, pre_transform=None): 45 | 46 | existing_dataset = ['20newsW100', 'ModelNet40', 'zoo', 47 | 'NTU2012', 'Mushroom', 48 | 'coauthor_cora', 'coauthor_dblp', 49 | 'yelp', 'amazon-reviews', 'walmart-trips', 'house-committees', 50 | 'walmart-trips-100', 'house-committees-100', 51 | 'cora', 'citeseer', 'pubmed'] 52 | if name not in existing_dataset: 53 | raise ValueError(f'name of hypergraph dataset must be one of: {existing_dataset}') 54 | else: 55 | self.name = name 56 | 57 | self.feature_noise = feature_noise 58 | 59 | self._train_percent = train_percent 60 | 61 | 62 | if (p2raw is not None) and osp.isdir(p2raw): 63 | self.p2raw = p2raw 64 | elif p2raw is None: 65 | self.p2raw = None 66 | elif not osp.isdir(p2raw): 67 | raise ValueError(f'path to raw hypergraph dataset "{p2raw}" does not exist!') 68 | 69 | if not osp.isdir(root): 70 | os.makedirs(root) 71 | 72 | self.root = root 73 | self.myraw_dir = osp.join(root, self.name, 'raw') 74 | self.myprocessed_dir = osp.join(root, self.name) 75 | 76 | super(dataset_Hypergraph, self).__init__(osp.join(root, name), transform, pre_transform) 77 | 78 | 79 | 80 | self.data, self.slices = torch.load(self.processed_paths[0]) 81 | self.train_percent = self.data.train_percent.item() 82 | 83 | # @property 84 | # def raw_dir(self): 85 | # return osp.join(self.root, self.name, 'raw') 86 | 87 | # @property 88 | # def processed_dir(self): 89 | # return osp.join(self.root, self.name, 'processed') 90 | 91 | 92 | @property 93 | def raw_file_names(self): 94 | if self.feature_noise is not None: 95 | file_names = [f'{self.name}_noise_{self.feature_noise}'] 96 | else: 97 | file_names = [self.name] 98 | return file_names 99 | 100 | @property 101 | def processed_file_names(self): 102 | if self.feature_noise is not None: 103 | file_names = [f'data_noise_{self.feature_noise}.pt'] 104 | else: 105 | file_names = ['data.pt'] 106 | return file_names 107 | 108 | @property 109 | def num_features(self): 110 | return self.data.num_node_features 111 | 112 | 113 | def download(self): 114 | for name in self.raw_file_names: 115 | p2f = osp.join(self.myraw_dir, name) 116 | if not osp.isfile(p2f): 117 | # file not exist, so we create it and save it there. 118 | print(p2f) 119 | print(self.p2raw) 120 | print(self.name) 121 | 122 | if self.name in ['cora', 'citeseer', 'pubmed']: 123 | tmp_data = load_citation_dataset(path = self.p2raw, 124 | dataset = self.name, 125 | train_percent = self._train_percent) 126 | 127 | elif self.name in ['coauthor_cora', 'coauthor_dblp']: 128 | assert 'coauthorship' in self.p2raw 129 | dataset_name = self.name.split('_')[-1] 130 | tmp_data = load_citation_dataset(path = self.p2raw, 131 | dataset = dataset_name, 132 | train_percent = self._train_percent) 133 | 134 | elif self.name in ['amazon-reviews', 'walmart-trips', 'house-committees']: 135 | if self.feature_noise is None: 136 | raise ValueError(f'for cornell datasets, feature noise cannot be {self.feature_noise}') 137 | tmp_data = load_cornell_dataset(path = self.p2raw, 138 | dataset = self.name, 139 | feature_noise = self.feature_noise, 140 | train_percent = self._train_percent) 141 | elif self.name in ['walmart-trips-100', 'house-committees-100']: 142 | if self.feature_noise is None: 143 | raise ValueError(f'for cornell datasets, feature noise cannot be {self.feature_noise}') 144 | feature_dim = int(self.name.split('-')[-1]) 145 | tmp_name = '-'.join(self.name.split('-')[:-1]) 146 | tmp_data = load_cornell_dataset(path = self.p2raw, 147 | dataset = tmp_name, 148 | feature_dim = feature_dim, 149 | feature_noise = self.feature_noise, 150 | train_percent = self._train_percent) 151 | 152 | 153 | elif self.name == 'yelp': 154 | tmp_data = load_yelp_dataset(path = self.p2raw, 155 | dataset = self.name, 156 | train_percent = self._train_percent) 157 | 158 | else: 159 | tmp_data = load_LE_dataset(path = self.p2raw, 160 | dataset = self.name, 161 | train_percent= self._train_percent) 162 | 163 | _ = save_data_to_pickle(tmp_data, 164 | p2root = self.myraw_dir, 165 | file_name = self.raw_file_names[0]) 166 | else: 167 | # file exists already. Do nothing. 168 | pass 169 | 170 | def process(self): 171 | p2f = osp.join(self.myraw_dir, self.raw_file_names[0]) 172 | with open(p2f, 'rb') as f: 173 | data = pickle.load(f) 174 | data = data if self.pre_transform is None else self.pre_transform(data) 175 | torch.save(self.collate([data]), self.processed_paths[0]) 176 | 177 | def __repr__(self): 178 | return '{}()'.format(self.name) 179 | 180 | 181 | if __name__ == '__main__': 182 | 183 | p2root = '../data/pyg_data/hypergraph_dataset_updated/' 184 | p2raw = '../data/AllSet_all_raw_data/' 185 | # dd = dataset_Hypergraph(root = p2root, name = 'walmart-trips-100', feature_noise = 0, 186 | # p2raw = p2raw) 187 | 188 | for f in ['walmart-trips-100', ]:# 'house-committees', 'amazon-reviews']: 189 | for feature_noise in [0.1, 1]: 190 | dd = dataset_Hypergraph(root = p2root, 191 | name = f, 192 | feature_noise = feature_noise, 193 | p2raw = p2raw) 194 | 195 | assert dd.data.num_nodes in dd.data.edge_index[0] 196 | print(dd, dd.data) 197 | 198 | 199 | p2root = '../data/pyg_data/hypergraph_dataset_updated/' 200 | p2raw = '../data/AllSet_all_raw_data/coauthorship/' 201 | for f in ['coauthor_cora', ]: #'coauthor_dblp']: 202 | dd = dataset_Hypergraph(root = p2root, 203 | name = f, 204 | p2raw = p2raw) 205 | assert dd.data.num_nodes in dd.data.edge_index[0] 206 | print(dd, dd.data) 207 | 208 | p2root = '../data/pyg_data/hypergraph_dataset_updated/' 209 | p2raw = '../data/AllSet_all_raw_data/cocitation/' 210 | for f in ['cora', 'citeseer']: 211 | dd = dataset_Hypergraph(root = p2root, 212 | name = f, 213 | p2raw = p2raw) 214 | assert dd.data.num_nodes in dd.data.edge_index[0] 215 | print(dd, dd.data) 216 | -------------------------------------------------------------------------------- /ours/load_other_datasets.py: -------------------------------------------------------------------------------- 1 | #! /usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | # vim:fenc=utf-8 4 | # 5 | # Copyright © 2021 6 | # 7 | # Distributed under terms of the MIT license. 8 | 9 | """ 10 | This script contains functions for loading the following datasets: 11 | co-authorship: (dblp, cora) 12 | walmart-trips (From cornell) 13 | Amazon-reviews 14 | U.S. House committee 15 | """ 16 | 17 | import torch 18 | import os 19 | import pickle 20 | #import ipdb 21 | 22 | import os.path as osp 23 | import numpy as np 24 | import pandas as pd 25 | import scipy.sparse as sp 26 | 27 | from torch_geometric.data import Data 28 | from torch_sparse import coalesce 29 | # from randomperm_code import random_planetoid_splits 30 | from sklearn.feature_extraction.text import CountVectorizer 31 | 32 | def load_LE_dataset(path=None, dataset="ModelNet40", train_percent = 0.025): 33 | # load edges, features, and labels. 34 | print('Loading {} dataset...'.format(dataset)) 35 | 36 | file_name = f'{dataset}.content' 37 | p2idx_features_labels = osp.join(path, dataset, file_name) 38 | idx_features_labels = np.genfromtxt(p2idx_features_labels, 39 | dtype=np.dtype(str)) 40 | # features = np.array(idx_features_labels[:, 1:-1]) 41 | features = sp.csr_matrix(idx_features_labels[:, 1:-1], dtype=np.float32) 42 | # labels = encode_onehot(idx_features_labels[:, -1]) 43 | labels = torch.LongTensor(idx_features_labels[:, -1].astype(float)) 44 | 45 | 46 | print ('load features') 47 | 48 | # build graph 49 | idx = np.array(idx_features_labels[:, 0], dtype=np.int32) 50 | idx_map = {j: i for i, j in enumerate(idx)} 51 | 52 | file_name = f'{dataset}.edges' 53 | p2edges_unordered = osp.join(path, dataset, file_name) 54 | edges_unordered = np.genfromtxt(p2edges_unordered, 55 | dtype=np.int32) 56 | 57 | 58 | edges = np.array(list(map(idx_map.get, edges_unordered.flatten())), 59 | dtype=np.int32).reshape(edges_unordered.shape) 60 | 61 | print ('load edges') 62 | 63 | 64 | projected_features = torch.FloatTensor(np.array(features.todense())) 65 | 66 | 67 | # From adjacency matrix to edge_list 68 | edge_index = edges.T 69 | # ipdb.set_trace() 70 | assert edge_index[0].max() == edge_index[1].min() - 1 71 | 72 | # check if values in edge_index is consecutive. i.e. no missing value for node_id/he_id. 73 | assert len(np.unique(edge_index)) == edge_index.max() + 1 74 | 75 | num_nodes = edge_index[0].max() + 1 76 | num_he = edge_index[1].max() - num_nodes + 1 77 | 78 | edge_index = np.hstack((edge_index, edge_index[::-1, :])) 79 | # ipdb.set_trace() 80 | 81 | # build torch data class 82 | data = Data( 83 | # x = projected_features, 84 | x = torch.FloatTensor(np.array(features[:num_nodes].todense())), 85 | edge_index = torch.LongTensor(edge_index), 86 | y = labels[:num_nodes]) 87 | 88 | 89 | # ipdb.set_trace() 90 | # data.coalesce() 91 | # There might be errors if edge_index.max() != num_nodes. 92 | # used user function to override the default function. 93 | # the following will also sort the edge_index and remove duplicates. 94 | total_num_node_id_he_id = len(np.unique(edge_index)) 95 | data.edge_index, data.edge_attr = coalesce(data.edge_index, 96 | None, 97 | total_num_node_id_he_id, 98 | total_num_node_id_he_id) 99 | 100 | 101 | 102 | 103 | # ipdb.set_trace() 104 | 105 | # # generate train, test, val mask. 106 | n_x = num_nodes 107 | # n_x = n_expanded 108 | num_class = len(np.unique(labels[:num_nodes].numpy())) 109 | val_lb = int(n_x * train_percent) 110 | percls_trn = int(round(train_percent * n_x / num_class)) 111 | # data = random_planetoid_splits(data, num_class, percls_trn, val_lb) 112 | data.n_x = n_x 113 | # add parameters to attribute 114 | 115 | 116 | data.train_percent = train_percent 117 | data.num_hyperedges = num_he 118 | 119 | return data 120 | 121 | def load_citation_dataset(path='../hyperGCN/data/', dataset = 'cora', train_percent = 0.025): 122 | ''' 123 | this will read the citation dataset from HyperGCN, and convert it edge_list to 124 | [[ -V- | -E- ] 125 | [ -E- | -V- ]] 126 | ''' 127 | print(f'Loading hypergraph dataset from hyperGCN: {dataset}') 128 | 129 | # first load node features: 130 | with open(osp.join(path, dataset, 'features.pickle'), 'rb') as f: 131 | features = pickle.load(f) 132 | features = features.todense() 133 | 134 | # then load node labels: 135 | with open(osp.join(path, dataset, 'labels.pickle'), 'rb') as f: 136 | labels = pickle.load(f) 137 | 138 | num_nodes, feature_dim = features.shape 139 | assert num_nodes == len(labels) 140 | print(f'number of nodes:{num_nodes}, feature dimension: {feature_dim}') 141 | 142 | features = torch.FloatTensor(features) 143 | labels = torch.LongTensor(labels) 144 | 145 | # The last, load hypergraph. 146 | with open(osp.join(path, dataset, 'hypergraph.pickle'), 'rb') as f: 147 | # hypergraph in hyperGCN is in the form of a dictionary. 148 | # { hyperedge: [list of nodes in the he], ...} 149 | hypergraph = pickle.load(f) 150 | 151 | print(f'number of hyperedges: {len(hypergraph)}') 152 | 153 | edge_idx = num_nodes 154 | node_list = [] 155 | edge_list = [] 156 | for he in hypergraph.keys(): 157 | cur_he = hypergraph[he] 158 | cur_size = len(cur_he) 159 | 160 | node_list += list(cur_he) 161 | edge_list += [edge_idx] * cur_size 162 | 163 | edge_idx += 1 164 | 165 | edge_index = np.array([ node_list + edge_list, 166 | edge_list + node_list], dtype = np.int) 167 | edge_index = torch.LongTensor(edge_index) 168 | 169 | data = Data(x = features, 170 | edge_index = edge_index, 171 | y = labels) 172 | 173 | # data.coalesce() 174 | # There might be errors if edge_index.max() != num_nodes. 175 | # used user function to override the default function. 176 | # the following will also sort the edge_index and remove duplicates. 177 | total_num_node_id_he_id = edge_index.max() + 1 178 | data.edge_index, data.edge_attr = coalesce(data.edge_index, 179 | None, 180 | total_num_node_id_he_id, 181 | total_num_node_id_he_id) 182 | 183 | 184 | n_x = num_nodes 185 | # n_x = n_expanded 186 | num_class = len(np.unique(labels.numpy())) 187 | val_lb = int(n_x * train_percent) 188 | percls_trn = int(round(train_percent * n_x / num_class)) 189 | # data = random_planetoid_splits(data, num_class, percls_trn, val_lb) 190 | data.n_x = n_x 191 | # add parameters to attribute 192 | 193 | data.train_percent = train_percent 194 | data.num_hyperedges = len(hypergraph) 195 | 196 | return data 197 | 198 | def load_yelp_dataset(path='../data/raw_data/yelp_raw_datasets/', dataset = 'yelp', 199 | name_dictionary_size = 1000, 200 | train_percent = 0.025): 201 | ''' 202 | this will read the yelp dataset from source files, and convert it edge_list to 203 | [[ -V- | -E- ] 204 | [ -E- | -V- ]] 205 | 206 | each node is a restaurant, a hyperedge represent a set of restaurants one user had been to. 207 | 208 | node features: 209 | - latitude, longitude 210 | - state, in one-hot coding. 211 | - city, in one-hot coding. 212 | - name, in bag-of-words 213 | 214 | node label: 215 | - average stars from 2-10, converted from original stars which is binned in x.5, min stars = 1 216 | ''' 217 | print(f'Loading hypergraph dataset from {dataset}') 218 | 219 | # first load node features: 220 | # load longtitude and latitude of restaurant. 221 | latlong = pd.read_csv(osp.join(path, 'yelp_restaurant_latlong.csv')).values 222 | 223 | # city - zipcode - state integer indicator dataframe. 224 | loc = pd.read_csv(osp.join(path, 'yelp_restaurant_locations.csv')) 225 | state_int = loc.state_int.values 226 | city_int = loc.city_int.values 227 | 228 | num_nodes = loc.shape[0] 229 | state_1hot = np.zeros((num_nodes, state_int.max())) 230 | state_1hot[np.arange(num_nodes), state_int - 1] = 1 231 | 232 | city_1hot = np.zeros((num_nodes, city_int.max())) 233 | city_1hot[np.arange(num_nodes), city_int - 1] = 1 234 | 235 | # convert restaurant name into bag-of-words feature. 236 | vectorizer = CountVectorizer(max_features = name_dictionary_size, stop_words = 'english', strip_accents = 'ascii') 237 | res_name = pd.read_csv(osp.join(path, 'yelp_restaurant_name.csv')).values.flatten() 238 | name_bow = vectorizer.fit_transform(res_name).todense() 239 | 240 | features = np.hstack([latlong, state_1hot, city_1hot, name_bow]) 241 | 242 | # then load node labels: 243 | df_labels = pd.read_csv(osp.join(path, 'yelp_restaurant_business_stars.csv')) 244 | labels = df_labels.values.flatten() 245 | 246 | num_nodes, feature_dim = features.shape 247 | assert num_nodes == len(labels) 248 | print(f'number of nodes:{num_nodes}, feature dimension: {feature_dim}') 249 | 250 | features = torch.FloatTensor(features) 251 | labels = torch.LongTensor(labels) 252 | 253 | # The last, load hypergraph. 254 | # Yelp restaurant review hypergraph is store in a incidence matrix. 255 | H = pd.read_csv(osp.join(path, 'yelp_restaurant_incidence_H.csv')) 256 | node_list = H.node.values - 1 257 | edge_list = H.he.values - 1 + num_nodes 258 | 259 | edge_index = np.vstack([node_list, edge_list]) 260 | edge_index = np.hstack([edge_index, edge_index[::-1, :]]) 261 | 262 | edge_index = torch.LongTensor(edge_index) 263 | 264 | data = Data(x = features, 265 | edge_index = edge_index, 266 | y = labels) 267 | 268 | # data.coalesce() 269 | # There might be errors if edge_index.max() != num_nodes. 270 | # used user function to override the default function. 271 | # the following will also sort the edge_index and remove duplicates. 272 | total_num_node_id_he_id = edge_index.max() + 1 273 | data.edge_index, data.edge_attr = coalesce(data.edge_index, 274 | None, 275 | total_num_node_id_he_id, 276 | total_num_node_id_he_id) 277 | 278 | 279 | n_x = num_nodes 280 | # n_x = n_expanded 281 | num_class = len(np.unique(labels.numpy())) 282 | val_lb = int(n_x * train_percent) 283 | percls_trn = int(round(train_percent * n_x / num_class)) 284 | # data = random_planetoid_splits(data, num_class, percls_trn, val_lb) 285 | data.n_x = n_x 286 | # add parameters to attribute 287 | 288 | data.train_percent = train_percent 289 | data.num_hyperedges = H.he.values.max() 290 | 291 | return data 292 | 293 | def load_cornell_dataset(path='../data/raw_data/', dataset = 'amazon', 294 | feature_noise = 0.1, 295 | feature_dim = None, 296 | train_percent = 0.025): 297 | ''' 298 | this will read the yelp dataset from source files, and convert it edge_list to 299 | [[ -V- | -E- ] 300 | [ -E- | -V- ]] 301 | 302 | each node is a restaurant, a hyperedge represent a set of restaurants one user had been to. 303 | 304 | node features: 305 | - add gaussian noise with sigma = nosie, mean = one hot coded label. 306 | 307 | node label: 308 | - average stars from 2-10, converted from original stars which is binned in x.5, min stars = 1 309 | ''' 310 | print(f'Loading hypergraph dataset from cornell: {dataset}') 311 | 312 | # first load node labels 313 | df_labels = pd.read_csv(osp.join(path, dataset, f'node-labels-{dataset}.txt'), names = ['node_label']) 314 | num_nodes = df_labels.shape[0] 315 | labels = df_labels.values.flatten() 316 | 317 | # then create node features. 318 | num_classes = df_labels.values.max() 319 | features = np.zeros((num_nodes, num_classes)) 320 | 321 | features[np.arange(num_nodes), labels - 1] = 1 322 | if feature_dim is not None: 323 | num_row, num_col = features.shape 324 | zero_col = np.zeros((num_row, feature_dim - num_col), dtype = features.dtype) 325 | features = np.hstack((features, zero_col)) 326 | 327 | features = np.random.normal(features, feature_noise, features.shape) 328 | print(f'number of nodes:{num_nodes}, feature dimension: {features.shape[1]}') 329 | 330 | features = torch.FloatTensor(features) 331 | labels = torch.LongTensor(labels) 332 | 333 | # The last, load hypergraph. 334 | # Corenll datasets are stored in lines of hyperedges. Each line is the set of nodes for that edge. 335 | p2hyperedge_list = osp.join(path, dataset, f'hyperedges-{dataset}.txt') 336 | node_list = [] 337 | he_list = [] 338 | he_id = num_nodes 339 | 340 | with open(p2hyperedge_list, 'r') as f: 341 | for line in f: 342 | if line[-1] == '\n': 343 | line = line[:-1] 344 | cur_set = line.split(',') 345 | cur_set = [int(x) for x in cur_set] 346 | 347 | node_list += cur_set 348 | he_list += [he_id] * len(cur_set) 349 | he_id += 1 350 | # shift node_idx to start with 0. 351 | node_idx_min = np.min(node_list) 352 | node_list = [x - node_idx_min for x in node_list] 353 | 354 | edge_index = [node_list + he_list, 355 | he_list + node_list] 356 | 357 | edge_index = torch.LongTensor(edge_index) 358 | 359 | data = Data(x = features, 360 | edge_index = edge_index, 361 | y = labels) 362 | 363 | # data.coalesce() 364 | # There might be errors if edge_index.max() != num_nodes. 365 | # used user function to override the default function. 366 | # the following will also sort the edge_index and remove duplicates. 367 | total_num_node_id_he_id = edge_index.max() + 1 368 | data.edge_index, data.edge_attr = coalesce(data.edge_index, 369 | None, 370 | total_num_node_id_he_id, 371 | total_num_node_id_he_id) 372 | 373 | 374 | n_x = num_nodes 375 | # n_x = n_expanded 376 | num_class = len(np.unique(labels.numpy())) 377 | val_lb = int(n_x * train_percent) 378 | percls_trn = int(round(train_percent * n_x / num_class)) 379 | # data = random_planetoid_splits(data, num_class, percls_trn, val_lb) 380 | data.n_x = n_x 381 | # add parameters to attribute 382 | 383 | data.train_percent = train_percent 384 | data.num_hyperedges = he_id - num_nodes 385 | 386 | return data 387 | 388 | if __name__ == '__main__': 389 | import ipdb 390 | ipdb.set_trace() 391 | # data = load_yelp_dataset() 392 | data = load_cornell_dataset(dataset = 'walmart-trips', feature_noise = 0.1) 393 | data = load_cornell_dataset(dataset = 'walmart-trips', feature_noise = 1) 394 | data = load_cornell_dataset(dataset = 'walmart-trips', feature_noise = 10) 395 | 396 | -------------------------------------------------------------------------------- /baselines_hypergnn/load_other_datasets.py: -------------------------------------------------------------------------------- 1 | #! /usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | # vim:fenc=utf-8 4 | # 5 | # Copyright © 2021 6 | # 7 | # Distributed under terms of the MIT license. 8 | 9 | """ 10 | This script contains functions for loading the following datasets: 11 | co-authorship: (dblp, cora) 12 | walmart-trips (From cornell) 13 | Amazon-reviews 14 | U.S. House committee 15 | """ 16 | 17 | import torch 18 | import os 19 | import pickle 20 | import ipdb 21 | 22 | import os.path as osp 23 | import numpy as np 24 | import pandas as pd 25 | import scipy.sparse as sp 26 | 27 | from torch_geometric.data import Data 28 | from torch_sparse import coalesce 29 | # from randomperm_code import random_planetoid_splits 30 | from sklearn.feature_extraction.text import CountVectorizer 31 | 32 | def load_LE_dataset(path=None, dataset="ModelNet40", train_percent = 0.025): 33 | # load edges, features, and labels. 34 | print('Loading {} dataset...'.format(dataset)) 35 | 36 | file_name = f'{dataset}.content' 37 | p2idx_features_labels = osp.join(path, dataset, file_name) 38 | idx_features_labels = np.genfromtxt(p2idx_features_labels, 39 | dtype=np.dtype(str)) 40 | # features = np.array(idx_features_labels[:, 1:-1]) 41 | features = sp.csr_matrix(idx_features_labels[:, 1:-1], dtype=np.float32) 42 | # labels = encode_onehot(idx_features_labels[:, -1]) 43 | labels = torch.LongTensor(idx_features_labels[:, -1].astype(float)) 44 | 45 | 46 | print ('load features') 47 | 48 | # build graph 49 | idx = np.array(idx_features_labels[:, 0], dtype=np.int32) 50 | idx_map = {j: i for i, j in enumerate(idx)} 51 | 52 | file_name = f'{dataset}.edges' 53 | p2edges_unordered = osp.join(path, dataset, file_name) 54 | edges_unordered = np.genfromtxt(p2edges_unordered, 55 | dtype=np.int32) 56 | 57 | 58 | edges = np.array(list(map(idx_map.get, edges_unordered.flatten())), 59 | dtype=np.int32).reshape(edges_unordered.shape) 60 | 61 | print ('load edges') 62 | 63 | 64 | projected_features = torch.FloatTensor(np.array(features.todense())) 65 | 66 | 67 | # From adjacency matrix to edge_list 68 | edge_index = edges.T 69 | # ipdb.set_trace() 70 | assert edge_index[0].max() == edge_index[1].min() - 1 71 | 72 | # check if values in edge_index is consecutive. i.e. no missing value for node_id/he_id. 73 | assert len(np.unique(edge_index)) == edge_index.max() + 1 74 | 75 | num_nodes = edge_index[0].max() + 1 76 | num_he = edge_index[1].max() - num_nodes + 1 77 | 78 | edge_index = np.hstack((edge_index, edge_index[::-1, :])) 79 | # ipdb.set_trace() 80 | 81 | # build torch data class 82 | data = Data( 83 | # x = projected_features, 84 | x = torch.FloatTensor(np.array(features[:num_nodes].todense())), 85 | edge_index = torch.LongTensor(edge_index), 86 | y = labels[:num_nodes]) 87 | 88 | 89 | # ipdb.set_trace() 90 | # data.coalesce() 91 | # There might be errors if edge_index.max() != num_nodes. 92 | # used user function to override the default function. 93 | # the following will also sort the edge_index and remove duplicates. 94 | total_num_node_id_he_id = len(np.unique(edge_index)) 95 | data.edge_index, data.edge_attr = coalesce(data.edge_index, 96 | None, 97 | total_num_node_id_he_id, 98 | total_num_node_id_he_id) 99 | 100 | 101 | 102 | 103 | # ipdb.set_trace() 104 | 105 | # # generate train, test, val mask. 106 | n_x = num_nodes 107 | # n_x = n_expanded 108 | num_class = len(np.unique(labels[:num_nodes].numpy())) 109 | val_lb = int(n_x * train_percent) 110 | percls_trn = int(round(train_percent * n_x / num_class)) 111 | # data = random_planetoid_splits(data, num_class, percls_trn, val_lb) 112 | data.n_x = n_x 113 | # add parameters to attribute 114 | 115 | 116 | data.train_percent = train_percent 117 | data.num_hyperedges = num_he 118 | 119 | return data 120 | 121 | def load_citation_dataset(path='../hyperGCN/data/', dataset = 'cora', train_percent = 0.025): 122 | ''' 123 | this will read the citation dataset from HyperGCN, and convert it edge_list to 124 | [[ -V- | -E- ] 125 | [ -E- | -V- ]] 126 | ''' 127 | print(f'Loading hypergraph dataset from hyperGCN: {dataset}') 128 | 129 | # first load node features: 130 | with open(osp.join(path, dataset, 'features.pickle'), 'rb') as f: 131 | features = pickle.load(f) 132 | features = features.todense() 133 | 134 | # then load node labels: 135 | with open(osp.join(path, dataset, 'labels.pickle'), 'rb') as f: 136 | labels = pickle.load(f) 137 | 138 | num_nodes, feature_dim = features.shape 139 | assert num_nodes == len(labels) 140 | print(f'number of nodes:{num_nodes}, feature dimension: {feature_dim}') 141 | 142 | features = torch.FloatTensor(features) 143 | labels = torch.LongTensor(labels) 144 | 145 | # The last, load hypergraph. 146 | with open(osp.join(path, dataset, 'hypergraph.pickle'), 'rb') as f: 147 | # hypergraph in hyperGCN is in the form of a dictionary. 148 | # { hyperedge: [list of nodes in the he], ...} 149 | hypergraph = pickle.load(f) 150 | 151 | print(f'number of hyperedges: {len(hypergraph)}') 152 | 153 | edge_idx = num_nodes 154 | node_list = [] 155 | edge_list = [] 156 | for he in hypergraph.keys(): 157 | cur_he = hypergraph[he] 158 | cur_size = len(cur_he) 159 | 160 | node_list += list(cur_he) 161 | edge_list += [edge_idx] * cur_size 162 | 163 | edge_idx += 1 164 | 165 | edge_index = np.array([ node_list + edge_list, 166 | edge_list + node_list], dtype = np.int) 167 | edge_index = torch.LongTensor(edge_index) 168 | 169 | data = Data(x = features, 170 | edge_index = edge_index, 171 | y = labels) 172 | 173 | # data.coalesce() 174 | # There might be errors if edge_index.max() != num_nodes. 175 | # used user function to override the default function. 176 | # the following will also sort the edge_index and remove duplicates. 177 | total_num_node_id_he_id = edge_index.max() + 1 178 | data.edge_index, data.edge_attr = coalesce(data.edge_index, 179 | None, 180 | total_num_node_id_he_id, 181 | total_num_node_id_he_id) 182 | 183 | 184 | n_x = num_nodes 185 | # n_x = n_expanded 186 | num_class = len(np.unique(labels.numpy())) 187 | val_lb = int(n_x * train_percent) 188 | percls_trn = int(round(train_percent * n_x / num_class)) 189 | # data = random_planetoid_splits(data, num_class, percls_trn, val_lb) 190 | data.n_x = n_x 191 | # add parameters to attribute 192 | 193 | data.train_percent = train_percent 194 | data.num_hyperedges = len(hypergraph) 195 | 196 | return data 197 | 198 | def load_yelp_dataset(path='../data/raw_data/yelp_raw_datasets/', dataset = 'yelp', 199 | name_dictionary_size = 1000, 200 | train_percent = 0.025): 201 | ''' 202 | this will read the yelp dataset from source files, and convert it edge_list to 203 | [[ -V- | -E- ] 204 | [ -E- | -V- ]] 205 | 206 | each node is a restaurant, a hyperedge represent a set of restaurants one user had been to. 207 | 208 | node features: 209 | - latitude, longitude 210 | - state, in one-hot coding. 211 | - city, in one-hot coding. 212 | - name, in bag-of-words 213 | 214 | node label: 215 | - average stars from 2-10, converted from original stars which is binned in x.5, min stars = 1 216 | ''' 217 | print(f'Loading hypergraph dataset from {dataset}') 218 | 219 | # first load node features: 220 | # load longtitude and latitude of restaurant. 221 | latlong = pd.read_csv(osp.join(path, 'yelp_restaurant_latlong.csv')).values 222 | 223 | # city - zipcode - state integer indicator dataframe. 224 | loc = pd.read_csv(osp.join(path, 'yelp_restaurant_locations.csv')) 225 | state_int = loc.state_int.values 226 | city_int = loc.city_int.values 227 | 228 | num_nodes = loc.shape[0] 229 | state_1hot = np.zeros((num_nodes, state_int.max())) 230 | state_1hot[np.arange(num_nodes), state_int - 1] = 1 231 | 232 | city_1hot = np.zeros((num_nodes, city_int.max())) 233 | city_1hot[np.arange(num_nodes), city_int - 1] = 1 234 | 235 | # convert restaurant name into bag-of-words feature. 236 | vectorizer = CountVectorizer(max_features = name_dictionary_size, stop_words = 'english', strip_accents = 'ascii') 237 | res_name = pd.read_csv(osp.join(path, 'yelp_restaurant_name.csv')).values.flatten() 238 | name_bow = vectorizer.fit_transform(res_name).todense() 239 | 240 | features = np.hstack([latlong, state_1hot, city_1hot, name_bow]) 241 | 242 | # then load node labels: 243 | df_labels = pd.read_csv(osp.join(path, 'yelp_restaurant_business_stars.csv')) 244 | labels = df_labels.values.flatten() 245 | 246 | num_nodes, feature_dim = features.shape 247 | assert num_nodes == len(labels) 248 | print(f'number of nodes:{num_nodes}, feature dimension: {feature_dim}') 249 | 250 | features = torch.FloatTensor(features) 251 | labels = torch.LongTensor(labels) 252 | 253 | # The last, load hypergraph. 254 | # Yelp restaurant review hypergraph is store in a incidence matrix. 255 | H = pd.read_csv(osp.join(path, 'yelp_restaurant_incidence_H.csv')) 256 | node_list = H.node.values - 1 257 | edge_list = H.he.values - 1 + num_nodes 258 | 259 | edge_index = np.vstack([node_list, edge_list]) 260 | edge_index = np.hstack([edge_index, edge_index[::-1, :]]) 261 | 262 | edge_index = torch.LongTensor(edge_index) 263 | 264 | data = Data(x = features, 265 | edge_index = edge_index, 266 | y = labels) 267 | 268 | # data.coalesce() 269 | # There might be errors if edge_index.max() != num_nodes. 270 | # used user function to override the default function. 271 | # the following will also sort the edge_index and remove duplicates. 272 | total_num_node_id_he_id = edge_index.max() + 1 273 | data.edge_index, data.edge_attr = coalesce(data.edge_index, 274 | None, 275 | total_num_node_id_he_id, 276 | total_num_node_id_he_id) 277 | 278 | 279 | n_x = num_nodes 280 | # n_x = n_expanded 281 | num_class = len(np.unique(labels.numpy())) 282 | val_lb = int(n_x * train_percent) 283 | percls_trn = int(round(train_percent * n_x / num_class)) 284 | # data = random_planetoid_splits(data, num_class, percls_trn, val_lb) 285 | data.n_x = n_x 286 | # add parameters to attribute 287 | 288 | data.train_percent = train_percent 289 | data.num_hyperedges = H.he.values.max() 290 | 291 | return data 292 | 293 | def load_cornell_dataset(path='../data/raw_data/', dataset = 'amazon', 294 | feature_noise = 0.1, 295 | feature_dim = None, 296 | train_percent = 0.025): 297 | ''' 298 | this will read the yelp dataset from source files, and convert it edge_list to 299 | [[ -V- | -E- ] 300 | [ -E- | -V- ]] 301 | 302 | each node is a restaurant, a hyperedge represent a set of restaurants one user had been to. 303 | 304 | node features: 305 | - add gaussian noise with sigma = nosie, mean = one hot coded label. 306 | 307 | node label: 308 | - average stars from 2-10, converted from original stars which is binned in x.5, min stars = 1 309 | ''' 310 | print(f'Loading hypergraph dataset from cornell: {dataset}') 311 | 312 | # first load node labels 313 | df_labels = pd.read_csv(osp.join(path, dataset, f'node-labels-{dataset}.txt'), names = ['node_label']) 314 | num_nodes = df_labels.shape[0] 315 | labels = df_labels.values.flatten() 316 | 317 | # then create node features. 318 | num_classes = df_labels.values.max() 319 | features = np.zeros((num_nodes, num_classes)) 320 | 321 | features[np.arange(num_nodes), labels - 1] = 1 322 | if feature_dim is not None: 323 | num_row, num_col = features.shape 324 | zero_col = np.zeros((num_row, feature_dim - num_col), dtype = features.dtype) 325 | features = np.hstack((features, zero_col)) 326 | 327 | features = np.random.normal(features, feature_noise, features.shape) 328 | print(f'number of nodes:{num_nodes}, feature dimension: {features.shape[1]}') 329 | 330 | features = torch.FloatTensor(features) 331 | labels = torch.LongTensor(labels) 332 | 333 | # The last, load hypergraph. 334 | # Corenll datasets are stored in lines of hyperedges. Each line is the set of nodes for that edge. 335 | p2hyperedge_list = osp.join(path, dataset, f'hyperedges-{dataset}.txt') 336 | node_list = [] 337 | he_list = [] 338 | he_id = num_nodes 339 | 340 | with open(p2hyperedge_list, 'r') as f: 341 | for line in f: 342 | if line[-1] == '\n': 343 | line = line[:-1] 344 | cur_set = line.split(',') 345 | cur_set = [int(x) for x in cur_set] 346 | 347 | node_list += cur_set 348 | he_list += [he_id] * len(cur_set) 349 | he_id += 1 350 | # shift node_idx to start with 0. 351 | node_idx_min = np.min(node_list) 352 | node_list = [x - node_idx_min for x in node_list] 353 | 354 | edge_index = [node_list + he_list, 355 | he_list + node_list] 356 | 357 | edge_index = torch.LongTensor(edge_index) 358 | 359 | data = Data(x = features, 360 | edge_index = edge_index, 361 | y = labels) 362 | 363 | # data.coalesce() 364 | # There might be errors if edge_index.max() != num_nodes. 365 | # used user function to override the default function. 366 | # the following will also sort the edge_index and remove duplicates. 367 | total_num_node_id_he_id = edge_index.max() + 1 368 | data.edge_index, data.edge_attr = coalesce(data.edge_index, 369 | None, 370 | total_num_node_id_he_id, 371 | total_num_node_id_he_id) 372 | 373 | 374 | n_x = num_nodes 375 | # n_x = n_expanded 376 | num_class = len(np.unique(labels.numpy())) 377 | val_lb = int(n_x * train_percent) 378 | percls_trn = int(round(train_percent * n_x / num_class)) 379 | # data = random_planetoid_splits(data, num_class, percls_trn, val_lb) 380 | data.n_x = n_x 381 | # add parameters to attribute 382 | 383 | data.train_percent = train_percent 384 | data.num_hyperedges = he_id - num_nodes 385 | 386 | return data 387 | 388 | if __name__ == '__main__': 389 | import ipdb 390 | ipdb.set_trace() 391 | # data = load_yelp_dataset() 392 | data = load_cornell_dataset(dataset = 'walmart-trips', feature_noise = 0.1) 393 | data = load_cornell_dataset(dataset = 'walmart-trips', feature_noise = 1) 394 | data = load_cornell_dataset(dataset = 'walmart-trips', feature_noise = 10) 395 | 396 | -------------------------------------------------------------------------------- /baselines_hypergnn/perturb.sh: -------------------------------------------------------------------------------- 1 | python train.py --method HGNN --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.1 2 | python train.py --method HGNN --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.3 3 | python train.py --method HGNN --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.5 4 | python train.py --method HGNN --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.7 5 | python train.py --method HGNN --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.9 6 | python train.py --method HGNN --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.1 7 | python train.py --method HGNN --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.3 8 | python train.py --method HGNN --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.5 9 | python train.py --method HGNN --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.7 10 | python train.py --method HGNN --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.9 11 | 12 | python train.py --method AllDeepSets --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.1 13 | python train.py --method AllDeepSets --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.3 14 | python train.py --method AllDeepSets --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.5 15 | python train.py --method AllDeepSets --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.7 16 | python train.py --method AllDeepSets --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.9 17 | python train.py --method AllDeepSets --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.1 18 | python train.py --method AllDeepSets --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.3 19 | python train.py --method AllDeepSets --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.5 20 | python train.py --method AllDeepSets --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.7 21 | python train.py --method AllDeepSets --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.9 22 | 23 | python train.py --method AllSetTransformer --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 8 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 4 --lr 0.001 --perturb_type replace --perturb_prop 0.1 24 | python train.py --method AllSetTransformer --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 8 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 4 --lr 0.001 --perturb_type replace --perturb_prop 0.3 25 | python train.py --method AllSetTransformer --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 8 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 4 --lr 0.001 --perturb_type replace --perturb_prop 0.5 26 | python train.py --method AllSetTransformer --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 8 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 4 --lr 0.001 --perturb_type replace --perturb_prop 0.7 27 | python train.py --method AllSetTransformer --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 8 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 4 --lr 0.001 --perturb_type replace --perturb_prop 0.9 28 | python train.py --method AllSetTransformer --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 4 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 4 --lr 0.001 --perturb_type replace --perturb_prop 0.1 29 | python train.py --method AllSetTransformer --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 4 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 4 --lr 0.001 --perturb_type replace --perturb_prop 0.3 30 | python train.py --method AllSetTransformer --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 4 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 4 --lr 0.001 --perturb_type replace --perturb_prop 0.5 31 | python train.py --method AllSetTransformer --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 4 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 4 --lr 0.001 --perturb_type replace --perturb_prop 0.7 32 | python train.py --method AllSetTransformer --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 4 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 4 --lr 0.001 --perturb_type replace --perturb_prop 0.9 33 | 34 | python train.py --method HyperGCN --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --Classifier_num_layers 1 --MLP_hidden 64 --Classifier_hidden 64 --HyperGCN_mediators --HyperGCN_fast --wd 0.00001 --epochs 500 --runs 20 --feature_noise 0.0 --cuda 0 --lr 0.01 --perturb_type replace --perturb_prop 0.1 35 | python train.py --method HyperGCN --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --Classifier_num_layers 1 --MLP_hidden 64 --Classifier_hidden 64 --HyperGCN_mediators --HyperGCN_fast --wd 0.00001 --epochs 500 --runs 20 --feature_noise 0.0 --cuda 0 --lr 0.01 --perturb_type replace --perturb_prop 0.3 36 | python train.py --method HyperGCN --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --Classifier_num_layers 1 --MLP_hidden 64 --Classifier_hidden 64 --HyperGCN_mediators --HyperGCN_fast --wd 0.00001 --epochs 500 --runs 20 --feature_noise 0.0 --cuda 0 --lr 0.01 --perturb_type replace --perturb_prop 0.5 37 | python train.py --method HyperGCN --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --Classifier_num_layers 1 --MLP_hidden 64 --Classifier_hidden 64 --HyperGCN_mediators --HyperGCN_fast --wd 0.00001 --epochs 500 --runs 20 --feature_noise 0.0 --cuda 0 --lr 0.01 --perturb_type replace --perturb_prop 0.7 38 | python train.py --method HyperGCN --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --Classifier_num_layers 1 --MLP_hidden 64 --Classifier_hidden 64 --HyperGCN_mediators --HyperGCN_fast --wd 0.00001 --epochs 500 --runs 20 --feature_noise 0.0 --cuda 0 --lr 0.01 --perturb_type replace --perturb_prop 0.9 39 | python train.py --method HyperGCN --dname cora --All_num_layers 1 --MLP_num_layers 2 --Classifier_num_layers 1 --MLP_hidden 64 --Classifier_hidden 64 --HyperGCN_mediators --HyperGCN_fast --wd 0.0 --epochs 500 --runs 20 --feature_noise 0.0 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.1 40 | python train.py --method HyperGCN --dname cora --All_num_layers 1 --MLP_num_layers 2 --Classifier_num_layers 1 --MLP_hidden 64 --Classifier_hidden 64 --HyperGCN_mediators --HyperGCN_fast --wd 0.0 --epochs 500 --runs 20 --feature_noise 0.0 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.3 41 | python train.py --method HyperGCN --dname cora --All_num_layers 1 --MLP_num_layers 2 --Classifier_num_layers 1 --MLP_hidden 64 --Classifier_hidden 64 --HyperGCN_mediators --HyperGCN_fast --wd 0.0 --epochs 500 --runs 20 --feature_noise 0.0 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.5 42 | python train.py --method HyperGCN --dname cora --All_num_layers 1 --MLP_num_layers 2 --Classifier_num_layers 1 --MLP_hidden 64 --Classifier_hidden 64 --HyperGCN_mediators --HyperGCN_fast --wd 0.0 --epochs 500 --runs 20 --feature_noise 0.0 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.7 43 | python train.py --method HyperGCN --dname cora --All_num_layers 1 --MLP_num_layers 2 --Classifier_num_layers 1 --MLP_hidden 64 --Classifier_hidden 64 --HyperGCN_mediators --HyperGCN_fast --wd 0.0 --epochs 500 --runs 20 --feature_noise 0.0 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.9 44 | 45 | python train.py --method HCHA --HCHA_symdegnorm --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 4 --lr 0.001 --perturb_type replace --perturb_prop 0.1 46 | python train.py --method HCHA --HCHA_symdegnorm --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 4 --lr 0.001 --perturb_type replace --perturb_prop 0.3 47 | python train.py --method HCHA --HCHA_symdegnorm --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 4 --lr 0.001 --perturb_type replace --perturb_prop 0.5 48 | python train.py --method HCHA --HCHA_symdegnorm --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 4 --lr 0.001 --perturb_type replace --perturb_prop 0.7 49 | python train.py --method HCHA --HCHA_symdegnorm --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 4 --lr 0.001 --perturb_type replace --perturb_prop 0.9 50 | python train.py --method HCHA --HCHA_symdegnorm --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 5 --lr 0.001 --perturb_type replace --perturb_prop 0.1 51 | python train.py --method HCHA --HCHA_symdegnorm --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 5 --lr 0.001 --perturb_type replace --perturb_prop 0.3 52 | python train.py --method HCHA --HCHA_symdegnorm --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 5 --lr 0.001 --perturb_type replace --perturb_prop 0.5 53 | python train.py --method HCHA --HCHA_symdegnorm --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 5 --lr 0.001 --perturb_type replace --perturb_prop 0.7 54 | python train.py --method HCHA --HCHA_symdegnorm --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 1 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 256 --wd 0.0 --epochs 500 --runs 20 --cuda 5 --lr 0.001 --perturb_type replace --perturb_prop 0.9 55 | 56 | python train.py --method UniGCNII --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 8 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.1 57 | python train.py --method UniGCNII --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 8 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.3 58 | python train.py --method UniGCNII --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 8 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.5 59 | python train.py --method UniGCNII --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 8 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.7 60 | python train.py --method UniGCNII --dname cora --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 8 --Classifier_num_layers 1 --MLP_hidden 512 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.9 61 | python train.py --method UniGCNII --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 8 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.1 62 | python train.py --method UniGCNII --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 8 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.3 63 | python train.py --method UniGCNII --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 8 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.5 64 | python train.py --method UniGCNII --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 8 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.7 65 | python train.py --method UniGCNII --dname coauthor_dblp --All_num_layers 1 --MLP_num_layers 2 --feature_noise 0.0 --heads 8 --Classifier_num_layers 1 --MLP_hidden 256 --Classifier_hidden 128 --wd 0.0 --epochs 500 --runs 20 --cuda 0 --lr 0.001 --perturb_type replace --perturb_prop 0.9 66 | 67 | -------------------------------------------------------------------------------- /ours/preprocessing.py: -------------------------------------------------------------------------------- 1 | #! /usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | # vim:fenc=utf-8 4 | # 5 | # Copyright © 2021 jianhao2 6 | # 7 | # Distributed under terms of the MIT license. 8 | 9 | """ 10 | 11 | """ 12 | 13 | import torch 14 | 15 | import numpy as np 16 | 17 | from collections import defaultdict, Counter 18 | from itertools import combinations 19 | from torch_scatter import scatter_add, scatter 20 | from torch_geometric.nn.conv.gcn_conv import gcn_norm 21 | 22 | def expand_edge_index(data, edge_th=0): 23 | ''' 24 | args: 25 | num_nodes: regular nodes. i.e. x.shape[0] 26 | num_edges: number of hyperedges. not the star expansion edges. 27 | 28 | this function will expand each n2he relations, [[n_1, n_2, n_3], 29 | [e_7, e_7, e_7]] 30 | to : 31 | [[n_1, n_1, n_2, n_2, n_3, n_3], 32 | [e_7_2, e_7_3, e_7_1, e_7_3, e_7_1, e_7_2]] 33 | 34 | and each he2n relations: [[e_7, e_7, e_7], 35 | [n_1, n_2, n_3]] 36 | to : 37 | [[e_7_1, e_7_2, e_7_3], 38 | [n_1, n_2, n_3]] 39 | 40 | and repeated for every hyperedge. 41 | ''' 42 | edge_index = data.edge_index 43 | num_nodes = data.n_x[0].item() 44 | if hasattr(data, 'totedges'): 45 | num_edges = data.totedges 46 | else: 47 | num_edges = data.num_hyperedges[0] 48 | 49 | expanded_n2he_index = [] 50 | # n2he_with_same_heid = [] 51 | 52 | # expanded_he2n_index = [] 53 | # he2n_with_same_heid = [] 54 | 55 | # start edge_id from the largest node_id + 1. 56 | cur_he_id = num_nodes 57 | # keep an mapping of new_edge_id to original edge_id for edge_size query. 58 | new_edge_id_2_original_edge_id = {} 59 | 60 | # do the expansion for all annotated he_id in the original edge_index 61 | # ipdb.set_trace() 62 | for he_idx in range(num_nodes, num_edges + num_nodes): 63 | # find all nodes within the same hyperedge. 64 | selected_he = edge_index[:, edge_index[1] == he_idx] 65 | size_of_he = selected_he.shape[1] 66 | 67 | # Trim a hyperedge if its size>edge_th 68 | if edge_th > 0: 69 | if size_of_he > edge_th: 70 | continue 71 | 72 | if size_of_he == 1: 73 | # there is only one node in this hyperedge -> self-loop node. add to graph. 74 | # n2he_with_same_heid.append(selected_he) 75 | 76 | new_n2he = selected_he.clone() 77 | new_n2he[1] = cur_he_id 78 | expanded_n2he_index.append(new_n2he) 79 | 80 | # ==== 81 | # new_he2n_same_heid = torch.flip(selected_he, dims = [0]) 82 | # he2n_with_same_heid.append(new_he2n_same_heid) 83 | 84 | # new_he2n = torch.flip(selected_he, dims = [0]) 85 | # new_he2n[0] = cur_he_id 86 | # expanded_he2n_index.append(new_he2n) 87 | 88 | cur_he_id += 1 89 | continue 90 | 91 | # ------------------------------- 92 | # # new_n2he_same_heid uses same he id for all nodes. 93 | # new_n2he_same_heid = selected_he.repeat_interleave(size_of_he - 1, dim = 1) 94 | # n2he_with_same_heid.append(new_n2he_same_heid) 95 | 96 | # for new_n2he mapping. connect the nodes to all repeated he first. 97 | # then remove those connection that corresponding to the node itself. 98 | new_n2he = selected_he.repeat_interleave(size_of_he, dim=1) 99 | 100 | # new_edge_ids start from the he_id from previous iteration (cur_he_id). 101 | new_edge_ids = torch.LongTensor( 102 | np.arange(cur_he_id, cur_he_id + size_of_he)).repeat(size_of_he) 103 | new_n2he[1] = new_edge_ids 104 | 105 | # build a mapping between node and it's corresponding edge. 106 | # e.g. {n_1: e_7_1, n_2: e_7_2} 107 | tmp_node_id_2_he_id_dict = {} 108 | for idx in range(size_of_he): 109 | new_edge_id_2_original_edge_id[cur_he_id] = he_idx 110 | cur_node_id = selected_he[0][idx].item() 111 | tmp_node_id_2_he_id_dict[cur_node_id] = cur_he_id 112 | cur_he_id += 1 113 | 114 | # create n2he by deleting the self-product edge. 115 | new_he_select_mask = torch.BoolTensor([True] * new_n2he.shape[1]) 116 | for col_idx in range(new_n2he.shape[1]): 117 | tmp_node_id, tmp_edge_id = new_n2he[0, col_idx].item( 118 | ), new_n2he[1, col_idx].item() 119 | if tmp_node_id_2_he_id_dict[tmp_node_id] == tmp_edge_id: 120 | new_he_select_mask[col_idx] = False 121 | new_n2he = new_n2he[:, new_he_select_mask] 122 | expanded_n2he_index.append(new_n2he) 123 | 124 | 125 | # # --------------------------- 126 | # # create he2n from mapping. 127 | # new_he2n = np.array([[he_id, node_id] for node_id, he_id in tmp_node_id_2_he_id_dict.items()]) 128 | # new_he2n = torch.from_numpy(new_he2n.T).to(device = edge_index.device) 129 | # expanded_he2n_index.append(new_he2n) 130 | 131 | # # create he2n with same heid as input edge_index. 132 | # new_he2n_same_heid = torch.zeros_like(new_he2n, device = edge_index.device) 133 | # new_he2n_same_heid[1] = new_he2n[1] 134 | # new_he2n_same_heid[0] = torch.ones_like(new_he2n[0]) * he_idx 135 | # he2n_with_same_heid.append(new_he2n_same_heid) 136 | 137 | new_edge_index = torch.cat(expanded_n2he_index, dim=1) 138 | # new_he2n_index = torch.cat(expanded_he2n_index, dim = 1) 139 | # new_edge_index = torch.cat([new_n2he_index, new_he2n_index], dim = 1) 140 | # sort the new_edge_index by first row. (node_ids) 141 | new_order = new_edge_index[0].argsort() 142 | data.edge_index = new_edge_index[:, new_order] 143 | 144 | return data 145 | 146 | 147 | # functions for processing/checkning the edge_index 148 | def get_HyperGCN_He_dict(data): 149 | # Assume edge_index = [V;E], sorted 150 | edge_index = np.array(data.edge_index) 151 | """ 152 | For each he, clique-expansion. Note that we allow the weighted edge. 153 | Note that if node pair (vi,vj) is contained in both he1, he2, we will have (vi,vj) twice in edge_index. (weighted version CE) 154 | We default no self loops so far. 155 | """ 156 | # # Construct a dictionary 157 | # He2V_List = [] 158 | # # Sort edge_index according to he_id 159 | # _, sorted_idx = torch.sort(edge_index[1]) 160 | # edge_index = edge_index[:,sorted_idx].type(torch.LongTensor) 161 | # current_heid = -1 162 | # for idx, he_id in enumerate(edge_index[1]): 163 | # if current_heid != he_id: 164 | # current_heid = he_id 165 | # if idx != 0 and len(he2v)>1: #drop original self loops 166 | # He2V_List.append(he2v) 167 | # he2v = [] 168 | # he2v.append(edge_index[0,idx].item()) 169 | # # Remember to append the last he 170 | # if len(he2v)>1: 171 | # He2V_List.append(he2v) 172 | # # Now, turn He2V_List into a dictionary 173 | edge_index[1, :] = edge_index[1, :]-edge_index[1, :].min() 174 | He_dict = {} 175 | for he in np.unique(edge_index[1, :]): 176 | # ipdb.set_trace() 177 | nodes_in_he = list(edge_index[0, :][edge_index[1, :] == he]) 178 | He_dict[he.item()] = nodes_in_he 179 | 180 | # for he_id, he in enumerate(He2V_List): 181 | # He_dict[he_id] = he 182 | 183 | return He_dict 184 | 185 | 186 | def ConstructH(data): 187 | """ 188 | Construct incidence matrix H of size (num_nodes,num_hyperedges) from edge_index = [V;E] 189 | """ 190 | # ipdb.set_trace() 191 | edge_index = np.array(data.edge_index) 192 | # Don't use edge_index[0].max()+1, as some nodes maybe isolated 193 | num_nodes = data.x.shape[0] 194 | num_hyperedges = np.max(edge_index[1])-np.min(edge_index[1])+1 195 | H = np.zeros((num_nodes, num_hyperedges)) 196 | cur_idx = 0 197 | for he in np.unique(edge_index[1]): 198 | nodes_in_he = edge_index[0][edge_index[1] == he] 199 | H[nodes_in_he, cur_idx] = 1. 200 | cur_idx += 1 201 | 202 | data.edge_index = H 203 | return data 204 | 205 | 206 | def ConstructH_HNHN(data): 207 | """ 208 | Construct incidence matrix H of size (num_nodes, num_hyperedges) from edge_index = [V;E] 209 | """ 210 | edge_index = np.array(data.edge_index) 211 | num_nodes = data.n_x[0] 212 | num_hyperedges = int(data.totedges) 213 | H = np.zeros((num_nodes, num_hyperedges)) 214 | cur_idx = 0 215 | for he in np.unique(edge_index[1]): 216 | nodes_in_he = edge_index[0][edge_index[1] == he] 217 | H[nodes_in_he, cur_idx] = 1. 218 | cur_idx += 1 219 | 220 | # data.incident_mat = H 221 | return H 222 | 223 | 224 | def generate_G_from_H(data): 225 | """ 226 | This function generate the propagation matrix G for HGNN from incidence matrix H. 227 | Here we assume data.edge_index is already the incidence matrix H. (can be done by ConstructH()) 228 | Adapted from HGNN github repo: https://github.com/iMoonLab/HGNN 229 | :param H: hypergraph incidence matrix H 230 | :param variable_weight: whether the weight of hyperedge is variable 231 | :return: G 232 | """ 233 | # ipdb.set_trace() 234 | H = data.edge_index 235 | H = np.array(H) 236 | n_edge = H.shape[1] 237 | # the weight of the hyperedge 238 | W = np.ones(n_edge) 239 | # the degree of the node 240 | DV = np.sum(H * W, axis=1) 241 | # the degree of the hyperedge 242 | DE = np.sum(H, axis=0) 243 | 244 | invDE = np.mat(np.diag(np.power(DE, -1))) 245 | DV2 = np.mat(np.diag(np.power(DV, -0.5))) 246 | # replace nan with 0. This is caused by isolated nodes 247 | DV2 = np.nan_to_num(DV2) 248 | W = np.mat(np.diag(W)) 249 | H = np.mat(H) 250 | HT = H.T 251 | 252 | # if variable_weight: 253 | # DV2_H = DV2 * H 254 | # invDE_HT_DV2 = invDE * HT * DV2 255 | # return DV2_H, W, invDE_HT_DV2 256 | # else: 257 | G = DV2 * H * W * invDE * HT * DV2 258 | data.edge_index = torch.Tensor(G) 259 | return data 260 | 261 | 262 | def generate_G_for_HNHN(data, args): 263 | """ 264 | This function generate the propagation matrix G_V2E and G_E2V for HNHN from incidence matrix H. 265 | Here we assume data.edge_index is already the incidence matrix H. (can be done by ConstructH()) 266 | 267 | :param H: hypergraph incidence matrix H 268 | :param variable_weight: whether the weight of hyperedge is variable 269 | :return: G 270 | """ 271 | # ipdb.set_trace() 272 | H = data.edge_index 273 | alpha = args.HNHN_alpha 274 | beta = args.HNHN_beta 275 | H = np.array(H) 276 | 277 | # the degree of the node 278 | DV = np.sum(H, axis=1) 279 | # the degree of the hyperedge 280 | DE = np.sum(H, axis=0) 281 | 282 | G_V2E = np.diag(DE**(-beta))@H.T@np.diag(DV**(beta)) 283 | G_E2V = np.diag(DV**(-alpha))@H@np.diag(DE**(alpha)) 284 | 285 | # if variable_weight: 286 | # DV2_H = DV2 * H 287 | # invDE_HT_DV2 = invDE * HT * DV2 288 | # return DV2_H, W, invDE_HT_DV2 289 | # else: 290 | data.G_V2E = torch.Tensor(G_V2E) 291 | data.G_E2V = torch.Tensor(G_E2V) 292 | return data 293 | 294 | 295 | def generate_norm_HNHN(H, data, args): 296 | """ 297 | :param H: hypergraph incidence matrix H 298 | :param variable_weight: whether the weight of hyperedge is variable 299 | :return: G 300 | """ 301 | # H = data.incident_mat 302 | alpha = args.HNHN_alpha 303 | beta = args.HNHN_beta 304 | H = np.array(H) 305 | 306 | # the degree of the node 307 | DV = np.sum(H, axis=1) 308 | # the degree of the hyperedge 309 | DE = np.sum(H, axis=0) 310 | 311 | num_nodes = data.n_x[0] 312 | num_hyperedges = int(data.totedges) 313 | # alpha part 314 | D_e_alpha = DE ** alpha 315 | D_v_alpha = np.zeros(num_nodes) 316 | for i in range(num_nodes): 317 | # which edges this node is in 318 | he_list = np.where(H[i] == 1)[0] 319 | D_v_alpha[i] = np.sum(DE[he_list] ** alpha) 320 | 321 | # beta part 322 | D_v_beta = DV ** beta 323 | D_e_beta = np.zeros(num_hyperedges) 324 | for i in range(num_hyperedges): 325 | # which nodes are in this hyperedge 326 | node_list = np.where(H[:, i] == 1)[0] 327 | D_e_beta[i] = np.sum(DV[node_list] ** beta) 328 | 329 | D_v_alpha_inv = 1.0 / D_v_alpha 330 | D_v_alpha_inv[D_v_alpha_inv == float("inf")] = 0 331 | 332 | D_e_beta_inv = 1.0 / D_e_beta 333 | D_e_beta_inv[D_e_beta_inv == float("inf")] = 0 334 | 335 | data.D_e_alpha = torch.from_numpy(D_e_alpha).float() 336 | data.D_v_alpha_inv = torch.from_numpy(D_v_alpha_inv).float() 337 | data.D_v_beta = torch.from_numpy(D_v_beta).float() 338 | data.D_e_beta_inv = torch.from_numpy(D_e_beta_inv).float() 339 | 340 | return data 341 | 342 | 343 | def ConstructV2V(data): 344 | # Assume edge_index = [V;E], sorted 345 | edge_index = np.array(data.edge_index) 346 | """ 347 | For each he, clique-expansion. Note that we DONT allow duplicated edges. 348 | Instead, we record its corresponding weights. 349 | We default no self loops so far. 350 | """ 351 | # # Note that the method below for CE can be memory expensive!!! 352 | # new_edge_index = [] 353 | # for he in np.unique(edge_index[1, :]): 354 | # nodes_in_he = edge_index[0, :][edge_index[1, :] == he] 355 | # if len(nodes_in_he) == 1: 356 | # continue #skip self loops 357 | # combs = combinations(nodes_in_he,2) 358 | # for comb in combs: 359 | # new_edge_index.append([comb[0],comb[1]]) 360 | 361 | 362 | # new_edge_index, new_edge_weight = torch.tensor(new_edge_index).type(torch.LongTensor).unique(dim=0,return_counts=True) 363 | # data.edge_index = new_edge_index.transpose(0,1) 364 | # data.norm = new_edge_weight.type(torch.float) 365 | 366 | # # Use the method below for better memory complexity 367 | edge_weight_dict = {} 368 | for he in np.unique(edge_index[1, :]): 369 | nodes_in_he = np.sort(edge_index[0, :][edge_index[1, :] == he]) 370 | if len(nodes_in_he) == 1: 371 | continue # skip self loops 372 | combs = combinations(nodes_in_he, 2) 373 | for comb in combs: 374 | if not comb in edge_weight_dict.keys(): 375 | edge_weight_dict[comb] = 1 376 | else: 377 | edge_weight_dict[comb] += 1 378 | 379 | # # Now, translate dict to edge_index and norm 380 | # 381 | new_edge_index = np.zeros((2, len(edge_weight_dict))) 382 | new_norm = np.zeros((len(edge_weight_dict))) 383 | cur_idx = 0 384 | for edge in edge_weight_dict: 385 | new_edge_index[:, cur_idx] = edge 386 | new_norm[cur_idx] = edge_weight_dict[edge] 387 | cur_idx += 1 388 | 389 | data.edge_index = torch.tensor(new_edge_index).type(torch.LongTensor) 390 | data.norm = torch.tensor(new_norm).type(torch.FloatTensor) 391 | return data 392 | 393 | 394 | def ExtractV2E(data): 395 | # Assume edge_index = [V|E;E|V] 396 | edge_index = data.edge_index 397 | # First, ensure the sorting is correct (increasing along edge_index[0]) 398 | _, sorted_idx = torch.sort(edge_index[0]) 399 | edge_index = edge_index[:, sorted_idx].type(torch.LongTensor) 400 | 401 | num_nodes = data.n_x[0] 402 | num_hyperedges = data.num_hyperedges[0] 403 | if not ((data.n_x[0]+data.num_hyperedges[0]-1) == data.edge_index[0].max().item()): 404 | print('num_hyperedges does not match! 1') 405 | return 406 | cidx = torch.where(edge_index[0] == num_nodes)[ 407 | 0].min() # cidx: [V...|cidx E...] 408 | data.edge_index = edge_index[:, :cidx].type(torch.LongTensor) 409 | return data 410 | 411 | 412 | def Add_Self_Loops(data): 413 | # update so we dont jump on some indices 414 | # Assume edge_index = [V;E]. If not, use ExtractV2E() 415 | edge_index = data.edge_index 416 | num_nodes = data.n_x[0] 417 | num_hyperedges = data.num_hyperedges[0] 418 | 419 | if not ((data.n_x[0] + data.num_hyperedges[0] - 1) == data.edge_index[1].max().item()): 420 | print('num_hyperedges does not match! 2') 421 | return 422 | 423 | hyperedge_appear_fre = Counter(edge_index[1].numpy()) 424 | # store the nodes that already have self-loops 425 | skip_node_lst = [] 426 | for edge in hyperedge_appear_fre: 427 | if hyperedge_appear_fre[edge] == 1: 428 | skip_node = edge_index[0][torch.where( 429 | edge_index[1] == edge)[0].item()] 430 | skip_node_lst.append(skip_node.item()) 431 | 432 | new_edge_idx = edge_index[1].max() + 1 433 | new_edges = torch.zeros( 434 | (2, num_nodes - len(skip_node_lst)), dtype=edge_index.dtype) 435 | tmp_count = 0 436 | for i in range(num_nodes): 437 | if i not in skip_node_lst: 438 | new_edges[0][tmp_count] = i 439 | new_edges[1][tmp_count] = new_edge_idx 440 | new_edge_idx += 1 441 | tmp_count += 1 442 | 443 | data.totedges = num_hyperedges + num_nodes - len(skip_node_lst) 444 | edge_index = torch.cat((edge_index, new_edges), dim=1) 445 | # Sort along w.r.t. nodes 446 | _, sorted_idx = torch.sort(edge_index[0]) 447 | data.edge_index = edge_index[:, sorted_idx].type(torch.LongTensor) 448 | return data 449 | 450 | 451 | def norm_contruction(data, option='all_one', TYPE='V2E'): 452 | if TYPE == 'V2E': 453 | if option == 'all_one': 454 | data.norm = torch.ones_like(data.edge_index[0]) 455 | 456 | elif option == 'deg_half_sym': 457 | edge_weight = torch.ones_like(data.edge_index[0]) 458 | cidx = data.edge_index[1].min() 459 | Vdeg = scatter_add(edge_weight, data.edge_index[0], dim=0) 460 | HEdeg = scatter_add(edge_weight, data.edge_index[1]-cidx, dim=0) 461 | V_norm = Vdeg**(-1/2) 462 | E_norm = HEdeg**(-1/2) 463 | data.norm = V_norm[data.edge_index[0]] * \ 464 | E_norm[data.edge_index[1]-cidx] 465 | 466 | elif TYPE == 'V2V': 467 | data.edge_index, data.norm = gcn_norm( 468 | data.edge_index, data.norm, add_self_loops=True) 469 | return data 470 | 471 | 472 | def rand_train_test_idx(label, train_prop=.5, valid_prop=.25, ignore_negative=True, balance=False): 473 | """ Adapted from https://github.com/CUAI/Non-Homophily-Benchmarks""" 474 | """ randomly splits label into train/valid/test splits """ 475 | if not balance: 476 | if ignore_negative: 477 | labeled_nodes = torch.where(label != -1)[0] 478 | else: 479 | labeled_nodes = label 480 | 481 | n = labeled_nodes.shape[0] 482 | train_num = int(n * train_prop) 483 | valid_num = int(n * valid_prop) 484 | 485 | perm = torch.as_tensor(np.random.permutation(n)) 486 | 487 | train_indices = perm[:train_num] 488 | val_indices = perm[train_num:train_num + valid_num] 489 | test_indices = perm[train_num + valid_num:] 490 | 491 | if not ignore_negative: 492 | return train_indices, val_indices, test_indices 493 | 494 | train_idx = labeled_nodes[train_indices] 495 | valid_idx = labeled_nodes[val_indices] 496 | test_idx = labeled_nodes[test_indices] 497 | 498 | split_idx = {'train': train_idx, 499 | 'valid': valid_idx, 500 | 'test': test_idx} 501 | else: 502 | # ipdb.set_trace() 503 | indices = [] 504 | for i in range(label.max()+1): 505 | index = torch.where((label == i))[0].view(-1) 506 | index = index[torch.randperm(index.size(0))] 507 | indices.append(index) 508 | 509 | percls_trn = int(train_prop/(label.max()+1)*len(label)) 510 | val_lb = int(valid_prop*len(label)) 511 | train_idx = torch.cat([i[:percls_trn] for i in indices], dim=0) 512 | rest_index = torch.cat([i[percls_trn:] for i in indices], dim=0) 513 | rest_index = rest_index[torch.randperm(rest_index.size(0))] 514 | valid_idx = rest_index[:val_lb] 515 | test_idx = rest_index[val_lb:] 516 | split_idx = {'train': train_idx, 517 | 'valid': valid_idx, 518 | 'test': test_idx} 519 | return split_idx 520 | 521 | 522 | -------------------------------------------------------------------------------- /ours/train.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # coding: utf-8 3 | 4 | import os 5 | import time 6 | # import math 7 | import torch 8 | import pickle 9 | import argparse 10 | import random 11 | 12 | import numpy as np 13 | import os.path as osp 14 | import scipy.sparse as sp 15 | import torch_sparse 16 | import torch.nn as nn 17 | import torch.nn.functional as F 18 | import matplotlib.pyplot as plt 19 | 20 | from tqdm import tqdm 21 | 22 | from layers import * 23 | from models import * 24 | from preprocessing import * 25 | 26 | from convert_datasets_to_pygDataset import dataset_Hypergraph 27 | 28 | def perturb_hyperedges(data, prop): 29 | data_p = data 30 | edge_index = data_p.edge_index 31 | num_node = data.x.shape[0] 32 | e_idxs = edge_index[1,:] - num_node 33 | edges = (edge_index[1,:].max()) - (edge_index[1,:].min()) 34 | p_num = ((edge_index[1,:].max()) - (edge_index[1,:].min())) * prop 35 | p_num = int(p_num) 36 | chosen_edges = torch.as_tensor(np.random.permutation(int(edges.numpy()))).to(edge_index.device) 37 | chosen_edges = chosen_edges[:p_num] 38 | for i in range(chosen_edges.shape[0]): 39 | chosen_edge = chosen_edges[i] 40 | edge_index = edge_index[:, (e_idxs != chosen_edge)] 41 | e_idxs = e_idxs[(e_idxs != chosen_edge)] 42 | edge_idxs = [edge_index] 43 | for i in range(chosen_edges.shape[0]): 44 | new_edge = torch.as_tensor(np.random.choice(int(num_node), 5, replace=False)).to(edge_index.device) 45 | for j in range(new_edge.shape[0]): 46 | edge_idx_i = torch.zeros([2,1]).to(edge_index.device) 47 | edge_idx_i[0,0] = new_edge[j] 48 | edge_idx_i[1,0] = chosen_edges[i] + num_node 49 | edge_idxs.append(edge_idx_i) 50 | edge_idxs = torch.cat(edge_idxs, dim=1) 51 | data_p.edge_index = edge_idxs.long() 52 | return data_p 53 | 54 | 55 | def parse_method(args): 56 | model = MLP_model(args) 57 | return model 58 | 59 | 60 | class Logger(object): 61 | """ Adapted from https://github.com/snap-stanford/ogb/ """ 62 | 63 | def __init__(self, runs, info=None): 64 | self.info = info 65 | self.results = [[] for _ in range(runs)] 66 | 67 | def add_result(self, run, result): 68 | assert len(result) == 3 69 | assert run >= 0 and run < len(self.results) 70 | self.results[run].append(result) 71 | 72 | def print_statistics(self, run=None): 73 | if run is not None: 74 | result = 100 * torch.tensor(self.results[run]) 75 | argmax = result[:, 1].argmax().item() 76 | print(f'Run {run + 1:02d}:') 77 | print(f'Highest Train: {result[:, 0].max():.2f}') 78 | print(f'Highest Valid: {result[:, 1].max():.2f}') 79 | print(f' Final Train: {result[argmax, 0]:.2f}') 80 | print(f' Final Test: {result[argmax, 2]:.2f}') 81 | else: 82 | result = 100 * torch.tensor(self.results) 83 | 84 | best_results = [] 85 | for r in result: 86 | train1 = r[:, 0].max().item() 87 | valid = r[:, 1].max().item() 88 | train2 = r[r[:, 1].argmax(), 0].item() 89 | test = r[r[:, 1].argmax(), 2].item() 90 | best_results.append((train1, valid, train2, test)) 91 | 92 | best_result = torch.tensor(best_results) 93 | 94 | print(f'All runs:') 95 | r = best_result[:, 0] 96 | print(f'Highest Train: {r.mean():.2f} ± {r.std():.2f}') 97 | r = best_result[:, 1] 98 | print(f'Highest Valid: {r.mean():.2f} ± {r.std():.2f}') 99 | r = best_result[:, 2] 100 | print(f' Final Train: {r.mean():.2f} ± {r.std():.2f}') 101 | r = best_result[:, 3] 102 | print(f' Final Test: {r.mean():.2f} ± {r.std():.2f}') 103 | 104 | return best_result[:, 1], best_result[:, 3] 105 | 106 | def plot_result(self, run=None): 107 | plt.style.use('seaborn') 108 | if run is not None: 109 | result = 100 * torch.tensor(self.results[run]) 110 | x = torch.arange(result.shape[0]) 111 | plt.figure() 112 | print(f'Run {run + 1:02d}:') 113 | plt.plot(x, result[:, 0], x, result[:, 1], x, result[:, 2]) 114 | plt.legend(['Train', 'Valid', 'Test']) 115 | else: 116 | result = 100 * torch.tensor(self.results[0]) 117 | x = torch.arange(result.shape[0]) 118 | plt.figure() 119 | # print(f'Run {run + 1:02d}:') 120 | plt.plot(x, result[:, 0], x, result[:, 1], x, result[:, 2]) 121 | plt.legend(['Train', 'Valid', 'Test']) 122 | 123 | 124 | @torch.no_grad() 125 | def evaluate(model, data, split_idx, eval_func, result=None): 126 | if result is not None: 127 | out = result 128 | else: 129 | model.eval() 130 | out, _ = model(data) 131 | out = F.log_softmax(out, dim=1) 132 | 133 | train_acc = eval_func( 134 | data.y[split_idx['train']], out[split_idx['train']]) 135 | valid_acc = eval_func( 136 | data.y[split_idx['valid']], out[split_idx['valid']]) 137 | test_acc = eval_func( 138 | data.y[split_idx['test']], out[split_idx['test']]) 139 | 140 | # Also keep track of losses 141 | train_loss = F.nll_loss( 142 | out[split_idx['train']], data.y[split_idx['train']]) 143 | valid_loss = F.nll_loss( 144 | out[split_idx['valid']], data.y[split_idx['valid']]) 145 | test_loss = F.nll_loss( 146 | out[split_idx['test']], data.y[split_idx['test']]) 147 | return train_acc, valid_acc, test_acc, train_loss, valid_loss, test_loss, out 148 | 149 | 150 | def eval_acc(y_true, y_pred): 151 | acc_list = [] 152 | y_true = y_true.detach().cpu().numpy() 153 | y_pred = y_pred.argmax(dim=-1, keepdim=False).detach().cpu().numpy() 154 | 155 | # ipdb.set_trace() 156 | # for i in range(y_true.shape[1]): 157 | is_labeled = y_true == y_true 158 | correct = y_true[is_labeled] == y_pred[is_labeled] 159 | acc_list.append(float(np.sum(correct))/len(correct)) 160 | 161 | return sum(acc_list)/len(acc_list) 162 | 163 | def count_parameters(model): 164 | return sum(p.numel() for p in model.parameters() if p.requires_grad) 165 | 166 | 167 | def smooth_loss(node_feature, n_idxs, e_idxs, edge_index, epoch): 168 | loss = 0.0 169 | edges = int(edge_index[1,:].max() - edge_index[1,:].min()) 170 | flag = False 171 | if(edges > 3000): 172 | setup_seed(epoch) 173 | chosen_edges = torch.as_tensor(np.random.permutation(edges)).to(node_feature.device) 174 | edges = 2000 175 | chosen_edges = chosen_edges[:edges] 176 | flag = True 177 | for i in range(edges): 178 | if(flag): 179 | idx = chosen_edges[i] 180 | else: 181 | idx = i 182 | temp_n = n_idxs[(e_idxs==idx)] 183 | node_featur_i = node_feature[temp_n, :] 184 | z_i = torch.cdist(node_featur_i, node_featur_i, compute_mode='donot_use_mm_for_euclid_dist') 185 | loss_i = torch.max(z_i) 186 | loss = loss + loss_i 187 | return loss / edges 188 | 189 | def setup_seed(seed): 190 | torch.manual_seed(seed) 191 | torch.cuda.manual_seed_all(seed) 192 | np.random.seed(seed) 193 | random.seed(seed) 194 | torch.backends.cudnn.deterministic = True 195 | torch.use_deterministic_algorithms = True 196 | torch.cuda.current_device() 197 | torch.cuda._initialized = True 198 | 199 | # --- Main part of the training --- 200 | # # Part 0: Parse arguments 201 | 202 | 203 | """ 204 | 205 | """ 206 | 207 | if __name__ == '__main__': 208 | parser = argparse.ArgumentParser() 209 | parser.add_argument('--train_prop', type=float, default=0.5) 210 | parser.add_argument('--valid_prop', type=float, default=0.25) 211 | parser.add_argument('--dname', default='walmart-trips-100') 212 | # method in ['SetGNN','CEGCN','CEGAT','HyperGCN','HGNN','HCHA'] 213 | parser.add_argument('--method', default='AllSetTransformer') 214 | parser.add_argument('--epochs', default=500, type=int) 215 | # Number of runs for each split (test fix, only shuffle train/val) 216 | parser.add_argument('--runs', default=20, type=int) 217 | parser.add_argument('--cuda', default=0, choices=[-1,0,1,2,3,4,5,6,7], type=int) 218 | parser.add_argument('--dropout', default=0.5, type=float) 219 | parser.add_argument('--lr', default=0.001, type=float) 220 | parser.add_argument('--wd', default=0.0, type=float) 221 | # How many layers of full NLConvs 222 | parser.add_argument('--All_num_layers', default=2, type=int) 223 | parser.add_argument('--MLP_num_layers', default=2, 224 | type=int) # How many layers of encoder 225 | parser.add_argument('--MLP_hidden', default=64, 226 | type=int) # Encoder hidden units 227 | parser.add_argument('--Classifier_num_layers', default=1, 228 | type=int) # How many layers of decoder 229 | parser.add_argument('--Classifier_hidden', default=256, 230 | type=int) # Decoder hidden units 231 | parser.add_argument('--display_step', type=int, default=-1) 232 | parser.add_argument('--alpha', type=float, default=1) 233 | parser.add_argument('--aggregate', default='mean', choices=['sum', 'mean']) 234 | # ['all_one','deg_half_sym'] 235 | parser.add_argument('--normtype', default='all_one') 236 | parser.add_argument('--add_self_loop', action='store_false') 237 | # NormLayer for MLP. ['bn','ln','None'] 238 | parser.add_argument('--normalization', default='ln') 239 | parser.add_argument('--deepset_input_norm', default = True) 240 | parser.add_argument('--GPR', action='store_false') # skip all but last dec 241 | # skip all but last dec 242 | parser.add_argument('--LearnMask', action='store_false') 243 | parser.add_argument('--num_features', default=0, type=int) # Placeholder 244 | parser.add_argument('--num_classes', default=0, type=int) # Placeholder 245 | # Choose std for synthetic feature noise 246 | parser.add_argument('--perturb_type', default='toxic', type=str) 247 | parser.add_argument('--perturb_prop', default=0.0, type=float) 248 | parser.add_argument('--feature_noise', default='1', type=str) 249 | parser.add_argument('--sth_type', default='max_s', type=str) 250 | # whether the he contain self node or not 251 | parser.add_argument('--exclude_self', action='store_true') 252 | parser.add_argument('--PMA', action='store_true') 253 | # Args for HyperGCN 254 | parser.add_argument('--HyperGCN_mediators', action='store_true') 255 | parser.add_argument('--HyperGCN_fast', action='store_true') 256 | # Args for Attentions: GAT and SetGNN 257 | parser.add_argument('--heads', default=1, type=int) # Placeholder 258 | parser.add_argument('--output_heads', default=1, type=int) # Placeholder 259 | # Args for HNHN 260 | parser.add_argument('--HNHN_alpha', default=-1.5, type=float) 261 | parser.add_argument('--HNHN_beta', default=-0.5, type=float) 262 | parser.add_argument('--HNHN_nonlinear_inbetween', default=True, type=bool) 263 | # Args for HCHA 264 | parser.add_argument('--HCHA_symdegnorm', action='store_true') 265 | # Args for UniGNN 266 | parser.add_argument('--UniGNN_use-norm', action="store_true", help='use norm in the final layer') 267 | parser.add_argument('--UniGNN_degV', default = 0) 268 | parser.add_argument('--UniGNN_degE', default = 0) 269 | parser.add_argument('--seed', type=int, default=1000, help='Random seed.') 270 | 271 | parser.set_defaults(PMA=True) # True: Use PMA. False: Use Deepsets. 272 | parser.set_defaults(add_self_loop=True) 273 | parser.set_defaults(exclude_self=False) 274 | parser.set_defaults(GPR=False) 275 | parser.set_defaults(LearnMask=False) 276 | parser.set_defaults(HyperGCN_mediators=True) 277 | parser.set_defaults(HyperGCN_fast=True) 278 | parser.set_defaults(HCHA_symdegnorm=False) 279 | 280 | # Use the line below for .py file 281 | args = parser.parse_args() 282 | # Use the line below for notebook 283 | # args = parser.parse_args([]) 284 | # args, _ = parser.parse_known_args() 285 | 286 | 287 | # # Part 1: Load data 288 | 289 | 290 | ### Load and preprocess data ### 291 | existing_dataset = ['20newsW100', 'ModelNet40', 'zoo', 292 | 'NTU2012', 'Mushroom', 293 | 'coauthor_cora', 'coauthor_dblp', 294 | 'yelp', 'amazon-reviews', 'walmart-trips', 'house-committees', 295 | 'walmart-trips-100', 'house-committees-100', 296 | 'cora', 'citeseer', 'pubmed'] 297 | 298 | synthetic_list = ['amazon-reviews', 'walmart-trips', 'house-committees', 'walmart-trips-100', 'house-committees-100'] 299 | 300 | if args.dname in existing_dataset: 301 | dname = args.dname 302 | f_noise = args.feature_noise 303 | if (f_noise is not None) and dname in synthetic_list: 304 | p2raw = '../data/AllSet_all_raw_data/' 305 | dataset = dataset_Hypergraph(name=dname, 306 | feature_noise=f_noise, 307 | p2raw = p2raw) 308 | else: 309 | if dname in ['cora', 'citeseer','pubmed']: 310 | p2raw = '../data/AllSet_all_raw_data/cocitation/' 311 | elif dname in ['coauthor_cora', 'coauthor_dblp']: 312 | p2raw = '../data/AllSet_all_raw_data/coauthorship/' 313 | elif dname in ['yelp']: 314 | p2raw = '../data/AllSet_all_raw_data/yelp/' 315 | else: 316 | p2raw = '../data/AllSet_all_raw_data/' 317 | dataset = dataset_Hypergraph(name=dname, root = '../data/pyg_data/hypergraph_dataset_updated/', 318 | p2raw = p2raw) 319 | data = dataset.data 320 | args.num_features = dataset.num_features 321 | args.num_classes = dataset.num_classes 322 | if args.dname in ['yelp', 'walmart-trips', 'house-committees', 'walmart-trips-100', 'house-committees-100']: 323 | # Shift the y label to start with 0 324 | args.num_classes = len(data.y.unique()) 325 | data.y = data.y - data.y.min() 326 | if not hasattr(data, 'n_x'): 327 | data.n_x = torch.tensor([data.x.shape[0]]) 328 | if not hasattr(data, 'num_hyperedges'): 329 | # note that we assume the he_id is consecutive. 330 | data.num_hyperedges = torch.tensor( 331 | [data.edge_index[0].max()-data.n_x[0]+1]) 332 | 333 | data = ExtractV2E(data) 334 | setup_seed(args.seed) 335 | if((args.perturb_type == 'toxic') and (args.perturb_prop > 0)): 336 | data = perturb_hyperedges(data, args.perturb_prop) 337 | # Get splits 338 | split_idx_lst = [] 339 | for run in range(args.runs): 340 | split_idx = rand_train_test_idx( 341 | data.y, train_prop=args.train_prop, valid_prop=args.valid_prop) 342 | split_idx_lst.append(split_idx) 343 | 344 | 345 | # # Part 2: Load model 346 | 347 | 348 | model = parse_method(args) 349 | # put things to device 350 | if args.cuda in [0,1,2,3,4,5,6,7]: 351 | device = torch.device('cuda:'+str(args.cuda) 352 | if torch.cuda.is_available() else 'cpu') 353 | else: 354 | device = torch.device('cpu') 355 | 356 | model, data = model.to(device), data.to(device) 357 | 358 | num_params = count_parameters(model) 359 | 360 | 361 | # # Part 3: Main. Training + Evaluation 362 | 363 | 364 | logger = Logger(args.runs, args) 365 | 366 | criterion = nn.NLLLoss() 367 | eval_func = eval_acc 368 | 369 | model.train() 370 | # print('MODEL:', model) 371 | 372 | ### Training loop ### 373 | edge_index = data.edge_index 374 | 375 | n_idxs = edge_index[0,:] - edge_index[0,:].min() 376 | e_idxs = edge_index[1,:] - edge_index[1,:].min() 377 | x = data.x 378 | 379 | train_acc_tensor = torch.zeros((args.runs, args.epochs)) 380 | val_acc_tensor = torch.zeros((args.runs, args.epochs)) 381 | test_acc_tensor = torch.zeros((args.runs, args.epochs)) 382 | smooth_loss_tensor = torch.zeros((args.runs, args.epochs)) 383 | for run in tqdm(range(args.runs)): 384 | setup_seed(run) 385 | split_idx = split_idx_lst[run] 386 | train_idx = split_idx['train'].to(device) 387 | model.reset_parameters() 388 | optimizer = torch.optim.Adam(model.parameters(), lr=args.lr, weight_decay=args.wd) 389 | # This is for HNHN only 390 | # if args.method == 'HNHN': 391 | # scheduler = torch.optim.lr_scheduler.StepLR(optimizer,step_size=100, gamma=0.51) 392 | best_val = float('-inf') 393 | for epoch in range(args.epochs): 394 | # Training part 395 | model.train() 396 | optimizer.zero_grad() 397 | out, x = model(data) 398 | out = F.log_softmax(out, dim=1) 399 | loss_sth = 0 400 | if(args.alpha > 0): 401 | loss_sth = smooth_loss(x, n_idxs, e_idxs, edge_index, epoch) * args.alpha 402 | loss_cls = criterion(out[train_idx], data.y[train_idx]) 403 | loss = loss_cls + loss_sth 404 | loss.backward() 405 | optimizer.step() 406 | # if args.method == 'HNHN': 407 | # scheduler.step() 408 | # Evaluation part 409 | result = evaluate(model, data, split_idx, eval_func) 410 | logger.add_result(run, result[:3]) 411 | 412 | train_acc_tensor[run, epoch] = result[0] 413 | val_acc_tensor[run, epoch] = result[1] 414 | test_acc_tensor[run, epoch] = result[2] 415 | smooth_loss_tensor[run, epoch] = loss_sth 416 | if(args.alpha > 0): 417 | smooth_loss_tensor[run, epoch] = loss_sth.cpu() 418 | 419 | if epoch % args.display_step == 0 and args.display_step > 0: 420 | print(f'Epoch: {epoch:02d}, ' 421 | f'Total Train Loss: {loss:.4f}, ' 422 | f'Smooth Train Loss: {loss_sth:.4f}, ' 423 | f'Cls Train Loss: {loss_cls:.4f}, ' 424 | f'Valid Loss: {result[4]:.4f}, ' 425 | f'Test Loss: {result[5]:.4f}, ' 426 | f'Train Acc: {100 * result[0]:.2f}%, ' 427 | f'Valid Acc: {100 * result[1]:.2f}%, ' 428 | f'Test Acc: {100 * result[2]:.2f}%') 429 | 430 | # logger.print_statistics(run) 431 | 432 | ### Save results ### 433 | best_val, best_test = logger.print_statistics() 434 | res_root = 'results' 435 | if not osp.isdir(res_root): 436 | os.makedirs(res_root) 437 | res_root = '{}/layer_{}'.format(res_root, args.All_num_layers) 438 | if not osp.isdir(res_root): 439 | os.makedirs(res_root) 440 | res_root = '{}/{}'.format(res_root, args.method) 441 | if not osp.isdir(res_root): 442 | os.makedirs(res_root) 443 | 444 | filename = f'{res_root}/{args.dname}_noise_{args.feature_noise}.csv' 445 | print(f"Saving results to {filename}") 446 | with open(filename, 'a+') as write_obj: 447 | cur_line = f'{args.method}_{args.lr}_{args.wd}_{args.alpha}\n' 448 | cur_line += f'{args.perturb_type}_{args.perturb_prop}\n' 449 | cur_line += f',{best_val.mean():.3f} ± {best_val.std():.3f}\n' 450 | cur_line += f',{best_test.mean():.3f} ± {best_test.std():.3f}\n' 451 | cur_line += f',{num_params}\n' 452 | cur_line += f'\n' 453 | write_obj.write(cur_line) 454 | 455 | all_args_file = f'{res_root}/all_args_{args.dname}_noise_{args.feature_noise}.csv' 456 | with open(all_args_file, 'a+') as f: 457 | f.write(str(args)) 458 | f.write('\n') 459 | 460 | res_root_2 = './storage' 461 | if not osp.isdir(res_root_2): 462 | os.makedirs(res_root_2) 463 | filename = f'{res_root_2}/{args.dname}_{args.feature_noise}_noise_{args.alpha}_alpha.pickle' 464 | data = { 465 | 'train_acc_tensor': train_acc_tensor, 466 | 'val_acc_tensor': val_acc_tensor, 467 | 'test_acc_tensor': test_acc_tensor, 468 | 'smooth_loss_tensor': smooth_loss_tensor 469 | } 470 | with open(filename, 'wb') as handle: 471 | pickle.dump(data, handle, protocol=4) 472 | 473 | print('All done! Exit python code') 474 | quit() 475 | -------------------------------------------------------------------------------- /baselines_hypergnn/preprocessing.py: -------------------------------------------------------------------------------- 1 | #! /usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | # vim:fenc=utf-8 4 | # 5 | # Copyright © 2021 jianhao2 6 | # 7 | # Distributed under terms of the MIT license. 8 | 9 | """ 10 | 11 | """ 12 | 13 | import torch 14 | 15 | import numpy as np 16 | 17 | from collections import defaultdict, Counter 18 | from itertools import combinations 19 | from torch_scatter import scatter_add, scatter 20 | from torch_geometric.nn.conv.gcn_conv import gcn_norm 21 | 22 | def expand_edge_index(data, edge_th=0): 23 | ''' 24 | args: 25 | num_nodes: regular nodes. i.e. x.shape[0] 26 | num_edges: number of hyperedges. not the star expansion edges. 27 | 28 | this function will expand each n2he relations, [[n_1, n_2, n_3], 29 | [e_7, e_7, e_7]] 30 | to : 31 | [[n_1, n_1, n_2, n_2, n_3, n_3], 32 | [e_7_2, e_7_3, e_7_1, e_7_3, e_7_1, e_7_2]] 33 | 34 | and each he2n relations: [[e_7, e_7, e_7], 35 | [n_1, n_2, n_3]] 36 | to : 37 | [[e_7_1, e_7_2, e_7_3], 38 | [n_1, n_2, n_3]] 39 | 40 | and repeated for every hyperedge. 41 | ''' 42 | edge_index = data.edge_index 43 | num_nodes = data.n_x[0].item() 44 | if hasattr(data, 'totedges'): 45 | num_edges = data.totedges 46 | else: 47 | num_edges = data.num_hyperedges[0] 48 | 49 | expanded_n2he_index = [] 50 | # n2he_with_same_heid = [] 51 | 52 | # expanded_he2n_index = [] 53 | # he2n_with_same_heid = [] 54 | 55 | # start edge_id from the largest node_id + 1. 56 | cur_he_id = num_nodes 57 | # keep an mapping of new_edge_id to original edge_id for edge_size query. 58 | new_edge_id_2_original_edge_id = {} 59 | 60 | # do the expansion for all annotated he_id in the original edge_index 61 | # ipdb.set_trace() 62 | for he_idx in range(num_nodes, num_edges + num_nodes): 63 | # find all nodes within the same hyperedge. 64 | selected_he = edge_index[:, edge_index[1] == he_idx] 65 | size_of_he = selected_he.shape[1] 66 | 67 | # Trim a hyperedge if its size>edge_th 68 | if edge_th > 0: 69 | if size_of_he > edge_th: 70 | continue 71 | 72 | if size_of_he == 1: 73 | # there is only one node in this hyperedge -> self-loop node. add to graph. 74 | # n2he_with_same_heid.append(selected_he) 75 | 76 | new_n2he = selected_he.clone() 77 | new_n2he[1] = cur_he_id 78 | expanded_n2he_index.append(new_n2he) 79 | 80 | # ==== 81 | # new_he2n_same_heid = torch.flip(selected_he, dims = [0]) 82 | # he2n_with_same_heid.append(new_he2n_same_heid) 83 | 84 | # new_he2n = torch.flip(selected_he, dims = [0]) 85 | # new_he2n[0] = cur_he_id 86 | # expanded_he2n_index.append(new_he2n) 87 | 88 | cur_he_id += 1 89 | continue 90 | 91 | # ------------------------------- 92 | # # new_n2he_same_heid uses same he id for all nodes. 93 | # new_n2he_same_heid = selected_he.repeat_interleave(size_of_he - 1, dim = 1) 94 | # n2he_with_same_heid.append(new_n2he_same_heid) 95 | 96 | # for new_n2he mapping. connect the nodes to all repeated he first. 97 | # then remove those connection that corresponding to the node itself. 98 | new_n2he = selected_he.repeat_interleave(size_of_he, dim=1) 99 | 100 | # new_edge_ids start from the he_id from previous iteration (cur_he_id). 101 | new_edge_ids = torch.LongTensor( 102 | np.arange(cur_he_id, cur_he_id + size_of_he)).repeat(size_of_he) 103 | new_n2he[1] = new_edge_ids 104 | 105 | # build a mapping between node and it's corresponding edge. 106 | # e.g. {n_1: e_7_1, n_2: e_7_2} 107 | tmp_node_id_2_he_id_dict = {} 108 | for idx in range(size_of_he): 109 | new_edge_id_2_original_edge_id[cur_he_id] = he_idx 110 | cur_node_id = selected_he[0][idx].item() 111 | tmp_node_id_2_he_id_dict[cur_node_id] = cur_he_id 112 | cur_he_id += 1 113 | 114 | # create n2he by deleting the self-product edge. 115 | new_he_select_mask = torch.BoolTensor([True] * new_n2he.shape[1]) 116 | for col_idx in range(new_n2he.shape[1]): 117 | tmp_node_id, tmp_edge_id = new_n2he[0, col_idx].item( 118 | ), new_n2he[1, col_idx].item() 119 | if tmp_node_id_2_he_id_dict[tmp_node_id] == tmp_edge_id: 120 | new_he_select_mask[col_idx] = False 121 | new_n2he = new_n2he[:, new_he_select_mask] 122 | expanded_n2he_index.append(new_n2he) 123 | 124 | 125 | # # --------------------------- 126 | # # create he2n from mapping. 127 | # new_he2n = np.array([[he_id, node_id] for node_id, he_id in tmp_node_id_2_he_id_dict.items()]) 128 | # new_he2n = torch.from_numpy(new_he2n.T).to(device = edge_index.device) 129 | # expanded_he2n_index.append(new_he2n) 130 | 131 | # # create he2n with same heid as input edge_index. 132 | # new_he2n_same_heid = torch.zeros_like(new_he2n, device = edge_index.device) 133 | # new_he2n_same_heid[1] = new_he2n[1] 134 | # new_he2n_same_heid[0] = torch.ones_like(new_he2n[0]) * he_idx 135 | # he2n_with_same_heid.append(new_he2n_same_heid) 136 | 137 | new_edge_index = torch.cat(expanded_n2he_index, dim=1) 138 | # new_he2n_index = torch.cat(expanded_he2n_index, dim = 1) 139 | # new_edge_index = torch.cat([new_n2he_index, new_he2n_index], dim = 1) 140 | # sort the new_edge_index by first row. (node_ids) 141 | new_order = new_edge_index[0].argsort() 142 | data.edge_index = new_edge_index[:, new_order] 143 | 144 | return data 145 | 146 | 147 | # functions for processing/checkning the edge_index 148 | def get_HyperGCN_He_dict(data): 149 | # Assume edge_index = [V;E], sorted 150 | edge_index = np.array(data.edge_index) 151 | """ 152 | For each he, clique-expansion. Note that we allow the weighted edge. 153 | Note that if node pair (vi,vj) is contained in both he1, he2, we will have (vi,vj) twice in edge_index. (weighted version CE) 154 | We default no self loops so far. 155 | """ 156 | # # Construct a dictionary 157 | # He2V_List = [] 158 | # # Sort edge_index according to he_id 159 | # _, sorted_idx = torch.sort(edge_index[1]) 160 | # edge_index = edge_index[:,sorted_idx].type(torch.LongTensor) 161 | # current_heid = -1 162 | # for idx, he_id in enumerate(edge_index[1]): 163 | # if current_heid != he_id: 164 | # current_heid = he_id 165 | # if idx != 0 and len(he2v)>1: #drop original self loops 166 | # He2V_List.append(he2v) 167 | # he2v = [] 168 | # he2v.append(edge_index[0,idx].item()) 169 | # # Remember to append the last he 170 | # if len(he2v)>1: 171 | # He2V_List.append(he2v) 172 | # # Now, turn He2V_List into a dictionary 173 | edge_index[1, :] = edge_index[1, :]-edge_index[1, :].min() 174 | He_dict = {} 175 | for he in np.unique(edge_index[1, :]): 176 | # ipdb.set_trace() 177 | nodes_in_he = list(edge_index[0, :][edge_index[1, :] == he]) 178 | He_dict[he.item()] = nodes_in_he 179 | 180 | # for he_id, he in enumerate(He2V_List): 181 | # He_dict[he_id] = he 182 | 183 | return He_dict 184 | 185 | 186 | def ConstructH(data): 187 | """ 188 | Construct incidence matrix H of size (num_nodes,num_hyperedges) from edge_index = [V;E] 189 | """ 190 | # ipdb.set_trace() 191 | edge_index = np.array(data.edge_index) 192 | # Don't use edge_index[0].max()+1, as some nodes maybe isolated 193 | num_nodes = data.x.shape[0] 194 | num_hyperedges = np.max(edge_index[1])-np.min(edge_index[1])+1 195 | H = np.zeros((num_nodes, num_hyperedges)) 196 | cur_idx = 0 197 | for he in np.unique(edge_index[1]): 198 | nodes_in_he = edge_index[0][edge_index[1] == he] 199 | H[nodes_in_he, cur_idx] = 1. 200 | cur_idx += 1 201 | 202 | data.edge_index = H 203 | return data 204 | 205 | 206 | def ConstructH_HNHN(data): 207 | """ 208 | Construct incidence matrix H of size (num_nodes, num_hyperedges) from edge_index = [V;E] 209 | """ 210 | edge_index = np.array(data.edge_index) 211 | num_nodes = data.n_x[0] 212 | num_hyperedges = int(data.totedges) 213 | H = np.zeros((num_nodes, num_hyperedges)) 214 | cur_idx = 0 215 | for he in np.unique(edge_index[1]): 216 | nodes_in_he = edge_index[0][edge_index[1] == he] 217 | H[nodes_in_he, cur_idx] = 1. 218 | cur_idx += 1 219 | 220 | # data.incident_mat = H 221 | return H 222 | 223 | 224 | def generate_G_from_H(data): 225 | """ 226 | This function generate the propagation matrix G for HGNN from incidence matrix H. 227 | Here we assume data.edge_index is already the incidence matrix H. (can be done by ConstructH()) 228 | Adapted from HGNN github repo: https://github.com/iMoonLab/HGNN 229 | :param H: hypergraph incidence matrix H 230 | :param variable_weight: whether the weight of hyperedge is variable 231 | :return: G 232 | """ 233 | # ipdb.set_trace() 234 | H = data.edge_index 235 | H = np.array(H) 236 | n_edge = H.shape[1] 237 | # the weight of the hyperedge 238 | W = np.ones(n_edge) 239 | # the degree of the node 240 | DV = np.sum(H * W, axis=1) 241 | # the degree of the hyperedge 242 | DE = np.sum(H, axis=0) 243 | 244 | invDE = np.mat(np.diag(np.power(DE, -1))) 245 | DV2 = np.mat(np.diag(np.power(DV, -0.5))) 246 | # replace nan with 0. This is caused by isolated nodes 247 | DV2 = np.nan_to_num(DV2) 248 | W = np.mat(np.diag(W)) 249 | H = np.mat(H) 250 | HT = H.T 251 | 252 | # if variable_weight: 253 | # DV2_H = DV2 * H 254 | # invDE_HT_DV2 = invDE * HT * DV2 255 | # return DV2_H, W, invDE_HT_DV2 256 | # else: 257 | G = DV2 * H * W * invDE * HT * DV2 258 | data.edge_index = torch.Tensor(G) 259 | return data 260 | 261 | 262 | def generate_G_for_HNHN(data, args): 263 | """ 264 | This function generate the propagation matrix G_V2E and G_E2V for HNHN from incidence matrix H. 265 | Here we assume data.edge_index is already the incidence matrix H. (can be done by ConstructH()) 266 | 267 | :param H: hypergraph incidence matrix H 268 | :param variable_weight: whether the weight of hyperedge is variable 269 | :return: G 270 | """ 271 | # ipdb.set_trace() 272 | H = data.edge_index 273 | alpha = args.HNHN_alpha 274 | beta = args.HNHN_beta 275 | H = np.array(H) 276 | 277 | # the degree of the node 278 | DV = np.sum(H, axis=1) 279 | # the degree of the hyperedge 280 | DE = np.sum(H, axis=0) 281 | 282 | G_V2E = np.diag(DE**(-beta))@H.T@np.diag(DV**(beta)) 283 | G_E2V = np.diag(DV**(-alpha))@H@np.diag(DE**(alpha)) 284 | 285 | # if variable_weight: 286 | # DV2_H = DV2 * H 287 | # invDE_HT_DV2 = invDE * HT * DV2 288 | # return DV2_H, W, invDE_HT_DV2 289 | # else: 290 | data.G_V2E = torch.Tensor(G_V2E) 291 | data.G_E2V = torch.Tensor(G_E2V) 292 | return data 293 | 294 | 295 | def generate_norm_HNHN(H, data, args): 296 | """ 297 | :param H: hypergraph incidence matrix H 298 | :param variable_weight: whether the weight of hyperedge is variable 299 | :return: G 300 | """ 301 | # H = data.incident_mat 302 | alpha = args.HNHN_alpha 303 | beta = args.HNHN_beta 304 | H = np.array(H) 305 | 306 | # the degree of the node 307 | DV = np.sum(H, axis=1) 308 | # the degree of the hyperedge 309 | DE = np.sum(H, axis=0) 310 | 311 | num_nodes = data.n_x[0] 312 | num_hyperedges = int(data.totedges) 313 | # alpha part 314 | D_e_alpha = DE ** alpha 315 | D_v_alpha = np.zeros(num_nodes) 316 | for i in range(num_nodes): 317 | # which edges this node is in 318 | he_list = np.where(H[i] == 1)[0] 319 | D_v_alpha[i] = np.sum(DE[he_list] ** alpha) 320 | 321 | # beta part 322 | D_v_beta = DV ** beta 323 | D_e_beta = np.zeros(num_hyperedges) 324 | for i in range(num_hyperedges): 325 | # which nodes are in this hyperedge 326 | node_list = np.where(H[:, i] == 1)[0] 327 | D_e_beta[i] = np.sum(DV[node_list] ** beta) 328 | 329 | D_v_alpha_inv = 1.0 / D_v_alpha 330 | D_v_alpha_inv[D_v_alpha_inv == float("inf")] = 0 331 | 332 | D_e_beta_inv = 1.0 / D_e_beta 333 | D_e_beta_inv[D_e_beta_inv == float("inf")] = 0 334 | 335 | data.D_e_alpha = torch.from_numpy(D_e_alpha).float() 336 | data.D_v_alpha_inv = torch.from_numpy(D_v_alpha_inv).float() 337 | data.D_v_beta = torch.from_numpy(D_v_beta).float() 338 | data.D_e_beta_inv = torch.from_numpy(D_e_beta_inv).float() 339 | 340 | return data 341 | 342 | 343 | def ConstructV2V(data): 344 | # Assume edge_index = [V;E], sorted 345 | edge_index = np.array(data.edge_index) 346 | """ 347 | For each he, clique-expansion. Note that we DONT allow duplicated edges. 348 | Instead, we record its corresponding weights. 349 | We default no self loops so far. 350 | """ 351 | # # Note that the method below for CE can be memory expensive!!! 352 | # new_edge_index = [] 353 | # for he in np.unique(edge_index[1, :]): 354 | # nodes_in_he = edge_index[0, :][edge_index[1, :] == he] 355 | # if len(nodes_in_he) == 1: 356 | # continue #skip self loops 357 | # combs = combinations(nodes_in_he,2) 358 | # for comb in combs: 359 | # new_edge_index.append([comb[0],comb[1]]) 360 | 361 | 362 | # new_edge_index, new_edge_weight = torch.tensor(new_edge_index).type(torch.LongTensor).unique(dim=0,return_counts=True) 363 | # data.edge_index = new_edge_index.transpose(0,1) 364 | # data.norm = new_edge_weight.type(torch.float) 365 | 366 | # # Use the method below for better memory complexity 367 | edge_weight_dict = {} 368 | for he in np.unique(edge_index[1, :]): 369 | nodes_in_he = np.sort(edge_index[0, :][edge_index[1, :] == he]) 370 | if len(nodes_in_he) == 1: 371 | continue # skip self loops 372 | combs = combinations(nodes_in_he, 2) 373 | for comb in combs: 374 | if not comb in edge_weight_dict.keys(): 375 | edge_weight_dict[comb] = 1 376 | else: 377 | edge_weight_dict[comb] += 1 378 | 379 | # # Now, translate dict to edge_index and norm 380 | # 381 | new_edge_index = np.zeros((2, len(edge_weight_dict))) 382 | new_norm = np.zeros((len(edge_weight_dict))) 383 | cur_idx = 0 384 | for edge in edge_weight_dict: 385 | new_edge_index[:, cur_idx] = edge 386 | new_norm[cur_idx] = edge_weight_dict[edge] 387 | cur_idx += 1 388 | 389 | data.edge_index = torch.tensor(new_edge_index).type(torch.LongTensor) 390 | data.norm = torch.tensor(new_norm).type(torch.FloatTensor) 391 | return data 392 | 393 | 394 | def ExtractV2E(data): 395 | # Assume edge_index = [V|E;E|V] 396 | edge_index = data.edge_index 397 | # First, ensure the sorting is correct (increasing along edge_index[0]) 398 | _, sorted_idx = torch.sort(edge_index[0]) 399 | edge_index = edge_index[:, sorted_idx].type(torch.LongTensor) 400 | 401 | num_nodes = data.n_x[0] 402 | num_hyperedges = data.num_hyperedges[0] 403 | if not ((data.n_x[0]+data.num_hyperedges[0]-1) == data.edge_index[0].max().item()): 404 | print('num_hyperedges does not match! 1') 405 | return 406 | cidx = torch.where(edge_index[0] == num_nodes)[ 407 | 0].min() # cidx: [V...|cidx E...] 408 | data.edge_index = edge_index[:, :cidx].type(torch.LongTensor) 409 | return data 410 | 411 | 412 | def Add_Self_Loops(data): 413 | # update so we dont jump on some indices 414 | # Assume edge_index = [V;E]. If not, use ExtractV2E() 415 | edge_index = data.edge_index 416 | num_nodes = data.n_x[0] 417 | num_hyperedges = data.num_hyperedges[0] 418 | 419 | #if not ((data.n_x[0] + data.num_hyperedges[0] - 1) == data.edge_index[1].max().item()): 420 | # print('num_hyperedges does not match! 2') 421 | # return 422 | 423 | hyperedge_appear_fre = Counter(edge_index[1].numpy()) 424 | # store the nodes that already have self-loops 425 | skip_node_lst = [] 426 | for edge in hyperedge_appear_fre: 427 | if hyperedge_appear_fre[edge] == 1: 428 | skip_node = edge_index[0][torch.where( 429 | edge_index[1] == edge)[0].item()] 430 | skip_node_lst.append(skip_node.item()) 431 | 432 | new_edge_idx = edge_index[1].max() + 1 433 | new_edges = torch.zeros( 434 | (2, num_nodes - len(skip_node_lst)), dtype=edge_index.dtype) 435 | tmp_count = 0 436 | for i in range(num_nodes): 437 | if i not in skip_node_lst: 438 | new_edges[0][tmp_count] = i 439 | new_edges[1][tmp_count] = new_edge_idx 440 | new_edge_idx += 1 441 | tmp_count += 1 442 | 443 | data.totedges = num_hyperedges + num_nodes - len(skip_node_lst) 444 | edge_index = torch.cat((edge_index, new_edges), dim=1) 445 | # Sort along w.r.t. nodes 446 | _, sorted_idx = torch.sort(edge_index[0]) 447 | data.edge_index = edge_index[:, sorted_idx].type(torch.LongTensor) 448 | return data 449 | 450 | 451 | def norm_contruction(data, option='all_one', TYPE='V2E'): 452 | if TYPE == 'V2E': 453 | if option == 'all_one': 454 | data.norm = torch.ones_like(data.edge_index[0]) 455 | 456 | elif option == 'deg_half_sym': 457 | edge_weight = torch.ones_like(data.edge_index[0]) 458 | cidx = data.edge_index[1].min() 459 | Vdeg = scatter_add(edge_weight, data.edge_index[0], dim=0) 460 | HEdeg = scatter_add(edge_weight, data.edge_index[1]-cidx, dim=0) 461 | V_norm = Vdeg**(-1/2) 462 | E_norm = HEdeg**(-1/2) 463 | data.norm = V_norm[data.edge_index[0]] * \ 464 | E_norm[data.edge_index[1]-cidx] 465 | 466 | elif TYPE == 'V2V': 467 | data.edge_index, data.norm = gcn_norm( 468 | data.edge_index, data.norm, add_self_loops=True) 469 | return data 470 | 471 | 472 | def rand_train_test_idx(label, train_prop=.5, valid_prop=.25, ignore_negative=True, balance=False): 473 | """ Adapted from https://github.com/CUAI/Non-Homophily-Benchmarks""" 474 | """ randomly splits label into train/valid/test splits """ 475 | if not balance: 476 | if ignore_negative: 477 | labeled_nodes = torch.where(label != -1)[0] 478 | else: 479 | labeled_nodes = label 480 | 481 | n = labeled_nodes.shape[0] 482 | train_num = int(n * train_prop) 483 | valid_num = int(n * valid_prop) 484 | 485 | perm = torch.as_tensor(np.random.permutation(n)) 486 | 487 | train_indices = perm[:train_num] 488 | val_indices = perm[train_num:train_num + valid_num] 489 | test_indices = perm[train_num + valid_num:] 490 | 491 | if not ignore_negative: 492 | return train_indices, val_indices, test_indices 493 | 494 | train_idx = labeled_nodes[train_indices] 495 | valid_idx = labeled_nodes[val_indices] 496 | test_idx = labeled_nodes[test_indices] 497 | 498 | split_idx = {'train': train_idx, 499 | 'valid': valid_idx, 500 | 'test': test_idx} 501 | else: 502 | # ipdb.set_trace() 503 | indices = [] 504 | for i in range(label.max()+1): 505 | index = torch.where((label == i))[0].view(-1) 506 | index = index[torch.randperm(index.size(0))] 507 | indices.append(index) 508 | 509 | percls_trn = int(train_prop/(label.max()+1)*len(label)) 510 | val_lb = int(valid_prop*len(label)) 511 | train_idx = torch.cat([i[:percls_trn] for i in indices], dim=0) 512 | rest_index = torch.cat([i[percls_trn:] for i in indices], dim=0) 513 | rest_index = rest_index[torch.randperm(rest_index.size(0))] 514 | valid_idx = rest_index[:val_lb] 515 | test_idx = rest_index[val_lb:] 516 | split_idx = {'train': train_idx, 517 | 'valid': valid_idx, 518 | 'test': test_idx} 519 | return split_idx 520 | 521 | 522 | def kfold_train_test_idx(label, k=5, train_prop=.5): 523 | """ Adapted from https://github.com/CUAI/Non-Homophily-Benchmarks""" 524 | """ randomly splits label into train/valid/test splits """ 525 | 526 | labeled_nodes = torch.where(label != -1)[0] 527 | 528 | n = labeled_nodes.shape[0] 529 | test_prop = 1 / k 530 | test_num = int(n * test_prop) 531 | train_num = int(n * train_prop) 532 | 533 | perm = torch.as_tensor(np.random.permutation(n)) 534 | 535 | split_idx_lst = [] 536 | test_start = 0 537 | for _ in range((k-1)): 538 | test_end = test_start + test_num 539 | test_indices = perm[test_start:test_end] 540 | temp_perm = torch.cat([perm[0:test_start], perm[test_end:]]) 541 | train_indices = temp_perm[0:train_num] 542 | val_indices = temp_perm[train_num:] 543 | 544 | train_idx = labeled_nodes[train_indices] 545 | valid_idx = labeled_nodes[val_indices] 546 | test_idx = labeled_nodes[test_indices] 547 | 548 | split_idx = {'train': train_idx, 549 | 'valid': valid_idx, 550 | 'test': test_idx} 551 | split_idx_lst.append(split_idx) 552 | test_start = test_end 553 | test_indices = perm[test_start:] 554 | temp_perm = perm[0:test_start] 555 | train_indices = temp_perm[0:train_num] 556 | val_indices = temp_perm[train_num:] 557 | 558 | train_idx = labeled_nodes[train_indices] 559 | valid_idx = labeled_nodes[val_indices] 560 | test_idx = labeled_nodes[test_indices] 561 | split_idx = {'train': train_idx, 562 | 'valid': valid_idx, 563 | 'test': test_idx} 564 | split_idx_lst.append(split_idx) 565 | return split_idx_lst 566 | 567 | -------------------------------------------------------------------------------- /ours/inference_time_test.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # coding: utf-8 3 | 4 | import os 5 | import time 6 | import math 7 | import torch 8 | import pickle 9 | import argparse 10 | import random 11 | import copy 12 | 13 | import numpy as np 14 | import os.path as osp 15 | import scipy.sparse as sp 16 | import torch_sparse 17 | import torch.nn as nn 18 | import torch.nn.functional as F 19 | import matplotlib.pyplot as plt 20 | 21 | from tqdm import tqdm 22 | 23 | from layers import * 24 | from models import * 25 | from preprocessing import * 26 | 27 | from convert_datasets_to_pygDataset import dataset_Hypergraph 28 | 29 | def perturb_hyperedges(data, prop, perturb_type='delete'): 30 | data_p = data 31 | edge_index = data_p.edge_index 32 | num_node = (edge_index[0,:].max()) - (edge_index[0,:].min()) + 1 33 | e_idxs = edge_index[1,:] - num_node 34 | num_edge = (edge_index[1,:].max()) - (edge_index[1,:].min()) + 1 35 | if((perturb_type == 'delete') or (perturb_type == 'replace')): 36 | p_num = num_edge * prop 37 | p_num = int(p_num) 38 | chosen_edges = torch.as_tensor(np.random.permutation(int(num_edge.cpu().numpy()))).to(edge_index.device) 39 | chosen_edges = chosen_edges[:p_num] 40 | if(perturb_type == 'delete'): 41 | data_p.edge_index = delete_edges(edge_index, chosen_edges, e_idxs) 42 | else: # replace = add + delete 43 | data_p.edge_index = replace_edges(edge_index, chosen_edges, e_idxs, num_node) 44 | elif(perturb_type == 'add'): 45 | # p_num = num_edge * prop / (1 - prop) 46 | p_num = num_edge * prop 47 | p_num = int(p_num) 48 | data_p.edge_index = add_edges(edge_index, p_num, num_node) 49 | return data_p 50 | 51 | def delete_edges(edge_index, chosen_edges, e_idxs): 52 | for i in range(chosen_edges.shape[0]): 53 | chosen_edge = chosen_edges[i] 54 | edge_index = edge_index[:, (e_idxs != chosen_edge)] 55 | e_idxs = e_idxs[(e_idxs != chosen_edge)] 56 | return edge_index 57 | 58 | def replace_edges(edge_index, chosen_edges, e_idxs, num_node): 59 | edge_index = delete_edges(edge_index, chosen_edges, e_idxs) 60 | edge_index = add_edges(edge_index, chosen_edges.shape[0], num_node) 61 | return edge_index 62 | 63 | def add_edges(edge_index, p_num, num_node): 64 | start_e_idx = edge_index[1,:].max() + 1 65 | edge_idxs = [edge_index] 66 | for i in range(p_num): 67 | new_edge = torch.as_tensor(np.random.choice(int(num_node.cpu().numpy()), 5, replace=False)).to(edge_index.device) 68 | for j in range(new_edge.shape[0]): 69 | edge_idx_i = torch.zeros([2,1]).to(edge_index.device) 70 | edge_idx_i[0,0] = new_edge[j] 71 | edge_idx_i[1,0] = start_e_idx 72 | edge_idxs.append(edge_idx_i) 73 | start_e_idx = start_e_idx + 1 74 | edge_idxs = torch.cat(edge_idxs, dim=1) 75 | return torch.tensor(edge_idxs, dtype=torch.int64) 76 | 77 | def parse_method(args, data): 78 | # Currently we don't set hyperparameters w.r.t. different dataset 79 | if args.method == 'AllSetTransformer': 80 | if args.LearnMask: 81 | model = SetGNN(args, data.norm) 82 | else: 83 | model = SetGNN(args) 84 | 85 | elif args.method == 'AllDeepSets': 86 | args.PMA = False 87 | args.aggregate = 'add' 88 | if args.LearnMask: 89 | model = SetGNN(args,data.norm) 90 | else: 91 | model = SetGNN(args) 92 | 93 | # elif args.method == 'SetGPRGNN': 94 | # model = SetGPRGNN(args) 95 | 96 | elif args.method == 'CEGCN': 97 | model = CEGCN(in_dim=args.num_features, 98 | hid_dim=args.MLP_hidden, # Use args.enc_hidden to control the number of hidden layers 99 | out_dim=args.num_classes, 100 | num_layers=args.All_num_layers, 101 | dropout=args.dropout, 102 | Normalization=args.normalization) 103 | 104 | elif args.method == 'CEGAT': 105 | model = CEGAT(in_dim=args.num_features, 106 | hid_dim=args.MLP_hidden, # Use args.enc_hidden to control the number of hidden layers 107 | out_dim=args.num_classes, 108 | num_layers=args.All_num_layers, 109 | heads=args.heads, 110 | output_heads=args.output_heads, 111 | dropout=args.dropout, 112 | Normalization=args.normalization) 113 | 114 | elif args.method == 'HyperGCN': 115 | # ipdb.set_trace() 116 | He_dict = get_HyperGCN_He_dict(data) 117 | model = HyperGCN(V=data.x.shape[0], 118 | E=He_dict, 119 | X=data.x, 120 | num_features=args.num_features, 121 | num_layers=args.All_num_layers, 122 | num_classses=args.num_classes, 123 | args=args 124 | ) 125 | 126 | elif args.method == 'HGNN': 127 | # model = HGNN(in_ch=args.num_features, 128 | # n_class=args.num_classes, 129 | # n_hid=args.MLP_hidden, 130 | # dropout=args.dropout) 131 | model = HCHA(args) 132 | 133 | elif args.method == 'HNHN': 134 | model = HNHN(args) 135 | 136 | elif args.method == 'HCHA': 137 | model = HCHA(args) 138 | 139 | elif args.method == 'MLP': 140 | model = MLP_model(args) 141 | 142 | elif args.method == 'res_mlp': 143 | model = MLP_res_model(args) 144 | 145 | elif args.method == 'trans': 146 | model = transformer_model(args) 147 | 148 | elif args.method == 'UniGCNII': 149 | if args.cuda in [0,1,2,3,4,5,6,7]: 150 | device = torch.device('cuda:'+str(args.cuda) if torch.cuda.is_available() else 'cpu') 151 | else: 152 | device = torch.device('cpu') 153 | (row, col), value = torch_sparse.from_scipy(data.edge_index) 154 | V, E = row, col 155 | V, E = V.to(device), E.to(device) 156 | model = UniGCNII(args, nfeat=args.num_features, nhid=args.MLP_hidden, nclass=args.num_classes, nlayer=args.All_num_layers, nhead=args.heads, 157 | V=V, E=E) 158 | # Below we can add different model, such as HyperGCN and so on 159 | return model 160 | 161 | 162 | class Logger(object): 163 | """ Adapted from https://github.com/snap-stanford/ogb/ """ 164 | 165 | def __init__(self, runs, info=None): 166 | self.info = info 167 | self.results = [[] for _ in range(runs)] 168 | 169 | def add_result(self, run, result): 170 | assert len(result) == 3 171 | assert run >= 0 and run < len(self.results) 172 | self.results[run].append(result) 173 | 174 | def print_statistics(self, run=None): 175 | if run is not None: 176 | result = 100 * torch.tensor(self.results[run]) 177 | argmax = result[:, 1].argmax().item() 178 | print(f'Run {run + 1:02d}:') 179 | print(f'Highest Train: {result[:, 0].max():.2f}') 180 | print(f'Highest Valid: {result[:, 1].max():.2f}') 181 | print(f' Final Train: {result[argmax, 0]:.2f}') 182 | print(f' Final Test: {result[argmax, 2]:.2f}') 183 | else: 184 | result = 100 * torch.tensor(self.results) 185 | 186 | best_results = [] 187 | for r in result: 188 | train1 = r[:, 0].max().item() 189 | valid = r[:, 1].max().item() 190 | train2 = r[r[:, 1].argmax(), 0].item() 191 | test = r[r[:, 1].argmax(), 2].item() 192 | best_results.append((train1, valid, train2, test)) 193 | 194 | best_result = torch.tensor(best_results) 195 | 196 | print(f'All runs:') 197 | r = best_result[:, 0] 198 | print(f'Highest Train: {r.mean():.2f} ± {r.std():.2f}') 199 | r = best_result[:, 1] 200 | print(f'Highest Valid: {r.mean():.2f} ± {r.std():.2f}') 201 | r = best_result[:, 2] 202 | print(f' Final Train: {r.mean():.2f} ± {r.std():.2f}') 203 | r = best_result[:, 3] 204 | print(f' Final Test: {r.mean():.2f} ± {r.std():.2f}') 205 | 206 | return best_result[:, 1], best_result[:, 3] 207 | 208 | def plot_result(self, run=None): 209 | plt.style.use('seaborn') 210 | if run is not None: 211 | result = 100 * torch.tensor(self.results[run]) 212 | x = torch.arange(result.shape[0]) 213 | plt.figure() 214 | print(f'Run {run + 1:02d}:') 215 | plt.plot(x, result[:, 0], x, result[:, 1], x, result[:, 2]) 216 | plt.legend(['Train', 'Valid', 'Test']) 217 | else: 218 | result = 100 * torch.tensor(self.results[0]) 219 | x = torch.arange(result.shape[0]) 220 | plt.figure() 221 | # print(f'Run {run + 1:02d}:') 222 | plt.plot(x, result[:, 0], x, result[:, 1], x, result[:, 2]) 223 | plt.legend(['Train', 'Valid', 'Test']) 224 | 225 | 226 | @torch.no_grad() 227 | def evaluate(model, data, split_idx, eval_func, result=None): 228 | if result is not None: 229 | out = result 230 | else: 231 | model.eval() 232 | out, _ = model(data) 233 | out = F.log_softmax(out, dim=1) 234 | 235 | train_acc = eval_func( 236 | data.y[split_idx['train']], out[split_idx['train']]) 237 | valid_acc = eval_func( 238 | data.y[split_idx['valid']], out[split_idx['valid']]) 239 | test_acc = eval_func( 240 | data.y[split_idx['test']], out[split_idx['test']]) 241 | 242 | # Also keep track of losses 243 | train_loss = F.nll_loss( 244 | out[split_idx['train']], data.y[split_idx['train']]) 245 | valid_loss = F.nll_loss( 246 | out[split_idx['valid']], data.y[split_idx['valid']]) 247 | test_loss = F.nll_loss( 248 | out[split_idx['test']], data.y[split_idx['test']]) 249 | return train_acc, valid_acc, test_acc, train_loss, valid_loss, test_loss, out 250 | 251 | 252 | def eval_acc(y_true, y_pred): 253 | acc_list = [] 254 | y_true = y_true.detach().cpu().numpy() 255 | y_pred = y_pred.argmax(dim=-1, keepdim=False).detach().cpu().numpy() 256 | 257 | # ipdb.set_trace() 258 | # for i in range(y_true.shape[1]): 259 | is_labeled = y_true == y_true 260 | correct = y_true[is_labeled] == y_pred[is_labeled] 261 | acc_list.append(float(np.sum(correct))/len(correct)) 262 | 263 | return sum(acc_list)/len(acc_list) 264 | 265 | def count_parameters(model): 266 | return sum(p.numel() for p in model.parameters() if p.requires_grad) 267 | 268 | 269 | def setup_seed(seed): 270 | torch.manual_seed(seed) 271 | torch.cuda.manual_seed_all(seed) 272 | np.random.seed(seed) 273 | random.seed(seed) 274 | torch.backends.cudnn.deterministic = True 275 | torch.use_deterministic_algorithms = True 276 | torch.cuda.current_device() 277 | torch.cuda._initialized = True 278 | 279 | # --- Main part of the training --- 280 | # # Part 0: Parse arguments 281 | 282 | 283 | """ 284 | 285 | """ 286 | 287 | if __name__ == '__main__': 288 | parser = argparse.ArgumentParser() 289 | parser.add_argument('--train_prop', type=float, default=0.5) 290 | parser.add_argument('--valid_prop', type=float, default=0.25) 291 | parser.add_argument('--dname', default='walmart-trips-100') 292 | # method in ['SetGNN','CEGCN','CEGAT','HyperGCN','HGNN','HCHA'] 293 | parser.add_argument('--method', default='AllSetTransformer') 294 | parser.add_argument('--epochs', default=1, type=int) 295 | # Number of runs for each split (test fix, only shuffle train/val) 296 | parser.add_argument('--runs', default=20, type=int) 297 | parser.add_argument('--cuda', default=0, choices=[-1,0,1,2,3,4,5,6,7], type=int) 298 | parser.add_argument('--dropout', default=0.5, type=float) 299 | parser.add_argument('--lr', default=0.001, type=float) 300 | parser.add_argument('--wd', default=0.0, type=float) 301 | # How many layers of full NLConvs 302 | parser.add_argument('--All_num_layers', default=2, type=int) 303 | parser.add_argument('--MLP_num_layers', default=2, 304 | type=int) # How many layers of encoder 305 | parser.add_argument('--MLP_hidden', default=64, 306 | type=int) # Encoder hidden units 307 | parser.add_argument('--Classifier_num_layers', default=1, 308 | type=int) # How many layers of decoder 309 | parser.add_argument('--Classifier_hidden', default=256, 310 | type=int) # Decoder hidden units 311 | parser.add_argument('--display_step', type=int, default=-1) 312 | parser.add_argument('--alpha', type=float, default=1) 313 | parser.add_argument('--aggregate', default='mean', choices=['sum', 'mean']) 314 | # ['all_one','deg_half_sym'] 315 | parser.add_argument('--normtype', default='all_one') 316 | parser.add_argument('--add_self_loop', action='store_false') 317 | # NormLayer for MLP. ['bn','ln','None'] 318 | parser.add_argument('--normalization', default='ln') 319 | parser.add_argument('--deepset_input_norm', default = True) 320 | parser.add_argument('--GPR', action='store_false') # skip all but last dec 321 | # skip all but last dec 322 | parser.add_argument('--LearnMask', action='store_false') 323 | parser.add_argument('--num_features', default=0, type=int) # Placeholder 324 | parser.add_argument('--num_classes', default=0, type=int) # Placeholder 325 | # Choose std for synthetic feature noise 326 | parser.add_argument('--feature_noise', default='1', type=str) 327 | parser.add_argument('--perturb_type', default='delete', type=str) 328 | parser.add_argument('--perturb_prop', default=0.0, type=float) 329 | parser.add_argument('--sth_type', default='max_s', type=str) 330 | # whether the he contain self node or not 331 | parser.add_argument('--exclude_self', action='store_true') 332 | parser.add_argument('--PMA', action='store_true') 333 | # Args for HyperGCN 334 | parser.add_argument('--HyperGCN_mediators', action='store_true') 335 | parser.add_argument('--HyperGCN_fast', action='store_true') 336 | # Args for Attentions: GAT and SetGNN 337 | parser.add_argument('--heads', default=1, type=int) # Placeholder 338 | parser.add_argument('--output_heads', default=1, type=int) # Placeholder 339 | # Args for HNHN 340 | parser.add_argument('--HNHN_alpha', default=-1.5, type=float) 341 | parser.add_argument('--HNHN_beta', default=-0.5, type=float) 342 | parser.add_argument('--HNHN_nonlinear_inbetween', default=True, type=bool) 343 | # Args for HCHA 344 | parser.add_argument('--HCHA_symdegnorm', action='store_true') 345 | # Args for UniGNN 346 | parser.add_argument('--UniGNN_use-norm', action="store_true", help='use norm in the final layer') 347 | parser.add_argument('--UniGNN_degV', default = 0) 348 | parser.add_argument('--UniGNN_degE', default = 0) 349 | parser.add_argument('--seed', type=int, default=1000, help='Random seed.') 350 | 351 | parser.set_defaults(PMA=True) # True: Use PMA. False: Use Deepsets. 352 | parser.set_defaults(add_self_loop=True) 353 | parser.set_defaults(exclude_self=False) 354 | parser.set_defaults(GPR=False) 355 | parser.set_defaults(LearnMask=False) 356 | parser.set_defaults(HyperGCN_mediators=True) 357 | parser.set_defaults(HyperGCN_fast=True) 358 | parser.set_defaults(HCHA_symdegnorm=False) 359 | 360 | # Use the line below for .py file 361 | args = parser.parse_args() 362 | # Use the line below for notebook 363 | # args = parser.parse_args([]) 364 | # args, _ = parser.parse_known_args() 365 | 366 | 367 | # # Part 1: Load data 368 | 369 | 370 | ### Load and preprocess data ### 371 | existing_dataset = ['20newsW100', 'ModelNet40', 'zoo', 372 | 'NTU2012', 'Mushroom', 373 | 'coauthor_cora', 'coauthor_dblp', 374 | 'yelp', 'amazon-reviews', 'walmart-trips', 'house-committees', 375 | 'walmart-trips-100', 'house-committees-100', 376 | 'cora', 'citeseer', 'pubmed'] 377 | 378 | synthetic_list = ['amazon-reviews', 'walmart-trips', 'house-committees', 'walmart-trips-100', 'house-committees-100'] 379 | 380 | if args.dname in existing_dataset: 381 | dname = args.dname 382 | f_noise = args.feature_noise 383 | if (f_noise is not None) and dname in synthetic_list: 384 | p2raw = '../data/AllSet_all_raw_data/' 385 | dataset = dataset_Hypergraph(name=dname, 386 | feature_noise=f_noise, 387 | p2raw = p2raw) 388 | else: 389 | if dname in ['cora', 'citeseer','pubmed']: 390 | p2raw = '../data/AllSet_all_raw_data/cocitation/' 391 | elif dname in ['coauthor_cora', 'coauthor_dblp']: 392 | p2raw = '../data/AllSet_all_raw_data/coauthorship/' 393 | elif dname in ['yelp']: 394 | p2raw = '../data/AllSet_all_raw_data/yelp/' 395 | else: 396 | p2raw = '../data/AllSet_all_raw_data/' 397 | dataset = dataset_Hypergraph(name=dname,root = '../data/pyg_data/hypergraph_dataset_updated/', 398 | p2raw = p2raw) 399 | data = dataset.data 400 | args.num_features = dataset.num_features 401 | args.num_classes = dataset.num_classes 402 | if args.dname in ['yelp', 'walmart-trips', 'house-committees', 'walmart-trips-100', 'house-committees-100']: 403 | # Shift the y label to start with 0 404 | args.num_classes = len(data.y.unique()) 405 | data.y = data.y - data.y.min() 406 | if not hasattr(data, 'n_x'): 407 | data.n_x = torch.tensor([data.x.shape[0]]) 408 | if not hasattr(data, 'num_hyperedges'): 409 | # note that we assume the he_id is consecutive. 410 | data.num_hyperedges = torch.tensor( 411 | [data.edge_index[0].max()-data.n_x[0]+1]) 412 | 413 | setup_seed(args.seed) 414 | data = ExtractV2E(data) 415 | data = perturb_hyperedges(data, args.perturb_prop, args.perturb_type) 416 | # Get splits 417 | split_idx_lst = [] 418 | for run in range(args.runs): 419 | split_idx = rand_train_test_idx( 420 | data.y, train_prop=args.train_prop, valid_prop=args.valid_prop) 421 | split_idx_lst.append(split_idx) 422 | 423 | 424 | # # Part 2: Load model 425 | 426 | 427 | model = parse_method(args, data) 428 | # put things to device 429 | if args.cuda in [0,1,2,3,4,5,6,7]: 430 | device = torch.device('cuda:'+str(args.cuda) 431 | if torch.cuda.is_available() else 'cpu') 432 | else: 433 | device = torch.device('cpu') 434 | 435 | model, data = model.to(device), data.to(device) 436 | if args.method == 'UniGCNII': 437 | args.UniGNN_degV = args.UniGNN_degV.to(device) 438 | args.UniGNN_degE = args.UniGNN_degE.to(device) 439 | 440 | num_params = count_parameters(model) 441 | 442 | 443 | # # Part 3: Main. Training + Evaluation 444 | 445 | 446 | logger = Logger(args.runs, args) 447 | 448 | criterion = nn.NLLLoss() 449 | eval_func = eval_acc 450 | 451 | model.train() 452 | # print('MODEL:', model) 453 | 454 | ### Training loop ### 455 | runtime_list = [] 456 | for run in tqdm(range(args.runs)): 457 | split_idx = split_idx_lst[run] 458 | train_idx = split_idx['train'].to(device) 459 | setup_seed(run) 460 | model.reset_parameters() 461 | if args.method == 'UniGCNII': 462 | optimizer = torch.optim.Adam([ 463 | dict(params=model.reg_params, weight_decay=0.01), 464 | dict(params=model.non_reg_params, weight_decay=5e-4) 465 | ], lr=0.01) 466 | else: 467 | optimizer = torch.optim.Adam(model.parameters(), lr=args.lr, weight_decay=args.wd) 468 | # This is for HNHN only 469 | # if args.method == 'HNHN': 470 | # scheduler = torch.optim.lr_scheduler.StepLR(optimizer,step_size=100, gamma=0.51) 471 | model.eval() 472 | if ((args.method == 'UniGCNII') or (args.method == 'HyperGCN')): 473 | test_flag = True 474 | data_input = [data, test_flag] 475 | else: 476 | data_input = data 477 | start_time = time.time() 478 | out, _ = model(data_input) 479 | out = F.log_softmax(out, dim=1) 480 | end_time = time.time() 481 | runtime_list.append((end_time - start_time)*1000) 482 | 483 | # logger.print_statistics(run) 484 | 485 | ### Save results ### 486 | avg_time, std_time = np.mean(runtime_list), np.std(runtime_list) 487 | res_root1 = 'log_time/{}'.format(args.method)#method 488 | if not osp.isdir(res_root1): 489 | os.makedirs(res_root1) 490 | res_root = '{}/{}'.format(res_root1, args.perturb_type) 491 | if not osp.isdir(res_root): 492 | os.makedirs(res_root) 493 | 494 | filename = f'{res_root}/{args.dname}_noise_{args.feature_noise}.csv' 495 | print(f"Saving results to {filename}") 496 | with open(filename, 'a+') as write_obj: 497 | cur_line = f'{args.runs}\n' 498 | cur_line += f'{avg_time}\n' 499 | cur_line += f'{std_time}\n' 500 | write_obj.write(cur_line) 501 | ''' 502 | all_args_file = f'{res_root}/all_args_{args.dname}_noise_{args.feature_noise}.csv' 503 | with open(all_args_file, 'a+') as f: 504 | f.write(str(args)) 505 | f.write('\n') 506 | res_root_2 = 'log_time_store/{}'.format(args.method) 507 | if not osp.isdir(res_root_2): 508 | os.makedirs(res_root_2) 509 | filename = f'{res_root_2}/{args.dname}_noise_{args.feature_noise}_alpha_{args.alpha}.pickle' 510 | data = { 511 | 'runtime_list': runtime_list 512 | } 513 | with open(filename, 'wb') as handle: 514 | pickle.dump(data, handle, protocol=4) 515 | ''' 516 | print(len(runtime_list)) 517 | print('All done! Exit python code') 518 | quit() 519 | 520 | -------------------------------------------------------------------------------- /baselines_hypergnn/inference_time_test.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # coding: utf-8 3 | 4 | import os 5 | import time 6 | # import math 7 | import torch 8 | import pickle 9 | import argparse 10 | import random 11 | import copy 12 | 13 | import numpy as np 14 | import os.path as osp 15 | import scipy.sparse as sp 16 | import torch_sparse 17 | import torch.nn as nn 18 | import torch.nn.functional as F 19 | import matplotlib.pyplot as plt 20 | 21 | from tqdm import tqdm 22 | 23 | from layers import * 24 | from models import * 25 | from preprocessing import * 26 | 27 | from convert_datasets_to_pygDataset import dataset_Hypergraph 28 | 29 | 30 | def parse_method(args, data, data_p): 31 | # Currently we don't set hyperparameters w.r.t. different dataset 32 | if args.method == 'AllSetTransformer': 33 | if args.LearnMask: 34 | model = SetGNN(args, data.norm) 35 | else: 36 | model = SetGNN(args) 37 | 38 | elif args.method == 'AllDeepSets': 39 | args.PMA = False 40 | args.aggregate = 'add' 41 | if args.LearnMask: 42 | model = SetGNN(args,data.norm) 43 | else: 44 | model = SetGNN(args) 45 | 46 | # elif args.method == 'SetGPRGNN': 47 | # model = SetGPRGNN(args) 48 | 49 | elif args.method == 'CEGCN': 50 | model = CEGCN(in_dim=args.num_features, 51 | hid_dim=args.MLP_hidden, # Use args.enc_hidden to control the number of hidden layers 52 | out_dim=args.num_classes, 53 | num_layers=args.All_num_layers, 54 | dropout=args.dropout, 55 | Normalization=args.normalization) 56 | 57 | elif args.method == 'CEGAT': 58 | model = CEGAT(in_dim=args.num_features, 59 | hid_dim=args.MLP_hidden, # Use args.enc_hidden to control the number of hidden layers 60 | out_dim=args.num_classes, 61 | num_layers=args.All_num_layers, 62 | heads=args.heads, 63 | output_heads=args.output_heads, 64 | dropout=args.dropout, 65 | Normalization=args.normalization) 66 | 67 | elif args.method == 'HyperGCN': 68 | # ipdb.set_trace() 69 | He_dict = get_HyperGCN_He_dict(data) 70 | He_dict_p = get_HyperGCN_He_dict(data_p) 71 | model = HyperGCN(V=data.x.shape[0], 72 | E=He_dict, 73 | E_p=He_dict_p, 74 | X=data.x, 75 | num_features=args.num_features, 76 | num_layers=args.All_num_layers, 77 | num_classses=args.num_classes, 78 | args=args 79 | ) 80 | 81 | elif args.method == 'HGNN': 82 | # model = HGNN(in_ch=args.num_features, 83 | # n_class=args.num_classes, 84 | # n_hid=args.MLP_hidden, 85 | # dropout=args.dropout) 86 | model = HCHA(args) 87 | 88 | elif args.method == 'HNHN': 89 | model = HNHN(args) 90 | 91 | elif args.method == 'HCHA': 92 | model = HCHA(args) 93 | 94 | elif args.method == 'MLP': 95 | model = MLP_model(args) 96 | elif args.method == 'UniGCNII': 97 | if args.cuda in [0,1,2,3,4,5,6,7]: 98 | device = torch.device('cuda:'+str(args.cuda) if torch.cuda.is_available() else 'cpu') 99 | else: 100 | device = torch.device('cpu') 101 | (row, col), value = torch_sparse.from_scipy(data.edge_index) 102 | V, E = row, col 103 | V, E = V.to(device), E.to(device) 104 | 105 | (row_p, col_p), value_p = torch_sparse.from_scipy(data_p.edge_index) 106 | V_p, E_p = row_p, col_p 107 | V_p, E_p = V_p.to(device), E_p.to(device) 108 | model = UniGCNII(args, nfeat=args.num_features, nhid=args.MLP_hidden, nclass=args.num_classes, nlayer=args.All_num_layers, nhead=args.heads, 109 | V=V, E=E, V_p=V_p, E_p=E_p) 110 | # Below we can add different model, such as HyperGCN and so on 111 | return model 112 | 113 | 114 | class Logger(object): 115 | """ Adapted from https://github.com/snap-stanford/ogb/ """ 116 | 117 | def __init__(self, runs, info=None): 118 | self.info = info 119 | self.results = [[] for _ in range(runs)] 120 | 121 | def add_result(self, run, result): 122 | assert len(result) == 3 123 | assert run >= 0 and run < len(self.results) 124 | self.results[run].append(result) 125 | 126 | def print_statistics(self, run=None): 127 | if run is not None: 128 | result = 100 * torch.tensor(self.results[run]) 129 | argmax = result[:, 1].argmax().item() 130 | print(f'Run {run + 1:02d}:') 131 | print(f'Highest Train: {result[:, 0].max():.2f}') 132 | print(f'Highest Valid: {result[:, 1].max():.2f}') 133 | print(f' Final Train: {result[argmax, 0]:.2f}') 134 | print(f' Final Test: {result[argmax, 2]:.2f}') 135 | else: 136 | result = 100 * torch.tensor(self.results) 137 | 138 | best_results = [] 139 | for r in result: 140 | train1 = r[:, 0].max().item() 141 | valid = r[:, 1].max().item() 142 | train2 = r[r[:, 1].argmax(), 0].item() 143 | test = r[r[:, 1].argmax(), 2].item() 144 | best_results.append((train1, valid, train2, test)) 145 | 146 | best_result = torch.tensor(best_results) 147 | 148 | print(f'All runs:') 149 | r = best_result[:, 0] 150 | print(f'Highest Train: {r.mean():.2f} ± {r.std():.2f}') 151 | r = best_result[:, 1] 152 | print(f'Highest Valid: {r.mean():.2f} ± {r.std():.2f}') 153 | r = best_result[:, 2] 154 | print(f' Final Train: {r.mean():.2f} ± {r.std():.2f}') 155 | r = best_result[:, 3] 156 | print(f' Final Test: {r.mean():.2f} ± {r.std():.2f}') 157 | 158 | return best_result[:, 1], best_result[:, 3] 159 | 160 | def plot_result(self, run=None): 161 | plt.style.use('seaborn') 162 | if run is not None: 163 | result = 100 * torch.tensor(self.results[run]) 164 | x = torch.arange(result.shape[0]) 165 | plt.figure() 166 | print(f'Run {run + 1:02d}:') 167 | plt.plot(x, result[:, 0], x, result[:, 1], x, result[:, 2]) 168 | plt.legend(['Train', 'Valid', 'Test']) 169 | else: 170 | result = 100 * torch.tensor(self.results[0]) 171 | x = torch.arange(result.shape[0]) 172 | plt.figure() 173 | # print(f'Run {run + 1:02d}:') 174 | plt.plot(x, result[:, 0], x, result[:, 1], x, result[:, 2]) 175 | plt.legend(['Train', 'Valid', 'Test']) 176 | 177 | 178 | @torch.no_grad() 179 | def evaluate(args, model, data, split_idx, eval_func, result=None): 180 | test_flag = True 181 | if ((args.method == 'UniGCNII') or (args.method == 'HyperGCN')): 182 | data_input = [data, test_flag] 183 | else: 184 | data_input = data 185 | if result is not None: 186 | out = result 187 | else: 188 | model.eval() 189 | out = model(data_input) 190 | out = F.log_softmax(out, dim=1) 191 | train_acc = eval_func( 192 | data.y[split_idx['train']], out[split_idx['train']]) 193 | valid_acc = eval_func( 194 | data.y[split_idx['valid']], out[split_idx['valid']]) 195 | test_acc = eval_func( 196 | data.y[split_idx['test']], out[split_idx['test']]) 197 | 198 | # Also keep track of losses 199 | train_loss = F.nll_loss( 200 | out[split_idx['train']], data.y[split_idx['train']]) 201 | valid_loss = F.nll_loss( 202 | out[split_idx['valid']], data.y[split_idx['valid']]) 203 | test_loss = F.nll_loss( 204 | out[split_idx['test']], data.y[split_idx['test']]) 205 | return train_acc, valid_acc, test_acc, train_loss, valid_loss, test_loss, out 206 | 207 | 208 | def eval_acc(y_true, y_pred): 209 | acc_list = [] 210 | y_true = y_true.detach().cpu().numpy() 211 | y_pred = y_pred.argmax(dim=-1, keepdim=False).detach().cpu().numpy() 212 | 213 | # ipdb.set_trace() 214 | # for i in range(y_true.shape[1]): 215 | is_labeled = y_true == y_true 216 | correct = y_true[is_labeled] == y_pred[is_labeled] 217 | acc_list.append(float(np.sum(correct))/len(correct)) 218 | 219 | return sum(acc_list)/len(acc_list) 220 | 221 | def count_parameters(model): 222 | return sum(p.numel() for p in model.parameters() if p.requires_grad) 223 | 224 | def perturb_hyperedges(data, prop, perturb_type='delete'): 225 | data_p = copy.deepcopy(data) 226 | edge_index = data_p.edge_index 227 | num_node = (edge_index[0,:].max()) - (edge_index[0,:].min()) + 1 228 | e_idxs = edge_index[1,:] - num_node 229 | num_edge = (edge_index[1,:].max()) - (edge_index[1,:].min()) + 1 230 | if((perturb_type == 'delete') or (perturb_type == 'replace')): 231 | p_num = num_edge * prop 232 | p_num = int(p_num) 233 | chosen_edges = torch.as_tensor(np.random.permutation(int(num_edge.cpu().numpy()))).to(edge_index.device) 234 | chosen_edges = chosen_edges[:p_num] 235 | if(perturb_type == 'delete'): 236 | data_p.edge_index = delete_edges(edge_index, chosen_edges, e_idxs) 237 | else: # replace = add + delete 238 | data_p.edge_index = replace_edges(edge_index, chosen_edges, e_idxs, num_node) 239 | elif(perturb_type == 'add'): 240 | # p_num = num_edge * prop / (1 - prop) 241 | p_num = num_edge * prop 242 | p_num = int(p_num) 243 | data_p.edge_index = add_edges(edge_index, p_num, num_node) 244 | return data_p 245 | 246 | def delete_edges(edge_index, chosen_edges, e_idxs): 247 | for i in range(chosen_edges.shape[0]): 248 | chosen_edge = chosen_edges[i] 249 | edge_index = edge_index[:, (e_idxs != chosen_edge)] 250 | e_idxs = e_idxs[(e_idxs != chosen_edge)] 251 | return edge_index 252 | 253 | def replace_edges(edge_index, chosen_edges, e_idxs, num_node): 254 | edge_index = delete_edges(edge_index, chosen_edges, e_idxs) 255 | edge_index = add_edges(edge_index, chosen_edges.shape[0], num_node) 256 | return edge_index 257 | 258 | def add_edges(edge_index, p_num, num_node): 259 | start_e_idx = edge_index[1,:].max() + 1 260 | edge_idxs = [edge_index] 261 | for i in range(p_num): 262 | new_edge = torch.as_tensor(np.random.choice(int(num_node.cpu().numpy()), 5, replace=False)).to(edge_index.device) 263 | for j in range(new_edge.shape[0]): 264 | edge_idx_i = torch.zeros([2,1]).to(edge_index.device) 265 | edge_idx_i[0,0] = new_edge[j] 266 | edge_idx_i[1,0] = start_e_idx 267 | edge_idxs.append(edge_idx_i) 268 | start_e_idx = start_e_idx + 1 269 | edge_idxs = torch.cat(edge_idxs, dim=1) 270 | return torch.tensor(edge_idxs, dtype=torch.int64) 271 | 272 | def unignn_ini_ve(data, device): 273 | data = ConstructH(data) 274 | data.edge_index = sp.csr_matrix(data.edge_index) 275 | # Compute degV and degE 276 | (row, col), value = torch_sparse.from_scipy(data.edge_index) 277 | V, E = row, col 278 | return V, E 279 | 280 | def unignn_get_deg(V, E): 281 | degV = torch.from_numpy(data.edge_index.sum(1)).view(-1, 1).float().to(device) 282 | from torch_scatter import scatter 283 | degE = scatter(degV[V], E, dim=0, reduce='mean') 284 | degE = degE.pow(-0.5) 285 | degV = degV.pow(-0.5) 286 | degV[torch.isinf(degV)] = 1 287 | return degV, degE 288 | 289 | def setup_seed(seed): 290 | torch.manual_seed(seed) 291 | torch.cuda.manual_seed_all(seed) 292 | np.random.seed(seed) 293 | random.seed(seed) 294 | torch.backends.cudnn.deterministic = True 295 | torch.use_deterministic_algorithms = True 296 | torch.cuda.current_device() 297 | torch.cuda._initialized = True 298 | 299 | # --- Main part of the training --- 300 | # # Part 0: Parse arguments 301 | 302 | 303 | """ 304 | 305 | """ 306 | 307 | if __name__ == '__main__': 308 | parser = argparse.ArgumentParser() 309 | parser.add_argument('--train_prop', type=float, default=0.5) 310 | parser.add_argument('--valid_prop', type=float, default=0.25) 311 | parser.add_argument('--dname', default='walmart-trips-100') 312 | # method in ['SetGNN','CEGCN','CEGAT','HyperGCN','HGNN','HCHA'] 313 | parser.add_argument('--method', default='AllSetTransformer') 314 | parser.add_argument('--epochs', default=1, type=int) 315 | parser.add_argument('--seed', default=1, type=int) 316 | # Number of runs for each split (test fix, only shuffle train/val) 317 | parser.add_argument('--runs', default=20, type=int) 318 | parser.add_argument('--cuda', default=0, choices=[-1,0,1,2,3,4,5,6,7], type=int) 319 | parser.add_argument('--dropout', default=0.5, type=float) 320 | parser.add_argument('--lr', default=0.001, type=float) 321 | parser.add_argument('--wd', default=0.0, type=float) 322 | # How many layers of full NLConvs 323 | parser.add_argument('--All_num_layers', default=2, type=int) 324 | parser.add_argument('--MLP_num_layers', default=2, 325 | type=int) # How many layers of encoder 326 | parser.add_argument('--MLP_hidden', default=64, 327 | type=int) # Encoder hidden units 328 | parser.add_argument('--Classifier_num_layers', default=2, 329 | type=int) # How many layers of decoder 330 | parser.add_argument('--Classifier_hidden', default=64, 331 | type=int) # Decoder hidden units 332 | parser.add_argument('--display_step', type=int, default=-1) 333 | parser.add_argument('--aggregate', default='mean', choices=['sum', 'mean']) 334 | # ['all_one','deg_half_sym'] 335 | parser.add_argument('--normtype', default='all_one') 336 | parser.add_argument('--add_self_loop', action='store_false') 337 | # NormLayer for MLP. ['bn','ln','None'] 338 | parser.add_argument('--normalization', default='ln') 339 | parser.add_argument('--deepset_input_norm', default = True) 340 | parser.add_argument('--GPR', action='store_false') # skip all but last dec 341 | # skip all but last dec 342 | parser.add_argument('--LearnMask', action='store_false') 343 | parser.add_argument('--num_features', default=0, type=int) # Placeholder 344 | parser.add_argument('--num_classes', default=0, type=int) # Placeholder 345 | # Choose std for synthetic feature noise 346 | parser.add_argument('--feature_noise', default='1', type=str) 347 | parser.add_argument('--perturb_type', default='delete', type=str) 348 | parser.add_argument('--perturb_prop', default=0.0, type=float) 349 | # whether the he contain self node or not 350 | parser.add_argument('--exclude_self', action='store_true') 351 | parser.add_argument('--PMA', action='store_true') 352 | # Args for HyperGCN 353 | parser.add_argument('--HyperGCN_mediators', action='store_true') 354 | parser.add_argument('--HyperGCN_fast', action='store_true') 355 | # Args for Attentions: GAT and SetGNN 356 | parser.add_argument('--heads', default=1, type=int) # Placeholder 357 | parser.add_argument('--output_heads', default=1, type=int) # Placeholder 358 | # Args for HNHN 359 | parser.add_argument('--HNHN_alpha', default=-1.5, type=float) 360 | parser.add_argument('--HNHN_beta', default=-0.5, type=float) 361 | parser.add_argument('--HNHN_nonlinear_inbetween', default=True, type=bool) 362 | # Args for HCHA 363 | parser.add_argument('--HCHA_symdegnorm', action='store_true') 364 | # Args for UniGNN 365 | parser.add_argument('--UniGNN_use-norm', action="store_true", help='use norm in the final layer') 366 | parser.add_argument('--UniGNN_degV', default = 0) 367 | parser.add_argument('--UniGNN_degE', default = 0) 368 | 369 | parser.set_defaults(PMA=True) # True: Use PMA. False: Use Deepsets. 370 | parser.set_defaults(add_self_loop=True) 371 | parser.set_defaults(exclude_self=False) 372 | parser.set_defaults(GPR=False) 373 | parser.set_defaults(LearnMask=False) 374 | parser.set_defaults(HyperGCN_mediators=True) 375 | parser.set_defaults(HyperGCN_fast=True) 376 | parser.set_defaults(HCHA_symdegnorm=False) 377 | 378 | # Use the line below for .py file 379 | args = parser.parse_args() 380 | # Use the line below for notebook 381 | # args = parser.parse_args([]) 382 | # args, _ = parser.parse_known_args() 383 | 384 | 385 | # # Part 1: Load data 386 | 387 | 388 | ### Load and preprocess data ### 389 | existing_dataset = ['20newsW100', 'ModelNet40', 'zoo', 390 | 'NTU2012', 'Mushroom', 391 | 'coauthor_cora', 'coauthor_dblp', 392 | 'yelp', 'amazon-reviews', 'walmart-trips', 'house-committees', 393 | 'walmart-trips-100', 'house-committees-100', 394 | 'cora', 'citeseer', 'pubmed'] 395 | 396 | synthetic_list = ['amazon-reviews', 'walmart-trips', 'house-committees', 'walmart-trips-100', 'house-committees-100'] 397 | 398 | if args.dname in existing_dataset: 399 | dname = args.dname 400 | f_noise = args.feature_noise 401 | if (f_noise is not None) and dname in synthetic_list: 402 | p2raw = '../data/AllSet_all_raw_data/' 403 | dataset = dataset_Hypergraph(name=dname, 404 | feature_noise=f_noise, 405 | p2raw = p2raw) 406 | else: 407 | if dname in ['cora', 'citeseer','pubmed']: 408 | p2raw = '../data/AllSet_all_raw_data/cocitation/' 409 | elif dname in ['coauthor_cora', 'coauthor_dblp']: 410 | p2raw = '../data/AllSet_all_raw_data/coauthorship/' 411 | elif dname in ['yelp']: 412 | p2raw = '../data/AllSet_all_raw_data/yelp/' 413 | else: 414 | p2raw = '../data/AllSet_all_raw_data/' 415 | dataset = dataset_Hypergraph(name=dname,root = '../data/pyg_data/hypergraph_dataset_updated/', 416 | p2raw = p2raw) 417 | data = dataset.data 418 | args.num_features = dataset.num_features 419 | args.num_classes = dataset.num_classes 420 | if args.dname in ['yelp', 'walmart-trips', 'house-committees', 'walmart-trips-100', 'house-committees-100']: 421 | # Shift the y label to start with 0 422 | args.num_classes = len(data.y.unique()) 423 | data.y = data.y - data.y.min() 424 | if not hasattr(data, 'n_x'): 425 | data.n_x = torch.tensor([data.x.shape[0]]) 426 | if not hasattr(data, 'num_hyperedges'): 427 | # note that we assume the he_id is consecutive. 428 | data.num_hyperedges = torch.tensor( 429 | [data.edge_index[0].max()-data.n_x[0]+1]) 430 | 431 | # ipdb.set_trace() 432 | # Preprocessing 433 | # if args.method in ['SetGNN', 'SetGPRGNN', 'SetGNN-DeepSet']: 434 | setup_seed(1000) 435 | if args.method in ['AllSetTransformer', 'AllDeepSets']: 436 | data = ExtractV2E(data) 437 | data_p = perturb_hyperedges(data, args.perturb_prop, args.perturb_type) 438 | if args.add_self_loop: 439 | data = Add_Self_Loops(data) 440 | data_p = Add_Self_Loops(data_p) 441 | if args.exclude_self: 442 | data = expand_edge_index(data) 443 | data_p = expand_edge_index(data_p) 444 | 445 | # Compute deg normalization: option in ['all_one','deg_half_sym'] (use args.normtype) 446 | # data.norm = torch.ones_like(data.edge_index[0]) 447 | data = norm_contruction(data, option=args.normtype) 448 | data_p = norm_contruction(data_p, option=args.normtype) 449 | 450 | elif args.method in ['CEGCN', 'CEGAT']: 451 | data = ExtractV2E(data) 452 | data_p = perturb_hyperedges(data, args.perturb_prop, args.perturb_type) 453 | data = ConstructV2V(data) 454 | data = norm_contruction(data, TYPE='V2V') 455 | data_p = ConstructV2V(data_p) 456 | data_p = norm_contruction(data_p, TYPE='V2V') 457 | 458 | elif args.method in ['HyperGCN']: 459 | data = ExtractV2E(data) 460 | data_p = perturb_hyperedges(data, args.perturb_prop, args.perturb_type) 461 | # ipdb.set_trace() 462 | # Feature normalization, default option in HyperGCN 463 | # X = data.x 464 | # X = sp.csr_matrix(utils.normalise(np.array(X)), dtype=np.float32) 465 | # X = torch.FloatTensor(np.array(X.todense())) 466 | # data.x = X 467 | 468 | # elif args.method in ['HGNN']: 469 | # data = ExtractV2E(data) 470 | # if args.add_self_loop: 471 | # data = Add_Self_Loops(data) 472 | # data = ConstructH(data) 473 | # data = generate_G_from_H(data) 474 | 475 | elif args.method in ['HNHN']: 476 | data = ExtractV2E(data) 477 | data_p = perturb_hyperedges(data, args.perturb_prop, args.perturb_type) 478 | if args.add_self_loop: 479 | data_p = Add_Self_Loops(data_p) 480 | data = Add_Self_Loops(data) 481 | 482 | H = ConstructH_HNHN(data) 483 | data = generate_norm_HNHN(H, data, args) 484 | data.edge_index[1] -= data.edge_index[1].min() 485 | 486 | H_p = ConstructH_HNHN(data_p) 487 | data_p = generate_norm_HNHN(H_p, data_p, args) 488 | data_p.edge_index[1] -= data_p.edge_index[1].min() 489 | 490 | elif args.method in ['HCHA', 'HGNN']: 491 | data = ExtractV2E(data) 492 | data_p = perturb_hyperedges(data, args.perturb_prop, args.perturb_type) 493 | if args.add_self_loop: 494 | data = Add_Self_Loops(data) 495 | data_p = Add_Self_Loops(data_p) 496 | # Make the first he_id to be 0 497 | data_p.edge_index[1] -= data_p.edge_index[1].min() 498 | data.edge_index[1] -= data.edge_index[1].min() 499 | 500 | elif args.method in ['UniGCNII']: 501 | data = ExtractV2E(data) 502 | data_p = perturb_hyperedges(data, args.perturb_prop, args.perturb_type) 503 | if args.add_self_loop: 504 | data = Add_Self_Loops(data) 505 | data_p = Add_Self_Loops(data_p) 506 | 507 | if args.cuda in [0,1,2,3,4,5,6,7]: 508 | device = torch.device('cuda:'+str(args.cuda) if torch.cuda.is_available() else 'cpu') 509 | else: 510 | device = torch.device('cpu') 511 | 512 | V, E = unignn_ini_ve(data, args) 513 | V, E = V.to(device), E.to(device) 514 | 515 | V_p, E_p = unignn_ini_ve(data_p, args) 516 | V_p, E_p = V_p.to(device), E_p.to(device) 517 | 518 | args.UniGNN_degV, args.UniGNN_degE = unignn_get_deg(V, E) 519 | args.UniGNN_degV_p, args.UniGNN_degE_p = unignn_get_deg(V_p, E_p) 520 | 521 | V, E = V.cpu(), E.cpu() 522 | del V 523 | del E 524 | V_p, E_p = V_p.cpu(), E_p.cpu() 525 | del V_p 526 | del E_p 527 | 528 | # Get splits 529 | split_idx_lst = [] 530 | for run in range(args.runs): 531 | split_idx = rand_train_test_idx( 532 | data.y, train_prop=args.train_prop, valid_prop=args.valid_prop) 533 | split_idx_lst.append(split_idx) 534 | 535 | #split_idx_lst = kfold_train_test_idx(data.y, args.runs) 536 | # # Part 2: Load model 537 | 538 | model = parse_method(args, data, data_p) 539 | # put things to device 540 | if args.cuda in [0,1,2,3,4,5,6,7]: 541 | device = torch.device('cuda:'+str(args.cuda) 542 | if torch.cuda.is_available() else 'cpu') 543 | else: 544 | device = torch.device('cpu') 545 | 546 | model, data, data_p = model.to(device), data.to(device), data_p.to(device) 547 | if args.method == 'UniGCNII': 548 | args.UniGNN_degV = args.UniGNN_degV.to(device) 549 | args.UniGNN_degE = args.UniGNN_degE.to(device) 550 | args.UniGNN_degV_p = args.UniGNN_degV_p.to(device) 551 | args.UniGNN_degE_p = args.UniGNN_degE_p.to(device) 552 | 553 | num_params = count_parameters(model) 554 | # # Part 3: Main. Training + Evaluation 555 | 556 | 557 | logger = Logger(args.runs, args) 558 | 559 | criterion = nn.NLLLoss() 560 | eval_func = eval_acc 561 | 562 | model.train() 563 | # print('MODEL:', model) 564 | 565 | ### Training loop ### 566 | runtime_list = [] 567 | for run in tqdm(range(args.runs)): 568 | split_idx = split_idx_lst[run] 569 | train_idx = split_idx['train'].to(device) 570 | setup_seed(run) 571 | model.reset_parameters() 572 | if args.method == 'UniGCNII': 573 | optimizer = torch.optim.Adam([ 574 | dict(params=model.reg_params, weight_decay=0.01), 575 | dict(params=model.non_reg_params, weight_decay=5e-4) 576 | ], lr=0.01) 577 | else: 578 | optimizer = torch.optim.Adam(model.parameters(), lr=args.lr, weight_decay=args.wd) 579 | # This is for HNHN only 580 | # if args.method == 'HNHN': 581 | # scheduler = torch.optim.lr_scheduler.StepLR(optimizer,step_size=100, gamma=0.51) 582 | model.eval() 583 | if ((args.method == 'UniGCNII') or (args.method == 'HyperGCN')): 584 | test_flag = True 585 | data_input = [data, test_flag] 586 | else: 587 | data_input = data 588 | start_time = time.time() 589 | out = model(data_input) 590 | out = F.log_softmax(out, dim=1) 591 | end_time = time.time() 592 | runtime_list.append(((end_time - start_time)*1000)) 593 | 594 | # logger.print_statistics(run) 595 | 596 | ### Save results ### 597 | avg_time, std_time = np.mean(runtime_list), np.std(runtime_list) 598 | res_root1 = 'log_time/{}'.format(args.method)#method 599 | if not osp.isdir(res_root1): 600 | os.makedirs(res_root1) 601 | res_root = '{}/{}'.format(res_root1, args.perturb_type) 602 | if not osp.isdir(res_root): 603 | os.makedirs(res_root) 604 | 605 | filename = f'{res_root}/{args.dname}_noise_{args.feature_noise}.csv' 606 | print(f"Saving results to {filename}") 607 | with open(filename, 'a+') as write_obj: 608 | cur_line = f'{args.runs}\n' 609 | cur_line += f'{avg_time}\n' 610 | cur_line += f'{std_time}\n' 611 | write_obj.write(cur_line) 612 | ''' 613 | all_args_file = f'{res_root}/all_args_{args.dname}_noise_{args.feature_noise}.csv' 614 | with open(all_args_file, 'a+') as f: 615 | f.write(str(args)) 616 | f.write('\n') 617 | res_root_2 = 'log_time_store/{}'.format(args.method) 618 | if not osp.isdir(res_root_2): 619 | os.makedirs(res_root_2) 620 | filename = f'{res_root_2}/{args.dname}_noise_{args.feature_noise}_alpha_{args.alpha}.pickle' 621 | data = { 622 | 'runtime_list': runtime_list 623 | } 624 | with open(filename, 'wb') as handle: 625 | pickle.dump(data, handle, protocol=4) 626 | 627 | ''' 628 | print(len(runtime_list)) 629 | print('All done! Exit python code') 630 | quit() 631 | -------------------------------------------------------------------------------- /baselines_hypergnn/layers.py: -------------------------------------------------------------------------------- 1 | #! /usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | # vim:fenc=utf-8 4 | # 5 | # Copyright © 2021 6 | # 7 | # Distributed under terms of the MIT license. 8 | 9 | """ 10 | This script contains layers used in AllSet and all other tested methods. 11 | """ 12 | 13 | import math 14 | import torch 15 | 16 | import torch.nn as nn 17 | import torch.nn.functional as F 18 | 19 | from torch import Tensor 20 | from torch.nn import Linear 21 | from torch.nn import Parameter 22 | from torch_geometric.nn.conv import MessagePassing 23 | from torch_geometric.utils import softmax 24 | from torch_scatter import scatter_add, scatter 25 | from torch_geometric.typing import Adj, Size, OptTensor 26 | from typing import Optional 27 | 28 | # This part is for PMA. 29 | # Modified from GATConv in pyg. 30 | # Method for initialization 31 | def glorot(tensor): 32 | if tensor is not None: 33 | stdv = math.sqrt(6.0 / (tensor.size(-2) + tensor.size(-1))) 34 | tensor.data.uniform_(-stdv, stdv) 35 | 36 | 37 | def zeros(tensor): 38 | if tensor is not None: 39 | tensor.data.fill_(0) 40 | 41 | 42 | class PMA(MessagePassing): 43 | """ 44 | PMA part: 45 | Note that in original PMA, we need to compute the inner product of the seed and neighbor nodes. 46 | i.e. e_ij = a(Wh_i,Wh_j), where a should be the inner product, h_i is the seed and h_j are neightbor nodes. 47 | In GAT, a(x,y) = a^T[x||y]. We use the same logic. 48 | """ 49 | _alpha: OptTensor 50 | 51 | def __init__(self, in_channels, hid_dim, 52 | out_channels, num_layers, heads=1, concat=True, 53 | negative_slope=0.2, dropout=0.0, bias=False, **kwargs): 54 | # kwargs.setdefault('aggr', 'add') 55 | super(PMA, self).__init__(node_dim=0, **kwargs) 56 | 57 | self.in_channels = in_channels 58 | self.hidden = hid_dim // heads 59 | self.out_channels = out_channels 60 | self.heads = heads 61 | self.concat = concat 62 | self.negative_slope = negative_slope 63 | self.dropout = 0. 64 | self.aggr = 'add' 65 | # self.input_seed = input_seed 66 | 67 | # This is the encoder part. Where we use 1 layer NN (Theta*x_i in the GATConv description) 68 | # Now, no seed as input. Directly learn the importance weights alpha_ij. 69 | # self.lin_O = Linear(heads*self.hidden, self.hidden) # For heads combining 70 | # For neighbor nodes (source side, key) 71 | self.lin_K = Linear(in_channels, self.heads*self.hidden) 72 | # For neighbor nodes (source side, value) 73 | self.lin_V = Linear(in_channels, self.heads*self.hidden) 74 | self.att_r = Parameter(torch.Tensor( 75 | 1, heads, self.hidden)) # Seed vector 76 | self.rFF = MLP(in_channels=self.heads*self.hidden, 77 | hidden_channels=self.heads*self.hidden, 78 | out_channels=out_channels, 79 | num_layers=num_layers, 80 | dropout=.0, Normalization='None',) 81 | self.ln0 = nn.LayerNorm(self.heads*self.hidden) 82 | self.ln1 = nn.LayerNorm(self.heads*self.hidden) 83 | # if bias and concat: 84 | # self.bias = Parameter(torch.Tensor(heads * out_channels)) 85 | # elif bias and not concat: 86 | # self.bias = Parameter(torch.Tensor(out_channels)) 87 | # else: 88 | 89 | # Always no bias! (For now) 90 | self.register_parameter('bias', None) 91 | 92 | self._alpha = None 93 | 94 | self.reset_parameters() 95 | 96 | def reset_parameters(self): 97 | # glorot(self.lin_l.weight) 98 | glorot(self.lin_K.weight) 99 | glorot(self.lin_V.weight) 100 | self.rFF.reset_parameters() 101 | self.ln0.reset_parameters() 102 | self.ln1.reset_parameters() 103 | # glorot(self.att_l) 104 | nn.init.xavier_uniform_(self.att_r) 105 | # zeros(self.bias) 106 | 107 | def forward(self, x, edge_index: Adj, 108 | size: Size = None, return_attention_weights=None): 109 | # type: (Union[Tensor, OptPairTensor], Tensor, Size, NoneType) -> Tensor # noqa 110 | # type: (Union[Tensor, OptPairTensor], SparseTensor, Size, NoneType) -> Tensor # noqa 111 | # type: (Union[Tensor, OptPairTensor], Tensor, Size, bool) -> Tuple[Tensor, Tuple[Tensor, Tensor]] # noqa 112 | # type: (Union[Tensor, OptPairTensor], SparseTensor, Size, bool) -> Tuple[Tensor, SparseTensor] # noqa 113 | r""" 114 | Args: 115 | return_attention_weights (bool, optional): If set to :obj:`True`, 116 | will additionally return the tuple 117 | :obj:`(edge_index, attention_weights)`, holding the computed 118 | attention weights for each edge. (default: :obj:`None`) 119 | """ 120 | H, C = self.heads, self.hidden 121 | 122 | x_l: OptTensor = None 123 | x_r: OptTensor = None 124 | alpha_l: OptTensor = None 125 | alpha_r: OptTensor = None 126 | if isinstance(x, Tensor): 127 | assert x.dim() == 2, 'Static graphs not supported in `GATConv`.' 128 | x_K = self.lin_K(x).view(-1, H, C) 129 | x_V = self.lin_V(x).view(-1, H, C) 130 | alpha_r = (x_K * self.att_r).sum(dim=-1) 131 | # else: 132 | # x_l, x_r = x[0], x[1] 133 | # assert x[0].dim() == 2, 'Static graphs not supported in `GATConv`.' 134 | # x_l = self.lin_l(x_l).view(-1, H, C) 135 | # alpha_l = (x_l * self.att_l).sum(dim=-1) 136 | # if x_r is not None: 137 | # x_r = self.lin_r(x_r).view(-1, H, C) 138 | # alpha_r = (x_r * self.att_r).sum(dim=-1) 139 | 140 | # assert x_l is not None 141 | # assert alpha_l is not None 142 | 143 | # propagate_type: (x: OptPairTensor, alpha: OptPairTensor) 144 | # ipdb.set_trace() 145 | out = self.propagate(edge_index, x=x_V, 146 | alpha=alpha_r, aggr=self.aggr) 147 | 148 | alpha = self._alpha 149 | self._alpha = None 150 | 151 | # Note that in the original code of GMT paper, they do not use additional W^O to combine heads. 152 | # This is because O = softmax(QK^T)V and V = V_in*W^V. So W^O can be effectively taken care by W^V!!! 153 | out += self.att_r # This is Seed + Multihead 154 | # concat heads then LayerNorm. Z (rhs of Eq(7)) in GMT paper. 155 | out = self.ln0(out.view(-1, self.heads * self.hidden)) 156 | # rFF and skip connection. Lhs of eq(7) in GMT paper. 157 | out = self.ln1(out+F.relu(self.rFF(out))) 158 | 159 | if isinstance(return_attention_weights, bool): 160 | assert alpha is not None 161 | if isinstance(edge_index, Tensor): 162 | return out, (edge_index, alpha) 163 | elif isinstance(edge_index, SparseTensor): 164 | return out, edge_index.set_value(alpha, layout='coo') 165 | else: 166 | return out 167 | 168 | def message(self, x_j, alpha_j, 169 | index, ptr, 170 | size_j): 171 | # ipdb.set_trace() 172 | alpha = alpha_j 173 | alpha = F.leaky_relu(alpha, self.negative_slope) 174 | alpha = softmax(alpha, index, ptr, index.max()+1) 175 | self._alpha = alpha 176 | alpha = F.dropout(alpha, p=self.dropout, training=self.training) 177 | return x_j * alpha.unsqueeze(-1) 178 | 179 | def aggregate(self, inputs, index, 180 | dim_size=None, aggr=None): 181 | r"""Aggregates messages from neighbors as 182 | :math:`\square_{j \in \mathcal{N}(i)}`. 183 | 184 | Takes in the output of message computation as first argument and any 185 | argument which was initially passed to :meth:`propagate`. 186 | 187 | By default, this function will delegate its call to scatter functions 188 | that support "add", "mean" and "max" operations as specified in 189 | :meth:`__init__` by the :obj:`aggr` argument. 190 | """ 191 | # ipdb.set_trace() 192 | if aggr is None: 193 | raise ValeuError("aggr was not passed!") 194 | return scatter(inputs, index, dim=self.node_dim, reduce=aggr) 195 | 196 | def __repr__(self): 197 | return '{}({}, {}, heads={})'.format(self.__class__.__name__, 198 | self.in_channels, 199 | self.out_channels, self.heads) 200 | 201 | 202 | class HGNN_conv(nn.Module): 203 | def __init__(self, in_ft, out_ft, bias=True): 204 | super(HGNN_conv, self).__init__() 205 | 206 | self.lin = Linear(in_ft, out_ft, bias=bias) 207 | # self.weight = Parameter(torch.Tensor(in_ft, out_ft)) 208 | # if bias: 209 | # self.bias = Parameter(torch.Tensor(out_ft)) 210 | # else: 211 | # self.register_parameter('bias', None) 212 | # self.reset_parameters() 213 | 214 | def reset_parameters(self): 215 | self.lin.reset_parameters() 216 | # stdv = 1. / math.sqrt(self.weight.size(1)) 217 | # self.weight.data.uniform_(-stdv, stdv) 218 | # if self.bias is not None: 219 | # self.bias.data.uniform_(-stdv, stdv) 220 | 221 | def forward(self, x, G): 222 | # x = data.x 223 | # G = data.edge_index 224 | 225 | x = self.lin(x) 226 | # x = x.matmul(self.weight) 227 | # if self.bias is not None: 228 | # x = x + self.bias 229 | x = torch.matmul(G, x) 230 | return x 231 | 232 | 233 | class HNHNConv(MessagePassing): 234 | def __init__(self, in_channels, hidden_channels, out_channels, heads=1, nonlinear_inbetween=True, 235 | concat=True, bias=True, **kwargs): 236 | kwargs.setdefault('aggr', 'add') 237 | super(HNHNConv, self).__init__(node_dim=0, **kwargs) 238 | 239 | self.in_channels = in_channels 240 | self.hidden_channels = hidden_channels 241 | self.out_channels = out_channels 242 | self.nonlinear_inbetween = nonlinear_inbetween 243 | 244 | # preserve variable heads for later use (attention) 245 | self.heads = heads 246 | self.concat = True 247 | # self.weight = Parameter(torch.Tensor(in_channels, out_channels)) 248 | self.weight_v2e = Linear(in_channels, hidden_channels) 249 | self.weight_e2v = Linear(hidden_channels, out_channels) 250 | 251 | self.reset_parameters() 252 | 253 | def reset_parameters(self): 254 | self.weight_v2e.reset_parameters() 255 | self.weight_e2v.reset_parameters() 256 | # glorot(self.weight_v2e) 257 | # glorot(self.weight_e2v) 258 | # zeros(self.bias) 259 | 260 | def forward(self, x, data): 261 | r""" 262 | Args: 263 | x (Tensor): Node feature matrix :math:`\mathbf{X}` 264 | hyperedge_index (LongTensor): The hyperedge indices, *i.e.* 265 | the sparse incidence matrix 266 | :math:`\mathbf{H} \in {\{ 0, 1 \}}^{N \times M}` mapping from 267 | nodes to edges. 268 | hyperedge_weight (Tensor, optional): Sparse hyperedge weights 269 | :math:`\mathbf{W} \in \mathbb{R}^M`. (default: :obj:`None`) 270 | """ 271 | hyperedge_index = data.edge_index 272 | hyperedge_weight = None 273 | num_nodes, num_edges = x.size(0), 0 274 | if hyperedge_index.numel() > 0: 275 | num_edges = int(hyperedge_index[1].max()) + 1 276 | 277 | if hyperedge_weight is None: 278 | hyperedge_weight = x.new_ones(num_edges) 279 | 280 | x = self.weight_v2e(x) 281 | 282 | # ipdb.set_trace() 283 | # x = torch.matmul(torch.diag(data.D_v_beta), x) 284 | x = data.D_v_beta.unsqueeze(-1) * x 285 | 286 | self.flow = 'source_to_target' 287 | out = self.propagate(hyperedge_index, x=x, norm=data.D_e_beta_inv, 288 | size=(num_nodes, num_edges)) 289 | 290 | if self.nonlinear_inbetween: 291 | out = F.relu(out) 292 | 293 | # sanity check 294 | out = torch.squeeze(out, dim=1) 295 | 296 | out = self.weight_e2v(out) 297 | 298 | # out = torch.matmul(torch.diag(data.D_e_alpha), out) 299 | out = data.D_e_alpha.unsqueeze(-1) * out 300 | 301 | self.flow = 'target_to_source' 302 | out = self.propagate(hyperedge_index, x=out, norm=data.D_v_alpha_inv, 303 | size=(num_edges, num_nodes)) 304 | 305 | return out 306 | 307 | def message(self, x_j, norm_i): 308 | 309 | out = norm_i.view(-1, 1) * x_j 310 | 311 | return out 312 | 313 | def __repr__(self): 314 | return "{}({}, {}, {})".format(self.__class__.__name__, self.in_channels, 315 | self.hidden_channels, self.out_channels) 316 | 317 | 318 | class HypergraphConv(MessagePassing): 319 | r"""The hypergraph convolutional operator from the `"Hypergraph Convolution 320 | and Hypergraph Attention" `_ paper 321 | 322 | .. math:: 323 | \mathbf{X}^{\prime} = \mathbf{D}^{-1} \mathbf{H} \mathbf{W} 324 | \mathbf{B}^{-1} \mathbf{H}^{\top} \mathbf{X} \mathbf{\Theta} 325 | 326 | where :math:`\mathbf{H} \in {\{ 0, 1 \}}^{N \times M}` is the incidence 327 | matrix, :math:`\mathbf{W} \in \mathbb{R}^M` is the diagonal hyperedge 328 | weight matrix, and 329 | :math:`\mathbf{D}` and :math:`\mathbf{B}` are the corresponding degree 330 | matrices. 331 | 332 | For example, in the hypergraph scenario 333 | :math:`\mathcal{G} = (\mathcal{V}, \mathcal{E})` with 334 | :math:`\mathcal{V} = \{ 0, 1, 2, 3 \}` and 335 | :math:`\mathcal{E} = \{ \{ 0, 1, 2 \}, \{ 1, 2, 3 \} \}`, the 336 | :obj:`hyperedge_index` is represented as: 337 | 338 | .. code-block:: python 339 | 340 | hyperedge_index = torch.tensor([ 341 | [0, 1, 2, 1, 2, 3], 342 | [0, 0, 0, 1, 1, 1], 343 | ]) 344 | 345 | Args: 346 | in_channels (int): Size of each input sample. 347 | out_channels (int): Size of each output sample. 348 | use_attention (bool, optional): If set to :obj:`True`, attention 349 | will be added to this layer. (default: :obj:`False`) 350 | heads (int, optional): Number of multi-head-attentions. 351 | (default: :obj:`1`) 352 | concat (bool, optional): If set to :obj:`False`, the multi-head 353 | attentions are averaged instead of concatenated. 354 | (default: :obj:`True`) 355 | negative_slope (float, optional): LeakyReLU angle of the negative 356 | slope. (default: :obj:`0.2`) 357 | dropout (float, optional): Dropout probability of the normalized 358 | attention coefficients which exposes each node to a stochastically 359 | sampled neighborhood during training. (default: :obj:`0`) 360 | bias (bool, optional): If set to :obj:`False`, the layer will not learn 361 | an additive bias. (default: :obj:`True`) 362 | **kwargs (optional): Additional arguments of 363 | :class:`torch_geometric.nn.conv.MessagePassing`. 364 | """ 365 | 366 | def __init__(self, in_channels, out_channels, symdegnorm=False, use_attention=False, heads=1, 367 | concat=True, negative_slope=0.2, dropout=0, bias=True, 368 | **kwargs): 369 | kwargs.setdefault('aggr', 'add') 370 | super(HypergraphConv, self).__init__(node_dim=0, **kwargs) 371 | 372 | self.in_channels = in_channels 373 | self.out_channels = out_channels 374 | self.use_attention = use_attention 375 | self.symdegnorm = symdegnorm 376 | 377 | if self.use_attention: 378 | self.heads = heads 379 | self.concat = concat 380 | self.negative_slope = negative_slope 381 | self.dropout = dropout 382 | self.weight = Parameter( 383 | torch.Tensor(in_channels, heads * out_channels)) 384 | self.att = Parameter(torch.Tensor(1, heads, 2 * out_channels)) 385 | else: 386 | self.heads = 1 387 | self.concat = True 388 | self.weight = Parameter(torch.Tensor(in_channels, out_channels)) 389 | 390 | if bias and concat: 391 | self.bias = Parameter(torch.Tensor(heads * out_channels)) 392 | elif bias and not concat: 393 | self.bias = Parameter(torch.Tensor(out_channels)) 394 | else: 395 | self.register_parameter('bias', None) 396 | 397 | self.reset_parameters() 398 | 399 | def reset_parameters(self): 400 | glorot(self.weight) 401 | if self.use_attention: 402 | glorot(self.att) 403 | zeros(self.bias) 404 | 405 | def forward(self, x: Tensor, hyperedge_index: Tensor, 406 | hyperedge_weight: Optional[Tensor] = None) -> Tensor: 407 | r""" 408 | Args: 409 | x (Tensor): Node feature matrix :math:`\mathbf{X}` 410 | hyperedge_index (LongTensor): The hyperedge indices, *i.e.* 411 | the sparse incidence matrix 412 | :math:`\mathbf{H} \in {\{ 0, 1 \}}^{N \times M}` mapping from 413 | nodes to edges. 414 | hyperedge_weight (Tensor, optional): Sparse hyperedge weights 415 | :math:`\mathbf{W} \in \mathbb{R}^M`. (default: :obj:`None`) 416 | """ 417 | num_nodes, num_edges = x.size(0), 0 418 | if hyperedge_index.numel() > 0: 419 | num_edges = int(hyperedge_index[1].max()) + 1 420 | 421 | if hyperedge_weight is None: 422 | hyperedge_weight = x.new_ones(num_edges) 423 | 424 | x = torch.matmul(x, self.weight) 425 | 426 | alpha = None 427 | if self.use_attention: 428 | assert num_edges <= num_edges 429 | x = x.view(-1, self.heads, self.out_channels) 430 | x_i, x_j = x[hyperedge_index[0]], x[hyperedge_index[1]] 431 | alpha = (torch.cat([x_i, x_j], dim=-1) * self.att).sum(dim=-1) 432 | alpha = F.leaky_relu(alpha, self.negative_slope) 433 | alpha = softmax(alpha, hyperedge_index[0], num_nodes=x.size(0)) 434 | alpha = F.dropout(alpha, p=self.dropout, training=self.training) 435 | 436 | if not self.symdegnorm: 437 | D = scatter_add(hyperedge_weight[hyperedge_index[1]], 438 | hyperedge_index[0], dim=0, dim_size=num_nodes) 439 | D = 1.0 / D 440 | D[D == float("inf")] = 0 441 | 442 | B = scatter_add(x.new_ones(hyperedge_index.size(1)), 443 | hyperedge_index[1], dim=0, dim_size=num_edges) 444 | B = 1.0 / B 445 | B[B == float("inf")] = 0 446 | 447 | self.flow = 'source_to_target' 448 | out = self.propagate(hyperedge_index, x=x, norm=B, alpha=alpha, 449 | size=(num_nodes, num_edges)) 450 | self.flow = 'target_to_source' 451 | out = self.propagate(hyperedge_index, x=out, norm=D, alpha=alpha, 452 | size=(num_edges, num_nodes)) 453 | else: # this correspond to HGNN 454 | D = scatter_add(hyperedge_weight[hyperedge_index[1]], 455 | hyperedge_index[0], dim=0, dim_size=num_nodes) 456 | D = 1.0 / D**(0.5) 457 | D[D == float("inf")] = 0 458 | 459 | B = scatter_add(x.new_ones(hyperedge_index.size(1)), 460 | hyperedge_index[1], dim=0, dim_size=num_edges) 461 | B = 1.0 / B 462 | B[B == float("inf")] = 0 463 | 464 | x = D.unsqueeze(-1)*x 465 | self.flow = 'source_to_target' 466 | out = self.propagate(hyperedge_index, x=x, norm=B, alpha=alpha, 467 | size=(num_nodes, num_edges)) 468 | self.flow = 'target_to_source' 469 | out = self.propagate(hyperedge_index, x=out, norm=D, alpha=alpha, 470 | size=(num_edges, num_nodes)) 471 | 472 | if self.concat is True: 473 | out = out.view(-1, self.heads * self.out_channels) 474 | else: 475 | out = out.mean(dim=1) 476 | 477 | if self.bias is not None: 478 | out = out + self.bias 479 | 480 | return out 481 | 482 | def message(self, x_j: Tensor, norm_i: Tensor, alpha: Tensor) -> Tensor: 483 | H, F = self.heads, self.out_channels 484 | 485 | out = norm_i.view(-1, 1, 1) * x_j.view(-1, H, F) 486 | 487 | if alpha is not None: 488 | out = alpha.view(-1, self.heads, 1) * out 489 | 490 | return out 491 | 492 | def __repr__(self): 493 | return "{}({}, {})".format(self.__class__.__name__, self.in_channels, 494 | self.out_channels) 495 | 496 | class MLP(nn.Module): 497 | """ adapted from https://github.com/CUAI/CorrectAndSmooth/blob/master/gen_models.py """ 498 | 499 | def __init__(self, in_channels, hidden_channels, out_channels, num_layers, 500 | dropout=.5, Normalization='bn', InputNorm=False): 501 | super(MLP, self).__init__() 502 | self.lins = nn.ModuleList() 503 | self.normalizations = nn.ModuleList() 504 | self.InputNorm = InputNorm 505 | 506 | assert Normalization in ['bn', 'ln', 'None'] 507 | if Normalization == 'bn': 508 | if num_layers == 1: 509 | # just linear layer i.e. logistic regression 510 | if InputNorm: 511 | self.normalizations.append(nn.BatchNorm1d(in_channels)) 512 | else: 513 | self.normalizations.append(nn.Identity()) 514 | self.lins.append(nn.Linear(in_channels, out_channels)) 515 | else: 516 | if InputNorm: 517 | self.normalizations.append(nn.BatchNorm1d(in_channels)) 518 | else: 519 | self.normalizations.append(nn.Identity()) 520 | self.lins.append(nn.Linear(in_channels, hidden_channels)) 521 | self.normalizations.append(nn.BatchNorm1d(hidden_channels)) 522 | for _ in range(num_layers - 2): 523 | self.lins.append( 524 | nn.Linear(hidden_channels, hidden_channels)) 525 | self.normalizations.append(nn.BatchNorm1d(hidden_channels)) 526 | self.lins.append(nn.Linear(hidden_channels, out_channels)) 527 | elif Normalization == 'ln': 528 | if num_layers == 1: 529 | # just linear layer i.e. logistic regression 530 | if InputNorm: 531 | self.normalizations.append(nn.LayerNorm(in_channels)) 532 | else: 533 | self.normalizations.append(nn.Identity()) 534 | self.lins.append(nn.Linear(in_channels, out_channels)) 535 | else: 536 | if InputNorm: 537 | self.normalizations.append(nn.LayerNorm(in_channels)) 538 | else: 539 | self.normalizations.append(nn.Identity()) 540 | self.lins.append(nn.Linear(in_channels, hidden_channels)) 541 | self.normalizations.append(nn.LayerNorm(hidden_channels)) 542 | for _ in range(num_layers - 2): 543 | self.lins.append( 544 | nn.Linear(hidden_channels, hidden_channels)) 545 | self.normalizations.append(nn.LayerNorm(hidden_channels)) 546 | self.lins.append(nn.Linear(hidden_channels, out_channels)) 547 | else: 548 | if num_layers == 1: 549 | # just linear layer i.e. logistic regression 550 | self.normalizations.append(nn.Identity()) 551 | self.lins.append(nn.Linear(in_channels, out_channels)) 552 | else: 553 | self.normalizations.append(nn.Identity()) 554 | self.lins.append(nn.Linear(in_channels, hidden_channels)) 555 | self.normalizations.append(nn.Identity()) 556 | for _ in range(num_layers - 2): 557 | self.lins.append( 558 | nn.Linear(hidden_channels, hidden_channels)) 559 | self.normalizations.append(nn.Identity()) 560 | self.lins.append(nn.Linear(hidden_channels, out_channels)) 561 | 562 | self.dropout = dropout 563 | 564 | def reset_parameters(self): 565 | for lin in self.lins: 566 | lin.reset_parameters() 567 | for normalization in self.normalizations: 568 | if not (normalization.__class__.__name__ == 'Identity'): 569 | normalization.reset_parameters() 570 | 571 | def forward(self, x): 572 | x = self.normalizations[0](x) 573 | for i, lin in enumerate(self.lins[:-1]): 574 | x = lin(x) 575 | x = F.relu(x, inplace=True) 576 | x = self.normalizations[i+1](x) 577 | x = F.dropout(x, p=self.dropout, training=self.training) 578 | x = self.lins[-1](x) 579 | return x 580 | 581 | 582 | class HalfNLHconv(MessagePassing): 583 | def __init__(self, 584 | in_dim, 585 | hid_dim, 586 | out_dim, 587 | num_layers, 588 | dropout, 589 | Normalization='bn', 590 | InputNorm=False, 591 | heads=1, 592 | attention=True 593 | ): 594 | super(HalfNLHconv, self).__init__() 595 | 596 | self.attention = attention 597 | self.dropout = dropout 598 | 599 | if self.attention: 600 | self.prop = PMA(in_dim, hid_dim, out_dim, num_layers, heads=heads) 601 | else: 602 | if num_layers > 0: 603 | self.f_enc = MLP(in_dim, hid_dim, hid_dim, num_layers, dropout, Normalization, InputNorm) 604 | self.f_dec = MLP(hid_dim, hid_dim, out_dim, num_layers, dropout, Normalization, InputNorm) 605 | else: 606 | self.f_enc = nn.Identity() 607 | self.f_dec = nn.Identity() 608 | # self.bn = nn.BatchNorm1d(dec_hid_dim) 609 | # self.dropout = dropout 610 | # self.Prop = S2SProp() 611 | 612 | def reset_parameters(self): 613 | 614 | if self.attention: 615 | self.prop.reset_parameters() 616 | else: 617 | if not (self.f_enc.__class__.__name__ == 'Identity'): 618 | self.f_enc.reset_parameters() 619 | if not (self.f_dec.__class__.__name__ == 'Identity'): 620 | self.f_dec.reset_parameters() 621 | # self.bn.reset_parameters() 622 | 623 | def forward(self, x, edge_index, norm, aggr='add'): 624 | """ 625 | input -> MLP -> Prop 626 | """ 627 | 628 | if self.attention: 629 | x = self.prop(x, edge_index) 630 | else: 631 | x = F.relu(self.f_enc(x)) 632 | x = F.dropout(x, p=self.dropout, training=self.training) 633 | x = self.propagate(edge_index, x=x, norm=norm, aggr=aggr) 634 | x = F.relu(self.f_dec(x)) 635 | 636 | return x 637 | 638 | def message(self, x_j, norm): 639 | return norm.view(-1, 1) * x_j 640 | 641 | def aggregate(self, inputs, index, 642 | dim_size=None, aggr=None): 643 | r"""Aggregates messages from neighbors as 644 | :math:`\square_{j \in \mathcal{N}(i)}`. 645 | 646 | Takes in the output of message computation as first argument and any 647 | argument which was initially passed to :meth:`propagate`. 648 | 649 | By default, this function will delegate its call to scatter functions 650 | that support "add", "mean" and "max" operations as specified in 651 | :meth:`__init__` by the :obj:`aggr` argument. 652 | """ 653 | # ipdb.set_trace() 654 | if aggr is None: 655 | raise ValeuError("aggr was not passed!") 656 | return scatter(inputs, index, dim=self.node_dim, reduce=aggr) 657 | 658 | 659 | --------------------------------------------------------------------------------