├── LICENSE ├── README.md ├── initial.py ├── jointD ├── cnn.txt ├── init.cpp ├── init.so ├── make.sh ├── network.py ├── network.pyc ├── pr_plot.py ├── test.py └── train.py ├── jointE ├── KATT │ ├── cnn.txt │ ├── init.cpp │ ├── make.sh │ ├── network.py │ ├── network.pyc │ ├── pr_plot.py │ ├── test.py │ └── train.py └── SATT │ ├── cnn.txt │ ├── init.cpp │ ├── make.sh │ ├── network.py │ ├── network.pyc │ ├── pr_plot.py │ ├── res.png │ └── train.py └── original └── baselines ├── test ├── init_cnn.cpp ├── init_know.cpp ├── test_JointD+ATT.py ├── test_JointD+ONE.py ├── test_JointE+ATT.py └── test_JointE+ONE.py └── train ├── JointD+ATT.py ├── JointD+ONE.py ├── JointE+ATT.py └── JointE+ONE.py /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 THUNLP 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # JointNRE 2 | 3 | This repository is a subproject of THU-OpenSK, and all subprojects of THU-OpenSK are as follows. 4 | 5 | - [OpenNE](https://www.github.com/thunlp/OpenNE) 6 | - [OpenKE](https://www.github.com/thunlp/OpenKE) 7 | - [KB2E](https://www.github.com/thunlp/KB2E) 8 | - [TensorFlow-Transx](https://www.github.com/thunlp/TensorFlow-Transx) 9 | - [Fast-TransX](https://www.github.com/thunlp/Fast-TransX) 10 | - [OpenNRE](https://www.github.com/thunlp/OpenNRE) 11 | - [JointNRE](https://www.github.com/thunlp/JointNRE) 12 | 13 | Codes and datasets for our paper "Neural Knowledge Acquisition via Mutual Attention between Knowledge Graph and Text" 14 | 15 | 16 | Some Introduction 17 | === 18 | 19 | This implementation is a fast and stable version. 20 | 21 | We have made some simplifications for the original model so that to train a joint model just needs around 15min. 22 | 23 | We also encapsulate more neural architectures into our framework to encode sentences. 24 | 25 | The code and datasets mainly for the task relation extraction. 26 | 27 | Data 28 | ========== 29 | 30 | We provide the datasets used for the task relation extraction. 31 | 32 | New York Times Corpus: The data used in relation extraction from text is published by "Modeling relations and their mentions without labeled text". The data should be obtained from [[LDC]](https://catalog.ldc.upenn.edu/LDC2008T19) first. 33 | 34 | Datasets are required in the folder data/ in the following format, containing at least 4 files: 35 | 36 | + kg/train.txt: the knowledge graph for training, format (e1, e2, rel). 37 | 38 | + text/relation2id.txt: the relation needed to be predicted for RE, format (rel, id). 39 | 40 | + text/train.txt: the text for training, format (e1, e2, name1, name2, rel, sentence). 41 | 42 | + text/vec.txt: the initial word embeddings. 43 | 44 | + [[Download (Baidu Cloud)]](https://pan.baidu.com/s/1q7rctsoJ_YdlLa55yckwbQ) 45 | + [[Download (Tsinghua Cloud)]](https://cloud.tsinghua.edu.cn/f/28ba8ac5262349dd9622/?dl=1) 46 | 47 | For FB15K-NYT, we directly give the data for our code [[Download (Tsinghua Cloud)]](https://cloud.tsinghua.edu.cn/f/384836aacb1f4aee9fa3/?dl=1), as we cannot release the original data limited by the license of LDC. 48 | 49 | Run the experiments 50 | ========== 51 | 52 | ### To run the experiments, unpack the datasets first: 53 | 54 | ``` 55 | unzip origin_data.zip -d origin_data/ 56 | mkdir data/ 57 | python initial.py 58 | ``` 59 | 60 | ### Run the corresponding python scripts to train models: 61 | 62 | ``` 63 | cd jointE 64 | bash make.sh 65 | python train.py 66 | ``` 67 | 68 | ### Change the corresponding python code to set hyperparameters: 69 | 70 | ``` 71 | tf.app.flags.DEFINE_float('nbatch_kg',100,'entity numbers used each training time') 72 | tf.app.flags.DEFINE_float('margin',1.0,'entity numbers used each training time') 73 | tf.app.flags.DEFINE_float('learning_rate_kg',0.001,'learning rate for kg') 74 | tf.app.flags.DEFINE_float('ent_total',lib.getEntityTotal(),'total of entities') 75 | tf.app.flags.DEFINE_float('rel_total',lib.getRelationTotal(),'total of relations') 76 | tf.app.flags.DEFINE_float('tri_total',lib.getTripleTotal(),'total of triples') 77 | tf.app.flags.DEFINE_float('katt_flag', 1, '1 for katt, 0 for att') 78 | 79 | tf.app.flags.DEFINE_string('model', 'cnn', 'neural models to encode sentences') 80 | tf.app.flags.DEFINE_float('max_length',config['fixlen'],'maximum of number of words in one sentence') 81 | tf.app.flags.DEFINE_float('pos_num', config['maxlen'] * 2 + 1,'number of position embedding vectors') 82 | tf.app.flags.DEFINE_float('num_classes', config['textual_rel_total'],'maximum of relations') 83 | 84 | tf.app.flags.DEFINE_float('hidden_size',230,'hidden feature size') 85 | tf.app.flags.DEFINE_float('pos_size',5,'position embedding size') 86 | 87 | tf.app.flags.DEFINE_float('max_epoch',20,'maximum of training epochs') 88 | tf.app.flags.DEFINE_float('batch_size',160,'entity numbers used each training time') 89 | tf.app.flags.DEFINE_float('learning_rate',0.5,'learning rate for nn') 90 | tf.app.flags.DEFINE_float('weight_decay',0.00001,'weight_decay') 91 | tf.app.flags.DEFINE_float('keep_prob',0.5,'dropout rate') 92 | 93 | tf.app.flags.DEFINE_string('model_dir','./model/','path to store model') 94 | tf.app.flags.DEFINE_string('summary_dir','./summary','path to store summary_dir') 95 | ``` 96 | 97 | ### Run the corresponding python scripts to test models: 98 | 99 | ``` 100 | cd jointE 101 | bash make.sh 102 | python test.py 103 | ``` 104 | 105 | Note that the hyperparameters in the train.py and the test.py must be the same. 106 | 107 | ### Run the corresponding python script to get PR-curve results: 108 | 109 | ``` 110 | cd jointE 111 | python pr_plot.py 112 | ``` 113 | 114 | Citation 115 | === 116 | 117 | ``` 118 | @inproceedings{han2018neural, 119 | title={Neural Knowledge Acquisition via Mutual Attention between Knowledge Graph and Text}, 120 | author={Han, Xu and Liu, Zhiyuan and Sun, Maosong}, 121 | booktitle={Proceedings of AAAI}, 122 | year={2018} 123 | } 124 | ``` 125 | 126 | 127 | 128 | 129 | 130 | 131 | 132 | -------------------------------------------------------------------------------- /initial.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import os 3 | import json 4 | 5 | # folder of training datasets 6 | data_path = "./origin_data/" 7 | # files to export data 8 | export_path = "./data/" 9 | #length of sentence 10 | fixlen = 120 11 | #max length of position embedding is 100 (-100~+100) 12 | maxlen = 100 13 | 14 | word2id = {} 15 | relation2id = {} 16 | word_size = 0 17 | word_vec = None 18 | 19 | def pos_embed(x): 20 | return max(0, min(x + maxlen, maxlen + maxlen + 1)) 21 | 22 | def find_index(x,y): 23 | for index, item in enumerate(y): 24 | if x == item: 25 | return index 26 | return -1 27 | 28 | def init_word(): 29 | # reading word embedding data... 30 | global word2id, word_size 31 | res = [] 32 | ff = open(export_path + "/entity2id.txt", "w") 33 | f = open(data_path + "kg/train.txt", "r") 34 | while True: 35 | content = f.readline() 36 | if content == "": 37 | break 38 | h, t, r = content.strip().split("\t") 39 | if not h in word2id: 40 | word2id[h] = len(word2id) 41 | ff.write("%s\t%d\n"%(h, word2id[h])) 42 | if not t in word2id: 43 | word2id[t] = len(word2id) 44 | ff.write("%s\t%d\n"%(t, word2id[t])) 45 | f.close() 46 | f = open(data_path + "text/train.txt", "r") 47 | while True: 48 | content = f.readline() 49 | if content == "": 50 | break 51 | h,t = content.strip().split("\t")[:2] 52 | if not h in word2id: 53 | word2id[h] = len(word2id) 54 | ff.write("%s\t%d\n"%(h, word2id[h])) 55 | if not t in word2id: 56 | word2id[t] = len(word2id) 57 | ff.write("%s\t%d\n"%(t, word2id[t])) 58 | f.close() 59 | f = open(data_path + "text/test.txt", "r") 60 | while True: 61 | content = f.readline() 62 | if content == "": 63 | break 64 | h,t = content.strip().split("\t")[:2] 65 | if not h in word2id: 66 | word2id[h] = len(word2id) 67 | ff.write("%s\t%d\n"%(h, word2id[h])) 68 | if not t in word2id: 69 | word2id[t] = len(word2id) 70 | ff.write("%s\t%d\n"%(t, word2id[t])) 71 | f.close() 72 | res.append(len(word2id)) 73 | ff.close() 74 | 75 | print 'reading word embedding data...' 76 | f = open(data_path + 'text/vec.txt', "r") 77 | total, size = f.readline().strip().split()[:2] 78 | total = (int)(total) 79 | word_size = (int)(size) 80 | vec = np.ones((total + res[0], word_size), dtype = np.float32) 81 | for i in range(total): 82 | content = f.readline().strip().split() 83 | word2id[content[0]] = len(word2id) 84 | for j in range(word_size): 85 | vec[i + res[0]][j] = (float)(content[j+1]) 86 | f.close() 87 | word2id['UNK'] = len(word2id) 88 | word2id['BLANK'] = len(word2id) 89 | global word_vec 90 | word_vec = vec 91 | res.append(len(word2id)) 92 | return res 93 | 94 | def init_relation(): 95 | # reading relation ids... 96 | global relation2id 97 | print 'reading relation ids...' 98 | res = [] 99 | ff = open(export_path + "/relation2id.txt", "w") 100 | f = open(data_path + "text/relation2id.txt","r") 101 | total = (int)(f.readline().strip()) 102 | for i in range(total): 103 | content = f.readline().strip().split() 104 | if not content[0] in relation2id: 105 | relation2id[content[0]] = len(relation2id) 106 | ff.write("%s\t%d\n"%(content[0], relation2id[content[0]])) 107 | f.close() 108 | res.append(len(relation2id)) 109 | f = open(data_path + "kg/train.txt", "r") 110 | for i in f.readlines(): 111 | h, t, r = i.strip().split("\t") 112 | if not r in relation2id: 113 | relation2id[r] = len(relation2id) 114 | ff.write("%s\t%d\n"%(r, relation2id[r])) 115 | f.close() 116 | ff.close() 117 | res.append(len(relation2id)) 118 | return res 119 | 120 | def sort_files(name, limit): 121 | hash = {} 122 | f = open(data_path + "text/" + name + '.txt','r') 123 | s = 0 124 | while True: 125 | content = f.readline() 126 | if content == '': 127 | break 128 | s = s + 1 129 | origin_data = content 130 | content = content.strip().split() 131 | en1_id = content[0] 132 | en2_id = content[1] 133 | rel_name = content[4] 134 | if (rel_name in relation2id) and ((int)(relation2id[rel_name]) < limit[0]): 135 | relation = relation2id[rel_name] 136 | else: 137 | relation = relation2id['NA'] 138 | id1 = str(en1_id)+"#"+str(en2_id) 139 | id2 = str(relation) 140 | if not id1 in hash: 141 | hash[id1] = {} 142 | if not id2 in hash[id1]: 143 | hash[id1][id2] = [] 144 | hash[id1][id2].append(origin_data) 145 | f.close() 146 | f = open(data_path + name + "_sort.txt", "w") 147 | f.write("%d\n"%(s)) 148 | for i in hash: 149 | for j in hash[i]: 150 | for k in hash[i][j]: 151 | f.write(k) 152 | f.close() 153 | 154 | def init_train_files(name, limit): 155 | print 'reading ' + name +' data...' 156 | f = open(data_path + name + '.txt','r') 157 | total = (int)(f.readline().strip()) 158 | sen_word = np.zeros((total, fixlen), dtype = np.int32) 159 | sen_pos1 = np.zeros((total, fixlen), dtype = np.int32) 160 | sen_pos2 = np.zeros((total, fixlen), dtype = np.int32) 161 | sen_mask = np.zeros((total, fixlen), dtype = np.int32) 162 | sen_len = np.zeros((total), dtype = np.int32) 163 | sen_label = np.zeros((total), dtype = np.int32) 164 | sen_head = np.zeros((total), dtype = np.int32) 165 | sen_tail = np.zeros((total), dtype = np.int32) 166 | instance_scope = [] 167 | instance_triple = [] 168 | for s in range(total): 169 | content = f.readline().strip().split() 170 | sentence = content[5:-1] 171 | en1_id = content[0] 172 | en2_id = content[1] 173 | en1_name = content[2] 174 | en2_name = content[3] 175 | rel_name = content[4] 176 | if rel_name in relation2id and ((int)(relation2id[rel_name]) < limit[0]): 177 | relation = relation2id[rel_name] 178 | else: 179 | relation = relation2id['NA'] 180 | en1pos = 0 181 | en2pos = 0 182 | for i in range(len(sentence)): 183 | if sentence[i] == en1_name: 184 | sentence[i] = en1_id 185 | en1pos = i 186 | sen_head[s] = word2id[en1_id] 187 | if sentence[i] == en2_name: 188 | sentence[i] = en2_id 189 | en2pos = i 190 | sen_tail[s] = word2id[en2_id] 191 | en_first = min(en1pos,en2pos) 192 | en_second = en1pos + en2pos - en_first 193 | for i in range(fixlen): 194 | sen_word[s][i] = word2id['BLANK'] 195 | sen_pos1[s][i] = pos_embed(i - en1pos) 196 | sen_pos2[s][i] = pos_embed(i - en2pos) 197 | if i >= len(sentence): 198 | sen_mask[s][i] = 0 199 | elif i - en_first<=0: 200 | sen_mask[s][i] = 1 201 | elif i - en_second<=0: 202 | sen_mask[s][i] = 2 203 | else: 204 | sen_mask[s][i] = 3 205 | for i, word in enumerate(sentence): 206 | if i >= fixlen: 207 | break 208 | elif not word in word2id: 209 | sen_word[s][i] = word2id['UNK'] 210 | else: 211 | sen_word[s][i] = word2id[word] 212 | sen_len[s] = min(fixlen, len(sentence)) 213 | sen_label[s] = relation 214 | #put the same entity pair sentences into a dict 215 | tup = (en1_id,en2_id,relation) 216 | if instance_triple == [] or instance_triple[len(instance_triple) - 1] != tup: 217 | instance_triple.append(tup) 218 | instance_scope.append([s,s]) 219 | instance_scope[len(instance_triple) - 1][1] = s 220 | if (s+1) % 100 == 0: 221 | print s 222 | return np.array(instance_triple), np.array(instance_scope), sen_len, sen_label, sen_word, sen_pos1, sen_pos2, sen_mask, sen_head, sen_tail 223 | 224 | def init_kg(): 225 | ff = open(export_path + "/triple2id.txt", "w") 226 | f = open(data_path + "kg/train.txt", "r") 227 | content = f.readlines() 228 | ff.write("%d\n"%(len(content))) 229 | for i in content: 230 | h,t,r = i.strip().split("\t") 231 | ff.write("%d\t%d\t%d\n"%(word2id[h], word2id[t], relation2id[r])) 232 | f.close() 233 | ff.close() 234 | 235 | f = open(export_path + "/entity2id.txt", "r") 236 | content = f.readlines() 237 | f.close() 238 | f = open(export_path + "/entity2id.txt", "w") 239 | f.write("%d\n"%(len(content))) 240 | for i in content: 241 | f.write(i.strip()+"\n") 242 | f.close() 243 | 244 | f = open(export_path + "/relation2id.txt", "r") 245 | content = f.readlines() 246 | f.close() 247 | f = open(export_path + "/relation2id.txt", "w") 248 | f.write("%d\n"%(len(content))) 249 | for i in content: 250 | f.write(i.strip()+"\n") 251 | f.close() 252 | 253 | textual_rel_total, rel_total = init_relation() 254 | entity_total, word_total = init_word() 255 | 256 | print textual_rel_total 257 | print rel_total 258 | print entity_total 259 | print word_total 260 | print word_vec.shape 261 | f = open(data_path + "word2id.txt", "w") 262 | for i in word2id: 263 | f.write("%s\t%d\n"%(i, word2id[i])) 264 | f.close() 265 | 266 | init_kg() 267 | np.save(export_path+'vec', word_vec) 268 | f = open(export_path+'config', "w") 269 | f.write(json.dumps({"word2id":word2id,"relation2id":relation2id,"word_size":word_size, "fixlen":fixlen, "maxlen":maxlen, "entity_total":entity_total, "word_total":word_total, "rel_total":rel_total, "textual_rel_total":textual_rel_total})) 270 | f.close() 271 | sort_files("train", [textual_rel_total, rel_total]) 272 | sort_files("test", [textual_rel_total, rel_total]) 273 | 274 | # word_vec = np.load(export_path + 'vec.npy') 275 | # f = open(export_path + "config", 'r') 276 | # config = json.loads(f.read()) 277 | # f.close() 278 | # relation2id = config["relation2id"] 279 | # word2id = config["word2id"] 280 | 281 | instance_triple, instance_scope, train_len, train_label, train_word, train_pos1, train_pos2, train_mask, train_head, train_tail = init_train_files("train_sort", [textual_rel_total, rel_total]) 282 | np.save(export_path+'train_instance_triple', instance_triple) 283 | np.save(export_path+'train_instance_scope', instance_scope) 284 | np.save(export_path+'train_len', train_len) 285 | np.save(export_path+'train_label', train_label) 286 | np.save(export_path+'train_word', train_word) 287 | np.save(export_path+'train_pos1', train_pos1) 288 | np.save(export_path+'train_pos2', train_pos2) 289 | np.save(export_path+'train_mask', train_mask) 290 | np.save(export_path+'train_head', train_head) 291 | np.save(export_path+'train_tail', train_tail) 292 | 293 | instance_triple, instance_scope, test_len, test_label, test_word, test_pos1, test_pos2, test_mask, test_head, test_tail = init_train_files("test_sort", [textual_rel_total, rel_total]) 294 | np.save(export_path+'test_instance_triple', instance_triple) 295 | np.save(export_path+'test_instance_scope', instance_scope) 296 | np.save(export_path+'test_len', test_len) 297 | np.save(export_path+'test_label', test_label) 298 | np.save(export_path+'test_word', test_word) 299 | np.save(export_path+'test_pos1', test_pos1) 300 | np.save(export_path+'test_pos2', test_pos2) 301 | np.save(export_path+'test_mask', test_mask) 302 | np.save(export_path+'test_head', test_head) 303 | np.save(export_path+'test_tail', test_tail) 304 | -------------------------------------------------------------------------------- /jointD/init.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | 9 | using namespace std; 10 | 11 | string inPath = "./data/"; 12 | 13 | extern "C" 14 | void setInPath(char *path) { 15 | int len = strlen(path); 16 | inPath = ""; 17 | for (int i = 0; i < len; i++) 18 | inPath = inPath + path[i]; 19 | printf("Input Files Path : %s\n", inPath.c_str()); 20 | } 21 | 22 | int *lefHead, *rigHead; 23 | int *lefTail, *rigTail; 24 | 25 | struct Triple { 26 | int h, r, t; 27 | }; 28 | 29 | struct cmp_head { 30 | bool operator()(const Triple &a, const Triple &b) { 31 | return (a.h < b.h)||(a.h == b.h && a.r < b.r)||(a.h == b.h && a.r == b.r && a.t < b.t); 32 | } 33 | }; 34 | 35 | struct cmp_tail { 36 | bool operator()(const Triple &a, const Triple &b) { 37 | return (a.t < b.t)||(a.t == b.t && a.r < b.r)||(a.t == b.t && a.r == b.r && a.h < b.h); 38 | } 39 | }; 40 | 41 | struct cmp_list { 42 | int minimal(int a,int b) { 43 | if (a > b) return b; 44 | return a; 45 | } 46 | bool operator()(const Triple &a, const Triple &b) { 47 | return (minimal(a.h, a.t) > minimal(b.h, b.t)); 48 | } 49 | }; 50 | 51 | Triple *trainHead, *trainTail, *trainList; 52 | int relationTotal, entityTotal, tripleTotal; 53 | int *freqRel, *freqEnt; 54 | float *left_mean, *right_mean; 55 | 56 | extern "C" 57 | void init() { 58 | 59 | FILE *fin; 60 | int tmp; 61 | 62 | fin = fopen((inPath + "relation2id.txt").c_str(), "r"); 63 | tmp = fscanf(fin, "%d", &relationTotal); 64 | fclose(fin); 65 | printf("%d\n", relationTotal); 66 | 67 | freqRel = (int *)calloc(relationTotal, sizeof(int)); 68 | 69 | fin = fopen((inPath + "entity2id.txt").c_str(), "r"); 70 | tmp = fscanf(fin, "%d", &entityTotal); 71 | fclose(fin); 72 | printf("%d\n", entityTotal); 73 | 74 | freqEnt = (int *)calloc(entityTotal, sizeof(int)); 75 | 76 | fin = fopen((inPath + "triple2id.txt").c_str(), "r"); 77 | tmp = fscanf(fin, "%d", &tripleTotal); 78 | printf("%d\n", tripleTotal); 79 | trainHead = (Triple *)calloc(tripleTotal, sizeof(Triple)); 80 | trainTail = (Triple *)calloc(tripleTotal, sizeof(Triple)); 81 | trainList = (Triple *)calloc(tripleTotal, sizeof(Triple)); 82 | tripleTotal = 0; 83 | while (fscanf(fin, "%d", &trainList[tripleTotal].h) == 1) { 84 | tmp = fscanf(fin, "%d", &trainList[tripleTotal].t); 85 | tmp = fscanf(fin, "%d", &trainList[tripleTotal].r); 86 | freqEnt[trainList[tripleTotal].t]++; 87 | freqEnt[trainList[tripleTotal].h]++; 88 | freqRel[trainList[tripleTotal].r]++; 89 | trainHead[tripleTotal].h = trainList[tripleTotal].h; 90 | trainHead[tripleTotal].t = trainList[tripleTotal].t; 91 | trainHead[tripleTotal].r = trainList[tripleTotal].r; 92 | trainTail[tripleTotal].h = trainList[tripleTotal].h; 93 | trainTail[tripleTotal].t = trainList[tripleTotal].t; 94 | trainTail[tripleTotal].r = trainList[tripleTotal].r; 95 | tripleTotal++; 96 | } 97 | fclose(fin); 98 | 99 | sort(trainHead, trainHead + tripleTotal, cmp_head()); 100 | sort(trainTail, trainTail + tripleTotal, cmp_tail()); 101 | 102 | lefHead = (int *)calloc(entityTotal, sizeof(int)); 103 | rigHead = (int *)calloc(entityTotal, sizeof(int)); 104 | lefTail = (int *)calloc(entityTotal, sizeof(int)); 105 | rigTail = (int *)calloc(entityTotal, sizeof(int)); 106 | memset(rigHead, -1, sizeof(rigHead)); 107 | memset(rigTail, -1, sizeof(rigTail)); 108 | for (int i = 1; i < tripleTotal; i++) { 109 | if (trainTail[i].t != trainTail[i - 1].t) { 110 | rigTail[trainTail[i - 1].t] = i - 1; 111 | lefTail[trainTail[i].t] = i; 112 | } 113 | if (trainHead[i].h != trainHead[i - 1].h) { 114 | rigHead[trainHead[i - 1].h] = i - 1; 115 | lefHead[trainHead[i].h] = i; 116 | } 117 | } 118 | rigHead[trainHead[tripleTotal - 1].h] = tripleTotal - 1; 119 | rigTail[trainTail[tripleTotal - 1].t] = tripleTotal - 1; 120 | 121 | left_mean = (float *)calloc(relationTotal,sizeof(float)); 122 | right_mean = (float *)calloc(relationTotal,sizeof(float)); 123 | for (int i = 0; i < entityTotal; i++) { 124 | for (int j = lefHead[i] + 1; j < rigHead[i]; j++) 125 | if (trainHead[j].r != trainHead[j - 1].r) 126 | left_mean[trainHead[j].r] += 1.0; 127 | if (lefHead[i] <= rigHead[i]) 128 | left_mean[trainHead[lefHead[i]].r] += 1.0; 129 | for (int j = lefTail[i] + 1; j < rigTail[i]; j++) 130 | if (trainTail[j].r != trainTail[j - 1].r) 131 | right_mean[trainTail[j].r] += 1.0; 132 | if (lefTail[i] <= rigTail[i]) 133 | right_mean[trainTail[lefTail[i]].r] += 1.0; 134 | } 135 | for (int i = 0; i < relationTotal; i++) { 136 | left_mean[i] = freqRel[i] / left_mean[i]; 137 | right_mean[i] = freqRel[i] / right_mean[i]; 138 | } 139 | } 140 | 141 | extern "C" 142 | int getEntityTotal() { 143 | return entityTotal; 144 | } 145 | 146 | extern "C" 147 | int getRelationTotal() { 148 | return relationTotal; 149 | } 150 | 151 | extern "C" 152 | int getTripleTotal() { 153 | return tripleTotal; 154 | } 155 | 156 | // unsigned long long *next_random; 157 | unsigned long long next_random = 3; 158 | 159 | unsigned long long randd(int id) { 160 | next_random = next_random * (unsigned long long)25214903917 + 11; 161 | return next_random; 162 | } 163 | 164 | int rand_max(int id, int x) { 165 | int res = randd(id) % x; 166 | while (res<0) 167 | res+=x; 168 | return res; 169 | } 170 | 171 | int corrupt_head(int id, int h, int r) { 172 | int lef, rig, mid, ll, rr; 173 | lef = lefHead[h] - 1; 174 | rig = rigHead[h]; 175 | while (lef + 1 < rig) { 176 | mid = (lef + rig) >> 1; 177 | if (trainHead[mid].r >= r) rig = mid; else 178 | lef = mid; 179 | } 180 | ll = rig; 181 | lef = lefHead[h]; 182 | rig = rigHead[h] + 1; 183 | while (lef + 1 < rig) { 184 | mid = (lef + rig) >> 1; 185 | if (trainHead[mid].r <= r) lef = mid; else 186 | rig = mid; 187 | } 188 | rr = lef; 189 | int tmp = rand_max(id, entityTotal - (rr - ll + 1)); 190 | if (tmp < trainHead[ll].t) return tmp; 191 | if (tmp > trainHead[rr].t - rr + ll - 1) return tmp + rr - ll + 1; 192 | lef = ll, rig = rr + 1; 193 | while (lef + 1 < rig) { 194 | mid = (lef + rig) >> 1; 195 | if (trainHead[mid].t - mid + ll - 1 < tmp) 196 | lef = mid; 197 | else 198 | rig = mid; 199 | } 200 | return tmp + lef - ll + 1; 201 | } 202 | 203 | int corrupt_tail(int id, int t, int r) { 204 | int lef, rig, mid, ll, rr; 205 | lef = lefTail[t] - 1; 206 | rig = rigTail[t]; 207 | while (lef + 1 < rig) { 208 | mid = (lef + rig) >> 1; 209 | if (trainTail[mid].r >= r) rig = mid; else 210 | lef = mid; 211 | } 212 | ll = rig; 213 | lef = lefTail[t]; 214 | rig = rigTail[t] + 1; 215 | while (lef + 1 < rig) { 216 | mid = (lef + rig) >> 1; 217 | if (trainTail[mid].r <= r) lef = mid; else 218 | rig = mid; 219 | } 220 | rr = lef; 221 | int tmp = rand_max(id, entityTotal - (rr - ll + 1)); 222 | if (tmp < trainTail[ll].h) return tmp; 223 | if (tmp > trainTail[rr].h - rr + ll - 1) return tmp + rr - ll + 1; 224 | lef = ll, rig = rr + 1; 225 | while (lef + 1 < rig) { 226 | mid = (lef + rig) >> 1; 227 | if (trainTail[mid].h - mid + ll - 1 < tmp) 228 | lef = mid; 229 | else 230 | rig = mid; 231 | } 232 | return tmp + lef - ll + 1; 233 | } 234 | 235 | extern "C" 236 | void getBatch(int *ph, int *pt, int *pr, int *nh, int *nt, int *nr, int batchSize, int id = 0) { 237 | for (int batch = 0; batch < batchSize; batch++) { 238 | int i = rand_max(id, tripleTotal), j; 239 | float prob = 1000 * right_mean[trainList[i].r] / (right_mean[trainList[i].r] + left_mean[trainList[i].r]); 240 | if (randd(id) % 1000 < prob) { 241 | j = corrupt_head(id, trainList[i].h, trainList[i].r); 242 | ph[batch] = trainList[i].h; 243 | pt[batch] = trainList[i].t; 244 | pr[batch] = trainList[i].r; 245 | nh[batch] = trainList[i].h; 246 | nt[batch] = j; 247 | nr[batch] = trainList[i].r; 248 | } else { 249 | j = corrupt_tail(id, trainList[i].t, trainList[i].r); 250 | ph[batch] = trainList[i].h; 251 | pt[batch] = trainList[i].t; 252 | pr[batch] = trainList[i].r; 253 | nh[batch] = j; 254 | nt[batch] = trainList[i].t; 255 | nr[batch] = trainList[i].r; 256 | } 257 | } 258 | } -------------------------------------------------------------------------------- /jointD/init.so: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thunlp/JointNRE/85c6009828b64439ef31b49e230fbd6ec24a6f8a/jointD/init.so -------------------------------------------------------------------------------- /jointD/make.sh: -------------------------------------------------------------------------------- 1 | g++ init.cpp -o init.so -fPIC -shared -pthread -O3 -march=native 2 | -------------------------------------------------------------------------------- /jointD/network.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thunlp/JointNRE/85c6009828b64439ef31b49e230fbd6ec24a6f8a/jointD/network.pyc -------------------------------------------------------------------------------- /jointD/pr_plot.py: -------------------------------------------------------------------------------- 1 | import os 2 | import numpy as np 3 | from sklearn.metrics import precision_recall_curve 4 | from sklearn.metrics import average_precision_score 5 | import matplotlib 6 | matplotlib.use('Agg') 7 | import matplotlib.pyplot as plt 8 | import numpy as np 9 | import sys 10 | 11 | from matplotlib.backends.backend_pdf import PdfPages 12 | 13 | 14 | ff = plt.figure() 15 | 16 | MODEL = 'cnn' 17 | 18 | def guolv(recall, precision): 19 | a = [recall[0]] 20 | b = [precision[0]] 21 | print len(recall) 22 | for i in range(1, len(recall)): 23 | if a[len(a) - 1] == recall[i]: 24 | if precision[i] > b[len(b)-1]: 25 | b[len(b)-1] = precision[i] 26 | else: 27 | a.append(recall[i]) 28 | b.append(precision[i]) 29 | 30 | recall = np.array(a) 31 | precision = np.array(b) 32 | xnew = np.linspace(recall.min(),recall.max(), 500) #300 represents number of points to make between T.min and T.max 33 | print recall 34 | print precision 35 | power_smooth = spline(recall,precision,xnew) 36 | return xnew, power_smooth 37 | 38 | def PrecisionAtRecall(pAll, rAll, rMark): 39 | length = len(rAll) 40 | lo = 0 41 | hi = length - 1 42 | mark = length >> 1 43 | error = rMark - rAll[mark] 44 | while np.abs(error) > 0.005: 45 | if error > 0: 46 | hi = mark - 1 47 | else: 48 | lo = mark + 1 49 | mark = (hi + lo) >> 1 50 | error = rMark - rAll[mark] 51 | return pAll[mark], rAll[mark], mark 52 | 53 | 54 | color = ['red', 'turquoise', 'darkorange', 'cornflowerblue', 'teal'] 55 | 56 | test_model = ['cnn'+'+sen_att'] 57 | test_epoch = ['9'] 58 | avg_pres = [] 59 | for temp, (model, step) in enumerate(zip(test_model, test_epoch)): 60 | y_scores = np.load(model+'_all_prob' + '_' + step + '.npy') 61 | y_true = np.load(model+'_all_label' + '_' + step + '.npy') 62 | y_scores = np.reshape(y_scores,(-1)) 63 | y_true = np.reshape(y_true,(-1)) 64 | precision,recall,threshold = precision_recall_curve(y_true,y_scores) 65 | p,r,i = PrecisionAtRecall(precision, recall, 0.1) 66 | print('precison: {}, recall: {}'.format(p, r)) 67 | p,r,i = PrecisionAtRecall(precision, recall, 0.2) 68 | print('precison: {}, recall: {}'.format(p, r)) 69 | p,r,i = PrecisionAtRecall(precision, recall, 0.3) 70 | print('precison: {}, recall: {}'.format(p, r)) 71 | average_precision = average_precision_score(y_true, y_scores) 72 | avg_pres.append(average_precision) 73 | recall = recall[::-1] 74 | precision = precision[::-1] 75 | plt.plot(recall[:], precision[:], lw=2, color=color[1],label="CNN+ATT") 76 | 77 | lines_cnn = open('cnn.txt').readlines() 78 | lines_cnn = [t.strip().split()[:2] for t in lines_cnn] 79 | precision_cnn = np.array([t[0] for t in lines_cnn], dtype=np.float32) 80 | recall_cnn = np.array([t[1] for t in lines_cnn], dtype=np.float32) 81 | plt.plot(recall_cnn, precision_cnn, lw=2, color=color[-1], label="CNN+ATT") 82 | 83 | 84 | plt.xlabel('Recall') 85 | plt.ylabel('Precision') 86 | plt.ylim([0.3, 1.0]) 87 | plt.xlim([0.0, 0.4]) 88 | plt.title('Precision-Recall Area={0:0.4f}'.format(avg_pres[-1])) 89 | plt.legend(loc="upper right") 90 | plt.grid(True) 91 | plt.savefig('sgd_'+MODEL) 92 | plt.plot(range(10), range(10), "o") 93 | plt.show() 94 | ff.savefig("pr.pdf", bbox_inches='tight') 95 | -------------------------------------------------------------------------------- /jointD/test.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | import numpy as np 3 | import time 4 | import datetime 5 | import os 6 | import network 7 | import json 8 | import sys 9 | from sklearn.metrics import average_precision_score 10 | import ctypes 11 | 12 | export_path = "../data/" 13 | 14 | word_vec = np.load(export_path + 'vec.npy') 15 | f = open(export_path + "config", 'r') 16 | config = json.loads(f.read()) 17 | f.close() 18 | 19 | ll = ctypes.cdll.LoadLibrary 20 | lib = ll("./init.so") 21 | lib.setInPath("../data/") 22 | lib.init() 23 | 24 | FLAGS = tf.app.flags.FLAGS 25 | 26 | tf.app.flags.DEFINE_integer('nbatch_kg',100,'entity numbers used each training time') 27 | tf.app.flags.DEFINE_float('margin',1.0,'entity numbers used each training time') 28 | tf.app.flags.DEFINE_float('learning_rate_kg',0.001,'learning rate for kg') 29 | tf.app.flags.DEFINE_integer('ent_total',lib.getEntityTotal(),'total of entities') 30 | tf.app.flags.DEFINE_integer('rel_total',lib.getRelationTotal(),'total of relations') 31 | tf.app.flags.DEFINE_integer('tri_total',lib.getTripleTotal(),'total of triples') 32 | tf.app.flags.DEFINE_integer('katt_flag', 1, '1 for katt, 0 for att') 33 | 34 | tf.app.flags.DEFINE_string('model', 'cnn', 'neural models to encode sentences') 35 | tf.app.flags.DEFINE_integer('max_length',config['fixlen'],'maximum of number of words in one sentence') 36 | tf.app.flags.DEFINE_integer('pos_num', config['maxlen'] * 2 + 1,'number of position embedding vectors') 37 | tf.app.flags.DEFINE_integer('num_classes', config['textual_rel_total'],'maximum of relations') 38 | 39 | tf.app.flags.DEFINE_integer('hidden_size',230,'hidden feature size') 40 | tf.app.flags.DEFINE_integer('pos_size',5,'position embedding size') 41 | 42 | tf.app.flags.DEFINE_integer('max_epoch',30,'maximum of training epochs') 43 | tf.app.flags.DEFINE_integer('batch_size',131*2,'entity numbers used each training time') 44 | tf.app.flags.DEFINE_float('learning_rate',0.1,'entity numbers used each training time') 45 | tf.app.flags.DEFINE_float('weight_decay',0.00001,'weight_decay') 46 | tf.app.flags.DEFINE_float('keep_prob',1.0,'dropout rate') 47 | 48 | tf.app.flags.DEFINE_integer('test_batch_size',131*2,'entity numbers used each test time') 49 | tf.app.flags.DEFINE_string('checkpoint_path','./model/','path to store model') 50 | 51 | 52 | def make_shape(array,last_dim): 53 | output = [] 54 | for i in array: 55 | for j in i: 56 | output.append(j) 57 | output = np.array(output) 58 | if np.shape(output)[-1]==last_dim: 59 | return output 60 | 61 | else: 62 | print 'Make Shape Error!' 63 | 64 | def main(_): 65 | 66 | print 'reading word embedding' 67 | word_vec = np.load(export_path + 'vec.npy') 68 | print 'reading test data' 69 | test_instance_triple = np.load(export_path + 'test_instance_triple.npy') 70 | test_instance_scope = np.load(export_path + 'test_instance_scope.npy') 71 | test_len = np.load(export_path + 'test_len.npy') 72 | test_label = np.load(export_path + 'test_label.npy') 73 | test_word = np.load(export_path + 'test_word.npy') 74 | test_pos1 = np.load(export_path + 'test_pos1.npy') 75 | test_pos2 = np.load(export_path + 'test_pos2.npy') 76 | test_mask = np.load(export_path + 'test_mask.npy') 77 | test_head = np.load(export_path + 'test_head.npy') 78 | test_tail = np.load(export_path + 'test_tail.npy') 79 | print 'reading finished' 80 | print 'mentions : %d' % (len(test_instance_triple)) 81 | print 'sentences : %d' % (len(test_len)) 82 | print 'relations : %d' % (FLAGS.num_classes) 83 | print 'word size : %d' % (len(word_vec[0])) 84 | print 'position size : %d' % (FLAGS.pos_size) 85 | print 'hidden size : %d' % (FLAGS.hidden_size) 86 | print 'reading finished' 87 | 88 | print 'building network...' 89 | sess = tf.Session() 90 | if FLAGS.model.lower() == "cnn": 91 | model = network.CNN(is_training = False, word_embeddings = word_vec) 92 | elif FLAGS.model.lower() == "pcnn": 93 | model = network.PCNN(is_training = False, word_embeddings = word_vec) 94 | elif FLAGS.model.lower() == "lstm": 95 | model = network.RNN(is_training = False, word_embeddings = word_vec, cell_name = "LSTM", simple_position = True) 96 | elif FLAGS.model.lower() == "gru": 97 | model = network.RNN(is_training = False, word_embeddings = word_vec, cell_name = "GRU", simple_position = True) 98 | elif FLAGS.model.lower() == "bi-lstm" or FLAGS.model.lower() == "bilstm": 99 | model = network.BiRNN(is_training = False, word_embeddings = word_vec, cell_name = "LSTM", simple_position = True) 100 | elif FLAGS.model.lower() == "bi-gru" or FLAGS.model.lower() == "bigru": 101 | model = network.BiRNN(is_training = False, word_embeddings = word_vec, cell_name = "GRU", simple_position = True) 102 | sess.run(tf.global_variables_initializer()) 103 | saver = tf.train.Saver() 104 | 105 | def test_step(head, tail, word, pos1, pos2, mask, leng, label_index, label, scope): 106 | feed_dict = { 107 | model.head_index: head, 108 | model.tail_index: tail, 109 | model.word: word, 110 | model.pos1: pos1, 111 | model.pos2: pos2, 112 | model.mask: mask, 113 | model.len : leng, 114 | model.label_index: label_index, 115 | model.label: label, 116 | model.scope: scope, 117 | model.keep_prob: FLAGS.keep_prob 118 | } 119 | output = sess.run(model.test_output, feed_dict) 120 | return output 121 | 122 | f = open('results.txt','w') 123 | f.write('iteration\taverage precision\n') 124 | for iters in range(1,30): 125 | print iters 126 | saver.restore(sess, FLAGS.checkpoint_path + FLAGS.model+str(FLAGS.katt_flag)+"-"+str(3664*iters)) 127 | 128 | stack_output = [] 129 | stack_label = [] 130 | 131 | iteration = len(test_instance_scope)/FLAGS.test_batch_size 132 | for i in range(iteration): 133 | temp_str= 'running '+str(i)+'/'+str(iteration)+'...' 134 | sys.stdout.write(temp_str+'\r') 135 | sys.stdout.flush() 136 | input_scope = test_instance_scope[i * FLAGS.test_batch_size:(i+1)*FLAGS.test_batch_size] 137 | index = [] 138 | scope = [0] 139 | label = [] 140 | for num in input_scope: 141 | index = index + range(num[0], num[1] + 1) 142 | label.append(test_label[num[0]]) 143 | scope.append(scope[len(scope)-1] + num[1] - num[0] + 1) 144 | label_ = np.zeros((FLAGS.test_batch_size, FLAGS.num_classes)) 145 | label_[np.arange(FLAGS.test_batch_size), label] = 1 146 | output = test_step(test_head[index], test_tail[index], test_word[index,:], test_pos1[index,:], test_pos2[index,:], test_mask[index,:], test_len[index], test_label[index], label_, np.array(scope)) 147 | stack_output.append(output) 148 | stack_label.append(label_) 149 | 150 | print 'evaluating...' 151 | 152 | stack_output = np.concatenate(stack_output, axis=0) 153 | stack_label = np.concatenate(stack_label, axis = 0) 154 | 155 | exclude_na_flatten_output = stack_output[:,1:] 156 | exclude_na_flatten_label = stack_label[:,1:] 157 | print exclude_na_flatten_output.shape 158 | print exclude_na_flatten_label.shape 159 | 160 | average_precision = average_precision_score(exclude_na_flatten_label,exclude_na_flatten_output, average = "micro") 161 | 162 | np.save('./'+FLAGS.model+'+sen_att_all_prob_'+str(iters)+'.npy', exclude_na_flatten_output) 163 | np.save('./'+FLAGS.model+'+sen_att_all_label_'+str(iters)+'.npy',exclude_na_flatten_label) 164 | 165 | print 'pr: '+str(average_precision) 166 | f.write(str(average_precision)+'\n') 167 | f.close() 168 | 169 | if __name__ == "__main__": 170 | tf.app.run() 171 | -------------------------------------------------------------------------------- /jointD/train.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | import numpy as np 3 | import time 4 | import datetime 5 | import os 6 | import network 7 | import json 8 | from sklearn.metrics import average_precision_score 9 | import sys 10 | import ctypes 11 | import threading 12 | 13 | export_path = "../data/" 14 | 15 | word_vec = np.load(export_path + 'vec.npy') 16 | f = open(export_path + "config", 'r') 17 | config = json.loads(f.read()) 18 | f.close() 19 | 20 | ll = ctypes.cdll.LoadLibrary 21 | lib = ll("./init.so") 22 | lib.setInPath("../data/") 23 | lib.init() 24 | 25 | FLAGS = tf.app.flags.FLAGS 26 | 27 | tf.app.flags.DEFINE_float('nbatch_kg',100,'entity numbers used each training time') 28 | tf.app.flags.DEFINE_float('margin',1.0,'entity numbers used each training time') 29 | tf.app.flags.DEFINE_float('learning_rate_kg',0.001,'learning rate for kg') 30 | tf.app.flags.DEFINE_integer('ent_total',lib.getEntityTotal(),'total of entities') 31 | tf.app.flags.DEFINE_integer('rel_total',lib.getRelationTotal(),'total of relations') 32 | tf.app.flags.DEFINE_integer('tri_total',lib.getTripleTotal(),'total of triples') 33 | tf.app.flags.DEFINE_integer('katt_flag', 1, '1 for katt, 0 for att') 34 | 35 | tf.app.flags.DEFINE_string('model', 'cnn', 'neural models to encode sentences') 36 | tf.app.flags.DEFINE_integer('max_length',config['fixlen'],'maximum of number of words in one sentence') 37 | tf.app.flags.DEFINE_integer('pos_num', config['maxlen'] * 2 + 1,'number of position embedding vectors') 38 | tf.app.flags.DEFINE_integer('num_classes', config['textual_rel_total'],'maximum of relations') 39 | 40 | tf.app.flags.DEFINE_integer('hidden_size',230,'hidden feature size') 41 | tf.app.flags.DEFINE_integer('pos_size',5,'position embedding size') 42 | 43 | tf.app.flags.DEFINE_integer('max_epoch',20,'maximum of training epochs') 44 | tf.app.flags.DEFINE_integer('batch_size',160,'entity numbers used each training time') 45 | tf.app.flags.DEFINE_float('learning_rate',0.5,'learning rate for nn') 46 | tf.app.flags.DEFINE_float('weight_decay',0.00001,'weight_decay') 47 | tf.app.flags.DEFINE_float('keep_prob',0.5,'dropout rate') 48 | 49 | tf.app.flags.DEFINE_string('model_dir','./model/','path to store model') 50 | tf.app.flags.DEFINE_string('summary_dir','./summary','path to store summary_dir') 51 | 52 | 53 | def MakeSummary(name, value): 54 | """Creates a tf.Summary proto with the given name and value.""" 55 | summary = tf.Summary() 56 | val = summary.value.add() 57 | val.tag = str(name) 58 | val.simple_value = float(value) 59 | return summary 60 | 61 | def make_shape(array,last_dim): 62 | output = [] 63 | for i in array: 64 | for j in i: 65 | output.append(j) 66 | output = np.array(output) 67 | if np.shape(output)[-1]==last_dim: 68 | return output 69 | else: 70 | print 'Make Shape Error!' 71 | 72 | def main(_): 73 | 74 | print 'reading word embedding' 75 | word_vec = np.load(export_path + 'vec.npy') 76 | print 'reading training data' 77 | instance_triple = np.load(export_path + 'train_instance_triple.npy') 78 | instance_scope = np.load(export_path + 'train_instance_scope.npy') 79 | train_len = np.load(export_path + 'train_len.npy') 80 | train_label = np.load(export_path + 'train_label.npy') 81 | train_word = np.load(export_path + 'train_word.npy') 82 | train_pos1 = np.load(export_path + 'train_pos1.npy') 83 | train_pos2 = np.load(export_path + 'train_pos2.npy') 84 | train_mask = np.load(export_path + 'train_mask.npy') 85 | train_head = np.load(export_path + 'train_head.npy') 86 | train_tail = np.load(export_path + 'train_tail.npy') 87 | print 'reading finished' 88 | print 'mentions : %d' % (len(instance_triple)) 89 | print 'sentences : %d' % (len(train_len)) 90 | print 'relations : %d' % (FLAGS.num_classes) 91 | print 'word size : %d' % (len(word_vec[0])) 92 | print 'position size : %d' % (FLAGS.pos_size) 93 | print 'hidden size : %d' % (FLAGS.hidden_size) 94 | reltot = {} 95 | for index, i in enumerate(train_label): 96 | if not i in reltot: 97 | reltot[i] = 1.0 98 | else: 99 | reltot[i] += 1.0 100 | for i in reltot: 101 | reltot[i] = 1/(reltot[i] ** (0.05)) 102 | print 'building network...' 103 | sess = tf.Session() 104 | if FLAGS.model.lower() == "cnn": 105 | model = network.CNN(is_training = True, word_embeddings = word_vec) 106 | elif FLAGS.model.lower() == "pcnn": 107 | model = network.PCNN(is_training = True, word_embeddings = word_vec) 108 | elif FLAGS.model.lower() == "lstm": 109 | model = network.RNN(is_training = True, word_embeddings = word_vec, cell_name = "LSTM", simple_position = True) 110 | elif FLAGS.model.lower() == "gru": 111 | model = network.RNN(is_training = True, word_embeddings = word_vec, cell_name = "GRU", simple_position = True) 112 | elif FLAGS.model.lower() == "bi-lstm" or FLAGS.model.lower() == "bilstm": 113 | model = network.BiRNN(is_training = True, word_embeddings = word_vec, cell_name = "LSTM", simple_position = True) 114 | elif FLAGS.model.lower() == "bi-gru" or FLAGS.model.lower() == "bigru": 115 | model = network.BiRNN(is_training = True, word_embeddings = word_vec, cell_name = "GRU", simple_position = True) 116 | 117 | global_step = tf.Variable(0,name='global_step',trainable=False) 118 | global_step_kg = tf.Variable(0,name='global_step_kg',trainable=False) 119 | tf.summary.scalar('learning_rate', FLAGS.learning_rate) 120 | tf.summary.scalar('learning_rate_kg', FLAGS.learning_rate_kg) 121 | 122 | optimizer = tf.train.GradientDescentOptimizer(FLAGS.learning_rate) 123 | grads_and_vars = optimizer.compute_gradients(model.loss) 124 | train_op = optimizer.apply_gradients(grads_and_vars, global_step = global_step) 125 | 126 | optimizer_kg = tf.train.GradientDescentOptimizer(FLAGS.learning_rate_kg) 127 | grads_and_vars_kg = optimizer_kg.compute_gradients(model.loss_kg) 128 | train_op_kg = optimizer_kg.apply_gradients(grads_and_vars_kg, global_step = global_step_kg) 129 | 130 | merged_summary = tf.summary.merge_all() 131 | summary_writer = tf.summary.FileWriter(FLAGS.summary_dir, sess.graph) 132 | sess.run(tf.global_variables_initializer()) 133 | saver = tf.train.Saver(max_to_keep=None) 134 | print 'building finished' 135 | 136 | def train_kg(coord): 137 | def train_step_kg(pos_h_batch, pos_t_batch, pos_r_batch, neg_h_batch, neg_t_batch, neg_r_batch): 138 | feed_dict = { 139 | model.pos_h: pos_h_batch, 140 | model.pos_t: pos_t_batch, 141 | model.pos_r: pos_r_batch, 142 | model.neg_h: neg_h_batch, 143 | model.neg_t: neg_t_batch, 144 | model.neg_r: neg_r_batch 145 | } 146 | _, step, loss = sess.run( 147 | [train_op_kg, global_step_kg, model.loss_kg], feed_dict) 148 | return loss 149 | 150 | batch_size = (FLAGS.ent_total / FLAGS.nbatch_kg) 151 | ph = np.zeros(batch_size, dtype = np.int32) 152 | pt = np.zeros(batch_size, dtype = np.int32) 153 | pr = np.zeros(batch_size, dtype = np.int32) 154 | nh = np.zeros(batch_size, dtype = np.int32) 155 | nt = np.zeros(batch_size, dtype = np.int32) 156 | nr = np.zeros(batch_size, dtype = np.int32) 157 | ph_addr = ph.__array_interface__['data'][0] 158 | pt_addr = pt.__array_interface__['data'][0] 159 | pr_addr = pr.__array_interface__['data'][0] 160 | nh_addr = nh.__array_interface__['data'][0] 161 | nt_addr = nt.__array_interface__['data'][0] 162 | nr_addr = nr.__array_interface__['data'][0] 163 | lib.getBatch.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_int] 164 | times_kg = 0 165 | while not coord.should_stop(): 166 | times_kg += 1 167 | res = 0.0 168 | for batch in range(FLAGS.nbatch_kg): 169 | lib.getBatch(ph_addr, pt_addr, pr_addr, nh_addr, nt_addr, nr_addr, batch_size) 170 | res += train_step_kg(ph, pt, pr, nh, nt, nr) 171 | time_str = datetime.datetime.now().isoformat() 172 | print "batch %d time %s | loss : %f" % (times_kg, time_str, res) 173 | 174 | 175 | def train_nn(coord): 176 | def train_step(head, tail, word, pos1, pos2, mask, leng, label_index, label, scope, weights): 177 | feed_dict = { 178 | model.head_index: head, 179 | model.tail_index: tail, 180 | model.word: word, 181 | model.pos1: pos1, 182 | model.pos2: pos2, 183 | model.mask: mask, 184 | model.len : leng, 185 | model.label_index: label_index, 186 | model.label: label, 187 | model.scope: scope, 188 | model.keep_prob: FLAGS.keep_prob, 189 | model.weights: weights 190 | } 191 | _, step, loss, summary, output, correct_predictions = sess.run([train_op, global_step, model.loss, merged_summary, model.output, model.correct_predictions], feed_dict) 192 | summary_writer.add_summary(summary, step) 193 | return output, loss, correct_predictions 194 | 195 | stack_output = [] 196 | stack_label = [] 197 | stack_ce_loss = [] 198 | 199 | train_order = range(len(instance_triple)) 200 | 201 | save_epoch = 2 202 | eval_step = 300 203 | 204 | for one_epoch in range(FLAGS.max_epoch): 205 | 206 | print('epoch '+str(one_epoch+1)+' starts!') 207 | np.random.shuffle(train_order) 208 | s1 = 0.0 209 | s2 = 0.0 210 | tot1 = 0.0 211 | tot2 = 0.0 212 | losstot = 0.0 213 | for i in range(int(len(train_order)/float(FLAGS.batch_size))): 214 | input_scope = np.take(instance_scope, train_order[i * FLAGS.batch_size:(i+1)*FLAGS.batch_size], axis=0) 215 | index = [] 216 | scope = [0] 217 | label = [] 218 | weights = [] 219 | for num in input_scope: 220 | index = index + range(num[0], num[1] + 1) 221 | label.append(train_label[num[0]]) 222 | if train_label[num[0]] > 53: 223 | print train_label[num[0]] 224 | scope.append(scope[len(scope)-1] + num[1] - num[0] + 1) 225 | weights.append(reltot[train_label[num[0]]]) 226 | label_ = np.zeros((FLAGS.batch_size, FLAGS.num_classes)) 227 | label_[np.arange(FLAGS.batch_size), label] = 1 228 | output, loss, correct_predictions = train_step(train_head[index], train_tail[index], train_word[index,:], train_pos1[index,:], train_pos2[index,:], train_mask[index,:], train_len[index],train_label[index], label_, np.array(scope), weights) 229 | num = 0 230 | s = 0 231 | losstot += loss 232 | for num in correct_predictions: 233 | if label[s] == 0: 234 | tot1 += 1.0 235 | if num: 236 | s1+= 1.0 237 | else: 238 | tot2 += 1.0 239 | if num: 240 | s2 += 1.0 241 | s = s + 1 242 | 243 | time_str = datetime.datetime.now().isoformat() 244 | print "batch %d step %d time %s | loss : %f, NA accuracy: %f, not NA accuracy: %f" % (one_epoch, i, time_str, loss, s1 / tot1, s2 / tot2) 245 | current_step = tf.train.global_step(sess, global_step) 246 | 247 | if (one_epoch + 1) % save_epoch == 0: 248 | print 'epoch '+str(one_epoch+1)+' has finished' 249 | print 'saving model...' 250 | path = saver.save(sess,FLAGS.model_dir+FLAGS.model+str(FLAGS.katt_flag), global_step=current_step) 251 | print 'have savde model to '+path 252 | 253 | coord.request_stop() 254 | 255 | 256 | coord = tf.train.Coordinator() 257 | threads = [] 258 | threads.append(threading.Thread(target=train_kg, args=(coord,))) 259 | threads.append(threading.Thread(target=train_nn, args=(coord,))) 260 | for t in threads: t.start() 261 | coord.join(threads) 262 | 263 | if __name__ == "__main__": 264 | tf.app.run() 265 | -------------------------------------------------------------------------------- /jointE/KATT/init.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | 9 | using namespace std; 10 | 11 | string inPath = "./data/"; 12 | 13 | extern "C" 14 | void setInPath(char *path) { 15 | int len = strlen(path); 16 | inPath = ""; 17 | for (int i = 0; i < len; i++) 18 | inPath = inPath + path[i]; 19 | printf("Input Files Path : %s\n", inPath.c_str()); 20 | } 21 | 22 | int *lefHead, *rigHead; 23 | int *lefTail, *rigTail; 24 | 25 | struct Triple { 26 | int h, r, t; 27 | }; 28 | 29 | struct cmp_head { 30 | bool operator()(const Triple &a, const Triple &b) { 31 | return (a.h < b.h)||(a.h == b.h && a.r < b.r)||(a.h == b.h && a.r == b.r && a.t < b.t); 32 | } 33 | }; 34 | 35 | struct cmp_tail { 36 | bool operator()(const Triple &a, const Triple &b) { 37 | return (a.t < b.t)||(a.t == b.t && a.r < b.r)||(a.t == b.t && a.r == b.r && a.h < b.h); 38 | } 39 | }; 40 | 41 | struct cmp_list { 42 | int minimal(int a,int b) { 43 | if (a > b) return b; 44 | return a; 45 | } 46 | bool operator()(const Triple &a, const Triple &b) { 47 | return (minimal(a.h, a.t) > minimal(b.h, b.t)); 48 | } 49 | }; 50 | 51 | Triple *trainHead, *trainTail, *trainList; 52 | int relationTotal, entityTotal, tripleTotal; 53 | int *freqRel, *freqEnt; 54 | float *left_mean, *right_mean; 55 | 56 | extern "C" 57 | void init() { 58 | 59 | FILE *fin; 60 | int tmp; 61 | 62 | fin = fopen((inPath + "relation2id.txt").c_str(), "r"); 63 | tmp = fscanf(fin, "%d", &relationTotal); 64 | fclose(fin); 65 | printf("%d\n", relationTotal); 66 | 67 | freqRel = (int *)calloc(relationTotal, sizeof(int)); 68 | 69 | fin = fopen((inPath + "entity2id.txt").c_str(), "r"); 70 | tmp = fscanf(fin, "%d", &entityTotal); 71 | fclose(fin); 72 | printf("%d\n", entityTotal); 73 | 74 | freqEnt = (int *)calloc(entityTotal, sizeof(int)); 75 | 76 | fin = fopen((inPath + "triple2id.txt").c_str(), "r"); 77 | tmp = fscanf(fin, "%d", &tripleTotal); 78 | printf("%d\n", tripleTotal); 79 | trainHead = (Triple *)calloc(tripleTotal, sizeof(Triple)); 80 | trainTail = (Triple *)calloc(tripleTotal, sizeof(Triple)); 81 | trainList = (Triple *)calloc(tripleTotal, sizeof(Triple)); 82 | tripleTotal = 0; 83 | while (fscanf(fin, "%d", &trainList[tripleTotal].h) == 1) { 84 | tmp = fscanf(fin, "%d", &trainList[tripleTotal].t); 85 | tmp = fscanf(fin, "%d", &trainList[tripleTotal].r); 86 | freqEnt[trainList[tripleTotal].t]++; 87 | freqEnt[trainList[tripleTotal].h]++; 88 | freqRel[trainList[tripleTotal].r]++; 89 | trainHead[tripleTotal].h = trainList[tripleTotal].h; 90 | trainHead[tripleTotal].t = trainList[tripleTotal].t; 91 | trainHead[tripleTotal].r = trainList[tripleTotal].r; 92 | trainTail[tripleTotal].h = trainList[tripleTotal].h; 93 | trainTail[tripleTotal].t = trainList[tripleTotal].t; 94 | trainTail[tripleTotal].r = trainList[tripleTotal].r; 95 | tripleTotal++; 96 | } 97 | fclose(fin); 98 | 99 | sort(trainHead, trainHead + tripleTotal, cmp_head()); 100 | sort(trainTail, trainTail + tripleTotal, cmp_tail()); 101 | 102 | lefHead = (int *)calloc(entityTotal, sizeof(int)); 103 | rigHead = (int *)calloc(entityTotal, sizeof(int)); 104 | lefTail = (int *)calloc(entityTotal, sizeof(int)); 105 | rigTail = (int *)calloc(entityTotal, sizeof(int)); 106 | memset(rigHead, -1, sizeof(rigHead)); 107 | memset(rigTail, -1, sizeof(rigTail)); 108 | for (int i = 1; i < tripleTotal; i++) { 109 | if (trainTail[i].t != trainTail[i - 1].t) { 110 | rigTail[trainTail[i - 1].t] = i - 1; 111 | lefTail[trainTail[i].t] = i; 112 | } 113 | if (trainHead[i].h != trainHead[i - 1].h) { 114 | rigHead[trainHead[i - 1].h] = i - 1; 115 | lefHead[trainHead[i].h] = i; 116 | } 117 | } 118 | rigHead[trainHead[tripleTotal - 1].h] = tripleTotal - 1; 119 | rigTail[trainTail[tripleTotal - 1].t] = tripleTotal - 1; 120 | 121 | left_mean = (float *)calloc(relationTotal,sizeof(float)); 122 | right_mean = (float *)calloc(relationTotal,sizeof(float)); 123 | for (int i = 0; i < entityTotal; i++) { 124 | for (int j = lefHead[i] + 1; j < rigHead[i]; j++) 125 | if (trainHead[j].r != trainHead[j - 1].r) 126 | left_mean[trainHead[j].r] += 1.0; 127 | if (lefHead[i] <= rigHead[i]) 128 | left_mean[trainHead[lefHead[i]].r] += 1.0; 129 | for (int j = lefTail[i] + 1; j < rigTail[i]; j++) 130 | if (trainTail[j].r != trainTail[j - 1].r) 131 | right_mean[trainTail[j].r] += 1.0; 132 | if (lefTail[i] <= rigTail[i]) 133 | right_mean[trainTail[lefTail[i]].r] += 1.0; 134 | } 135 | for (int i = 0; i < relationTotal; i++) { 136 | left_mean[i] = freqRel[i] / left_mean[i]; 137 | right_mean[i] = freqRel[i] / right_mean[i]; 138 | } 139 | } 140 | 141 | extern "C" 142 | int getEntityTotal() { 143 | return entityTotal; 144 | } 145 | 146 | extern "C" 147 | int getRelationTotal() { 148 | return relationTotal; 149 | } 150 | 151 | extern "C" 152 | int getTripleTotal() { 153 | return tripleTotal; 154 | } 155 | 156 | // unsigned long long *next_random; 157 | unsigned long long next_random = 3; 158 | 159 | unsigned long long randd(int id) { 160 | next_random = next_random * (unsigned long long)25214903917 + 11; 161 | return next_random; 162 | } 163 | 164 | int rand_max(int id, int x) { 165 | int res = randd(id) % x; 166 | while (res<0) 167 | res+=x; 168 | return res; 169 | } 170 | 171 | int corrupt_head(int id, int h, int r) { 172 | int lef, rig, mid, ll, rr; 173 | lef = lefHead[h] - 1; 174 | rig = rigHead[h]; 175 | while (lef + 1 < rig) { 176 | mid = (lef + rig) >> 1; 177 | if (trainHead[mid].r >= r) rig = mid; else 178 | lef = mid; 179 | } 180 | ll = rig; 181 | lef = lefHead[h]; 182 | rig = rigHead[h] + 1; 183 | while (lef + 1 < rig) { 184 | mid = (lef + rig) >> 1; 185 | if (trainHead[mid].r <= r) lef = mid; else 186 | rig = mid; 187 | } 188 | rr = lef; 189 | int tmp = rand_max(id, entityTotal - (rr - ll + 1)); 190 | if (tmp < trainHead[ll].t) return tmp; 191 | if (tmp > trainHead[rr].t - rr + ll - 1) return tmp + rr - ll + 1; 192 | lef = ll, rig = rr + 1; 193 | while (lef + 1 < rig) { 194 | mid = (lef + rig) >> 1; 195 | if (trainHead[mid].t - mid + ll - 1 < tmp) 196 | lef = mid; 197 | else 198 | rig = mid; 199 | } 200 | return tmp + lef - ll + 1; 201 | } 202 | 203 | int corrupt_tail(int id, int t, int r) { 204 | int lef, rig, mid, ll, rr; 205 | lef = lefTail[t] - 1; 206 | rig = rigTail[t]; 207 | while (lef + 1 < rig) { 208 | mid = (lef + rig) >> 1; 209 | if (trainTail[mid].r >= r) rig = mid; else 210 | lef = mid; 211 | } 212 | ll = rig; 213 | lef = lefTail[t]; 214 | rig = rigTail[t] + 1; 215 | while (lef + 1 < rig) { 216 | mid = (lef + rig) >> 1; 217 | if (trainTail[mid].r <= r) lef = mid; else 218 | rig = mid; 219 | } 220 | rr = lef; 221 | int tmp = rand_max(id, entityTotal - (rr - ll + 1)); 222 | if (tmp < trainTail[ll].h) return tmp; 223 | if (tmp > trainTail[rr].h - rr + ll - 1) return tmp + rr - ll + 1; 224 | lef = ll, rig = rr + 1; 225 | while (lef + 1 < rig) { 226 | mid = (lef + rig) >> 1; 227 | if (trainTail[mid].h - mid + ll - 1 < tmp) 228 | lef = mid; 229 | else 230 | rig = mid; 231 | } 232 | return tmp + lef - ll + 1; 233 | } 234 | 235 | extern "C" 236 | void getBatch(int *ph, int *pt, int *pr, int *nh, int *nt, int *nr, int batchSize, int id = 0) { 237 | for (int batch = 0; batch < batchSize; batch++) { 238 | int i = rand_max(id, tripleTotal), j; 239 | float prob = 1000 * right_mean[trainList[i].r] / (right_mean[trainList[i].r] + left_mean[trainList[i].r]); 240 | if (randd(id) % 1000 < prob) { 241 | j = corrupt_head(id, trainList[i].h, trainList[i].r); 242 | ph[batch] = trainList[i].h; 243 | pt[batch] = trainList[i].t; 244 | pr[batch] = trainList[i].r; 245 | nh[batch] = trainList[i].h; 246 | nt[batch] = j; 247 | nr[batch] = trainList[i].r; 248 | } else { 249 | j = corrupt_tail(id, trainList[i].t, trainList[i].r); 250 | ph[batch] = trainList[i].h; 251 | pt[batch] = trainList[i].t; 252 | pr[batch] = trainList[i].r; 253 | nh[batch] = j; 254 | nt[batch] = trainList[i].t; 255 | nr[batch] = trainList[i].r; 256 | } 257 | } 258 | } -------------------------------------------------------------------------------- /jointE/KATT/make.sh: -------------------------------------------------------------------------------- 1 | g++ init.cpp -o init.so -fPIC -shared -pthread -O3 -march=native 2 | -------------------------------------------------------------------------------- /jointE/KATT/network.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thunlp/JointNRE/85c6009828b64439ef31b49e230fbd6ec24a6f8a/jointE/KATT/network.pyc -------------------------------------------------------------------------------- /jointE/KATT/pr_plot.py: -------------------------------------------------------------------------------- 1 | import os 2 | import numpy as np 3 | from sklearn.metrics import precision_recall_curve 4 | from sklearn.metrics import average_precision_score 5 | import matplotlib 6 | matplotlib.use('Agg') 7 | import matplotlib.pyplot as plt 8 | import numpy as np 9 | import sys 10 | 11 | from matplotlib.backends.backend_pdf import PdfPages 12 | 13 | 14 | ff = plt.figure() 15 | 16 | MODEL = 'cnn' 17 | 18 | def guolv(recall, precision): 19 | a = [recall[0]] 20 | b = [precision[0]] 21 | print len(recall) 22 | for i in range(1, len(recall)): 23 | if a[len(a) - 1] == recall[i]: 24 | if precision[i] > b[len(b)-1]: 25 | b[len(b)-1] = precision[i] 26 | else: 27 | a.append(recall[i]) 28 | b.append(precision[i]) 29 | 30 | recall = np.array(a) 31 | precision = np.array(b) 32 | xnew = np.linspace(recall.min(),recall.max(), 500) #300 represents number of points to make between T.min and T.max 33 | print recall 34 | print precision 35 | power_smooth = spline(recall,precision,xnew) 36 | return xnew, power_smooth 37 | 38 | def PrecisionAtRecall(pAll, rAll, rMark): 39 | length = len(rAll) 40 | lo = 0 41 | hi = length - 1 42 | mark = length >> 1 43 | error = rMark - rAll[mark] 44 | while np.abs(error) > 0.005: 45 | if error > 0: 46 | hi = mark - 1 47 | else: 48 | lo = mark + 1 49 | mark = (hi + lo) >> 1 50 | error = rMark - rAll[mark] 51 | return pAll[mark], rAll[mark], mark 52 | 53 | 54 | color = ['red', 'turquoise', 'darkorange', 'cornflowerblue', 'teal'] 55 | 56 | test_model = ['cnn'+'+sen_att'] 57 | test_epoch = ['9'] 58 | avg_pres = [] 59 | for temp, (model, step) in enumerate(zip(test_model, test_epoch)): 60 | y_scores = np.load(model+'_all_prob' + '_' + step + '.npy') 61 | y_true = np.load(model+'_all_label' + '_' + step + '.npy') 62 | y_scores = np.reshape(y_scores,(-1)) 63 | y_true = np.reshape(y_true,(-1)) 64 | precision,recall,threshold = precision_recall_curve(y_true,y_scores) 65 | p,r,i = PrecisionAtRecall(precision, recall, 0.1) 66 | print('precison: {}, recall: {}'.format(p, r)) 67 | p,r,i = PrecisionAtRecall(precision, recall, 0.2) 68 | print('precison: {}, recall: {}'.format(p, r)) 69 | p,r,i = PrecisionAtRecall(precision, recall, 0.3) 70 | print('precison: {}, recall: {}'.format(p, r)) 71 | average_precision = average_precision_score(y_true, y_scores) 72 | avg_pres.append(average_precision) 73 | recall = recall[::-1] 74 | precision = precision[::-1] 75 | plt.plot(recall[:], precision[:], lw=2, color=color[1],label="CNN+ATT") 76 | 77 | lines_cnn = open('cnn.txt').readlines() 78 | lines_cnn = [t.strip().split()[:2] for t in lines_cnn] 79 | precision_cnn = np.array([t[0] for t in lines_cnn], dtype=np.float32) 80 | recall_cnn = np.array([t[1] for t in lines_cnn], dtype=np.float32) 81 | plt.plot(recall_cnn, precision_cnn, lw=2, color=color[-1], label="CNN+ATT") 82 | 83 | 84 | plt.xlabel('Recall') 85 | plt.ylabel('Precision') 86 | plt.ylim([0.3, 1.0]) 87 | plt.xlim([0.0, 0.4]) 88 | plt.title('Precision-Recall Area={0:0.4f}'.format(avg_pres[-1])) 89 | plt.legend(loc="upper right") 90 | plt.grid(True) 91 | plt.savefig('sgd_'+MODEL) 92 | plt.plot(range(10), range(10), "o") 93 | plt.show() 94 | ff.savefig("pr.pdf", bbox_inches='tight') 95 | -------------------------------------------------------------------------------- /jointE/KATT/test.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | import numpy as np 3 | import time 4 | import datetime 5 | import os 6 | import network 7 | import json 8 | import sys 9 | from sklearn.metrics import average_precision_score 10 | import ctypes 11 | 12 | export_path = "../data/" 13 | 14 | word_vec = np.load(export_path + 'vec.npy') 15 | f = open(export_path + "config", 'r') 16 | config = json.loads(f.read()) 17 | f.close() 18 | 19 | ll = ctypes.cdll.LoadLibrary 20 | lib = ll("./init.so") 21 | lib.setInPath("../data/") 22 | lib.init() 23 | 24 | FLAGS = tf.app.flags.FLAGS 25 | 26 | tf.app.flags.DEFINE_integer('nbatch_kg',100,'entity numbers used each training time') 27 | tf.app.flags.DEFINE_float('margin',1.0,'entity numbers used each training time') 28 | tf.app.flags.DEFINE_float('learning_rate_kg',0.001,'learning rate for kg') 29 | tf.app.flags.DEFINE_integer('ent_total',lib.getEntityTotal(),'total of entities') 30 | tf.app.flags.DEFINE_integer('rel_total',lib.getRelationTotal(),'total of relations') 31 | tf.app.flags.DEFINE_integer('tri_total',lib.getTripleTotal(),'total of triples') 32 | tf.app.flags.DEFINE_integer('katt_flag', 1, '1 for katt, 0 for att') 33 | 34 | tf.app.flags.DEFINE_string('model', 'cnn', 'neural models to encode sentences') 35 | tf.app.flags.DEFINE_integer('max_length',config['fixlen'],'maximum of number of words in one sentence') 36 | tf.app.flags.DEFINE_integer('pos_num', config['maxlen'] * 2 + 1,'number of position embedding vectors') 37 | tf.app.flags.DEFINE_integer('num_classes', config['textual_rel_total'],'maximum of relations') 38 | 39 | tf.app.flags.DEFINE_integer('hidden_size',230,'hidden feature size') 40 | tf.app.flags.DEFINE_integer('pos_size',5,'position embedding size') 41 | 42 | tf.app.flags.DEFINE_integer('max_epoch',30,'maximum of training epochs') 43 | tf.app.flags.DEFINE_integer('batch_size',131*2,'entity numbers used each training time') 44 | tf.app.flags.DEFINE_float('learning_rate',0.1,'entity numbers used each training time') 45 | tf.app.flags.DEFINE_float('weight_decay',0.00001,'weight_decay') 46 | tf.app.flags.DEFINE_float('keep_prob',1.0,'dropout rate') 47 | 48 | tf.app.flags.DEFINE_integer('test_batch_size',131*2,'entity numbers used each test time') 49 | tf.app.flags.DEFINE_string('checkpoint_path','./model/','path to store model') 50 | 51 | 52 | def make_shape(array,last_dim): 53 | output = [] 54 | for i in array: 55 | for j in i: 56 | output.append(j) 57 | output = np.array(output) 58 | if np.shape(output)[-1]==last_dim: 59 | return output 60 | 61 | else: 62 | print 'Make Shape Error!' 63 | 64 | def main(_): 65 | 66 | print 'reading word embedding' 67 | word_vec = np.load(export_path + 'vec.npy') 68 | print 'reading test data' 69 | test_instance_triple = np.load(export_path + 'test_instance_triple.npy') 70 | test_instance_scope = np.load(export_path + 'test_instance_scope.npy') 71 | test_len = np.load(export_path + 'test_len.npy') 72 | test_label = np.load(export_path + 'test_label.npy') 73 | test_word = np.load(export_path + 'test_word.npy') 74 | test_pos1 = np.load(export_path + 'test_pos1.npy') 75 | test_pos2 = np.load(export_path + 'test_pos2.npy') 76 | test_mask = np.load(export_path + 'test_mask.npy') 77 | test_head = np.load(export_path + 'test_head.npy') 78 | test_tail = np.load(export_path + 'test_tail.npy') 79 | print 'reading finished' 80 | print 'mentions : %d' % (len(test_instance_triple)) 81 | print 'sentences : %d' % (len(test_len)) 82 | print 'relations : %d' % (FLAGS.num_classes) 83 | print 'word size : %d' % (len(word_vec[0])) 84 | print 'position size : %d' % (FLAGS.pos_size) 85 | print 'hidden size : %d' % (FLAGS.hidden_size) 86 | print 'reading finished' 87 | 88 | print 'building network...' 89 | sess = tf.Session() 90 | if FLAGS.model.lower() == "cnn": 91 | model = network.CNN(is_training = False, word_embeddings = word_vec) 92 | elif FLAGS.model.lower() == "pcnn": 93 | model = network.PCNN(is_training = False, word_embeddings = word_vec) 94 | elif FLAGS.model.lower() == "lstm": 95 | model = network.RNN(is_training = False, word_embeddings = word_vec, cell_name = "LSTM", simple_position = True) 96 | elif FLAGS.model.lower() == "gru": 97 | model = network.RNN(is_training = False, word_embeddings = word_vec, cell_name = "GRU", simple_position = True) 98 | elif FLAGS.model.lower() == "bi-lstm" or FLAGS.model.lower() == "bilstm": 99 | model = network.BiRNN(is_training = False, word_embeddings = word_vec, cell_name = "LSTM", simple_position = True) 100 | elif FLAGS.model.lower() == "bi-gru" or FLAGS.model.lower() == "bigru": 101 | model = network.BiRNN(is_training = False, word_embeddings = word_vec, cell_name = "GRU", simple_position = True) 102 | sess.run(tf.global_variables_initializer()) 103 | saver = tf.train.Saver() 104 | 105 | def test_step(head, tail, word, pos1, pos2, mask, leng, label_index, label, scope): 106 | feed_dict = { 107 | model.head_index: head, 108 | model.tail_index: tail, 109 | model.word: word, 110 | model.pos1: pos1, 111 | model.pos2: pos2, 112 | model.mask: mask, 113 | model.len : leng, 114 | model.label_index: label_index, 115 | model.label: label, 116 | model.scope: scope, 117 | model.keep_prob: FLAGS.keep_prob 118 | } 119 | output = sess.run(model.test_output, feed_dict) 120 | return output 121 | 122 | f = open('results.txt','w') 123 | f.write('iteration\taverage precision\n') 124 | for iters in range(1,30): 125 | print iters 126 | saver.restore(sess, FLAGS.checkpoint_path + FLAGS.model+str(FLAGS.katt_flag)+"-"+str(3664*iters)) 127 | 128 | stack_output = [] 129 | stack_label = [] 130 | 131 | iteration = len(test_instance_scope)/FLAGS.test_batch_size 132 | for i in range(iteration): 133 | temp_str= 'running '+str(i)+'/'+str(iteration)+'...' 134 | sys.stdout.write(temp_str+'\r') 135 | sys.stdout.flush() 136 | input_scope = test_instance_scope[i * FLAGS.test_batch_size:(i+1)*FLAGS.test_batch_size] 137 | index = [] 138 | scope = [0] 139 | label = [] 140 | for num in input_scope: 141 | index = index + range(num[0], num[1] + 1) 142 | label.append(test_label[num[0]]) 143 | scope.append(scope[len(scope)-1] + num[1] - num[0] + 1) 144 | label_ = np.zeros((FLAGS.test_batch_size, FLAGS.num_classes)) 145 | label_[np.arange(FLAGS.test_batch_size), label] = 1 146 | output = test_step(test_head[index], test_tail[index], test_word[index,:], test_pos1[index,:], test_pos2[index,:], test_mask[index,:], test_len[index], test_label[index], label_, np.array(scope)) 147 | stack_output.append(output) 148 | stack_label.append(label_) 149 | 150 | print 'evaluating...' 151 | 152 | stack_output = np.concatenate(stack_output, axis=0) 153 | stack_label = np.concatenate(stack_label, axis = 0) 154 | 155 | exclude_na_flatten_output = stack_output[:,1:] 156 | exclude_na_flatten_label = stack_label[:,1:] 157 | print exclude_na_flatten_output.shape 158 | print exclude_na_flatten_label.shape 159 | 160 | average_precision = average_precision_score(exclude_na_flatten_label,exclude_na_flatten_output, average = "micro") 161 | 162 | np.save('./'+FLAGS.model+'+sen_att_all_prob_'+str(iters)+'.npy', exclude_na_flatten_output) 163 | np.save('./'+FLAGS.model+'+sen_att_all_label_'+str(iters)+'.npy',exclude_na_flatten_label) 164 | 165 | print 'pr: '+str(average_precision) 166 | f.write(str(average_precision)+'\n') 167 | f.close() 168 | 169 | if __name__ == "__main__": 170 | tf.app.run() 171 | -------------------------------------------------------------------------------- /jointE/KATT/train.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | import numpy as np 3 | import time 4 | import datetime 5 | import os 6 | import network 7 | import json 8 | from sklearn.metrics import average_precision_score 9 | import sys 10 | import ctypes 11 | import threading 12 | 13 | export_path = "../data/" 14 | 15 | word_vec = np.load(export_path + 'vec.npy') 16 | f = open(export_path + "config", 'r') 17 | config = json.loads(f.read()) 18 | f.close() 19 | 20 | ll = ctypes.cdll.LoadLibrary 21 | lib = ll("./init.so") 22 | lib.setInPath("../data/") 23 | lib.init() 24 | 25 | FLAGS = tf.app.flags.FLAGS 26 | 27 | tf.app.flags.DEFINE_float('nbatch_kg',100,'entity numbers used each training time') 28 | tf.app.flags.DEFINE_float('margin',1.0,'entity numbers used each training time') 29 | tf.app.flags.DEFINE_float('learning_rate_kg',0.001,'learning rate for kg') 30 | tf.app.flags.DEFINE_integer('ent_total',lib.getEntityTotal(),'total of entities') 31 | tf.app.flags.DEFINE_integer('rel_total',lib.getRelationTotal(),'total of relations') 32 | tf.app.flags.DEFINE_integer('tri_total',lib.getTripleTotal(),'total of triples') 33 | tf.app.flags.DEFINE_integer('katt_flag', 1, '1 for katt, 0 for att') 34 | 35 | tf.app.flags.DEFINE_string('model', 'cnn', 'neural models to encode sentences') 36 | tf.app.flags.DEFINE_integer('max_length',config['fixlen'],'maximum of number of words in one sentence') 37 | tf.app.flags.DEFINE_integer('pos_num', config['maxlen'] * 2 + 1,'number of position embedding vectors') 38 | tf.app.flags.DEFINE_integer('num_classes', config['textual_rel_total'],'maximum of relations') 39 | 40 | tf.app.flags.DEFINE_integer('hidden_size',230,'hidden feature size') 41 | tf.app.flags.DEFINE_integer('pos_size',5,'position embedding size') 42 | 43 | tf.app.flags.DEFINE_integer('max_epoch',20,'maximum of training epochs') 44 | tf.app.flags.DEFINE_integer('batch_size',160,'entity numbers used each training time') 45 | tf.app.flags.DEFINE_float('learning_rate',0.5,'learning rate for nn') 46 | tf.app.flags.DEFINE_float('weight_decay',0.00001,'weight_decay') 47 | tf.app.flags.DEFINE_float('keep_prob',0.5,'dropout rate') 48 | 49 | tf.app.flags.DEFINE_string('model_dir','./model/','path to store model') 50 | tf.app.flags.DEFINE_string('summary_dir','./summary','path to store summary_dir') 51 | 52 | 53 | def MakeSummary(name, value): 54 | """Creates a tf.Summary proto with the given name and value.""" 55 | summary = tf.Summary() 56 | val = summary.value.add() 57 | val.tag = str(name) 58 | val.simple_value = float(value) 59 | return summary 60 | 61 | def make_shape(array,last_dim): 62 | output = [] 63 | for i in array: 64 | for j in i: 65 | output.append(j) 66 | output = np.array(output) 67 | if np.shape(output)[-1]==last_dim: 68 | return output 69 | else: 70 | print 'Make Shape Error!' 71 | 72 | def main(_): 73 | 74 | print 'reading word embedding' 75 | word_vec = np.load(export_path + 'vec.npy') 76 | print 'reading training data' 77 | 78 | instance_triple = np.load(export_path + 'train_instance_triple.npy') 79 | instance_scope = np.load(export_path + 'train_instance_scope.npy') 80 | train_len = np.load(export_path + 'train_len.npy') 81 | train_label = np.load(export_path + 'train_label.npy') 82 | train_word = np.load(export_path + 'train_word.npy') 83 | train_pos1 = np.load(export_path + 'train_pos1.npy') 84 | train_pos2 = np.load(export_path + 'train_pos2.npy') 85 | train_mask = np.load(export_path + 'train_mask.npy') 86 | train_head = np.load(export_path + 'train_head.npy') 87 | train_tail = np.load(export_path + 'train_tail.npy') 88 | 89 | print 'reading finished' 90 | print 'mentions : %d' % (len(instance_triple)) 91 | print 'sentences : %d' % (len(train_len)) 92 | print 'relations : %d' % (FLAGS.num_classes) 93 | print 'word size : %d' % (len(word_vec[0])) 94 | print 'position size : %d' % (FLAGS.pos_size) 95 | print 'hidden size : %d' % (FLAGS.hidden_size) 96 | reltot = {} 97 | for index, i in enumerate(train_label): 98 | if not i in reltot: 99 | reltot[i] = 1.0 100 | else: 101 | reltot[i] += 1.0 102 | for i in reltot: 103 | reltot[i] = 1/(reltot[i] ** (0.05)) 104 | print 'building network...' 105 | sess = tf.Session() 106 | if FLAGS.model.lower() == "cnn": 107 | model = network.CNN(is_training = True, word_embeddings = word_vec) 108 | elif FLAGS.model.lower() == "pcnn": 109 | model = network.PCNN(is_training = True, word_embeddings = word_vec) 110 | elif FLAGS.model.lower() == "lstm": 111 | model = network.RNN(is_training = True, word_embeddings = word_vec, cell_name = "LSTM", simple_position = True) 112 | elif FLAGS.model.lower() == "gru": 113 | model = network.RNN(is_training = True, word_embeddings = word_vec, cell_name = "GRU", simple_position = True) 114 | elif FLAGS.model.lower() == "bi-lstm" or FLAGS.model.lower() == "bilstm": 115 | model = network.BiRNN(is_training = True, word_embeddings = word_vec, cell_name = "LSTM", simple_position = True) 116 | elif FLAGS.model.lower() == "bi-gru" or FLAGS.model.lower() == "bigru": 117 | model = network.BiRNN(is_training = True, word_embeddings = word_vec, cell_name = "GRU", simple_position = True) 118 | 119 | global_step = tf.Variable(0,name='global_step',trainable=False) 120 | global_step_kg = tf.Variable(0,name='global_step_kg',trainable=False) 121 | tf.summary.scalar('learning_rate', FLAGS.learning_rate) 122 | tf.summary.scalar('learning_rate_kg', FLAGS.learning_rate_kg) 123 | 124 | optimizer = tf.train.GradientDescentOptimizer(FLAGS.learning_rate) 125 | grads_and_vars = optimizer.compute_gradients(model.loss) 126 | train_op = optimizer.apply_gradients(grads_and_vars, global_step = global_step) 127 | 128 | optimizer_kg = tf.train.GradientDescentOptimizer(FLAGS.learning_rate_kg) 129 | grads_and_vars_kg = optimizer_kg.compute_gradients(model.loss_kg) 130 | train_op_kg = optimizer_kg.apply_gradients(grads_and_vars_kg, global_step = global_step_kg) 131 | 132 | merged_summary = tf.summary.merge_all() 133 | summary_writer = tf.summary.FileWriter(FLAGS.summary_dir, sess.graph) 134 | sess.run(tf.global_variables_initializer()) 135 | saver = tf.train.Saver(max_to_keep=None) 136 | print 'building finished' 137 | 138 | def train_kg(coord): 139 | def train_step_kg(pos_h_batch, pos_t_batch, pos_r_batch, neg_h_batch, neg_t_batch, neg_r_batch): 140 | feed_dict = { 141 | model.pos_h: pos_h_batch, 142 | model.pos_t: pos_t_batch, 143 | model.pos_r: pos_r_batch, 144 | model.neg_h: neg_h_batch, 145 | model.neg_t: neg_t_batch, 146 | model.neg_r: neg_r_batch 147 | } 148 | _, step, loss = sess.run( 149 | [train_op_kg, global_step_kg, model.loss_kg], feed_dict) 150 | return loss 151 | 152 | batch_size = (FLAGS.ent_total / FLAGS.nbatch_kg) 153 | ph = np.zeros(batch_size, dtype = np.int32) 154 | pt = np.zeros(batch_size, dtype = np.int32) 155 | pr = np.zeros(batch_size, dtype = np.int32) 156 | nh = np.zeros(batch_size, dtype = np.int32) 157 | nt = np.zeros(batch_size, dtype = np.int32) 158 | nr = np.zeros(batch_size, dtype = np.int32) 159 | ph_addr = ph.__array_interface__['data'][0] 160 | pt_addr = pt.__array_interface__['data'][0] 161 | pr_addr = pr.__array_interface__['data'][0] 162 | nh_addr = nh.__array_interface__['data'][0] 163 | nt_addr = nt.__array_interface__['data'][0] 164 | nr_addr = nr.__array_interface__['data'][0] 165 | lib.getBatch.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_int] 166 | times_kg = 0 167 | while not coord.should_stop(): 168 | times_kg += 1 169 | res = 0.0 170 | for batch in range(FLAGS.nbatch_kg): 171 | lib.getBatch(ph_addr, pt_addr, pr_addr, nh_addr, nt_addr, nr_addr, batch_size) 172 | res += train_step_kg(ph, pt, pr, nh, nt, nr) 173 | time_str = datetime.datetime.now().isoformat() 174 | print "batch %d time %s | loss : %f" % (times_kg, time_str, res) 175 | 176 | 177 | def train_nn(coord): 178 | def train_step(head, tail, word, pos1, pos2, mask, leng, label_index, label, scope, weights): 179 | feed_dict = { 180 | model.head_index: head, 181 | model.tail_index: tail, 182 | model.word: word, 183 | model.pos1: pos1, 184 | model.pos2: pos2, 185 | model.mask: mask, 186 | model.len : leng, 187 | model.label_index: label_index, 188 | model.label: label, 189 | model.scope: scope, 190 | model.keep_prob: FLAGS.keep_prob, 191 | model.weights: weights 192 | } 193 | _, step, loss, summary, output, correct_predictions = sess.run([train_op, global_step, model.loss, merged_summary, model.output, model.correct_predictions], feed_dict) 194 | summary_writer.add_summary(summary, step) 195 | return output, loss, correct_predictions 196 | 197 | stack_output = [] 198 | stack_label = [] 199 | stack_ce_loss = [] 200 | 201 | train_order = range(len(instance_triple)) 202 | 203 | save_epoch = 2 204 | eval_step = 300 205 | 206 | for one_epoch in range(FLAGS.max_epoch): 207 | 208 | print('epoch '+str(one_epoch+1)+' starts!') 209 | np.random.shuffle(train_order) 210 | s1 = 0.0 211 | s2 = 0.0 212 | tot1 = 0.0 213 | tot2 = 0.0 214 | losstot = 0.0 215 | for i in range(int(len(train_order)/float(FLAGS.batch_size))): 216 | input_scope = np.take(instance_scope, train_order[i * FLAGS.batch_size:(i+1)*FLAGS.batch_size], axis=0) 217 | index = [] 218 | scope = [0] 219 | label = [] 220 | weights = [] 221 | for num in input_scope: 222 | index = index + range(num[0], num[1] + 1) 223 | label.append(train_label[num[0]]) 224 | if train_label[num[0]] > 53: 225 | print train_label[num[0]] 226 | scope.append(scope[len(scope)-1] + num[1] - num[0] + 1) 227 | weights.append(reltot[train_label[num[0]]]) 228 | label_ = np.zeros((FLAGS.batch_size, FLAGS.num_classes)) 229 | label_[np.arange(FLAGS.batch_size), label] = 1 230 | output, loss, correct_predictions = train_step(train_head[index], train_tail[index], train_word[index,:], train_pos1[index,:], train_pos2[index,:], train_mask[index,:], train_len[index],train_label[index], label_, np.array(scope), weights) 231 | num = 0 232 | s = 0 233 | losstot += loss 234 | for num in correct_predictions: 235 | if label[s] == 0: 236 | tot1 += 1.0 237 | if num: 238 | s1+= 1.0 239 | else: 240 | tot2 += 1.0 241 | if num: 242 | s2 += 1.0 243 | s = s + 1 244 | 245 | time_str = datetime.datetime.now().isoformat() 246 | print "batch %d step %d time %s | loss : %f, NA accuracy: %f, not NA accuracy: %f" % (one_epoch, i, time_str, loss, s1 / tot1, s2 / tot2) 247 | current_step = tf.train.global_step(sess, global_step) 248 | 249 | if (one_epoch + 1) % save_epoch == 0: 250 | print 'epoch '+str(one_epoch+1)+' has finished' 251 | print 'saving model...' 252 | path = saver.save(sess,FLAGS.model_dir+FLAGS.model+str(FLAGS.katt_flag), global_step=current_step) 253 | print 'have savde model to '+path 254 | 255 | coord.request_stop() 256 | 257 | 258 | coord = tf.train.Coordinator() 259 | threads = [] 260 | threads.append(threading.Thread(target=train_kg, args=(coord,))) 261 | threads.append(threading.Thread(target=train_nn, args=(coord,))) 262 | for t in threads: t.start() 263 | coord.join(threads) 264 | 265 | if __name__ == "__main__": 266 | tf.app.run() 267 | -------------------------------------------------------------------------------- /jointE/SATT/init.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | 9 | using namespace std; 10 | 11 | string inPath = "./data/"; 12 | 13 | extern "C" 14 | void setInPath(char *path) { 15 | int len = strlen(path); 16 | inPath = ""; 17 | for (int i = 0; i < len; i++) 18 | inPath = inPath + path[i]; 19 | printf("Input Files Path : %s\n", inPath.c_str()); 20 | } 21 | 22 | int *lefHead, *rigHead; 23 | int *lefTail, *rigTail; 24 | 25 | struct Triple { 26 | int h, r, t; 27 | }; 28 | 29 | struct cmp_head { 30 | bool operator()(const Triple &a, const Triple &b) { 31 | return (a.h < b.h)||(a.h == b.h && a.r < b.r)||(a.h == b.h && a.r == b.r && a.t < b.t); 32 | } 33 | }; 34 | 35 | struct cmp_tail { 36 | bool operator()(const Triple &a, const Triple &b) { 37 | return (a.t < b.t)||(a.t == b.t && a.r < b.r)||(a.t == b.t && a.r == b.r && a.h < b.h); 38 | } 39 | }; 40 | 41 | struct cmp_list { 42 | int minimal(int a,int b) { 43 | if (a > b) return b; 44 | return a; 45 | } 46 | bool operator()(const Triple &a, const Triple &b) { 47 | return (minimal(a.h, a.t) > minimal(b.h, b.t)); 48 | } 49 | }; 50 | 51 | Triple *trainHead, *trainTail, *trainList; 52 | int relationTotal, entityTotal, tripleTotal; 53 | int *freqRel, *freqEnt; 54 | float *left_mean, *right_mean; 55 | 56 | extern "C" 57 | void init() { 58 | 59 | FILE *fin; 60 | int tmp; 61 | 62 | fin = fopen((inPath + "relation2id.txt").c_str(), "r"); 63 | tmp = fscanf(fin, "%d", &relationTotal); 64 | fclose(fin); 65 | printf("%d\n", relationTotal); 66 | 67 | freqRel = (int *)calloc(relationTotal, sizeof(int)); 68 | 69 | fin = fopen((inPath + "entity2id.txt").c_str(), "r"); 70 | tmp = fscanf(fin, "%d", &entityTotal); 71 | fclose(fin); 72 | printf("%d\n", entityTotal); 73 | 74 | freqEnt = (int *)calloc(entityTotal, sizeof(int)); 75 | 76 | fin = fopen((inPath + "triple2id.txt").c_str(), "r"); 77 | tmp = fscanf(fin, "%d", &tripleTotal); 78 | printf("%d\n", tripleTotal); 79 | trainHead = (Triple *)calloc(tripleTotal, sizeof(Triple)); 80 | trainTail = (Triple *)calloc(tripleTotal, sizeof(Triple)); 81 | trainList = (Triple *)calloc(tripleTotal, sizeof(Triple)); 82 | tripleTotal = 0; 83 | while (fscanf(fin, "%d", &trainList[tripleTotal].h) == 1) { 84 | tmp = fscanf(fin, "%d", &trainList[tripleTotal].t); 85 | tmp = fscanf(fin, "%d", &trainList[tripleTotal].r); 86 | freqEnt[trainList[tripleTotal].t]++; 87 | freqEnt[trainList[tripleTotal].h]++; 88 | freqRel[trainList[tripleTotal].r]++; 89 | trainHead[tripleTotal].h = trainList[tripleTotal].h; 90 | trainHead[tripleTotal].t = trainList[tripleTotal].t; 91 | trainHead[tripleTotal].r = trainList[tripleTotal].r; 92 | trainTail[tripleTotal].h = trainList[tripleTotal].h; 93 | trainTail[tripleTotal].t = trainList[tripleTotal].t; 94 | trainTail[tripleTotal].r = trainList[tripleTotal].r; 95 | tripleTotal++; 96 | } 97 | fclose(fin); 98 | 99 | sort(trainHead, trainHead + tripleTotal, cmp_head()); 100 | sort(trainTail, trainTail + tripleTotal, cmp_tail()); 101 | 102 | lefHead = (int *)calloc(entityTotal, sizeof(int)); 103 | rigHead = (int *)calloc(entityTotal, sizeof(int)); 104 | lefTail = (int *)calloc(entityTotal, sizeof(int)); 105 | rigTail = (int *)calloc(entityTotal, sizeof(int)); 106 | memset(rigHead, -1, sizeof(rigHead)); 107 | memset(rigTail, -1, sizeof(rigTail)); 108 | for (int i = 1; i < tripleTotal; i++) { 109 | if (trainTail[i].t != trainTail[i - 1].t) { 110 | rigTail[trainTail[i - 1].t] = i - 1; 111 | lefTail[trainTail[i].t] = i; 112 | } 113 | if (trainHead[i].h != trainHead[i - 1].h) { 114 | rigHead[trainHead[i - 1].h] = i - 1; 115 | lefHead[trainHead[i].h] = i; 116 | } 117 | } 118 | rigHead[trainHead[tripleTotal - 1].h] = tripleTotal - 1; 119 | rigTail[trainTail[tripleTotal - 1].t] = tripleTotal - 1; 120 | 121 | left_mean = (float *)calloc(relationTotal,sizeof(float)); 122 | right_mean = (float *)calloc(relationTotal,sizeof(float)); 123 | for (int i = 0; i < entityTotal; i++) { 124 | for (int j = lefHead[i] + 1; j <= rigHead[i]; j++) 125 | if (trainHead[j].r != trainHead[j - 1].r) 126 | left_mean[trainHead[j].r] += 1.0; 127 | if (lefHead[i] <= rigHead[i]) 128 | left_mean[trainHead[lefHead[i]].r] += 1.0; 129 | for (int j = lefTail[i] + 1; j <= rigTail[i]; j++) 130 | if (trainTail[j].r != trainTail[j - 1].r) 131 | right_mean[trainTail[j].r] += 1.0; 132 | if (lefTail[i] <= rigTail[i]) 133 | right_mean[trainTail[lefTail[i]].r] += 1.0; 134 | } 135 | for (int i = 0; i < relationTotal; i++) { 136 | left_mean[i] = freqRel[i] / left_mean[i]; 137 | right_mean[i] = freqRel[i] / right_mean[i]; 138 | } 139 | } 140 | 141 | extern "C" 142 | int getEntityTotal() { 143 | return entityTotal; 144 | } 145 | 146 | extern "C" 147 | int getRelationTotal() { 148 | return relationTotal; 149 | } 150 | 151 | extern "C" 152 | int getTripleTotal() { 153 | return tripleTotal; 154 | } 155 | 156 | // unsigned long long *next_random; 157 | unsigned long long next_random = 3; 158 | 159 | unsigned long long randd(int id) { 160 | next_random = next_random * (unsigned long long)25214903917 + 11; 161 | return next_random; 162 | } 163 | 164 | int rand_max(int id, int x) { 165 | int res = randd(id) % x; 166 | while (res<0) 167 | res+=x; 168 | return res; 169 | } 170 | 171 | int corrupt_head(int id, int h, int r) { 172 | int lef, rig, mid, ll, rr; 173 | lef = lefHead[h] - 1; 174 | rig = rigHead[h]; 175 | while (lef + 1 < rig) { 176 | mid = (lef + rig) >> 1; 177 | if (trainHead[mid].r >= r) rig = mid; else 178 | lef = mid; 179 | } 180 | ll = rig; 181 | lef = lefHead[h]; 182 | rig = rigHead[h] + 1; 183 | while (lef + 1 < rig) { 184 | mid = (lef + rig) >> 1; 185 | if (trainHead[mid].r <= r) lef = mid; else 186 | rig = mid; 187 | } 188 | rr = lef; 189 | int tmp = rand_max(id, entityTotal - (rr - ll + 1)); 190 | if (tmp < trainHead[ll].t) return tmp; 191 | if (tmp > trainHead[rr].t - rr + ll - 1) return tmp + rr - ll + 1; 192 | lef = ll, rig = rr + 1; 193 | while (lef + 1 < rig) { 194 | mid = (lef + rig) >> 1; 195 | if (trainHead[mid].t - mid + ll - 1 < tmp) 196 | lef = mid; 197 | else 198 | rig = mid; 199 | } 200 | return tmp + lef - ll + 1; 201 | } 202 | 203 | int corrupt_tail(int id, int t, int r) { 204 | int lef, rig, mid, ll, rr; 205 | lef = lefTail[t] - 1; 206 | rig = rigTail[t]; 207 | while (lef + 1 < rig) { 208 | mid = (lef + rig) >> 1; 209 | if (trainTail[mid].r >= r) rig = mid; else 210 | lef = mid; 211 | } 212 | ll = rig; 213 | lef = lefTail[t]; 214 | rig = rigTail[t] + 1; 215 | while (lef + 1 < rig) { 216 | mid = (lef + rig) >> 1; 217 | if (trainTail[mid].r <= r) lef = mid; else 218 | rig = mid; 219 | } 220 | rr = lef; 221 | int tmp = rand_max(id, entityTotal - (rr - ll + 1)); 222 | if (tmp < trainTail[ll].h) return tmp; 223 | if (tmp > trainTail[rr].h - rr + ll - 1) return tmp + rr - ll + 1; 224 | lef = ll, rig = rr + 1; 225 | while (lef + 1 < rig) { 226 | mid = (lef + rig) >> 1; 227 | if (trainTail[mid].h - mid + ll - 1 < tmp) 228 | lef = mid; 229 | else 230 | rig = mid; 231 | } 232 | return tmp + lef - ll + 1; 233 | } 234 | 235 | extern "C" 236 | void getBatch(int *ph, int *pt, int *pr, int *nh, int *nt, int *nr, int batchSize, int id = 0) { 237 | for (int batch = 0; batch < batchSize; batch++) { 238 | int i = rand_max(id, tripleTotal), j; 239 | // float prob = 1000 * right_mean[trainList[i].r] / (right_mean[trainList[i].r] + left_mean[trainList[i].r]); 240 | float prob = 500; 241 | if (randd(id) % 1000 < prob) { 242 | j = corrupt_head(id, trainList[i].h, trainList[i].r); 243 | ph[batch] = trainList[i].h; 244 | pt[batch] = trainList[i].t; 245 | pr[batch] = trainList[i].r; 246 | nh[batch] = trainList[i].h; 247 | nt[batch] = j; 248 | nr[batch] = trainList[i].r; 249 | } else { 250 | j = corrupt_tail(id, trainList[i].t, trainList[i].r); 251 | ph[batch] = trainList[i].h; 252 | pt[batch] = trainList[i].t; 253 | pr[batch] = trainList[i].r; 254 | nh[batch] = j; 255 | nt[batch] = trainList[i].t; 256 | nr[batch] = trainList[i].r; 257 | } 258 | } 259 | } -------------------------------------------------------------------------------- /jointE/SATT/make.sh: -------------------------------------------------------------------------------- 1 | g++ init.cpp -o init.so -fPIC -shared -pthread -O3 -march=native 2 | -------------------------------------------------------------------------------- /jointE/SATT/network.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thunlp/JointNRE/85c6009828b64439ef31b49e230fbd6ec24a6f8a/jointE/SATT/network.pyc -------------------------------------------------------------------------------- /jointE/SATT/pr_plot.py: -------------------------------------------------------------------------------- 1 | import os 2 | import numpy as np 3 | from sklearn.metrics import precision_recall_curve 4 | from sklearn.metrics import average_precision_score 5 | import matplotlib 6 | matplotlib.use('Agg') 7 | import matplotlib.pyplot as plt 8 | import numpy as np 9 | import sys 10 | 11 | from matplotlib.backends.backend_pdf import PdfPages 12 | 13 | 14 | ff = plt.figure() 15 | 16 | MODEL = 'cnn' 17 | 18 | def guolv(recall, precision): 19 | a = [recall[0]] 20 | b = [precision[0]] 21 | print len(recall) 22 | for i in range(1, len(recall)): 23 | if a[len(a) - 1] == recall[i]: 24 | if precision[i] > b[len(b)-1]: 25 | b[len(b)-1] = precision[i] 26 | else: 27 | a.append(recall[i]) 28 | b.append(precision[i]) 29 | 30 | recall = np.array(a) 31 | precision = np.array(b) 32 | xnew = np.linspace(recall.min(),recall.max(), 500) #300 represents number of points to make between T.min and T.max 33 | print recall 34 | print precision 35 | power_smooth = spline(recall,precision,xnew) 36 | return xnew, power_smooth 37 | 38 | def PrecisionAtRecall(pAll, rAll, rMark): 39 | length = len(rAll) 40 | lo = 0 41 | hi = length - 1 42 | mark = length >> 1 43 | error = rMark - rAll[mark] 44 | while np.abs(error) > 0.005: 45 | if error > 0: 46 | hi = mark - 1 47 | else: 48 | lo = mark + 1 49 | mark = (hi + lo) >> 1 50 | error = rMark - rAll[mark] 51 | return pAll[mark], rAll[mark], mark 52 | 53 | 54 | color = ['red', 'turquoise', 'darkorange', 'cornflowerblue', 'teal'] 55 | 56 | test_model = ['cnn'+'+sen_att'] 57 | test_epoch = ['9'] 58 | avg_pres = [] 59 | for temp, (model, step) in enumerate(zip(test_model, test_epoch)): 60 | y_scores = np.load(model+'_all_prob' + '_' + step + '.npy') 61 | y_true = np.load(model+'_all_label' + '_' + step + '.npy') 62 | y_scores = np.reshape(y_scores,(-1)) 63 | y_true = np.reshape(y_true,(-1)) 64 | precision,recall,threshold = precision_recall_curve(y_true,y_scores) 65 | p,r,i = PrecisionAtRecall(precision, recall, 0.1) 66 | print('precison: {}, recall: {}'.format(p, r)) 67 | p,r,i = PrecisionAtRecall(precision, recall, 0.2) 68 | print('precison: {}, recall: {}'.format(p, r)) 69 | p,r,i = PrecisionAtRecall(precision, recall, 0.3) 70 | print('precison: {}, recall: {}'.format(p, r)) 71 | average_precision = average_precision_score(y_true, y_scores) 72 | avg_pres.append(average_precision) 73 | recall = recall[::-1] 74 | precision = precision[::-1] 75 | plt.plot(recall[:], precision[:], lw=2, color=color[1],label="CNN+ATT") 76 | 77 | lines_cnn = open('cnn.txt').readlines() 78 | lines_cnn = [t.strip().split()[:2] for t in lines_cnn] 79 | precision_cnn = np.array([t[0] for t in lines_cnn], dtype=np.float32) 80 | recall_cnn = np.array([t[1] for t in lines_cnn], dtype=np.float32) 81 | plt.plot(recall_cnn, precision_cnn, lw=2, color=color[-1], label="CNN+ATT") 82 | 83 | 84 | plt.xlabel('Recall') 85 | plt.ylabel('Precision') 86 | plt.ylim([0.3, 1.0]) 87 | plt.xlim([0.0, 0.4]) 88 | plt.title('Precision-Recall Area={0:0.4f}'.format(avg_pres[-1])) 89 | plt.legend(loc="upper right") 90 | plt.grid(True) 91 | plt.savefig('sgd_'+MODEL) 92 | plt.plot(range(10), range(10), "o") 93 | plt.show() 94 | ff.savefig("pr.pdf", bbox_inches='tight') 95 | -------------------------------------------------------------------------------- /jointE/SATT/res.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thunlp/JointNRE/85c6009828b64439ef31b49e230fbd6ec24a6f8a/jointE/SATT/res.png -------------------------------------------------------------------------------- /jointE/SATT/train.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | import numpy as np 3 | import time 4 | import datetime 5 | import os 6 | import network 7 | import json 8 | import sys 9 | import ctypes 10 | import threading 11 | 12 | export_path = "../data_15k/" 13 | 14 | word_vec = np.load(export_path + 'vec.npy') 15 | f = open(export_path + "config", 'r') 16 | config = json.loads(f.read()) 17 | f.close() 18 | 19 | ll = ctypes.cdll.LoadLibrary 20 | lib = ll("./init.so") 21 | lib.setInPath("../data_15k/") 22 | lib.init() 23 | 24 | triple_per_rel = {} 25 | f = open(export_path + 'triple2id.txt', "r") 26 | content = f.readlines()[1:] 27 | for i in content: 28 | h,t,r = i.strip().split("\t") 29 | if not (int)(r) in triple_per_rel: 30 | triple_per_rel[(int)(r)] = [] 31 | triple_per_rel[(int)(r)].append([(int)(h),(int)(t)]) 32 | for rel in triple_per_rel: 33 | triple_per_rel[rel] = np.array(triple_per_rel[rel]) 34 | f.close() 35 | for i in range(config['textual_rel_total']): 36 | print (len(triple_per_rel[i])) 37 | 38 | 39 | 40 | FLAGS = tf.app.flags.FLAGS 41 | 42 | 43 | tf.app.flags.DEFINE_integer('nbatch_kg',100,'entity numbers used each training time') 44 | tf.app.flags.DEFINE_float('margin',1.0,'entity numbers used each training time') 45 | tf.app.flags.DEFINE_float('learning_rate_kg',0.001,'learning rate for kg') 46 | tf.app.flags.DEFINE_integer('ent_total',lib.getEntityTotal(),'total of entities') 47 | tf.app.flags.DEFINE_integer('rel_total',lib.getRelationTotal(),'total of relations') 48 | tf.app.flags.DEFINE_integer('tri_total',lib.getTripleTotal(),'total of triples') 49 | tf.app.flags.DEFINE_integer('katt_flag', 1, '1 for katt, 0 for att') 50 | 51 | tf.app.flags.DEFINE_string('model', 'cnn', 'neural models to encode sentences') 52 | tf.app.flags.DEFINE_integer('max_length',config['fixlen'],'maximum of number of words in one sentence') 53 | tf.app.flags.DEFINE_integer('pos_num', config['maxlen'] * 2 + 1,'number of position embedding vectors') 54 | print (config['textual_rel_total']) 55 | tf.app.flags.DEFINE_integer('num_classes', config['textual_rel_total'],'maximum of relations') 56 | 57 | tf.app.flags.DEFINE_integer('hidden_size',230,'hidden feature size') 58 | tf.app.flags.DEFINE_integer('embedding_size',100,'input feature size') 59 | tf.app.flags.DEFINE_integer('pos_size',5,'position embedding size') 60 | 61 | tf.app.flags.DEFINE_integer('max_epoch',100,'maximum of training epochs') 62 | tf.app.flags.DEFINE_integer('batch_size',16,'entity numbers used each training time') 63 | tf.app.flags.DEFINE_float('learning_rate',0.01,'learning rate for nn') 64 | tf.app.flags.DEFINE_float('weight_decay',0.00001,'weight_decay') 65 | tf.app.flags.DEFINE_float('keep_prob',0.5,'dropout rate') 66 | 67 | tf.app.flags.DEFINE_string('model_dir','./model/','path to store model') 68 | tf.app.flags.DEFINE_string('summary_dir','./summary','path to store summary_dir') 69 | tf.app.flags.DEFINE_integer('store_kg_flag', 1, 'store kg embeddings') 70 | 71 | 72 | def MakeSummary(name, value): 73 | """Creates a tf.Summary proto with the given name and value.""" 74 | summary = tf.Summary() 75 | val = summary.value.add() 76 | val.tag = str(name) 77 | val.simple_value = float(value) 78 | return summary 79 | 80 | def make_shape(array,last_dim): 81 | output = [] 82 | for i in array: 83 | for j in i: 84 | output.append(j) 85 | output = np.array(output) 86 | if np.shape(output)[-1]==last_dim: 87 | return output 88 | else: 89 | print 'Make Shape Error!' 90 | 91 | def main(_): 92 | 93 | print 'reading word embedding' 94 | word_vec = np.load(export_path + 'vec.npy') 95 | print 'reading training data' 96 | 97 | instance_triple = np.load(export_path + 'train_instance_triple.npy') 98 | instance_scope = np.load(export_path + 'train_instance_scope.npy') 99 | train_len = np.load(export_path + 'train_len.npy') 100 | train_label = np.load(export_path + 'train_label.npy') 101 | train_word = np.load(export_path + 'train_word.npy') 102 | train_pos1 = np.load(export_path + 'train_pos1.npy') 103 | train_pos2 = np.load(export_path + 'train_pos2.npy') 104 | train_mask = np.load(export_path + 'train_mask.npy') 105 | train_head = np.load(export_path + 'train_head.npy') 106 | train_tail = np.load(export_path + 'train_tail.npy') 107 | 108 | print 'reading finished' 109 | print 'mentions : %d' % (len(instance_triple)) 110 | print 'sentences : %d' % (len(train_len)) 111 | print 'relations : %d' % (FLAGS.num_classes) 112 | print 'word size : %d' % (len(word_vec[0])) 113 | print 'position size : %d' % (FLAGS.pos_size) 114 | print 'hidden size : %d' % (FLAGS.hidden_size) 115 | reltot = {} 116 | for index, i in enumerate(train_label): 117 | if not i in reltot: 118 | reltot[i] = 1.0 119 | else: 120 | reltot[i] += 1.0 121 | for i in reltot: 122 | reltot[i] = 1/(reltot[i] ** (0.05)) 123 | print 'building network...' 124 | sess = tf.Session() 125 | if FLAGS.model.lower() == "cnn": 126 | model = network.CNN(is_training = True, word_embeddings = word_vec) 127 | elif FLAGS.model.lower() == "pcnn": 128 | model = network.PCNN(is_training = True, word_embeddings = word_vec) 129 | elif FLAGS.model.lower() == "lstm": 130 | model = network.RNN(is_training = True, word_embeddings = word_vec, cell_name = "LSTM", simple_position = True) 131 | elif FLAGS.model.lower() == "gru": 132 | model = network.RNN(is_training = True, word_embeddings = word_vec, cell_name = "GRU", simple_position = True) 133 | elif FLAGS.model.lower() == "bi-lstm" or FLAGS.model.lower() == "bilstm": 134 | model = network.BiRNN(is_training = True, word_embeddings = word_vec, cell_name = "LSTM", simple_position = True) 135 | elif FLAGS.model.lower() == "bi-gru" or FLAGS.model.lower() == "bigru": 136 | model = network.BiRNN(is_training = True, word_embeddings = word_vec, cell_name = "GRU", simple_position = True) 137 | 138 | global_step = tf.Variable(0,name='global_step',trainable=False) 139 | global_step_kg = tf.Variable(0,name='global_step_kg',trainable=False) 140 | global_step_kg_satt = tf.Variable(0,name='global_step_kg_satt',trainable=False) 141 | tf.summary.scalar('learning_rate', FLAGS.learning_rate) 142 | tf.summary.scalar('learning_rate_kg', FLAGS.learning_rate_kg) 143 | 144 | optimizer = tf.train.GradientDescentOptimizer(FLAGS.learning_rate) 145 | grads_and_vars = optimizer.compute_gradients(model.loss) 146 | train_op = optimizer.apply_gradients(grads_and_vars, global_step = global_step) 147 | 148 | optimizer_kg = tf.train.GradientDescentOptimizer(FLAGS.learning_rate_kg) 149 | grads_and_vars_kg = optimizer_kg.compute_gradients(model.loss_kg) 150 | train_op_kg = optimizer_kg.apply_gradients(grads_and_vars_kg, global_step = global_step_kg) 151 | 152 | optimizer_kg_satt = tf.train.GradientDescentOptimizer(FLAGS.learning_rate_kg) 153 | grads_and_vars_kg_satt = optimizer_kg_satt.compute_gradients(model.loss_kg_att) 154 | train_op_kg_satt = optimizer_kg_satt.apply_gradients(grads_and_vars_kg_satt, global_step = global_step_kg_satt) 155 | 156 | merged_summary = tf.summary.merge_all() 157 | summary_writer = tf.summary.FileWriter(FLAGS.summary_dir, sess.graph) 158 | sess.run(tf.global_variables_initializer()) 159 | saver = tf.train.Saver(max_to_keep=None) 160 | 161 | print 'building finished' 162 | 163 | def train_kg_att(coord): 164 | 165 | def train_step_kg_att(h_batch, t_batch, r_batch, r_scope, r_label): 166 | feed_dict = { 167 | model.pos_h: h_batch, 168 | model.pos_t: t_batch, 169 | model.pos_r: r_batch, 170 | model.r_scope: r_scope, 171 | model.r_label: r_label, 172 | model.r_length: np.array([len(r_label)]), 173 | } 174 | _, loss = sess.run([train_op_kg_satt, model.loss_kg_att], feed_dict) 175 | return loss 176 | 177 | def merge(head, tail, rel): 178 | hash = {} 179 | for (h,t,r) in zip(head,tail,rel): 180 | if r < FLAGS.num_classes: 181 | if not r in hash: 182 | hash[r] = [] 183 | hash[r].append((h,t)) 184 | rel = [] 185 | head = [] 186 | tail = [] 187 | rel_label = [] 188 | rel_config = [0] 189 | for r in hash: 190 | if len(hash[r]) != 0: 191 | rel_config.append(rel_config[-1]) 192 | rel_label.append(r) 193 | for h,t in hash[r]: 194 | rel_config[-1]+=1 195 | head.append(h) 196 | tail.append(t) 197 | rel.append(r) 198 | return np.array(head), np.array(tail), np.array(rel), np.array(rel_config), np.array(rel_label) 199 | 200 | batch_size = (FLAGS.tri_total / FLAGS.nbatch_kg) 201 | ph = np.zeros(batch_size, dtype = np.int32) 202 | pt = np.zeros(batch_size, dtype = np.int32) 203 | pr = np.zeros(batch_size, dtype = np.int32) 204 | nh = np.zeros(batch_size, dtype = np.int32) 205 | nt = np.zeros(batch_size, dtype = np.int32) 206 | nr = np.zeros(batch_size, dtype = np.int32) 207 | ph_addr = ph.__array_interface__['data'][0] 208 | pt_addr = pt.__array_interface__['data'][0] 209 | pr_addr = pr.__array_interface__['data'][0] 210 | nh_addr = nh.__array_interface__['data'][0] 211 | nt_addr = nt.__array_interface__['data'][0] 212 | nr_addr = nr.__array_interface__['data'][0] 213 | lib.getBatch.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_int] 214 | times_kg = 0 215 | while not coord.should_stop(): 216 | times_kg += 1 217 | # if times_kg == 3000: 218 | # coord.request_stop() 219 | res = 0.0 220 | for batch in range(FLAGS.nbatch_kg): 221 | lib.getBatch(ph_addr, pt_addr, pr_addr, nh_addr, nt_addr, nr_addr, batch_size) 222 | h, t, r, r_range, r_label = merge(ph, pt, pr) 223 | res += train_step_kg_att(h, t, r, r_range, r_label) 224 | time_str = datetime.datetime.now().isoformat() 225 | print "batch %d time %s | loss : %f" % (times_kg, time_str, res) 226 | 227 | def train_kg(coord): 228 | 229 | def train_step_kg(pos_h_batch, pos_t_batch, pos_r_batch, neg_h_batch, neg_t_batch, neg_r_batch): 230 | feed_dict = { 231 | model.pos_h: pos_h_batch, 232 | model.pos_t: pos_t_batch, 233 | model.pos_r: pos_r_batch, 234 | model.neg_h: neg_h_batch, 235 | model.neg_t: neg_t_batch, 236 | model.neg_r: neg_r_batch 237 | } 238 | _, step, loss = sess.run( 239 | [train_op_kg, global_step_kg, model.loss_kg], feed_dict) 240 | return loss 241 | 242 | batch_size = (FLAGS.tri_total / FLAGS.nbatch_kg) 243 | ph = np.zeros(batch_size, dtype = np.int32) 244 | pt = np.zeros(batch_size, dtype = np.int32) 245 | pr = np.zeros(batch_size, dtype = np.int32) 246 | nh = np.zeros(batch_size, dtype = np.int32) 247 | nt = np.zeros(batch_size, dtype = np.int32) 248 | nr = np.zeros(batch_size, dtype = np.int32) 249 | ph_addr = ph.__array_interface__['data'][0] 250 | pt_addr = pt.__array_interface__['data'][0] 251 | pr_addr = pr.__array_interface__['data'][0] 252 | nh_addr = nh.__array_interface__['data'][0] 253 | nt_addr = nt.__array_interface__['data'][0] 254 | nr_addr = nr.__array_interface__['data'][0] 255 | lib.getBatch.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_int] 256 | times_kg = 0 257 | while not coord.should_stop(): 258 | times_kg += 1 259 | # if times_kg == 3000: 260 | # coord.request_stop() 261 | res = 0.0 262 | for batch in range(FLAGS.nbatch_kg): 263 | lib.getBatch(ph_addr, pt_addr, pr_addr, nh_addr, nt_addr, nr_addr, batch_size) 264 | res += train_step_kg(ph, pt, pr, nh, nt, nr) 265 | time_str = datetime.datetime.now().isoformat() 266 | print "batch %d time %s | loss : %f" % (times_kg, time_str, res) 267 | 268 | 269 | def train_nn(coord): 270 | def train_step(head, tail, word, pos1, pos2, mask, leng, label_index, label, scope, weights): 271 | feed_dict = { 272 | model.head_index: head, 273 | model.tail_index: tail, 274 | model.word: word, 275 | model.pos1: pos1, 276 | model.pos2: pos2, 277 | model.mask: mask, 278 | model.len : leng, 279 | model.label_index: label_index, 280 | model.label: label, 281 | model.scope: scope, 282 | model.keep_prob: FLAGS.keep_prob, 283 | model.weights: weights 284 | } 285 | _, step, loss, summary, output, correct_predictions = sess.run([train_op, global_step, model.loss, merged_summary, model.output, model.correct_predictions], feed_dict) 286 | summary_writer.add_summary(summary, step) 287 | return output, loss, correct_predictions 288 | 289 | stack_output = [] 290 | stack_label = [] 291 | stack_ce_loss = [] 292 | 293 | train_order = range(len(instance_triple)) 294 | 295 | save_epoch = 2 296 | eval_step = 300 297 | 298 | for one_epoch in range(FLAGS.max_epoch): 299 | 300 | print('epoch '+str(one_epoch+1)+' starts!') 301 | np.random.shuffle(train_order) 302 | s1 = 0.0 303 | s2 = 0.0 304 | tot1 = 1.0 305 | tot2 = 1.0 306 | losstot = 0.0 307 | for i in range(int(len(train_order)/float(FLAGS.batch_size))): 308 | input_scope = np.take(instance_scope, train_order[i * FLAGS.batch_size:(i+1)*FLAGS.batch_size], axis=0) 309 | index = [] 310 | scope = [0] 311 | label = [] 312 | weights = [] 313 | for num in input_scope: 314 | index = index + range(num[0], num[1] + 1) 315 | label.append(train_label[num[0]]) 316 | scope.append(scope[len(scope)-1] + num[1] - num[0] + 1) 317 | weights.append(reltot[train_label[num[0]]]) 318 | label_ = np.zeros((FLAGS.batch_size, FLAGS.num_classes)) 319 | label_[np.arange(FLAGS.batch_size), label] = 1 320 | output, loss, correct_predictions = train_step(train_head[index], train_tail[index], train_word[index,:], train_pos1[index,:], train_pos2[index,:], train_mask[index,:], train_len[index],train_label[index], label_, np.array(scope), weights) 321 | num = 0 322 | s = 0 323 | losstot += loss 324 | for num in correct_predictions: 325 | if label[s] == 0: 326 | tot1 += 1.0 327 | if num: 328 | s1+= 1.0 329 | else: 330 | tot2 += 1.0 331 | if num: 332 | s2 += 1.0 333 | s = s + 1 334 | 335 | time_str = datetime.datetime.now().isoformat() 336 | # print "batch %d step %d time %s | loss : %f, NA accuracy: %f, not NA accuracy: %f" % (one_epoch, i, time_str, loss, s1 / tot1, s2 / tot2) 337 | current_step = tf.train.global_step(sess, global_step) 338 | 339 | if (one_epoch + 1) % save_epoch == 0: 340 | print 'epoch '+str(one_epoch+1)+' has finished' 341 | print 'saving model...' 342 | path = saver.save(sess,FLAGS.model_dir+FLAGS.model+str(FLAGS.katt_flag), global_step=current_step) 343 | coord.request_stop() 344 | 345 | coord = tf.train.Coordinator() 346 | threads = [] 347 | threads.append(threading.Thread(target=train_kg, args=(coord,))) 348 | threads.append(threading.Thread(target=train_nn, args=(coord,))) 349 | threads.append(threading.Thread(target=train_kg_att, args=(coord,))) 350 | for t in threads: t.start() 351 | coord.join(threads) 352 | if (FLAGS.store_kg_flag != 0): 353 | print 'saving kg...' 354 | ent_embedding, rel_embedding = sess.run([model.word_embedding, model.rel_embeddings]) 355 | ent_embedding = ent_embedding.tolist() 356 | rel_embedding = rel_embedding.tolist() 357 | f = open("entity2vec", "w") 358 | f.write(json.dumps(ent_embedding)) 359 | f.close() 360 | f = open("relation2vec", "w") 361 | f.write(json.dumps(rel_embedding)) 362 | f.close() 363 | 364 | 365 | if __name__ == "__main__": 366 | tf.app.run() 367 | -------------------------------------------------------------------------------- /original/baselines/test/init_cnn.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | 7 | using namespace std; 8 | 9 | const float pi = 3.141592653589793238462643383; 10 | string path = "../data/FB60K/"; 11 | 12 | float rand(float min, float max) { 13 | return min + (max - min) * rand() / (RAND_MAX + 1.0); 14 | } 15 | 16 | float normal(float x, float miu,float sigma) { 17 | return 1.0/sqrt(2*pi)/sigma*exp(-1*(x-miu)*(x-miu)/(2*sigma*sigma)); 18 | } 19 | 20 | float randn(float miu,float sigma, float min ,float max) { 21 | float x, y, dScope; 22 | do { 23 | x = rand(min,max); 24 | y = normal(x,miu,sigma); 25 | dScope=rand(0.0,normal(miu,miu,sigma)); 26 | } while (dScope > y); 27 | return x; 28 | } 29 | 30 | int word_size; 31 | int entity_size; 32 | int dimension; 33 | int PositionLimit = 30; 34 | int LenLimit = 100; 35 | int NA = -1; 36 | float *word_embeddings; 37 | 38 | struct Tip { 39 | int h; 40 | int t; 41 | int r; 42 | int tot; 43 | int *lists; 44 | }; 45 | Tip *tipList; 46 | int tipTotal; 47 | int relationTotal = 0; 48 | int sentenceTotal; 49 | int instanceTot = 0; 50 | int *sentence, *posH, *posT, *bags_train; 51 | 52 | extern "C" 53 | void setNA(int con) { 54 | NA = con; 55 | } 56 | 57 | int getPosition(int position) { 58 | if (position < -PositionLimit) return 0; 59 | if (position > PositionLimit) return 2 * PositionLimit; 60 | return position + PositionLimit; 61 | } 62 | 63 | extern "C" 64 | void readWordVec() { 65 | 66 | FILE *fin1 = fopen((path + "entity2id.txt").c_str(), "r"); 67 | FILE *fin2 = fopen((path + "vec.txt").c_str(), "r"); 68 | fscanf(fin2, "%d%d\n", &word_size, &dimension); 69 | fscanf(fin1, "%d", &entity_size); 70 | 71 | char buffer[100]; 72 | printf("hx\n"); 73 | word_size += 2 + entity_size; 74 | word_embeddings = (float *)calloc(word_size * dimension, sizeof(float)); 75 | int last = 0; 76 | for (int i = entity_size * dimension; i > 0 ; i--) 77 | word_embeddings[last++] = randn(0, 1.0 / dimension, -6 / sqrt(dimension), 6 / sqrt(dimension)); 78 | last += 2 * dimension; 79 | printf("hx\n"); 80 | for (int i = entity_size + 2; i < word_size; i++) { 81 | fscanf(fin2, "%s", buffer); 82 | for (int i = 0; i < dimension; i++) 83 | fscanf(fin2, "%f", &word_embeddings[last++]); 84 | } 85 | 86 | fclose(fin2); 87 | fclose(fin1); 88 | } 89 | 90 | extern "C" 91 | void getWordVec(float *con) { 92 | for (int i = word_size * dimension - 1; i >= 0; i--) 93 | con[i] = word_embeddings[i]; 94 | } 95 | 96 | int last = 0; 97 | extern "C" 98 | int batch_iter(int *x_batch, int *p_h_batch, int *p_t_batch, int *y_batch, int *r_batch, float *r_n_batch, int *h_batch, int *t_batch) { 99 | int n = last; 100 | last++; 101 | if (n >= tipTotal) { 102 | n = 0; 103 | last = 0; 104 | } 105 | int instance = tipList[n].tot; 106 | int last = 0; 107 | for (int i = 0; i < instance; i++) { 108 | int j = tipList[n].lists[i]; 109 | int last1 = j * LenLimit; 110 | for (int k = 0; k < LenLimit; k++) { 111 | x_batch[last] = sentence[last1]; 112 | p_h_batch[last] = posH[last1]; 113 | p_t_batch[last] = posT[last1]; 114 | last++; 115 | last1++; 116 | } 117 | } 118 | r_batch[0] = tipList[n].r; 119 | if (r_batch[0] == NA) 120 | r_n_batch[0] = 0; 121 | else 122 | r_n_batch[0] = 1; 123 | for (int j = 0; j < relationTotal; j++) 124 | y_batch[j] = 0; 125 | y_batch[tipList[n].r] = 1; 126 | h_batch[0] = tipList[n].h; 127 | t_batch[0] = tipList[n].t; 128 | return instance; 129 | } 130 | 131 | extern "C" 132 | int getTipTotal() { 133 | return tipTotal; 134 | } 135 | 136 | extern "C" 137 | int getLenLimit() { 138 | return LenLimit; 139 | } 140 | 141 | extern "C" 142 | int getRelationTotal() { 143 | return relationTotal; 144 | } 145 | 146 | extern "C" 147 | int getWordTotal() { 148 | return word_size; 149 | } 150 | 151 | extern "C" 152 | int getPositionLimit() { 153 | return PositionLimit; 154 | } 155 | 156 | extern "C" 157 | int getWordDimension() { 158 | return dimension; 159 | } 160 | 161 | extern "C" 162 | int getInstanceTot() { 163 | return instanceTot; 164 | } 165 | 166 | extern "C" 167 | void readFromFile() { 168 | FILE *f = fopen((path + "test2id.txt").c_str(), "r"); 169 | fscanf(f, "%d\n", &tipTotal); 170 | fscanf(f, "%d\n", &sentenceTotal); 171 | tipList = (Tip *)calloc(tipTotal, sizeof(Tip)); 172 | sentence = (int *)calloc(sentenceTotal * LenLimit, sizeof(int)); 173 | posH = (int *)calloc(sentenceTotal * LenLimit, sizeof(int)); 174 | posT = (int *)calloc(sentenceTotal * LenLimit, sizeof(int)); 175 | bags_train = (int *)calloc(sentenceTotal, sizeof(int)); 176 | int h, t, post, posh, r, len, tip; 177 | for (int i = 0; i < sentenceTotal; i++) { 178 | fscanf(f, "%d%d%d%d%d%d%d",&h, &t, &posh, &post, &r, &tip, &len); 179 | int last = i * LenLimit; 180 | for (int j = 0; j < len; j++) { 181 | fscanf(f, "%d", &sentence[last + j]); 182 | posH[last + j] = getPosition(j - posh); 183 | posT[last + j] = getPosition(j - post); 184 | } 185 | bags_train[i] = tip; 186 | tipList[tip].tot++; 187 | tipList[tip].h = h; 188 | tipList[tip].r = r; 189 | tipList[tip].t = t; 190 | if (r + 1 > relationTotal) 191 | relationTotal = r + 1; 192 | if (tipList[tip].tot > instanceTot) 193 | instanceTot = tipList[tip].tot; 194 | } 195 | fclose(f); 196 | for (int i = 0; i < tipTotal; i++) { 197 | tipList[i].lists = new int[tipList[i].tot]; 198 | tipList[i].tot = 0; 199 | } 200 | for (int i = 0; i < sentenceTotal; i++) 201 | tipList[bags_train[i]].lists[tipList[bags_train[i]].tot++] = i; 202 | relationTotal = 56; 203 | } 204 | 205 | int main() { 206 | readWordVec(); 207 | readFromFile(); 208 | return 0; 209 | } 210 | -------------------------------------------------------------------------------- /original/baselines/test/init_know.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | 9 | using namespace std; 10 | 11 | string inPath = "../data/FB60K/"; 12 | 13 | int *lefHead, *rigHead; 14 | int *lefTail, *rigTail; 15 | 16 | struct Triple { 17 | int h, r, t; 18 | }; 19 | 20 | struct cmp_head { 21 | bool operator()(const Triple &a, const Triple &b) { 22 | return (a.h < b.h)||(a.h == b.h && a.r < b.r)||(a.h == b.h && a.r == b.r && a.t < b.t); 23 | } 24 | }; 25 | 26 | struct cmp_tail { 27 | bool operator()(const Triple &a, const Triple &b) { 28 | return (a.t < b.t)||(a.t == b.t && a.r < b.r)||(a.t == b.t && a.r == b.r && a.h < b.h); 29 | } 30 | }; 31 | 32 | struct cmp_list { 33 | int minimal(int a,int b) { 34 | if (a > b) return b; 35 | return a; 36 | } 37 | bool operator()(const Triple &a, const Triple &b) { 38 | return (minimal(a.h, a.t) > minimal(b.h, b.t)); 39 | } 40 | }; 41 | 42 | Triple *trainHead, *trainTail, *trainList; 43 | int relationTotal, entityTotal, tripleTotal; 44 | int *freqRel, *freqEnt; 45 | float *left_mean, *right_mean; 46 | 47 | extern "C" 48 | void init() { 49 | 50 | FILE *fin; 51 | int tmp; 52 | 53 | fin = fopen((inPath + "relation2id.txt").c_str(), "r"); 54 | tmp = fscanf(fin, "%d", &relationTotal); 55 | fclose(fin); 56 | 57 | freqRel = (int *)calloc(relationTotal, sizeof(int)); 58 | 59 | fin = fopen((inPath + "entity2id.txt").c_str(), "r"); 60 | tmp = fscanf(fin, "%d", &entityTotal); 61 | fclose(fin); 62 | 63 | freqEnt = (int *)calloc(entityTotal, sizeof(int)); 64 | 65 | fin = fopen((inPath + "triple2id.txt").c_str(), "r"); 66 | tmp = fscanf(fin, "%d", &tripleTotal); 67 | trainHead = (Triple *)calloc(tripleTotal, sizeof(Triple)); 68 | trainTail = (Triple *)calloc(tripleTotal, sizeof(Triple)); 69 | trainList = (Triple *)calloc(tripleTotal, sizeof(Triple)); 70 | tripleTotal = 0; 71 | while (fscanf(fin, "%d", &trainList[tripleTotal].h) == 1) { 72 | tmp = fscanf(fin, "%d", &trainList[tripleTotal].t); 73 | tmp = fscanf(fin, "%d", &trainList[tripleTotal].r); 74 | freqEnt[trainList[tripleTotal].t]++; 75 | freqEnt[trainList[tripleTotal].h]++; 76 | freqRel[trainList[tripleTotal].r]++; 77 | trainHead[tripleTotal].h = trainList[tripleTotal].h; 78 | trainHead[tripleTotal].t = trainList[tripleTotal].t; 79 | trainHead[tripleTotal].r = trainList[tripleTotal].r; 80 | trainTail[tripleTotal].h = trainList[tripleTotal].h; 81 | trainTail[tripleTotal].t = trainList[tripleTotal].t; 82 | trainTail[tripleTotal].r = trainList[tripleTotal].r; 83 | tripleTotal++; 84 | } 85 | fclose(fin); 86 | 87 | sort(trainHead, trainHead + tripleTotal, cmp_head()); 88 | sort(trainTail, trainTail + tripleTotal, cmp_tail()); 89 | 90 | lefHead = (int *)calloc(entityTotal, sizeof(int)); 91 | rigHead = (int *)calloc(entityTotal, sizeof(int)); 92 | lefTail = (int *)calloc(entityTotal, sizeof(int)); 93 | rigTail = (int *)calloc(entityTotal, sizeof(int)); 94 | memset(rigHead, -1, sizeof(rigHead)); 95 | memset(rigTail, -1, sizeof(rigTail)); 96 | for (int i = 1; i < tripleTotal; i++) { 97 | if (trainTail[i].t != trainTail[i - 1].t) { 98 | rigTail[trainTail[i - 1].t] = i - 1; 99 | lefTail[trainTail[i].t] = i; 100 | } 101 | if (trainHead[i].h != trainHead[i - 1].h) { 102 | rigHead[trainHead[i - 1].h] = i - 1; 103 | lefHead[trainHead[i].h] = i; 104 | } 105 | } 106 | rigHead[trainHead[tripleTotal - 1].h] = tripleTotal - 1; 107 | rigTail[trainTail[tripleTotal - 1].t] = tripleTotal - 1; 108 | 109 | left_mean = (float *)calloc(relationTotal,sizeof(float)); 110 | right_mean = (float *)calloc(relationTotal,sizeof(float)); 111 | for (int i = 0; i < entityTotal; i++) { 112 | for (int j = lefHead[i] + 1; j < rigHead[i]; j++) 113 | if (trainHead[j].r != trainHead[j - 1].r) 114 | left_mean[trainHead[j].r] += 1.0; 115 | if (lefHead[i] <= rigHead[i]) 116 | left_mean[trainHead[lefHead[i]].r] += 1.0; 117 | for (int j = lefTail[i] + 1; j < rigTail[i]; j++) 118 | if (trainTail[j].r != trainTail[j - 1].r) 119 | right_mean[trainTail[j].r] += 1.0; 120 | if (lefTail[i] <= rigTail[i]) 121 | right_mean[trainTail[lefTail[i]].r] += 1.0; 122 | } 123 | for (int i = 0; i < relationTotal; i++) { 124 | left_mean[i] = freqRel[i] / left_mean[i]; 125 | right_mean[i] = freqRel[i] / right_mean[i]; 126 | } 127 | } 128 | 129 | extern "C" 130 | int getEntityTotal() { 131 | return entityTotal; 132 | } 133 | 134 | extern "C" 135 | int getRelationTotal() { 136 | return relationTotal; 137 | } 138 | 139 | extern "C" 140 | int getTripleTotal() { 141 | return tripleTotal; 142 | } 143 | 144 | // unsigned long long *next_random; 145 | unsigned long long next_random = 3; 146 | 147 | unsigned long long randd(int id) { 148 | next_random = next_random * (unsigned long long)25214903917 + 11; 149 | return next_random; 150 | } 151 | 152 | int rand_max(int id, int x) { 153 | int res = randd(id) % x; 154 | while (res<0) 155 | res+=x; 156 | return res; 157 | } 158 | 159 | int corrupt_head(int id, int h, int r) { 160 | int lef, rig, mid, ll, rr; 161 | lef = lefHead[h] - 1; 162 | rig = rigHead[h]; 163 | while (lef + 1 < rig) { 164 | mid = (lef + rig) >> 1; 165 | if (trainHead[mid].r >= r) rig = mid; else 166 | lef = mid; 167 | } 168 | ll = rig; 169 | lef = lefHead[h]; 170 | rig = rigHead[h] + 1; 171 | while (lef + 1 < rig) { 172 | mid = (lef + rig) >> 1; 173 | if (trainHead[mid].r <= r) lef = mid; else 174 | rig = mid; 175 | } 176 | rr = lef; 177 | int tmp = rand_max(id, entityTotal - (rr - ll + 1)); 178 | if (tmp < trainHead[ll].t) return tmp; 179 | if (tmp > trainHead[rr].t - rr + ll - 1) return tmp + rr - ll + 1; 180 | lef = ll, rig = rr + 1; 181 | while (lef + 1 < rig) { 182 | mid = (lef + rig) >> 1; 183 | if (trainHead[mid].t - mid + ll - 1 < tmp) 184 | lef = mid; 185 | else 186 | rig = mid; 187 | } 188 | return tmp + lef - ll + 1; 189 | } 190 | 191 | int corrupt_tail(int id, int t, int r) { 192 | int lef, rig, mid, ll, rr; 193 | lef = lefTail[t] - 1; 194 | rig = rigTail[t]; 195 | while (lef + 1 < rig) { 196 | mid = (lef + rig) >> 1; 197 | if (trainTail[mid].r >= r) rig = mid; else 198 | lef = mid; 199 | } 200 | ll = rig; 201 | lef = lefTail[t]; 202 | rig = rigTail[t] + 1; 203 | while (lef + 1 < rig) { 204 | mid = (lef + rig) >> 1; 205 | if (trainTail[mid].r <= r) lef = mid; else 206 | rig = mid; 207 | } 208 | rr = lef; 209 | int tmp = rand_max(id, entityTotal - (rr - ll + 1)); 210 | if (tmp < trainTail[ll].h) return tmp; 211 | if (tmp > trainTail[rr].h - rr + ll - 1) return tmp + rr - ll + 1; 212 | lef = ll, rig = rr + 1; 213 | while (lef + 1 < rig) { 214 | mid = (lef + rig) >> 1; 215 | if (trainTail[mid].h - mid + ll - 1 < tmp) 216 | lef = mid; 217 | else 218 | rig = mid; 219 | } 220 | return tmp + lef - ll + 1; 221 | } 222 | 223 | extern "C" 224 | void getBatch(int *ph, int *pt, int *pr, int *nh, int *nt, int *nr, int batchSize, int id = 0) { 225 | for (int batch = 0; batch < batchSize; batch++) { 226 | int i = rand_max(id, tripleTotal), j; 227 | float prob = 1000 * right_mean[trainList[i].r] / (right_mean[trainList[i].r] + left_mean[trainList[i].r]); 228 | if (randd(id) % 1000 < prob) { 229 | j = corrupt_head(id, trainList[i].h, trainList[i].r); 230 | ph[batch] = trainList[i].h; 231 | pt[batch] = trainList[i].t; 232 | pr[batch] = trainList[i].r; 233 | nh[batch] = trainList[i].h; 234 | nt[batch] = j; 235 | nr[batch] = trainList[i].r; 236 | } else { 237 | j = corrupt_tail(id, trainList[i].t, trainList[i].r); 238 | ph[batch] = trainList[i].h; 239 | pt[batch] = trainList[i].t; 240 | pr[batch] = trainList[i].r; 241 | nh[batch] = j; 242 | nt[batch] = trainList[i].t; 243 | nr[batch] = trainList[i].r; 244 | } 245 | ph[batchSize + batch] = trainList[i].h; 246 | pt[batchSize + batch] = trainList[i].t; 247 | pr[batchSize + batch] = trainList[i].r; 248 | nh[batchSize + batch] = trainList[i].h; 249 | nt[batchSize + batch] = trainList[i].t; 250 | nr[batchSize + batch] = rand_max(id, relationTotal); 251 | } 252 | } 253 | 254 | int main() { 255 | init(); 256 | return 0; 257 | } -------------------------------------------------------------------------------- /original/baselines/test/test_JointD+ATT.py: -------------------------------------------------------------------------------- 1 | #coding:utf-8 2 | import numpy as np 3 | import tensorflow as tf 4 | import os 5 | import time 6 | import datetime 7 | import ctypes 8 | import threading 9 | import json 10 | 11 | ll1 = ctypes.cdll.LoadLibrary 12 | lib_cnn = ll1("./init_cnn.so") 13 | ll2 = ctypes.cdll.LoadLibrary 14 | lib_kg = ll2("./init_know.so") 15 | 16 | class Config(object): 17 | def __init__(self): 18 | self.instanceTot = lib_cnn.getInstanceTot() 19 | self.sequence_size = lib_cnn.getLenLimit() 20 | self.num_classes = lib_cnn.getRelationTotal() 21 | self.num_words = lib_cnn.getWordTotal() 22 | self.num_positions = 2 * lib_cnn.getPositionLimit() + 1 23 | self.word_size = lib_cnn.getWordDimension() 24 | self.position_size = 5 25 | self.embedding_size = self.word_size + self.position_size * 2 26 | self.filter_size = 3 27 | self.num_filters = 230 28 | self.relation_size = self.word_size 29 | self.dropout_keep_prob = 0.5 30 | self.l2_lambda = 0.0001 31 | lib_cnn.setNA(51) 32 | self.margin = 1.0 33 | self.nbatches = 100 34 | self.trainTimes = 15 35 | self.entityTotal = 0 36 | self.relationTotal = 0 37 | 38 | class Model(object): 39 | 40 | 41 | def calc(self, e, t, r): 42 | return e + tf.reduce_sum(e * t, 1, keep_dims = True) * r 43 | 44 | def __init__(self, config): 45 | sequence_size = config.sequence_size 46 | num_classes = config.num_classes 47 | num_words = config.num_words 48 | num_positions = config.num_positions 49 | 50 | embedding_size = config.embedding_size 51 | word_size = config.word_size 52 | position_size = config.position_size 53 | relation_size = config.relation_size 54 | filter_size = config.filter_size 55 | num_filters = config.num_filters 56 | dropout_keep_prob = config.dropout_keep_prob 57 | 58 | margin = config.margin 59 | l2_lambda = config.l2_lambda 60 | 61 | self.input_x = tf.placeholder(tf.int32, [None, sequence_size], name = "input_x") 62 | self.input_p_h = tf.placeholder(tf.int32, [None, sequence_size], name = "input_p_h") 63 | self.input_p_t = tf.placeholder(tf.int32, [None, sequence_size], name = "input_p_t") 64 | self.input_r = tf.placeholder(tf.int32, [1, 1], name = "input_r") 65 | self.input_r_n = tf.placeholder(tf.float32, [1, 1], name = "input_r_n") 66 | self.input_h = tf.placeholder(tf.int32, [1, 1], name = "input_t") 67 | self.input_t = tf.placeholder(tf.int32, [1, 1], name = "input_t") 68 | self.input_y = tf.placeholder(tf.float32, [1, num_classes], name = "input_y") 69 | 70 | self.pos_h = tf.placeholder(tf.int32, [None]) 71 | self.pos_t = tf.placeholder(tf.int32, [None]) 72 | self.pos_r = tf.placeholder(tf.int32, [None]) 73 | self.neg_h = tf.placeholder(tf.int32, [None]) 74 | self.neg_t = tf.placeholder(tf.int32, [None]) 75 | self.neg_r = tf.placeholder(tf.int32, [None]) 76 | 77 | l2_loss = tf.constant(0.0) 78 | with tf.name_scope("embedding-lookup"): 79 | self.word_embeddings = tf.Variable(word_embeddings, name="word_embeddings") 80 | self.relation_embeddings = tf.Variable(relation_embeddings, name="relation_embeddings") 81 | self.position_embeddings = tf.Variable(position_embeddings, name="position_embeddings") 82 | self.relation_attention = tf.Variable(relation_attention, name="relation_attention") 83 | self.NAattention = tf.Variable(NAattention, name="NAattention") 84 | self.attention = tf.Variable(attention, name="attention") 85 | self.ent_transfer = tf.Variable(ent_transfer, name = "ent_transfer") 86 | self.rel_transfer = tf.Variable(rel_transfer, name = "rel_transfer") 87 | 88 | self.r = tf.nn.embedding_lookup(self.attention, self.input_r) 89 | 90 | #know 91 | pos_h_e = tf.nn.embedding_lookup(self.word_embeddings, self.pos_h) 92 | pos_t_e = tf.nn.embedding_lookup(self.word_embeddings, self.pos_t) 93 | pos_r_e = tf.nn.embedding_lookup(self.relation_embeddings, self.pos_r) 94 | pos_h_t = tf.nn.embedding_lookup(self.ent_transfer, self.pos_h) 95 | pos_t_t = tf.nn.embedding_lookup(self.ent_transfer, self.pos_t) 96 | pos_r_t = tf.nn.embedding_lookup(self.rel_transfer, self.pos_r) 97 | neg_h_e = tf.nn.embedding_lookup(self.word_embeddings, self.neg_h) 98 | neg_t_e = tf.nn.embedding_lookup(self.word_embeddings, self.neg_t) 99 | neg_r_e = tf.nn.embedding_lookup(self.relation_embeddings, self.neg_r) 100 | neg_h_t = tf.nn.embedding_lookup(self.ent_transfer, self.neg_h) 101 | neg_t_t = tf.nn.embedding_lookup(self.ent_transfer, self.neg_t) 102 | neg_r_t = tf.nn.embedding_lookup(self.rel_transfer, self.neg_r) 103 | pos_h_e = self.calc(pos_h_e, pos_h_t, pos_r_t) 104 | pos_t_e = self.calc(pos_t_e, pos_t_t, pos_r_t) 105 | neg_h_e = self.calc(neg_h_e, neg_h_t, neg_r_t) 106 | neg_t_e = self.calc(neg_t_e, neg_t_t, neg_r_t) 107 | 108 | #cnn 109 | self.x_initial = tf.nn.embedding_lookup(self.word_embeddings, self.input_x) 110 | self.x_p_h = tf.nn.embedding_lookup(self.position_embeddings, self.input_p_h) 111 | self.x_p_t = tf.nn.embedding_lookup(self.position_embeddings, self.input_p_t) 112 | self.x = tf.expand_dims(tf.concat(2, [self.x_initial, self.x_p_h, self.x_p_t]), -1) 113 | self.head = tf.nn.embedding_lookup(self.word_embeddings, self.input_h) 114 | self.tail = tf.nn.embedding_lookup(self.word_embeddings, self.input_t) 115 | self.head_t = tf.nn.embedding_lookup(self.ent_transfer, self.input_h) 116 | self.tail_t = tf.nn.embedding_lookup(self.ent_transfer, self.input_t) 117 | self.r_t = tf.nn.embedding_lookup(self.rel_transfer, self.input_r) 118 | self.head = self.calc(self.head, self.head_t, self.r_t) 119 | self.tail = self.calc(self.tail, self.tail_t, self.r_t) 120 | l2_loss += tf.nn.l2_loss(self.attention) 121 | 122 | with tf.name_scope("conv-maxpool"): 123 | self.W = tf.Variable(W, name="W") 124 | self.b = tf.Variable(B, name="b") 125 | conv = tf.nn.conv2d(self.x, self.W, strides=[1, 1, 1, 1], padding="VALID", name="conv") 126 | h = tf.nn.tanh(tf.nn.bias_add(conv, self.b), name="tanh") 127 | self.y = tf.nn.max_pool(h, ksize=[1, sequence_size - filter_size + 1, 1, 1], strides=[1, 1, 1, 1], padding='VALID', name="pool") 128 | l2_loss += tf.nn.l2_loss(self.W) 129 | l2_loss += tf.nn.l2_loss(self.b) 130 | self.y = tf.reshape(self.y, [-1, num_filters]) 131 | 132 | with tf.name_scope('attention'): 133 | self.r = tf.reshape(self.r, [relation_size, -1]) 134 | self.e = tf.matmul(tf.nn.tanh(tf.matmul(self.y, self.attention)), self.r) 135 | alpha = tf.reshape(self.e, [1, -1]) 136 | self.alpha_reshape = tf.nn.softmax(alpha) 137 | self.y_attention = tf.matmul(self.alpha_reshape, self.y) 138 | 139 | with tf.name_scope("dropout"): 140 | self.transfer_w = tf.Variable(transfer_w, name="transfer_w") 141 | self.h_drop = tf.nn.l2_normalize(self.y_attention, 1) 142 | self.scores = tf.matmul(self.h_drop, self.transfer_w) 143 | self.scoress = tf.nn.softmax(tf.reshape(self.scores, [1, -1])) 144 | 145 | bags_sum = 0.0 146 | bags_hit_NA = 0.0 147 | sum_NA = 0.0 148 | sum_fNA = 0.0 149 | bags_hit = 0.0 150 | loss_sum = 0.0 151 | ss = [] 152 | flag = True 153 | 154 | if __name__ == "__main__": 155 | 156 | lib_cnn.readWordVec() 157 | lib_cnn.readFromFile() 158 | lib_kg.init() 159 | 160 | np.random.seed(0) 161 | tf.set_random_seed(0) 162 | config = Config() 163 | 164 | word_embeddings = np.zeros(config.num_words * config.word_size, dtype = np.float32) 165 | lib_cnn.getWordVec.argtypes = [ctypes.c_void_p] 166 | lib_cnn.getWordVec(word_embeddings.__array_interface__['data'][0]) 167 | word_embeddings.resize((config.num_words,config.word_size)) 168 | 169 | config.batch_size = lib_kg.getTripleTotal() / config.nbatches 170 | config.entityTotal = lib_kg.getEntityTotal() 171 | config.relationTotal = lib_kg.getRelationTotal() 172 | 173 | config.num_classes = 56 174 | log = open("log14.txt", "r") 175 | word_embeddings = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_words , config.word_size)) 176 | relation_embeddings = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.relationTotal, config.word_size)) 177 | position_embeddings = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_positions, config.position_size)) 178 | relation_attention = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_classes, config.relation_size)) 179 | attention = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_filters, config.relation_size)) 180 | W = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.filter_size, config.embedding_size, 1, config.num_filters)) 181 | B = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_filters)) 182 | transfer_w = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape(config.num_filters, config.num_classes) 183 | NAattention = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.relation_size, 1)) 184 | ent_transfer = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_words , config.word_size)) 185 | rel_transfer = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.relationTotal, config.word_size)) 186 | log.close() 187 | 188 | 189 | with tf.Graph().as_default(): 190 | conf = tf.ConfigProto() 191 | sess = tf.Session(config=conf) 192 | with sess.as_default(): 193 | initializer = tf.contrib.layers.xavier_initializer() 194 | with tf.variable_scope("model", reuse=None, initializer = initializer): 195 | m = Model(config = config) 196 | 197 | global_step_cnn = tf.Variable(0, name="global_step_cnn", trainable=False) 198 | sess.run(tf.initialize_all_variables()) 199 | 200 | x_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 201 | p_t_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 202 | p_h_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 203 | r_batch = np.zeros((1, 1), dtype = np.int32) 204 | y_batch = np.zeros((1, config.num_classes), dtype = np.int32) 205 | r_n_batch = np.zeros((1, 1), dtype = np.int32) 206 | h_batch = np.zeros((1, 1), dtype = np.int32) 207 | t_batch = np.zeros((1, 1), dtype = np.int32) 208 | 209 | x_batch_addr = x_batch.__array_interface__['data'][0] 210 | p_t_batch_addr = p_t_batch.__array_interface__['data'][0] 211 | p_h_batch_addr = p_h_batch.__array_interface__['data'][0] 212 | y_batch_addr = y_batch.__array_interface__['data'][0] 213 | r_batch_addr = r_batch.__array_interface__['data'][0] 214 | r_n_batch_addr = r_n_batch.__array_interface__['data'][0] 215 | h_batch_addr = h_batch.__array_interface__['data'][0] 216 | t_batch_addr = t_batch.__array_interface__['data'][0] 217 | lib_cnn.batch_iter.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p] 218 | tipTotal = lib_cnn.getTipTotal() 219 | 220 | def train_step_cnn(x_batch, p_h_batch, p_t_batch, y_batch, r_batch, r_n_batch, h_batch, t_batch,i): 221 | global ss, bags_sum, bags_hit, loss_sum, bags_hit_NA, bags_hit, sum_fNA, sum_NA, flag 222 | feed_dict = { 223 | m.input_x: x_batch, 224 | m.input_p_h: p_h_batch, 225 | m.input_p_t: p_t_batch, 226 | m.input_r: r_batch, 227 | m.input_r_n: r_n_batch, 228 | m.input_y: y_batch, 229 | m.input_h: h_batch, 230 | m.input_t: t_batch 231 | } 232 | scores, step, accuracy, accuracy1,scoresgg,scores1gg = sess.run( 233 | [m.scoress, global_step_cnn, m.accuracy,m.accuracy1,m.y_attention,m.h_drop], feed_dict) 234 | time_str = datetime.datetime.now().isoformat() 235 | for i in range(config.num_classes): 236 | if (r_batch[0] == i): 237 | ss.append((i,scores[0][i],1)) 238 | else: 239 | ss.append((i,scores[0][i],0)) 240 | if bags_sum % 100 == 0: 241 | print("{}: step {}".format(time_str, step)) 242 | 243 | for i in range(tipTotal): 244 | length = lib_cnn.batch_iter(x_batch_addr, p_h_batch_addr, p_t_batch_addr, y_batch_addr, r_batch_addr, r_n_batch_addr, h_batch_addr, t_batch_addr) 245 | train_step_cnn(x_batch[0:length,], p_h_batch[0:length,], p_t_batch[0:length,], y_batch, r_batch, r_n_batch, h_batch, t_batch,i) 246 | if not flag: 247 | break 248 | 249 | f = open("res.txt", "w") 250 | for i in ss: 251 | f.write("%d\t%f\t%d\n"%(i[0],i[1],i[2])) 252 | f.close() 253 | -------------------------------------------------------------------------------- /original/baselines/test/test_JointD+ONE.py: -------------------------------------------------------------------------------- 1 | #coding:utf-8 2 | import numpy as np 3 | import tensorflow as tf 4 | import os 5 | import time 6 | import datetime 7 | import ctypes 8 | import threading 9 | import json 10 | 11 | ll1 = ctypes.cdll.LoadLibrary 12 | lib_cnn = ll1("./init_cnn.so") 13 | ll2 = ctypes.cdll.LoadLibrary 14 | lib_kg = ll2("./init_know.so") 15 | 16 | class Config(object): 17 | def __init__(self): 18 | self.instanceTot = lib_cnn.getInstanceTot() 19 | self.sequence_size = lib_cnn.getLenLimit() 20 | self.num_classes = lib_cnn.getRelationTotal() 21 | self.num_words = lib_cnn.getWordTotal() 22 | self.num_positions = 2 * lib_cnn.getPositionLimit() + 1 23 | self.word_size = lib_cnn.getWordDimension() 24 | self.position_size = 5 25 | self.embedding_size = self.word_size + self.position_size * 2 26 | self.filter_size = 3 27 | self.num_filters = 230 28 | self.relation_size = self.word_size 29 | self.dropout_keep_prob = 0.5 30 | self.l2_lambda = 0.0001 31 | lib_cnn.setNA(51) 32 | self.margin = 1.0 33 | self.nbatches = 100 34 | self.trainTimes = 15 35 | self.entityTotal = 0 36 | self.relationTotal = 0 37 | 38 | class Model(object): 39 | 40 | 41 | def calc(self, e, t, r): 42 | return e + tf.reduce_sum(e * t, 1, keep_dims = True) * r 43 | 44 | def __init__(self, config): 45 | sequence_size = config.sequence_size 46 | num_classes = config.num_classes 47 | num_words = config.num_words 48 | num_positions = config.num_positions 49 | 50 | embedding_size = config.embedding_size 51 | word_size = config.word_size 52 | position_size = config.position_size 53 | relation_size = config.relation_size 54 | filter_size = config.filter_size 55 | num_filters = config.num_filters 56 | dropout_keep_prob = config.dropout_keep_prob 57 | 58 | margin = config.margin 59 | l2_lambda = config.l2_lambda 60 | 61 | self.input_x = tf.placeholder(tf.int32, [None, sequence_size], name = "input_x") 62 | self.input_p_h = tf.placeholder(tf.int32, [None, sequence_size], name = "input_p_h") 63 | self.input_p_t = tf.placeholder(tf.int32, [None, sequence_size], name = "input_p_t") 64 | self.input_r = tf.placeholder(tf.int32, [1, 1], name = "input_r") 65 | self.input_r_n = tf.placeholder(tf.float32, [1, 1], name = "input_r_n") 66 | self.input_h = tf.placeholder(tf.int32, [1, 1], name = "input_t") 67 | self.input_t = tf.placeholder(tf.int32, [1, 1], name = "input_t") 68 | self.input_y = tf.placeholder(tf.float32, [1, num_classes], name = "input_y") 69 | 70 | self.pos_h = tf.placeholder(tf.int32, [None]) 71 | self.pos_t = tf.placeholder(tf.int32, [None]) 72 | self.pos_r = tf.placeholder(tf.int32, [None]) 73 | self.neg_h = tf.placeholder(tf.int32, [None]) 74 | self.neg_t = tf.placeholder(tf.int32, [None]) 75 | self.neg_r = tf.placeholder(tf.int32, [None]) 76 | 77 | l2_loss = tf.constant(0.0) 78 | with tf.name_scope("embedding-lookup"): 79 | self.word_embeddings = tf.Variable(word_embeddings, name="word_embeddings") 80 | self.relation_embeddings = tf.Variable(relation_embeddings, name="relation_embeddings") 81 | self.position_embeddings = tf.Variable(position_embeddings, name="position_embeddings") 82 | self.relation_attention = tf.Variable(relation_attention, name="relation_attention") 83 | self.NAattention = tf.Variable(NAattention, name="NAattention") 84 | self.attention = tf.Variable(attention, name="attention") 85 | self.ent_transfer = tf.Variable(ent_transfer, name = "ent_transfer") 86 | self.rel_transfer = tf.Variable(rel_transfer, name = "rel_transfer") 87 | 88 | #know 89 | pos_h_e = tf.nn.embedding_lookup(self.word_embeddings, self.pos_h) 90 | pos_t_e = tf.nn.embedding_lookup(self.word_embeddings, self.pos_t) 91 | pos_r_e = tf.nn.embedding_lookup(self.relation_embeddings, self.pos_r) 92 | pos_h_t = tf.nn.embedding_lookup(self.ent_transfer, self.pos_h) 93 | pos_t_t = tf.nn.embedding_lookup(self.ent_transfer, self.pos_t) 94 | pos_r_t = tf.nn.embedding_lookup(self.rel_transfer, self.pos_r) 95 | neg_h_e = tf.nn.embedding_lookup(self.word_embeddings, self.neg_h) 96 | neg_t_e = tf.nn.embedding_lookup(self.word_embeddings, self.neg_t) 97 | neg_r_e = tf.nn.embedding_lookup(self.relation_embeddings, self.neg_r) 98 | neg_h_t = tf.nn.embedding_lookup(self.ent_transfer, self.neg_h) 99 | neg_t_t = tf.nn.embedding_lookup(self.ent_transfer, self.neg_t) 100 | neg_r_t = tf.nn.embedding_lookup(self.rel_transfer, self.neg_r) 101 | pos_h_e = self.calc(pos_h_e, pos_h_t, pos_r_t) 102 | pos_t_e = self.calc(pos_t_e, pos_t_t, pos_r_t) 103 | neg_h_e = self.calc(neg_h_e, neg_h_t, neg_r_t) 104 | neg_t_e = self.calc(neg_t_e, neg_t_t, neg_r_t) 105 | 106 | #cnn 107 | self.x_initial = tf.nn.embedding_lookup(self.word_embeddings, self.input_x) 108 | self.x_p_h = tf.nn.embedding_lookup(self.position_embeddings, self.input_p_h) 109 | self.x_p_t = tf.nn.embedding_lookup(self.position_embeddings, self.input_p_t) 110 | self.x = tf.expand_dims(tf.concat(2, [self.x_initial, self.x_p_h, self.x_p_t]), -1) 111 | self.head = tf.nn.embedding_lookup(self.word_embeddings, self.input_h) 112 | self.tail = tf.nn.embedding_lookup(self.word_embeddings, self.input_t) 113 | self.head_t = tf.nn.embedding_lookup(self.ent_transfer, self.input_h) 114 | self.tail_t = tf.nn.embedding_lookup(self.ent_transfer, self.input_t) 115 | self.r_t = tf.nn.embedding_lookup(self.rel_transfer, self.input_r) 116 | self.head = self.calc(self.head, self.head_t, self.r_t) 117 | self.tail = self.calc(self.tail, self.tail_t, self.r_t) 118 | l2_loss += tf.nn.l2_loss(self.attention) 119 | 120 | with tf.name_scope("conv-maxpool"): 121 | self.W = tf.Variable(W, name="W") 122 | self.b = tf.Variable(B, name="b") 123 | conv = tf.nn.conv2d(self.x, self.W, strides=[1, 1, 1, 1], padding="VALID", name="conv") 124 | h = tf.nn.tanh(tf.nn.bias_add(conv, self.b), name="tanh") 125 | self.y = tf.nn.max_pool(h, ksize=[1, sequence_size - filter_size + 1, 1, 1], strides=[1, 1, 1, 1], padding='VALID', name="pool") 126 | l2_loss += tf.nn.l2_loss(self.W) 127 | l2_loss += tf.nn.l2_loss(self.b) 128 | self.y = tf.reshape(self.y, [-1, num_filters]) 129 | 130 | with tf.name_scope('attention'): 131 | self.y_attention = tf.reduce_max(self.y, 0 , keep_dims = True) 132 | 133 | with tf.name_scope("dropout"): 134 | self.transfer_w = tf.Variable(transfer_w, name="transfer_w") 135 | self.h_drop = tf.nn.l2_normalize(self.y_attention, 1) 136 | self.scores = tf.matmul(self.h_drop, self.transfer_w) 137 | self.scoress = tf.nn.softmax(tf.reshape(self.scores, [1, -1])) 138 | 139 | bags_sum = 0.0 140 | bags_hit_NA = 0.0 141 | sum_NA = 0.0 142 | sum_fNA = 0.0 143 | bags_hit = 0.0 144 | loss_sum = 0.0 145 | ss = [] 146 | flag = True 147 | 148 | if __name__ == "__main__": 149 | 150 | lib_cnn.readWordVec() 151 | lib_cnn.readFromFile() 152 | lib_kg.init() 153 | 154 | np.random.seed(0) 155 | tf.set_random_seed(0) 156 | config = Config() 157 | 158 | word_embeddings = np.zeros(config.num_words * config.word_size, dtype = np.float32) 159 | lib_cnn.getWordVec.argtypes = [ctypes.c_void_p] 160 | lib_cnn.getWordVec(word_embeddings.__array_interface__['data'][0]) 161 | word_embeddings.resize((config.num_words,config.word_size)) 162 | 163 | config.batch_size = lib_kg.getTripleTotal() / config.nbatches 164 | config.entityTotal = lib_kg.getEntityTotal() 165 | config.relationTotal = lib_kg.getRelationTotal() 166 | 167 | config.num_classes = 56 168 | log = open("log14.txt", "r") 169 | word_embeddings = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_words , config.word_size)) 170 | relation_embeddings = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.relationTotal, config.word_size)) 171 | position_embeddings = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_positions, config.position_size)) 172 | relation_attention = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_classes, config.relation_size)) 173 | attention = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_filters, config.relation_size)) 174 | W = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.filter_size, config.embedding_size, 1, config.num_filters)) 175 | B = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_filters)) 176 | transfer_w = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape(config.num_filters, config.num_classes) 177 | NAattention = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.relation_size, 1)) 178 | ent_transfer = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_words , config.word_size)) 179 | rel_transfer = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.relationTotal, config.word_size)) 180 | log.close() 181 | 182 | 183 | with tf.Graph().as_default(): 184 | conf = tf.ConfigProto() 185 | sess = tf.Session(config=conf) 186 | with sess.as_default(): 187 | initializer = tf.contrib.layers.xavier_initializer() 188 | with tf.variable_scope("model", reuse=None, initializer = initializer): 189 | m = Model(config = config) 190 | 191 | global_step_cnn = tf.Variable(0, name="global_step_cnn", trainable=False) 192 | sess.run(tf.initialize_all_variables()) 193 | 194 | x_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 195 | p_t_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 196 | p_h_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 197 | r_batch = np.zeros((1, 1), dtype = np.int32) 198 | y_batch = np.zeros((1, config.num_classes), dtype = np.int32) 199 | r_n_batch = np.zeros((1, 1), dtype = np.int32) 200 | h_batch = np.zeros((1, 1), dtype = np.int32) 201 | t_batch = np.zeros((1, 1), dtype = np.int32) 202 | 203 | x_batch_addr = x_batch.__array_interface__['data'][0] 204 | p_t_batch_addr = p_t_batch.__array_interface__['data'][0] 205 | p_h_batch_addr = p_h_batch.__array_interface__['data'][0] 206 | y_batch_addr = y_batch.__array_interface__['data'][0] 207 | r_batch_addr = r_batch.__array_interface__['data'][0] 208 | r_n_batch_addr = r_n_batch.__array_interface__['data'][0] 209 | h_batch_addr = h_batch.__array_interface__['data'][0] 210 | t_batch_addr = t_batch.__array_interface__['data'][0] 211 | lib_cnn.batch_iter.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p] 212 | tipTotal = lib_cnn.getTipTotal() 213 | 214 | def train_step_cnn(x_batch, p_h_batch, p_t_batch, y_batch, r_batch, r_n_batch, h_batch, t_batch,i): 215 | global ss, bags_sum, bags_hit, loss_sum, bags_hit_NA, bags_hit, sum_fNA, sum_NA, flag 216 | feed_dict = { 217 | m.input_x: x_batch, 218 | m.input_p_h: p_h_batch, 219 | m.input_p_t: p_t_batch, 220 | m.input_r: r_batch, 221 | m.input_r_n: r_n_batch, 222 | m.input_y: y_batch, 223 | m.input_h: h_batch, 224 | m.input_t: t_batch 225 | } 226 | scores, step, accuracy, accuracy1,scoresgg,scores1gg = sess.run( 227 | [m.scoress, global_step_cnn, m.accuracy,m.accuracy1,m.y_attention,m.h_drop], feed_dict) 228 | time_str = datetime.datetime.now().isoformat() 229 | for i in range(config.num_classes): 230 | if (r_batch[0] == i): 231 | ss.append((i,scores[0][i],1)) 232 | else: 233 | ss.append((i,scores[0][i],0)) 234 | if bags_sum % 100 == 0: 235 | print("{}: step {}".format(time_str, step)) 236 | 237 | for i in range(tipTotal): 238 | length = lib_cnn.batch_iter(x_batch_addr, p_h_batch_addr, p_t_batch_addr, y_batch_addr, r_batch_addr, r_n_batch_addr, h_batch_addr, t_batch_addr) 239 | train_step_cnn(x_batch[0:length,], p_h_batch[0:length,], p_t_batch[0:length,], y_batch, r_batch, r_n_batch, h_batch, t_batch,i) 240 | if not flag: 241 | break 242 | 243 | f = open("res.txt", "w") 244 | for i in ss: 245 | f.write("%d\t%f\t%d\n"%(i[0],i[1],i[2])) 246 | f.close() 247 | -------------------------------------------------------------------------------- /original/baselines/test/test_JointE+ATT.py: -------------------------------------------------------------------------------- 1 | #coding:utf-8 2 | import numpy as np 3 | import tensorflow as tf 4 | import os 5 | import time 6 | import datetime 7 | import ctypes 8 | import threading 9 | import json 10 | 11 | ll1 = ctypes.cdll.LoadLibrary 12 | lib_cnn = ll1("./init_cnn.so") 13 | ll2 = ctypes.cdll.LoadLibrary 14 | lib_kg = ll2("./init_know.so") 15 | 16 | class Config(object): 17 | def __init__(self): 18 | self.instanceTot = lib_cnn.getInstanceTot() 19 | self.sequence_size = lib_cnn.getLenLimit() 20 | self.num_classes = lib_cnn.getRelationTotal() 21 | self.num_words = lib_cnn.getWordTotal() 22 | self.num_positions = 2 * lib_cnn.getPositionLimit() + 1 23 | self.word_size = lib_cnn.getWordDimension() 24 | self.position_size = 5 25 | self.embedding_size = self.word_size + self.position_size * 2 26 | self.filter_size = 3 27 | self.num_filters = 230 28 | self.relation_size = self.word_size#230 29 | self.dropout_keep_prob = 0.5 30 | self.l2_lambda = 0.0001 31 | lib_cnn.setNA(51) 32 | self.margin = 1.0 33 | self.nbatches = 100 34 | self.trainTimes = 15 35 | self.entityTotal = 0 36 | self.relationTotal = 0 37 | 38 | class Model(object): 39 | 40 | def __init__(self, config): 41 | sequence_size = config.sequence_size 42 | num_classes = config.num_classes 43 | num_words = config.num_words 44 | num_positions = config.num_positions 45 | 46 | embedding_size = config.embedding_size 47 | word_size = config.word_size 48 | position_size = config.position_size 49 | relation_size = config.relation_size 50 | filter_size = config.filter_size 51 | num_filters = config.num_filters 52 | dropout_keep_prob = config.dropout_keep_prob 53 | 54 | margin = config.margin 55 | l2_lambda = config.l2_lambda 56 | 57 | self.input_x = tf.placeholder(tf.int32, [None, sequence_size], name = "input_x") 58 | self.input_p_h = tf.placeholder(tf.int32, [None, sequence_size], name = "input_p_h") 59 | self.input_p_t = tf.placeholder(tf.int32, [None, sequence_size], name = "input_p_t") 60 | self.input_r = tf.placeholder(tf.int32, [1, 1], name = "input_r") 61 | self.input_r_n = tf.placeholder(tf.float32, [1, 1], name = "input_r_n") 62 | self.input_h = tf.placeholder(tf.int32, [1, 1], name = "input_t") 63 | self.input_t = tf.placeholder(tf.int32, [1, 1], name = "input_t") 64 | self.input_y = tf.placeholder(tf.float32, [1, num_classes], name = "input_y") 65 | 66 | self.pos_h = tf.placeholder(tf.int32, [None]) 67 | self.pos_t = tf.placeholder(tf.int32, [None]) 68 | self.pos_r = tf.placeholder(tf.int32, [None]) 69 | self.neg_h = tf.placeholder(tf.int32, [None]) 70 | self.neg_t = tf.placeholder(tf.int32, [None]) 71 | self.neg_r = tf.placeholder(tf.int32, [None]) 72 | 73 | l2_loss = tf.constant(0.0) 74 | with tf.name_scope("embedding-lookup"): 75 | self.word_embeddings = tf.Variable(word_embeddings, name="word_embeddings") 76 | self.relation_embeddings = tf.Variable(relation_embeddings, name="relation_embeddings") 77 | self.position_embeddings = tf.Variable(position_embeddings, name="position_embeddings") 78 | self.relation_attention = tf.Variable(relation_attention, name="relation_attention") 79 | self.NAattention = tf.Variable(NAattention, name="NAattention") 80 | self.attention = tf.Variable(attention, name="attention") 81 | 82 | self.r = tf.nn.embedding_lookup(self.attention, self.input_r) 83 | 84 | #know 85 | pos_h_e = tf.nn.embedding_lookup(self.word_embeddings, self.pos_h) 86 | pos_t_e = tf.nn.embedding_lookup(self.word_embeddings, self.pos_t) 87 | pos_r_e = tf.nn.embedding_lookup(self.relation_embeddings, self.pos_r) 88 | neg_h_e = tf.nn.embedding_lookup(self.word_embeddings, self.neg_h) 89 | neg_t_e = tf.nn.embedding_lookup(self.word_embeddings, self.neg_t) 90 | neg_r_e = tf.nn.embedding_lookup(self.relation_embeddings, self.neg_r) 91 | 92 | #cnn 93 | self.x_initial = tf.nn.embedding_lookup(self.word_embeddings, self.input_x) 94 | self.x_p_h = tf.nn.embedding_lookup(self.position_embeddings, self.input_p_h) 95 | self.x_p_t = tf.nn.embedding_lookup(self.position_embeddings, self.input_p_t) 96 | self.x = tf.expand_dims(tf.concat(2, [self.x_initial, self.x_p_h, self.x_p_t]), -1) 97 | # self.r = tf.nn.embedding_lookup(self.relation_attention, self.input_r) 98 | self.head = tf.nn.embedding_lookup(self.word_embeddings, self.input_h) 99 | self.tail = tf.nn.embedding_lookup(self.word_embeddings, self.input_t) 100 | l2_loss += tf.nn.l2_loss(self.attention) 101 | 102 | with tf.name_scope("conv-maxpool"): 103 | self.W = tf.Variable(W, name="W") 104 | self.b = tf.Variable(B, name="b") 105 | conv = tf.nn.conv2d(self.x, self.W, strides=[1, 1, 1, 1], padding="VALID", name="conv") 106 | h = tf.nn.tanh(tf.nn.bias_add(conv, self.b), name="tanh") 107 | self.y = tf.nn.max_pool(h, ksize=[1, sequence_size - filter_size + 1, 1, 1], strides=[1, 1, 1, 1], padding='VALID', name="pool") 108 | l2_loss += tf.nn.l2_loss(self.W) 109 | l2_loss += tf.nn.l2_loss(self.b) 110 | self.y = tf.reshape(self.y, [-1, num_filters]) 111 | 112 | with tf.name_scope('attention'): 113 | self.r = tf.reshape(self.r, [relation_size, -1]) 114 | self.e = tf.matmul(tf.nn.tanh(tf.matmul(self.y, self.attention)), self.r) 115 | alpha = tf.reshape(self.e, [1, -1]) 116 | self.alpha_reshape = tf.nn.softmax(alpha) 117 | self.y_attention = tf.matmul(self.alpha_reshape, self.y) 118 | 119 | with tf.name_scope("dropout"): 120 | self.transfer_w = tf.Variable(transfer_w, name="transfer_w") 121 | self.h_drop = tf.nn.l2_normalize(self.y_attention, 1) 122 | self.scores = tf.matmul(self.h_drop, self.transfer_w) 123 | self.scoress = tf.nn.softmax(tf.reshape(self.scores, [1, -1])) 124 | 125 | bags_sum = 0.0 126 | bags_hit_NA = 0.0 127 | sum_NA = 0.0 128 | sum_fNA = 0.0 129 | bags_hit = 0.0 130 | loss_sum = 0.0 131 | ss = [] 132 | flag = True 133 | 134 | if __name__ == "__main__": 135 | 136 | lib_cnn.readWordVec() 137 | lib_cnn.readFromFile() 138 | lib_kg.init() 139 | 140 | np.random.seed(0) 141 | tf.set_random_seed(0) 142 | config = Config() 143 | 144 | word_embeddings = np.zeros(config.num_words * config.word_size, dtype = np.float32) 145 | lib_cnn.getWordVec.argtypes = [ctypes.c_void_p] 146 | lib_cnn.getWordVec(word_embeddings.__array_interface__['data'][0]) 147 | word_embeddings.resize((config.num_words,config.word_size)) 148 | 149 | config.batch_size = lib_kg.getTripleTotal() / config.nbatches 150 | config.entityTotal = lib_kg.getEntityTotal() 151 | config.relationTotal = lib_kg.getRelationTotal() 152 | 153 | config.num_classes = 56 154 | log = open("log14.txt", "r") 155 | word_embeddings = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_words , config.word_size)) 156 | relation_embeddings = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.relationTotal, config.word_size)) 157 | position_embeddings = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_positions, config.position_size)) 158 | relation_attention = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_classes, config.relation_size)) 159 | attention = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_filters, config.relation_size)) 160 | W = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.filter_size, config.embedding_size, 1, config.num_filters)) 161 | B = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_filters)) 162 | transfer_w = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape(config.num_filters, config.num_classes) 163 | NAattention = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.relation_size, 1)) 164 | log.close() 165 | 166 | 167 | with tf.Graph().as_default(): 168 | conf = tf.ConfigProto() 169 | sess = tf.Session(config=conf) 170 | with sess.as_default(): 171 | initializer = tf.contrib.layers.xavier_initializer() 172 | with tf.variable_scope("model", reuse=None, initializer = initializer): 173 | m = Model(config = config) 174 | 175 | global_step_cnn = tf.Variable(0, name="global_step_cnn", trainable=False) 176 | sess.run(tf.initialize_all_variables()) 177 | 178 | x_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 179 | p_t_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 180 | p_h_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 181 | r_batch = np.zeros((1, 1), dtype = np.int32) 182 | y_batch = np.zeros((1, config.num_classes), dtype = np.int32) 183 | r_n_batch = np.zeros((1, 1), dtype = np.int32) 184 | h_batch = np.zeros((1, 1), dtype = np.int32) 185 | t_batch = np.zeros((1, 1), dtype = np.int32) 186 | 187 | x_batch_addr = x_batch.__array_interface__['data'][0] 188 | p_t_batch_addr = p_t_batch.__array_interface__['data'][0] 189 | p_h_batch_addr = p_h_batch.__array_interface__['data'][0] 190 | y_batch_addr = y_batch.__array_interface__['data'][0] 191 | r_batch_addr = r_batch.__array_interface__['data'][0] 192 | r_n_batch_addr = r_n_batch.__array_interface__['data'][0] 193 | h_batch_addr = h_batch.__array_interface__['data'][0] 194 | t_batch_addr = t_batch.__array_interface__['data'][0] 195 | lib_cnn.batch_iter.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p] 196 | tipTotal = lib_cnn.getTipTotal() 197 | 198 | def train_step_cnn(x_batch, p_h_batch, p_t_batch, y_batch, r_batch, r_n_batch, h_batch, t_batch,i): 199 | global ss, bags_sum, bags_hit, loss_sum, bags_hit_NA, bags_hit, sum_fNA, sum_NA, flag 200 | feed_dict = { 201 | m.input_x: x_batch, 202 | m.input_p_h: p_h_batch, 203 | m.input_p_t: p_t_batch, 204 | m.input_r: r_batch, 205 | m.input_r_n: r_n_batch, 206 | m.input_y: y_batch, 207 | m.input_h: h_batch, 208 | m.input_t: t_batch 209 | } 210 | scores, step, accuracy, accuracy1,scoresgg,scores1gg = sess.run( 211 | [m.scoress, global_step_cnn, m.accuracy,m.accuracy1,m.y_attention,m.h_drop], feed_dict) 212 | time_str = datetime.datetime.now().isoformat() 213 | for i in range(config.num_classes): 214 | if (r_batch[0] == i): 215 | ss.append((i,scores[0][i],1)) 216 | else: 217 | ss.append((i,scores[0][i],0)) 218 | bags_sum += 1 219 | if bags_sum % 100 == 0: 220 | print("{}: step {}".format(time_str, step)) 221 | 222 | for i in range(tipTotal): 223 | length = lib_cnn.batch_iter(x_batch_addr, p_h_batch_addr, p_t_batch_addr, y_batch_addr, r_batch_addr, r_n_batch_addr, h_batch_addr, t_batch_addr) 224 | train_step_cnn(x_batch[0:length,], p_h_batch[0:length,], p_t_batch[0:length,], y_batch, r_batch, r_n_batch, h_batch, t_batch,i) 225 | if not flag: 226 | break 227 | 228 | f = open("res.txt", "w") 229 | for i in ss: 230 | f.write("%d\t%f\t%d\n"%(i[0],i[1],i[2])) 231 | f.close() 232 | -------------------------------------------------------------------------------- /original/baselines/test/test_JointE+ONE.py: -------------------------------------------------------------------------------- 1 | #coding:utf-8 2 | import numpy as np 3 | import tensorflow as tf 4 | import os 5 | import time 6 | import datetime 7 | import ctypes 8 | import threading 9 | import json 10 | 11 | ll1 = ctypes.cdll.LoadLibrary 12 | lib_cnn = ll1("./init_cnn.so") 13 | ll2 = ctypes.cdll.LoadLibrary 14 | lib_kg = ll2("./init_know.so") 15 | 16 | class Config(object): 17 | def __init__(self): 18 | self.instanceTot = lib_cnn.getInstanceTot() 19 | self.sequence_size = lib_cnn.getLenLimit() 20 | self.num_classes = lib_cnn.getRelationTotal() 21 | self.num_words = lib_cnn.getWordTotal() 22 | self.num_positions = 2 * lib_cnn.getPositionLimit() + 1 23 | self.word_size = lib_cnn.getWordDimension() 24 | self.position_size = 5 25 | self.embedding_size = self.word_size + self.position_size * 2 26 | self.filter_size = 3 27 | self.num_filters = 230 28 | self.relation_size = self.word_size 29 | self.dropout_keep_prob = 0.5 30 | self.l2_lambda = 0.0001 31 | lib_cnn.setNA(51) 32 | self.margin = 1.0 33 | self.nbatches = 100 34 | self.trainTimes = 15 35 | self.entityTotal = 0 36 | self.relationTotal = 0 37 | 38 | class Model(object): 39 | 40 | def __init__(self, config): 41 | sequence_size = config.sequence_size 42 | num_classes = config.num_classes 43 | num_words = config.num_words 44 | num_positions = config.num_positions 45 | 46 | embedding_size = config.embedding_size 47 | word_size = config.word_size 48 | position_size = config.position_size 49 | relation_size = config.relation_size 50 | filter_size = config.filter_size 51 | num_filters = config.num_filters 52 | dropout_keep_prob = config.dropout_keep_prob 53 | 54 | margin = config.margin 55 | l2_lambda = config.l2_lambda 56 | 57 | self.input_x = tf.placeholder(tf.int32, [None, sequence_size], name = "input_x") 58 | self.input_p_h = tf.placeholder(tf.int32, [None, sequence_size], name = "input_p_h") 59 | self.input_p_t = tf.placeholder(tf.int32, [None, sequence_size], name = "input_p_t") 60 | self.input_r = tf.placeholder(tf.int32, [1, 1], name = "input_r") 61 | self.input_r_n = tf.placeholder(tf.float32, [1, 1], name = "input_r_n") 62 | self.input_h = tf.placeholder(tf.int32, [1, 1], name = "input_t") 63 | self.input_t = tf.placeholder(tf.int32, [1, 1], name = "input_t") 64 | self.input_y = tf.placeholder(tf.float32, [1, num_classes], name = "input_y") 65 | 66 | self.pos_h = tf.placeholder(tf.int32, [None]) 67 | self.pos_t = tf.placeholder(tf.int32, [None]) 68 | self.pos_r = tf.placeholder(tf.int32, [None]) 69 | self.neg_h = tf.placeholder(tf.int32, [None]) 70 | self.neg_t = tf.placeholder(tf.int32, [None]) 71 | self.neg_r = tf.placeholder(tf.int32, [None]) 72 | 73 | l2_loss = tf.constant(0.0) 74 | with tf.name_scope("embedding-lookup"): 75 | self.word_embeddings = tf.Variable(word_embeddings, name="word_embeddings") 76 | self.relation_embeddings = tf.Variable(relation_embeddings, name="relation_embeddings") 77 | self.position_embeddings = tf.Variable(position_embeddings, name="position_embeddings") 78 | self.relation_attention = tf.Variable(relation_attention, name="relation_attention") 79 | self.NAattention = tf.Variable(NAattention, name="NAattention") 80 | self.attention = tf.Variable(attention, name="attention") 81 | 82 | #know 83 | pos_h_e = tf.nn.embedding_lookup(self.word_embeddings, self.pos_h) 84 | pos_t_e = tf.nn.embedding_lookup(self.word_embeddings, self.pos_t) 85 | pos_r_e = tf.nn.embedding_lookup(self.relation_embeddings, self.pos_r) 86 | neg_h_e = tf.nn.embedding_lookup(self.word_embeddings, self.neg_h) 87 | neg_t_e = tf.nn.embedding_lookup(self.word_embeddings, self.neg_t) 88 | neg_r_e = tf.nn.embedding_lookup(self.relation_embeddings, self.neg_r) 89 | 90 | #cnn 91 | self.x_initial = tf.nn.embedding_lookup(self.word_embeddings, self.input_x) 92 | self.x_p_h = tf.nn.embedding_lookup(self.position_embeddings, self.input_p_h) 93 | self.x_p_t = tf.nn.embedding_lookup(self.position_embeddings, self.input_p_t) 94 | self.x = tf.expand_dims(tf.concat(2, [self.x_initial, self.x_p_h, self.x_p_t]), -1) 95 | self.head = tf.nn.embedding_lookup(self.word_embeddings, self.input_h) 96 | self.tail = tf.nn.embedding_lookup(self.word_embeddings, self.input_t) 97 | l2_loss += tf.nn.l2_loss(self.attention) 98 | 99 | with tf.name_scope("conv-maxpool"): 100 | self.W = tf.Variable(W, name="W") 101 | self.b = tf.Variable(B, name="b") 102 | conv = tf.nn.conv2d(self.x, self.W, strides=[1, 1, 1, 1], padding="VALID", name="conv") 103 | h = tf.nn.tanh(tf.nn.bias_add(conv, self.b), name="tanh") 104 | self.y = tf.nn.max_pool(h, ksize=[1, sequence_size - filter_size + 1, 1, 1], strides=[1, 1, 1, 1], padding='VALID', name="pool") 105 | l2_loss += tf.nn.l2_loss(self.W) 106 | l2_loss += tf.nn.l2_loss(self.b) 107 | self.y = tf.reshape(self.y, [-1, num_filters]) 108 | 109 | with tf.name_scope('attention'): 110 | self.y_attention = tf.reduce_max(self.y, 0 , keep_dims = True) 111 | 112 | 113 | with tf.name_scope("dropout"): 114 | print self.y_attention.get_shape() 115 | self.transfer_w = tf.Variable(transfer_w, name="transfer_w") 116 | self.h_drop = tf.nn.l2_normalize(self.y_attention, 1) 117 | self.scores = tf.matmul(self.h_drop, self.transfer_w) 118 | self.scoress = tf.nn.softmax(tf.reshape(self.scores, [1, -1])) 119 | 120 | bags_sum = 0.0 121 | bags_hit_NA = 0.0 122 | sum_NA = 0.0 123 | sum_fNA = 0.0 124 | bags_hit = 0.0 125 | loss_sum = 0.0 126 | ss = [] 127 | flag = True 128 | 129 | if __name__ == "__main__": 130 | 131 | lib_cnn.readWordVec() 132 | lib_cnn.readFromFile() 133 | lib_kg.init() 134 | 135 | np.random.seed(0) 136 | tf.set_random_seed(0) 137 | config = Config() 138 | 139 | word_embeddings = np.zeros(config.num_words * config.word_size, dtype = np.float32) 140 | lib_cnn.getWordVec.argtypes = [ctypes.c_void_p] 141 | lib_cnn.getWordVec(word_embeddings.__array_interface__['data'][0]) 142 | word_embeddings.resize((config.num_words,config.word_size)) 143 | 144 | config.batch_size = lib_kg.getTripleTotal() / config.nbatches 145 | config.entityTotal = lib_kg.getEntityTotal() 146 | config.relationTotal = lib_kg.getRelationTotal() 147 | 148 | config.num_classes = 56 149 | log = open("log14.txt", "r") 150 | word_embeddings = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_words , config.word_size)) 151 | relation_embeddings = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.relationTotal, config.word_size)) 152 | position_embeddings = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_positions, config.position_size)) 153 | relation_attention = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_classes, config.relation_size)) 154 | attention = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_filters, config.relation_size)) 155 | W = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.filter_size, config.embedding_size, 1, config.num_filters)) 156 | B = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.num_filters)) 157 | transfer_w = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape(config.num_filters, config.num_classes) 158 | NAattention = np.array(json.loads(log.readline().strip()), dtype = np.float32).reshape((config.relation_size, 1)) 159 | log.close() 160 | 161 | 162 | with tf.Graph().as_default(): 163 | conf = tf.ConfigProto() 164 | sess = tf.Session(config=conf) 165 | with sess.as_default(): 166 | initializer = tf.contrib.layers.xavier_initializer() 167 | with tf.variable_scope("model", reuse=None, initializer = initializer): 168 | m = Model(config = config) 169 | 170 | global_step_cnn = tf.Variable(0, name="global_step_cnn", trainable=False) 171 | sess.run(tf.initialize_all_variables()) 172 | 173 | x_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 174 | p_t_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 175 | p_h_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 176 | r_batch = np.zeros((1, 1), dtype = np.int32) 177 | y_batch = np.zeros((1, config.num_classes), dtype = np.int32) 178 | r_n_batch = np.zeros((1, 1), dtype = np.int32) 179 | h_batch = np.zeros((1, 1), dtype = np.int32) 180 | t_batch = np.zeros((1, 1), dtype = np.int32) 181 | 182 | x_batch_addr = x_batch.__array_interface__['data'][0] 183 | p_t_batch_addr = p_t_batch.__array_interface__['data'][0] 184 | p_h_batch_addr = p_h_batch.__array_interface__['data'][0] 185 | y_batch_addr = y_batch.__array_interface__['data'][0] 186 | r_batch_addr = r_batch.__array_interface__['data'][0] 187 | r_n_batch_addr = r_n_batch.__array_interface__['data'][0] 188 | h_batch_addr = h_batch.__array_interface__['data'][0] 189 | t_batch_addr = t_batch.__array_interface__['data'][0] 190 | lib_cnn.batch_iter.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p] 191 | tipTotal = lib_cnn.getTipTotal() 192 | 193 | def train_step_cnn(x_batch, p_h_batch, p_t_batch, y_batch, r_batch, r_n_batch, h_batch, t_batch,i): 194 | global ss, bags_sum, bags_hit, loss_sum, bags_hit_NA, bags_hit, sum_fNA, sum_NA, flag 195 | feed_dict = { 196 | m.input_x: x_batch, 197 | m.input_p_h: p_h_batch, 198 | m.input_p_t: p_t_batch, 199 | m.input_r: r_batch, 200 | m.input_r_n: r_n_batch, 201 | m.input_y: y_batch, 202 | m.input_h: h_batch, 203 | m.input_t: t_batch 204 | } 205 | scores, step, accuracy, accuracy1,scoresgg,scores1gg = sess.run( 206 | [m.scoress, global_step_cnn, m.accuracy,m.accuracy1,m.y_attention,m.h_drop], feed_dict) 207 | time_str = datetime.datetime.now().isoformat() 208 | for i in range(config.num_classes): 209 | if (r_batch[0] == i): 210 | ss.append((i,scores[0][i],1)) 211 | else: 212 | ss.append((i,scores[0][i],0)) 213 | bags_sum += 1 214 | if bags_sum % 100 == 0: 215 | print("{}: step {}".format(time_str, step)) 216 | 217 | for i in range(tipTotal): 218 | length = lib_cnn.batch_iter(x_batch_addr, p_h_batch_addr, p_t_batch_addr, y_batch_addr, r_batch_addr, r_n_batch_addr, h_batch_addr, t_batch_addr) 219 | train_step_cnn(x_batch[0:length,], p_h_batch[0:length,], p_t_batch[0:length,], y_batch, r_batch, r_n_batch, h_batch, t_batch,i) 220 | if not flag: 221 | break 222 | 223 | f = open("res.txt", "w") 224 | for i in ss: 225 | f.write("%d\t%f\t%d\n"%(i[0],i[1],i[2])) 226 | f.close() 227 | -------------------------------------------------------------------------------- /original/baselines/train/JointD+ATT.py: -------------------------------------------------------------------------------- 1 | #coding:utf-8 2 | import numpy as np 3 | import tensorflow as tf 4 | import os 5 | import time 6 | import datetime 7 | import ctypes 8 | import threading 9 | import json 10 | 11 | ll1 = ctypes.cdll.LoadLibrary 12 | lib_cnn = ll1("./init_cnn.so") 13 | ll2 = ctypes.cdll.LoadLibrary 14 | lib_kg = ll2("./init_know.so") 15 | 16 | class Config(object): 17 | def __init__(self): 18 | self.instanceTot = lib_cnn.getInstanceTot() 19 | self.sequence_size = lib_cnn.getLenLimit() 20 | self.num_classes = lib_cnn.getRelationTotal() 21 | self.num_words = lib_cnn.getWordTotal() 22 | self.num_positions = 2 * lib_cnn.getPositionLimit() + 1 23 | self.word_size = lib_cnn.getWordDimension() 24 | self.position_size = 5 25 | self.embedding_size = self.word_size + self.position_size * 2 26 | self.filter_size = 3 27 | self.num_filters = 230 28 | self.relation_size = self.word_size 29 | self.dropout_keep_prob = 0.5 30 | self.l2_lambda = 0.0001 31 | self.NA = 51 32 | lib_cnn.setNA(self.NA) 33 | lib_cnn.setRate(3) 34 | self.margin = 1.0 35 | self.nbatches = 100 36 | self.trainTimes = 15 37 | self.entityTotal = 0 38 | self.relationTotal = 0 39 | 40 | class Model(object): 41 | 42 | def calc(self, e, t, r): 43 | return e + tf.reduce_sum(e * t, 1, keep_dims = True) * r 44 | 45 | 46 | def __init__(self, config): 47 | sequence_size = config.sequence_size 48 | num_classes = config.num_classes 49 | num_words = config.num_words 50 | num_positions = config.num_positions 51 | 52 | embedding_size = config.embedding_size 53 | word_size = config.word_size 54 | position_size = config.position_size 55 | relation_size = config.relation_size 56 | filter_size = config.filter_size 57 | num_filters = config.num_filters 58 | dropout_keep_prob = config.dropout_keep_prob 59 | 60 | margin = config.margin 61 | l2_lambda = config.l2_lambda 62 | 63 | self.input_x = tf.placeholder(tf.int32, [None, sequence_size], name = "input_x") 64 | self.input_p_h = tf.placeholder(tf.int32, [None, sequence_size], name = "input_p_h") 65 | self.input_p_t = tf.placeholder(tf.int32, [None, sequence_size], name = "input_p_t") 66 | self.input_r = tf.placeholder(tf.int32, [1, 1], name = "input_r") 67 | self.input_r_n = tf.placeholder(tf.float32, [1, 1], name = "input_r_n") 68 | self.input_h = tf.placeholder(tf.int32, [1, 1], name = "input_h") 69 | self.input_t = tf.placeholder(tf.int32, [1, 1], name = "input_t") 70 | self.input_y = tf.placeholder(tf.float32, [1, num_classes], name = "input_y") 71 | 72 | self.pos_h = tf.placeholder(tf.int32, [None]) 73 | self.pos_t = tf.placeholder(tf.int32, [None]) 74 | self.pos_r = tf.placeholder(tf.int32, [None]) 75 | self.neg_h = tf.placeholder(tf.int32, [None]) 76 | self.neg_t = tf.placeholder(tf.int32, [None]) 77 | self.neg_r = tf.placeholder(tf.int32, [None]) 78 | 79 | l2_loss = tf.constant(0.0) 80 | with tf.name_scope("embedding-lookup"): 81 | self.word_embeddings = tf.Variable(word_embeddings, name="word_embeddings") 82 | self.relation_embeddings = tf.get_variable("relation_embeddings", [config.relationTotal, word_size]) 83 | self.position_embeddings = tf.get_variable("position_embeddings", [num_positions, position_size]) 84 | self.relation_attention = tf.get_variable("relation_attention", [num_classes, relation_size]) 85 | self.NAattention = tf.get_variable("NAattention", [relation_size, 1]) 86 | self.attention = tf.get_variable("attention", [num_filters, relation_size]) 87 | self.ent_transfer = tf.get_variable("ent_transfer", shape = [len(word_embeddings), word_size]) 88 | self.rel_transfer = tf.get_variable("rel_transfer", shape = [config.relationTotal, word_size]) 89 | 90 | self.r = tf.nn.embedding_lookup(self.attention, self.input_r) 91 | 92 | #know 93 | pos_h_e = tf.nn.embedding_lookup(self.word_embeddings, self.pos_h) 94 | pos_t_e = tf.nn.embedding_lookup(self.word_embeddings, self.pos_t) 95 | pos_r_e = tf.nn.embedding_lookup(self.relation_embeddings, self.pos_r) 96 | pos_h_t = tf.nn.embedding_lookup(self.ent_transfer, self.pos_h) 97 | pos_t_t = tf.nn.embedding_lookup(self.ent_transfer, self.pos_t) 98 | pos_r_t = tf.nn.embedding_lookup(self.rel_transfer, self.pos_r) 99 | neg_h_e = tf.nn.embedding_lookup(self.word_embeddings, self.neg_h) 100 | neg_t_e = tf.nn.embedding_lookup(self.word_embeddings, self.neg_t) 101 | neg_r_e = tf.nn.embedding_lookup(self.relation_embeddings, self.neg_r) 102 | neg_h_t = tf.nn.embedding_lookup(self.ent_transfer, self.neg_h) 103 | neg_t_t = tf.nn.embedding_lookup(self.ent_transfer, self.neg_t) 104 | neg_r_t = tf.nn.embedding_lookup(self.rel_transfer, self.neg_r) 105 | pos_h_e = self.calc(pos_h_e, pos_h_t, pos_r_t) 106 | pos_t_e = self.calc(pos_t_e, pos_t_t, pos_r_t) 107 | neg_h_e = self.calc(neg_h_e, neg_h_t, neg_r_t) 108 | neg_t_e = self.calc(neg_t_e, neg_t_t, neg_r_t) 109 | 110 | #cnn 111 | self.x_initial = tf.nn.embedding_lookup(self.word_embeddings, self.input_x) 112 | self.x_p_h = tf.nn.embedding_lookup(self.position_embeddings, self.input_p_h) 113 | self.x_p_t = tf.nn.embedding_lookup(self.position_embeddings, self.input_p_t) 114 | self.x = tf.expand_dims(tf.concat(2, [self.x_initial, self.x_p_h, self.x_p_t]), -1) 115 | self.head = tf.nn.embedding_lookup(self.word_embeddings, self.input_h) 116 | self.tail = tf.nn.embedding_lookup(self.word_embeddings, self.input_t) 117 | self.head_t = tf.nn.embedding_lookup(self.ent_transfer, self.input_h) 118 | self.tail_t = tf.nn.embedding_lookup(self.ent_transfer, self.input_t) 119 | self.r_t = tf.nn.embedding_lookup(self.rel_transfer, self.input_r) 120 | self.head = self.calc(self.head, self.head_t, self.r_t) 121 | self.tail = self.calc(self.tail, self.tail_t, self.r_t) 122 | l2_loss += tf.nn.l2_loss(self.attention) 123 | 124 | with tf.name_scope("conv-maxpool"): 125 | self.W = tf.get_variable("W", [filter_size, embedding_size, 1, num_filters]) 126 | self.b = tf.get_variable("b", [num_filters]) 127 | conv = tf.nn.conv2d(self.x, self.W, strides=[1, 1, 1, 1], padding="VALID", name="conv") 128 | h = tf.nn.tanh(tf.nn.bias_add(conv, self.b), name="tanh") 129 | self.y = tf.nn.max_pool(h, ksize=[1, sequence_size - filter_size + 1, 1, 1], strides=[1, 1, 1, 1], padding='VALID', name="pool") 130 | l2_loss += tf.nn.l2_loss(self.W) 131 | l2_loss += tf.nn.l2_loss(self.b) 132 | self.y = tf.reshape(self.y, [-1, num_filters]) 133 | 134 | with tf.name_scope('attention'): 135 | self.r = tf.reshape(self.r, [relation_size, -1]) 136 | self.e = tf.matmul(tf.matmul(self.y, self.attention), self.r) 137 | alpha = tf.reshape(self.e, [1, -1]) 138 | self.alpha_reshape = tf.nn.softmax(alpha) 139 | self.y_attention = tf.matmul(self.alpha_reshape, self.y) 140 | 141 | with tf.name_scope("dropout"): 142 | self.y_attention = tf.nn.l2_normalize(self.y_attention, 1) 143 | self.h_drop = tf.nn.dropout(self.y_attention, dropout_keep_prob) 144 | self.transfer_w = tf.get_variable("transfer_w", [num_filters, num_classes]) 145 | self.scores = tf.matmul(self.h_drop, self.transfer_w) 146 | l2_loss += tf.nn.l2_loss(self.transfer_w) 147 | 148 | with tf.name_scope("loss"): 149 | cross_entropy = tf.nn.softmax_cross_entropy_with_logits(self.scores, self.input_y) 150 | self.loss_cnn = tf.reduce_mean(cross_entropy) + l2_lambda * l2_loss 151 | 152 | self.pos = tf.reduce_sum(abs(pos_h_e + pos_r_e - pos_t_e), 1, keep_dims = True) 153 | self.neg = tf.reduce_sum(abs(neg_h_e + neg_r_e - neg_t_e), 1, keep_dims = True) 154 | self.loss_kg = tf.reduce_sum(tf.maximum(self.pos - self.neg + margin, 0)) 155 | 156 | with tf.name_scope("accuracy"): 157 | self.predictions = tf.argmax(self.scores, 1, name="predictions") 158 | correct_predictions = tf.equal(self.predictions, tf.argmax(self.input_y, 1)) 159 | self.accuracy = tf.reduce_mean(tf.cast(correct_predictions, "float"), name="accuracy") 160 | 161 | 162 | bags_sum = 0.0 163 | bags_hit_NA = 0.0 164 | sum_NA = 0.0 165 | sum_fNA = 0.0 166 | bags_hit = 0.0 167 | loss_sum = 0.0 168 | 169 | if __name__ == "__main__": 170 | 171 | lib_cnn.readWordVec() 172 | lib_cnn.readFromFile() 173 | lib_kg.init() 174 | 175 | np.random.seed(0) 176 | tf.set_random_seed(0) 177 | config = Config() 178 | 179 | word_embeddings = np.zeros(config.num_words * config.word_size, dtype = np.float32) 180 | lib_cnn.getWordVec.argtypes = [ctypes.c_void_p] 181 | lib_cnn.getWordVec(word_embeddings.__array_interface__['data'][0]) 182 | word_embeddings.resize((config.num_words,config.word_size)) 183 | 184 | config.batch_size = lib_kg.getTripleTotal() / config.nbatches 185 | config.entityTotal = lib_kg.getEntityTotal() 186 | config.relationTotal = lib_kg.getRelationTotal() 187 | 188 | with tf.Graph().as_default(): 189 | conf = tf.ConfigProto() 190 | sess = tf.Session(config=conf) 191 | with sess.as_default(): 192 | initializer = tf.contrib.layers.xavier_initializer() 193 | with tf.variable_scope("model", reuse=None, initializer = initializer): 194 | m = Model(config = config) 195 | 196 | global_step_cnn = tf.Variable(0, name="global_step_cnn", trainable=False) 197 | 198 | optimizer_cnn = tf.train.GradientDescentOptimizer(0.01) 199 | grads_and_vars_cnn = optimizer_cnn.compute_gradients(m.loss_cnn) 200 | train_op_cnn = optimizer_cnn.apply_gradients(grads_and_vars_cnn, global_step = global_step_cnn) 201 | 202 | global_step_kg = tf.Variable(0, name="global_step_kg", trainable=False) 203 | optimizer_kg = tf.train.GradientDescentOptimizer(0.001) 204 | grads_and_vars_kg = optimizer_kg.compute_gradients(m.loss_kg) 205 | train_op_kg = optimizer_kg.apply_gradients(grads_and_vars_kg, global_step=global_step_kg) 206 | 207 | sess.run(tf.initialize_all_variables()) 208 | 209 | def outEmbedding(str1): 210 | word_embeddings, relation_embeddings, position_embeddings, relation_attention, attention, W, B, transfer_w, transfer_b, softmax_w, softmax_b = sess.run([m.word_embeddings, m.relation_embeddings, m.position_embeddings, m.relation_attention, m.attention, m.W, m.b, m.transfer_w, m.transfer_b, m.softmax_w, m.softmax_b]) 211 | log = open("log"+str1+".txt", "w") 212 | log.write(json.dumps(word_embeddings.tolist())+"\n") 213 | log.write(json.dumps(relation_embeddings.tolist())+"\n") 214 | log.write(json.dumps(position_embeddings.tolist())+"\n") 215 | log.write(json.dumps(relation_attention.tolist())+"\n") 216 | log.write(json.dumps(attention.tolist())+"\n") 217 | log.write(json.dumps(W.tolist())+"\n") 218 | log.write(json.dumps(B.tolist())+"\n") 219 | log.write(json.dumps(transfer_w.tolist())+"\n") 220 | NAattention = sess.run(m.NAattention) 221 | log.write(json.dumps(NAattention.tolist()) + "\n") 222 | ent_transfer = sess.run(m.ent_transfer) 223 | log.write(json.dumps(ent_transfer.tolist()) + "\n") 224 | rel_transfer = sess.run(m.rel_transfer) 225 | log.write(json.dumps(rel_transfer.tolist()) + "\n") 226 | log.close() 227 | 228 | x_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 229 | p_t_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 230 | p_h_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 231 | r_batch = np.zeros((1, 1), dtype = np.int32) 232 | y_batch = np.zeros((1, config.num_classes), dtype = np.int32) 233 | r_n_batch = np.zeros((1, 1), dtype = np.float32) 234 | h_batch = np.zeros((1, 1), dtype = np.int32) 235 | t_batch = np.zeros((1, 1), dtype = np.int32) 236 | 237 | x_batch_addr = x_batch.__array_interface__['data'][0] 238 | p_t_batch_addr = p_t_batch.__array_interface__['data'][0] 239 | p_h_batch_addr = p_h_batch.__array_interface__['data'][0] 240 | y_batch_addr = y_batch.__array_interface__['data'][0] 241 | r_batch_addr = r_batch.__array_interface__['data'][0] 242 | r_n_batch_addr = r_n_batch.__array_interface__['data'][0] 243 | h_batch_addr = h_batch.__array_interface__['data'][0] 244 | t_batch_addr = t_batch.__array_interface__['data'][0] 245 | lib_cnn.batch_iter.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p] 246 | tipTotal = lib_cnn.getTipTotal() 247 | loop = 0 248 | 249 | def train_cnn(coord): 250 | def train_step_cnn(x_batch, p_h_batch, p_t_batch, y_batch, r_batch, r_n_batch, h_batch, t_batch): 251 | global bags_sum, bags_hit, loss_sum, bags_hit_NA, bags_hit, sum_fNA, sum_NA 252 | feed_dict = { 253 | m.input_x: x_batch, 254 | m.input_p_h: p_h_batch, 255 | m.input_p_t: p_t_batch, 256 | m.input_r: r_batch, 257 | m.input_r_n: r_n_batch, 258 | m.input_y: y_batch, 259 | m.input_h: h_batch, 260 | m.input_t: t_batch 261 | } 262 | _, step, loss, accuracy = sess.run( 263 | [train_op_cnn, global_step_cnn, m.loss_cnn, m.accuracy], feed_dict) 264 | time_str = datetime.datetime.now().isoformat() 265 | loss_sum += loss 266 | bags_sum += 1 267 | if (r_batch[0]!=config.NA): 268 | sum_fNA += 1 269 | if accuracy > 0.5: 270 | bags_hit += 1.0 271 | else: 272 | sum_NA += 1 273 | if accuracy > 0.5: 274 | bags_hit_NA += 1.0 275 | if bags_sum % 1000 == 0: 276 | if (sum_NA == 0): 277 | sum_NA+=1 278 | if (sum_fNA == 0): 279 | sum_fNA+=1 280 | print("{}: step {}, loss {:g}, acc {:g} acc {:g} {} {}".format(time_str, step, loss_sum/bags_sum, bags_hit_NA/sum_NA, bags_hit/sum_fNA, sum_NA, sum_fNA)) 281 | 282 | 283 | global loop 284 | while not coord.should_stop(): 285 | print 'Looping ', loop 286 | outEmbedding(str(loop)) 287 | for i in range(tipTotal): 288 | length = lib_cnn.batch_iter(x_batch_addr, p_h_batch_addr, p_t_batch_addr, y_batch_addr, r_batch_addr, r_n_batch_addr, h_batch_addr, t_batch_addr) 289 | train_step_cnn(x_batch[0:length,], p_h_batch[0:length,], p_t_batch[0:length,], y_batch, r_batch, r_n_batch, h_batch, t_batch) 290 | global bags_sum, bags_hit, loss_sum, bags_hit_NA, bags_hit, sum_fNA, sum_NA 291 | bags_sum = 0 292 | bags_hit = 0 293 | bags_hit_NA = 0 294 | loss_sum = 0 295 | sum_fNA = 0 296 | sum_NA = 0 297 | loop += 1 298 | if loop == config.trainTimes: 299 | coord.request_stop() 300 | 301 | ph = np.zeros(config.batch_size, dtype = np.int32) 302 | pt = np.zeros(config.batch_size, dtype = np.int32) 303 | pr = np.zeros(config.batch_size, dtype = np.int32) 304 | nh = np.zeros(config.batch_size, dtype = np.int32) 305 | nt = np.zeros(config.batch_size, dtype = np.int32) 306 | nr = np.zeros(config.batch_size, dtype = np.int32) 307 | ph_addr = ph.__array_interface__['data'][0] 308 | pt_addr = pt.__array_interface__['data'][0] 309 | pr_addr = pr.__array_interface__['data'][0] 310 | nh_addr = nh.__array_interface__['data'][0] 311 | nt_addr = nt.__array_interface__['data'][0] 312 | nr_addr = nr.__array_interface__['data'][0] 313 | lib_kg.getBatch.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_int] 314 | times_kg = 0 315 | 316 | def train_kg(coord): 317 | def train_step_kg(pos_h_batch, pos_t_batch, pos_r_batch, neg_h_batch, neg_t_batch, neg_r_batch): 318 | feed_dict = { 319 | m.pos_h: pos_h_batch, 320 | m.pos_t: pos_t_batch, 321 | m.pos_r: pos_r_batch, 322 | m.neg_h: neg_h_batch, 323 | m.neg_t: neg_t_batch, 324 | m.neg_r: neg_r_batch 325 | } 326 | _, step, loss, = sess.run( 327 | [train_op_kg, global_step_kg, m.loss_kg], feed_dict) 328 | return loss 329 | global times_kg 330 | while not coord.should_stop(): 331 | times_kg += 1 332 | res = 0.0 333 | for batch in range(config.nbatches): 334 | lib_kg.getBatch(ph_addr, pt_addr, pr_addr, nh_addr, nt_addr, nr_addr, config.batch_size) 335 | res += train_step_kg(ph, pt, pr, nh, nt, nr) 336 | 337 | coord = tf.train.Coordinator() 338 | threads = [] 339 | threads.append(threading.Thread(target=train_kg, args=(coord,))) 340 | threads.append(threading.Thread(target=train_cnn, args=(coord,))) 341 | for t in threads: t.start() 342 | coord.join(threads) 343 | 344 | 345 | 346 | -------------------------------------------------------------------------------- /original/baselines/train/JointD+ONE.py: -------------------------------------------------------------------------------- 1 | #coding:utf-8 2 | import numpy as np 3 | import tensorflow as tf 4 | import os 5 | import time 6 | import datetime 7 | import ctypes 8 | import threading 9 | import json 10 | 11 | ll1 = ctypes.cdll.LoadLibrary 12 | lib_cnn = ll1("./init_cnn.so") 13 | ll2 = ctypes.cdll.LoadLibrary 14 | lib_kg = ll2("./init_know.so") 15 | 16 | class Config(object): 17 | def __init__(self): 18 | self.instanceTot = lib_cnn.getInstanceTot() 19 | self.sequence_size = lib_cnn.getLenLimit() 20 | self.num_classes = lib_cnn.getRelationTotal() 21 | self.num_words = lib_cnn.getWordTotal() 22 | self.num_positions = 2 * lib_cnn.getPositionLimit() + 1 23 | self.word_size = lib_cnn.getWordDimension() 24 | self.position_size = 5 25 | self.embedding_size = self.word_size + self.position_size * 2 26 | self.filter_size = 3 27 | self.num_filters = 230 28 | self.relation_size = self.word_size 29 | self.dropout_keep_prob = 0.5 30 | self.l2_lambda = 0.0001 31 | self.NA = 51 32 | lib_cnn.setNA(self.NA) 33 | lib_cnn.setRate(3) 34 | self.margin = 1.0 35 | self.nbatches = 100 36 | self.trainTimes = 15 37 | self.entityTotal = 0 38 | self.relationTotal = 0 39 | 40 | class Model(object): 41 | 42 | def calc(self, e, t, r): 43 | return e + tf.reduce_sum(e * t, 1, keep_dims = True) * r 44 | 45 | 46 | def __init__(self, config): 47 | sequence_size = config.sequence_size 48 | num_classes = config.num_classes 49 | num_words = config.num_words 50 | num_positions = config.num_positions 51 | 52 | embedding_size = config.embedding_size 53 | word_size = config.word_size 54 | position_size = config.position_size 55 | relation_size = config.relation_size 56 | filter_size = config.filter_size 57 | num_filters = config.num_filters 58 | dropout_keep_prob = config.dropout_keep_prob 59 | 60 | margin = config.margin 61 | l2_lambda = config.l2_lambda 62 | 63 | self.input_x = tf.placeholder(tf.int32, [None, sequence_size], name = "input_x") 64 | self.input_p_h = tf.placeholder(tf.int32, [None, sequence_size], name = "input_p_h") 65 | self.input_p_t = tf.placeholder(tf.int32, [None, sequence_size], name = "input_p_t") 66 | self.input_r = tf.placeholder(tf.int32, [1, 1], name = "input_r") 67 | self.input_r_n = tf.placeholder(tf.float32, [1, 1], name = "input_r_n") 68 | self.input_h = tf.placeholder(tf.int32, [1, 1], name = "input_h") 69 | self.input_t = tf.placeholder(tf.int32, [1, 1], name = "input_t") 70 | self.input_y = tf.placeholder(tf.float32, [1, num_classes], name = "input_y") 71 | 72 | self.pos_h = tf.placeholder(tf.int32, [None]) 73 | self.pos_t = tf.placeholder(tf.int32, [None]) 74 | self.pos_r = tf.placeholder(tf.int32, [None]) 75 | self.neg_h = tf.placeholder(tf.int32, [None]) 76 | self.neg_t = tf.placeholder(tf.int32, [None]) 77 | self.neg_r = tf.placeholder(tf.int32, [None]) 78 | 79 | l2_loss = tf.constant(0.0) 80 | with tf.name_scope("embedding-lookup"): 81 | self.word_embeddings = tf.Variable(word_embeddings, name="word_embeddings") 82 | self.relation_embeddings = tf.get_variable("relation_embeddings", [config.relationTotal, word_size]) 83 | self.position_embeddings = tf.get_variable("position_embeddings", [num_positions, position_size]) 84 | self.relation_attention = tf.get_variable("relation_attention", [num_classes, relation_size]) 85 | self.NAattention = tf.get_variable("NAattention", [relation_size, 1]) 86 | self.attention = tf.get_variable("attention", [num_filters, relation_size]) 87 | self.ent_transfer = tf.get_variable("ent_transfer", shape = [len(word_embeddings), word_size]) 88 | self.rel_transfer = tf.get_variable("rel_transfer", shape = [config.relationTotal, word_size]) 89 | 90 | #know 91 | pos_h_e = tf.nn.embedding_lookup(self.word_embeddings, self.pos_h) 92 | pos_t_e = tf.nn.embedding_lookup(self.word_embeddings, self.pos_t) 93 | pos_r_e = tf.nn.embedding_lookup(self.relation_embeddings, self.pos_r) 94 | pos_h_t = tf.nn.embedding_lookup(self.ent_transfer, self.pos_h) 95 | pos_t_t = tf.nn.embedding_lookup(self.ent_transfer, self.pos_t) 96 | pos_r_t = tf.nn.embedding_lookup(self.rel_transfer, self.pos_r) 97 | neg_h_e = tf.nn.embedding_lookup(self.word_embeddings, self.neg_h) 98 | neg_t_e = tf.nn.embedding_lookup(self.word_embeddings, self.neg_t) 99 | neg_r_e = tf.nn.embedding_lookup(self.relation_embeddings, self.neg_r) 100 | neg_h_t = tf.nn.embedding_lookup(self.ent_transfer, self.neg_h) 101 | neg_t_t = tf.nn.embedding_lookup(self.ent_transfer, self.neg_t) 102 | neg_r_t = tf.nn.embedding_lookup(self.rel_transfer, self.neg_r) 103 | pos_h_e = self.calc(pos_h_e, pos_h_t, pos_r_t) 104 | pos_t_e = self.calc(pos_t_e, pos_t_t, pos_r_t) 105 | neg_h_e = self.calc(neg_h_e, neg_h_t, neg_r_t) 106 | neg_t_e = self.calc(neg_t_e, neg_t_t, neg_r_t) 107 | 108 | #cnn 109 | self.x_initial = tf.nn.embedding_lookup(self.word_embeddings, self.input_x) 110 | self.x_p_h = tf.nn.embedding_lookup(self.position_embeddings, self.input_p_h) 111 | self.x_p_t = tf.nn.embedding_lookup(self.position_embeddings, self.input_p_t) 112 | self.x = tf.expand_dims(tf.concat(2, [self.x_initial, self.x_p_h, self.x_p_t]), -1) 113 | self.head = tf.nn.embedding_lookup(self.word_embeddings, self.input_h) 114 | self.tail = tf.nn.embedding_lookup(self.word_embeddings, self.input_t) 115 | self.head_t = tf.nn.embedding_lookup(self.ent_transfer, self.input_h) 116 | self.tail_t = tf.nn.embedding_lookup(self.ent_transfer, self.input_t) 117 | self.r_t = tf.nn.embedding_lookup(self.rel_transfer, self.input_r) 118 | self.head = self.calc(self.head, self.head_t, self.r_t) 119 | self.tail = self.calc(self.tail, self.tail_t, self.r_t) 120 | l2_loss += tf.nn.l2_loss(self.attention) 121 | 122 | with tf.name_scope("conv-maxpool"): 123 | self.W = tf.get_variable("W", [filter_size, embedding_size, 1, num_filters]) 124 | self.b = tf.get_variable("b", [num_filters]) 125 | conv = tf.nn.conv2d(self.x, self.W, strides=[1, 1, 1, 1], padding="VALID", name="conv") 126 | h = tf.nn.tanh(tf.nn.bias_add(conv, self.b), name="tanh") 127 | self.y = tf.nn.max_pool(h, ksize=[1, sequence_size - filter_size + 1, 1, 1], strides=[1, 1, 1, 1], padding='VALID', name="pool") 128 | l2_loss += tf.nn.l2_loss(self.W) 129 | l2_loss += tf.nn.l2_loss(self.b) 130 | self.y = tf.reshape(self.y, [-1, num_filters]) 131 | 132 | with tf.name_scope('attention'): 133 | self.y_attention = tf.reduce_max(self.y, 0 , keep_dims = True) 134 | 135 | with tf.name_scope("dropout"): 136 | self.y_attention = tf.nn.l2_normalize(self.y_attention, 1) 137 | self.h_drop = tf.nn.dropout(self.y_attention, dropout_keep_prob) 138 | self.transfer_w = tf.get_variable("transfer_w", [num_filters, num_classes]) 139 | self.scores = tf.matmul(self.h_drop, self.transfer_w) 140 | l2_loss += tf.nn.l2_loss(self.transfer_w) 141 | 142 | with tf.name_scope("loss"): 143 | cross_entropy = tf.nn.softmax_cross_entropy_with_logits(self.scores, self.input_y) 144 | self.loss_cnn = tf.reduce_mean(cross_entropy) + l2_lambda * l2_loss 145 | 146 | self.pos = tf.reduce_sum(abs(pos_h_e + pos_r_e - pos_t_e), 1, keep_dims = True) 147 | self.neg = tf.reduce_sum(abs(neg_h_e + neg_r_e - neg_t_e), 1, keep_dims = True) 148 | self.loss_kg = tf.reduce_sum(tf.maximum(self.pos - self.neg + margin, 0)) 149 | 150 | with tf.name_scope("accuracy"): 151 | self.predictions = tf.argmax(self.scores, 1, name="predictions") 152 | correct_predictions = tf.equal(self.predictions, tf.argmax(self.input_y, 1)) 153 | self.accuracy = tf.reduce_mean(tf.cast(correct_predictions, "float"), name="accuracy") 154 | 155 | 156 | bags_sum = 0.0 157 | bags_hit_NA = 0.0 158 | sum_NA = 0.0 159 | sum_fNA = 0.0 160 | bags_hit = 0.0 161 | loss_sum = 0.0 162 | 163 | if __name__ == "__main__": 164 | 165 | lib_cnn.readWordVec() 166 | lib_cnn.readFromFile() 167 | lib_kg.init() 168 | 169 | np.random.seed(0) 170 | tf.set_random_seed(0) 171 | config = Config() 172 | 173 | word_embeddings = np.zeros(config.num_words * config.word_size, dtype = np.float32) 174 | lib_cnn.getWordVec.argtypes = [ctypes.c_void_p] 175 | lib_cnn.getWordVec(word_embeddings.__array_interface__['data'][0]) 176 | word_embeddings.resize((config.num_words,config.word_size)) 177 | 178 | config.batch_size = lib_kg.getTripleTotal() / config.nbatches 179 | config.entityTotal = lib_kg.getEntityTotal() 180 | config.relationTotal = lib_kg.getRelationTotal() 181 | 182 | with tf.Graph().as_default(): 183 | conf = tf.ConfigProto() 184 | sess = tf.Session(config=conf) 185 | with sess.as_default(): 186 | initializer = tf.contrib.layers.xavier_initializer() 187 | with tf.variable_scope("model", reuse=None, initializer = initializer): 188 | m = Model(config = config) 189 | 190 | global_step_cnn = tf.Variable(0, name="global_step_cnn", trainable=False) 191 | 192 | optimizer_cnn = tf.train.GradientDescentOptimizer(0.01) 193 | grads_and_vars_cnn = optimizer_cnn.compute_gradients(m.loss_cnn) 194 | train_op_cnn = optimizer_cnn.apply_gradients(grads_and_vars_cnn, global_step = global_step_cnn) 195 | 196 | global_step_kg = tf.Variable(0, name="global_step_kg", trainable=False) 197 | optimizer_kg = tf.train.GradientDescentOptimizer(0.001) 198 | grads_and_vars_kg = optimizer_kg.compute_gradients(m.loss_kg) 199 | train_op_kg = optimizer_kg.apply_gradients(grads_and_vars_kg, global_step=global_step_kg) 200 | 201 | sess.run(tf.initialize_all_variables()) 202 | 203 | def outEmbedding(str1): 204 | word_embeddings, relation_embeddings, position_embeddings, relation_attention, attention, W, B, transfer_w, transfer_b, softmax_w, softmax_b = sess.run([m.word_embeddings, m.relation_embeddings, m.position_embeddings, m.relation_attention, m.attention, m.W, m.b, m.transfer_w, m.transfer_b, m.softmax_w, m.softmax_b]) 205 | log = open("log"+str1+".txt", "w") 206 | log.write(json.dumps(word_embeddings.tolist())+"\n") 207 | log.write(json.dumps(relation_embeddings.tolist())+"\n") 208 | log.write(json.dumps(position_embeddings.tolist())+"\n") 209 | log.write(json.dumps(relation_attention.tolist())+"\n") 210 | log.write(json.dumps(attention.tolist())+"\n") 211 | log.write(json.dumps(W.tolist())+"\n") 212 | log.write(json.dumps(B.tolist())+"\n") 213 | log.write(json.dumps(transfer_w.tolist())+"\n") 214 | NAattention = sess.run(m.NAattention) 215 | log.write(json.dumps(NAattention.tolist()) + "\n") 216 | ent_transfer = sess.run(m.ent_transfer) 217 | log.write(json.dumps(ent_transfer.tolist()) + "\n") 218 | rel_transfer = sess.run(m.rel_transfer) 219 | log.write(json.dumps(rel_transfer.tolist()) + "\n") 220 | log.close() 221 | 222 | x_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 223 | p_t_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 224 | p_h_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 225 | r_batch = np.zeros((1, 1), dtype = np.int32) 226 | y_batch = np.zeros((1, config.num_classes), dtype = np.int32) 227 | r_n_batch = np.zeros((1, 1), dtype = np.float32) 228 | h_batch = np.zeros((1, 1), dtype = np.int32) 229 | t_batch = np.zeros((1, 1), dtype = np.int32) 230 | 231 | x_batch_addr = x_batch.__array_interface__['data'][0] 232 | p_t_batch_addr = p_t_batch.__array_interface__['data'][0] 233 | p_h_batch_addr = p_h_batch.__array_interface__['data'][0] 234 | y_batch_addr = y_batch.__array_interface__['data'][0] 235 | r_batch_addr = r_batch.__array_interface__['data'][0] 236 | r_n_batch_addr = r_n_batch.__array_interface__['data'][0] 237 | h_batch_addr = h_batch.__array_interface__['data'][0] 238 | t_batch_addr = t_batch.__array_interface__['data'][0] 239 | lib_cnn.batch_iter.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p] 240 | tipTotal = lib_cnn.getTipTotal() 241 | loop = 0 242 | 243 | def train_cnn(coord): 244 | def train_step_cnn(x_batch, p_h_batch, p_t_batch, y_batch, r_batch, r_n_batch, h_batch, t_batch): 245 | global bags_sum, bags_hit, loss_sum, bags_hit_NA, bags_hit, sum_fNA, sum_NA 246 | feed_dict = { 247 | m.input_x: x_batch, 248 | m.input_p_h: p_h_batch, 249 | m.input_p_t: p_t_batch, 250 | m.input_r: r_batch, 251 | m.input_r_n: r_n_batch, 252 | m.input_y: y_batch, 253 | m.input_h: h_batch, 254 | m.input_t: t_batch 255 | } 256 | _, step, loss, accuracy = sess.run( 257 | [train_op_cnn, global_step_cnn, m.loss_cnn, m.accuracy], feed_dict) 258 | time_str = datetime.datetime.now().isoformat() 259 | loss_sum += loss 260 | bags_sum += 1 261 | if (r_batch[0]!=config.NA): 262 | sum_fNA += 1 263 | if accuracy > 0.5: 264 | bags_hit += 1.0 265 | else: 266 | sum_NA += 1 267 | if accuracy > 0.5: 268 | bags_hit_NA += 1.0 269 | if bags_sum % 1000 == 0: 270 | if (sum_NA == 0): 271 | sum_NA+=1 272 | if (sum_fNA == 0): 273 | sum_fNA+=1 274 | print("{}: step {}, loss {:g}, acc {:g} acc {:g} {} {}".format(time_str, step, loss_sum/bags_sum, bags_hit_NA/sum_NA, bags_hit/sum_fNA, sum_NA, sum_fNA)) 275 | 276 | 277 | global loop 278 | while not coord.should_stop(): 279 | print 'Looping ', loop 280 | outEmbedding(str(loop)) 281 | for i in range(tipTotal): 282 | length = lib_cnn.batch_iter(x_batch_addr, p_h_batch_addr, p_t_batch_addr, y_batch_addr, r_batch_addr, r_n_batch_addr, h_batch_addr, t_batch_addr) 283 | train_step_cnn(x_batch[0:length,], p_h_batch[0:length,], p_t_batch[0:length,], y_batch, r_batch, r_n_batch, h_batch, t_batch) 284 | global bags_sum, bags_hit, loss_sum, bags_hit_NA, bags_hit, sum_fNA, sum_NA 285 | bags_sum = 0 286 | bags_hit = 0 287 | bags_hit_NA = 0 288 | loss_sum = 0 289 | sum_fNA = 0 290 | sum_NA = 0 291 | loop += 1 292 | if loop == config.trainTimes: 293 | coord.request_stop() 294 | 295 | ph = np.zeros(config.batch_size, dtype = np.int32) 296 | pt = np.zeros(config.batch_size, dtype = np.int32) 297 | pr = np.zeros(config.batch_size, dtype = np.int32) 298 | nh = np.zeros(config.batch_size, dtype = np.int32) 299 | nt = np.zeros(config.batch_size, dtype = np.int32) 300 | nr = np.zeros(config.batch_size, dtype = np.int32) 301 | ph_addr = ph.__array_interface__['data'][0] 302 | pt_addr = pt.__array_interface__['data'][0] 303 | pr_addr = pr.__array_interface__['data'][0] 304 | nh_addr = nh.__array_interface__['data'][0] 305 | nt_addr = nt.__array_interface__['data'][0] 306 | nr_addr = nr.__array_interface__['data'][0] 307 | lib_kg.getBatch.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_int] 308 | times_kg = 0 309 | 310 | def train_kg(coord): 311 | def train_step_kg(pos_h_batch, pos_t_batch, pos_r_batch, neg_h_batch, neg_t_batch, neg_r_batch): 312 | feed_dict = { 313 | m.pos_h: pos_h_batch, 314 | m.pos_t: pos_t_batch, 315 | m.pos_r: pos_r_batch, 316 | m.neg_h: neg_h_batch, 317 | m.neg_t: neg_t_batch, 318 | m.neg_r: neg_r_batch 319 | } 320 | _, step, loss, = sess.run( 321 | [train_op_kg, global_step_kg, m.loss_kg], feed_dict) 322 | return loss 323 | global times_kg 324 | while not coord.should_stop(): 325 | times_kg += 1 326 | res = 0.0 327 | for batch in range(config.nbatches): 328 | lib_kg.getBatch(ph_addr, pt_addr, pr_addr, nh_addr, nt_addr, nr_addr, config.batch_size) 329 | res += train_step_kg(ph, pt, pr, nh, nt, nr) 330 | 331 | coord = tf.train.Coordinator() 332 | threads = [] 333 | threads.append(threading.Thread(target=train_kg, args=(coord,))) 334 | threads.append(threading.Thread(target=train_cnn, args=(coord,))) 335 | for t in threads: t.start() 336 | coord.join(threads) 337 | 338 | 339 | 340 | -------------------------------------------------------------------------------- /original/baselines/train/JointE+ATT.py: -------------------------------------------------------------------------------- 1 | #coding:utf-8 2 | import numpy as np 3 | import tensorflow as tf 4 | import os 5 | import time 6 | import datetime 7 | import ctypes 8 | import threading 9 | import json 10 | 11 | ll1 = ctypes.cdll.LoadLibrary 12 | lib_cnn = ll1("./init_cnn.so") 13 | ll2 = ctypes.cdll.LoadLibrary 14 | lib_kg = ll2("./init_know.so") 15 | 16 | class Config(object): 17 | def __init__(self): 18 | self.instanceTot = lib_cnn.getInstanceTot() 19 | self.sequence_size = lib_cnn.getLenLimit() 20 | self.num_classes = lib_cnn.getRelationTotal() 21 | self.num_words = lib_cnn.getWordTotal() 22 | self.num_positions = 2 * lib_cnn.getPositionLimit() + 1 23 | self.word_size = lib_cnn.getWordDimension() 24 | self.position_size = 5 25 | self.embedding_size = self.word_size + self.position_size * 2 26 | self.filter_size = 3 27 | self.num_filters = 230 28 | self.relation_size = self.word_size 29 | self.dropout_keep_prob = 0.5 30 | self.l2_lambda = 0.0001 31 | self.NA = 51 32 | lib_cnn.setNA(self.NA) 33 | lib_cnn.setRate(3) 34 | self.margin = 1.0 35 | self.nbatches = 100 36 | self.trainTimes = 15 37 | self.entityTotal = 0 38 | self.relationTotal = 0 39 | 40 | class Model(object): 41 | 42 | def __init__(self, config): 43 | sequence_size = config.sequence_size 44 | num_classes = config.num_classes 45 | num_words = config.num_words 46 | num_positions = config.num_positions 47 | 48 | embedding_size = config.embedding_size 49 | word_size = config.word_size 50 | position_size = config.position_size 51 | relation_size = config.relation_size 52 | filter_size = config.filter_size 53 | num_filters = config.num_filters 54 | dropout_keep_prob = config.dropout_keep_prob 55 | 56 | margin = config.margin 57 | l2_lambda = config.l2_lambda 58 | 59 | self.input_x = tf.placeholder(tf.int32, [None, sequence_size], name = "input_x") 60 | self.input_p_h = tf.placeholder(tf.int32, [None, sequence_size], name = "input_p_h") 61 | self.input_p_t = tf.placeholder(tf.int32, [None, sequence_size], name = "input_p_t") 62 | self.input_r = tf.placeholder(tf.int32, [1, 1], name = "input_r") 63 | self.input_r_n = tf.placeholder(tf.float32, [1, 1], name = "input_r_n") 64 | self.input_h = tf.placeholder(tf.int32, [1, 1], name = "input_h") 65 | self.input_t = tf.placeholder(tf.int32, [1, 1], name = "input_t") 66 | self.input_y = tf.placeholder(tf.float32, [1, num_classes], name = "input_y") 67 | 68 | self.pos_h = tf.placeholder(tf.int32, [None]) 69 | self.pos_t = tf.placeholder(tf.int32, [None]) 70 | self.pos_r = tf.placeholder(tf.int32, [None]) 71 | self.neg_h = tf.placeholder(tf.int32, [None]) 72 | self.neg_t = tf.placeholder(tf.int32, [None]) 73 | self.neg_r = tf.placeholder(tf.int32, [None]) 74 | 75 | l2_loss = tf.constant(0.0) 76 | with tf.name_scope("embedding-lookup"): 77 | self.word_embeddings = tf.Variable(word_embeddings, name="word_embeddings") 78 | self.relation_embeddings = tf.get_variable("relation_embeddings", [config.relationTotal, word_size]) 79 | self.position_embeddings = tf.get_variable("position_embeddings", [num_positions, position_size]) 80 | self.relation_attention = tf.get_variable("relation_attention", [num_classes, relation_size]) 81 | self.NAattention = tf.get_variable("NAattention", [relation_size, 1]) 82 | self.attention = tf.get_variable("attention", [num_filters, relation_size]) 83 | self.r = tf.nn.embedding_lookup(self.attention, self.input_r) 84 | 85 | #know 86 | pos_h_e = tf.nn.embedding_lookup(self.word_embeddings, self.pos_h) 87 | pos_t_e = tf.nn.embedding_lookup(self.word_embeddings, self.pos_t) 88 | pos_r_e = tf.nn.embedding_lookup(self.relation_embeddings, self.pos_r) 89 | neg_h_e = tf.nn.embedding_lookup(self.word_embeddings, self.neg_h) 90 | neg_t_e = tf.nn.embedding_lookup(self.word_embeddings, self.neg_t) 91 | neg_r_e = tf.nn.embedding_lookup(self.relation_embeddings, self.neg_r) 92 | 93 | #cnn 94 | self.x_initial = tf.nn.embedding_lookup(self.word_embeddings, self.input_x) 95 | self.x_p_h = tf.nn.embedding_lookup(self.position_embeddings, self.input_p_h) 96 | self.x_p_t = tf.nn.embedding_lookup(self.position_embeddings, self.input_p_t) 97 | self.x = tf.expand_dims(tf.concat(2, [self.x_initial, self.x_p_h, self.x_p_t]), -1) 98 | self.head = tf.nn.embedding_lookup(self.word_embeddings, self.input_h) 99 | self.tail = tf.nn.embedding_lookup(self.word_embeddings, self.input_t) 100 | l2_loss += tf.nn.l2_loss(self.attention) 101 | 102 | with tf.name_scope("conv-maxpool"): 103 | self.W = tf.get_variable("W", [filter_size, embedding_size, 1, num_filters]) 104 | self.b = tf.get_variable("b", [num_filters]) 105 | conv = tf.nn.conv2d(self.x, self.W, strides=[1, 1, 1, 1], padding="VALID", name="conv") 106 | h = tf.nn.tanh(tf.nn.bias_add(conv, self.b), name="tanh") 107 | self.y = tf.nn.max_pool(h, ksize=[1, sequence_size - filter_size + 1, 1, 1], strides=[1, 1, 1, 1], padding='VALID', name="pool") 108 | l2_loss += tf.nn.l2_loss(self.W) 109 | l2_loss += tf.nn.l2_loss(self.b) 110 | self.y = tf.reshape(self.y, [-1, num_filters]) 111 | 112 | with tf.name_scope('attention'): 113 | self.r = tf.reshape(self.r, [relation_size, -1]) 114 | self.e = tf.matmul(tf.matmul(self.y, self.attention), self.r) 115 | alpha = tf.reshape(self.e, [1, -1]) 116 | self.alpha_reshape = tf.nn.softmax(alpha) 117 | self.y_attention = tf.matmul(self.alpha_reshape, self.y) 118 | 119 | with tf.name_scope("dropout"): 120 | self.y_attention = tf.nn.l2_normalize(self.y_attention, 1) 121 | self.h_drop = tf.nn.dropout(self.y_attention, dropout_keep_prob) 122 | self.transfer_w = tf.get_variable("transfer_w", [num_filters, num_classes]) 123 | self.scores = tf.matmul(self.h_drop, self.transfer_w) 124 | l2_loss += tf.nn.l2_loss(self.transfer_w) 125 | 126 | with tf.name_scope("loss"): 127 | cross_entropy = tf.nn.softmax_cross_entropy_with_logits(self.scores, self.input_y) 128 | self.loss_cnn = tf.reduce_mean(cross_entropy) + l2_lambda * l2_loss 129 | 130 | pos = tf.reduce_sum(abs(pos_h_e + pos_r_e - pos_t_e), 1, keep_dims = True) 131 | neg = tf.reduce_sum(abs(neg_h_e + neg_r_e - neg_t_e), 1, keep_dims = True) 132 | self.loss_kg = tf.reduce_sum(tf.maximum(pos - neg + margin, 0)) 133 | 134 | with tf.name_scope("accuracy"): 135 | self.predictions = tf.argmax(self.scores, 1, name="predictions") 136 | correct_predictions = tf.equal(self.predictions, tf.argmax(self.input_y, 1)) 137 | self.accuracy = tf.reduce_mean(tf.cast(correct_predictions, "float"), name="accuracy") 138 | 139 | 140 | bags_sum = 0.0 141 | bags_hit_NA = 0.0 142 | sum_NA = 0.0 143 | sum_fNA = 0.0 144 | bags_hit = 0.0 145 | loss_sum = 0.0 146 | 147 | if __name__ == "__main__": 148 | 149 | lib_cnn.readWordVec() 150 | lib_cnn.readFromFile() 151 | lib_kg.init() 152 | 153 | np.random.seed(0) 154 | tf.set_random_seed(0) 155 | config = Config() 156 | 157 | word_embeddings = np.zeros(config.num_words * config.word_size, dtype = np.float32) 158 | lib_cnn.getWordVec.argtypes = [ctypes.c_void_p] 159 | lib_cnn.getWordVec(word_embeddings.__array_interface__['data'][0]) 160 | word_embeddings.resize((config.num_words,config.word_size)) 161 | 162 | config.batch_size = lib_kg.getTripleTotal() / config.nbatches 163 | config.entityTotal = lib_kg.getEntityTotal() 164 | config.relationTotal = lib_kg.getRelationTotal() 165 | 166 | with tf.Graph().as_default(): 167 | conf = tf.ConfigProto() 168 | sess = tf.Session(config=conf) 169 | with sess.as_default(): 170 | initializer = tf.contrib.layers.xavier_initializer() 171 | with tf.variable_scope("model", reuse=None, initializer = initializer): 172 | m = Model(config = config) 173 | 174 | global_step_cnn = tf.Variable(0, name="global_step_cnn", trainable=False) 175 | 176 | optimizer_cnn = tf.train.GradientDescentOptimizer(0.01) 177 | grads_and_vars_cnn = optimizer_cnn.compute_gradients(m.loss_cnn) 178 | train_op_cnn = optimizer_cnn.apply_gradients(grads_and_vars_cnn, global_step = global_step_cnn) 179 | 180 | global_step_kg = tf.Variable(0, name="global_step_kg", trainable=False) 181 | optimizer_kg = tf.train.GradientDescentOptimizer(0.001) 182 | grads_and_vars_kg = optimizer_kg.compute_gradients(m.loss_kg) 183 | train_op_kg = optimizer_kg.apply_gradients(grads_and_vars_kg, global_step=global_step_kg) 184 | 185 | sess.run(tf.initialize_all_variables()) 186 | 187 | def outEmbedding(str1): 188 | word_embeddings, relation_embeddings, position_embeddings, relation_attention, attention, W, B, transfer_w, transfer_b, softmax_w, softmax_b = sess.run([m.word_embeddings, m.relation_embeddings, m.position_embeddings, m.relation_attention, m.attention, m.W, m.b, m.transfer_w, m.transfer_b, m.softmax_w, m.softmax_b]) 189 | log = open("log"+str1+".txt", "w") 190 | log.write(json.dumps(word_embeddings.tolist())+"\n") 191 | log.write(json.dumps(relation_embeddings.tolist())+"\n") 192 | log.write(json.dumps(position_embeddings.tolist())+"\n") 193 | log.write(json.dumps(relation_attention.tolist())+"\n") 194 | log.write(json.dumps(attention.tolist())+"\n") 195 | log.write(json.dumps(W.tolist())+"\n") 196 | log.write(json.dumps(B.tolist())+"\n") 197 | log.write(json.dumps(transfer_w.tolist())+"\n") 198 | NAattention = sess.run(m.NAattention) 199 | log.write(json.dumps(NAattention.tolist()) + "\n") 200 | log.close() 201 | 202 | x_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 203 | p_t_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 204 | p_h_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 205 | r_batch = np.zeros((1, 1), dtype = np.int32) 206 | y_batch = np.zeros((1, config.num_classes), dtype = np.int32) 207 | r_n_batch = np.zeros((1, 1), dtype = np.float32) 208 | h_batch = np.zeros((1, 1), dtype = np.int32) 209 | t_batch = np.zeros((1, 1), dtype = np.int32) 210 | 211 | x_batch_addr = x_batch.__array_interface__['data'][0] 212 | p_t_batch_addr = p_t_batch.__array_interface__['data'][0] 213 | p_h_batch_addr = p_h_batch.__array_interface__['data'][0] 214 | y_batch_addr = y_batch.__array_interface__['data'][0] 215 | r_batch_addr = r_batch.__array_interface__['data'][0] 216 | r_n_batch_addr = r_n_batch.__array_interface__['data'][0] 217 | h_batch_addr = h_batch.__array_interface__['data'][0] 218 | t_batch_addr = t_batch.__array_interface__['data'][0] 219 | lib_cnn.batch_iter.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p] 220 | tipTotal = lib_cnn.getTipTotal() 221 | loop = 0 222 | 223 | def train_cnn(coord): 224 | def train_step_cnn(x_batch, p_h_batch, p_t_batch, y_batch, r_batch, r_n_batch, h_batch, t_batch): 225 | global bags_sum, bags_hit, loss_sum, bags_hit_NA, bags_hit, sum_fNA, sum_NA 226 | feed_dict = { 227 | m.input_x: x_batch, 228 | m.input_p_h: p_h_batch, 229 | m.input_p_t: p_t_batch, 230 | m.input_r: r_batch, 231 | m.input_r_n: r_n_batch, 232 | m.input_y: y_batch, 233 | m.input_h: h_batch, 234 | m.input_t: t_batch 235 | } 236 | _, step, loss, accuracy = sess.run( 237 | [train_op_cnn, global_step_cnn, m.loss_cnn, m.accuracy], feed_dict) 238 | time_str = datetime.datetime.now().isoformat() 239 | loss_sum += loss 240 | bags_sum += 1 241 | if (r_batch[0]!=config.NA): 242 | sum_fNA += 1 243 | if accuracy > 0.5: 244 | bags_hit += 1.0 245 | else: 246 | sum_NA += 1 247 | if accuracy > 0.5: 248 | bags_hit_NA += 1.0 249 | if bags_sum % 1000 == 0: 250 | if (sum_NA == 0): 251 | sum_NA+=1 252 | if (sum_fNA == 0): 253 | sum_fNA+=1 254 | print("{}: step {}, loss {:g}, acc {:g} acc {:g} {} {}".format(time_str, step, loss_sum/bags_sum, bags_hit_NA/sum_NA, bags_hit/sum_fNA, sum_NA, sum_fNA)) 255 | 256 | 257 | global loop 258 | while not coord.should_stop(): 259 | print 'Looping ', loop 260 | outEmbedding(str(loop)) 261 | for i in range(tipTotal): 262 | length = lib_cnn.batch_iter(x_batch_addr, p_h_batch_addr, p_t_batch_addr, y_batch_addr, r_batch_addr, r_n_batch_addr, h_batch_addr, t_batch_addr) 263 | train_step_cnn(x_batch[0:length,], p_h_batch[0:length,], p_t_batch[0:length,], y_batch, r_batch, r_n_batch, h_batch, t_batch) 264 | global bags_sum, bags_hit, loss_sum, bags_hit_NA, bags_hit, sum_fNA, sum_NA 265 | bags_sum = 0 266 | bags_hit = 0 267 | bags_hit_NA = 0 268 | loss_sum = 0 269 | sum_fNA = 0 270 | sum_NA = 0 271 | loop += 1 272 | if loop == config.trainTimes: 273 | coord.request_stop() 274 | 275 | ph = np.zeros(config.batch_size, dtype = np.int32) 276 | pt = np.zeros(config.batch_size, dtype = np.int32) 277 | pr = np.zeros(config.batch_size, dtype = np.int32) 278 | nh = np.zeros(config.batch_size, dtype = np.int32) 279 | nt = np.zeros(config.batch_size, dtype = np.int32) 280 | nr = np.zeros(config.batch_size, dtype = np.int32) 281 | ph_addr = ph.__array_interface__['data'][0] 282 | pt_addr = pt.__array_interface__['data'][0] 283 | pr_addr = pr.__array_interface__['data'][0] 284 | nh_addr = nh.__array_interface__['data'][0] 285 | nt_addr = nt.__array_interface__['data'][0] 286 | nr_addr = nr.__array_interface__['data'][0] 287 | lib_kg.getBatch.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_int] 288 | times_kg = 0 289 | 290 | def train_kg(coord): 291 | def train_step_kg(pos_h_batch, pos_t_batch, pos_r_batch, neg_h_batch, neg_t_batch, neg_r_batch): 292 | feed_dict = { 293 | m.pos_h: pos_h_batch, 294 | m.pos_t: pos_t_batch, 295 | m.pos_r: pos_r_batch, 296 | m.neg_h: neg_h_batch, 297 | m.neg_t: neg_t_batch, 298 | m.neg_r: neg_r_batch 299 | } 300 | _, step, loss = sess.run( 301 | [train_op_kg, global_step_kg, m.loss_kg], feed_dict) 302 | return loss 303 | global times_kg 304 | while not coord.should_stop(): 305 | times_kg += 1 306 | res = 0.0 307 | for batch in range(config.nbatches): 308 | lib_kg.getBatch(ph_addr, pt_addr, pr_addr, nh_addr, nt_addr, nr_addr, config.batch_size) 309 | res += train_step_kg(ph, pt, pr, nh, nt, nr) 310 | 311 | coord = tf.train.Coordinator() 312 | threads = [] 313 | threads.append(threading.Thread(target=train_kg, args=(coord,))) 314 | threads.append(threading.Thread(target=train_cnn, args=(coord,))) 315 | for t in threads: t.start() 316 | coord.join(threads) 317 | 318 | 319 | 320 | -------------------------------------------------------------------------------- /original/baselines/train/JointE+ONE.py: -------------------------------------------------------------------------------- 1 | #coding:utf-8 2 | import numpy as np 3 | import tensorflow as tf 4 | import os 5 | import time 6 | import datetime 7 | import ctypes 8 | import threading 9 | import json 10 | 11 | ll1 = ctypes.cdll.LoadLibrary 12 | lib_cnn = ll1("./init_cnn.so") 13 | ll2 = ctypes.cdll.LoadLibrary 14 | lib_kg = ll2("./init_know.so") 15 | 16 | class Config(object): 17 | def __init__(self): 18 | self.instanceTot = lib_cnn.getInstanceTot() 19 | self.sequence_size = lib_cnn.getLenLimit() 20 | self.num_classes = lib_cnn.getRelationTotal() 21 | self.num_words = lib_cnn.getWordTotal() 22 | self.num_positions = 2 * lib_cnn.getPositionLimit() + 1 23 | self.word_size = lib_cnn.getWordDimension() 24 | self.position_size = 5 25 | self.embedding_size = self.word_size + self.position_size * 2 26 | self.filter_size = 3 27 | self.num_filters = 230 28 | self.relation_size = self.word_size#230 29 | self.dropout_keep_prob = 0.5 30 | self.l2_lambda = 0.0001 31 | self.NA = 51 32 | lib_cnn.setNA(self.NA) 33 | lib_cnn.setRate(3) 34 | self.margin = 1.0 35 | self.nbatches = 100 36 | self.trainTimes = 15 37 | self.entityTotal = 0 38 | self.relationTotal = 0 39 | 40 | class Model(object): 41 | 42 | def __init__(self, config): 43 | sequence_size = config.sequence_size 44 | num_classes = config.num_classes 45 | num_words = config.num_words 46 | num_positions = config.num_positions 47 | 48 | embedding_size = config.embedding_size 49 | word_size = config.word_size 50 | position_size = config.position_size 51 | relation_size = config.relation_size 52 | filter_size = config.filter_size 53 | num_filters = config.num_filters 54 | dropout_keep_prob = config.dropout_keep_prob 55 | 56 | margin = config.margin 57 | l2_lambda = config.l2_lambda 58 | 59 | self.input_x = tf.placeholder(tf.int32, [None, sequence_size], name = "input_x") 60 | self.input_p_h = tf.placeholder(tf.int32, [None, sequence_size], name = "input_p_h") 61 | self.input_p_t = tf.placeholder(tf.int32, [None, sequence_size], name = "input_p_t") 62 | self.input_r = tf.placeholder(tf.float32, [1, 1], name = "input_r") 63 | self.input_r_n = tf.placeholder(tf.float32, [1, 1], name = "input_r_n") 64 | self.input_h = tf.placeholder(tf.int32, [1, 1], name = "input_h") 65 | self.input_t = tf.placeholder(tf.int32, [1, 1], name = "input_t") 66 | self.input_y = tf.placeholder(tf.float32, [1, num_classes], name = "input_y") 67 | 68 | self.pos_h = tf.placeholder(tf.int32, [None]) 69 | self.pos_t = tf.placeholder(tf.int32, [None]) 70 | self.pos_r = tf.placeholder(tf.int32, [None]) 71 | self.neg_h = tf.placeholder(tf.int32, [None]) 72 | self.neg_t = tf.placeholder(tf.int32, [None]) 73 | self.neg_r = tf.placeholder(tf.int32, [None]) 74 | 75 | l2_loss = tf.constant(0.0) 76 | with tf.name_scope("embedding-lookup"): 77 | self.word_embeddings = tf.Variable(word_embeddings, name="word_embeddings") 78 | self.relation_embeddings = tf.get_variable("relation_embeddings", [config.relationTotal, word_size]) 79 | self.position_embeddings = tf.get_variable("position_embeddings", [num_positions, position_size]) 80 | self.relation_attention = tf.get_variable("relation_attention", [num_classes, relation_size]) 81 | self.NAattention = tf.get_variable("NAattention", [relation_size, 1]) 82 | self.attention = tf.get_variable("attention", [num_filters, relation_size]) 83 | 84 | #know 85 | pos_h_e = tf.nn.embedding_lookup(self.word_embeddings, self.pos_h) 86 | pos_t_e = tf.nn.embedding_lookup(self.word_embeddings, self.pos_t) 87 | pos_r_e = tf.nn.embedding_lookup(self.relation_embeddings, self.pos_r) 88 | neg_h_e = tf.nn.embedding_lookup(self.word_embeddings, self.neg_h) 89 | neg_t_e = tf.nn.embedding_lookup(self.word_embeddings, self.neg_t) 90 | neg_r_e = tf.nn.embedding_lookup(self.relation_embeddings, self.neg_r) 91 | 92 | #cnn 93 | self.x_initial = tf.nn.embedding_lookup(self.word_embeddings, self.input_x) 94 | self.x_p_h = tf.nn.embedding_lookup(self.position_embeddings, self.input_p_h) 95 | self.x_p_t = tf.nn.embedding_lookup(self.position_embeddings, self.input_p_t) 96 | self.x = tf.expand_dims(tf.concat(2, [self.x_initial, self.x_p_h, self.x_p_t]), -1) 97 | self.head = tf.nn.embedding_lookup(self.word_embeddings, self.input_h) 98 | self.tail = tf.nn.embedding_lookup(self.word_embeddings, self.input_t) 99 | l2_loss += tf.nn.l2_loss(self.attention) 100 | 101 | with tf.name_scope("conv-maxpool"): 102 | self.W = tf.get_variable("W", [filter_size, embedding_size, 1, num_filters]) 103 | self.b = tf.get_variable("b", [num_filters]) 104 | conv = tf.nn.conv2d(self.x, self.W, strides=[1, 1, 1, 1], padding="VALID", name="conv") 105 | h = tf.nn.tanh(tf.nn.bias_add(conv, self.b), name="tanh") 106 | self.y = tf.nn.max_pool(h, ksize=[1, sequence_size - filter_size + 1, 1, 1], strides=[1, 1, 1, 1], padding='VALID', name="pool") 107 | l2_loss += tf.nn.l2_loss(self.W) 108 | l2_loss += tf.nn.l2_loss(self.b) 109 | self.y = tf.reshape(self.y, [-1, num_filters]) 110 | 111 | with tf.name_scope('attention'): 112 | self.y_attention = tf.reduce_max(self.y, 0 , keep_dims = True) 113 | 114 | with tf.name_scope("dropout"): 115 | self.y_attention = tf.nn.l2_normalize(self.y_attention, 1) 116 | self.h_drop = tf.nn.dropout(self.y_attention, dropout_keep_prob) 117 | self.transfer_w = tf.get_variable("transfer_w", [num_filters, num_classes]) 118 | self.scores = tf.matmul(self.h_drop, self.transfer_w) 119 | l2_loss += tf.nn.l2_loss(self.transfer_w) 120 | 121 | with tf.name_scope("loss"): 122 | cross_entropy = tf.nn.softmax_cross_entropy_with_logits(self.scores, self.input_y) 123 | self.loss_cnn = tf.reduce_mean(cross_entropy) + l2_lambda * l2_loss 124 | 125 | pos = tf.reduce_sum(abs(pos_h_e + pos_r_e - pos_t_e), 1, keep_dims = True) 126 | neg = tf.reduce_sum(abs(neg_h_e + neg_r_e - neg_t_e), 1, keep_dims = True) 127 | self.loss_kg = tf.reduce_sum(tf.maximum(pos - neg + margin, 0)) 128 | 129 | with tf.name_scope("accuracy"): 130 | self.predictions = tf.argmax(self.scores, 1, name="predictions") 131 | correct_predictions = tf.equal(self.predictions, tf.argmax(self.input_y, 1)) 132 | self.accuracy = tf.reduce_mean(tf.cast(correct_predictions, "float"), name="accuracy") 133 | 134 | 135 | bags_sum = 0.0 136 | bags_hit_NA = 0.0 137 | sum_NA = 0.0 138 | sum_fNA = 0.0 139 | bags_hit = 0.0 140 | loss_sum = 0.0 141 | 142 | if __name__ == "__main__": 143 | 144 | lib_cnn.readWordVec() 145 | lib_cnn.readFromFile() 146 | lib_kg.init() 147 | 148 | np.random.seed(0) 149 | tf.set_random_seed(0) 150 | config = Config() 151 | 152 | word_embeddings = np.zeros(config.num_words * config.word_size, dtype = np.float32) 153 | lib_cnn.getWordVec.argtypes = [ctypes.c_void_p] 154 | lib_cnn.getWordVec(word_embeddings.__array_interface__['data'][0]) 155 | word_embeddings.resize((config.num_words,config.word_size)) 156 | 157 | config.batch_size = lib_kg.getTripleTotal() / config.nbatches 158 | config.entityTotal = lib_kg.getEntityTotal() 159 | config.relationTotal = lib_kg.getRelationTotal() 160 | 161 | with tf.Graph().as_default(): 162 | conf = tf.ConfigProto() 163 | sess = tf.Session(config=conf) 164 | with sess.as_default(): 165 | initializer = tf.contrib.layers.xavier_initializer() 166 | with tf.variable_scope("model", reuse=None, initializer = initializer): 167 | m = Model(config = config) 168 | 169 | global_step_cnn = tf.Variable(0, name="global_step_cnn", trainable=False) 170 | 171 | optimizer_cnn = tf.train.GradientDescentOptimizer(0.01) 172 | grads_and_vars_cnn = optimizer_cnn.compute_gradients(m.loss_cnn) 173 | train_op_cnn = optimizer_cnn.apply_gradients(grads_and_vars_cnn, global_step = global_step_cnn) 174 | 175 | global_step_kg = tf.Variable(0, name="global_step_kg", trainable=False) 176 | optimizer_kg = tf.train.GradientDescentOptimizer(0.001) 177 | grads_and_vars_kg = optimizer_kg.compute_gradients(m.loss_kg) 178 | train_op_kg = optimizer_kg.apply_gradients(grads_and_vars_kg, global_step=global_step_kg) 179 | 180 | sess.run(tf.initialize_all_variables()) 181 | 182 | def outEmbedding(str1): 183 | word_embeddings, relation_embeddings, position_embeddings, relation_attention, attention, W, B, transfer_w, transfer_b, softmax_w, softmax_b = sess.run([m.word_embeddings, m.relation_embeddings, m.position_embeddings, m.relation_attention, m.attention, m.W, m.b, m.transfer_w, m.transfer_b, m.softmax_w, m.softmax_b]) 184 | log = open("log"+str1+".txt", "w") 185 | log.write(json.dumps(word_embeddings.tolist())+"\n") 186 | log.write(json.dumps(relation_embeddings.tolist())+"\n") 187 | log.write(json.dumps(position_embeddings.tolist())+"\n") 188 | log.write(json.dumps(relation_attention.tolist())+"\n") 189 | log.write(json.dumps(attention.tolist())+"\n") 190 | log.write(json.dumps(W.tolist())+"\n") 191 | log.write(json.dumps(B.tolist())+"\n") 192 | log.write(json.dumps(transfer_w.tolist())+"\n") 193 | NAattention = sess.run(m.NAattention) 194 | log.write(json.dumps(NAattention.tolist()) + "\n") 195 | log.close() 196 | 197 | x_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 198 | p_t_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 199 | p_h_batch = np.zeros((config.instanceTot,config.sequence_size), dtype = np.int32) 200 | r_batch = np.zeros((1, 1), dtype = np.int32) 201 | y_batch = np.zeros((1, config.num_classes), dtype = np.int32) 202 | r_n_batch = np.zeros((1, 1), dtype = np.float32) 203 | h_batch = np.zeros((1, 1), dtype = np.int32) 204 | t_batch = np.zeros((1, 1), dtype = np.int32) 205 | 206 | x_batch_addr = x_batch.__array_interface__['data'][0] 207 | p_t_batch_addr = p_t_batch.__array_interface__['data'][0] 208 | p_h_batch_addr = p_h_batch.__array_interface__['data'][0] 209 | y_batch_addr = y_batch.__array_interface__['data'][0] 210 | r_batch_addr = r_batch.__array_interface__['data'][0] 211 | r_n_batch_addr = r_n_batch.__array_interface__['data'][0] 212 | h_batch_addr = h_batch.__array_interface__['data'][0] 213 | t_batch_addr = t_batch.__array_interface__['data'][0] 214 | lib_cnn.batch_iter.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p] 215 | tipTotal = lib_cnn.getTipTotal() 216 | loop = 0 217 | 218 | def train_cnn(coord): 219 | def train_step_cnn(x_batch, p_h_batch, p_t_batch, y_batch, r_batch, r_n_batch, h_batch, t_batch): 220 | global bags_sum, bags_hit, loss_sum, bags_hit_NA, bags_hit, sum_fNA, sum_NA 221 | feed_dict = { 222 | m.input_x: x_batch, 223 | m.input_p_h: p_h_batch, 224 | m.input_p_t: p_t_batch, 225 | m.input_r: r_batch, 226 | m.input_r_n: r_n_batch, 227 | m.input_y: y_batch, 228 | m.input_h: h_batch, 229 | m.input_t: t_batch 230 | } 231 | _, step, loss, accuracy = sess.run( 232 | [train_op_cnn, global_step_cnn, m.loss_cnn, m.accuracy], feed_dict) 233 | time_str = datetime.datetime.now().isoformat() 234 | loss_sum += loss 235 | bags_sum += 1 236 | if (r_batch[0]!=config.NA): 237 | sum_fNA += 1 238 | if accuracy > 0.5: 239 | bags_hit += 1.0 240 | else: 241 | sum_NA += 1 242 | if accuracy > 0.5: 243 | bags_hit_NA += 1.0 244 | if bags_sum % 1000 == 0: 245 | if (sum_NA == 0): 246 | sum_NA+=1 247 | if (sum_fNA == 0): 248 | sum_fNA+=1 249 | print("{}: step {}, loss {:g}, acc {:g} acc {:g} {} {}".format(time_str, step, loss_sum/bags_sum, bags_hit_NA/sum_NA, bags_hit/sum_fNA, sum_NA, sum_fNA)) 250 | 251 | 252 | global loop 253 | while not coord.should_stop(): 254 | print 'Looping ', loop 255 | outEmbedding(str(loop)) 256 | for i in range(tipTotal): 257 | length = lib_cnn.batch_iter(x_batch_addr, p_h_batch_addr, p_t_batch_addr, y_batch_addr, r_batch_addr, r_n_batch_addr, h_batch_addr, t_batch_addr) 258 | train_step_cnn(x_batch[0:length,], p_h_batch[0:length,], p_t_batch[0:length,], y_batch, r_batch, r_n_batch, h_batch, t_batch) 259 | global bags_sum, bags_hit, loss_sum, bags_hit_NA, bags_hit, sum_fNA, sum_NA 260 | bags_sum = 0 261 | bags_hit = 0 262 | bags_hit_NA = 0 263 | loss_sum = 0 264 | sum_fNA = 0 265 | sum_NA = 0 266 | loop += 1 267 | if loop == config.trainTimes: 268 | coord.request_stop() 269 | 270 | ph = np.zeros(config.batch_size * 2, dtype = np.int32) 271 | pt = np.zeros(config.batch_size * 2, dtype = np.int32) 272 | pr = np.zeros(config.batch_size * 2, dtype = np.int32) 273 | nh = np.zeros(config.batch_size * 2, dtype = np.int32) 274 | nt = np.zeros(config.batch_size * 2, dtype = np.int32) 275 | nr = np.zeros(config.batch_size * 2, dtype = np.int32) 276 | ph_addr = ph.__array_interface__['data'][0] 277 | pt_addr = pt.__array_interface__['data'][0] 278 | pr_addr = pr.__array_interface__['data'][0] 279 | nh_addr = nh.__array_interface__['data'][0] 280 | nt_addr = nt.__array_interface__['data'][0] 281 | nr_addr = nr.__array_interface__['data'][0] 282 | lib_kg.getBatch.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_int] 283 | times_kg = 0 284 | 285 | def train_kg(coord): 286 | def train_step_kg(pos_h_batch, pos_t_batch, pos_r_batch, neg_h_batch, neg_t_batch, neg_r_batch): 287 | feed_dict = { 288 | m.pos_h: pos_h_batch, 289 | m.pos_t: pos_t_batch, 290 | m.pos_r: pos_r_batch, 291 | m.neg_h: neg_h_batch, 292 | m.neg_t: neg_t_batch, 293 | m.neg_r: neg_r_batch 294 | } 295 | _, step, loss = sess.run( 296 | [train_op_kg, global_step_kg, m.loss_kg], feed_dict) 297 | return loss 298 | global times_kg 299 | while not coord.should_stop(): 300 | times_kg += 1 301 | res = 0.0 302 | for batch in range(config.nbatches): 303 | lib_kg.getBatch(ph_addr, pt_addr, pr_addr, nh_addr, nt_addr, nr_addr, config.batch_size) 304 | res += train_step_kg(ph, pt, pr, nh, nt, nr) 305 | 306 | coord = tf.train.Coordinator() 307 | threads = [] 308 | threads.append(threading.Thread(target=train_kg, args=(coord,))) 309 | threads.append(threading.Thread(target=train_cnn, args=(coord,))) 310 | for t in threads: t.start() 311 | coord.join(threads) 312 | 313 | 314 | 315 | --------------------------------------------------------------------------------