├── saved_embeddings └── my │ └── none.txt ├── processed_data └── notes.txt ├── README.md ├── model_my_variant1.py ├── my_utils.py ├── model_my_variant2.py ├── Dataset_CDAE.py ├── model_my_variant3.py ├── model_my.py ├── model_my_extention.py ├── main_my_variant1.py ├── main_my_variant2.py ├── main_my_variant3.py └── main_my.py /saved_embeddings/my/none.txt: -------------------------------------------------------------------------------- 1 | none 2 | -------------------------------------------------------------------------------- /processed_data/notes.txt: -------------------------------------------------------------------------------- 1 | The original data has large memory and we cannot upload to Github directly. 2 | Instead, please download it from Google drive or Baidu disk by following the advice on the project page. 3 | 4 | Thanks. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # ETL-master 2 | This is a pytorch implementation of our ACM Transactions on Information Systems (ACM TOIS) paper: "Towards Equivalent Transformation of User Preferences in Cross Domain Recommendation" 3 | 4 | **Running Tips** 5 | This folder contains the thhree datasets used in our paper and the codes of our ETL model. Specifically, main_my.py is our model and the other four main_xxx.py files are four variants of ETL in the abalation study of our paper. 6 | 7 | **The directory originization** 8 | ETL-master/ 9 | ----processed_data/ 10 | ----saved_embeddings/my/none.txt 11 | ----code_name1.py 12 | ----code_name2.py 13 | 14 | **0. Data preparation:** 15 | Download the processed data files from Google drive: https://drive.google.com/drive/folders/1oIABLZE0UcEylLwiJrWY5R3rw7t739Zz?usp=sharing 16 | and Baidu disk: https://pan.baidu.com/s/1UD5ezVqjsMBmIgQXl1CA-g password:yk2i. 17 | When you download the processed_data, you can replace the dir here and directly run our codes! Note that the data_filter_amazon.py under processed_data dir is the file that pre-processes the original data. You can skip the pre-processing step if you download the processed data files. 18 | 19 | **1. Requirements:** 20 | python3.5; pytorch=1.4.0; tqdm=4.27.0; tensorboardX=1.8; pandas=0.25; numpy=1.15; networkx=2.2; logger=1.4; scipy=1.1; scikit-learn=0.20 21 | 22 | **2.Runing commands** 23 | Run the codes with the following commands on different datasets (amazon means "Movie & Book", amazon2 means "Movie & Music" and amazon3 means "Music & Book"). 24 | 25 | -->on Movie & Book dataset: 26 | CUDA_VISIBLE_DEVICES=gpu_num python main_my.py --dataset amazon --reg 5.0 27 | 28 | -->on Movie & Music dataset: 29 | CUDA_VISIBLE_DEVICES=gpu_num python main_my.py --dataset amazon2 --reg 0.5 30 | 31 | -->on Music & Book dataset: 32 | CUDA_VISIBLE_DEVICES=gpu_num python main_my.py --dataset amazon3 --reg 1.0 33 | 34 | In this way, you can get he results. Besides, if you want to run the variants of ETL, just follow the same way while with different main files. 35 | 36 | Moreover, if you want to do experments with different data sparsity, you can add "--s_percent ${ratio} --t_percent ${ratio}" to the above running commands. 37 | 38 | **If you find this paper or codes useful, please cite our paper. Thank you!** 39 | 40 | 41 | @article{chen2020equivalent, 42 | title={Towards Equivalent Transformation of User Preferences in Cross Domain Recommendation}, 43 | author={Xu Chen and Ya Zhang and Ivor Tsang and Yuangang Pan and Jingchao Su}, 44 | year={2020}, 45 | eprint={2009.06884}, 46 | archivePrefix={ACM Transactions on Information Systems}, 47 | } 48 | 49 | -------------------------------------------------------------------------------- /model_my_variant1.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | import argparse 3 | import numpy as np 4 | import torch 5 | import torch.utils.data 6 | from torch import nn, optim 7 | import torch.nn.functional as F 8 | 9 | class CDAE(nn.Module): 10 | def __init__(self, NUM_USER, NUM_MOVIE, NUM_BOOK, EMBED_SIZE, dropout, is_sparse=False): 11 | super(CDAE, self).__init__() 12 | self.NUM_MOVIE = NUM_MOVIE 13 | self.NUM_BOOK = NUM_BOOK 14 | self.NUM_USER = NUM_USER 15 | self.emb_size = EMBED_SIZE 16 | 17 | self.user_embeddings = nn.Embedding(self.NUM_USER, EMBED_SIZE, sparse=is_sparse) 18 | self.user_embeddings.weight.data = torch.from_numpy(np.random.normal(0, 0.01, size=[self.NUM_USER, EMBED_SIZE])).float() 19 | 20 | self.encoder_x = nn.Sequential( 21 | nn.Linear(self.NUM_MOVIE, EMBED_SIZE), 22 | nn.ReLU(), 23 | nn.Linear(EMBED_SIZE, EMBED_SIZE) 24 | ) 25 | self.decoder_x = nn.Sequential( 26 | nn.Linear(EMBED_SIZE, EMBED_SIZE), 27 | nn.ReLU(), 28 | nn.Linear(EMBED_SIZE, self.NUM_MOVIE) 29 | ) 30 | self.encoder_y = nn.Sequential( 31 | nn.Linear(self.NUM_BOOK, EMBED_SIZE), 32 | nn.ReLU(), 33 | nn.Linear(EMBED_SIZE, EMBED_SIZE) 34 | ) 35 | self.decoder_y = nn.Sequential( 36 | nn.Linear(EMBED_SIZE, EMBED_SIZE), 37 | nn.ReLU(), 38 | nn.Linear(EMBED_SIZE, self.NUM_BOOK) 39 | ) 40 | 41 | self.orthogonal_w = nn.Parameter(nn.init.xavier_uniform(torch.Tensor(EMBED_SIZE, EMBED_SIZE).type( 42 | torch.cuda.FloatTensor if torch.cuda.is_available() else torch.FloatTensor), gain=np.sqrt(2.0)), 43 | requires_grad=True) 44 | self.dropout = nn.Dropout(dropout) 45 | self.relu = nn.ReLU 46 | 47 | def orthogonal_map(self, z_x, z_y): 48 | mapped_z_x = torch.matmul(z_x, self.orthogonal_w) 49 | mapped_z_y = torch.matmul(z_y, torch.transpose(self.orthogonal_w, 1, 0)) 50 | return mapped_z_x, mapped_z_y 51 | 52 | def forward(self, batch_user, batch_user_x, batch_user_y): 53 | h_user_x = self.encoder_x(self.dropout(batch_user_x)) 54 | h_user_y = self.encoder_y(self.dropout(batch_user_y)) 55 | h_user = self.user_embeddings(batch_user) 56 | feature_x = torch.add(h_user_x, h_user) 57 | feature_y = torch.add(h_user_y, h_user) 58 | z_x = F.relu(feature_x) 59 | z_y = F.relu(feature_y) 60 | preds_x = self.decoder_x(z_x) 61 | preds_y = self.decoder_y(z_y) 62 | mapped_z_x, mapped_z_y = self.orthogonal_map(z_x, z_y) 63 | preds_x2y = self.decoder_y(mapped_z_x) 64 | preds_y2x = self.decoder_x(mapped_z_y) 65 | 66 | # define orthogonal constraint loss 67 | z_x_ = torch.matmul(mapped_z_x, torch.transpose(self.orthogonal_w, 1, 0)) 68 | z_y_ = torch.matmul(mapped_z_y, self.orthogonal_w) 69 | z_x_reg_loss = torch.norm(z_x - z_x_, p=1, dim=1) 70 | z_y_reg_loss = torch.norm(z_y - z_y_, p=1, dim=1) 71 | 72 | return preds_x, preds_y, preds_x2y, preds_y2x, feature_x, feature_y, z_x_reg_loss, z_y_reg_loss 73 | def get_user_embedding(self, batch_user_x, batch_user_y): 74 | h_user_x = self.encoder_x(self.dropout(batch_user_x)) 75 | h_user_y = self.encoder_y(self.dropout(batch_user_y)) 76 | return h_user_x, h_user_y 77 | 78 | # class MI_Map(nn.Module): 79 | # def __init__(self, n_input): 80 | # super(MI_Map, self).__init__() 81 | # 82 | # self.fc_x = nn.Linear(n_input, n_input, bias=False) 83 | # # self.fc_y = nn.Linear(n_input, n_input, bias=False) 84 | # 85 | # def forward(self, z_x, z_y): 86 | # # make mlp for discriminator 87 | # logits = torch.sum(self.fc_x(z_x)*z_y, dim=1) 88 | # return logits 89 | 90 | class Discriminator(nn.Module): 91 | def __init__(self, n_fts, dropout): 92 | super(Discriminator, self).__init__() 93 | self.dropout = dropout 94 | self.training = True 95 | 96 | self.disc = nn.Sequential( 97 | nn.Linear(n_fts, int(n_fts/2)), 98 | nn.ReLU(), 99 | nn.Dropout(dropout), 100 | nn.Linear(int(n_fts/2), 1)) 101 | 102 | def forward(self, x): 103 | # make mlp for discriminator 104 | h = self.disc(x) 105 | return h 106 | 107 | def save_embedding_process(model, save_loader, feed_data, is_cuda): 108 | fts1 = feed_data['fts1'] 109 | fts2 = feed_data['fts2'] 110 | 111 | user_embedding1_list = [] 112 | user_embedding2_list = [] 113 | model.eval() 114 | for batch_idx, data in enumerate(save_loader): 115 | data = data.reshape([-1]) 116 | val_user_arr = data.numpy() 117 | v_item1 = fts1[val_user_arr] 118 | v_item2 = fts2[val_user_arr] 119 | if is_cuda: 120 | v_user = torch.LongTensor(val_user_arr).cuda() 121 | v_item1 = torch.FloatTensor(v_item1).cuda() 122 | v_item2 = torch.FloatTensor(v_item2).cuda() 123 | else: 124 | v_user = torch.LongTensor(val_user_arr) 125 | v_item1 = torch.FloatTensor(v_item1) 126 | v_item2 = torch.FloatTensor(v_item2) 127 | 128 | res = model.get_user_embedding(v_item1, v_item2) 129 | user_embedding1 = res[0] 130 | user_embedding2 = res[1] 131 | if is_cuda: 132 | user_embedding1 = user_embedding1.detach().cpu().numpy() 133 | user_embedding2 = user_embedding2.detach().cpu().numpy() 134 | else: 135 | user_embedding1 = user_embedding1.detach().numpy() 136 | user_embedding2 = user_embedding2.detach().numpy() 137 | 138 | user_embedding1_list.append(user_embedding1) 139 | user_embedding2_list.append(user_embedding2) 140 | 141 | return np.concatenate(user_embedding1_list, axis=0), np.concatenate(user_embedding2_list, axis=0) -------------------------------------------------------------------------------- /my_utils.py: -------------------------------------------------------------------------------- 1 | import os 2 | import torch 3 | import torch.nn as nn 4 | import torch.nn.functional as F 5 | import torch.autograd as autograd 6 | from sklearn.metrics import auc 7 | import numpy as np 8 | import scipy.sparse as sp 9 | from scipy.sparse import csr_matrix 10 | from tqdm import tqdm 11 | 12 | def MMD(X, Y, biased=True): 13 | # set params to calculate MMD distance 14 | sigma_list = [1e-2, 1e-1, 1, 10, 100] 15 | sigma_list = torch.FloatTensor(np.array(sigma_list)) 16 | K_XX, K_XY, K_YY, d = _mix_rbf_kernel(X, Y, sigma_list) 17 | # return _mmd2(K_XX, K_XY, K_YY, const_diagonal=d, biased=biased) 18 | return _mmd2(K_XX, K_XY, K_YY, const_diagonal=False, biased=biased) 19 | 20 | def _mix_rbf_kernel(X, Y, sigma_list): 21 | assert(X.size(0) == Y.size(0)) 22 | m = X.size(0) 23 | 24 | Z = torch.cat((X, Y), 0) 25 | ZZT = torch.mm(Z, Z.t()) 26 | diag_ZZT = torch.diag(ZZT).unsqueeze(1) 27 | Z_norm_sqr = diag_ZZT.expand_as(ZZT) 28 | exponent = Z_norm_sqr - 2 * ZZT + Z_norm_sqr.t() 29 | 30 | K = 0.0 31 | for sigma in sigma_list: 32 | gamma = 1.0 / (2 * sigma**2) 33 | K += torch.exp(-gamma * exponent) 34 | 35 | return K[:m, :m], K[:m, m:], K[m:, m:], len(sigma_list) 36 | 37 | def _mmd2(K_XX, K_XY, K_YY, const_diagonal=False, biased=False): 38 | m = K_XX.size(0) # assume X, Y are same shape 39 | 40 | # Get the various sums of kernels that we'll use 41 | # Kts drop the diagonal, but we don't need to compute them explicitly 42 | if const_diagonal is not False: 43 | diag_X = diag_Y = const_diagonal 44 | sum_diag_X = sum_diag_Y = m * const_diagonal 45 | else: 46 | diag_X = torch.diag(K_XX) # (m,) 47 | diag_Y = torch.diag(K_YY) # (m,) 48 | sum_diag_X = torch.sum(diag_X) 49 | sum_diag_Y = torch.sum(diag_Y) 50 | 51 | Kt_XX_sums = K_XX.sum(dim=1) - diag_X # \tilde{K}_XX * e = K_XX * e - diag_X 52 | Kt_YY_sums = K_YY.sum(dim=1) - diag_Y # \tilde{K}_YY * e = K_YY * e - diag_Y 53 | K_XY_sums_0 = K_XY.sum(dim=0) # K_{XY}^T * e 54 | 55 | Kt_XX_sum = Kt_XX_sums.sum() # e^T * \tilde{K}_XX * e 56 | Kt_YY_sum = Kt_YY_sums.sum() # e^T * \tilde{K}_YY * e 57 | K_XY_sum = K_XY_sums_0.sum() # e^T * K_{XY} * e 58 | 59 | if biased: 60 | mmd2 = ((Kt_XX_sum + sum_diag_X) / (m * m) 61 | + (Kt_YY_sum + sum_diag_Y) / (m * m) 62 | - 2.0 * K_XY_sum / (m * m)) 63 | else: 64 | mmd2 = (Kt_XX_sum / (m * (m - 1)) 65 | + Kt_YY_sum / (m * (m - 1)) 66 | - 2.0 * K_XY_sum / (m * m)) 67 | 68 | return mmd2 69 | 70 | def evaluation(preds, topk): 71 | sort = np.argsort(-preds, axis=1)[:, :topk] 72 | hr_arr = np.zeros(shape=[sort.shape[0]]) 73 | ndcg_arr = np.zeros(shape=[sort.shape[0]]) 74 | mrr_arr = np.zeros(shape=[sort.shape[0]]) 75 | rows = np.where(sort==99)[0] 76 | cols = np.where(sort==99)[1] 77 | hr_arr[rows] = 1.0 78 | ndcg_arr[rows] = np.log(2) / np.log(cols + 2.0) 79 | mrr_arr[rows] = 1.0 / (cols + 1.0) 80 | return hr_arr.tolist(), ndcg_arr.tolist(), mrr_arr.tolist() 81 | 82 | def test_process(model, train_loader, feed_data, is_cuda, topK, mode='val'): 83 | all_hr1_list = [] 84 | all_ndcg1_list = [] 85 | all_mrr1_list = [] 86 | all_hr2_list = [] 87 | all_ndcg2_list = [] 88 | all_mrr2_list = [] 89 | fts1 = feed_data['fts1'] 90 | fts2 = feed_data['fts2'] 91 | 92 | if mode == 'val': 93 | movie_nega = feed_data['movie_nega'] 94 | movie_test = feed_data['movie_vali'] 95 | book_nega = feed_data['book_nega'] 96 | book_test = feed_data['book_vali'] 97 | elif mode=='test': 98 | movie_nega = feed_data['movie_nega'] 99 | movie_test = feed_data['movie_test'] 100 | book_nega = feed_data['book_nega'] 101 | book_test = feed_data['book_test'] 102 | else: 103 | raise Exception 104 | 105 | # user_embeddings_x = [] 106 | # user_embeddings_y = [] 107 | for batch_idx, data in enumerate(train_loader): 108 | data = data.reshape([-1]) 109 | val_user_arr = data.numpy() 110 | v_item1 = fts1[val_user_arr] 111 | v_item2 = fts2[val_user_arr] 112 | if is_cuda: 113 | v_user = torch.LongTensor(val_user_arr).cuda() 114 | v_item1 = torch.FloatTensor(v_item1).cuda() 115 | v_item2 = torch.FloatTensor(v_item2).cuda() 116 | else: 117 | v_user = torch.LongTensor(val_user_arr) 118 | v_item1 = torch.FloatTensor(v_item1) 119 | v_item2 = torch.FloatTensor(v_item2) 120 | 121 | res = model.forward(v_user, v_item1, v_item2) 122 | y1 = res[0] 123 | y2 = res[1] 124 | if is_cuda: 125 | y1 = y1.detach().cpu().numpy() 126 | y2 = y2.detach().cpu().numpy() 127 | else: 128 | y1 = y1.detach().numpy() 129 | y2 = y2.detach().numpy() 130 | 131 | 132 | nega_vali1 = np.array([movie_nega[ele] + [movie_test[ele]] for ele in val_user_arr]) 133 | nega_vali2 = np.array([book_nega[ele] + [book_test[ele]] for ele in val_user_arr]) 134 | pred1 = np.stack([y1[xx][nega_vali1[xx]] for xx in range(nega_vali1.shape[0])]) 135 | pred2 = np.stack([y2[xx][nega_vali2[xx]] for xx in range(nega_vali2.shape[0])]) 136 | hr1_list, ndcg1_list, mrr1_list = evaluation(pred1, topK) 137 | hr2_list, ndcg2_list, mrr2_list = evaluation(pred2, topK) 138 | all_hr1_list += hr1_list 139 | all_ndcg1_list += ndcg1_list 140 | all_mrr1_list += mrr1_list 141 | all_hr2_list += hr2_list 142 | all_ndcg2_list += ndcg2_list 143 | all_mrr2_list += mrr2_list 144 | 145 | avg_hr1 = np.mean(all_hr1_list) 146 | avg_ndcg1 = np.mean(all_ndcg1_list) 147 | avg_mrr1 = np.mean(all_mrr1_list) 148 | avg_hr2 = np.mean(all_hr2_list) 149 | avg_ndcg2 = np.mean(all_ndcg2_list) 150 | avg_mrr2 = np.mean(all_mrr2_list) 151 | 152 | return avg_hr1, avg_ndcg1, avg_mrr1, avg_hr2, avg_ndcg2, avg_mrr2 153 | 154 | 155 | -------------------------------------------------------------------------------- /model_my_variant2.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | import argparse 3 | import numpy as np 4 | import torch 5 | import torch.utils.data 6 | from torch import nn, optim 7 | import torch.nn.functional as F 8 | 9 | class CDAE(nn.Module): 10 | def __init__(self, NUM_USER, NUM_MOVIE, NUM_BOOK, EMBED_SIZE, dropout, is_sparse=False): 11 | super(CDAE, self).__init__() 12 | self.NUM_MOVIE = NUM_MOVIE 13 | self.NUM_BOOK = NUM_BOOK 14 | self.NUM_USER = NUM_USER 15 | self.emb_size = EMBED_SIZE 16 | 17 | self.user_embeddings = nn.Embedding(self.NUM_USER, EMBED_SIZE, sparse=is_sparse) 18 | self.user_embeddings.weight.data = torch.from_numpy(np.random.normal(0, 0.01, size=[self.NUM_USER, EMBED_SIZE])).float() 19 | 20 | self.encoder_x = nn.Sequential( 21 | nn.Linear(self.NUM_MOVIE, EMBED_SIZE), 22 | nn.ReLU(), 23 | nn.Linear(EMBED_SIZE, EMBED_SIZE) 24 | ) 25 | self.decoder_x = nn.Sequential( 26 | nn.Linear(EMBED_SIZE, EMBED_SIZE), 27 | nn.ReLU(), 28 | nn.Linear(EMBED_SIZE, self.NUM_MOVIE) 29 | ) 30 | self.encoder_y = nn.Sequential( 31 | nn.Linear(self.NUM_BOOK, EMBED_SIZE), 32 | nn.ReLU(), 33 | nn.Linear(EMBED_SIZE, EMBED_SIZE) 34 | ) 35 | self.decoder_y = nn.Sequential( 36 | nn.Linear(EMBED_SIZE, EMBED_SIZE), 37 | nn.ReLU(), 38 | nn.Linear(EMBED_SIZE, self.NUM_BOOK) 39 | ) 40 | 41 | self.orthogonal_w1 = nn.Parameter(nn.init.xavier_uniform(torch.Tensor(EMBED_SIZE, EMBED_SIZE).type( 42 | torch.cuda.FloatTensor if torch.cuda.is_available() else torch.FloatTensor), gain=np.sqrt(2.0)), 43 | requires_grad=True) 44 | 45 | self.orthogonal_w2 = nn.Parameter(nn.init.xavier_uniform(torch.Tensor(EMBED_SIZE, EMBED_SIZE).type( 46 | torch.cuda.FloatTensor if torch.cuda.is_available() else torch.FloatTensor), gain=np.sqrt(2.0)), 47 | requires_grad=True) 48 | self.dropout = nn.Dropout(dropout) 49 | self.relu = nn.ReLU 50 | 51 | def orthogonal_map(self, z_x, z_y): 52 | mapped_z_x = torch.matmul(z_x, self.orthogonal_w1) 53 | mapped_z_y = torch.matmul(z_y, self.orthogonal_w2) 54 | return mapped_z_x, mapped_z_y 55 | 56 | def forward(self, batch_user, batch_user_x, batch_user_y): 57 | h_user_x = self.encoder_x(self.dropout(batch_user_x)) 58 | h_user_y = self.encoder_y(self.dropout(batch_user_y)) 59 | h_user = self.user_embeddings(batch_user) 60 | feature_x = torch.add(h_user_x, h_user) 61 | feature_y = torch.add(h_user_y, h_user) 62 | z_x = F.relu(feature_x) 63 | z_y = F.relu(feature_y) 64 | preds_x = self.decoder_x(z_x) 65 | preds_y = self.decoder_y(z_y) 66 | mapped_z_x, mapped_z_y = self.orthogonal_map(z_x, z_y) 67 | preds_x2y = self.decoder_y(mapped_z_x) 68 | preds_y2x = self.decoder_x(mapped_z_y) 69 | 70 | # # define orthogonal constraint loss 71 | z_x_ = torch.matmul(mapped_z_x, self.orthogonal_w2) 72 | z_y_ = torch.matmul(mapped_z_y, self.orthogonal_w1) 73 | z_x_reg_loss = torch.norm(z_x - z_x_, p=1, dim=1) 74 | z_y_reg_loss = torch.norm(z_y - z_y_, p=1, dim=1) 75 | 76 | return preds_x, preds_y, preds_x2y, preds_y2x, feature_x, feature_y, z_x_reg_loss, z_y_reg_loss 77 | def get_user_embedding(self, batch_user_x, batch_user_y): 78 | h_user_x = self.encoder_x(self.dropout(batch_user_x)) 79 | h_user_y = self.encoder_y(self.dropout(batch_user_y)) 80 | return h_user_x, h_user_y 81 | 82 | # class MI_Map(nn.Module): 83 | # def __init__(self, n_input): 84 | # super(MI_Map, self).__init__() 85 | # 86 | # self.fc_x = nn.Linear(n_input, n_input, bias=False) 87 | # # self.fc_y = nn.Linear(n_input, n_input, bias=False) 88 | # 89 | # def forward(self, z_x, z_y): 90 | # # make mlp for discriminator 91 | # logits = torch.sum(self.fc_x(z_x)*z_y, dim=1) 92 | # return logits 93 | 94 | class Discriminator(nn.Module): 95 | def __init__(self, n_fts, dropout): 96 | super(Discriminator, self).__init__() 97 | self.dropout = dropout 98 | self.training = True 99 | 100 | self.disc = nn.Sequential( 101 | nn.Linear(n_fts, int(n_fts/2)), 102 | nn.ReLU(), 103 | nn.Dropout(dropout), 104 | nn.Linear(int(n_fts/2), 1)) 105 | 106 | def forward(self, x): 107 | # make mlp for discriminator 108 | h = self.disc(x) 109 | return h 110 | 111 | def save_embedding_process(model, save_loader, feed_data, is_cuda): 112 | fts1 = feed_data['fts1'] 113 | fts2 = feed_data['fts2'] 114 | 115 | user_embedding1_list = [] 116 | user_embedding2_list = [] 117 | model.eval() 118 | for batch_idx, data in enumerate(save_loader): 119 | data = data.reshape([-1]) 120 | val_user_arr = data.numpy() 121 | v_item1 = fts1[val_user_arr] 122 | v_item2 = fts2[val_user_arr] 123 | if is_cuda: 124 | v_user = torch.LongTensor(val_user_arr).cuda() 125 | v_item1 = torch.FloatTensor(v_item1).cuda() 126 | v_item2 = torch.FloatTensor(v_item2).cuda() 127 | else: 128 | v_user = torch.LongTensor(val_user_arr) 129 | v_item1 = torch.FloatTensor(v_item1) 130 | v_item2 = torch.FloatTensor(v_item2) 131 | 132 | res = model.get_user_embedding(v_item1, v_item2) 133 | user_embedding1 = res[0] 134 | user_embedding2 = res[1] 135 | if is_cuda: 136 | user_embedding1 = user_embedding1.detach().cpu().numpy() 137 | user_embedding2 = user_embedding2.detach().cpu().numpy() 138 | else: 139 | user_embedding1 = user_embedding1.detach().numpy() 140 | user_embedding2 = user_embedding2.detach().numpy() 141 | 142 | user_embedding1_list.append(user_embedding1) 143 | user_embedding2_list.append(user_embedding2) 144 | 145 | return np.concatenate(user_embedding1_list, axis=0), np.concatenate(user_embedding2_list, axis=0) -------------------------------------------------------------------------------- /Dataset_CDAE.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import os 3 | import pickle 4 | import random 5 | from tqdm import tqdm 6 | from sklearn.utils import shuffle 7 | 8 | class Dataset(object): 9 | def __init__(self, BATCH_SIZE, dataset='douban'): 10 | self.batch_size = BATCH_SIZE 11 | self.peo2item_movie, self.peo2item_book, self.item2peo_movie, self.item2peo_book= self.load_data(dataset) 12 | self.movie_set = self.item2peo_movie.keys() 13 | self.book_set = self.item2peo_book.keys() 14 | self.num_user = len(self.peo2item_movie) 15 | self.num_movie = len(self.item2peo_movie) 16 | self.num_book = len(self.item2peo_book) 17 | self.movie_vali, self.movie_test, self.movie_nega, self.book_vali, self.book_test, self.book_nega = self.get_test_data(dataset) 18 | 19 | print('load data down') 20 | 21 | self.test_count = 0 22 | self.batch_count = 0 23 | self.count = 0 24 | self.epoch = 1 25 | 26 | def load_data(self, dataset): 27 | print('loading data...{}') 28 | peo2movie = pickle.load(open(os.path.join('processed_data', dataset,'peo2movie_id.pkl'), 'rb')) 29 | peo2book = pickle.load(open(os.path.join('processed_data', dataset,'peo2book_id.pkl'), 'rb')) 30 | movie2peo = pickle.load(open(os.path.join('processed_data', dataset,'movie2peo_id.pkl'), 'rb')) 31 | book2peo = pickle.load(open(os.path.join('processed_data', dataset,'book2peo_id.pkl'), 'rb')) 32 | 33 | return peo2movie, peo2book, movie2peo, book2peo 34 | 35 | # def get_train_indices(self, domain): 36 | # row, col = [], [] 37 | # 38 | # if domain == 'movie': 39 | # dict = self.peo2item_movie 40 | # vali = self.movie_vali 41 | # elif domain == 'book': 42 | # dict = self.peo2item_book 43 | # vali = self.book_vali 44 | # else: 45 | # return 46 | # 47 | # for user in dict: 48 | # dict[user].remove(vali[user]) 49 | # for each in dict[user]: 50 | # row.append(user) 51 | # col.append(each) 52 | # 53 | # row = np.array(row) 54 | # col = np.array(col) 55 | # 56 | # return row, col 57 | 58 | def get_part_train_indices(self, domain, percent): 59 | row, col = [], [] 60 | 61 | if domain == 'movie': 62 | dict = self.peo2item_movie 63 | vali = self.movie_vali 64 | test = self.movie_test 65 | elif domain == 'book': 66 | dict = self.peo2item_book 67 | vali = self.book_vali 68 | test = self.book_test 69 | else: 70 | return 71 | 72 | for user in dict: 73 | if len(dict[user])>2: 74 | dict[user].remove(vali[user]) 75 | dict[user].remove(test[user]) 76 | else: 77 | dict[user].remove(vali[user]) 78 | 79 | dict[user] = shuffle(dict[user], random_state=72) 80 | num_item = int(round(percent * len(dict[user]))) 81 | 82 | for i in range(num_item): 83 | row.append(user) 84 | col.append(dict[user][i]) 85 | 86 | row = np.array(row) 87 | col = np.array(col) 88 | 89 | return row, col 90 | 91 | def get_test_data(self, dataset): 92 | if not os.path.exists(os.path.join(os.getcwd(), 'processed_data', dataset, 'movie_vali.pkl')) or \ 93 | not os.path.exists(os.path.join(os.getcwd(), 'processed_data', dataset, 'movie_test.pkl')): 94 | movie_vali = {} 95 | movie_test = {} 96 | book_vali = {} 97 | book_test = {} 98 | movie_nega = {} 99 | book_nega = {} 100 | for i in tqdm(self.peo2item_movie): 101 | items = self.peo2item_movie[i] 102 | if len(items)>=2: 103 | items_choices = shuffle(items, random_state=2020)[:2] 104 | movie_vali[i] = items_choices[0] 105 | movie_test[i] = items_choices[1] 106 | else: 107 | movie_vali[i] = items[0] 108 | movie_test[i] = items[0] 109 | 110 | all_nega_movies = list((set(list(range(self.num_movie))) - set(items))) 111 | movie_nega[i] = shuffle(all_nega_movies, random_state=2020)[:99] 112 | 113 | for i in tqdm(self.peo2item_book): 114 | items = self.peo2item_book[i] 115 | if len(items)>=2: 116 | items_choices = shuffle(items, random_state=2020)[:2] 117 | book_vali[i] = items_choices[0] 118 | book_test[i] = items_choices[1] 119 | else: 120 | book_vali[i] = items[0] 121 | book_test[i] = items[0] 122 | 123 | all_nega_books = list((set(list(range(self.num_book))) - set(items))) 124 | book_nega[i] = shuffle(all_nega_books, random_state=2020)[:99] 125 | 126 | pickle.dump(movie_vali, open(os.path.join('processed_data', dataset, 'movie_vali.pkl'), 'wb')) 127 | pickle.dump(book_vali, open(os.path.join('processed_data', dataset, 'book_vali.pkl'), 'wb')) 128 | 129 | pickle.dump(movie_test, open(os.path.join('processed_data', dataset, 'movie_test.pkl'), 'wb')) 130 | pickle.dump(book_test, open(os.path.join('processed_data', dataset, 'book_test.pkl'), 'wb')) 131 | 132 | pickle.dump(movie_nega, open(os.path.join('processed_data', dataset, 'movie_nega.pkl'), 'wb')) 133 | pickle.dump(book_nega, open(os.path.join('processed_data', dataset, 'book_nega.pkl'), 'wb')) 134 | 135 | else: 136 | movie_vali = pickle.load(open(os.path.join('processed_data', dataset, 'movie_vali.pkl'), 'rb')) 137 | book_vali = pickle.load(open(os.path.join('processed_data', dataset, 'book_vali.pkl'), 'rb')) 138 | 139 | movie_test = pickle.load(open(os.path.join('processed_data', dataset, 'movie_test.pkl'), 'rb')) 140 | book_test = pickle.load(open(os.path.join('processed_data', dataset, 'book_test.pkl'), 'rb')) 141 | 142 | movie_nega = pickle.load(open(os.path.join('processed_data', dataset, 'movie_nega.pkl'), 'rb')) 143 | book_nega = pickle.load(open(os.path.join('processed_data', dataset, 'book_nega.pkl'), 'rb')) 144 | 145 | return movie_vali, movie_test, movie_nega, book_vali, book_test, book_nega 146 | 147 | 148 | 149 | if __name__ == '__main__': 150 | dataset = Dataset(1000, 4, True) 151 | i = 1 152 | vali = dataset.movie_vali[i] 153 | nega = dataset.movie_nega[i] 154 | nega_vali = np.concatenate([nega, vali],1) 155 | 156 | 157 | 158 | 159 | 160 | 161 | -------------------------------------------------------------------------------- /model_my_variant3.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | import argparse 3 | import numpy as np 4 | import torch 5 | import torch.utils.data 6 | from torch import nn, optim 7 | import torch.nn.functional as F 8 | 9 | class CDAE(nn.Module): 10 | def __init__(self, NUM_USER, NUM_MOVIE, NUM_BOOK, EMBED_SIZE, dropout, is_sparse=False): 11 | super(CDAE, self).__init__() 12 | self.NUM_MOVIE = NUM_MOVIE 13 | self.NUM_BOOK = NUM_BOOK 14 | self.NUM_USER = NUM_USER 15 | self.emb_size = EMBED_SIZE 16 | 17 | self.user_embeddings = nn.Embedding(self.NUM_USER, EMBED_SIZE, sparse=is_sparse) 18 | self.user_embeddings.weight.data = torch.from_numpy(np.random.normal(0, 0.01, size=[self.NUM_USER, EMBED_SIZE])).float() 19 | 20 | self.encoder_x = nn.Sequential( 21 | nn.Linear(self.NUM_MOVIE, EMBED_SIZE), 22 | nn.ReLU(), 23 | nn.Linear(EMBED_SIZE, EMBED_SIZE) 24 | ) 25 | self.decoder_x = nn.Sequential( 26 | nn.Linear(EMBED_SIZE, EMBED_SIZE), 27 | nn.ReLU(), 28 | nn.Linear(EMBED_SIZE, self.NUM_MOVIE) 29 | ) 30 | self.encoder_y = nn.Sequential( 31 | nn.Linear(self.NUM_BOOK, EMBED_SIZE), 32 | nn.ReLU(), 33 | nn.Linear(EMBED_SIZE, EMBED_SIZE) 34 | ) 35 | self.decoder_y = nn.Sequential( 36 | nn.Linear(EMBED_SIZE, EMBED_SIZE), 37 | nn.ReLU(), 38 | nn.Linear(EMBED_SIZE, self.NUM_BOOK) 39 | ) 40 | 41 | self.transformer_x_layer1 = nn.Linear(EMBED_SIZE, EMBED_SIZE, bias=False) 42 | self.transformer_x_layer2 = nn.Linear(EMBED_SIZE, EMBED_SIZE, bias=False) 43 | 44 | self.transformer_y_layer1 = nn.Linear(EMBED_SIZE, EMBED_SIZE, bias=False) 45 | self.transformer_y_layer2 = nn.Linear(EMBED_SIZE, EMBED_SIZE, bias=False) 46 | 47 | # self.orthogonal_w1 = nn.Parameter(nn.init.xavier_uniform(torch.Tensor(EMBED_SIZE, EMBED_SIZE).type( 48 | # torch.cuda.FloatTensor if torch.cuda.is_available() else torch.FloatTensor), gain=np.sqrt(2.0)), 49 | # requires_grad=True) 50 | # 51 | # self.orthogonal_w2 = nn.Parameter(nn.init.xavier_uniform(torch.Tensor(EMBED_SIZE, EMBED_SIZE).type( 52 | # torch.cuda.FloatTensor if torch.cuda.is_available() else torch.FloatTensor), gain=np.sqrt(2.0)), 53 | # requires_grad=True) 54 | self.dropout = nn.Dropout(dropout) 55 | self.relu = nn.ReLU 56 | 57 | def orthogonal_map(self, z_x, z_y): 58 | mapped_z_x = self.transformer_x_layer1(z_x) 59 | mapped_z_x = self.transformer_x_layer2(F.relu(mapped_z_x)) 60 | mapped_z_y = self.transformer_y_layer1(z_y) 61 | mapped_z_y = self.transformer_y_layer2(F.relu(mapped_z_y)) 62 | return mapped_z_x, mapped_z_y 63 | 64 | def inver_orthogonal_map(self, z_x, z_y): 65 | mapped_z_x = self.transformer_y_layer1(z_x) 66 | mapped_z_x = self.transformer_y_layer2(F.relu(mapped_z_x)) 67 | mapped_z_y = self.transformer_x_layer1(z_y) 68 | mapped_z_y = self.transformer_x_layer2(F.relu(mapped_z_y)) 69 | return mapped_z_x, mapped_z_y 70 | 71 | def forward(self, batch_user, batch_user_x, batch_user_y): 72 | h_user_x = self.encoder_x(self.dropout(batch_user_x)) 73 | h_user_y = self.encoder_y(self.dropout(batch_user_y)) 74 | h_user = self.user_embeddings(batch_user) 75 | feature_x = torch.add(h_user_x, h_user) 76 | feature_y = torch.add(h_user_y, h_user) 77 | z_x = F.relu(feature_x) 78 | z_y = F.relu(feature_y) 79 | preds_x = self.decoder_x(z_x) 80 | preds_y = self.decoder_y(z_y) 81 | mapped_z_x, mapped_z_y = self.orthogonal_map(z_x, z_y) 82 | preds_x2y = self.decoder_y(mapped_z_x) 83 | preds_y2x = self.decoder_x(mapped_z_y) 84 | 85 | # define orthogonal constraint loss 86 | z_x_, z_y_ = self.inver_orthogonal_map(z_x, z_y) 87 | z_x_reg_loss = torch.norm(z_x - z_x_, p=1, dim=1) 88 | z_y_reg_loss = torch.norm(z_y - z_y_, p=1, dim=1) 89 | 90 | return preds_x, preds_y, preds_x2y, preds_y2x, feature_x, feature_y, z_x_reg_loss, z_y_reg_loss 91 | def get_user_embedding(self, batch_user_x, batch_user_y): 92 | h_user_x = self.encoder_x(self.dropout(batch_user_x)) 93 | h_user_y = self.encoder_y(self.dropout(batch_user_y)) 94 | return h_user_x, h_user_y 95 | 96 | # class MI_Map(nn.Module): 97 | # def __init__(self, n_input): 98 | # super(MI_Map, self).__init__() 99 | # 100 | # self.fc_x = nn.Linear(n_input, n_input, bias=False) 101 | # # self.fc_y = nn.Linear(n_input, n_input, bias=False) 102 | # 103 | # def forward(self, z_x, z_y): 104 | # # make mlp for discriminator 105 | # logits = torch.sum(self.fc_x(z_x)*z_y, dim=1) 106 | # return logits 107 | 108 | class Discriminator(nn.Module): 109 | def __init__(self, n_fts, dropout): 110 | super(Discriminator, self).__init__() 111 | self.dropout = dropout 112 | self.training = True 113 | 114 | self.disc = nn.Sequential( 115 | nn.Linear(n_fts, int(n_fts/2)), 116 | nn.ReLU(), 117 | nn.Dropout(dropout), 118 | nn.Linear(int(n_fts/2), 1)) 119 | 120 | def forward(self, x): 121 | # make mlp for discriminator 122 | h = self.disc(x) 123 | return h 124 | 125 | def save_embedding_process(model, save_loader, feed_data, is_cuda): 126 | fts1 = feed_data['fts1'] 127 | fts2 = feed_data['fts2'] 128 | 129 | user_embedding1_list = [] 130 | user_embedding2_list = [] 131 | model.eval() 132 | for batch_idx, data in enumerate(save_loader): 133 | data = data.reshape([-1]) 134 | val_user_arr = data.numpy() 135 | v_item1 = fts1[val_user_arr] 136 | v_item2 = fts2[val_user_arr] 137 | if is_cuda: 138 | v_user = torch.LongTensor(val_user_arr).cuda() 139 | v_item1 = torch.FloatTensor(v_item1).cuda() 140 | v_item2 = torch.FloatTensor(v_item2).cuda() 141 | else: 142 | v_user = torch.LongTensor(val_user_arr) 143 | v_item1 = torch.FloatTensor(v_item1) 144 | v_item2 = torch.FloatTensor(v_item2) 145 | 146 | res = model.get_user_embedding(v_item1, v_item2) 147 | user_embedding1 = res[0] 148 | user_embedding2 = res[1] 149 | if is_cuda: 150 | user_embedding1 = user_embedding1.detach().cpu().numpy() 151 | user_embedding2 = user_embedding2.detach().cpu().numpy() 152 | else: 153 | user_embedding1 = user_embedding1.detach().numpy() 154 | user_embedding2 = user_embedding2.detach().numpy() 155 | 156 | user_embedding1_list.append(user_embedding1) 157 | user_embedding2_list.append(user_embedding2) 158 | 159 | return np.concatenate(user_embedding1_list, axis=0), np.concatenate(user_embedding2_list, axis=0) -------------------------------------------------------------------------------- /model_my.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | import argparse 3 | import numpy as np 4 | import torch 5 | import torch.utils.data 6 | from torch import nn, optim 7 | import torch.nn.functional as F 8 | 9 | class CDAE(nn.Module): 10 | def __init__(self, NUM_USER, NUM_MOVIE, NUM_BOOK, EMBED_SIZE, dropout, is_sparse=False): 11 | super(CDAE, self).__init__() 12 | self.NUM_MOVIE = NUM_MOVIE 13 | self.NUM_BOOK = NUM_BOOK 14 | self.NUM_USER = NUM_USER 15 | self.emb_size = EMBED_SIZE 16 | 17 | self.user_embeddings = nn.Embedding(self.NUM_USER, EMBED_SIZE, sparse=is_sparse) 18 | self.user_embeddings.weight.data = torch.from_numpy(np.random.normal(0, 0.01, size=[self.NUM_USER, EMBED_SIZE])).float() 19 | 20 | self.encoder_x = nn.Sequential( 21 | nn.Linear(self.NUM_MOVIE, EMBED_SIZE), 22 | nn.ReLU(), 23 | nn.Linear(EMBED_SIZE, EMBED_SIZE) 24 | ) 25 | self.decoder_x = nn.Sequential( 26 | nn.Linear(EMBED_SIZE, EMBED_SIZE), 27 | nn.ReLU(), 28 | nn.Linear(EMBED_SIZE, self.NUM_MOVIE) 29 | ) 30 | self.encoder_y = nn.Sequential( 31 | nn.Linear(self.NUM_BOOK, EMBED_SIZE), 32 | nn.ReLU(), 33 | nn.Linear(EMBED_SIZE, EMBED_SIZE) 34 | ) 35 | self.decoder_y = nn.Sequential( 36 | nn.Linear(EMBED_SIZE, EMBED_SIZE), 37 | nn.ReLU(), 38 | nn.Linear(EMBED_SIZE, self.NUM_BOOK) 39 | ) 40 | 41 | self.orthogonal_w = nn.Parameter(nn.init.xavier_uniform(torch.Tensor(EMBED_SIZE, EMBED_SIZE).type( 42 | torch.cuda.FloatTensor if torch.cuda.is_available() else torch.FloatTensor), gain=np.sqrt(2.0)), 43 | requires_grad=True) 44 | self.dropout = nn.Dropout(dropout) 45 | self.relu = nn.ReLU 46 | 47 | def orthogonal_map(self, z_x, z_y): 48 | mapped_z_x = torch.matmul(z_x, self.orthogonal_w) 49 | mapped_z_y = torch.matmul(z_y, torch.transpose(self.orthogonal_w, 1, 0)) 50 | return mapped_z_x, mapped_z_y 51 | 52 | def forward(self, batch_user, batch_user_x, batch_user_y): 53 | h_user_x = self.encoder_x(self.dropout(batch_user_x)) 54 | h_user_y = self.encoder_y(self.dropout(batch_user_y)) 55 | h_user = self.user_embeddings(batch_user) 56 | feature_x = torch.add(h_user_x, h_user) 57 | feature_y = torch.add(h_user_y, h_user) 58 | z_x = F.relu(feature_x) 59 | z_y = F.relu(feature_y) 60 | preds_x = self.decoder_x(z_x) 61 | preds_y = self.decoder_y(z_y) 62 | mapped_z_x, mapped_z_y = self.orthogonal_map(z_x, z_y) 63 | preds_x2y = self.decoder_y(mapped_z_x) 64 | preds_y2x = self.decoder_x(mapped_z_y) 65 | 66 | # define orthogonal constraint loss 67 | z_x_ = torch.matmul(mapped_z_x, torch.transpose(self.orthogonal_w, 1, 0)) 68 | z_y_ = torch.matmul(mapped_z_y, self.orthogonal_w) 69 | z_x_reg_loss = torch.norm(z_x - z_x_, p=1, dim=1) 70 | z_y_reg_loss = torch.norm(z_y - z_y_, p=1, dim=1) 71 | 72 | return preds_x, preds_y, preds_x2y, preds_y2x, feature_x, feature_y, z_x_reg_loss, z_y_reg_loss 73 | def get_user_embedding(self, batch_user_x, batch_user_y): 74 | # this is for SIGIR's experiment 75 | h_user_x = self.encoder_x(self.dropout(batch_user_x)) 76 | h_user_y = self.encoder_y(self.dropout(batch_user_y)) 77 | return h_user_x, h_user_y 78 | 79 | def get_latent_z(self, batch_user, batch_user_x, batch_user_y): 80 | # this is for clustering visualization 81 | h_user_x = self.encoder_x(self.dropout(batch_user_x)) 82 | h_user_y = self.encoder_y(self.dropout(batch_user_y)) 83 | h_user = self.user_embeddings(batch_user) 84 | feature_x = torch.add(h_user_x, h_user) 85 | feature_y = torch.add(h_user_y, h_user) 86 | z_x = F.relu(feature_x) 87 | z_y = F.relu(feature_y) 88 | 89 | return z_x, z_y 90 | 91 | 92 | 93 | 94 | class Discriminator(nn.Module): 95 | def __init__(self, n_fts, dropout): 96 | super(Discriminator, self).__init__() 97 | self.dropout = dropout 98 | self.training = True 99 | 100 | self.disc = nn.Sequential( 101 | nn.Linear(n_fts, int(n_fts/2)), 102 | nn.ReLU(), 103 | nn.Dropout(dropout), 104 | nn.Linear(int(n_fts/2), 1)) 105 | 106 | def forward(self, x): 107 | # make mlp for discriminator 108 | h = self.disc(x) 109 | return h 110 | 111 | def save_embedding_process(model, save_loader, feed_data, is_cuda): 112 | fts1 = feed_data['fts1'] 113 | fts2 = feed_data['fts2'] 114 | 115 | user_embedding1_list = [] 116 | user_embedding2_list = [] 117 | model.eval() 118 | for batch_idx, data in enumerate(save_loader): 119 | data = data.reshape([-1]) 120 | val_user_arr = data.numpy() 121 | v_item1 = fts1[val_user_arr] 122 | v_item2 = fts2[val_user_arr] 123 | if is_cuda: 124 | v_user = torch.LongTensor(val_user_arr).cuda() 125 | v_item1 = torch.FloatTensor(v_item1).cuda() 126 | v_item2 = torch.FloatTensor(v_item2).cuda() 127 | else: 128 | v_user = torch.LongTensor(val_user_arr) 129 | v_item1 = torch.FloatTensor(v_item1) 130 | v_item2 = torch.FloatTensor(v_item2) 131 | 132 | res = model.get_user_embedding(v_item1, v_item2) 133 | user_embedding1 = res[0] 134 | user_embedding2 = res[1] 135 | if is_cuda: 136 | user_embedding1 = user_embedding1.detach().cpu().numpy() 137 | user_embedding2 = user_embedding2.detach().cpu().numpy() 138 | else: 139 | user_embedding1 = user_embedding1.detach().numpy() 140 | user_embedding2 = user_embedding2.detach().numpy() 141 | 142 | user_embedding1_list.append(user_embedding1) 143 | user_embedding2_list.append(user_embedding2) 144 | 145 | return np.concatenate(user_embedding1_list, axis=0), np.concatenate(user_embedding2_list, axis=0) 146 | 147 | def save_z_process(model, save_loader, feed_data, is_cuda): 148 | fts1 = feed_data['fts1'] 149 | fts2 = feed_data['fts2'] 150 | 151 | user_embedding1_list = [] 152 | user_embedding2_list = [] 153 | model.eval() 154 | for batch_idx, data in enumerate(save_loader): 155 | data = data.reshape([-1]) 156 | val_user_arr = data.numpy() 157 | v_item1 = fts1[val_user_arr] 158 | v_item2 = fts2[val_user_arr] 159 | if is_cuda: 160 | v_user = torch.LongTensor(val_user_arr).cuda() 161 | v_item1 = torch.FloatTensor(v_item1).cuda() 162 | v_item2 = torch.FloatTensor(v_item2).cuda() 163 | else: 164 | v_user = torch.LongTensor(val_user_arr) 165 | v_item1 = torch.FloatTensor(v_item1) 166 | v_item2 = torch.FloatTensor(v_item2) 167 | 168 | res = model.get_latent_z(v_user, v_item1, v_item2) 169 | user_embedding1 = res[0] 170 | user_embedding2 = res[1] 171 | if is_cuda: 172 | user_embedding1 = user_embedding1.detach().cpu().numpy() 173 | user_embedding2 = user_embedding2.detach().cpu().numpy() 174 | else: 175 | user_embedding1 = user_embedding1.detach().numpy() 176 | user_embedding2 = user_embedding2.detach().numpy() 177 | 178 | user_embedding1_list.append(user_embedding1) 179 | user_embedding2_list.append(user_embedding2) 180 | 181 | return np.concatenate(user_embedding1_list, axis=0), np.concatenate(user_embedding2_list, axis=0) -------------------------------------------------------------------------------- /model_my_extention.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | import argparse 3 | import numpy as np 4 | import torch 5 | import torch.utils.data 6 | from torch import nn, optim 7 | import torch.nn.functional as F 8 | 9 | # class Cluster_layer(nn.Module): 10 | # def __init__(self, emb_size, num_cluster=2, iters=4, tau=1.0, **kwargs): 11 | # super(Cluster_layer, self).__init__() 12 | # self.n_cluster = num_cluster 13 | # self.iters = iters 14 | # self.tau = tau 15 | # self.centers = nn.Parameter(torch.nn.init.xavier_uniform_(torch.empty(self.n_cluster, emb_size))) 16 | # 17 | # def forward(self, u_vecs): 18 | # for i in range(self.iters): 19 | # distance = torch.matmul(u_vecs.unsqueeze(1), F.normalize(self.centers, p=1, dim=1).T) 20 | # assigns = F.softmax(distance*self.tau) 21 | # 22 | # 23 | # return o, c 24 | 25 | 26 | class CDAE(nn.Module): 27 | def __init__(self, NUM_USER, NUM_MOVIE, NUM_BOOK, EMBED_SIZE, dropout, x_cluster_num=10, y_cluster_num=10, is_sparse=False): 28 | super(CDAE, self).__init__() 29 | self.NUM_MOVIE = NUM_MOVIE 30 | self.NUM_BOOK = NUM_BOOK 31 | self.NUM_USER = NUM_USER 32 | self.emb_size = EMBED_SIZE 33 | 34 | self.user_embeddings = nn.Embedding(self.NUM_USER, EMBED_SIZE, sparse=is_sparse) 35 | self.user_embeddings.weight.data = torch.from_numpy( 36 | np.random.normal(0, 0.01, size=[self.NUM_USER, EMBED_SIZE])).float() 37 | 38 | self.encoder_x = nn.Sequential( 39 | nn.Linear(self.NUM_MOVIE, EMBED_SIZE), 40 | nn.ReLU(), 41 | nn.Linear(EMBED_SIZE, EMBED_SIZE) 42 | ) 43 | self.decoder_x = nn.Sequential( 44 | nn.Linear(EMBED_SIZE, EMBED_SIZE), 45 | nn.ReLU(), 46 | nn.Linear(EMBED_SIZE, self.NUM_MOVIE) 47 | ) 48 | self.encoder_y = nn.Sequential( 49 | nn.Linear(self.NUM_BOOK, EMBED_SIZE), 50 | nn.ReLU(), 51 | nn.Linear(EMBED_SIZE, EMBED_SIZE) 52 | ) 53 | self.decoder_y = nn.Sequential( 54 | nn.Linear(EMBED_SIZE, EMBED_SIZE), 55 | nn.ReLU(), 56 | nn.Linear(EMBED_SIZE, self.NUM_BOOK) 57 | ) 58 | 59 | self.x_clustering = Cluster_layer(num_cluster=x_cluster_num) 60 | self.y_clustering = Cluster_layer(num_cluster=y_cluster_num) 61 | self.epsilon = torch.tensor(1e-10).type(torch.FloatTensor) # .cuda() 62 | 63 | self.dropout = nn.Dropout(dropout) 64 | self.relu = nn.ReLU 65 | 66 | 67 | def forward(self, batch_user, batch_user_x, batch_user_y): 68 | # encoding for x and y domain 69 | h_user_x = self.encoder_x(self.dropout(batch_user_x)) 70 | h_user_y = self.encoder_y(self.dropout(batch_user_y)) 71 | 72 | # disentangling for user representations in x and y domains 73 | x_centers, x_assigns = self.x_clustering(h_user_x.unsqueeze(1)) 74 | y_centers, y_assigns = self.y_clustering(h_user_y.unsqueeze(1)) 75 | 76 | x_centers_ = torch.sum(x_centers ** 2, dim=2, keepdim=True) 77 | y_centers_ = torch.sum(y_centers ** 2, dim=2, keepdim=True) 78 | 79 | distance_ = torch.sqrt(torch.max(x_centers_ - 2 * torch.bmm(x_centers, y_centers.permute(0, 2, 1)) + y_centers_.permute(0, 2, 1), 80 | self.epsilon)) 81 | distance = torch.min(distance_, dim=1) 82 | distance = distance.values 83 | 84 | # adding the ont-hot user encoding for user representations in x and y domains 85 | # h_user_x_ = 86 | h_user = self.user_embeddings(batch_user) 87 | feature_x = torch.add(h_user_x, h_user) 88 | feature_y = torch.add(h_user_y, h_user) 89 | z_x = F.relu(feature_x) 90 | z_y = F.relu(feature_y) 91 | 92 | # decoding for x and y domain 93 | preds_x = self.decoder_x(z_x) 94 | preds_y = self.decoder_y(z_y) 95 | 96 | return preds_x, preds_y, feature_x, feature_y, distance 97 | 98 | def get_user_embedding(self, batch_user_x, batch_user_y): 99 | # this is for SIGIR's experiment 100 | h_user_x = self.encoder_x(self.dropout(batch_user_x)) 101 | h_user_y = self.encoder_y(self.dropout(batch_user_y)) 102 | return h_user_x, h_user_y 103 | 104 | def get_latent_z(self, batch_user, batch_user_x, batch_user_y): 105 | # this is for clustering visualization 106 | h_user_x = self.encoder_x(self.dropout(batch_user_x)) 107 | h_user_y = self.encoder_y(self.dropout(batch_user_y)) 108 | h_user = self.user_embeddings(batch_user) 109 | feature_x = torch.add(h_user_x, h_user) 110 | feature_y = torch.add(h_user_y, h_user) 111 | z_x = F.relu(feature_x) 112 | z_y = F.relu(feature_y) 113 | 114 | return z_x, z_y 115 | 116 | 117 | class Discriminator(nn.Module): 118 | def __init__(self, n_fts, dropout): 119 | super(Discriminator, self).__init__() 120 | self.dropout = dropout 121 | self.training = True 122 | 123 | self.disc = nn.Sequential( 124 | nn.Linear(n_fts, int(n_fts / 2)), 125 | nn.ReLU(), 126 | nn.Dropout(dropout), 127 | nn.Linear(int(n_fts / 2), 1)) 128 | 129 | def forward(self, x): 130 | # make mlp for discriminator 131 | h = self.disc(x) 132 | return h 133 | 134 | 135 | def save_embedding_process(model, save_loader, feed_data, is_cuda): 136 | fts1 = feed_data['fts1'] 137 | fts2 = feed_data['fts2'] 138 | 139 | user_embedding1_list = [] 140 | user_embedding2_list = [] 141 | model.eval() 142 | for batch_idx, data in enumerate(save_loader): 143 | data = data.reshape([-1]) 144 | val_user_arr = data.numpy() 145 | v_item1 = fts1[val_user_arr] 146 | v_item2 = fts2[val_user_arr] 147 | if is_cuda: 148 | v_user = torch.LongTensor(val_user_arr).cuda() 149 | v_item1 = torch.FloatTensor(v_item1).cuda() 150 | v_item2 = torch.FloatTensor(v_item2).cuda() 151 | else: 152 | v_user = torch.LongTensor(val_user_arr) 153 | v_item1 = torch.FloatTensor(v_item1) 154 | v_item2 = torch.FloatTensor(v_item2) 155 | 156 | res = model.get_user_embedding(v_item1, v_item2) 157 | user_embedding1 = res[0] 158 | user_embedding2 = res[1] 159 | if is_cuda: 160 | user_embedding1 = user_embedding1.detach().cpu().numpy() 161 | user_embedding2 = user_embedding2.detach().cpu().numpy() 162 | else: 163 | user_embedding1 = user_embedding1.detach().numpy() 164 | user_embedding2 = user_embedding2.detach().numpy() 165 | 166 | user_embedding1_list.append(user_embedding1) 167 | user_embedding2_list.append(user_embedding2) 168 | 169 | return np.concatenate(user_embedding1_list, axis=0), np.concatenate(user_embedding2_list, axis=0) 170 | 171 | 172 | def save_z_process(model, save_loader, feed_data, is_cuda): 173 | fts1 = feed_data['fts1'] 174 | fts2 = feed_data['fts2'] 175 | 176 | user_embedding1_list = [] 177 | user_embedding2_list = [] 178 | model.eval() 179 | for batch_idx, data in enumerate(save_loader): 180 | data = data.reshape([-1]) 181 | val_user_arr = data.numpy() 182 | v_item1 = fts1[val_user_arr] 183 | v_item2 = fts2[val_user_arr] 184 | if is_cuda: 185 | v_user = torch.LongTensor(val_user_arr).cuda() 186 | v_item1 = torch.FloatTensor(v_item1).cuda() 187 | v_item2 = torch.FloatTensor(v_item2).cuda() 188 | else: 189 | v_user = torch.LongTensor(val_user_arr) 190 | v_item1 = torch.FloatTensor(v_item1) 191 | v_item2 = torch.FloatTensor(v_item2) 192 | 193 | res = model.get_latent_z(v_user, v_item1, v_item2) 194 | user_embedding1 = res[0] 195 | user_embedding2 = res[1] 196 | if is_cuda: 197 | user_embedding1 = user_embedding1.detach().cpu().numpy() 198 | user_embedding2 = user_embedding2.detach().cpu().numpy() 199 | else: 200 | user_embedding1 = user_embedding1.detach().numpy() 201 | user_embedding2 = user_embedding2.detach().numpy() 202 | 203 | user_embedding1_list.append(user_embedding1) 204 | user_embedding2_list.append(user_embedding2) 205 | 206 | return np.concatenate(user_embedding1_list, axis=0), np.concatenate(user_embedding2_list, axis=0) -------------------------------------------------------------------------------- /main_my_variant1.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | import matplotlib 3 | matplotlib.use('Agg') 4 | import argparse 5 | import torch 6 | import torch.utils.data 7 | from torch import nn, optim 8 | from torch.autograd import Variable 9 | import time 10 | import random 11 | import os 12 | import math 13 | import pickle 14 | import scipy.sparse as sp 15 | import matplotlib.pyplot as plt 16 | import matplotlib.ticker as ticker 17 | import numpy as np 18 | from Dataset_CDAE import Dataset 19 | from model_my_variant1 import CDAE, Discriminator, save_embedding_process 20 | import time 21 | import itertools 22 | import pandas as pd 23 | from my_utils import * 24 | from scipy.sparse import csr_matrix 25 | 26 | ''' 27 | variant of my. keep W, WT and without L1 loss constraint 28 | ''' 29 | 30 | method_name = 'my_variant1' 31 | # os.environ['CUDA_VISIBLE_DEVICES'] = '1' 32 | topK_list = [5, 10] 33 | 34 | parser = argparse.ArgumentParser() 35 | parser.add_argument('--prior', type=str, default='Gaussian', help='Gaussian, MVGaussian') 36 | parser.add_argument('--seed', type=int, default=72, help='Random seed.') 37 | parser.add_argument('--epochs', type=int, default=300, help='Number of epochs to train.') 38 | parser.add_argument('--batch', type=int, default=256, help='batch size.') 39 | parser.add_argument('--emb_size', type=int, default=200, help='embed size.') 40 | parser.add_argument('--lr', type=float, default=0.001, help='Initial learning rate.') 41 | parser.add_argument('--weight_decay', type=float, default=0.001, help='Weight decay (L2 loss on parameters).') 42 | parser.add_argument('--dropout', type=float, default=0.5, help='Dropout rate (1 - keep probability).') 43 | parser.add_argument('--log', type=str, default='logs/{}'.format(method_name), help='log directory') 44 | parser.add_argument('--pos-weight', type=float, default=1.0, help='weight for positive samples') 45 | 46 | parser.add_argument('--reg', type=float, default=0.0, help='lambda reg always 0.0 for this variant') 47 | 48 | parser.add_argument('--self', type=float, default=1.0, help='lambda rec') 49 | parser.add_argument('--cross', type=float, default=1.0, help='lambda rec') 50 | parser.add_argument('--gan', type=float, default=1.0, help='lambda gan') 51 | parser.add_argument('--d_epoch', type=int, default=2, help='d epoch') 52 | parser.add_argument('--t_percent', type=float, default=1.0, help='target percent') 53 | parser.add_argument('--s_percent', type=float, default=1.0, help='source percent') 54 | parser.add_argument('--dataset', type=str, default='amazon3', help='amazon') 55 | 56 | args = parser.parse_args() 57 | 58 | args.cuda = torch.cuda.is_available() 59 | 60 | kwargs = {'num_workers': 1, 'pin_memory': True} if args.cuda else {} 61 | 62 | def sigmoid(x): 63 | return 1 / (1 + np.exp(-x)) 64 | 65 | def main(): 66 | log = os.path.join(args.log, '{}_{}_{}_{}_{}_{}_{}_{}_{}'.format(args.dataset, args.emb_size, args.weight_decay, args.self, 67 | args.cross, args.gan, args.reg, args.t_percent, args.s_percent)) 68 | if os.path.isdir(log): 69 | print("%s already exist. are you sure to override? Ok, I'll wait for 5 seconds. Ctrl-C to abort." % log) 70 | time.sleep(5) 71 | os.system('rm -rf %s/' % log) 72 | 73 | os.makedirs(log) 74 | print("made the log directory", log) 75 | 76 | print('preparing data...') 77 | dataset = Dataset(args.batch, dataset=args.dataset) 78 | 79 | NUM_USER = dataset.num_user 80 | NUM_MOVIE = dataset.num_movie 81 | NUM_BOOK = dataset.num_book 82 | 83 | print('Preparing the training data......') 84 | # prepare data for X 85 | row, col = dataset.get_part_train_indices('movie', args.s_percent) 86 | values = np.ones(row.shape[0]) 87 | user_x = csr_matrix((values, (row, col)), shape=(NUM_USER, NUM_MOVIE)).toarray() 88 | 89 | # prepare data fot Y 90 | row, col = dataset.get_part_train_indices('book', args.t_percent) 91 | values = np.ones(row.shape[0]) 92 | user_y = csr_matrix((values, (row, col)), shape=(NUM_USER, NUM_BOOK)).toarray() 93 | 94 | print('Preparing the training data over......') 95 | # for shared user one-hot representation 96 | user_id = np.arange(NUM_USER).reshape([NUM_USER, 1]) 97 | 98 | user_x = torch.FloatTensor(user_x) 99 | user_y = torch.FloatTensor(user_y) 100 | 101 | train_loader = torch.utils.data.DataLoader(torch.from_numpy(user_id), 102 | batch_size=args.batch, 103 | shuffle=True, **kwargs) 104 | save_loader = torch.utils.data.DataLoader(torch.from_numpy(user_id), 105 | batch_size=args.batch, 106 | shuffle=False, **kwargs) 107 | 108 | pos_weight = torch.FloatTensor([args.pos_weight]) 109 | 110 | if args.cuda: 111 | pos_weight = pos_weight.cuda() 112 | 113 | model = CDAE(NUM_USER=NUM_USER, NUM_MOVIE=NUM_MOVIE, NUM_BOOK=NUM_BOOK, 114 | EMBED_SIZE=args.emb_size, dropout=args.dropout) 115 | disc1 = Discriminator(args.emb_size, args.dropout) 116 | disc2 = Discriminator(args.emb_size, args.dropout) 117 | optimizer_g = optim.Adam(model.parameters(), lr=args.lr, weight_decay=args.weight_decay) 118 | optimizer_d = optim.SGD(itertools.chain(disc1.parameters(), disc2.parameters()), lr=args.lr, weight_decay=args.weight_decay) 119 | 120 | BCEWL = torch.nn.BCEWithLogitsLoss(reduction='none', pos_weight=pos_weight) 121 | BCEL = torch.nn.BCEWithLogitsLoss(reduction='none') 122 | 123 | if args.cuda: 124 | model = model.cuda() 125 | disc1 = disc1.cuda() 126 | disc2 = disc2.cuda() 127 | 128 | # prepare data fot test process 129 | movie_vali, movie_test, movie_nega = dataset.movie_vali, dataset.movie_test, dataset.movie_nega 130 | book_vali, book_test, book_nega = dataset.book_vali, dataset.book_test, dataset.book_nega 131 | feed_data = {} 132 | feed_data['fts1'] = user_x 133 | feed_data['fts2'] = user_y 134 | feed_data['movie_vali'] = movie_vali 135 | feed_data['book_vali'] = book_vali 136 | feed_data['movie_test'] = movie_test 137 | feed_data['book_test'] = book_test 138 | feed_data['movie_nega'] = movie_nega 139 | feed_data['book_nega'] = book_nega 140 | 141 | best_hr1, best_ndcg1, best_mrr1 = 0.0, 0.0, 0.0 142 | best_hr2, best_ndcg2, best_mrr2 = 0.0, 0.0, 0.0 143 | val_hr1_list, val_ndcg1_list, val_mrr1_list = [], [], [] 144 | val_hr2_list, val_ndcg2_list, val_mrr2_list = [], [], [] 145 | 146 | G_loss_list = [] 147 | D_loss_list = [] 148 | reg_loss_list = [] 149 | loss_list = [] 150 | self_loss_list = [] 151 | cross_loss_list = [] 152 | JRL_loss_list = [] 153 | 154 | for epoch in range(args.epochs): 155 | model.train() 156 | batch_G_loss_list = [] 157 | batch_D_loss_list = [] 158 | batch_loss_list = [] 159 | batch_reg_loss_list = [] 160 | batch_self_loss_list = [] 161 | batch_cross_loss_list = [] 162 | batch_JRL_loss_list = [] 163 | 164 | for batch_idx, data in enumerate(train_loader): 165 | data = data.reshape([-1]) 166 | 167 | if (batch_idx+1) % (args.d_epoch + 1) == 0: 168 | optimizer_d.zero_grad() 169 | prior = torch.from_numpy(np.random.normal(0, 1.0, size=[data.shape[0], args.emb_size])).float() 170 | if args.cuda: 171 | prior = prior.cuda() 172 | 173 | if args.cuda: 174 | batch_user = data.cuda() 175 | batch_user_x = user_x[data].cuda() 176 | batch_user_y = user_y[data].cuda() 177 | else: 178 | batch_user = data 179 | batch_user_x = user_x[data] 180 | batch_user_y = user_y[data] 181 | 182 | _, _, _, _, z_x, z_y, z_x_reg_loss, z_y_reg_loss = model.forward(batch_user, batch_user_x, batch_user_y) 183 | 184 | true1 = disc1(prior).reshape([-1]) 185 | true2 = disc2(prior).reshape([-1]) 186 | fake1 = disc1(z_x).reshape([-1]) 187 | fake2 = disc2(z_y).reshape([-1]) 188 | 189 | dis_loss1 = BCEL(torch.cat([true1, fake1], 0), 190 | torch.cat([torch.ones_like(true1), torch.zeros_like(fake1)], 0)).sum() 191 | dis_loss2 = BCEL(torch.cat([true2, fake2], 0), 192 | torch.cat([torch.ones_like(true2), torch.zeros_like(fake2)], 0)).sum() 193 | 194 | D_loss = dis_loss1 + dis_loss2 195 | 196 | D_loss.backward() 197 | optimizer_d.step() 198 | batch_D_loss_list.append(D_loss.item()) 199 | 200 | else: 201 | optimizer_g.zero_grad() 202 | 203 | if args.cuda: 204 | batch_user = data.cuda() 205 | batch_user_x = user_x[data].cuda() 206 | batch_user_y = user_y[data].cuda() 207 | else: 208 | batch_user_x = user_x[data] 209 | batch_user_y = user_y[data] 210 | 211 | pred_x, pred_y, pred_x2y, pred_y2x, z_x, z_y, z_x_reg_loss, z_y_reg_loss = model.forward(batch_user, batch_user_x, batch_user_y) 212 | 213 | loss_x = BCEWL(pred_x, batch_user_x).sum() 214 | loss_y = BCEWL(pred_y, batch_user_y).sum() 215 | loss_x2y = BCEWL(pred_x2y, batch_user_y).sum() 216 | loss_y2x = BCEWL(pred_y2x, batch_user_x).sum() 217 | 218 | reg_loss = z_x_reg_loss.sum() + z_y_reg_loss.sum() 219 | 220 | fake_x = disc1(z_x).reshape([-1]) 221 | fake_y = disc2(z_y).reshape([-1]) 222 | 223 | G_loss1 = BCEL(fake_x, torch.ones_like(fake_x)).sum() 224 | G_loss2 = BCEL(fake_y, torch.ones_like(fake_y)).sum() 225 | G_loss = G_loss1+G_loss2 226 | 227 | # get plot JRL loss 228 | JRL_loss = args.self * (loss_x + loss_y) + args.cross * (loss_x2y + loss_y2x) + args.reg*reg_loss 229 | batch_JRL_loss_list.append(JRL_loss.item()/args.batch) 230 | 231 | # get the whole loss 232 | loss = G_loss + args.self * (loss_x + loss_y) + args.cross * (loss_x2y + loss_y2x) + args.reg*reg_loss 233 | loss.backward() 234 | optimizer_g.step() 235 | 236 | batch_G_loss_list.append(G_loss.item()) 237 | batch_self_loss_list.append(args.self * (loss_x + loss_y).item()) 238 | batch_cross_loss_list.append(args.cross * (loss_x2y + loss_y2x).item()) 239 | batch_loss_list.append(loss.item()) 240 | batch_reg_loss_list.append(args.reg*reg_loss.item()) 241 | 242 | epoch_G_loss = np.mean(batch_G_loss_list) 243 | epoch_D_loss = np.mean(batch_D_loss_list) 244 | epoch_reg_loss = np.mean(batch_reg_loss_list) 245 | epoch_loss = np.mean(batch_loss_list) 246 | epoch_self_loss = np.mean(batch_self_loss_list) 247 | epoch_cross_loss = np.mean(batch_cross_loss_list) 248 | epoch_JRL_Loss = np.mean(batch_JRL_loss_list) 249 | 250 | G_loss_list.append(epoch_G_loss) 251 | D_loss_list.append(epoch_D_loss) 252 | loss_list.append(epoch_loss) 253 | self_loss_list.append(epoch_self_loss) 254 | cross_loss_list.append(epoch_cross_loss) 255 | reg_loss_list.append(epoch_reg_loss) 256 | JRL_loss_list.append(epoch_JRL_Loss) 257 | 258 | print('epoch:{}, self loss:{:.4f}, cross loss:{:.4f}, G loss:{:.4f}, D loss:{:.4f}, reg loss:{:.4f}'.format(epoch,epoch_self_loss, 259 | epoch_cross_loss, 260 | epoch_G_loss, 261 | epoch_D_loss, 262 | epoch_reg_loss)) 263 | with open(log + '/tmp.txt', 'a') as f: 264 | f.write('epoch:{}, self loss:{:.4f}, cross loss:{:.4f}, G loss:{:.4f}, D loss:{:.4f}, reg loss:{:.4f}'.format( 265 | epoch, 266 | epoch_self_loss, epoch_cross_loss, epoch_G_loss, epoch_D_loss, epoch_reg_loss)) 267 | 268 | if epoch % 1 == 0: 269 | model.eval() 270 | 271 | avg_hr1, avg_ndcg1, avg_mrr1, avg_hr2, avg_ndcg2, avg_mrr2 = test_process(model, train_loader, feed_data, 272 | args.cuda, topK_list[1], mode='val') 273 | 274 | val_hr1_list.append(avg_hr1) 275 | val_ndcg1_list.append(avg_ndcg1) 276 | val_mrr1_list.append(avg_mrr1) 277 | val_hr2_list.append(avg_hr2) 278 | val_ndcg2_list.append(avg_ndcg2) 279 | val_mrr2_list.append(avg_mrr2) 280 | 281 | print('test: movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 282 | .format(avg_hr1, avg_ndcg1, avg_mrr1, avg_hr2, avg_ndcg2, avg_mrr2)) 283 | with open(log + '/tmp.txt', 'a') as f: 284 | f.write('test: movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 285 | .format(avg_hr1, avg_ndcg1, avg_mrr1, avg_hr2, avg_ndcg2, avg_mrr2)) 286 | 287 | if avg_hr1 > best_hr1: 288 | best_hr1 = avg_hr1 289 | torch.save(model.state_dict(), os.path.join(log, 'best_hr1.pkl')) 290 | 291 | if avg_ndcg1 > best_ndcg1: 292 | torch.save(model.state_dict(), os.path.join(log, 'best_ndcg1.pkl')) 293 | best_ndcg1 = avg_ndcg1 294 | if avg_mrr1 > best_mrr1: 295 | torch.save(model.state_dict(), os.path.join(log, 'best_mrr1.pkl')) 296 | best_mrr1 = avg_mrr1 297 | if avg_hr2 > best_hr2: 298 | torch.save(model.state_dict(), os.path.join(log, 'best_hr2.pkl')) 299 | best_hr2 = avg_hr2 300 | if avg_ndcg2 > best_ndcg2: 301 | torch.save(model.state_dict(), os.path.join(log, 'best_ndcg2.pkl')) 302 | best_ndcg2 = avg_ndcg2 303 | if avg_mrr2 > best_mrr2: 304 | torch.save(model.state_dict(), os.path.join(log, 'best_mrr2.pkl')) 305 | best_mrr2 = avg_mrr2 306 | 307 | 308 | print('best val movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 309 | .format(best_hr1, best_ndcg1, best_mrr1, best_hr2, best_ndcg2, best_mrr2)) 310 | with open(log + '/tmp.txt', 'a') as f: 311 | f.write('best val movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 312 | .format(best_hr1, best_ndcg1, best_mrr1, best_hr2, best_ndcg2, best_mrr2)) 313 | 314 | # # save necessary plot loss and validation metric 315 | # pickle.dump(JRL_loss_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 316 | # '{}_{}_JRL_loss_list'.format(args.dataset, args.gan)), 'wb')) 317 | # pickle.dump(val_hr1_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 318 | # '{}_{}_val_hr1_list'.format(args.dataset, args.gan)), 'wb')) 319 | # pickle.dump(val_ndcg1_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 320 | # '{}_{}_val_ndcg1_list'.format(args.dataset, args.gan)), 'wb')) 321 | # pickle.dump(val_mrr1_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 322 | # '{}_{}_val_mrr1_list'.format(args.dataset, args.gan)), 'wb')) 323 | # pickle.dump(val_hr2_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 324 | # '{}_{}_val_hr2_list'.format(args.dataset, args.gan)), 'wb')) 325 | # pickle.dump(val_ndcg2_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 326 | # '{}_{}_val_ndcg2_list'.format(args.dataset, args.gan)), 'wb')) 327 | # pickle.dump(val_mrr2_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 328 | # '{}_{}_val_mrr2_list'.format(args.dataset, args.gan)), 'wb')) 329 | 330 | print('Val process over!') 331 | print('Test process......') 332 | for topK in topK_list: 333 | model.load_state_dict(torch.load(os.path.join(log, 'best_hr1.pkl'))) 334 | test_hr1, _, _, _, _, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 335 | 336 | # if topK==10: 337 | # user_embedding1, user_embedding2 = save_embedding_process(model, save_loader, feed_data, args.cuda) 338 | # pickle.dump(user_embedding1, open( 339 | # os.path.join(os.getcwd(), 'saved_embeddings', method_name, '{}_Z_movie.pkl'.format(args.dataset)), 'wb')) 340 | # pickle.dump(user_embedding2, open( 341 | # os.path.join(os.getcwd(), 'saved_embeddings', method_name, '{}_Z_book.pkl'.format(args.dataset)), 'wb')) 342 | 343 | model.load_state_dict(torch.load(os.path.join(log, 'best_ndcg1.pkl'))) 344 | _, test_ndcg1, _, _, _, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 345 | model.load_state_dict(torch.load(os.path.join(log, 'best_mrr1.pkl'))) 346 | _, _, test_mrr1, _, _, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 347 | model.load_state_dict(torch.load(os.path.join(log, 'best_hr2.pkl'))) 348 | _, _, _, test_hr2, _, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 349 | model.load_state_dict(torch.load(os.path.join(log, 'best_ndcg2.pkl'))) 350 | _, _, _, _, test_ndcg2, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 351 | model.load_state_dict(torch.load(os.path.join(log, 'best_mrr2.pkl'))) 352 | _, _, _, _, _, test_mrr2 = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 353 | print('Test TopK:{} ---> movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 354 | .format(topK, test_hr1, test_ndcg1, test_mrr1, test_hr2, test_ndcg2, test_mrr2)) 355 | with open(log + '/tmp.txt', 'a') as f: 356 | f.write('Test TopK:{} ---> movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 357 | .format(topK, test_hr1, test_ndcg1, test_mrr1, test_hr2, test_ndcg2, test_mrr2)) 358 | 359 | 360 | 361 | if __name__ == "__main__": 362 | print(args) 363 | main() 364 | print(args) -------------------------------------------------------------------------------- /main_my_variant2.py: -------------------------------------------------------------------------------- 1 | 2 | from __future__ import print_function 3 | import matplotlib 4 | matplotlib.use('Agg') 5 | import argparse 6 | import torch 7 | import torch.utils.data 8 | from torch import nn, optim 9 | from torch.autograd import Variable 10 | import time 11 | import random 12 | import os 13 | import math 14 | import pickle 15 | import scipy.sparse as sp 16 | import matplotlib.pyplot as plt 17 | import matplotlib.ticker as ticker 18 | import numpy as np 19 | from Dataset_CDAE import Dataset 20 | from model_my_variant2 import CDAE, Discriminator, save_embedding_process 21 | import time 22 | import itertools 23 | import pandas as pd 24 | from my_utils import * 25 | from scipy.sparse import csr_matrix 26 | 27 | ''' 28 | variant of my. Set W1, W2 and do not has constraint 29 | ''' 30 | 31 | method_name = 'my_variant2' 32 | # os.environ['CUDA_VISIBLE_DEVICES'] = '1' 33 | topK_list = [5, 10] 34 | 35 | parser = argparse.ArgumentParser() 36 | parser.add_argument('--prior', type=str, default='Gaussian', help='Gaussian, MVGaussian') 37 | parser.add_argument('--seed', type=int, default=72, help='Random seed.') 38 | parser.add_argument('--epochs', type=int, default=300, help='Number of epochs to train.') 39 | parser.add_argument('--batch', type=int, default=256, help='batch size.') 40 | parser.add_argument('--emb_size', type=int, default=200, help='embed size.') 41 | parser.add_argument('--lr', type=float, default=0.001, help='Initial learning rate.') 42 | parser.add_argument('--weight_decay', type=float, default=0.001, help='Weight decay (L2 loss on parameters).') 43 | parser.add_argument('--dropout', type=float, default=0.5, help='Dropout rate (1 - keep probability).') 44 | parser.add_argument('--log', type=str, default='logs/{}'.format(method_name), help='log directory') 45 | parser.add_argument('--pos-weight', type=float, default=1.0, help='weight for positive samples') 46 | 47 | parser.add_argument('--reg', type=float, default=1.0, help='lambda reg always 0.0 for this variant') 48 | 49 | parser.add_argument('--self', type=float, default=1.0, help='lambda rec') 50 | parser.add_argument('--cross', type=float, default=1.0, help='lambda rec') 51 | parser.add_argument('--gan', type=float, default=1.0, help='lambda gan') 52 | parser.add_argument('--d_epoch', type=int, default=2, help='d epoch') 53 | parser.add_argument('--t_percent', type=float, default=1.0, help='target percent') 54 | parser.add_argument('--s_percent', type=float, default=1.0, help='source percent') 55 | parser.add_argument('--dataset', type=str, default='amazon3', help='amazon') 56 | 57 | args = parser.parse_args() 58 | 59 | args.cuda = torch.cuda.is_available() 60 | 61 | kwargs = {'num_workers': 1, 'pin_memory': True} if args.cuda else {} 62 | 63 | def sigmoid(x): 64 | return 1 / (1 + np.exp(-x)) 65 | 66 | def main(): 67 | log = os.path.join(args.log, '{}_{}_{}_{}_{}_{}_{}_{}_{}'.format(args.dataset, args.emb_size, args.weight_decay, args.self, 68 | args.cross, args.gan, args.reg, args.t_percent, args.s_percent)) 69 | if os.path.isdir(log): 70 | print("%s already exist. are you sure to override? Ok, I'll wait for 5 seconds. Ctrl-C to abort." % log) 71 | time.sleep(5) 72 | os.system('rm -rf %s/' % log) 73 | 74 | os.makedirs(log) 75 | print("made the log directory", log) 76 | 77 | print('preparing data...') 78 | dataset = Dataset(args.batch, dataset=args.dataset) 79 | 80 | NUM_USER = dataset.num_user 81 | NUM_MOVIE = dataset.num_movie 82 | NUM_BOOK = dataset.num_book 83 | 84 | print('Preparing the training data......') 85 | # prepare data for X 86 | row, col = dataset.get_part_train_indices('movie', args.s_percent) 87 | values = np.ones(row.shape[0]) 88 | user_x = csr_matrix((values, (row, col)), shape=(NUM_USER, NUM_MOVIE)).toarray() 89 | 90 | # prepare data fot Y 91 | row, col = dataset.get_part_train_indices('book', args.t_percent) 92 | values = np.ones(row.shape[0]) 93 | user_y = csr_matrix((values, (row, col)), shape=(NUM_USER, NUM_BOOK)).toarray() 94 | 95 | print('Preparing the training data over......') 96 | # for shared user one-hot representation 97 | user_id = np.arange(NUM_USER).reshape([NUM_USER, 1]) 98 | 99 | user_x = torch.FloatTensor(user_x) 100 | user_y = torch.FloatTensor(user_y) 101 | 102 | train_loader = torch.utils.data.DataLoader(torch.from_numpy(user_id), 103 | batch_size=args.batch, 104 | shuffle=True, **kwargs) 105 | save_loader = torch.utils.data.DataLoader(torch.from_numpy(user_id), 106 | batch_size=args.batch, 107 | shuffle=False, **kwargs) 108 | 109 | pos_weight = torch.FloatTensor([args.pos_weight]) 110 | 111 | if args.cuda: 112 | pos_weight = pos_weight.cuda() 113 | 114 | model = CDAE(NUM_USER=NUM_USER, NUM_MOVIE=NUM_MOVIE, NUM_BOOK=NUM_BOOK, 115 | EMBED_SIZE=args.emb_size, dropout=args.dropout) 116 | disc1 = Discriminator(args.emb_size, args.dropout) 117 | disc2 = Discriminator(args.emb_size, args.dropout) 118 | optimizer_g = optim.Adam(model.parameters(), lr=args.lr, weight_decay=args.weight_decay) 119 | optimizer_d = optim.SGD(itertools.chain(disc1.parameters(), disc2.parameters()), lr=args.lr, weight_decay=args.weight_decay) 120 | 121 | BCEWL = torch.nn.BCEWithLogitsLoss(reduction='none', pos_weight=pos_weight) 122 | BCEL = torch.nn.BCEWithLogitsLoss(reduction='none') 123 | 124 | if args.cuda: 125 | model = model.cuda() 126 | disc1 = disc1.cuda() 127 | disc2 = disc2.cuda() 128 | 129 | # prepare data fot test process 130 | movie_vali, movie_test, movie_nega = dataset.movie_vali, dataset.movie_test, dataset.movie_nega 131 | book_vali, book_test, book_nega = dataset.book_vali, dataset.book_test, dataset.book_nega 132 | feed_data = {} 133 | feed_data['fts1'] = user_x 134 | feed_data['fts2'] = user_y 135 | feed_data['movie_vali'] = movie_vali 136 | feed_data['book_vali'] = book_vali 137 | feed_data['movie_test'] = movie_test 138 | feed_data['book_test'] = book_test 139 | feed_data['movie_nega'] = movie_nega 140 | feed_data['book_nega'] = book_nega 141 | 142 | best_hr1, best_ndcg1, best_mrr1 = 0.0, 0.0, 0.0 143 | best_hr2, best_ndcg2, best_mrr2 = 0.0, 0.0, 0.0 144 | val_hr1_list, val_ndcg1_list, val_mrr1_list = [], [], [] 145 | val_hr2_list, val_ndcg2_list, val_mrr2_list = [], [], [] 146 | 147 | G_loss_list = [] 148 | D_loss_list = [] 149 | reg_loss_list = [] 150 | loss_list = [] 151 | self_loss_list = [] 152 | cross_loss_list = [] 153 | JRL_loss_list = [] 154 | 155 | for epoch in range(args.epochs): 156 | model.train() 157 | batch_G_loss_list = [] 158 | batch_D_loss_list = [] 159 | batch_loss_list = [] 160 | batch_reg_loss_list = [] 161 | batch_self_loss_list = [] 162 | batch_cross_loss_list = [] 163 | batch_JRL_loss_list = [] 164 | 165 | for batch_idx, data in enumerate(train_loader): 166 | data = data.reshape([-1]) 167 | 168 | if (batch_idx+1) % (args.d_epoch + 1) == 0: 169 | optimizer_d.zero_grad() 170 | prior = torch.from_numpy(np.random.normal(0, 1.0, size=[data.shape[0], args.emb_size])).float() 171 | if args.cuda: 172 | prior = prior.cuda() 173 | 174 | if args.cuda: 175 | batch_user = data.cuda() 176 | batch_user_x = user_x[data].cuda() 177 | batch_user_y = user_y[data].cuda() 178 | else: 179 | batch_user = data 180 | batch_user_x = user_x[data] 181 | batch_user_y = user_y[data] 182 | 183 | _, _, _, _, z_x, z_y, z_x_reg_loss, z_y_reg_loss = model.forward(batch_user, batch_user_x, batch_user_y) 184 | 185 | true1 = disc1(prior).reshape([-1]) 186 | true2 = disc2(prior).reshape([-1]) 187 | fake1 = disc1(z_x).reshape([-1]) 188 | fake2 = disc2(z_y).reshape([-1]) 189 | 190 | dis_loss1 = BCEL(torch.cat([true1, fake1], 0), 191 | torch.cat([torch.ones_like(true1), torch.zeros_like(fake1)], 0)).sum() 192 | dis_loss2 = BCEL(torch.cat([true2, fake2], 0), 193 | torch.cat([torch.ones_like(true2), torch.zeros_like(fake2)], 0)).sum() 194 | 195 | D_loss = dis_loss1 + dis_loss2 196 | 197 | D_loss.backward() 198 | optimizer_d.step() 199 | batch_D_loss_list.append(D_loss.item()) 200 | 201 | else: 202 | optimizer_g.zero_grad() 203 | 204 | if args.cuda: 205 | batch_user = data.cuda() 206 | batch_user_x = user_x[data].cuda() 207 | batch_user_y = user_y[data].cuda() 208 | else: 209 | batch_user_x = user_x[data] 210 | batch_user_y = user_y[data] 211 | 212 | pred_x, pred_y, pred_x2y, pred_y2x, z_x, z_y, z_x_reg_loss, z_y_reg_loss = model.forward(batch_user, batch_user_x, batch_user_y) 213 | 214 | loss_x = BCEWL(pred_x, batch_user_x).sum() 215 | loss_y = BCEWL(pred_y, batch_user_y).sum() 216 | loss_x2y = BCEWL(pred_x2y, batch_user_y).sum() 217 | loss_y2x = BCEWL(pred_y2x, batch_user_x).sum() 218 | 219 | reg_loss = z_x_reg_loss.sum() + z_y_reg_loss.sum() 220 | 221 | fake_x = disc1(z_x).reshape([-1]) 222 | fake_y = disc2(z_y).reshape([-1]) 223 | 224 | G_loss1 = BCEL(fake_x, torch.ones_like(fake_x)).sum() 225 | G_loss2 = BCEL(fake_y, torch.ones_like(fake_y)).sum() 226 | G_loss = G_loss1+G_loss2 227 | 228 | # get plot JRL loss 229 | JRL_loss = args.self * (loss_x + loss_y) + args.cross * (loss_x2y + loss_y2x) + args.reg*reg_loss 230 | batch_JRL_loss_list.append(JRL_loss.item()/args.batch) 231 | 232 | # get the whole loss 233 | loss = G_loss + args.self * (loss_x + loss_y) + args.cross * (loss_x2y + loss_y2x) + args.reg*reg_loss 234 | loss.backward() 235 | optimizer_g.step() 236 | 237 | batch_G_loss_list.append(G_loss.item()) 238 | batch_self_loss_list.append(args.self * (loss_x + loss_y).item()) 239 | batch_cross_loss_list.append(args.cross * (loss_x2y + loss_y2x).item()) 240 | batch_loss_list.append(loss.item()) 241 | batch_reg_loss_list.append(args.reg*reg_loss.item()) 242 | 243 | epoch_G_loss = np.mean(batch_G_loss_list) 244 | epoch_D_loss = np.mean(batch_D_loss_list) 245 | epoch_reg_loss = np.mean(batch_reg_loss_list) 246 | epoch_loss = np.mean(batch_loss_list) 247 | epoch_self_loss = np.mean(batch_self_loss_list) 248 | epoch_cross_loss = np.mean(batch_cross_loss_list) 249 | epoch_JRL_Loss = np.mean(batch_JRL_loss_list) 250 | 251 | G_loss_list.append(epoch_G_loss) 252 | D_loss_list.append(epoch_D_loss) 253 | loss_list.append(epoch_loss) 254 | self_loss_list.append(epoch_self_loss) 255 | cross_loss_list.append(epoch_cross_loss) 256 | reg_loss_list.append(epoch_reg_loss) 257 | JRL_loss_list.append(epoch_JRL_Loss) 258 | 259 | print('epoch:{}, self loss:{:.4f}, cross loss:{:.4f}, G loss:{:.4f}, D loss:{:.4f}, reg loss:{:.4f}'.format(epoch,epoch_self_loss, 260 | epoch_cross_loss, 261 | epoch_G_loss, 262 | epoch_D_loss, 263 | epoch_reg_loss)) 264 | with open(log + '/tmp.txt', 'a') as f: 265 | f.write('epoch:{}, self loss:{:.4f}, cross loss:{:.4f}, G loss:{:.4f}, D loss:{:.4f}, reg loss:{:.4f}'.format( 266 | epoch, 267 | epoch_self_loss, epoch_cross_loss, epoch_G_loss, epoch_D_loss, epoch_reg_loss)) 268 | 269 | if epoch % 1 == 0: 270 | model.eval() 271 | 272 | avg_hr1, avg_ndcg1, avg_mrr1, avg_hr2, avg_ndcg2, avg_mrr2 = test_process(model, train_loader, feed_data, 273 | args.cuda, topK_list[1], mode='val') 274 | 275 | val_hr1_list.append(avg_hr1) 276 | val_ndcg1_list.append(avg_ndcg1) 277 | val_mrr1_list.append(avg_mrr1) 278 | val_hr2_list.append(avg_hr2) 279 | val_ndcg2_list.append(avg_ndcg2) 280 | val_mrr2_list.append(avg_mrr2) 281 | 282 | print('test: movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 283 | .format(avg_hr1, avg_ndcg1, avg_mrr1, avg_hr2, avg_ndcg2, avg_mrr2)) 284 | with open(log + '/tmp.txt', 'a') as f: 285 | f.write('test: movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 286 | .format(avg_hr1, avg_ndcg1, avg_mrr1, avg_hr2, avg_ndcg2, avg_mrr2)) 287 | 288 | if avg_hr1 > best_hr1: 289 | best_hr1 = avg_hr1 290 | torch.save(model.state_dict(), os.path.join(log, 'best_hr1.pkl')) 291 | 292 | if avg_ndcg1 > best_ndcg1: 293 | torch.save(model.state_dict(), os.path.join(log, 'best_ndcg1.pkl')) 294 | best_ndcg1 = avg_ndcg1 295 | if avg_mrr1 > best_mrr1: 296 | torch.save(model.state_dict(), os.path.join(log, 'best_mrr1.pkl')) 297 | best_mrr1 = avg_mrr1 298 | if avg_hr2 > best_hr2: 299 | torch.save(model.state_dict(), os.path.join(log, 'best_hr2.pkl')) 300 | best_hr2 = avg_hr2 301 | if avg_ndcg2 > best_ndcg2: 302 | torch.save(model.state_dict(), os.path.join(log, 'best_ndcg2.pkl')) 303 | best_ndcg2 = avg_ndcg2 304 | if avg_mrr2 > best_mrr2: 305 | torch.save(model.state_dict(), os.path.join(log, 'best_mrr2.pkl')) 306 | best_mrr2 = avg_mrr2 307 | 308 | 309 | print('best val movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 310 | .format(best_hr1, best_ndcg1, best_mrr1, best_hr2, best_ndcg2, best_mrr2)) 311 | with open(log + '/tmp.txt', 'a') as f: 312 | f.write('best val movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 313 | .format(best_hr1, best_ndcg1, best_mrr1, best_hr2, best_ndcg2, best_mrr2)) 314 | 315 | # # save necessary plot loss and validation metric 316 | # pickle.dump(JRL_loss_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 317 | # '{}_{}_JRL_loss_list'.format(args.dataset, args.gan)), 'wb')) 318 | # pickle.dump(val_hr1_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 319 | # '{}_{}_val_hr1_list'.format(args.dataset, args.gan)), 'wb')) 320 | # pickle.dump(val_ndcg1_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 321 | # '{}_{}_val_ndcg1_list'.format(args.dataset, args.gan)), 'wb')) 322 | # pickle.dump(val_mrr1_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 323 | # '{}_{}_val_mrr1_list'.format(args.dataset, args.gan)), 'wb')) 324 | # pickle.dump(val_hr2_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 325 | # '{}_{}_val_hr2_list'.format(args.dataset, args.gan)), 'wb')) 326 | # pickle.dump(val_ndcg2_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 327 | # '{}_{}_val_ndcg2_list'.format(args.dataset, args.gan)), 'wb')) 328 | # pickle.dump(val_mrr2_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 329 | # '{}_{}_val_mrr2_list'.format(args.dataset, args.gan)), 'wb')) 330 | 331 | print('Val process over!') 332 | print('Test process......') 333 | for topK in topK_list: 334 | model.load_state_dict(torch.load(os.path.join(log, 'best_hr1.pkl'))) 335 | test_hr1, _, _, _, _, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 336 | 337 | # if topK==10: 338 | # user_embedding1, user_embedding2 = save_embedding_process(model, save_loader, feed_data, args.cuda) 339 | # pickle.dump(user_embedding1, open( 340 | # os.path.join(os.getcwd(), 'saved_embeddings', method_name, '{}_Z_movie.pkl'.format(args.dataset)), 'wb')) 341 | # pickle.dump(user_embedding2, open( 342 | # os.path.join(os.getcwd(), 'saved_embeddings', method_name, '{}_Z_book.pkl'.format(args.dataset)), 'wb')) 343 | 344 | model.load_state_dict(torch.load(os.path.join(log, 'best_ndcg1.pkl'))) 345 | _, test_ndcg1, _, _, _, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 346 | model.load_state_dict(torch.load(os.path.join(log, 'best_mrr1.pkl'))) 347 | _, _, test_mrr1, _, _, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 348 | model.load_state_dict(torch.load(os.path.join(log, 'best_hr2.pkl'))) 349 | _, _, _, test_hr2, _, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 350 | model.load_state_dict(torch.load(os.path.join(log, 'best_ndcg2.pkl'))) 351 | _, _, _, _, test_ndcg2, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 352 | model.load_state_dict(torch.load(os.path.join(log, 'best_mrr2.pkl'))) 353 | _, _, _, _, _, test_mrr2 = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 354 | print('Test TopK:{} ---> movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 355 | .format(topK, test_hr1, test_ndcg1, test_mrr1, test_hr2, test_ndcg2, test_mrr2)) 356 | with open(log + '/tmp.txt', 'a') as f: 357 | f.write('Test TopK:{} ---> movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 358 | .format(topK, test_hr1, test_ndcg1, test_mrr1, test_hr2, test_ndcg2, test_mrr2)) 359 | 360 | 361 | 362 | if __name__ == "__main__": 363 | print(args) 364 | main() 365 | print(args) -------------------------------------------------------------------------------- /main_my_variant3.py: -------------------------------------------------------------------------------- 1 | 2 | from __future__ import print_function 3 | import matplotlib 4 | matplotlib.use('Agg') 5 | import argparse 6 | import torch 7 | import torch.utils.data 8 | from torch import nn, optim 9 | from torch.autograd import Variable 10 | import time 11 | import random 12 | import os 13 | import math 14 | import pickle 15 | import scipy.sparse as sp 16 | import matplotlib.pyplot as plt 17 | import matplotlib.ticker as ticker 18 | import numpy as np 19 | from Dataset_CDAE import Dataset 20 | from model_my_variant3 import CDAE, Discriminator, save_embedding_process 21 | import time 22 | import itertools 23 | import pandas as pd 24 | from my_utils import * 25 | from scipy.sparse import csr_matrix 26 | 27 | ''' 28 | variant of my. Set W1, W2 are two-layer non-linear transformation 29 | ''' 30 | 31 | method_name = 'my_variant3' 32 | # os.environ['CUDA_VISIBLE_DEVICES'] = '1' 33 | topK_list = [5, 10] 34 | 35 | parser = argparse.ArgumentParser() 36 | parser.add_argument('--prior', type=str, default='Gaussian', help='Gaussian, MVGaussian') 37 | parser.add_argument('--seed', type=int, default=72, help='Random seed.') 38 | parser.add_argument('--epochs', type=int, default=300, help='Number of epochs to train.') 39 | parser.add_argument('--batch', type=int, default=256, help='batch size.') 40 | parser.add_argument('--emb_size', type=int, default=200, help='embed size.') 41 | parser.add_argument('--lr', type=float, default=0.001, help='Initial learning rate.') 42 | parser.add_argument('--weight_decay', type=float, default=0.001, help='Weight decay (L2 loss on parameters).') 43 | parser.add_argument('--dropout', type=float, default=0.5, help='Dropout rate (1 - keep probability).') 44 | parser.add_argument('--log', type=str, default='logs/{}'.format(method_name), help='log directory') 45 | parser.add_argument('--pos-weight', type=float, default=1.0, help='weight for positive samples') 46 | 47 | parser.add_argument('--reg', type=float, default=1.0, help='lambda reg always 0.0 for this variant') 48 | 49 | parser.add_argument('--self', type=float, default=1.0, help='lambda rec') 50 | parser.add_argument('--cross', type=float, default=1.0, help='lambda rec') 51 | parser.add_argument('--gan', type=float, default=1.0, help='lambda gan') 52 | parser.add_argument('--d_epoch', type=int, default=2, help='d epoch') 53 | parser.add_argument('--t_percent', type=float, default=1.0, help='target percent') 54 | parser.add_argument('--s_percent', type=float, default=1.0, help='source percent') 55 | parser.add_argument('--dataset', type=str, default='amazon3', help='amazon') 56 | 57 | args = parser.parse_args() 58 | 59 | args.cuda = torch.cuda.is_available() 60 | 61 | kwargs = {'num_workers': 1, 'pin_memory': True} if args.cuda else {} 62 | 63 | def sigmoid(x): 64 | return 1 / (1 + np.exp(-x)) 65 | 66 | def main(): 67 | log = os.path.join(args.log, '{}_{}_{}_{}_{}_{}_{}_{}_{}'.format(args.dataset, args.emb_size, args.weight_decay, args.self, 68 | args.cross, args.gan, args.reg, args.t_percent, args.s_percent)) 69 | if os.path.isdir(log): 70 | print("%s already exist. are you sure to override? Ok, I'll wait for 5 seconds. Ctrl-C to abort." % log) 71 | time.sleep(5) 72 | os.system('rm -rf %s/' % log) 73 | 74 | os.makedirs(log) 75 | print("made the log directory", log) 76 | 77 | print('preparing data...') 78 | dataset = Dataset(args.batch, dataset=args.dataset) 79 | 80 | NUM_USER = dataset.num_user 81 | NUM_MOVIE = dataset.num_movie 82 | NUM_BOOK = dataset.num_book 83 | 84 | print('Preparing the training data......') 85 | # prepare data for X 86 | row, col = dataset.get_part_train_indices('movie', args.s_percent) 87 | values = np.ones(row.shape[0]) 88 | user_x = csr_matrix((values, (row, col)), shape=(NUM_USER, NUM_MOVIE)).toarray() 89 | 90 | # prepare data fot Y 91 | row, col = dataset.get_part_train_indices('book', args.t_percent) 92 | values = np.ones(row.shape[0]) 93 | user_y = csr_matrix((values, (row, col)), shape=(NUM_USER, NUM_BOOK)).toarray() 94 | 95 | print('Preparing the training data over......') 96 | # for shared user one-hot representation 97 | user_id = np.arange(NUM_USER).reshape([NUM_USER, 1]) 98 | 99 | user_x = torch.FloatTensor(user_x) 100 | user_y = torch.FloatTensor(user_y) 101 | 102 | train_loader = torch.utils.data.DataLoader(torch.from_numpy(user_id), 103 | batch_size=args.batch, 104 | shuffle=True, **kwargs) 105 | save_loader = torch.utils.data.DataLoader(torch.from_numpy(user_id), 106 | batch_size=args.batch, 107 | shuffle=False, **kwargs) 108 | 109 | pos_weight = torch.FloatTensor([args.pos_weight]) 110 | 111 | if args.cuda: 112 | pos_weight = pos_weight.cuda() 113 | 114 | model = CDAE(NUM_USER=NUM_USER, NUM_MOVIE=NUM_MOVIE, NUM_BOOK=NUM_BOOK, 115 | EMBED_SIZE=args.emb_size, dropout=args.dropout) 116 | disc1 = Discriminator(args.emb_size, args.dropout) 117 | disc2 = Discriminator(args.emb_size, args.dropout) 118 | optimizer_g = optim.Adam(model.parameters(), lr=args.lr, weight_decay=args.weight_decay) 119 | optimizer_d = optim.SGD(itertools.chain(disc1.parameters(), disc2.parameters()), lr=args.lr, weight_decay=args.weight_decay) 120 | 121 | BCEWL = torch.nn.BCEWithLogitsLoss(reduction='none', pos_weight=pos_weight) 122 | BCEL = torch.nn.BCEWithLogitsLoss(reduction='none') 123 | 124 | if args.cuda: 125 | model = model.cuda() 126 | disc1 = disc1.cuda() 127 | disc2 = disc2.cuda() 128 | 129 | # prepare data fot test process 130 | movie_vali, movie_test, movie_nega = dataset.movie_vali, dataset.movie_test, dataset.movie_nega 131 | book_vali, book_test, book_nega = dataset.book_vali, dataset.book_test, dataset.book_nega 132 | feed_data = {} 133 | feed_data['fts1'] = user_x 134 | feed_data['fts2'] = user_y 135 | feed_data['movie_vali'] = movie_vali 136 | feed_data['book_vali'] = book_vali 137 | feed_data['movie_test'] = movie_test 138 | feed_data['book_test'] = book_test 139 | feed_data['movie_nega'] = movie_nega 140 | feed_data['book_nega'] = book_nega 141 | 142 | best_hr1, best_ndcg1, best_mrr1 = 0.0, 0.0, 0.0 143 | best_hr2, best_ndcg2, best_mrr2 = 0.0, 0.0, 0.0 144 | val_hr1_list, val_ndcg1_list, val_mrr1_list = [], [], [] 145 | val_hr2_list, val_ndcg2_list, val_mrr2_list = [], [], [] 146 | 147 | G_loss_list = [] 148 | D_loss_list = [] 149 | reg_loss_list = [] 150 | loss_list = [] 151 | self_loss_list = [] 152 | cross_loss_list = [] 153 | JRL_loss_list = [] 154 | 155 | for epoch in range(args.epochs): 156 | model.train() 157 | batch_G_loss_list = [] 158 | batch_D_loss_list = [] 159 | batch_loss_list = [] 160 | batch_reg_loss_list = [] 161 | batch_self_loss_list = [] 162 | batch_cross_loss_list = [] 163 | batch_JRL_loss_list = [] 164 | 165 | for batch_idx, data in enumerate(train_loader): 166 | data = data.reshape([-1]) 167 | 168 | if (batch_idx+1) % (args.d_epoch + 1) == 0: 169 | optimizer_d.zero_grad() 170 | prior = torch.from_numpy(np.random.normal(0, 1.0, size=[data.shape[0], args.emb_size])).float() 171 | if args.cuda: 172 | prior = prior.cuda() 173 | 174 | if args.cuda: 175 | batch_user = data.cuda() 176 | batch_user_x = user_x[data].cuda() 177 | batch_user_y = user_y[data].cuda() 178 | else: 179 | batch_user = data 180 | batch_user_x = user_x[data] 181 | batch_user_y = user_y[data] 182 | 183 | _, _, _, _, z_x, z_y, z_x_reg_loss, z_y_reg_loss = model.forward(batch_user, batch_user_x, batch_user_y) 184 | 185 | true1 = disc1(prior).reshape([-1]) 186 | true2 = disc2(prior).reshape([-1]) 187 | fake1 = disc1(z_x).reshape([-1]) 188 | fake2 = disc2(z_y).reshape([-1]) 189 | 190 | dis_loss1 = BCEL(torch.cat([true1, fake1], 0), 191 | torch.cat([torch.ones_like(true1), torch.zeros_like(fake1)], 0)).sum() 192 | dis_loss2 = BCEL(torch.cat([true2, fake2], 0), 193 | torch.cat([torch.ones_like(true2), torch.zeros_like(fake2)], 0)).sum() 194 | 195 | D_loss = dis_loss1 + dis_loss2 196 | 197 | D_loss.backward() 198 | optimizer_d.step() 199 | batch_D_loss_list.append(D_loss.item()) 200 | 201 | else: 202 | optimizer_g.zero_grad() 203 | 204 | if args.cuda: 205 | batch_user = data.cuda() 206 | batch_user_x = user_x[data].cuda() 207 | batch_user_y = user_y[data].cuda() 208 | else: 209 | batch_user_x = user_x[data] 210 | batch_user_y = user_y[data] 211 | 212 | pred_x, pred_y, pred_x2y, pred_y2x, z_x, z_y, z_x_reg_loss, z_y_reg_loss = model.forward(batch_user, batch_user_x, batch_user_y) 213 | 214 | loss_x = BCEWL(pred_x, batch_user_x).sum() 215 | loss_y = BCEWL(pred_y, batch_user_y).sum() 216 | loss_x2y = BCEWL(pred_x2y, batch_user_y).sum() 217 | loss_y2x = BCEWL(pred_y2x, batch_user_x).sum() 218 | 219 | reg_loss = z_x_reg_loss.sum() + z_y_reg_loss.sum() 220 | 221 | fake_x = disc1(z_x).reshape([-1]) 222 | fake_y = disc2(z_y).reshape([-1]) 223 | 224 | G_loss1 = BCEL(fake_x, torch.ones_like(fake_x)).sum() 225 | G_loss2 = BCEL(fake_y, torch.ones_like(fake_y)).sum() 226 | G_loss = G_loss1+G_loss2 227 | 228 | # get plot JRL loss 229 | JRL_loss = args.self * (loss_x + loss_y) + args.cross * (loss_x2y + loss_y2x) + args.reg*reg_loss 230 | batch_JRL_loss_list.append(JRL_loss.item()/args.batch) 231 | 232 | # get the whole loss 233 | loss = G_loss + args.self * (loss_x + loss_y) + args.cross * (loss_x2y + loss_y2x) + args.reg*reg_loss 234 | loss.backward() 235 | optimizer_g.step() 236 | 237 | batch_G_loss_list.append(G_loss.item()) 238 | batch_self_loss_list.append(args.self * (loss_x + loss_y).item()) 239 | batch_cross_loss_list.append(args.cross * (loss_x2y + loss_y2x).item()) 240 | batch_loss_list.append(loss.item()) 241 | batch_reg_loss_list.append(args.reg*reg_loss.item()) 242 | 243 | epoch_G_loss = np.mean(batch_G_loss_list) 244 | epoch_D_loss = np.mean(batch_D_loss_list) 245 | epoch_reg_loss = np.mean(batch_reg_loss_list) 246 | epoch_loss = np.mean(batch_loss_list) 247 | epoch_self_loss = np.mean(batch_self_loss_list) 248 | epoch_cross_loss = np.mean(batch_cross_loss_list) 249 | epoch_JRL_Loss = np.mean(batch_JRL_loss_list) 250 | 251 | G_loss_list.append(epoch_G_loss) 252 | D_loss_list.append(epoch_D_loss) 253 | loss_list.append(epoch_loss) 254 | self_loss_list.append(epoch_self_loss) 255 | cross_loss_list.append(epoch_cross_loss) 256 | reg_loss_list.append(epoch_reg_loss) 257 | JRL_loss_list.append(epoch_JRL_Loss) 258 | 259 | print('epoch:{}, self loss:{:.4f}, cross loss:{:.4f}, G loss:{:.4f}, D loss:{:.4f}, reg loss:{:.4f}'.format(epoch,epoch_self_loss, 260 | epoch_cross_loss, 261 | epoch_G_loss, 262 | epoch_D_loss, 263 | epoch_reg_loss)) 264 | with open(log + '/tmp.txt', 'a') as f: 265 | f.write('epoch:{}, self loss:{:.4f}, cross loss:{:.4f}, G loss:{:.4f}, D loss:{:.4f}, reg loss:{:.4f}'.format( 266 | epoch, 267 | epoch_self_loss, epoch_cross_loss, epoch_G_loss, epoch_D_loss, epoch_reg_loss)) 268 | 269 | if epoch % 1 == 0: 270 | model.eval() 271 | 272 | avg_hr1, avg_ndcg1, avg_mrr1, avg_hr2, avg_ndcg2, avg_mrr2 = test_process(model, train_loader, feed_data, 273 | args.cuda, topK_list[1], mode='val') 274 | 275 | val_hr1_list.append(avg_hr1) 276 | val_ndcg1_list.append(avg_ndcg1) 277 | val_mrr1_list.append(avg_mrr1) 278 | val_hr2_list.append(avg_hr2) 279 | val_ndcg2_list.append(avg_ndcg2) 280 | val_mrr2_list.append(avg_mrr2) 281 | 282 | print('test: movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 283 | .format(avg_hr1, avg_ndcg1, avg_mrr1, avg_hr2, avg_ndcg2, avg_mrr2)) 284 | with open(log + '/tmp.txt', 'a') as f: 285 | f.write('test: movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 286 | .format(avg_hr1, avg_ndcg1, avg_mrr1, avg_hr2, avg_ndcg2, avg_mrr2)) 287 | 288 | if avg_hr1 > best_hr1: 289 | best_hr1 = avg_hr1 290 | torch.save(model.state_dict(), os.path.join(log, 'best_hr1.pkl')) 291 | 292 | if avg_ndcg1 > best_ndcg1: 293 | torch.save(model.state_dict(), os.path.join(log, 'best_ndcg1.pkl')) 294 | best_ndcg1 = avg_ndcg1 295 | if avg_mrr1 > best_mrr1: 296 | torch.save(model.state_dict(), os.path.join(log, 'best_mrr1.pkl')) 297 | best_mrr1 = avg_mrr1 298 | if avg_hr2 > best_hr2: 299 | torch.save(model.state_dict(), os.path.join(log, 'best_hr2.pkl')) 300 | best_hr2 = avg_hr2 301 | if avg_ndcg2 > best_ndcg2: 302 | torch.save(model.state_dict(), os.path.join(log, 'best_ndcg2.pkl')) 303 | best_ndcg2 = avg_ndcg2 304 | if avg_mrr2 > best_mrr2: 305 | torch.save(model.state_dict(), os.path.join(log, 'best_mrr2.pkl')) 306 | best_mrr2 = avg_mrr2 307 | 308 | 309 | print('best val movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 310 | .format(best_hr1, best_ndcg1, best_mrr1, best_hr2, best_ndcg2, best_mrr2)) 311 | with open(log + '/tmp.txt', 'a') as f: 312 | f.write('best val movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 313 | .format(best_hr1, best_ndcg1, best_mrr1, best_hr2, best_ndcg2, best_mrr2)) 314 | 315 | # # save necessary plot loss and validation metric 316 | # pickle.dump(JRL_loss_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 317 | # '{}_{}_JRL_loss_list'.format(args.dataset, args.gan)), 'wb')) 318 | # pickle.dump(val_hr1_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 319 | # '{}_{}_val_hr1_list'.format(args.dataset, args.gan)), 'wb')) 320 | # pickle.dump(val_ndcg1_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 321 | # '{}_{}_val_ndcg1_list'.format(args.dataset, args.gan)), 'wb')) 322 | # pickle.dump(val_mrr1_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 323 | # '{}_{}_val_mrr1_list'.format(args.dataset, args.gan)), 'wb')) 324 | # pickle.dump(val_hr2_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 325 | # '{}_{}_val_hr2_list'.format(args.dataset, args.gan)), 'wb')) 326 | # pickle.dump(val_ndcg2_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 327 | # '{}_{}_val_ndcg2_list'.format(args.dataset, args.gan)), 'wb')) 328 | # pickle.dump(val_mrr2_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 329 | # '{}_{}_val_mrr2_list'.format(args.dataset, args.gan)), 'wb')) 330 | 331 | print('Val process over!') 332 | print('Test process......') 333 | for topK in topK_list: 334 | model.load_state_dict(torch.load(os.path.join(log, 'best_hr1.pkl'))) 335 | test_hr1, _, _, _, _, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 336 | 337 | # if topK==10: 338 | # user_embedding1, user_embedding2 = save_embedding_process(model, save_loader, feed_data, args.cuda) 339 | # pickle.dump(user_embedding1, open( 340 | # os.path.join(os.getcwd(), 'saved_embeddings', method_name, '{}_Z_movie.pkl'.format(args.dataset)), 'wb')) 341 | # pickle.dump(user_embedding2, open( 342 | # os.path.join(os.getcwd(), 'saved_embeddings', method_name, '{}_Z_book.pkl'.format(args.dataset)), 'wb')) 343 | 344 | model.load_state_dict(torch.load(os.path.join(log, 'best_ndcg1.pkl'))) 345 | _, test_ndcg1, _, _, _, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 346 | model.load_state_dict(torch.load(os.path.join(log, 'best_mrr1.pkl'))) 347 | _, _, test_mrr1, _, _, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 348 | model.load_state_dict(torch.load(os.path.join(log, 'best_hr2.pkl'))) 349 | _, _, _, test_hr2, _, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 350 | model.load_state_dict(torch.load(os.path.join(log, 'best_ndcg2.pkl'))) 351 | _, _, _, _, test_ndcg2, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 352 | model.load_state_dict(torch.load(os.path.join(log, 'best_mrr2.pkl'))) 353 | _, _, _, _, _, test_mrr2 = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 354 | print('Test TopK:{} ---> movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 355 | .format(topK, test_hr1, test_ndcg1, test_mrr1, test_hr2, test_ndcg2, test_mrr2)) 356 | with open(log + '/tmp.txt', 'a') as f: 357 | f.write('Test TopK:{} ---> movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 358 | .format(topK, test_hr1, test_ndcg1, test_mrr1, test_hr2, test_ndcg2, test_mrr2)) 359 | 360 | 361 | 362 | if __name__ == "__main__": 363 | print(args) 364 | main() 365 | print(args) -------------------------------------------------------------------------------- /main_my.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | import matplotlib 3 | matplotlib.use('Agg') 4 | import argparse 5 | import torch 6 | import torch.utils.data 7 | from torch import nn, optim 8 | from torch.autograd import Variable 9 | import time 10 | import random 11 | import os 12 | import math 13 | import pickle 14 | import scipy.sparse as sp 15 | import matplotlib.pyplot as plt 16 | import matplotlib.ticker as ticker 17 | import numpy as np 18 | from Dataset_CDAE import Dataset 19 | from model_my import CDAE, Discriminator, save_embedding_process, save_z_process 20 | import time 21 | import itertools 22 | import pandas as pd 23 | from my_utils import * 24 | from scipy.sparse import csr_matrix 25 | 26 | method_name = 'my' 27 | # os.environ['CUDA_VISIBLE_DEVICES'] = '1' 28 | topK_list = [5, 10] 29 | 30 | parser = argparse.ArgumentParser() 31 | parser.add_argument('--prior', type=str, default='Gaussian', help='Gaussian, MVGaussian') 32 | parser.add_argument('--seed', type=int, default=72, help='Random seed.') 33 | parser.add_argument('--epochs', type=int, default=300, help='Number of epochs to train.') 34 | parser.add_argument('--batch', type=int, default=256, help='batch size.') 35 | parser.add_argument('--emb_size', type=int, default=200, help='embed size.') 36 | parser.add_argument('--lr', type=float, default=0.001, help='Initial learning rate.') 37 | parser.add_argument('--weight_decay', type=float, default=0.001, help='Weight decay (L2 loss on parameters).') 38 | parser.add_argument('--dropout', type=float, default=0.5, help='Dropout rate (1 - keep probability).') 39 | parser.add_argument('--log', type=str, default='logs/{}'.format(method_name), help='log directory') 40 | parser.add_argument('--pos-weight', type=float, default=1.0, help='weight for positive samples') 41 | 42 | parser.add_argument('--reg', type=float, default=1.0, help='lambda reg') 43 | 44 | parser.add_argument('--self', type=float, default=1.0, help='lambda rec') 45 | parser.add_argument('--cross', type=float, default=1.0, help='lambda rec') 46 | parser.add_argument('--gan', type=float, default=1.0, help='lambda gan') 47 | parser.add_argument('--d_epoch', type=int, default=2, help='d epoch') 48 | parser.add_argument('--t_percent', type=float, default=1.0, help='target percent') 49 | parser.add_argument('--s_percent', type=float, default=1.0, help='source percent') 50 | parser.add_argument('--dataset', type=str, default='amazon3', help='amazon') 51 | 52 | args = parser.parse_args() 53 | 54 | args.cuda = torch.cuda.is_available() 55 | 56 | # kwargs = {'num_workers': 1, 'pin_memory': True} if args.cuda else {} 57 | 58 | def sigmoid(x): 59 | return 1 / (1 + np.exp(-x)) 60 | 61 | def main(): 62 | log = os.path.join(args.log, '{}_{}_{}_{}_{}_{}_{}_{}_{}'.format(args.dataset, args.emb_size, args.weight_decay, args.self, 63 | args.cross, args.gan, args.reg, args.t_percent, args.s_percent)) 64 | if os.path.isdir(log): 65 | print("%s already exist. are you sure to override? Ok, I'll wait for 5 seconds. Ctrl-C to abort." % log) 66 | time.sleep(5) 67 | os.system('rm -rf %s/' % log) 68 | 69 | os.makedirs(log) 70 | print("made the log directory", log) 71 | 72 | print('preparing data...') 73 | dataset = Dataset(args.batch, dataset=args.dataset) 74 | 75 | NUM_USER = dataset.num_user 76 | NUM_MOVIE = dataset.num_movie 77 | NUM_BOOK = dataset.num_book 78 | 79 | print('Preparing the training data......') 80 | # prepare data for X 81 | row, col = dataset.get_part_train_indices('movie', args.s_percent) 82 | values = np.ones(row.shape[0]) 83 | user_x = csr_matrix((values, (row, col)), shape=(NUM_USER, NUM_MOVIE)).toarray() 84 | 85 | # prepare data fot Y 86 | row, col = dataset.get_part_train_indices('book', args.t_percent) 87 | values = np.ones(row.shape[0]) 88 | user_y = csr_matrix((values, (row, col)), shape=(NUM_USER, NUM_BOOK)).toarray() 89 | 90 | print('Preparing the training data over......') 91 | # for shared user one-hot representation 92 | user_id = np.arange(NUM_USER).reshape([NUM_USER, 1]) 93 | 94 | user_x = torch.FloatTensor(user_x) 95 | user_y = torch.FloatTensor(user_y) 96 | 97 | train_loader = torch.utils.data.DataLoader(torch.from_numpy(user_id), 98 | batch_size=args.batch, 99 | shuffle=True) 100 | save_loader = torch.utils.data.DataLoader(torch.from_numpy(user_id), 101 | batch_size=args.batch, 102 | shuffle=False) 103 | 104 | pos_weight = torch.FloatTensor([args.pos_weight]) 105 | 106 | if args.cuda: 107 | pos_weight = pos_weight.cuda() 108 | 109 | model = CDAE(NUM_USER=NUM_USER, NUM_MOVIE=NUM_MOVIE, NUM_BOOK=NUM_BOOK, 110 | EMBED_SIZE=args.emb_size, dropout=args.dropout) 111 | disc1 = Discriminator(args.emb_size, args.dropout) 112 | disc2 = Discriminator(args.emb_size, args.dropout) 113 | optimizer_g = optim.Adam(model.parameters(), lr=args.lr, weight_decay=args.weight_decay) 114 | optimizer_d = optim.SGD(itertools.chain(disc1.parameters(), disc2.parameters()), lr=args.lr, weight_decay=args.weight_decay) 115 | 116 | BCEWL = torch.nn.BCEWithLogitsLoss(reduction='none', pos_weight=pos_weight) 117 | BCEL = torch.nn.BCEWithLogitsLoss(reduction='none') 118 | 119 | if args.cuda: 120 | model = model.cuda() 121 | disc1 = disc1.cuda() 122 | disc2 = disc2.cuda() 123 | 124 | # prepare data fot test process 125 | movie_vali, movie_test, movie_nega = dataset.movie_vali, dataset.movie_test, dataset.movie_nega 126 | book_vali, book_test, book_nega = dataset.book_vali, dataset.book_test, dataset.book_nega 127 | feed_data = {} 128 | feed_data['fts1'] = user_x 129 | feed_data['fts2'] = user_y 130 | feed_data['movie_vali'] = movie_vali 131 | feed_data['book_vali'] = book_vali 132 | feed_data['movie_test'] = movie_test 133 | feed_data['book_test'] = book_test 134 | feed_data['movie_nega'] = movie_nega 135 | feed_data['book_nega'] = book_nega 136 | 137 | best_hr1, best_ndcg1, best_mrr1 = 0.0, 0.0, 0.0 138 | best_hr2, best_ndcg2, best_mrr2 = 0.0, 0.0, 0.0 139 | val_hr1_list, val_ndcg1_list, val_mrr1_list = [], [], [] 140 | val_hr2_list, val_ndcg2_list, val_mrr2_list = [], [], [] 141 | 142 | G_loss_list = [] 143 | D_loss_list = [] 144 | reg_loss_list = [] 145 | loss_list = [] 146 | self_loss_list = [] 147 | cross_loss_list = [] 148 | JRL_loss_list = [] 149 | 150 | epoch_time_list = [] 151 | for epoch in range(args.epochs): 152 | model.train() 153 | batch_G_loss_list = [] 154 | batch_D_loss_list = [] 155 | batch_loss_list = [] 156 | batch_reg_loss_list = [] 157 | batch_self_loss_list = [] 158 | batch_cross_loss_list = [] 159 | batch_JRL_loss_list = [] 160 | 161 | epoch_time = 0.0 162 | for batch_idx, data in enumerate(train_loader): 163 | data = data.reshape([-1]) 164 | 165 | if (batch_idx+1) % (args.d_epoch + 1) == 0: 166 | optimizer_d.zero_grad() 167 | # standard Gaussian distribution 168 | prior = torch.from_numpy(np.random.normal(0, 1.0, size=[data.shape[0], args.emb_size])).float() 169 | # laplace distribution 170 | # prior = torch.from_numpy(np.random.laplace(0, 1.0, size=[data.shape[0], args.emb_size])).float() 171 | # uniform distribution 172 | # prior = torch.from_numpy(np.random.uniform(0, 1.0, size=[data.shape[0], args.emb_size])).float() 173 | # multi-variate Gaussian distribution 174 | # prior = 0.5*torch.from_numpy(np.random.normal(0, 1.0, size=[data.shape[0], args.emb_size])+ 175 | # np.random.normal(3, 1.0, size=[data.shape[0], args.emb_size])).float() 176 | if args.cuda: 177 | prior = prior.cuda() 178 | 179 | if args.cuda: 180 | batch_user = data.cuda() 181 | batch_user_x = user_x[data].cuda() 182 | batch_user_y = user_y[data].cuda() 183 | else: 184 | batch_user = data 185 | batch_user_x = user_x[data] 186 | batch_user_y = user_y[data] 187 | 188 | time1 = time.time() 189 | _, _, _, _, z_x, z_y, z_x_reg_loss, z_y_reg_loss = model.forward(batch_user, batch_user_x, batch_user_y) 190 | time2 = time.time() 191 | epoch_time += time2-time1 192 | 193 | true1 = disc1(prior).reshape([-1]) 194 | true2 = disc2(prior).reshape([-1]) 195 | fake1 = disc1(z_x).reshape([-1]) 196 | fake2 = disc2(z_y).reshape([-1]) 197 | 198 | dis_loss1 = BCEL(torch.cat([true1, fake1], 0), 199 | torch.cat([torch.ones_like(true1), torch.zeros_like(fake1)], 0)).sum() 200 | dis_loss2 = BCEL(torch.cat([true2, fake2], 0), 201 | torch.cat([torch.ones_like(true2), torch.zeros_like(fake2)], 0)).sum() 202 | 203 | D_loss = dis_loss1 + dis_loss2 204 | 205 | D_loss.backward() 206 | optimizer_d.step() 207 | batch_D_loss_list.append(D_loss.item()) 208 | 209 | else: 210 | optimizer_g.zero_grad() 211 | 212 | if args.cuda: 213 | batch_user = data.cuda() 214 | batch_user_x = user_x[data].cuda() 215 | batch_user_y = user_y[data].cuda() 216 | else: 217 | batch_user_x = user_x[data] 218 | batch_user_y = user_y[data] 219 | 220 | time1 = time.time() 221 | pred_x, pred_y, pred_x2y, pred_y2x, z_x, z_y, z_x_reg_loss, z_y_reg_loss = model.forward(batch_user, batch_user_x, batch_user_y) 222 | time2 = time.time() 223 | epoch_time += time2 - time1 224 | 225 | loss_x = BCEWL(pred_x, batch_user_x).sum() 226 | loss_y = BCEWL(pred_y, batch_user_y).sum() 227 | loss_x2y = BCEWL(pred_x2y, batch_user_y).sum() 228 | loss_y2x = BCEWL(pred_y2x, batch_user_x).sum() 229 | 230 | reg_loss = z_x_reg_loss.sum() + z_y_reg_loss.sum() 231 | 232 | fake_x = disc1(z_x).reshape([-1]) 233 | fake_y = disc2(z_y).reshape([-1]) 234 | 235 | G_loss1 = BCEL(fake_x, torch.ones_like(fake_x)).sum() 236 | G_loss2 = BCEL(fake_y, torch.ones_like(fake_y)).sum() 237 | G_loss = G_loss1+G_loss2 238 | 239 | # get plot JRL loss 240 | JRL_loss = args.self * (loss_x + loss_y) + args.cross * (loss_x2y + loss_y2x) + args.reg*reg_loss 241 | batch_JRL_loss_list.append(JRL_loss.item()/args.batch) 242 | 243 | # get the whole loss 244 | loss = G_loss + args.self * (loss_x + loss_y) + args.cross * (loss_x2y + loss_y2x) + args.reg*reg_loss 245 | loss.backward() 246 | optimizer_g.step() 247 | 248 | batch_G_loss_list.append(G_loss.item()) 249 | batch_self_loss_list.append(args.self * (loss_x + loss_y).item()) 250 | batch_cross_loss_list.append(args.cross * (loss_x2y + loss_y2x).item()) 251 | batch_loss_list.append(loss.item()) 252 | batch_reg_loss_list.append(args.reg*reg_loss.item()) 253 | 254 | # epoch_time_list.append(epoch_time) 255 | # print('epoch time:{:.4f}'.format(epoch_time)) 256 | 257 | epoch_G_loss = np.mean(batch_G_loss_list) 258 | epoch_D_loss = np.mean(batch_D_loss_list) 259 | epoch_reg_loss = np.mean(batch_reg_loss_list) 260 | epoch_loss = np.mean(batch_loss_list) 261 | epoch_self_loss = np.mean(batch_self_loss_list) 262 | epoch_cross_loss = np.mean(batch_cross_loss_list) 263 | epoch_JRL_Loss = np.mean(batch_JRL_loss_list) 264 | 265 | G_loss_list.append(epoch_G_loss) 266 | D_loss_list.append(epoch_D_loss) 267 | loss_list.append(epoch_loss) 268 | self_loss_list.append(epoch_self_loss) 269 | cross_loss_list.append(epoch_cross_loss) 270 | reg_loss_list.append(epoch_reg_loss) 271 | JRL_loss_list.append(epoch_JRL_Loss) 272 | 273 | print('epoch:{}, self loss:{:.4f}, cross loss:{:.4f}, G loss:{:.4f}, D loss:{:.4f}, reg loss:{:.4f}'.format(epoch,epoch_self_loss, 274 | epoch_cross_loss, 275 | epoch_G_loss, 276 | epoch_D_loss, 277 | epoch_reg_loss)) 278 | with open(log + '/tmp.txt', 'a') as f: 279 | f.write('epoch:{}, self loss:{:.4f}, cross loss:{:.4f}, G loss:{:.4f}, D loss:{:.4f}, reg loss:{:.4f}\n'.format( 280 | epoch, 281 | epoch_self_loss, epoch_cross_loss, epoch_G_loss, epoch_D_loss, epoch_reg_loss)) 282 | 283 | if epoch % 1 == 0: 284 | model.eval() 285 | 286 | avg_hr1, avg_ndcg1, avg_mrr1, avg_hr2, avg_ndcg2, avg_mrr2 = test_process(model, train_loader, feed_data, 287 | args.cuda, topK_list[1], mode='val') 288 | 289 | val_hr1_list.append(avg_hr1) 290 | val_ndcg1_list.append(avg_ndcg1) 291 | val_mrr1_list.append(avg_mrr1) 292 | val_hr2_list.append(avg_hr2) 293 | val_ndcg2_list.append(avg_ndcg2) 294 | val_mrr2_list.append(avg_mrr2) 295 | 296 | print('test: movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 297 | .format(avg_hr1, avg_ndcg1, avg_mrr1, avg_hr2, avg_ndcg2, avg_mrr2)) 298 | with open(log + '/tmp.txt', 'a') as f: 299 | f.write('test: movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}\n' 300 | .format(avg_hr1, avg_ndcg1, avg_mrr1, avg_hr2, avg_ndcg2, avg_mrr2)) 301 | 302 | if avg_hr1 > best_hr1: 303 | best_hr1 = avg_hr1 304 | torch.save(model.state_dict(), os.path.join(log, 'best_hr1.pkl')) 305 | 306 | if avg_ndcg1 > best_ndcg1: 307 | torch.save(model.state_dict(), os.path.join(log, 'best_ndcg1.pkl')) 308 | best_ndcg1 = avg_ndcg1 309 | if avg_mrr1 > best_mrr1: 310 | torch.save(model.state_dict(), os.path.join(log, 'best_mrr1.pkl')) 311 | best_mrr1 = avg_mrr1 312 | if avg_hr2 > best_hr2: 313 | torch.save(model.state_dict(), os.path.join(log, 'best_hr2.pkl')) 314 | best_hr2 = avg_hr2 315 | if avg_ndcg2 > best_ndcg2: 316 | torch.save(model.state_dict(), os.path.join(log, 'best_ndcg2.pkl')) 317 | best_ndcg2 = avg_ndcg2 318 | if avg_mrr2 > best_mrr2: 319 | torch.save(model.state_dict(), os.path.join(log, 'best_mrr2.pkl')) 320 | best_mrr2 = avg_mrr2 321 | 322 | 323 | print('best val movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 324 | .format(best_hr1, best_ndcg1, best_mrr1, best_hr2, best_ndcg2, best_mrr2)) 325 | with open(log + '/tmp.txt', 'a') as f: 326 | f.write('best val movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}\n' 327 | .format(best_hr1, best_ndcg1, best_mrr1, best_hr2, best_ndcg2, best_mrr2)) 328 | 329 | # # save necessary plot loss and validation metric 330 | # pickle.dump(loss_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 331 | # '{}_{}_loss_list'.format(args.dataset, args.gan)), 'wb')) 332 | # pickle.dump(JRL_loss_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 333 | # '{}_{}_JRL_loss_list'.format(args.dataset, args.gan)), 'wb')) 334 | # pickle.dump(val_hr1_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 335 | # '{}_{}_val_hr1_list'.format(args.dataset, args.gan)), 'wb')) 336 | # pickle.dump(val_ndcg1_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 337 | # '{}_{}_val_ndcg1_list'.format(args.dataset, args.gan)), 'wb')) 338 | # pickle.dump(val_mrr1_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 339 | # '{}_{}_val_mrr1_list'.format(args.dataset, args.gan)), 'wb')) 340 | # pickle.dump(val_hr2_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 341 | # '{}_{}_val_hr2_list'.format(args.dataset, args.gan)), 'wb')) 342 | # pickle.dump(val_ndcg2_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 343 | # '{}_{}_val_ndcg2_list'.format(args.dataset, args.gan)), 'wb')) 344 | # pickle.dump(val_mrr2_list, open(os.path.join(os.getcwd(), 'saved_embeddings', method_name, 345 | # '{}_{}_val_mrr2_list'.format(args.dataset, args.gan)), 'wb')) 346 | 347 | # print('mean epoch time:{:.4f}'.format(np.mean(epoch_time_list))) 348 | print('Val process over!') 349 | print('Test process......') 350 | for topK in topK_list: 351 | model.load_state_dict(torch.load(os.path.join(log, 'best_hr1.pkl'))) 352 | test_hr1, _, _, _, _, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 353 | 354 | # if topK==10: 355 | # user_embedding1, user_embedding2 = save_embedding_process(model, save_loader, feed_data, args.cuda) 356 | # pickle.dump(user_embedding1, open( 357 | # os.path.join(os.getcwd(), 'saved_embeddings', method_name, '{}_Z_movie.pkl'.format(args.dataset)), 'wb')) 358 | # pickle.dump(user_embedding2, open( 359 | # os.path.join(os.getcwd(), 'saved_embeddings', method_name, '{}_Z_book.pkl'.format(args.dataset)), 'wb' 360 | 361 | model.load_state_dict(torch.load(os.path.join(log, 'best_ndcg1.pkl'))) 362 | _, test_ndcg1, _, _, _, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 363 | model.load_state_dict(torch.load(os.path.join(log, 'best_mrr1.pkl'))) 364 | _, _, test_mrr1, _, _, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 365 | model.load_state_dict(torch.load(os.path.join(log, 'best_hr2.pkl'))) 366 | _, _, _, test_hr2, _, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 367 | model.load_state_dict(torch.load(os.path.join(log, 'best_ndcg2.pkl'))) 368 | _, _, _, _, test_ndcg2, _ = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 369 | model.load_state_dict(torch.load(os.path.join(log, 'best_mrr2.pkl'))) 370 | _, _, _, _, _, test_mrr2 = test_process(model, train_loader, feed_data, args.cuda, topK, mode='test') 371 | print('Test TopK:{} ---> movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}' 372 | .format(topK, test_hr1, test_ndcg1, test_mrr1, test_hr2, test_ndcg2, test_mrr2)) 373 | with open(log + '/tmp.txt', 'a') as f: 374 | f.write('Test TopK:{} ---> movie: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}, book: hr:{:.4f},ndcg:{:.4f},mrr:{:.4f}\n' 375 | .format(topK, test_hr1, test_ndcg1, test_mrr1, test_hr2, test_ndcg2, test_mrr2)) 376 | 377 | 378 | 379 | if __name__ == "__main__": 380 | print(args) 381 | main() 382 | print(args) --------------------------------------------------------------------------------