├── .gitignore ├── .idea ├── .gitignore ├── ddqn_quantization.iml ├── inspectionProfiles │ └── profiles_settings.xml ├── misc.xml └── modules.xml ├── README.md ├── __pycache__ ├── agent.cpython-38.pyc ├── dataloaders.cpython-38.pyc ├── dqn.cpython-38.pyc ├── fcn.cpython-38.pyc ├── helper.cpython-38.pyc ├── identical_compressor.cpython-38.pyc ├── logger.cpython-38.pyc ├── mec_env.cpython-38.pyc ├── options.cpython-38.pyc ├── pe.cpython-38.pyc ├── ps_quantizer.cpython-38.pyc ├── qsgd_compressor.cpython-38.pyc └── resnet.cpython-38.pyc ├── agent.py ├── bar_chart.py ├── dataloaders.py ├── dqn.py ├── fcn.py ├── federated_learning.py ├── helper.py ├── identical_compressor.py ├── line_chart.py ├── logger.py ├── mec_env.py ├── model ├── __pycache__ │ └── pe.cpython-38.pyc ├── checkpoint ├── train_model-q.data-00000-of-00001 ├── train_model-q.index └── train_model-q.meta ├── opt_weight ├── acc │ ├── dqn_0.1.txt │ ├── dqn_0.3.txt │ ├── dqn_0.5.txt │ ├── dqn_0.7.txt │ └── dqn_0.9.txt ├── loss │ ├── dqn_0.1.txt │ ├── dqn_0.3.txt │ ├── dqn_0.5.txt │ ├── dqn_0.7.txt │ └── dqn_0.9.txt ├── qerror.txt ├── reward.txt └── traintime.txt ├── options.py ├── pe.py ├── plot_reward.py ├── ps_quantizer.py ├── q_fl_dqn.py ├── qsgd_compressor.py ├── resnet.py ├── test ├── dir │ ├── testNov_03_2022_01_11_51.npz │ └── testNov_03_2022_17_30_04.npz └── step_result │ └── dir │ ├── step_resultNov_02_2022_15_21_32.npz │ ├── step_resultNov_02_2022_16_35_09.npz │ ├── step_resultNov_02_2022_17_48_54.npz │ ├── step_resultNov_02_2022_19_02_37.npz │ ├── step_resultNov_02_2022_20_16_47.npz │ ├── step_resultNov_02_2022_21_30_58.npz │ ├── step_resultNov_02_2022_22_44_32.npz │ ├── step_resultNov_02_2022_23_58_06.npz │ ├── step_resultNov_03_2022_01_11_51.npz │ └── step_resultNov_03_2022_17_30_04.npz ├── test_q_fl_dqn.py ├── test_results ├── acc │ ├── ada.txt │ ├── dqn_q.txt │ ├── fix_10bit.txt │ ├── fix_2bit.txt │ └── fix_6bit.txt ├── ada_time.txt ├── dqn_q_time.txt ├── error10.txt ├── error6.txt ├── error_ada.txt ├── error_dqn.txt ├── fix_10_time.txt ├── fix_6_time.txt ├── longtermreward1.txt ├── loss │ ├── ada.txt │ ├── dqn_q.txt │ ├── fix_10bit.txt │ ├── fix_2bit.txt │ └── fix_6bit.txt ├── qerror1.txt ├── reward10.txt ├── reward6.txt ├── rewardada.txt ├── rewarddqn.txt └── traintime1.txt └── train ├── a1bathnormal_env_max_ep2000_tep_1000_buffersize250000_maxstep500000_prestep25000_lr0.05_plot_minig+T_tfacter0.5_num4.npz ├── a2bathnormal_env_max_ep2000_tep_1000_buffersize250000_maxstep500000_prestep25000_lr0.05_plot_minig+T_tfacter0.5_num4.npz └── a3bathnormal_env_max_ep2000_tep_1000_buffersize250000_maxstep500000_prestep25000_lr0.05_plot_minig+T_tfacter0.5_num4.npz /.gitignore: -------------------------------------------------------------------------------- 1 | data/ 2 | -------------------------------------------------------------------------------- /.idea/.gitignore: -------------------------------------------------------------------------------- 1 | # Default ignored files 2 | /shelf/ 3 | /workspace.xml 4 | -------------------------------------------------------------------------------- /.idea/ddqn_quantization.iml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 12 | -------------------------------------------------------------------------------- /.idea/inspectionProfiles/profiles_settings.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 6 | -------------------------------------------------------------------------------- /.idea/misc.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | -------------------------------------------------------------------------------- /.idea/modules.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 数据集为公开数据集，请自行下载，在根目录下创建```data```文件夹，内容如下 2 | ``` 3 | | cifar-10-python.tar.gz 4 | | 5 | +---cifar-10-batches-py 6 | | batches.meta 7 | | data_batch_1 8 | | data_batch_2 9 | | data_batch_3 10 | | data_batch_4 11 | | data_batch_5 12 | | readme.html 13 | | test_batch 14 | | 15 | \---mnist 16 | +---MNIST 17 | | \---raw 18 | | t10k-images-idx3-ubyte 19 | | t10k-images-idx3-ubyte.gz 20 | | t10k-labels-idx1-ubyte 21 | | t10k-labels-idx1-ubyte.gz 22 | | train-images-idx3-ubyte 23 | | train-images-idx3-ubyte.gz 24 | | train-labels-idx1-ubyte 25 | | train-labels-idx1-ubyte.gz 26 | | 27 | +---processed 28 | | test.pt 29 | | training.pt 30 | | 31 | \---raw 32 | t10k-images-idx3-ubyte 33 | t10k-images-idx3-ubyte.gz 34 | t10k-labels-idx1-ubyte 35 | t10k-labels-idx1-ubyte.gz 36 | train-images-idx3-ubyte 37 | train-images-idx3-ubyte.gz 38 | train-labels-idx1-ubyte 39 | train-labels-idx1-ubyte.gz 40 | ``` 41 | -------------------------------------------------------------------------------- /__pycache__/agent.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/__pycache__/agent.cpython-38.pyc -------------------------------------------------------------------------------- /__pycache__/dataloaders.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/__pycache__/dataloaders.cpython-38.pyc -------------------------------------------------------------------------------- /__pycache__/dqn.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/__pycache__/dqn.cpython-38.pyc -------------------------------------------------------------------------------- /__pycache__/fcn.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/__pycache__/fcn.cpython-38.pyc -------------------------------------------------------------------------------- /__pycache__/helper.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/__pycache__/helper.cpython-38.pyc -------------------------------------------------------------------------------- /__pycache__/identical_compressor.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/__pycache__/identical_compressor.cpython-38.pyc -------------------------------------------------------------------------------- /__pycache__/logger.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/__pycache__/logger.cpython-38.pyc -------------------------------------------------------------------------------- /__pycache__/mec_env.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/__pycache__/mec_env.cpython-38.pyc -------------------------------------------------------------------------------- /__pycache__/options.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/__pycache__/options.cpython-38.pyc -------------------------------------------------------------------------------- /__pycache__/pe.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/__pycache__/pe.cpython-38.pyc -------------------------------------------------------------------------------- /__pycache__/ps_quantizer.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/__pycache__/ps_quantizer.cpython-38.pyc -------------------------------------------------------------------------------- /__pycache__/qsgd_compressor.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/__pycache__/qsgd_compressor.cpython-38.pyc -------------------------------------------------------------------------------- /__pycache__/resnet.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/__pycache__/resnet.cpython-38.pyc -------------------------------------------------------------------------------- /agent.py: -------------------------------------------------------------------------------- 1 | import os 2 | import random 3 | 4 | import numpy as np 5 | import tensorflow.compat.v1 as tf 6 | from options import args_parser 7 | from dqn import * 8 | import ipdb as pdb 9 | import matplotlib.pyplot as plt 10 | 11 | args = args_parser() 12 | 13 | alpha = 2.0 14 | ref_loss = 0.001 15 | # args.num_users = 10 16 | INITIAL_EPSILON = 0.5 17 | FINAL_EPSILON = 0.01 18 | 19 | class DQNAgent(object): 20 | """docstring for DQNAgent""" 21 | 22 | def __init__(self, sess, user_config, train_config): 23 | self.sess = sess 24 | self.user_id = user_config['id'] 25 | self.state_dim = user_config['state_dim'] 26 | self.action_dim = user_config['action_dim'] 27 | self.action_bound = user_config['action_bound'] 28 | self.action_level = user_config['action_level'] 29 | self.minibatch_size = int(train_config['minibatch_size']) 30 | self.epsilon = float(train_config['epsilon']) 31 | 32 | 33 | self.action_nums = 25# 1 -->2 34 | # for i in range(self.action_dim): 35 | # self.action_nums *= self.action_level 36 | 37 | self.max_step = 500000 # 500000 38 | # self.pre_train_steps = 25000 39 | self.pre_train_steps = 25000 40 | self.total_step = 0 41 | self.DQN = DeepQNetwork(sess, self.state_dim, self.action_nums, float(train_config['critic_lr']), float(train_config['tau']), float(train_config['gamma']), self.user_id) 42 | self.replay_buffer = ReplayBuffer(int(train_config['buffer_size']), int(train_config['random_seed'])) 43 | 44 | def init_target_network(self): 45 | self.DQN.update_target_network() 46 | 47 | def predict(self, s): 48 | if self.total_step <= self.max_step: 49 | self.epsilon *= 0.9999953948404178 50 | # print (self.epsilon) 51 | # print (np.random.rand(1) < self.epsilon or self.total_step < self.pre_train_steps) 52 | # random.seed(1) # np.random(1) 53 | if np.random.rand(1) < self.epsilon or self.total_step < self.pre_train_steps: 54 | #self.epsilon -= (INITIAL_EPSILON - FINAL_EPSILON) / 10000 55 | action = np.random.randint(self.action_nums) 56 | else: 57 | #self.epsilon -= (INITIAL_EPSILON - FINAL_EPSILON) / 10000 58 | action, _ = self.DQN.predict(np.reshape(s, (1, self.state_dim))) 59 | 60 | self.total_step += 1 61 | # print ('self.total_step:',self.total_step) 62 | # print (' self.epsilon:', self.epsilon) 63 | return action 64 | 65 | def update(self, s, a, r, t, s2): 66 | self.replay_buffer.add(np.reshape(s, (self.state_dim,)), a, r, 67 | t, np.reshape(s2, (self.state_dim,))) 68 | 69 | if self.replay_buffer.size() > self.minibatch_size: 70 | s_batch, a_batch, r_batch, t_batch, s2_batch = \ 71 | self.replay_buffer.sample_batch(self.minibatch_size) 72 | 73 | # calculate targets 74 | _, q_out = self.DQN.predict(s_batch) 75 | target_prediction, target_q_out = self.DQN.predict_target(s2_batch) 76 | 77 | for k in range(self.minibatch_size): 78 | if t_batch[k]: 79 | q_out[k][a_batch[k]] = r_batch[k] # 10/5 [0]-2 80 | else: 81 | q_out[k][a_batch[k]] = r_batch[k] + self.DQN.gamma * target_q_out[k][target_prediction[k]] 82 | 83 | # Update the critic given the targets 84 | q_loss, _ = self.DQN.train( 85 | s_batch, q_out) 86 | 87 | # losses.append(q_loss) 88 | # Update target networks 89 | self.DQN.update_target_network() 90 | ''' 91 | def update(self, s, a, r, t, s2): 92 | self.replay_buffer.add(np.reshape(s, (self.state_dim,)), a, r, 93 | t, np.reshape(s2, (self.state_dim,))) 94 | 95 | if self.replay_buffer.size() > self.minibatch_size: 96 | s_batch, a_batch, r_batch, t_batch, s2_batch = \ 97 | self.replay_buffer.sample_batch(self.minibatch_size) 98 | 99 | # calculate targets 100 | _, q_out = self.DQN.predict(s_batch) 101 | target_prediction, target_q_out = self.DQN.predict_target(s2_batch) 102 | 103 | for k in range(self.minibatch_size): 104 | if t_batch[k]: 105 | q_out[k][a_batch[k]] = r_batch[k] # 10/5 [0]-2 106 | else: 107 | q_out[k][a_batch[k]] = r_batch[k] + self.DQN.gamma * target_q_out[k][target_prediction[k]] 108 | 109 | # Update the critic given the targets 110 | q_loss, _ = self.DQN.train( 111 | s_batch, q_out) 112 | 113 | # losses.append(q_loss) 114 | # Update target networks 115 | self.DQN.update_target_network() 116 | ''' -------------------------------------------------------------------------------- /bar_chart.py: -------------------------------------------------------------------------------- 1 | import matplotlib.pyplot as plt 2 | import numpy as np 3 | 4 | # 读取txt文件中的数据 5 | data = [] 6 | with open('opt_weight/traintime.txt', 'r') as file: 7 | for line in file: 8 | data.append(float(line.strip())) 9 | 10 | name = ["Value of w1"] 11 | x = np.arange(len(name)) 12 | width = 0.25 13 | # fig = plt.figure() 14 | plt.tick_params(bottom=False) 15 | plt.bar(x, data[0], width=width, label='w1=0.1', color='r') 16 | plt.bar(x + width, data[1], width=width, label='w1=0.3', color='#CC6600', tick_label="") 17 | plt.bar(x + 2 * width, data[2], width=width, label='w1=0.5', color='#66CCFF') 18 | plt.bar(x + 3 * width, data[3], width=width, label='w1=0.7', color='g') 19 | plt.bar(x + 4 * width, data[4], width=width, label='w1=0.9', color='m') 20 | 21 | plt.xticks() 22 | # plt.ylabel('Long-term Discounted Reward') 23 | # plt.ylabel('Average quantization error') 24 | plt.ylabel('Average total training time') 25 | plt.xlabel('Value of w1') 26 | plt.grid(axis='y', linestyle=':') 27 | plt.legend(loc='lower right') 28 | plt.savefig('time_w.eps', format='eps') 29 | plt.show() -------------------------------------------------------------------------------- /dataloaders.py: -------------------------------------------------------------------------------- 1 | import torch 2 | from torchvision import datasets, transforms 3 | 4 | 5 | def mnist(args): 6 | train_loader = torch.utils.data.DataLoader( 7 | datasets.MNIST('./data', train=True, download=True, 8 | transform=transforms.Compose([ 9 | transforms.ToTensor(), 10 | transforms.Normalize((0.1307,), (0.3081,)) 11 | ])), 12 | batch_size=args.batch_size*args.num_users, shuffle=True,) 13 | test_loader = torch.utils.data.DataLoader( 14 | datasets.MNIST('./data', train=False, transform=transforms.Compose([ 15 | transforms.ToTensor(), 16 | transforms.Normalize((0.1307,), (0.3081,)) 17 | ])), 18 | batch_size=args.test_batch_size, shuffle=True,) 19 | return train_loader, test_loader 20 | 21 | 22 | def cifar10(args): 23 | transform_train = transforms.Compose([ 24 | transforms.RandomCrop(32, padding=4), # 先四周填充0，在吧图像随机裁剪成32*32 25 | transforms.RandomHorizontalFlip(), # 图像一半的概率翻转，一半的概率不翻转 26 | transforms.ToTensor(), 27 | transforms.Normalize((0.4914, 0.4822, 0.4465), 28 | (0.2023, 0.1994, 0.2010)), # R,G,B每层的归一化用到的均值和方差 29 | ]) 30 | 31 | transform_test = transforms.Compose([ 32 | transforms.ToTensor(), 33 | transforms.Normalize((0.4914, 0.4822, 0.4465), 34 | (0.2023, 0.1994, 0.2010)), 35 | ]) 36 | 37 | trainset = datasets.CIFAR10( 38 | root='./data', train=True, download=True, transform=transform_train) # 训练数据集 39 | train_loader = torch.utils.data.DataLoader( 40 | trainset, batch_size=args.batch_size*args.num_users, shuffle=True, num_workers=2) # 生成一个个batch进行批训练，组成batch的时候顺序打乱取 41 | 42 | testset = datasets.CIFAR10( 43 | root='./data', train=False, download=True, transform=transform_test) 44 | test_loader = torch.utils.data.DataLoader( 45 | testset, args.test_batch_size, shuffle=False, num_workers=2) 46 | return train_loader, test_loader 47 | 48 | 49 | def cifar100(args): 50 | transform_train = transforms.Compose([ 51 | transforms.RandomCrop(32, padding=4), # 先四周填充0，在吧图像随机裁剪成32*32 52 | transforms.RandomHorizontalFlip(), # 图像一半的概率翻转，一半的概率不翻转 53 | transforms.ToTensor(), 54 | transforms.Normalize((0.4914, 0.4822, 0.4465), 55 | (0.2023, 0.1994, 0.2010)), # R,G,B每层的归一化用到的均值和方差 56 | ]) 57 | 58 | transform_test = transforms.Compose([ 59 | transforms.ToTensor(), 60 | transforms.Normalize((0.4914, 0.4822, 0.4465), 61 | (0.2023, 0.1994, 0.2010)), 62 | ]) 63 | 64 | trainset = datasets.CIFAR100( 65 | root='./data', train=True, download=True, transform=transform_train) # 训练数据集 66 | train_loader = torch.utils.data.DataLoader( 67 | trainset, batch_size=args.batch_size*args.num_users, shuffle=True, num_workers=2) # 生成一个个batch进行批训练，组成batch的时候顺序打乱取 68 | 69 | testset = datasets.CIFAR100( 70 | root='./data', train=False, download=True, transform=transform_test) 71 | test_loader = torch.utils.data.DataLoader( 72 | testset, args.test_batch_size, shuffle=False, num_workers=2) 73 | return train_loader, test_loader 74 | 75 | 76 | def stl10(args): 77 | transform_train = transforms.Compose([ 78 | transforms.RandomCrop(96, padding=4), # 先四周填充0，在吧图像随机裁剪成32*32 79 | transforms.RandomHorizontalFlip(), # 图像一半的概率翻转，一半的概率不翻转 80 | transforms.ToTensor(), 81 | transforms.Normalize((0.4914, 0.4822, 0.4465), 82 | (0.2023, 0.1994, 0.2010)), # R,G,B每层的归一化用到的均值和方差 83 | ]) 84 | 85 | transform_test = transforms.Compose([ 86 | transforms.ToTensor(), 87 | transforms.Normalize((0.4914, 0.4822, 0.4465), 88 | (0.2023, 0.1994, 0.2010)), 89 | ]) 90 | 91 | trainset = datasets.STL10( 92 | root='./data', split='train', download=True, transform=transform_train) # 训练数据集 93 | train_loader = torch.utils.data.DataLoader( 94 | trainset, batch_size=args.batch_size*args.num_users, shuffle=True, num_workers=2) # 生成一个个batch进行批训练，组成batch的时候顺序打乱取 95 | 96 | testset = datasets.STL10( 97 | root='./data', split='test', download=True, transform=transform_test) 98 | test_loader = torch.utils.data.DataLoader( 99 | testset, args.test_batch_size, shuffle=False, num_workers=2) 100 | return train_loader, test_loader 101 | 102 | 103 | 104 | def svhn(args): 105 | transform_train = transforms.Compose([ 106 | transforms.RandomCrop(32, padding=4), # 先四周填充0，在吧图像随机裁剪成32*32 107 | transforms.RandomHorizontalFlip(), # 图像一半的概率翻转，一半的概率不翻转 108 | transforms.ToTensor(), 109 | transforms.Normalize((0.4914, 0.4822, 0.4465), 110 | (0.2023, 0.1994, 0.2010)), # R,G,B每层的归一化用到的均值和方差 111 | ]) 112 | 113 | transform_test = transforms.Compose([ 114 | transforms.ToTensor(), 115 | transforms.Normalize((0.4914, 0.4822, 0.4465), 116 | (0.2023, 0.1994, 0.2010)), 117 | ]) 118 | 119 | trainset = datasets.SVHN( 120 | root='./data', split='train', download=True, transform=transform_train) # 训练数据集 121 | train_loader = torch.utils.data.DataLoader( 122 | trainset, batch_size=args.batch_size*args.num_users, shuffle=True, num_workers=2) # 生成一个个batch进行批训练，组成batch的时候顺序打乱取 123 | 124 | testset = datasets.SVHN( 125 | root='./data', split='test', download=True, transform=transform_test) 126 | test_loader = torch.utils.data.DataLoader( 127 | testset, args.test_batch_size, shuffle=False, num_workers=2) 128 | return train_loader, test_loader 129 | 130 | 131 | def tinyimgnet(args): 132 | 133 | traindir = './tinyimgnet/train' 134 | testdir = './tinyimgnet/val' 135 | normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406], 136 | std=[0.229, 0.224, 0.225]) 137 | 138 | train_dataset = datasets.ImageFolder( 139 | traindir, 140 | transforms.Compose([ 141 | transforms.RandomResizedCrop(224), 142 | transforms.RandomHorizontalFlip(), 143 | transforms.ToTensor(), 144 | normalize, 145 | ]) 146 | ) 147 | train_sampler = None 148 | train_loader = torch.utils.data.DataLoader( 149 | train_dataset, batch_size=args.batch_size*args.num_users, shuffle=(train_sampler is None), 150 | num_workers=8, pin_memory=True, sampler=train_sampler) 151 | 152 | test_loader = torch.utils.data.DataLoader( 153 | datasets.ImageFolder(testdir, transforms.Compose([ 154 | transforms.Resize(256), 155 | transforms.CenterCrop(224), 156 | transforms.ToTensor(), 157 | normalize, 158 | ])), 159 | batch_size=args.test_batch_size, shuffle=False, 160 | num_workers=8, pin_memory=True) 161 | 162 | return train_loader, test_loader 163 | 164 | -------------------------------------------------------------------------------- /dqn.py: -------------------------------------------------------------------------------- 1 | import tensorflow.compat.v1 as tf 2 | import numpy as np 3 | import tflearn 4 | from collections import deque 5 | import random 6 | import ipdb as pdb 7 | from options import args_parser 8 | 9 | args = args_parser() 10 | # args.num_users = 10 11 | 12 | # setting for hidden layers 13 | Layer1 = 400 14 | Layer2 = 300 15 | 16 | class DeepQNetwork(object): 17 | """ 18 | Input to the network is the state, output is a vector of Q(s,a). 19 | """ 20 | 21 | def __init__(self, sess, state_dim, action_dim, learning_rate, tau, gamma, user_id): 22 | self.sess = sess 23 | self.s_dim = state_dim 24 | self.a_dim = action_dim 25 | self.learning_rate = learning_rate 26 | self.tau = tau 27 | self.gamma = gamma 28 | self.user_id = user_id 29 | 30 | # Create the critic network 31 | self.inputs, self.q_out = self.create_deep_q_network() 32 | 33 | self.network_params = tf.trainable_variables() 34 | 35 | # Target Network 36 | self.target_inputs, self.target_q_out = self.create_deep_q_network() 37 | 38 | self.target_network_params = tf.trainable_variables()[len(self.network_params):] 39 | 40 | # Op for periodically updating target network with online network 41 | # weights with regularization 42 | self.update_target_network_params = \ 43 | [self.target_network_params[i].assign(tf.multiply(self.network_params[i], self.tau) \ 44 | + tf.multiply(self.target_network_params[i], 1. - self.tau)) 45 | for i in range(len(self.target_network_params))] 46 | 47 | # Network target (y_i) 48 | self.target_Q = tf.placeholder(tf.float32, [None, self.a_dim]) 49 | 50 | # Define loss and optimization Op 51 | self.loss = tflearn.mean_square(self.target_Q, self.q_out) 52 | self.optimize = tf.train.AdamOptimizer(self.learning_rate).minimize(self.loss) 53 | 54 | def create_deep_q_network(self): 55 | inputs = tflearn.input_data(shape=[None, self.s_dim], name="input_" + str(self.user_id)) 56 | net = tflearn.fully_connected(inputs, Layer1) 57 | net = tflearn.layers.normalization.batch_normalization(net) 58 | net = tflearn.activations.elu(net) 59 | 60 | net = tflearn.fully_connected(net, Layer2) 61 | net = tflearn.layers.normalization.batch_normalization(net) 62 | net = tflearn.activations.elu(net) 63 | 64 | # linear layer connected to 1 output representing Q(s,a) 65 | # Weights are init to Uniform[-3e-3, 3e-3] 66 | # w_init = tflearn.initializations.uniform(minval=-0.003, maxval=0.003) 67 | q_out = tflearn.fully_connected(net, self.a_dim, name="output_" + str(self.user_id)) 68 | return inputs, q_out 69 | 70 | def train(self, inputs, target_Q): 71 | return self.sess.run([self.loss, self.optimize], feed_dict={ 72 | self.inputs: inputs, 73 | self.target_Q: target_Q 74 | }) 75 | 76 | def predict(self, inputs): 77 | q_out = self.sess.run(self.q_out, feed_dict={ 78 | self.inputs: inputs 79 | }) 80 | return np.argmax(q_out, axis=1), q_out 81 | 82 | def predict_target(self, inputs): 83 | q_out = self.sess.run(self.target_q_out, feed_dict={ 84 | self.target_inputs: inputs 85 | }) 86 | return np.argmax(q_out, axis=1), q_out 87 | 88 | def update_target_network(self): 89 | self.sess.run(self.update_target_network_params) 90 | 91 | class ReplayBuffer(object): 92 | 93 | def __init__(self, buffer_size, random_seed=123): 94 | """ 95 | The right side of the deque contains the most recent experiences 96 | """ 97 | self.buffer_size = buffer_size 98 | self.count = 0 99 | self.buffer = deque() 100 | random.seed(random_seed) 101 | 102 | def add(self, s, a, r, t, s2): 103 | experience = (s, a, r, t, s2) 104 | if self.count < self.buffer_size: 105 | self.buffer.append(experience) 106 | self.count += 1 107 | else: 108 | self.buffer.popleft() 109 | self.buffer.append(experience) 110 | 111 | def size(self): 112 | return self.count 113 | 114 | def sample_batch(self, batch_size): 115 | batch = [] 116 | 117 | if self.count < batch_size: 118 | batch = random.sample(self.buffer, self.count) 119 | else: 120 | batch = random.sample(self.buffer, batch_size) 121 | 122 | s_batch = np.array([_[0] for _ in batch]) 123 | a_batch = np.array([_[1] for _ in batch]) 124 | r_batch = np.array([_[2] for _ in batch]) 125 | t_batch = np.array([_[3] for _ in batch]) 126 | s2_batch = np.array([_[4] for _ in batch]) 127 | 128 | return s_batch, a_batch, r_batch, t_batch, s2_batch 129 | 130 | def clear(self): 131 | self.buffer.clear() 132 | self.count = 0 -------------------------------------------------------------------------------- /fcn.py: -------------------------------------------------------------------------------- 1 | # https://pytorch.org/tutorials/beginner/pytorch_with_examples.html 2 | 3 | import torch 4 | 5 | class FCN(torch.nn.Module): 6 | def __init__(self, D_in=784, H=256, num_classes=10): 7 | """ 8 | In the constructor we instantiate two nn.Linear modules and assign them as 9 | member variables. 10 | """ 11 | super(FCN, self).__init__() 12 | self.linear1 = torch.nn.Linear(D_in, H) 13 | self.linear2 = torch.nn.Linear(H, num_classes) 14 | 15 | def forward(self, x): 16 | """ 17 | In the forward function we accept a Tensor of input data and we must return 18 | a Tensor of output data. We can use Modules defined in the constructor as 19 | well as arbitrary operators on Tensors. 20 | """ 21 | x = x.view(-1, 28 * 28) 22 | h_relu = self.linear1(x).clamp(min=0) 23 | y_pred = self.linear2(h_relu) 24 | return y_pred 25 | -------------------------------------------------------------------------------- /federated_learning.py: -------------------------------------------------------------------------------- 1 | # FL相关 2 | import tflearn 3 | 4 | from qsgd_compressor import * 5 | from datetime import datetime 6 | from identical_compressor import * 7 | import torch.nn as nn 8 | import torch.optim as optim 9 | import time 10 | from fcn import FCN 11 | from resnet import * 12 | from dataloaders import mnist 13 | from ps_quantizer import * 14 | import os 15 | from logger import Logger 16 | 17 | from mec_env import MecTerm, MecTermDQN, MecSvrEnv 18 | from helper import * 19 | from agent import * 20 | import tensorflow as tf 21 | from pe import * 22 | import matplotlib 23 | import matplotlib.pyplot as plt 24 | 25 | from options import * 26 | 27 | # DDPG相关 28 | os.environ['CUDA_VISIBLE_DEVICES'] = '-1' 29 | 30 | # FL相关初始化 31 | os.environ['KMP_DUPLICATE_LIB_OK'] = 'True' 32 | 33 | # DQN相关设置 34 | tf.compat.v1.reset_default_graph() 35 | MAX_EPISODE = 2 36 | MAX_EPISODE_LEN = 2 37 | NUM_R = args.num_users # 天线数 跟信道数有关 38 | 39 | SIGMA2 = 1e-9 # 噪声平方 10 -9 40 | 41 | config = {'state_dim': 4, 'action_dim': 2}; 42 | train_config = {'minibatch_size': 64, 43 | 'actor_lr': 0.0001, 44 | 'tau': 0.001, 45 | 'critic_lr': 0.05, # 0.001 46 | 'gamma': 0.99, 47 | 'buffer_size': 250000, # 250000 48 | 'random_seed': int(time.perf_counter() * 1000 % 1000), 49 | 'sigma2': SIGMA2, 50 | 'epsilon': 0.5} # 0.5 51 | 52 | IS_TRAIN = False 53 | 54 | res_path = 'train/' 55 | model_fold = 'model/' 56 | model_path = 'model/train_model-q' 57 | 58 | # 创建目录,用来保存dqn模型 59 | if not os.path.exists(res_path): 60 | os.mkdir(res_path) 61 | if not os.path.exists(model_fold): 62 | os.mkdir(model_fold) 63 | 64 | init_path = '' 65 | 66 | # choose the vehicle for training 67 | Train_vehicle_ID = 1 68 | 69 | # action_bound是需要后面调的 70 | user_config = [{'id': '1', 'model': 'AR', 'num_r': NUM_R, 'action_bound': 1}] 71 | 72 | # 0. initialize the session object 73 | sess = tf.compat.v1.Session() 74 | 75 | # 1. include all user in the system according to the user_config 76 | user_list = []; 77 | for info in user_config: 78 | info.update(config) 79 | info['model_path'] = model_path 80 | info['meta_path'] = info['model_path'] + '.meta' 81 | info['init_path'] = init_path 82 | info['action_level'] = 5 83 | user_list.append(MecTermDQN(sess, info, train_config)) 84 | print('Initialization OK!----> user ') 85 | 86 | # 2. create the simulation env 87 | env = MecSvrEnv(user_list, Train_vehicle_ID, SIGMA2, MAX_EPISODE_LEN) 88 | 89 | sess.run(tf.compat.v1.global_variables_initializer()) 90 | 91 | tflearn.config.is_training(is_training=IS_TRAIN, session=sess) 92 | 93 | env.init_target_network() 94 | 95 | res_r = [] # 每一步的平均奖励 96 | res_p = [] # 每个回合的平均动作 97 | 98 | args = args_parser() 99 | if args.logdir is None: 100 | assert False, "The logdir is not defined" 101 | LOGGER = Logger(args.logdir) 102 | 103 | args.no_cuda = args.no_cuda or not torch.cuda.is_available() 104 | 105 | torch.manual_seed(args.seed) 106 | device = torch.device("cpu" if args.no_cuda else "cuda") 107 | 108 | # train_loader, test_loader = mnist(args) # 235 10 109 | # model = FCN(num_classes=args.num_classes).to(device) 110 | # optimizer = optim.SGD(model.parameters(), lr=0.1, 111 | # momentum=args.momentum, weight_decay=args.weight_decay) 112 | ave_reward_ep = [] 113 | ave_ground_ep = [] 114 | ave_T_ep = [] 115 | ave_delay_ep = [] 116 | ave_error_ep = [] 117 | reward_step = [] 118 | train_loss, testing_accuracy = [], [] 119 | maxs = [] 120 | mins = [] 121 | # 开始训练DQN episode 122 | for episode in range(1, MAX_EPISODE): 123 | print(f'\n | Global Training Round/episode : {episode} |\n') 124 | # model.train() 125 | # batch_size = args.batch_size # 32 126 | # num_users = args.num_users # 8 127 | # train_data = list() 128 | # iteration = len(train_loader.dataset)//(num_users*batch_size) + \ 129 | # int(len(train_loader.dataset) % (num_users*batch_size) != 0) # 235 130 | # 记录间隔 # [23, 46, 69, 92, 115, 138, 161, 184, 207, 230] 131 | # log_interval = [iteration // args.log_epoch * (i+1) for i in range(args.log_epoch)] 132 | # Training 133 | 134 | tr_step_loss = [] 135 | tr_step_acc = [] 136 | val_acc_list, net_list = [], [] 137 | cv_loss, cv_acc = [], [] 138 | print_every = 2 139 | val_loss_pre, counter = 0, 0 140 | 141 | # DQN相关 142 | plt.ion() # 打开交互模式,画动态图 143 | cur_init_ds_ep = env.reset() # 环境重置 144 | cur_r_ep = 0 # 一个回合的总奖励 145 | cur_g_ep = 0 146 | cur_T_ep = 0 147 | cur_d_ep = 0 148 | cur_e_ep = 0 149 | step_cur_r_ep = [] 150 | count = 0 151 | # here the real batch size is (num_users * batch_size) 152 | # DQN时间步 & FL的一个通信轮次 153 | for epoch in range(1, MAX_EPISODE_LEN): 154 | i = Train_vehicle_ID - 1 155 | # if epoch == 1: 156 | # q_level1 = 1 157 | # if epoch % 5 == 0: 158 | actions = user_list[i].predict(True) 159 | print('step is : {}, q_level is: {}, power is: {}'.format(epoch, actions[0], actions[1])) 160 | # quantizer = Quantizer(QSGDCompressor, model.parameters(), args, actions[0]) 161 | # para = list(model.parameters()) 162 | # ini_para = para 163 | # FL本地训练迭代 164 | # for batch_idx, (data, target) in enumerate(train_loader): 165 | # user_batch_size = len(data) // num_users # 32 = 256//8 166 | # train_data.clear() 167 | # 给每个用户分配训练数据 168 | # for user_id in range(num_users-1): 169 | # train_data.append((data[user_id*user_batch_size:(user_id+1)*user_batch_size], 170 | # target[user_id*user_batch_size:(user_id+1)*user_batch_size])) 171 | # train_data.append((data[(num_users-1)*user_batch_size:], 172 | # target[(num_users-1)*user_batch_size:])) 173 | 174 | # 计算一次迭代的全局损失 175 | # loss = one_iter(model, device, LOSS_FUNC, optimizer, 176 | # quantizer, train_data, num_users, epoch=epoch) 177 | 178 | # 记录一个epoch的2个损失和测试精度 179 | # if (batch_idx+1) in log_interval: 180 | # train_loss.append(loss.item()) 181 | # test_accuracy = test(args, model, device, test_loader) 182 | # testing_accuracy.append(test_accuracy*100) 183 | # print('Train Episode: {} Train epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}\t Test Accuracy: {:.2f}%'.format( 184 | # episode, 185 | # epoch, 186 | # batch_idx * num_users * batch_size + len(data), 187 | # len(train_loader.dataset), 188 | # 100. * batch_idx / len(train_loader), 189 | # loss.item(), 190 | # test_accuracy*100)) 191 | 192 | # info = {'loss': loss.item(), 'accuracy(%)': test_accuracy*100} 193 | 194 | # for tag, value in info.items(): 195 | # LOGGER.scalar_summary( 196 | # tag, value, iteration*(epoch-1)+batch_idx) 197 | # para1 = list(model.parameters()) 198 | # maxx = max(para1[0].detach().numpy().reshape(1,200704).flatten()) 199 | # minn = min(abs(para1[0].detach().numpy().reshape(1,200704).flatten())) 200 | # maxs.append(maxx) 201 | # mins.append(minn) 202 | 203 | # print('Train Epoch: {} Done.\tLoss: {:.6f}'.format(epoch, loss.item())) 204 | 205 | reward = 0 206 | trs = 0 207 | deltas = 0 208 | diss = 0 209 | mini_ground = 0 210 | T_total = 0 211 | 212 | max_len = 1000 # MAX_EPISODE_LEN 213 | count += 1 214 | 215 | i = Train_vehicle_ID - 1 216 | # feedback the sinr to each user 217 | [reward, mini_ground, T_total, trs, deltas, diss, q_level, power, delay, q_error] = user_list[i].feedback( 218 | actions[0], actions[1]) 219 | print("reward is: {}, delay is: {}".format(reward, delay)) 220 | 221 | reward_step.append(reward) 222 | 223 | user_list[i].AgentUpdate(count >= max_len) # 训练数据个数逐渐增加 大于buffer的大小时 进行更新agent 224 | cur_r = reward 225 | cur_mg = mini_ground 226 | cur_T = T_total 227 | cur_q = q_level 228 | cur_d = delay 229 | cur_e = q_error 230 | done = count >= max_len # max_len即为MAX_EPISODE_LEN 231 | 232 | cur_r_ep += cur_r # 一个回合的总奖励 所有step的奖励之和 233 | cur_g_ep += cur_mg 234 | cur_T_ep += cur_T 235 | cur_d_ep += cur_d 236 | cur_e_ep += cur_e 237 | # for m in range(args.num_users): 238 | # cur_p_ep[m] += cur_p[m] 239 | 240 | # if done: # 一个episode结束 241 | # res_r.append(cur_r_ep / MAX_EPISODE_LEN) # 后面为了存储进模型 每一步的平均奖励 242 | 243 | # cur_p_ep1 = [0] * args.num_users 244 | # for m in range(args.num_users): 245 | # cur_p_ep1[m] = cur_p_ep[m] / MAX_EPISODE_LEN # 一个回合里平均每一个step的动作 246 | # res_p.append(cur_p_ep1) # 用来存储每个回合的平均动作 247 | 248 | print("episode = ", episode) 249 | print("r = ", cur_r_ep / MAX_EPISODE_LEN) 250 | print("delay = ", cur_d_ep / MAX_EPISODE_LEN) 251 | print("error = ", cur_e_ep / MAX_EPISODE_LEN) 252 | ave_reward_ep.append(cur_r_ep / MAX_EPISODE_LEN) 253 | ave_ground_ep.append(cur_g_ep / MAX_EPISODE_LEN) 254 | ave_T_ep.append(cur_T_ep / MAX_EPISODE_LEN) 255 | ave_delay_ep.append(cur_d_ep / MAX_EPISODE_LEN) 256 | ave_error_ep.append(cur_e_ep / MAX_EPISODE_LEN) 257 | # print("p_lambda = ", cur_p_ep1) # this is wrong 258 | # print("cur_r_ep = ", cur_r_ep) 259 | 260 | # line_reward = ax.plot(range(0, MAX_EPISODE), cur_r_ep, '#ff7f0e', label='reward', lw=1) 261 | # line_pro = ax2.plot(range(0, MAX_EPISODE), step_cur_r_ep[0], '#ff7f0e', label='第1辆车的the probility(选择每辆车的概率，即动作输出)', lw=1) 262 | plt.ioff() 263 | # upper = max(maxs) 264 | # lower = min(mins) 265 | # print("upper is", upper) 266 | # print("lower is", lower) 267 | # DDPG模型保存 268 | name = res_path + 'a1bathnormal_env_max_ep2000_tep_1000_buffersize250000_maxstep500000_prestep25000_lr0.05_plot_minig+T_tfacter0.5_num4' 269 | # np.savez(name, train_loss, testing_accuracy, ave_reward_ep, ave_delay_ep, ave_error_ep) # 保存平均每一步的q_level和奖励 # 为了后面的画图 270 | np.savez(name, ave_reward_ep, ave_ground_ep, ave_T_ep, ave_delay_ep, ave_error_ep) 271 | 272 | tflearn.config.is_training(is_training=False, session=sess) 273 | # Create a saver object which will save all the variables # 即保存模型参数 274 | saver = tf.compat.v1.train.Saver() 275 | saver.save(sess, model_path) 276 | sess.close() 277 | 278 | # if args.save_model: 279 | # filename = "saved_{}_{}.pt".format(args.network, datetime.now()) 280 | # torch.save(model.state_dict(), filename) 281 | 282 | matplotlib.use('TkAgg') 283 | os.environ["KMP_DUPLICATE_LIB_OK"] = "TRUE" 284 | 285 | ''' 286 | plt.figure() 287 | plt.plot(range(len(ave_reward_ep)), ave_reward_ep, color='b') 288 | plt.ylabel('Training reward') 289 | plt.xlabel('Episode') 290 | plt.rcParams['axes.unicode_minus']=False 291 | plt.show() 292 | # plt.savefig('train_reward.png') 293 | 294 | plt.figure() 295 | # plt.title('Training loss') 296 | plt.plot(range(len(train_loss)), train_loss, color='b') 297 | plt.ylabel('Training loss') 298 | plt.xlabel('Episode') 299 | plt.show() 300 | # plt.savefig('train_loss.png') 301 | ''' 302 | p_e() 303 | ''' 304 | plt.figure() 305 | # plt.title('Test Accuracy') 306 | plt.plot(range(len(testing_accuracy)), testing_accuracy, color='r') 307 | plt.ylabel('Testing Accuracy') 308 | plt.xlabel('Episode') 309 | plt.show() 310 | # plt.savefig('train_accuracy.png') 311 | 312 | plt.figure() 313 | # plt.title('Test Accuracy') 314 | plt.plot(range(len(ave_delay_ep)), ave_delay_ep, color='r') 315 | plt.ylabel('Average delay') 316 | plt.xlabel('Episode') 317 | plt.show() 318 | 319 | plt.figure() 320 | # plt.title('Test Accuracy') 321 | plt.plot(range(len(ave_error_ep)), ave_error_ep, color='r') 322 | plt.ylabel('Average error') 323 | plt.xlabel('Episode') 324 | plt.show() 325 | ''' 326 | -------------------------------------------------------------------------------- /helper.py: -------------------------------------------------------------------------------- 1 | import scipy.stats as stats 2 | import os 3 | import numpy as np 4 | import tensorflow as tf 5 | import ipdb as pdb 6 | import matplotlib.pyplot as plt 7 | from options import args_parser 8 | import torch.nn as nn 9 | import torch 10 | 11 | args = args_parser() 12 | LOSS_FUNC = nn.CrossEntropyLoss() 13 | 14 | # 进行AR信道建模 15 | def complexGaussian(row=1, col=1, amp=1.0): 16 | real = np.random.normal(size=[row, col])[0] * np.sqrt(0.5) 17 | # np.random.normal(size=[row,col])生成数据2维 第一维度包含row个数据 每个数据中又包含col个数据 18 | # np.sqrt(A)求A的开方 19 | img = np.random.normal(size=[row, col])[0] * np.sqrt(0.5) 20 | return amp * (real + 1j * img) # amp相当于系数 后面计算时替换成了根号下1-rou平方 (real + 1j*img)即为误差向量e(t) 21 | 22 | 23 | class ARModel(object): 24 | """docstring for AR channel Model""" 25 | 26 | def __init__(self, n_t=1, n_r=1, seed=123): 27 | self.n_t = n_t 28 | self.n_r = n_r 29 | np.random.seed([seed]) 30 | self.H1 = complexGaussian(self.n_t, self.n_r) # self.H就是hi 即信道增益。初始化定义. 31 | 32 | def sampleCh(self, dis, rho): 33 | for i in range(args.num_users): 34 | # self.H1[i] = rho[i] * self.H1[i] + complexGaussian(self.n_t, self.n_r, np.sqrt(1 - rho[i] * rho[i])) # 这是信道更新的方式 35 | self.H1[i] = rho[i] * self.H1[i] + complexGaussian(1, 1, np.sqrt(1 - rho[i] * rho[i])) #因为这里是一个一个算的 所以是一个一个复高斯数生成的 所以可以直接写成1,1 36 | return self.H1 37 | 38 | def one_iter(model, device, loss_func, optimizer, quantizer, train_data, num_users, epoch): 39 | assert num_users == len(train_data) 40 | model.train() 41 | user_gradients = [list() for _ in model.parameters()] # 4层梯度 42 | all_losses = [] 43 | for user_id in range(num_users): 44 | optimizer.zero_grad() 45 | _data, _target = train_data[user_id] 46 | data, target = _data.to(device), _target.to(device) 47 | pred = model(data) 48 | loss = loss_func(pred, target) 49 | # print(loss) 50 | all_losses.append(loss) 51 | loss.backward() 52 | #parameter = list(model.parameters()) 53 | quantizer.record(user_id, epoch=epoch) 54 | quantizer.apply() 55 | # parameter = list(model.parameters()) 56 | optimizer.step() 57 | #parameter1 = list(model.parameters()) 58 | return torch.stack(all_losses).mean() 59 | 60 | def test(args, model, device, test_loader): 61 | model.eval() 62 | test_loss = 0 63 | correct = 0 64 | with torch.no_grad(): # 在该模块下,所有计算得出的tensor的requires_grad都自动设置为False 65 | for data, target in test_loader: 66 | data, target = data.to(device), target.to(device) 67 | output = model(data) 68 | # sum up batch loss 69 | test_loss += LOSS_FUNC(output, target).sum().item() 70 | # get the index of the max log-probability 71 | pred = output.argmax(dim=1, keepdim=True) 72 | correct += pred.eq(target.view_as(pred)).sum().item() 73 | 74 | test_loss /= len(test_loader.dataset) 75 | 76 | #print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.2f}%)\n'.format( 77 | # test_loss, correct, len(test_loader.dataset), 78 | # 100. * correct / len(test_loader.dataset))) 79 | return correct / len(test_loader.dataset) -------------------------------------------------------------------------------- /identical_compressor.py: -------------------------------------------------------------------------------- 1 | class IdenticalCompressor(object): 2 | def __init__(self, size=None, shape=None, args=None): 3 | pass 4 | 5 | @staticmethod 6 | def compress(vec): 7 | return vec.clone() 8 | 9 | @staticmethod 10 | def decompress(signature): 11 | return signature 12 | -------------------------------------------------------------------------------- /line_chart.py: -------------------------------------------------------------------------------- 1 | import matplotlib.pyplot as plt 2 | 3 | # 读取txt文件中的数据 4 | data = [] 5 | with open('opt_weight/acc/dqn_0.1.txt', 'r') as file: 6 | for line in file: 7 | data.append(float(line.strip())) 8 | x = list(range(1, len(data) + 1)) 9 | y = data 10 | 11 | data1 = [] 12 | with open('opt_weight/acc/dqn_0.3.txt', 'r') as file: 13 | for line in file: 14 | data1.append(float(line.strip())) 15 | x1 = list(range(1, len(data1) + 1)) 16 | y1 = data1 17 | 18 | data2 = [] 19 | with open('opt_weight/acc/dqn_0.5.txt', 'r') as file: 20 | for line in file: 21 | data2.append(float(line.strip())) 22 | x2 = list(range(1, len(data2) + 1)) 23 | y2 = data2 24 | 25 | data3 = [] 26 | with open('opt_weight/acc/dqn_0.7.txt', 'r') as file: 27 | for line in file: 28 | data3.append(float(line.strip())) 29 | x3 = list(range(1, len(data3) + 1)) 30 | y3 = data3 31 | 32 | data4 = [] 33 | with open('opt_weight/acc/dqn_0.9.txt', 'r') as file: 34 | for line in file: 35 | data4.append(float(line.strip())) 36 | x4 = list(range(1, len(data4) + 1)) 37 | y4 = data4 38 | 39 | plt.plot(x, y, label='w1=0.1', linestyle='-.', color='g', lw=1.5) 40 | plt.plot(x1, y1, label='w1=0.3', linestyle=(0, (1, 2, 3)), color='m', lw=1.5) 41 | plt.plot(x2, y2, label='w1=0.5', linestyle='-', color='r', lw=1.5 ) 42 | plt.plot(x3, y3, label='w1=0.7', linestyle='--', color='#66CCFF', lw=1.5) 43 | plt.plot(x4, y4, label='w1=0.9', linestyle=':', color='#CC6600', lw=1.5) 44 | plt.legend() 45 | plt.grid(linestyle=':') 46 | plt.legend() 47 | plt.ylabel("Test accuracy") 48 | plt.xlabel("Episode") 49 | plt.savefig('test_acc_w.eps', format='eps') 50 | plt.show() -------------------------------------------------------------------------------- /logger.py: -------------------------------------------------------------------------------- 1 | # Code referenced from https://gist.github.com/gyglim/1f8dfb1b5c82627ae3efcfbbadb9f514 2 | import tensorflow.compat.v1 as tf 3 | import numpy as np 4 | import scipy.misc 5 | try: 6 | from StringIO import StringIO # Python 2.7 7 | except ImportError: 8 | from io import BytesIO # Python 3.x 9 | 10 | tf.disable_eager_execution() 11 | class Logger(object): 12 | 13 | def __init__(self, log_dir): 14 | """Create a summary writer logging to log_dir.""" 15 | self.writer = tf.summary.FileWriter(log_dir) 16 | 17 | def scalar_summary(self, tag, value, step): 18 | """Log a scalar variable.""" 19 | summary = tf.Summary(value=[tf.Summary.Value(tag=tag, simple_value=value)]) 20 | self.writer.add_summary(summary, step) 21 | 22 | def image_summary(self, tag, images, step): 23 | """Log a list of images.""" 24 | 25 | img_summaries = [] 26 | for i, img in enumerate(images): 27 | # Write the image to a string 28 | try: 29 | s = StringIO() 30 | except: 31 | s = BytesIO() 32 | scipy.misc.toimage(img).save(s, format="png") 33 | 34 | # Create an Image object 35 | img_sum = tf.Summary.Image(encoded_image_string=s.getvalue(), 36 | height=img.shape[0], 37 | width=img.shape[1]) 38 | # Create a Summary value 39 | img_summaries.append(tf.Summary.Value(tag='%s/%d' % (tag, i), image=img_sum)) 40 | 41 | # Create and write Summary 42 | summary = tf.Summary(value=img_summaries) 43 | self.writer.add_summary(summary, step) 44 | 45 | def histo_summary(self, tag, values, step, bins=1000): 46 | """Log a histogram of the tensor of values.""" 47 | 48 | # Create a histogram using numpy 49 | counts, bin_edges = np.histogram(values, bins=bins) 50 | 51 | # Fill the fields of the histogram proto 52 | hist = tf.HistogramProto() 53 | hist.min = float(np.min(values)) 54 | hist.max = float(np.max(values)) 55 | hist.num = int(np.prod(values.shape)) 56 | hist.sum = float(np.sum(values)) 57 | hist.sum_squares = float(np.sum(values**2)) 58 | 59 | # Drop the start of the first bin 60 | bin_edges = bin_edges[1:] 61 | 62 | # Add bin edges and counts 63 | for edge in bin_edges: 64 | hist.bucket_limit.append(edge) 65 | for c in counts: 66 | hist.bucket.append(c) 67 | 68 | # Create and write Summary 69 | summary = tf.Summary(value=[tf.Summary.Value(tag=tag, histo=hist)]) 70 | self.writer.add_summary(summary, step) 71 | self.writer.flush() -------------------------------------------------------------------------------- /model/__pycache__/pe.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/model/__pycache__/pe.cpython-38.pyc -------------------------------------------------------------------------------- /model/checkpoint: -------------------------------------------------------------------------------- 1 | model_checkpoint_path: "train_model-q" 2 | all_model_checkpoint_paths: "train_model-q" 3 | -------------------------------------------------------------------------------- /model/train_model-q.data-00000-of-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/model/train_model-q.data-00000-of-00001 -------------------------------------------------------------------------------- /model/train_model-q.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/model/train_model-q.index -------------------------------------------------------------------------------- /model/train_model-q.meta: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/model/train_model-q.meta -------------------------------------------------------------------------------- /opt_weight/acc/dqn_0.1.txt: -------------------------------------------------------------------------------- 1 | 2.907000000000000139e-01 2 | 3.250032236901025939e-01 3 | 3.438399801960485558e-01 4 | 3.853627144847479591e-01 5 | 4.096424179230718221e-01 6 | 4.297768061402184592e-01 7 | 4.425088457481199389e-01 8 | 4.543526035229121240e-01 9 | 4.647158915758551889e-01 10 | 4.735987099069493556e-01 11 | 4.815932464049339501e-01 12 | 4.889955950141790519e-01 13 | 4.999510709558618315e-01 14 | 5.087518781410144131e-01 15 | 5.191151661939575890e-01 16 | 5.259253269144630316e-01 17 | 5.348901563914986168e-01 18 | 5.452534444444417927e-01 19 | 5.538401688311660465e-01 20 | 5.591698598298225908e-01 21 | 5.650917387172186279e-01 22 | 5.713097115489843558e-01 23 | 5.769354964920104800e-01 24 | 5.807847177688180151e-01 25 | 5.846339390456255503e-01 26 | 5.923323815992403985e-01 27 | 5.982542604866364355e-01 28 | 6.047683272627720763e-01 29 | 6.073156073875015659e-01 30 | 6.114609226086787919e-01 31 | 6.164945196629653124e-01 32 | 6.231261518136999022e-01 33 | 6.284558428123564466e-01 34 | 6.331933459222732763e-01 35 | 6.391152248096693134e-01 36 | 6.457844767046464485e-01 37 | 6.490415100927142689e-01 38 | 6.511141677033028818e-01 39 | 6.523208088271853500e-01 40 | 6.551631047917583217e-01 41 | 6.581240442354564513e-01 42 | 6.626840968801284149e-01 43 | 6.687173024995409776e-01 44 | 6.725665237763484017e-01 45 | 6.743430874425673238e-01 46 | 6.761269678558946561e-01 47 | 6.796800951883322783e-01 48 | 6.832332225207699006e-01 49 | 6.873785377419472375e-01 50 | 6.883318652037586638e-01 51 | 6.915888985918264842e-01 52 | 6.939576501467848990e-01 53 | 6.989262015723695098e-01 54 | 7.024793289048071321e-01 55 | 7.051441744041354598e-01 56 | 7.081051138478334783e-01 57 | 7.101777714584219803e-01 58 | 7.128426169577503080e-01 59 | 7.160996503458180173e-01 60 | 7.184433080393392013e-01 61 | 7.202198717055580124e-01 62 | 7.217030792489738111e-01 63 | 7.238008307209997660e-01 64 | 7.283062074107227168e-01 65 | 7.322277910633406828e-01 66 | 7.355599246487859455e-01 67 | 7.376715466155804313e-01 68 | 7.394815083014041335e-01 69 | 7.415931302681985082e-01 70 | 7.437047522349929940e-01 71 | 7.452130536398460237e-01 72 | 7.470230153256697259e-01 73 | 7.476263358876110710e-01 74 | 7.485313167305228665e-01 75 | 7.494362975734348842e-01 76 | 7.512462592592585864e-01 77 | 7.537580758487978372e-01 78 | 7.552663772536510889e-01 79 | 7.558696978155922119e-01 80 | 7.572794648786711491e-01 81 | 7.596927471264360854e-01 82 | 7.608993882503186645e-01 83 | 7.621060293742010217e-01 84 | 7.636143307790542734e-01 85 | 7.645193116219660689e-01 86 | 7.657259527458486481e-01 87 | 7.660276130268192096e-01 88 | 7.687425555555549295e-01 89 | 7.708541775223493042e-01 90 | 7.726641392081731174e-01 91 | 7.741724406130261471e-01 92 | 7.759824022988498493e-01 93 | 7.786973448275854581e-01 94 | 7.802056462324385988e-01 95 | 7.808089667943797219e-01 96 | 7.814122873563210669e-01 97 | 7.835239093231155527e-01 98 | 7.852431499062856313e-01 99 | 7.870531115921092224e-01 100 | 7.876564321540504565e-01 101 | 7.886521340996160978e-01 102 | 7.889537943805867704e-01 103 | 7.910654163473811451e-01 104 | 7.910654163473811451e-01 105 | 7.931770383141755199e-01 106 | 7.928753780332048473e-01 107 | 7.955903205619404561e-01 108 | 7.955903205619406782e-01 109 | 7.958919808429111287e-01 110 | 7.952886602809697836e-01 111 | 7.964953014048523627e-01 112 | 7.996170829974600380e-01 113 | 8.014270446832837402e-01 114 | 8.014270446832837402e-01 115 | 8.022268467432942529e-01 116 | 8.028301673052355980e-01 117 | 8.049417892720299728e-01 118 | 8.061484303959124409e-01 119 | 8.082600523627068156e-01 120 | 8.091650332056187223e-01 121 | 8.100700140485305178e-01 122 | 8.100700140485305178e-01 123 | 8.118799757343543311e-01 124 | 8.121816360153250036e-01 125 | 8.130866168582366882e-01 126 | 8.134210935462981018e-01 127 | 8.143533244596996568e-01 128 | 8.149566450216410018e-01 129 | 8.155488329103804945e-01 130 | 8.164371147434899001e-01 131 | 8.179559672090302991e-01 132 | 8.188392643180533170e-01 133 | 8.185270529763953862e-01 134 | 8.182148416347373443e-01 135 | 8.179237324144540411e-01 136 | 8.182359437561119719e-01 137 | 8.185376040370826445e-01 138 | 8.185376040370826445e-01 139 | 8.191297919258223592e-01 140 | 8.182248110829104526e-01 141 | 8.194091868603896600e-01 142 | 8.202974686934990656e-01 143 | 8.230124112222346744e-01 144 | 8.233085051666044762e-01 145 | 8.221191446650387702e-01 146 | 8.223923916504909393e-01 147 | 8.230129928347489354e-01 148 | 8.242196339586315146e-01 149 | 8.248513678160911411e-01 150 | 8.248513678160911411e-01 151 | 8.263145569156233883e-01 152 | 8.268671988356430802e-01 153 | 8.268671988356430802e-01 154 | 8.275156317029050967e-01 155 | 8.281078195916448115e-01 156 | 8.296161209964978411e-01 157 | 8.299177812774685137e-01 158 | 8.293762720306506742e-01 159 | 8.305044028296072467e-01 160 | 8.305044028296073577e-01 161 | 8.317110439534897148e-01 162 | 8.317895542784156104e-01 163 | 8.332082126851411141e-01 164 | 8.338115332470824592e-01 165 | 8.344148538090236933e-01 166 | 8.345044968071512193e-01 167 | 8.350126080343640567e-01 168 | 8.356159285963054018e-01 169 | 8.367106613756570832e-01 170 | 8.374091912723266029e-01 171 | 8.377108515532972755e-01 172 | 8.384260804597692962e-01 173 | 8.392080202849486747e-01 174 | 8.401130011278606924e-01 175 | 8.405821479988677680e-01 176 | 8.413085095785414191e-01 177 | 8.407051890166002961e-01 178 | 8.410012829609700979e-01 179 | 8.414592971587756542e-01 180 | 8.434200464248334805e-01 181 | 8.440122343135730842e-01 182 | 8.441575406777087620e-01 183 | 8.444536346220785639e-01 184 | 8.450458225108180565e-01 185 | 8.441408416679063720e-01 186 | 8.439956204242730076e-01 187 | 8.438559655172405138e-01 188 | 8.444592860791818589e-01 189 | 8.444592860791818589e-01 190 | 8.445989409862142416e-01 191 | 8.461072423910672713e-01 192 | 8.465485575790704376e-01 193 | 8.470122232339792889e-01 194 | 8.465485575790704376e-01 195 | 8.470122232339791779e-01 196 | 8.467105629530086164e-01 197 | 8.468725683269467952e-01 198 | 8.471742286079174677e-01 199 | 8.476044111227188926e-01 200 | 8.482077316846602377e-01 201 | 8.488110522466013608e-01 202 | 8.492858505747117315e-01 203 | 8.500009943606814389e-01 204 | 8.503026546416521114e-01 205 | 8.512076354845640180e-01 206 | 8.507941519795649832e-01 207 | 8.510958122605355447e-01 208 | 8.510958122605356557e-01 209 | 8.520007931034474513e-01 210 | 8.521014836542741833e-01 211 | 8.524031439352448558e-01 212 | 8.527048042162155284e-01 213 | 8.532074342273299195e-01 214 | 8.520007931034475623e-01 215 | 8.520959173176733126e-01 216 | 8.520959173176733126e-01 217 | 8.539058790034971258e-01 218 | 8.542019729478669277e-01 219 | 8.551069537907788343e-01 220 | 8.554086140717495068e-01 221 | 8.559223767560656393e-01 222 | 8.556207164750949667e-01 223 | 8.556207164750949667e-01 224 | 8.554030477351486361e-01 225 | 8.554814729395721074e-01 226 | 8.558559920883652694e-01 227 | 8.557720005473411495e-01 228 | 8.544869342190350991e-01 229 | 8.520007931034475623e-01 230 | 8.513974725415062172e-01 231 | 8.520007931034475623e-01 232 | 8.544869342190350991e-01 233 | 8.544869342190350991e-01 234 | 8.541852739380644266e-01 235 | 8.535090945083005920e-01 236 | 8.547885945000057717e-01 237 | 8.547885945000057717e-01 238 | 8.553919150619470058e-01 239 | 8.559841029506864984e-01 240 | 8.574924043555397501e-01 241 | 8.577940646365102006e-01 242 | 8.577323384418892305e-01 243 | 8.586373192848011371e-01 244 | 8.592967997047625817e-01 245 | 8.592967997047625817e-01 246 | 8.586934791428212366e-01 247 | 8.589389795657718096e-01 248 | 8.592967997047625817e-01 249 | 8.599507137881232666e-01 250 | 8.590457329452112489e-01 251 | 8.592912333681615999e-01 252 | 8.592912333681618220e-01 253 | 8.601962142110736176e-01 254 | 8.601962142110736176e-01 255 | 8.592406398467423712e-01 256 | 8.592912333681618220e-01 257 | 8.592912333681615999e-01 258 | 8.598945539301029450e-01 259 | 8.595423001277131547e-01 260 | 8.595928936491324945e-01 261 | 8.607995347730149627e-01 262 | 8.620061758968974308e-01 263 | 8.616539220945074184e-01 264 | 8.610956287173846535e-01 265 | 8.598889875935021854e-01 266 | 8.598889875935021854e-01 267 | 8.607489412515955118e-01 268 | 8.616933829427252389e-01 269 | 8.626378246338547440e-01 270 | 8.623361643528840714e-01 271 | 8.622967035046663620e-01 272 | 8.628605632183899976e-01 273 | 8.625983637856369235e-01 274 | 8.632016843475781576e-01 275 | 8.626378246338547440e-01 276 | 8.619950432236956894e-01 277 | 8.620233713987118795e-01 278 | 8.613805899885528250e-01 279 | 8.626155592874513722e-01 280 | 8.622855708314647316e-01 281 | 8.635205401303632788e-01 282 | 8.631905516743766382e-01 283 | 8.640955325172884338e-01 284 | 8.640955325172884338e-01 285 | 8.650005133602003404e-01 286 | 8.646988530792296679e-01 287 | 8.652738454661550449e-01 288 | 8.652738454661549339e-01 289 | 8.661848891376795478e-01 290 | 8.664981785838637984e-01 291 | 8.662192801413083343e-01 292 | 8.656098967507542818e-01 293 | 8.652966073045701423e-01 294 | 8.655755057471256064e-01 295 | 8.655755057471256064e-01 296 | 8.655755057471256064e-01 297 | 8.661848891376795478e-01 298 | 8.664865494186502204e-01 299 | 8.670898699805914545e-01 300 | 8.670838071519787471e-01 301 | 8.682798120946714215e-01 302 | 8.688831326566127666e-01 303 | 8.694864532185540007e-01 304 | 8.691954291187731219e-01 305 | 8.691954291187731219e-01 306 | 8.691903592741841988e-01 307 | 8.685931015408557831e-01 308 | 8.679958438075271454e-01 309 | 8.674036559187875417e-01 310 | 8.682919377518970583e-01 311 | 8.691802195850062418e-01 312 | 8.703645953624855602e-01 313 | 8.703645953624855602e-01 314 | 8.706606893068554731e-01 315 | 8.706606893068552511e-01 316 | 8.712528771955949658e-01 317 | 8.712528771955949658e-01 318 | 8.715489711399646566e-01 319 | 8.715489711399647677e-01 320 | 8.715651736577553210e-01 321 | 8.715651736577553210e-01 322 | 8.715651736577553210e-01 323 | 8.706601928148435254e-01 324 | 8.703803013882641659e-01 325 | 8.697987496807142449e-01 326 | 8.704020702426555900e-01 327 | 8.704020702426555900e-01 328 | 8.707037305236261515e-01 329 | 8.707037305236261515e-01 330 | 8.707037305236262625e-01 331 | 8.715758098389451147e-01 332 | 8.727495494352345284e-01 333 | 8.730512097162052010e-01 334 | 8.724590218274655973e-01 335 | 8.715707399943561917e-01 336 | 8.712528771955949658e-01 337 | 8.709729857690157173e-01 338 | 8.709891882868062707e-01 339 | 8.710053908045968241e-01 340 | 8.709891882868062707e-01 341 | 8.709891882868062707e-01 342 | 8.709891882868062707e-01 343 | 8.710053908045968241e-01 344 | 8.713070510855674966e-01 345 | 8.716087113665381692e-01 346 | 8.719103716475087307e-01 347 | 8.722120319284792922e-01 348 | 8.725136922094499647e-01 349 | 8.721958294106887388e-01 350 | 8.715763063309570624e-01 351 | 8.712746460499863899e-01 352 | 8.718941691297180663e-01 353 | 8.725136922094499647e-01 354 | 8.728153524904206373e-01 355 | 8.719103716475086197e-01 356 | 8.713070510855674966e-01 357 | 8.707037305236262625e-01 358 | 8.716087113665381692e-01 359 | 8.722120319284792922e-01 360 | 8.722120319284792922e-01 361 | 8.722120319284792922e-01 362 | 8.719103716475086197e-01 363 | 8.722120319284792922e-01 364 | 8.719103716475087307e-01 365 | 8.719103716475087307e-01 366 | 8.719103716475087307e-01 367 | 8.725136922094499647e-01 368 | 8.731170127713911988e-01 369 | 8.737203333333324329e-01 370 | 8.737203333333324329e-01 371 | 8.737203333333324329e-01 372 | 8.737203333333324329e-01 373 | 8.737203333333324329e-01 374 | 8.737203333333324329e-01 375 | 8.737203333333324329e-01 376 | 8.737203333333324329e-01 377 | 8.737203333333324329e-01 378 | 8.740219936143031054e-01 379 | 8.743236538952737780e-01 380 | 8.749269744572149010e-01 381 | 8.755302950191562461e-01 382 | 8.764352758620681527e-01 383 | 8.770385964240093868e-01 384 | 8.767369361430387142e-01 385 | 8.767369361430387142e-01 386 | 8.764352758620681527e-01 387 | 8.764352758620681527e-01 388 | 8.758319553001268076e-01 389 | 8.758319553001268076e-01 390 | 8.749269744572151231e-01 391 | 8.754751281451600420e-01 392 | 8.757216215521342884e-01 393 | 8.763249421140755224e-01 394 | 8.754751281451600420e-01 395 | 8.740219936143031054e-01 396 | 8.743236538952737780e-01 397 | 8.737203333333325439e-01 398 | 8.743236538952737780e-01 399 | 8.740219936143031054e-01 400 | 8.746253141762443395e-01 401 | 8.746253141762443395e-01 402 | 8.760673160338995347e-01 403 | 8.760673160338994236e-01 404 | 8.766706365958407687e-01 405 | 8.758319553001268076e-01 406 | 8.764352758620681527e-01 407 | 8.764352758620681527e-01 408 | 8.772739571577821138e-01 409 | 8.772739571577821138e-01 410 | 8.772739571577821138e-01 411 | 8.764352758620681527e-01 412 | 8.764352758620681527e-01 413 | 8.764352758620681527e-01 414 | 8.772739571577821138e-01 415 | 8.781126384534959639e-01 416 | 8.793192795773784320e-01 417 | 8.796872394055470501e-01 418 | 8.796872394055469391e-01 419 | 8.793192795773784320e-01 420 | 8.793192795773784320e-01 421 | 8.793855791245763776e-01 422 | 8.794518786717743231e-01 423 | 8.790839188436058160e-01 424 | 8.790176192964079815e-01 425 | 8.789513197492099250e-01 426 | 8.789513197492099250e-01 427 | 8.787159590154370870e-01 428 | 8.793081469041768017e-01 429 | 8.799777670133158702e-01 430 | 8.799777670133160923e-01 431 | 8.799777670133160923e-01 432 | 8.796761067323454197e-01 433 | 8.807997492992275523e-01 434 | 8.805755212386564557e-01 435 | 8.810958432435972432e-01 436 | 8.810958432435972432e-01 437 | 8.816991638055385883e-01 438 | 8.817821623625389238e-01 439 | 8.815635006385686978e-01 440 | 8.814805020815682512e-01 441 | 8.805092216914585102e-01 442 | 8.805092216914585102e-01 443 | 8.802905599674882842e-01 444 | 8.808827478562278879e-01 445 | 8.811844081371985604e-01 446 | 8.811069759167988735e-01 447 | 8.802794272942867648e-01 448 | 8.799003347929165164e-01 449 | 8.790727861704041857e-01 450 | 8.790727861704041857e-01 451 | 8.794518786717743231e-01 452 | 8.794518786717744341e-01 453 | 8.802794272942865428e-01 454 | 8.796042408485466035e-01 455 | 8.804317894710589343e-01 456 | 8.804317894710589343e-01 457 | 8.804317894710589343e-01 458 | 8.808108819724291827e-01 459 | 8.805147880280593808e-01 460 | 8.805147880280593808e-01 461 | 8.798396015823193306e-01 462 | 8.793081469041768017e-01 463 | 8.793081469041768017e-01 464 | 8.793800127879755069e-01 465 | 8.799114674661181468e-01 466 | 8.799114674661180358e-01 467 | 8.798396015823193306e-01 468 | 8.798396015823193306e-01 469 | 8.798396015823193306e-01 470 | 8.787048263422355676e-01 471 | 8.787048263422355676e-01 472 | 8.787048263422355676e-01 473 | 8.804317894710589343e-01 474 | 8.810239773597986490e-01 475 | 8.804925226816560091e-01 476 | 8.784638992718619699e-01 477 | 8.769834295500130716e-01 478 | 8.778109781725254024e-01 479 | 8.795435076379495287e-01 480 | 8.802019950738871890e-01 481 | 8.802019950738871890e-01 482 | 8.790727861704041857e-01 483 | 8.784142987344666365e-01 484 | 8.781182047900969456e-01 485 | 8.789513197492099250e-01 486 | 8.798396015823193306e-01 487 | 8.799114674661181468e-01 488 | 8.799114674661181468e-01 489 | 8.796153735217483449e-01 490 | 8.795435076379495287e-01 491 | 8.792474136935797269e-01 492 | 8.789513197492099250e-01 493 | 8.783591318604703213e-01 494 | 8.780630379161005195e-01 495 | 8.780630379161005195e-01 496 | 8.786552258048402342e-01 497 | 8.786552258048402342e-01 498 | 8.798396015823192196e-01 499 | 8.795435076379495287e-01 500 | 8.798396015823192196e-01 501 | -------------------------------------------------------------------------------- /opt_weight/acc/dqn_0.3.txt: -------------------------------------------------------------------------------- 1 | 2.907000000000000139e-01 2 | 3.482726187739463297e-01 3 | 3.750049987228606940e-01 4 | 3.938010737547891771e-01 5 | 4.085176459770113455e-01 6 | 4.456643946360151487e-01 7 | 4.615920574712641944e-01 8 | 4.758907547892718415e-01 9 | 4.876555057471261279e-01 10 | 4.983342796934863150e-01 11 | 5.099180344827582978e-01 12 | 5.207778045977008441e-01 13 | 5.298276130268195772e-01 14 | 5.394204099616854986e-01 15 | 5.486512145593865908e-01 16 | 5.571580344827582465e-01 17 | 5.654838582375475431e-01 18 | 5.739906781609191988e-01 19 | 5.803255440613023231e-01 20 | 5.859364252873560108e-01 21 | 5.904613295019152108e-01 22 | 5.948052375478922738e-01 23 | 5.996921340996164140e-01 24 | 6.063889923371643675e-01 25 | 6.125428620689650216e-01 26 | 6.183347394636010685e-01 27 | 6.244886091954018337e-01 28 | 6.295565019157083331e-01 29 | 6.339004099616853960e-01 30 | 6.377013295019151595e-01 31 | 6.425882260536394108e-01 32 | 6.478371149425281583e-01 33 | 6.516380344827581439e-01 34 | 6.565249310344822842e-01 35 | 6.605068467432945178e-01 36 | 6.639457739463596742e-01 37 | 6.673847011494247194e-01 38 | 6.726335900383135780e-01 39 | 6.773394904214553591e-01 40 | 6.829503716475089359e-01 41 | 6.880182643678154353e-01 42 | 6.930861570881219347e-01 43 | 6.976110613026813567e-01 44 | 7.017739731800760605e-01 45 | 7.055748927203058241e-01 46 | 7.100997969348653571e-01 47 | 7.133577279693480433e-01 48 | 7.171586475095779178e-01 49 | 7.204165785440606040e-01 50 | 7.242174980842905896e-01 51 | 7.269324406130260874e-01 52 | 7.296473831417618072e-01 53 | 7.316383409961678685e-01 54 | 7.332673065134093227e-01 55 | 7.348962720306506657e-01 56 | 7.368872298850568381e-01 57 | 7.396021724137924469e-01 58 | 7.423171149425280557e-01 59 | 7.453940498084283828e-01 60 | 7.482899885057465728e-01 61 | 7.510049310344820706e-01 62 | 7.533578812260529611e-01 63 | 7.555298352490413816e-01 64 | 7.575207931034475539e-01 65 | 7.589687624521066489e-01 66 | 7.609597203065127102e-01 67 | 7.624076896551716942e-01 68 | 7.638556590038307892e-01 69 | 7.653036283524896621e-01 70 | 7.672945862068958345e-01 71 | 7.687425555555548184e-01 72 | 7.703715210727961615e-01 73 | 7.718194904214552565e-01 74 | 7.727244712643670521e-01 75 | 7.736294521072790698e-01 76 | 7.748964252873555836e-01 77 | 7.759824022988498493e-01 78 | 7.772493754789264742e-01 79 | 7.786973448275855691e-01 80 | 7.805073065134091603e-01 81 | 7.823172681992329736e-01 82 | 7.841272298850567868e-01 83 | 7.857561954022982409e-01 84 | 7.870231685823747547e-01 85 | 7.881091455938691315e-01 86 | 7.886521340996162088e-01 87 | 7.895571149425280044e-01 88 | 7.908240881226046293e-01 89 | 7.920910613026812541e-01 90 | 7.931770383141756309e-01 91 | 7.944440114942521447e-01 92 | 7.957109846743287696e-01 93 | 7.966159655172406762e-01 94 | 7.978829386973173010e-01 95 | 7.986069233716467375e-01 96 | 7.995119042145585331e-01 97 | 7.996929003831410032e-01 98 | 8.005978812260530209e-01 99 | 8.011408697317999872e-01 100 | 8.022268467432942529e-01 101 | 8.029508314176239114e-01 102 | 8.040368084291179551e-01 103 | 8.042178045977003142e-01 104 | 8.049417892720299728e-01 105 | 8.056657739463594092e-01 106 | 8.065707547892714269e-01 107 | 8.078377279693478297e-01 108 | 8.091047011494245655e-01 109 | 8.101906781609187203e-01 110 | 8.109146628352483788e-01 111 | 8.114576513409954561e-01 112 | 8.118196436781601744e-01 113 | 8.125436283524896108e-01 114 | 8.129056206896544401e-01 115 | 8.136296053639838766e-01 116 | 8.141725938697309539e-01 117 | 8.141725938697310649e-01 118 | 8.147155823754781423e-01 119 | 8.156205632183899379e-01 120 | 8.161635517241372373e-01 121 | 8.165255440613019555e-01 122 | 8.181545095785432986e-01 123 | 8.183355057471256577e-01 124 | 8.186974980842904870e-01 125 | 8.194214827586199235e-01 126 | 8.206884559386965483e-01 127 | 8.210504482758613776e-01 128 | 8.221364252873556433e-01 129 | 8.223174214559378914e-01 130 | 8.228604099616850798e-01 131 | 8.226794137931026096e-01 132 | 8.232224022988499090e-01 133 | 8.234033984674322681e-01 134 | 8.244893754789265339e-01 135 | 8.252133601532559704e-01 136 | 8.261183409961677659e-01 137 | 8.266613295019148433e-01 138 | 8.281092988505738273e-01 139 | 8.291952758620680930e-01 140 | 8.297382643678152814e-01 141 | 8.306432452107271880e-01 142 | 8.315482260536389836e-01 143 | 8.319102183908038128e-01 144 | 8.319102183908038128e-01 145 | 8.324532068965510012e-01 146 | 8.333581877394629078e-01 147 | 8.339011762452098742e-01 148 | 8.346251609195393106e-01 149 | 8.348061570881217808e-01 150 | 8.351681494252864990e-01 151 | 8.349871532567043619e-01 152 | 8.353491455938689692e-01 153 | 8.351681494252864990e-01 154 | 8.358921340996159355e-01 155 | 8.371591072796927824e-01 156 | 8.384260804597692962e-01 157 | 8.391500651340987327e-01 158 | 8.405980344827577166e-01 159 | 8.418650076628344525e-01 160 | 8.429509846743286072e-01 161 | 8.436749693486582657e-01 162 | 8.443989540229877022e-01 163 | 8.449419425287348906e-01 164 | 8.456659272030643271e-01 165 | 8.458469233716467972e-01 166 | 8.463899118773937635e-01 167 | 8.471138965517234221e-01 168 | 8.476568850574703884e-01 169 | 8.483808697317998249e-01 170 | 8.489238582375471243e-01 171 | 8.496478429118765607e-01 172 | 8.501908314176237491e-01 173 | 8.510958122605355447e-01 174 | 8.516388007662826221e-01 175 | 8.520007931034474513e-01 176 | 8.523627854406121696e-01 177 | 8.530867701149418281e-01 178 | 8.532677662835240762e-01 179 | 8.534487624521064353e-01 180 | 8.543537432950183419e-01 181 | 8.547157356321830601e-01 182 | 8.548967318007655303e-01 183 | 8.558017126436773259e-01 184 | 8.561637049808419331e-01 185 | 8.561637049808421551e-01 186 | 8.568876896551715916e-01 187 | 8.574306781609187800e-01 188 | 8.579736666666658573e-01 189 | 8.586976513409952938e-01 190 | 8.592406398467424822e-01 191 | 8.596026321839073114e-01 192 | 8.603266168582367479e-01 193 | 8.597836283524895595e-01 194 | 8.599646245210719187e-01 195 | 8.605076130268189960e-01 196 | 8.608696053639837142e-01 197 | 8.608696053639837142e-01 198 | 8.615935900383133728e-01 199 | 8.619555823754780910e-01 200 | 8.619555823754780910e-01 201 | 8.619555823754780910e-01 202 | 8.619555823754780910e-01 203 | 8.623175747126428092e-01 204 | 8.626795670498076385e-01 205 | 8.630415593869722457e-01 206 | 8.637655440613019042e-01 207 | 8.641275363984665114e-01 208 | 8.641275363984665114e-01 209 | 8.641275363984665114e-01 210 | 8.641275363984666225e-01 211 | 8.637655440613017932e-01 212 | 8.637655440613017932e-01 213 | 8.637655440613017932e-01 214 | 8.641275363984665114e-01 215 | 8.641275363984665114e-01 216 | 8.643085325670490926e-01 217 | 8.646705249042136998e-01 218 | 8.653945095785431363e-01 219 | 8.657565019157079655e-01 220 | 8.664804865900374020e-01 221 | 8.670234750957845904e-01 222 | 8.673854674329493086e-01 223 | 8.675664636015316677e-01 224 | 8.677474597701140269e-01 225 | 8.681094521072788561e-01 226 | 8.684714444444436854e-01 227 | 8.690144329501906517e-01 228 | 8.693764252873554810e-01 229 | 8.697384176245203102e-01 230 | 8.699194137931025583e-01 231 | 8.702814061302672766e-01 232 | 8.701004099616849174e-01 233 | 8.701004099616849174e-01 234 | 8.701004099616849174e-01 235 | 8.701004099616849174e-01 236 | 8.699194137931025583e-01 237 | 8.701004099616849174e-01 238 | 8.701004099616849174e-01 239 | 8.701004099616849174e-01 240 | 8.704624022988497467e-01 241 | 8.706433984674321058e-01 242 | 8.710053908045969351e-01 243 | 8.713673831417615423e-01 244 | 8.717293754789263716e-01 245 | 8.717293754789263716e-01 246 | 8.719103716475087307e-01 247 | 8.719103716475087307e-01 248 | 8.722723639846734489e-01 249 | 8.722723639846734489e-01 250 | 8.720913678160909788e-01 251 | 8.720913678160912008e-01 252 | 8.719103716475086197e-01 253 | 8.715483793103440124e-01 254 | 8.715483793103440124e-01 255 | 8.719103716475087307e-01 256 | 8.722723639846734489e-01 257 | 8.729963486590029964e-01 258 | 8.735393371647500738e-01 259 | 8.742633218390796213e-01 260 | 8.748063103448266986e-01 261 | 8.749873065134090577e-01 262 | 8.749873065134090577e-01 263 | 8.749873065134090577e-01 264 | 8.748063103448266986e-01 265 | 8.746253141762443395e-01 266 | 8.746253141762443395e-01 267 | 8.746253141762443395e-01 268 | 8.746253141762443395e-01 269 | 8.746253141762443395e-01 270 | 8.748063103448266986e-01 271 | 8.748063103448266986e-01 272 | 8.749873065134090577e-01 273 | 8.753492988505738870e-01 274 | 8.758922873563209643e-01 275 | 8.762542796934857936e-01 276 | 8.769782643678152301e-01 277 | 8.775212528735624185e-01 278 | 8.777022490421447776e-01 279 | 8.780642413793093848e-01 280 | 8.786072298850566842e-01 281 | 8.789692222222214024e-01 282 | 8.793312145593861207e-01 283 | 8.798742030651333090e-01 284 | 8.800551992337155571e-01 285 | 8.800551992337155571e-01 286 | 8.800551992337155571e-01 287 | 8.800551992337155571e-01 288 | 8.798742030651333090e-01 289 | 8.800551992337156682e-01 290 | 8.800551992337155571e-01 291 | 8.800551992337156682e-01 292 | 8.802361954022979162e-01 293 | 8.804171915708802754e-01 294 | 8.802361954022979162e-01 295 | 8.802361954022979162e-01 296 | 8.805981877394627455e-01 297 | 8.804171915708802754e-01 298 | 8.804171915708802754e-01 299 | 8.804171915708802754e-01 300 | 8.804171915708802754e-01 301 | 8.804171915708802754e-01 302 | 8.805981877394627455e-01 303 | 8.809601800766275748e-01 304 | 8.811411762452099339e-01 305 | 8.809601800766275748e-01 306 | 8.809601800766275748e-01 307 | 8.809601800766275748e-01 308 | 8.809601800766275748e-01 309 | 8.811411762452099339e-01 310 | 8.816841647509570112e-01 311 | 8.816841647509570112e-01 312 | 8.820461570881217295e-01 313 | 8.822271532567041996e-01 314 | 8.824081494252865587e-01 315 | 8.824081494252865587e-01 316 | 8.824081494252865587e-01 317 | 8.822271532567039776e-01 318 | 8.820461570881217295e-01 319 | 8.818651609195393704e-01 320 | 8.815031685823745411e-01 321 | 8.815031685823745411e-01 322 | 8.815031685823745411e-01 323 | 8.813221724137921820e-01 324 | 8.811411762452099339e-01 325 | 8.815031685823747631e-01 326 | 8.816841647509570112e-01 327 | 8.822271532567041996e-01 328 | 8.822271532567039776e-01 329 | 8.822271532567041996e-01 330 | 8.824081494252865587e-01 331 | 8.820461570881217295e-01 332 | 8.815031685823745411e-01 333 | 8.815031685823745411e-01 334 | 8.815031685823745411e-01 335 | 8.811411762452099339e-01 336 | 8.816841647509570112e-01 337 | 8.816841647509570112e-01 338 | 8.822271532567041996e-01 339 | 8.829511379310336361e-01 340 | 8.834941264367806024e-01 341 | 8.831321340996159952e-01 342 | 8.834941264367806024e-01 343 | 8.833131302681982433e-01 344 | 8.827701417624511659e-01 345 | 8.824081494252864477e-01 346 | 8.824081494252864477e-01 347 | 8.818651609195393704e-01 348 | 8.818651609195393704e-01 349 | 8.818651609195393704e-01 350 | 8.818651609195393704e-01 351 | 8.816841647509570112e-01 352 | 8.820461570881217295e-01 353 | 8.816841647509570112e-01 354 | 8.815031685823745411e-01 355 | 8.816841647509570112e-01 356 | 8.816841647509570112e-01 357 | 8.813221724137921820e-01 358 | 8.815031685823745411e-01 359 | 8.815031685823745411e-01 360 | 8.815031685823745411e-01 361 | 8.822271532567041996e-01 362 | 8.825891455938690289e-01 363 | 8.827701417624513880e-01 364 | 8.833131302681984653e-01 365 | 8.833131302681984653e-01 366 | 8.831321340996159952e-01 367 | 8.833131302681984653e-01 368 | 8.834941264367806024e-01 369 | 8.834941264367806024e-01 370 | 8.836751226053630726e-01 371 | 8.836751226053630726e-01 372 | 8.836751226053630726e-01 373 | 8.838561187739454317e-01 374 | 8.838561187739454317e-01 375 | 8.842181111111102609e-01 376 | 8.845801034482748681e-01 377 | 8.849420957854396974e-01 378 | 8.851230919540220565e-01 379 | 8.854850842911868858e-01 380 | 8.856660804597693559e-01 381 | 8.860280727969340742e-01 382 | 8.860280727969339631e-01 383 | 8.863900651340987924e-01 384 | 8.869330536398457587e-01 385 | 8.869330536398459808e-01 386 | 8.867520574712635106e-01 387 | 8.869330536398459808e-01 388 | 8.867520574712635106e-01 389 | 8.867520574712635106e-01 390 | 8.869330536398459808e-01 391 | 8.869330536398457587e-01 392 | 8.872950459770105880e-01 393 | 8.872950459770105880e-01 394 | 8.872950459770105880e-01 395 | 8.874760421455929471e-01 396 | 8.878380344827577764e-01 397 | 8.880190306513402465e-01 398 | 8.885620191570872128e-01 399 | 8.887430153256696830e-01 400 | 8.889240114942520421e-01 401 | 8.892860038314166493e-01 402 | 8.892860038314168714e-01 403 | 8.896479961685814786e-01 404 | 8.898289923371638377e-01 405 | 8.901909846743285559e-01 406 | 8.900099885057463078e-01 407 | 8.903719808429109150e-01 408 | 8.901909846743285559e-01 409 | 8.900099885057463078e-01 410 | 8.896479961685814786e-01 411 | 8.896479961685814786e-01 412 | 8.898289923371638377e-01 413 | 8.898289923371638377e-01 414 | 8.898289923371638377e-01 415 | 8.900099885057463078e-01 416 | 8.905529770114932742e-01 417 | 8.901909846743285559e-01 418 | 8.901909846743285559e-01 419 | 8.905529770114932742e-01 420 | 8.910959655172405736e-01 421 | 8.910959655172405736e-01 422 | 8.914579578544051808e-01 423 | 8.914579578544051808e-01 424 | 8.916389540229877619e-01 425 | 8.910959655172405736e-01 426 | 8.910959655172405736e-01 427 | 8.910959655172405736e-01 428 | 8.910959655172405736e-01 429 | 8.910959655172405736e-01 430 | 8.914579578544052918e-01 431 | 8.914579578544051808e-01 432 | 8.914579578544052918e-01 433 | 8.918199501915700100e-01 434 | 8.918199501915700100e-01 435 | 8.918199501915700100e-01 436 | 8.918199501915700100e-01 437 | 8.918199501915700100e-01 438 | 8.918199501915700100e-01 439 | 8.918199501915700100e-01 440 | 8.918199501915700100e-01 441 | 8.918199501915700100e-01 442 | 8.918199501915700100e-01 443 | 8.918199501915700100e-01 444 | 8.921819425287347283e-01 445 | 8.921819425287347283e-01 446 | 8.921819425287347283e-01 447 | 8.921819425287347283e-01 448 | 8.923629386973171984e-01 449 | 8.923629386973171984e-01 450 | 8.923629386973171984e-01 451 | 8.923629386973171984e-01 452 | 8.923629386973171984e-01 453 | 8.925439348658995575e-01 454 | 8.921819425287347283e-01 455 | 8.921819425287347283e-01 456 | 8.921819425287347283e-01 457 | 8.920009463601523692e-01 458 | 8.912769616858229327e-01 459 | 8.910959655172405736e-01 460 | 8.907339731800757443e-01 461 | 8.905529770114932742e-01 462 | 8.905529770114932742e-01 463 | 8.907339731800757443e-01 464 | 8.907339731800757443e-01 465 | 8.909149693486581034e-01 466 | 8.909149693486581034e-01 467 | 8.909149693486581034e-01 468 | 8.910959655172405736e-01 469 | 8.909149693486581034e-01 470 | 8.910959655172405736e-01 471 | 8.909149693486581034e-01 472 | 8.912769616858229327e-01 473 | 8.912769616858229327e-01 474 | 8.916389540229875399e-01 475 | 8.916389540229877619e-01 476 | 8.920009463601523692e-01 477 | 8.916389540229875399e-01 478 | 8.916389540229875399e-01 479 | 8.916389540229875399e-01 480 | 8.916389540229875399e-01 481 | 8.916389540229875399e-01 482 | 8.916389540229877619e-01 483 | 8.914579578544052918e-01 484 | 8.912769616858229327e-01 485 | 8.907339731800757443e-01 486 | 8.907339731800757443e-01 487 | 8.907339731800757443e-01 488 | 8.909149693486581034e-01 489 | 8.910959655172405736e-01 490 | 8.914579578544052918e-01 491 | 8.912769616858227106e-01 492 | 8.912769616858229327e-01 493 | 8.910959655172405736e-01 494 | 8.909149693486581034e-01 495 | 8.909149693486581034e-01 496 | 8.909149693486581034e-01 497 | 8.907339731800757443e-01 498 | 8.910959655172405736e-01 499 | 8.910959655172405736e-01 500 | 8.910959655172405736e-01 501 | -------------------------------------------------------------------------------- /opt_weight/acc/dqn_0.5.txt: -------------------------------------------------------------------------------- 1 | 2.907000000000000139e-01 2 | 4.256899999999999573e-01 3 | 4.530666666666668951e-01 4 | 4.707933333333332304e-01 5 | 5.036866666666665049e-01 6 | 5.363766666666666127e-01 7 | 5.557666666666661870e-01 8 | 5.655366666666665765e-01 9 | 5.739299999999999402e-01 10 | 5.822166666666664936e-01 11 | 5.954966666666664521e-01 12 | 6.054299999999999127e-01 13 | 6.130166666666665432e-01 14 | 6.201633333333330667e-01 15 | 6.256466666666665732e-01 16 | 6.313366666666664351e-01 17 | 6.381066666666665999e-01 18 | 6.429500000000000215e-01 19 | 6.480966666666666542e-01 20 | 6.527833333333332710e-01 21 | 6.591966666666666530e-01 22 | 6.639766666666666595e-01 23 | 6.682366666666667010e-01 24 | 6.725233333333332508e-01 25 | 6.777533333333333188e-01 26 | 6.816966666666666175e-01 27 | 6.851599999999999913e-01 28 | 6.902566666666667405e-01 29 | 6.943166666666668041e-01 30 | 6.997533333333334493e-01 31 | 7.035666666666668956e-01 32 | 7.064566666666668437e-01 33 | 7.095400000000002816e-01 34 | 7.118566666666670262e-01 35 | 7.148433333333337192e-01 36 | 7.176433333333336329e-01 37 | 7.209866666666671087e-01 38 | 7.224300000000005717e-01 39 | 7.261000000000005228e-01 40 | 7.290866666666672158e-01 41 | 7.332466666666671573e-01 42 | 7.363466666666670379e-01 43 | 7.385433333333337735e-01 44 | 7.410700000000005616e-01 45 | 7.447100000000005382e-01 46 | 7.471533333333338911e-01 47 | 7.491066666666672536e-01 48 | 7.512166666666671988e-01 49 | 7.542433333333338208e-01 50 | 7.561266666666672798e-01 51 | 7.588233333333339603e-01 52 | 7.606066666666674303e-01 53 | 7.626166666666674976e-01 54 | 7.639000000000008006e-01 55 | 7.658633333333340065e-01 56 | 7.678300000000006786e-01 57 | 7.693633333333340651e-01 58 | 7.709733333333339544e-01 59 | 7.726400000000005486e-01 60 | 7.744000000000004214e-01 61 | 7.758700000000005037e-01 62 | 7.788700000000002843e-01 63 | 7.808800000000002406e-01 64 | 7.827366666666667472e-01 65 | 7.850133333333334518e-01 66 | 7.853366666666669049e-01 67 | 7.867466666666668162e-01 68 | 7.893933333333333913e-01 69 | 7.897733333333332162e-01 70 | 7.911866666666665937e-01 71 | 7.928633333333332533e-01 72 | 7.942600000000000771e-01 73 | 7.957566666666664457e-01 74 | 7.968033333333330859e-01 75 | 7.980166666666664854e-01 76 | 7.995833333333329795e-01 77 | 8.009133333333328109e-01 78 | 8.025799999999994050e-01 79 | 8.031966666666660037e-01 80 | 8.052833333333324628e-01 81 | 8.054033333333324718e-01 82 | 8.060999999999988175e-01 83 | 8.074966666666651971e-01 84 | 8.088799999999987111e-01 85 | 8.101299999999987955e-01 86 | 8.108033333333319881e-01 87 | 8.116533333333317835e-01 88 | 8.127166666666648664e-01 89 | 8.139666666666651729e-01 90 | 8.144933333333319592e-01 91 | 8.154799999999985394e-01 92 | 8.147366666666653323e-01 93 | 8.153133333333322241e-01 94 | 8.161599999999989974e-01 95 | 8.170466666666655886e-01 96 | 8.190233333333319932e-01 97 | 8.200699999999988554e-01 98 | 8.206933333333320535e-01 99 | 8.220799999999987007e-01 100 | 8.230099999999987981e-01 101 | 8.221299999999990282e-01 102 | 8.226466666666657490e-01 103 | 8.233499999999992491e-01 104 | 8.246133333333323101e-01 105 | 8.241266666666656748e-01 106 | 8.245799999999988694e-01 107 | 8.251566666666657612e-01 108 | 8.252666666666660378e-01 109 | 8.252999999999995895e-01 110 | 8.254733333333329481e-01 111 | 8.263799999999993373e-01 112 | 8.266966666666661911e-01 113 | 8.272166666666663781e-01 114 | 8.277299999999996327e-01 115 | 8.284866666666663715e-01 116 | 8.292066666666665364e-01 117 | 8.304166666666669139e-01 118 | 8.311433333333340112e-01 119 | 8.313266666666673244e-01 120 | 8.321466666666670342e-01 121 | 8.321033333333337501e-01 122 | 8.338333333333338704e-01 123 | 8.342766666666672215e-01 124 | 8.348400000000002485e-01 125 | 8.348166666666670954e-01 126 | 8.353033333333338417e-01 127 | 8.360833333333336226e-01 128 | 8.369433333333338165e-01 129 | 8.371366666666674172e-01 130 | 8.375600000000005263e-01 131 | 8.390133333333338328e-01 132 | 8.389333333333335307e-01 133 | 8.389200000000002211e-01 134 | 8.386933333333336238e-01 135 | 8.398033333333335682e-01 136 | 8.402066666666669903e-01 137 | 8.413133333333334685e-01 138 | 8.415233333333332899e-01 139 | 8.430666666666667419e-01 140 | 8.438966666666668504e-01 141 | 8.444400000000001905e-01 142 | 8.454100000000001058e-01 143 | 8.458866666666670087e-01 144 | 8.464833333333335874e-01 145 | 8.469100000000001627e-01 146 | 8.477033333333333642e-01 147 | 8.484533333333327265e-01 148 | 8.490466666666658391e-01 149 | 8.499199999999992317e-01 150 | 8.504799999999997917e-01 151 | 8.511899999999996691e-01 152 | 8.515799999999994485e-01 153 | 8.522666666666661728e-01 154 | 8.526266666666661997e-01 155 | 8.532433333333325765e-01 156 | 8.535633333333326744e-01 157 | 8.536899999999992827e-01 158 | 8.540699999999991077e-01 159 | 8.547199999999989251e-01 160 | 8.546499999999994657e-01 161 | 8.550166666666664250e-01 162 | 8.557199999999994811e-01 163 | 8.564466666666665784e-01 164 | 8.568833333333329971e-01 165 | 8.570833333333335302e-01 166 | 8.571566666666664558e-01 167 | 8.570899999999994634e-01 168 | 8.579933333333324974e-01 169 | 8.583266666666660161e-01 170 | 8.586266666666659830e-01 171 | 8.588799999999991996e-01 172 | 8.594566666666660915e-01 173 | 8.601199999999996626e-01 174 | 8.603633333333330357e-01 175 | 8.599433333333327267e-01 176 | 8.603733333333328792e-01 177 | 8.601433333333337039e-01 178 | 8.605400000000003047e-01 179 | 8.606600000000005357e-01 180 | 8.607266666666665289e-01 181 | 8.611366666666668834e-01 182 | 8.614066666666673200e-01 183 | 8.617100000000003091e-01 184 | 8.624300000000004740e-01 185 | 8.628966666666675334e-01 186 | 8.631600000000010375e-01 187 | 8.629066666666678209e-01 188 | 8.631666666666679699e-01 189 | 8.633200000000016416e-01 190 | 8.637066666666678438e-01 191 | 8.646133333333343440e-01 192 | 8.653233333333351096e-01 193 | 8.657833333333343484e-01 194 | 8.666766666666679830e-01 195 | 8.670866666666673384e-01 196 | 8.677300000000012226e-01 197 | 8.680866666666683384e-01 198 | 8.684566666666683199e-01 199 | 8.689600000000012869e-01 200 | 8.692333333333341905e-01 201 | 8.695866666666679512e-01 202 | 8.694033333333341940e-01 203 | 8.691266666666668250e-01 204 | 8.690533333333330113e-01 205 | 8.695699999999997321e-01 206 | 8.695266666666668920e-01 207 | 8.697566666666659563e-01 208 | 8.698133333333326611e-01 209 | 8.698766666666651881e-01 210 | 8.703066666666653406e-01 211 | 8.700999999999984302e-01 212 | 8.702399999999983482e-01 213 | 8.707833333333316883e-01 214 | 8.710833333333312112e-01 215 | 8.713133333333312747e-01 216 | 8.715699999999979575e-01 217 | 8.724233333333312190e-01 218 | 8.728533333333312605e-01 219 | 8.733733333333304483e-01 220 | 8.739133333333304332e-01 221 | 8.739599999999967395e-01 222 | 8.737533333333298291e-01 223 | 8.740966666666631912e-01 224 | 8.738466666666634408e-01 225 | 8.735166666666638324e-01 226 | 8.731366666666635634e-01 227 | 8.731399999999970296e-01 228 | 8.732966666666641675e-01 229 | 8.733799999999973807e-01 230 | 8.734766666666644586e-01 231 | 8.731766666666639365e-01 232 | 8.738366666666640414e-01 233 | 8.743899999999977801e-01 234 | 8.752433333333319299e-01 235 | 8.757733333333325154e-01 236 | 8.760899999999982590e-01 237 | 8.764899999999983260e-01 238 | 8.765466666666649198e-01 239 | 8.768966666666652143e-01 240 | 8.766833333333323708e-01 241 | 8.769133333333324343e-01 242 | 8.776133333333320241e-01 243 | 8.773433333333324757e-01 244 | 8.774466666666663750e-01 245 | 8.770333333333326653e-01 246 | 8.767433333333324308e-01 247 | 8.761433333333324969e-01 248 | 8.763099999999989231e-01 249 | 8.763999999999990687e-01 250 | 8.762499999999988631e-01 251 | 8.760699999999995713e-01 252 | 8.762199999999997768e-01 253 | 8.762433333333338181e-01 254 | 8.767200000000002769e-01 255 | 8.775700000000000722e-01 256 | 8.783899999999997821e-01 257 | 8.788566666666668414e-01 258 | 8.789366666666665884e-01 259 | 8.789266666666663008e-01 260 | 8.792466666666665098e-01 261 | 8.800266666666668458e-01 262 | 8.797933333333333161e-01 263 | 8.796333333333327120e-01 264 | 8.797666666666656976e-01 265 | 8.799299999999987687e-01 266 | 8.800199999999989142e-01 267 | 8.800299999999993128e-01 268 | 8.802433333333330445e-01 269 | 8.805400000000001004e-01 270 | 8.811933333333333840e-01 271 | 8.817566666666664110e-01 272 | 8.811599999999998323e-01 273 | 8.813599999999998102e-01 274 | 8.815733333333326538e-01 275 | 8.824333333333328477e-01 276 | 8.831699999999993445e-01 277 | 8.842199999999991178e-01 278 | 8.843599999999990358e-01 279 | 8.843766666666662557e-01 280 | 8.847433333333327710e-01 281 | 8.852966666666655104e-01 282 | 8.855799999999988126e-01 283 | 8.859333333333315741e-01 284 | 8.859133333333317761e-01 285 | 8.860733333333314921e-01 286 | 8.861699999999984589e-01 287 | 8.862399999999979183e-01 288 | 8.862466666666648507e-01 289 | 8.863699999999984369e-01 290 | 8.866366666666645191e-01 291 | 8.866799999999973592e-01 292 | 8.865833333333312805e-01 293 | 8.869933333333316350e-01 294 | 8.866566666666642060e-01 295 | 8.867733333333309709e-01 296 | 8.869133333333308888e-01 297 | 8.870399999999970531e-01 298 | 8.873333333333306427e-01 299 | 8.873299999999962884e-01 300 | 8.872133333333295235e-01 301 | 8.873066666666640234e-01 302 | 8.876266666666642324e-01 303 | 8.879133333333300016e-01 304 | 8.876266666666642324e-01 305 | 8.876599999999986723e-01 306 | 8.872533333333326722e-01 307 | 8.868399999999998506e-01 308 | 8.866066666666664320e-01 309 | 8.864500000000001823e-01 310 | 8.867300000000000182e-01 311 | 8.868333333333330293e-01 312 | 8.867733333333328583e-01 313 | 8.866699999999989590e-01 314 | 8.868133333333313439e-01 315 | 8.868733333333315150e-01 316 | 8.866833333333318246e-01 317 | 8.864666666666655148e-01 318 | 8.868199999999991645e-01 319 | 8.871499999999987729e-01 320 | 8.870366666666654742e-01 321 | 8.876166666666658323e-01 322 | 8.883166666666663103e-01 323 | 8.885166666666672874e-01 324 | 8.886300000000005861e-01 325 | 8.886200000000011867e-01 326 | 8.885600000000011267e-01 327 | 8.887766666666683246e-01 328 | 8.885866666666667468e-01 329 | 8.885700000000004151e-01 330 | 8.886266666666661207e-01 331 | 8.887166666666643788e-01 332 | 8.889299999999972224e-01 333 | 8.888133333333314567e-01 334 | 8.888233333333308561e-01 335 | 8.887333333333307106e-01 336 | 8.891499999999969983e-01 337 | 8.896299999999958130e-01 338 | 8.897499999999960441e-01 339 | 8.899466666666626669e-01 340 | 8.892733333333296963e-01 341 | 8.890833333333300059e-01 342 | 8.893633333333298419e-01 343 | 8.891399999999975989e-01 344 | 8.890533333333318078e-01 345 | 8.892566666666652520e-01 346 | 8.893999999999986361e-01 347 | 8.894799999999974949e-01 348 | 8.892766666666640507e-01 349 | 8.892099999999970583e-01 350 | 8.891899999999982596e-01 351 | 8.887266666666656656e-01 352 | 8.883133333333319559e-01 353 | 8.881933333333317249e-01 354 | 8.880899999999997130e-01 355 | 8.878599999999986503e-01 356 | 8.881099999999985117e-01 357 | 8.881633333333316394e-01 358 | 8.884399999999990083e-01 359 | 8.886333333333330531e-01 360 | 8.886366666666674075e-01 361 | 8.887400000000014177e-01 362 | 8.890100000000018543e-01 363 | 8.891933333333346123e-01 364 | 8.893966666666680565e-01 365 | 8.896433333333334526e-01 366 | 8.899166666666663561e-01 367 | 8.897633333333335726e-01 368 | 8.899999999999995692e-01 369 | 8.902166666666649908e-01 370 | 8.910433333333325212e-01 371 | 8.912233333333328122e-01 372 | 8.912966666666667370e-01 373 | 8.915066666666670026e-01 374 | 8.914233333333337894e-01 375 | 8.912433333333334984e-01 376 | 8.914866666666664274e-01 377 | 8.918933333333333158e-01 378 | 8.918599999999988759e-01 379 | 8.919333333333326896e-01 380 | 8.918099999999981042e-01 381 | 8.918466666666650111e-01 382 | 8.923599999999983767e-01 383 | 8.925333333333317354e-01 384 | 8.928533333333310562e-01 385 | 8.935033333333308736e-01 386 | 8.937933333333318853e-01 387 | 8.938899999999989632e-01 388 | 8.940866666666654750e-01 389 | 8.937866666666650639e-01 390 | 8.938666666666639227e-01 391 | 8.936233333333309936e-01 392 | 8.935799999999971543e-01 393 | 8.936299999999979260e-01 394 | 8.940199999999985936e-01 395 | 8.944333333333323033e-01 396 | 8.941599999999993997e-01 397 | 8.943900000000003514e-01 398 | 8.943900000000003514e-01 399 | 8.946266666666683465e-01 400 | 8.947833333333334860e-01 401 | 8.950900000000009404e-01 402 | 8.949899999999995082e-01 403 | 8.949299999999993371e-01 404 | 8.951233333333316056e-01 405 | 8.951899999999984869e-01 406 | 8.952266666666655048e-01 407 | 8.952599999999999447e-01 408 | 8.954300000000008364e-01 409 | 8.949966666666682169e-01 410 | 8.952900000000018066e-01 411 | 8.954833333333340750e-01 412 | 8.952933333333343846e-01 413 | 8.953433333333331579e-01 414 | 8.955533333333335344e-01 415 | 8.953366666666682239e-01 416 | 8.951533333333353548e-01 417 | 8.948466666666680114e-01 418 | 8.945733333333351078e-01 419 | 8.948200000000023913e-01 420 | 8.949433333333369767e-01 421 | 8.953866666666707719e-01 422 | 8.953033333333375587e-01 423 | 8.954166666666708574e-01 424 | 8.953366666666701112e-01 425 | 8.951066666666709359e-01 426 | 8.953366666666701112e-01 427 | 8.954033333333352163e-01 428 | 8.952700000000031189e-01 429 | 8.951233333333353803e-01 430 | 8.950533333333358099e-01 431 | 8.947566666666678659e-01 432 | 8.948966666666687830e-01 433 | 8.950666666666677873e-01 434 | 8.954133333333345046e-01 435 | 8.954400000000021231e-01 436 | 8.953333333333356459e-01 437 | 8.953500000000019776e-01 438 | 8.957233333333344261e-01 439 | 8.963466666666666249e-01 440 | 8.966633333333333677e-01 441 | 8.967933333333329982e-01 442 | 8.969466666666657817e-01 443 | 8.967633333333329126e-01 444 | 8.966166666666651741e-01 445 | 8.965433333333313604e-01 446 | 8.964833333333311893e-01 447 | 8.965133333333312748e-01 448 | 8.965599999999975811e-01 449 | 8.965833333333307342e-01 450 | 8.965599999999975811e-01 451 | 8.963566666666641369e-01 452 | 8.963899999999985768e-01 453 | 8.961999999999988864e-01 454 | 8.963266666666659388e-01 455 | 8.962966666666659643e-01 456 | 8.961200000000000276e-01 457 | 8.961266666666669600e-01 458 | 8.960800000000006538e-01 459 | 8.961100000000006283e-01 460 | 8.959366666666672696e-01 461 | 8.960366666666687019e-01 462 | 8.959900000000023956e-01 463 | 8.960866666666693625e-01 464 | 8.960100000000029707e-01 465 | 8.959833333333354632e-01 466 | 8.958933333333353177e-01 467 | 8.957633333333357983e-01 468 | 8.957500000000038209e-01 469 | 8.956900000000037609e-01 470 | 8.955733333333379953e-01 471 | 8.956400000000049877e-01 472 | 8.955400000000054428e-01 473 | 8.957633333333395731e-01 474 | 8.958900000000066255e-01 475 | 8.959866666666737034e-01 476 | 8.958666666666734724e-01 477 | 8.957933333333414350e-01 478 | 8.955333333333422852e-01 479 | 8.954100000000076998e-01 480 | 8.955000000000078453e-01 481 | 8.956633333333400282e-01 482 | 8.954700000000059834e-01 483 | 8.955666666666710629e-01 484 | 8.954700000000040960e-01 485 | 8.955300000000041560e-01 486 | 8.956333333333380553e-01 487 | 8.958100000000058794e-01 488 | 8.960166666666717905e-01 489 | 8.959833333333392380e-01 490 | 8.961733333333389284e-01 491 | 8.958800000000053387e-01 492 | 8.957000000000050477e-01 493 | 8.956900000000056483e-01 494 | 8.952866666666731144e-01 495 | 8.952866666666731144e-01 496 | 8.954800000000072702e-01 497 | 8.955933333333405688e-01 498 | 8.954400000000058979e-01 499 | 8.954900000000065585e-01 500 | 8.954700000000059834e-01 501 | -------------------------------------------------------------------------------- /opt_weight/acc/dqn_0.7.txt: -------------------------------------------------------------------------------- 1 | 2.907000000000000139e-01 2 | 3.537025038314175474e-01 3 | 3.870714099616857085e-01 4 | 4.507926194125158048e-01 5 | 4.791486858237545832e-01 6 | 5.008682260536395647e-01 7 | 5.165545606641120946e-01 8 | 5.304309335887609222e-01 9 | 5.406873831417621235e-01 10 | 5.509438326947634357e-01 11 | 5.587869999999997006e-01 12 | 5.666301673052359655e-01 13 | 5.738700140485309964e-01 14 | 5.814115210727965888e-01 15 | 5.889530280970621812e-01 16 | 5.955895542784159780e-01 17 | 6.019244201787991022e-01 18 | 6.076559655172409924e-01 19 | 6.127841902937415375e-01 20 | 6.176107547892716321e-01 21 | 6.230406398467428497e-01 22 | 6.290738454661553014e-01 23 | 6.345037305236266301e-01 24 | 6.399336155810978477e-01 25 | 6.444585197956572697e-01 26 | 6.492850842911872533e-01 27 | 6.535083282247761138e-01 28 | 6.583348927203059864e-01 29 | 6.631614572158359699e-01 30 | 6.673847011494246084e-01 31 | 6.707029642401015623e-01 32 | 6.740212273307785162e-01 33 | 6.776411507024260317e-01 34 | 6.821660549169853427e-01 35 | 6.854843180076622966e-01 36 | 6.894059016602803736e-01 37 | 6.927241647509573275e-01 38 | 6.969474086845459659e-01 39 | 7.002656717752229198e-01 40 | 7.038855951468704353e-01 41 | 7.066005376756060441e-01 42 | 7.090138199233709804e-01 43 | 7.111254418901653551e-01 44 | 7.135387241379304024e-01 45 | 7.162536666666660112e-01 46 | 7.189686091954016200e-01 47 | 7.216835517241372289e-01 48 | 7.250018148148141828e-01 49 | 7.274150970625791190e-01 50 | 7.298283793103442774e-01 51 | 7.313366807151973070e-01 52 | 7.334483026819916818e-01 53 | 7.352582643678154950e-01 54 | 7.370682260536391972e-01 55 | 7.385765274584922269e-01 56 | 7.403864891443160401e-01 57 | 7.415931302681985082e-01 58 | 7.434030919540223215e-01 59 | 7.452130536398460237e-01 60 | 7.467213550446992754e-01 61 | 7.482296564495523050e-01 62 | 7.488329770114936501e-01 63 | 7.503412784163466798e-01 64 | 7.515479195402292589e-01 65 | 7.533578812260529611e-01 66 | 7.551678429118767744e-01 67 | 7.557711634738178974e-01 68 | 7.572794648786711491e-01 69 | 7.587877662835241788e-01 70 | 7.608993882503186645e-01 71 | 7.618043690932304601e-01 72 | 7.627093499361423667e-01 73 | 7.639159910600247239e-01 74 | 7.648209719029367415e-01 75 | 7.654242924648779756e-01 76 | 7.663292733077898822e-01 77 | 7.675359144316723503e-01 78 | 7.687425555555548184e-01 79 | 7.693458761174961635e-01 80 | 7.708541775223491932e-01 81 | 7.720608186462317724e-01 82 | 7.732674597701142405e-01 83 | 7.732674597701142405e-01 84 | 7.729657994891435679e-01 85 | 7.732674597701142405e-01 86 | 7.741724406130261471e-01 87 | 7.753790817369085042e-01 88 | 7.768873831417616449e-01 89 | 7.777923639846736625e-01 90 | 7.786973448275854581e-01 91 | 7.799039859514680373e-01 92 | 7.811106270753503944e-01 93 | 7.829205887611742076e-01 94 | 7.832222490421448802e-01 95 | 7.835239093231155527e-01 96 | 7.835239093231155527e-01 97 | 7.841272298850566758e-01 98 | 7.841272298850566758e-01 99 | 7.850322107279685824e-01 100 | 7.850322107279685824e-01 101 | 7.874454929757336297e-01 102 | 7.886521340996162088e-01 103 | 7.907637560664104726e-01 104 | 7.910654163473811451e-01 105 | 7.910654163473811451e-01 106 | 7.910654163473811451e-01 107 | 7.916687369093223792e-01 108 | 7.928753780332048473e-01 109 | 7.940820191570873154e-01 110 | 7.952886602809700056e-01 111 | 7.967969616858230353e-01 112 | 7.986069233716467375e-01 113 | 7.989085836526174100e-01 114 | 7.980036028097056144e-01 115 | 7.983052630906760649e-01 116 | 7.992102439335879716e-01 117 | 8.007185453384412233e-01 118 | 8.007185453384412233e-01 119 | 8.010202056194116738e-01 120 | 8.019251864623235804e-01 121 | 8.022268467432942529e-01 122 | 8.025285070242649255e-01 123 | 8.028301673052355980e-01 124 | 8.034334878671768321e-01 125 | 8.040368084291180661e-01 126 | 8.049417892720299728e-01 127 | 8.058467701149417683e-01 128 | 8.076567318007654706e-01 129 | 8.079583920817361431e-01 130 | 8.091650332056187223e-01 131 | 8.094666934865891728e-01 132 | 8.097683537675598453e-01 133 | 8.097683537675598453e-01 134 | 8.094666934865891728e-01 135 | 8.103716743295010794e-01 136 | 8.103716743295011904e-01 137 | 8.103716743295011904e-01 138 | 8.106733346104717519e-01 139 | 8.121816360153250036e-01 140 | 8.133882771392073607e-01 141 | 8.142932579821192673e-01 142 | 8.139915977011487058e-01 143 | 8.154998991060017355e-01 144 | 8.164048799489136421e-01 145 | 8.176115210727961102e-01 146 | 8.176115210727961102e-01 147 | 8.176115210727961102e-01 148 | 8.176115210727961102e-01 149 | 8.179131813537666718e-01 150 | 8.188181621966786894e-01 151 | 8.197231430395905960e-01 152 | 8.197231430395905960e-01 153 | 8.194214827586199235e-01 154 | 8.203264636015318301e-01 155 | 8.218347650063849708e-01 156 | 8.230414061302674389e-01 157 | 8.230414061302674389e-01 158 | 8.236447266922085619e-01 159 | 8.248513678160911411e-01 160 | 8.263596692209441708e-01 161 | 8.266613295019148433e-01 162 | 8.269629897828855158e-01 163 | 8.269629897828855158e-01 164 | 8.272646500638561884e-01 165 | 8.275663103448267499e-01 166 | 8.278679706257974225e-01 167 | 8.284712911877386565e-01 168 | 8.287729514687093291e-01 169 | 8.299795925925917972e-01 170 | 8.311862337164742653e-01 171 | 8.320912145593861720e-01 172 | 8.326945351213274060e-01 173 | 8.339011762452099852e-01 174 | 8.348061570881216698e-01 175 | 8.348061570881218918e-01 176 | 8.345044968071512193e-01 177 | 8.348061570881216698e-01 178 | 8.357111379310336874e-01 179 | 8.372194393358868281e-01 180 | 8.378227598978281732e-01 181 | 8.387277407407399688e-01 182 | 8.384260804597692962e-01 183 | 8.390294010217105303e-01 184 | 8.384260804597692962e-01 185 | 8.393310613026812028e-01 186 | 8.387277407407398577e-01 187 | 8.399343818646224369e-01 188 | 8.399343818646224369e-01 189 | 8.408393627075342325e-01 190 | 8.414426832694755776e-01 191 | 8.414426832694755776e-01 192 | 8.420460038314168116e-01 193 | 8.417443435504461391e-01 194 | 8.426493243933580457e-01 195 | 8.432526449552991687e-01 196 | 8.432526449552991687e-01 197 | 8.432526449552993908e-01 198 | 8.432526449552991687e-01 199 | 8.441576257982111864e-01 200 | 8.447609463601524205e-01 201 | 8.447609463601524205e-01 202 | 8.450626066411230930e-01 203 | 8.444592860791817479e-01 204 | 8.456659272030643271e-01 205 | 8.462692477650054501e-01 206 | 8.468725683269467952e-01 207 | 8.468725683269467952e-01 208 | 8.465709080459761227e-01 209 | 8.471742286079173567e-01 210 | 8.471742286079173567e-01 211 | 8.474758888888880293e-01 212 | 8.474758888888880293e-01 213 | 8.480792094508293744e-01 214 | 8.483808697318000469e-01 215 | 8.489841902937412810e-01 216 | 8.489841902937410589e-01 217 | 8.486825300127706084e-01 218 | 8.483808697317999359e-01 219 | 8.480792094508293744e-01 220 | 8.486825300127706084e-01 221 | 8.489841902937410589e-01 222 | 8.495875108556824040e-01 223 | 8.498891711366530766e-01 224 | 8.501908314176237491e-01 225 | 8.501908314176237491e-01 226 | 8.501908314176237491e-01 227 | 8.504924916985944217e-01 228 | 8.507941519795649832e-01 229 | 8.510958122605355447e-01 230 | 8.510958122605355447e-01 231 | 8.510958122605355447e-01 232 | 8.510958122605355447e-01 233 | 8.510958122605355447e-01 234 | 8.513974725415062172e-01 235 | 8.513974725415062172e-01 236 | 8.513974725415062172e-01 237 | 8.510958122605355447e-01 238 | 8.510958122605355447e-01 239 | 8.510958122605355447e-01 240 | 8.513974725415062172e-01 241 | 8.526041136653886854e-01 242 | 8.532074342273299195e-01 243 | 8.538107547892712645e-01 244 | 8.535090945083005920e-01 245 | 8.544140753512124986e-01 246 | 8.550173959131536217e-01 247 | 8.553190561941242942e-01 248 | 8.565256973180068734e-01 249 | 8.568273575989774349e-01 250 | 8.571290178799481074e-01 251 | 8.559223767560656393e-01 252 | 8.565256973180068734e-01 253 | 8.574306781609187800e-01 254 | 8.589389795657718096e-01 255 | 8.595423001277131547e-01 256 | 8.598439604086837162e-01 257 | 8.592406398467424822e-01 258 | 8.589389795657718096e-01 259 | 8.592406398467424822e-01 260 | 8.598439604086837162e-01 261 | 8.604472809706248393e-01 262 | 8.613522618135367459e-01 263 | 8.622572426564487635e-01 264 | 8.622572426564487635e-01 265 | 8.616539220945074184e-01 266 | 8.610506015325661844e-01 267 | 8.610506015325661844e-01 268 | 8.610506015325661844e-01 269 | 8.610506015325661844e-01 270 | 8.610506015325661844e-01 271 | 8.610506015325661844e-01 272 | 8.607489412515955118e-01 273 | 8.604472809706248393e-01 274 | 8.601456206896542778e-01 275 | 8.604472809706248393e-01 276 | 8.613522618135368569e-01 277 | 8.619555823754780910e-01 278 | 8.622572426564487635e-01 279 | 8.619555823754780910e-01 280 | 8.622572426564487635e-01 281 | 8.625589029374193251e-01 282 | 8.628605632183899976e-01 283 | 8.631622234993606702e-01 284 | 8.634638837803311207e-01 285 | 8.631622234993604481e-01 286 | 8.631622234993606702e-01 287 | 8.625589029374193251e-01 288 | 8.631622234993606702e-01 289 | 8.625589029374193251e-01 290 | 8.628605632183899976e-01 291 | 8.625589029374194361e-01 292 | 8.631622234993606702e-01 293 | 8.634638837803311207e-01 294 | 8.640672043422723547e-01 295 | 8.643688646232430273e-01 296 | 8.643688646232430273e-01 297 | 8.640672043422724657e-01 298 | 8.640672043422724657e-01 299 | 8.643688646232430273e-01 300 | 8.652738454661550449e-01 301 | 8.658771660280962790e-01 302 | 8.658771660280962790e-01 303 | 8.658771660280962790e-01 304 | 8.658771660280962790e-01 305 | 8.658771660280962790e-01 306 | 8.658771660280962790e-01 307 | 8.652738454661550449e-01 308 | 8.652738454661550449e-01 309 | 8.643688646232430273e-01 310 | 8.643688646232430273e-01 311 | 8.640672043422723547e-01 312 | 8.646705249042136998e-01 313 | 8.649721851851843724e-01 314 | 8.649721851851843724e-01 315 | 8.649721851851843724e-01 316 | 8.643688646232430273e-01 317 | 8.655755057471256064e-01 318 | 8.661788263090667295e-01 319 | 8.670838071519787471e-01 320 | 8.661788263090667295e-01 321 | 8.658771660280962790e-01 322 | 8.658771660280962790e-01 323 | 8.661788263090669515e-01 324 | 8.661788263090669515e-01 325 | 8.661788263090669515e-01 326 | 8.661788263090667295e-01 327 | 8.667821468710080746e-01 328 | 8.667821468710080746e-01 329 | 8.676871277139199812e-01 330 | 8.676871277139199812e-01 331 | 8.676871277139199812e-01 332 | 8.676871277139199812e-01 333 | 8.676871277139199812e-01 334 | 8.682904482758612152e-01 335 | 8.682904482758612152e-01 336 | 8.679887879948905427e-01 337 | 8.673854674329493086e-01 338 | 8.667821468710080746e-01 339 | 8.664804865900375130e-01 340 | 8.664804865900375130e-01 341 | 8.664804865900375130e-01 342 | 8.664804865900375130e-01 343 | 8.667821468710080746e-01 344 | 8.670838071519787471e-01 345 | 8.670838071519786361e-01 346 | 8.676871277139199812e-01 347 | 8.676871277139199812e-01 348 | 8.679887879948906537e-01 349 | 8.673854674329493086e-01 350 | 8.673854674329493086e-01 351 | 8.673854674329493086e-01 352 | 8.673854674329493086e-01 353 | 8.673854674329493086e-01 354 | 8.682904482758612152e-01 355 | 8.685921085568318878e-01 356 | 8.685921085568318878e-01 357 | 8.676871277139198702e-01 358 | 8.673854674329493086e-01 359 | 8.673854674329493086e-01 360 | 8.673854674329493086e-01 361 | 8.670838071519787471e-01 362 | 8.670838071519787471e-01 363 | 8.673854674329493086e-01 364 | 8.679887879948905427e-01 365 | 8.685921085568318878e-01 366 | 8.691954291187730108e-01 367 | 8.697987496807142449e-01 368 | 8.697987496807142449e-01 369 | 8.697987496807142449e-01 370 | 8.701004099616849174e-01 371 | 8.704020702426555900e-01 372 | 8.704020702426555900e-01 373 | 8.701004099616849174e-01 374 | 8.701004099616849174e-01 375 | 8.697987496807142449e-01 376 | 8.697987496807142449e-01 377 | 8.694970893997436834e-01 378 | 8.694970893997436834e-01 379 | 8.691954291187731219e-01 380 | 8.691954291187731219e-01 381 | 8.691954291187731219e-01 382 | 8.691954291187731219e-01 383 | 8.694970893997436834e-01 384 | 8.697987496807142449e-01 385 | 8.701004099616849174e-01 386 | 8.697987496807143559e-01 387 | 8.694970893997436834e-01 388 | 8.701004099616850285e-01 389 | 8.704020702426555900e-01 390 | 8.707037305236262625e-01 391 | 8.707037305236261515e-01 392 | 8.710053908045968241e-01 393 | 8.716087113665379471e-01 394 | 8.716087113665381692e-01 395 | 8.719103716475087307e-01 396 | 8.719103716475087307e-01 397 | 8.719103716475087307e-01 398 | 8.719103716475087307e-01 399 | 8.719103716475087307e-01 400 | 8.722120319284792922e-01 401 | 8.725136922094499647e-01 402 | 8.728153524904206373e-01 403 | 8.722120319284792922e-01 404 | 8.722120319284792922e-01 405 | 8.722120319284792922e-01 406 | 8.731170127713911988e-01 407 | 8.731170127713913098e-01 408 | 8.731170127713911988e-01 409 | 8.728153524904206373e-01 410 | 8.728153524904206373e-01 411 | 8.734186730523618714e-01 412 | 8.734186730523618714e-01 413 | 8.734186730523618714e-01 414 | 8.728153524904206373e-01 415 | 8.728153524904206373e-01 416 | 8.734186730523618714e-01 417 | 8.746253141762444505e-01 418 | 8.752286347381855736e-01 419 | 8.755302950191561351e-01 420 | 8.755302950191561351e-01 421 | 8.755302950191562461e-01 422 | 8.755302950191562461e-01 423 | 8.749269744572149010e-01 424 | 8.749269744572149010e-01 425 | 8.746253141762443395e-01 426 | 8.749269744572149010e-01 427 | 8.755302950191562461e-01 428 | 8.758319553001268076e-01 429 | 8.764352758620681527e-01 430 | 8.767369361430387142e-01 431 | 8.773402567049799483e-01 432 | 8.779435772669211824e-01 433 | 8.779435772669211824e-01 434 | 8.779435772669211824e-01 435 | 8.776419169859505098e-01 436 | 8.779435772669211824e-01 437 | 8.779435772669211824e-01 438 | 8.776419169859505098e-01 439 | 8.773402567049799483e-01 440 | 8.773402567049799483e-01 441 | 8.773402567049799483e-01 442 | 8.770385964240093868e-01 443 | 8.770385964240093868e-01 444 | 8.770385964240093868e-01 445 | 8.773402567049799483e-01 446 | 8.773402567049799483e-01 447 | 8.770385964240093868e-01 448 | 8.767369361430388253e-01 449 | 8.764352758620681527e-01 450 | 8.767369361430387142e-01 451 | 8.767369361430387142e-01 452 | 8.770385964240093868e-01 453 | 8.770385964240093868e-01 454 | 8.770385964240093868e-01 455 | 8.770385964240093868e-01 456 | 8.764352758620681527e-01 457 | 8.764352758620681527e-01 458 | 8.764352758620681527e-01 459 | 8.770385964240093868e-01 460 | 8.773402567049799483e-01 461 | 8.773402567049799483e-01 462 | 8.773402567049799483e-01 463 | 8.773402567049799483e-01 464 | 8.773402567049799483e-01 465 | 8.773402567049799483e-01 466 | 8.770385964240093868e-01 467 | 8.767369361430388253e-01 468 | 8.764352758620681527e-01 469 | 8.764352758620681527e-01 470 | 8.764352758620681527e-01 471 | 8.764352758620681527e-01 472 | 8.764352758620681527e-01 473 | 8.764352758620681527e-01 474 | 8.770385964240093868e-01 475 | 8.770385964240093868e-01 476 | 8.770385964240093868e-01 477 | 8.764352758620681527e-01 478 | 8.770385964240094978e-01 479 | 8.767369361430388253e-01 480 | 8.764352758620681527e-01 481 | 8.755302950191562461e-01 482 | 8.755302950191562461e-01 483 | 8.752286347381855736e-01 484 | 8.752286347381855736e-01 485 | 8.752286347381855736e-01 486 | 8.752286347381855736e-01 487 | 8.752286347381855736e-01 488 | 8.752286347381855736e-01 489 | 8.755302950191562461e-01 490 | 8.755302950191562461e-01 491 | 8.755302950191562461e-01 492 | 8.755302950191562461e-01 493 | 8.755302950191562461e-01 494 | 8.755302950191562461e-01 495 | 8.758319553001268076e-01 496 | 8.761336155810974802e-01 497 | 8.764352758620681527e-01 498 | 8.761336155810974802e-01 499 | 8.761336155810974802e-01 500 | 8.761336155810974802e-01 501 | -------------------------------------------------------------------------------- /opt_weight/acc/dqn_0.9.txt: -------------------------------------------------------------------------------- 1 | 2.907000000000000139e-01 2 | 3.559649559386972584e-01 3 | 3.876747305236269425e-01 4 | 4.100907289272028855e-01 5 | 4.293322053639844205e-01 6 | 4.760717509578541451e-01 7 | 5.005062337164748465e-01 8 | 5.209588007662832032e-01 9 | 5.379724406130265146e-01 10 | 5.508231685823752333e-01 11 | 5.616829386973176685e-01 12 | 5.707327471264364016e-01 13 | 5.799635517241374938e-01 14 | 5.891943563218386970e-01 15 | 5.980631685823750709e-01 16 | 6.063889923371643675e-01 17 | 6.141718275862064758e-01 18 | 6.212306781609191475e-01 19 | 6.279275363984669900e-01 20 | 6.342624022988501142e-01 21 | 6.409592605363979567e-01 22 | 6.478371149425281583e-01 23 | 6.539909846743290345e-01 24 | 6.596018659003826112e-01 25 | 6.648507547892714697e-01 26 | 6.695566551724132509e-01 27 | 6.728145862068959371e-01 28 | 6.764345095785435635e-01 29 | 6.800544329501909679e-01 30 | 6.838553524904209535e-01 31 | 6.876562720306507170e-01 32 | 6.916381877394630617e-01 33 | 6.950771149425281070e-01 34 | 6.981540498084285451e-01 35 | 7.014119808429113423e-01 36 | 7.041269233716468401e-01 37 | 7.072038582375472782e-01 38 | 7.106427854406124345e-01 39 | 7.137197203065127615e-01 40 | 7.166156590038308405e-01 41 | 7.195115977011486974e-01 42 | 7.224075363984667764e-01 43 | 7.247604865900376669e-01 44 | 7.278374214559379940e-01 45 | 7.303713678160913547e-01 46 | 7.329053141762446044e-01 47 | 7.354392605363978541e-01 48 | 7.381542030651334629e-01 49 | 7.405071532567043535e-01 50 | 7.435840881226046806e-01 51 | 7.466610229885051186e-01 52 | 7.495569616858229756e-01 53 | 7.517289157088116180e-01 54 | 7.540818659003825086e-01 55 | 7.557108314176238517e-01 56 | 7.575207931034475539e-01 57 | 7.587877662835242898e-01 58 | 7.605977279693479920e-01 59 | 7.622266934865893351e-01 60 | 7.638556590038307892e-01 61 | 7.654846245210721323e-01 62 | 7.674755823754783046e-01 63 | 7.692855440613020068e-01 64 | 7.707335134099608798e-01 65 | 7.720004865900376156e-01 66 | 7.732674597701142405e-01 67 | 7.741724406130260361e-01 68 | 7.752584176245204128e-01 69 | 7.763443946360146786e-01 70 | 7.777923639846736625e-01 71 | 7.790593371647502874e-01 72 | 7.801453141762444421e-01 73 | 7.815932835249034261e-01 74 | 7.830412528735625211e-01 75 | 7.837652375478919575e-01 76 | 7.843082260536390349e-01 77 | 7.857561954022981299e-01 78 | 7.864801800766276774e-01 79 | 7.875661570881218321e-01 80 | 7.891951226053632862e-01 81 | 7.910050842911869884e-01 82 | 7.924530536398459724e-01 83 | 7.940820191570873154e-01 84 | 7.951679961685815812e-01 85 | 7.962539731800759579e-01 86 | 7.969779578544053944e-01 87 | 7.978829386973173010e-01 88 | 7.986069233716468485e-01 89 | 7.998738965517234734e-01 90 | 8.011408697317999872e-01 91 | 8.020458505747118938e-01 92 | 8.029508314176239114e-01 93 | 8.038558122605357070e-01 94 | 8.043988007662827844e-01 95 | 8.049417892720297507e-01 96 | 8.058467701149417683e-01 97 | 8.063897586206889567e-01 98 | 8.074757356321831114e-01 99 | 8.081997203065126589e-01 100 | 8.085617126436772661e-01 101 | 8.094666934865892838e-01 102 | 8.103716743295011904e-01 103 | 8.109146628352481567e-01 104 | 8.112766551724129860e-01 105 | 8.121816360153248926e-01 106 | 8.125436283524896108e-01 107 | 8.130866168582369102e-01 108 | 8.134486091954015174e-01 109 | 8.143535900383133130e-01 110 | 8.150775747126429716e-01 111 | 8.158015593869724080e-01 112 | 8.161635517241372373e-01 113 | 8.168875363984666738e-01 114 | 8.176115210727961102e-01 115 | 8.181545095785432986e-01 116 | 8.186974980842904870e-01 117 | 8.192404865900375643e-01 118 | 8.194214827586199235e-01 119 | 8.197834750957847527e-01 120 | 8.201454674329493599e-01 121 | 8.205074597701141892e-01 122 | 8.212314444444436257e-01 123 | 8.214124406130259848e-01 124 | 8.221364252873556433e-01 125 | 8.223174214559378914e-01 126 | 8.230414061302674389e-01 127 | 8.230414061302674389e-01 128 | 8.241273831417617046e-01 129 | 8.243083793103440637e-01 130 | 8.250323639846735002e-01 131 | 8.253943563218383295e-01 132 | 8.261183409961677659e-01 133 | 8.259373448275854068e-01 134 | 8.261183409961677659e-01 135 | 8.261183409961677659e-01 136 | 8.261183409961677659e-01 137 | 8.261183409961677659e-01 138 | 8.261183409961677659e-01 139 | 8.261183409961677659e-01 140 | 8.259373448275854068e-01 141 | 8.255753524904207996e-01 142 | 8.255753524904205776e-01 143 | 8.261183409961677659e-01 144 | 8.257563486590029367e-01 145 | 8.257563486590029367e-01 146 | 8.262993371647502361e-01 147 | 8.259373448275854068e-01 148 | 8.259373448275854068e-01 149 | 8.259373448275854068e-01 150 | 8.266613295019149543e-01 151 | 8.266613295019148433e-01 152 | 8.268423256704972024e-01 153 | 8.272043180076620317e-01 154 | 8.275663103448268609e-01 155 | 8.272043180076620317e-01 156 | 8.273853141762443908e-01 157 | 8.281092988505738273e-01 158 | 8.282902950191562974e-01 159 | 8.288332835249034858e-01 160 | 8.299192605363977515e-01 161 | 8.301002567049801106e-01 162 | 8.311862337164743764e-01 163 | 8.317292222222214537e-01 164 | 8.328151992337156084e-01 165 | 8.333581877394629078e-01 166 | 8.340821724137923443e-01 167 | 8.342631685823747034e-01 168 | 8.348061570881217808e-01 169 | 8.349871532567041399e-01 170 | 8.351681494252864990e-01 171 | 8.351681494252864990e-01 172 | 8.346251609195395327e-01 173 | 8.340821724137923443e-01 174 | 8.337201800766276261e-01 175 | 8.333581877394629078e-01 176 | 8.337201800766275150e-01 177 | 8.340821724137923443e-01 178 | 8.344441647509570625e-01 179 | 8.346251609195393106e-01 180 | 8.348061570881217808e-01 181 | 8.344441647509570625e-01 182 | 8.346251609195393106e-01 183 | 8.349871532567041399e-01 184 | 8.353491455938689692e-01 185 | 8.358921340996161575e-01 186 | 8.371591072796926714e-01 187 | 8.378830919540222189e-01 188 | 8.382450842911868261e-01 189 | 8.386070766283516553e-01 190 | 8.387880727969341255e-01 191 | 8.391500651340988437e-01 192 | 8.389690689655164846e-01 193 | 8.396930536398459211e-01 194 | 8.400550459770107503e-01 195 | 8.407790306513401868e-01 196 | 8.405980344827579387e-01 197 | 8.409600268199225459e-01 198 | 8.405980344827577166e-01 199 | 8.402360421455931094e-01 200 | 8.400550459770107503e-01 201 | 8.402360421455931094e-01 202 | 8.402360421455931094e-01 203 | 8.404170383141755796e-01 204 | 8.409600268199225459e-01 205 | 8.407790306513401868e-01 206 | 8.405980344827579387e-01 207 | 8.407790306513401868e-01 208 | 8.409600268199225459e-01 209 | 8.411410229885050160e-01 210 | 8.416840114942519824e-01 211 | 8.424079961685816409e-01 212 | 8.429509846743286072e-01 213 | 8.433129770114934365e-01 214 | 8.436749693486582657e-01 215 | 8.438559655172405138e-01 216 | 8.438559655172405138e-01 217 | 8.438559655172405138e-01 218 | 8.438559655172405138e-01 219 | 8.440369616858228730e-01 220 | 8.440369616858228730e-01 221 | 8.442179578544053431e-01 222 | 8.443989540229877022e-01 223 | 8.445799501915700613e-01 224 | 8.447609463601523094e-01 225 | 8.451229386973171387e-01 226 | 8.453039348658994978e-01 227 | 8.454849310344819679e-01 228 | 8.460279195402291563e-01 229 | 8.460279195402289343e-01 230 | 8.465709080459762337e-01 231 | 8.471138965517234221e-01 232 | 8.476568850574703884e-01 233 | 8.478378812260528585e-01 234 | 8.485618659003822950e-01 235 | 8.480188773946352176e-01 236 | 8.480188773946352176e-01 237 | 8.480188773946352176e-01 238 | 8.483808697318000469e-01 239 | 8.485618659003822950e-01 240 | 8.491048544061294834e-01 241 | 8.491048544061294834e-01 242 | 8.491048544061294834e-01 243 | 8.483808697318000469e-01 244 | 8.480188773946352176e-01 245 | 8.480188773946352176e-01 246 | 8.478378812260528585e-01 247 | 8.478378812260528585e-01 248 | 8.481998735632174657e-01 249 | 8.481998735632174657e-01 250 | 8.483808697317998249e-01 251 | 8.489238582375471243e-01 252 | 8.491048544061294834e-01 253 | 8.492858505747118425e-01 254 | 8.494668467432940906e-01 255 | 8.494668467432943126e-01 256 | 8.494668467432940906e-01 257 | 8.496478429118766718e-01 258 | 8.500098352490412790e-01 259 | 8.503718275862061082e-01 260 | 8.510958122605355447e-01 261 | 8.516388007662826221e-01 262 | 8.520007931034474513e-01 263 | 8.525437816091946397e-01 264 | 8.532677662835240762e-01 265 | 8.536297586206889054e-01 266 | 8.539917509578535126e-01 267 | 8.547157356321830601e-01 268 | 8.552587241379303595e-01 269 | 8.552587241379301375e-01 270 | 8.554397203065124966e-01 271 | 8.554397203065124966e-01 272 | 8.554397203065124966e-01 273 | 8.552587241379301375e-01 274 | 8.556207164750949667e-01 275 | 8.558017126436773259e-01 276 | 8.559827088122597960e-01 277 | 8.558017126436773259e-01 278 | 8.558017126436773259e-01 279 | 8.556207164750949667e-01 280 | 8.556207164750949667e-01 281 | 8.558017126436773259e-01 282 | 8.561637049808421551e-01 283 | 8.563447011494244032e-01 284 | 8.565256973180068734e-01 285 | 8.568876896551715916e-01 286 | 8.570686858237539507e-01 287 | 8.574306781609187800e-01 288 | 8.577926704980833872e-01 289 | 8.581546628352482164e-01 290 | 8.581546628352482164e-01 291 | 8.588786475095776529e-01 292 | 8.590596436781601231e-01 293 | 8.592406398467424822e-01 294 | 8.596026321839073114e-01 295 | 8.596026321839070894e-01 296 | 8.590596436781601231e-01 297 | 8.590596436781601231e-01 298 | 8.592406398467424822e-01 299 | 8.594216360153247303e-01 300 | 8.599646245210719187e-01 301 | 8.605076130268191070e-01 302 | 8.610506015325661844e-01 303 | 8.614125938697310136e-01 304 | 8.615935900383133728e-01 305 | 8.617745862068957319e-01 306 | 8.623175747126428092e-01 307 | 8.626795670498076385e-01 308 | 8.626795670498076385e-01 309 | 8.630415593869724677e-01 310 | 8.635845478927194341e-01 311 | 8.635845478927194341e-01 312 | 8.635845478927194341e-01 313 | 8.639465402298840413e-01 314 | 8.637655440613019042e-01 315 | 8.634035517241370750e-01 316 | 8.634035517241370750e-01 317 | 8.634035517241370750e-01 318 | 8.632225555555546048e-01 319 | 8.634035517241370750e-01 320 | 8.634035517241370750e-01 321 | 8.630415593869724677e-01 322 | 8.630415593869722457e-01 323 | 8.630415593869724677e-01 324 | 8.630415593869722457e-01 325 | 8.634035517241370750e-01 326 | 8.639465402298840413e-01 327 | 8.637655440613019042e-01 328 | 8.641275363984666225e-01 329 | 8.643085325670488706e-01 330 | 8.643085325670488706e-01 331 | 8.643085325670488706e-01 332 | 8.650325172413785291e-01 333 | 8.650325172413785291e-01 334 | 8.653945095785433583e-01 335 | 8.657565019157079655e-01 336 | 8.661184942528727948e-01 337 | 8.657565019157079655e-01 338 | 8.657565019157079655e-01 339 | 8.653945095785431363e-01 340 | 8.652135134099608882e-01 341 | 8.648515210727960589e-01 342 | 8.648515210727960589e-01 343 | 8.648515210727960589e-01 344 | 8.648515210727960589e-01 345 | 8.648515210727960589e-01 346 | 8.646705249042136998e-01 347 | 8.648515210727960589e-01 348 | 8.646705249042136998e-01 349 | 8.646705249042136998e-01 350 | 8.644895287356313407e-01 351 | 8.643085325670490926e-01 352 | 8.641275363984665114e-01 353 | 8.643085325670490926e-01 354 | 8.643085325670490926e-01 355 | 8.639465402298842633e-01 356 | 8.643085325670488706e-01 357 | 8.639465402298842633e-01 358 | 8.635845478927194341e-01 359 | 8.632225555555546048e-01 360 | 8.634035517241370750e-01 361 | 8.632225555555546048e-01 362 | 8.632225555555548269e-01 363 | 8.632225555555546048e-01 364 | 8.632225555555546048e-01 365 | 8.630415593869724677e-01 366 | 8.630415593869724677e-01 367 | 8.630415593869724677e-01 368 | 8.632225555555546048e-01 369 | 8.634035517241370750e-01 370 | 8.635845478927194341e-01 371 | 8.639465402298842633e-01 372 | 8.644895287356313407e-01 373 | 8.648515210727960589e-01 374 | 8.655755057471254954e-01 375 | 8.662994904214551539e-01 376 | 8.662994904214551539e-01 377 | 8.662994904214549319e-01 378 | 8.662994904214549319e-01 379 | 8.659374980842903247e-01 380 | 8.659374980842903247e-01 381 | 8.664804865900375130e-01 382 | 8.664804865900375130e-01 383 | 8.668424789272022313e-01 384 | 8.670234750957845904e-01 385 | 8.666614827586199832e-01 386 | 8.661184942528727948e-01 387 | 8.662994904214551539e-01 388 | 8.661184942528727948e-01 389 | 8.662994904214549319e-01 390 | 8.662994904214551539e-01 391 | 8.664804865900375130e-01 392 | 8.664804865900374020e-01 393 | 8.664804865900375130e-01 394 | 8.662994904214551539e-01 395 | 8.666614827586197611e-01 396 | 8.672044712643669495e-01 397 | 8.672044712643669495e-01 398 | 8.677474597701142489e-01 399 | 8.677474597701140269e-01 400 | 8.675664636015317788e-01 401 | 8.670234750957845904e-01 402 | 8.670234750957845904e-01 403 | 8.664804865900375130e-01 404 | 8.664804865900375130e-01 405 | 8.666614827586199832e-01 406 | 8.668424789272022313e-01 407 | 8.668424789272022313e-01 408 | 8.666614827586197611e-01 409 | 8.666614827586197611e-01 410 | 8.662994904214551539e-01 411 | 8.664804865900375130e-01 412 | 8.662994904214551539e-01 413 | 8.666614827586197611e-01 414 | 8.670234750957845904e-01 415 | 8.673854674329494197e-01 416 | 8.670234750957845904e-01 417 | 8.672044712643669495e-01 418 | 8.666614827586197611e-01 419 | 8.659374980842903247e-01 420 | 8.657565019157079655e-01 421 | 8.653945095785431363e-01 422 | 8.657565019157079655e-01 423 | 8.664804865900375130e-01 424 | 8.673854674329494197e-01 425 | 8.677474597701140269e-01 426 | 8.688334367816082926e-01 427 | 8.691954291187731219e-01 428 | 8.695574214559378401e-01 429 | 8.697384176245203102e-01 430 | 8.699194137931025583e-01 431 | 8.701004099616849174e-01 432 | 8.701004099616849174e-01 433 | 8.699194137931026694e-01 434 | 8.699194137931026694e-01 435 | 8.699194137931025583e-01 436 | 8.697384176245200882e-01 437 | 8.697384176245200882e-01 438 | 8.699194137931025583e-01 439 | 8.701004099616849174e-01 440 | 8.702814061302672766e-01 441 | 8.702814061302672766e-01 442 | 8.702814061302672766e-01 443 | 8.704624022988497467e-01 444 | 8.701004099616849174e-01 445 | 8.701004099616849174e-01 446 | 8.701004099616849174e-01 447 | 8.701004099616849174e-01 448 | 8.699194137931025583e-01 449 | 8.701004099616849174e-01 450 | 8.701004099616849174e-01 451 | 8.701004099616849174e-01 452 | 8.699194137931026694e-01 453 | 8.697384176245203102e-01 454 | 8.691954291187731219e-01 455 | 8.690144329501908738e-01 456 | 8.686524406130260445e-01 457 | 8.684714444444436854e-01 458 | 8.682904482758612152e-01 459 | 8.684714444444434633e-01 460 | 8.682904482758612152e-01 461 | 8.682904482758612152e-01 462 | 8.682904482758612152e-01 463 | 8.682904482758612152e-01 464 | 8.684714444444436854e-01 465 | 8.684714444444436854e-01 466 | 8.684714444444436854e-01 467 | 8.684714444444436854e-01 468 | 8.684714444444436854e-01 469 | 8.682904482758612152e-01 470 | 8.682904482758612152e-01 471 | 8.682904482758612152e-01 472 | 8.682904482758612152e-01 473 | 8.682904482758612152e-01 474 | 8.686524406130260445e-01 475 | 8.686524406130260445e-01 476 | 8.690144329501906517e-01 477 | 8.690144329501906517e-01 478 | 8.690144329501906517e-01 479 | 8.686524406130260445e-01 480 | 8.686524406130260445e-01 481 | 8.682904482758612152e-01 482 | 8.682904482758612152e-01 483 | 8.682904482758612152e-01 484 | 8.682904482758612152e-01 485 | 8.682904482758612152e-01 486 | 8.682904482758612152e-01 487 | 8.682904482758612152e-01 488 | 8.682904482758612152e-01 489 | 8.686524406130260445e-01 490 | 8.686524406130258225e-01 491 | 8.686524406130260445e-01 492 | 8.686524406130260445e-01 493 | 8.686524406130260445e-01 494 | 8.682904482758612152e-01 495 | 8.684714444444436854e-01 496 | 8.686524406130260445e-01 497 | 8.688334367816084036e-01 498 | 8.690144329501906517e-01 499 | 8.690144329501906517e-01 500 | 8.688334367816084036e-01 501 | -------------------------------------------------------------------------------- /opt_weight/qerror.txt: -------------------------------------------------------------------------------- 1 | 0.038 2 | 0.097 3 | 0.265 4 | 0.534 5 | 0.6487 -------------------------------------------------------------------------------- /opt_weight/reward.txt: -------------------------------------------------------------------------------- 1 | -5882343.379201 2 | -5602683.283127 3 | -5578671.158516 4 | -5772002.832691 5 | -6173285.203756 -------------------------------------------------------------------------------- /opt_weight/traintime.txt: -------------------------------------------------------------------------------- 1 | 10527.925 2 | 11006.8739 3 | 11157.3423 4 | 11464.573 5 | 11695.3744 -------------------------------------------------------------------------------- /options.py: -------------------------------------------------------------------------------- 1 | import argparse # 引入模块 2 | 3 | def args_parser(): 4 | parser = argparse.ArgumentParser( 5 | description='Gradient Quantization Samples') 6 | parser.add_argument('--network', type=str, default='fcn') 7 | parser.add_argument('--dataset', type=str, default='mnist') 8 | parser.add_argument('--num-classes', type=int, default=10) 9 | parser.add_argument('--quantizer', type=str, default='qsgd') 10 | parser.add_argument('--mode', type=str, default='ps') 11 | parser.add_argument('--scale', type=str, default="exp") 12 | 13 | parser.add_argument('--c-dim', type=int, default=128) 14 | parser.add_argument('--k-bit', type=int, default=8) 15 | parser.add_argument('--n-bit', type=int, default=2) 16 | parser.add_argument('--cr', type=int, default=256) 17 | parser.add_argument('--random', type=int, default=True) 18 | 19 | parser.add_argument('--num-users', type=int, default=4, metavar='N', 20 | help='num of users for training (default: 8)') 21 | parser.add_argument('--logdir', type=str, default='logs_qsgd', 22 | help='For Saving the logs') 23 | parser.add_argument('--batch-size', type=int, default=128, metavar='N', 24 | help='input batch size for training (default: 32)') 25 | parser.add_argument('--test-batch-size', type=int, default=1000, metavar='N', 26 | help='input batch size for testing (default: 1000)') 27 | parser.add_argument('--epochs', type=int, default=350, metavar='N', 28 | help='number of epochs to train (default: 350)') 29 | 30 | parser.add_argument('--momentum', type=float, default=0.9, metavar='M', 31 | help='SGD momentum (default: 0.9)') 32 | parser.add_argument('--weight-decay', type=float, default=5e-4, metavar='M', 33 | help='weight decay momentum (default: 5e-4)') 34 | parser.add_argument('--no-cuda', action='store_true', default=False, 35 | help='disables CUDA training') 36 | parser.add_argument('--ef', action='store_true', default=False, 37 | help='enable error feedback') 38 | parser.add_argument('--seed', type=int, default=1, metavar='S', 39 | help='random seed (default: 1)') 40 | parser.add_argument('--log-epoch', type=int, default=1, metavar='N', 41 | help='logging training status at each epoch') 42 | parser.add_argument('--save-model', action='store_true', default=False, 43 | help='For Saving the current Model') 44 | parser.add_argument('--two-phase', action='store_true', default=False, 45 | help='For Compression two phases') 46 | args = parser.parse_args() # 属性给予args实例 47 | return args 48 | -------------------------------------------------------------------------------- /pe.py: -------------------------------------------------------------------------------- 1 | import os 2 | os.environ["KMP_DUPLICATE_LIB_OK"]="TRUE" 3 | # os.environ['CUDA_VISIBLE_DEVICES'] = '-1' 4 | import numpy as np 5 | import ipdb as pdb 6 | import matplotlib.pyplot as plt 7 | 8 | plt.rcParams['font.sans-serif']=['SimSun'] #显示中文 9 | plt.rcParams['axes.unicode_minus'] = False 10 | plt.rc("font",family="SimSun",size="15") 11 | 12 | 13 | def p_e(): 14 | 15 | dir = 'test_results/' 16 | a = np.loadtxt(dir + 'longtermreward1.txt') 17 | 18 | name = ["Method"] 19 | x = np.arange(len(name)) 20 | width = 0.25 21 | fig = plt.figure() 22 | plt.tick_params(bottom=False) 23 | plt.bar(x + width, a[0], width=width, label='固定 (10-比特)', color='#CC6600', tick_label="") 24 | plt.bar(x + 3 * width, a[1], width=width, label='固定 (6-比特)', color='#66CCFF') 25 | # plt.plot(tick_label="") 26 | plt.bar(x + 2 * width, a[2], width=width, label='自适应-梯度量化', color='m') 27 | plt.bar(x, a[3], width=width, label='DQN-梯度量化', color='r') 28 | plt.xticks() 29 | plt.ylabel('长期折扣奖励') 30 | plt.xlabel('方案') 31 | plt.grid(axis='y', linestyle=':') 32 | plt.legend() 33 | plt.show() 34 | 35 | b = np.loadtxt(dir + 'qerror1.txt') 36 | 37 | width = 0.25 38 | fig = plt.figure() 39 | plt.tick_params(bottom=False) 40 | plt.bar(x + width, b[0], width=width, label='固定 (10-比特)', color='#CC6600', tick_label="") 41 | plt.bar(x + 3 * width, b[1], width=width, label='固定 (6-比特)', color='#66CCFF') 42 | # plt.plot(tick_label="") 43 | plt.bar(x + 2 * width, b[2], width=width, label='自适应-梯度量化', color='m') 44 | plt.bar(x, b[3], width=width, label='DQN-梯度量化', color='r') 45 | plt.xticks() 46 | plt.ylim(0, 1) 47 | plt.ylabel('平均量化误差') 48 | plt.xlabel('方案') 49 | plt.grid(axis='y', linestyle=':') 50 | plt.legend() 51 | # plt.subplots_adjust(left=0.15) 52 | plt.show() 53 | 54 | c = np.loadtxt(dir + 'traintime1.txt') 55 | 56 | name = ["Method"] 57 | x = np.arange(len(name)) 58 | width = 0.25 59 | fig = plt.figure() 60 | plt.tick_params(bottom=False) 61 | plt.bar(x + width, c[0], width=width, label='固定 (10-比特)', color='#CC6600', tick_label="") 62 | plt.bar(x + 3 * width, c[1], width=width, label='固定 (6-比特)', color='#66CCFF') 63 | # plt.plot(tick_label="") 64 | plt.bar(x + 2 * width, c[2], width=width, label='自适应-梯度量化', color='m') 65 | plt.bar(x, c[3], width=width, label='DQN-梯度量化', color='r') 66 | plt.xticks() 67 | 68 | plt.ylabel('平均总训练时间') 69 | plt.xlabel('方案') 70 | plt.grid(axis='y', linestyle=':') 71 | plt.legend() 72 | # plt.subplots_adjust(left=0.15) 73 | plt.show() 74 | 75 | fileList = os.listdir(dir + 'loss/') 76 | fileList = [name for name in fileList if '.txt' in name] 77 | d = [np.loadtxt(dir + 'loss/' + i) for i in fileList] 78 | 79 | plt.plot(range(len(d[0])), d[0], label='自适应-梯度量化', linestyle=(0, (1, 2, 3)), color='m', lw=1.5) 80 | plt.plot(range(len(d[1])), d[1], label='DQN-梯度量化', linestyle='-', color='r', lw=1.5) 81 | plt.plot(range(len(d[3])), d[3], label='固定 (2-比特)', linestyle='-.', color='g', lw=1.5) 82 | plt.plot(range(len(d[4])), d[4], label='固定 (6-比特)', linestyle='--', color='#66CCFF', lw=1.5) 83 | plt.plot(range(len(d[2])), d[2], label='固定 (10-比特)', linestyle=':', color='#CC6600', lw=1.5) 84 | plt.legend() 85 | plt.grid(linestyle=':') 86 | plt.legend() 87 | plt.ylabel("训练损失") 88 | plt.xlabel("通信轮次") 89 | plt.show() 90 | 91 | fileList1 = os.listdir(dir + 'acc/') 92 | fileList1 = [name for name in fileList if '.txt' in name] 93 | e = [np.loadtxt(dir + 'acc/' + j) for j in fileList1] 94 | 95 | plt.plot(range(len(e[0])), e[0], label='自适应-梯度量化', linestyle=(0, (1, 2, 3)), color='m', lw=1.5) 96 | plt.plot(range(len(e[1])), e[1], label='DQN-梯度量化', linestyle='-', color='r', lw=1.5) 97 | plt.plot(range(len(e[3])), e[3], label='固定 (2-比特)', linestyle='-.', color='g', lw=1.5) 98 | plt.plot(range(len(e[4])), e[4], label='固定 (6-比特)', linestyle='--', color='#66CCFF', lw=1.5) 99 | plt.plot(range(len(e[2])), e[2], label='固定 (10-比特)', linestyle=':', color='#CC6600', lw=1.5) 100 | plt.legend() 101 | plt.grid(linestyle=':') 102 | plt.legend() 103 | plt.ylabel("测试精度") 104 | plt.xlabel("通信轮次") 105 | plt.show() 106 | 107 | fix_10 = np.loadtxt(dir + 'fix_10_time.txt') 108 | fix_6 = np.loadtxt(dir + 'fix_6_time.txt') 109 | ada = np.loadtxt(dir + 'ada_time.txt') 110 | dqn = np.loadtxt(dir + 'dqn_q_time.txt') 111 | x = [4, 6, 8, 10, 12] 112 | my_x_ticks = np.arange(4, 13, 2) # 原始数据有13个点，故此处为设置从0开始，间隔为1 113 | plt.xticks(my_x_ticks) 114 | plt.plot(x, fix_10, label='固定 (10-比特)', marker='s', color='#CC6600') 115 | plt.plot(x, fix_6, label='固定 (6-比特)', marker='d', color='#66CCFF') 116 | plt.plot(x, ada, label='自适应-梯度量化', marker='*', color='m') 117 | plt.plot(x, dqn, label='DQN-梯度量化', marker='o', color='r') 118 | plt.subplots_adjust(left=0.15) 119 | plt.subplots_adjust(bottom=0.13) 120 | plt.legend() 121 | plt.grid(linestyle=':') 122 | plt.legend() 123 | plt.ylabel("总训练时间 (s)") 124 | plt.xlabel("参与车辆数") 125 | plt.show() 126 | 127 | error_10 = np.loadtxt(dir + 'error10.txt') 128 | error_6 = np.loadtxt(dir + 'error6.txt') 129 | ada = np.loadtxt(dir + 'error_ada.txt') 130 | dqn = np.loadtxt(dir + 'error_dqn.txt') 131 | x = [4, 6, 8, 10, 12] 132 | my_x_ticks = np.arange(4, 13, 2) # 原始数据有13个点，故此处为设置从0开始，间隔为1 133 | plt.xticks(my_x_ticks) 134 | plt.plot(x, error_10, label='固定 (10-比特)', marker='s', color='#CC6600') 135 | plt.plot(x, error_6, label='固定 (6-比特)', marker='d', color='#66CCFF') 136 | plt.plot(x, ada, label='自适应-梯度量化', marker='*', color='m') 137 | plt.plot(x, dqn, label='DQN-梯度量化', marker='o', color='r') 138 | plt.xticks() 139 | plt.ylim(0, 1) 140 | plt.legend(loc=1) 141 | plt.grid(linestyle=':') 142 | plt.ylabel("量化误差") 143 | plt.xlabel("参与车辆数") 144 | plt.subplots_adjust(bottom=0.15) 145 | plt.show() 146 | 147 | reward_10 = np.loadtxt(dir + 'reward10.txt') 148 | reward_6 = np.loadtxt(dir + 'reward6.txt') 149 | ada = np.loadtxt(dir + 'rewardada.txt') 150 | dqn = np.loadtxt(dir + 'rewarddqn.txt') 151 | x = [4, 6, 8, 10, 12] 152 | my_x_ticks = np.arange(4, 13, 2) # 原始数据有13个点，故此处为设置从0开始，间隔为1 153 | 154 | plt.xticks(my_x_ticks) 155 | 156 | plt.plot(x, reward_10, label='固定 (10-比特)', marker='s', color='#CC6600') 157 | plt.plot(x, reward_6, label='固定 (6-比特)', marker='d', color='#66CCFF') 158 | plt.plot(x, ada, label='自适应-梯度量化', marker='*', color='m') 159 | plt.plot(x, dqn, label='DQN-梯度量化', marker='o', color='r') 160 | plt.ylabel('长期折扣奖励') 161 | plt.xlabel('参与车辆数') 162 | plt.grid(linestyle=':') 163 | plt.legend() 164 | # plt.subplots_adjust(bottom=0.15) 165 | plt.show() 166 | 167 | 168 | 169 | 170 | 171 | 172 | 173 | 174 | -------------------------------------------------------------------------------- /plot_reward.py: -------------------------------------------------------------------------------- 1 | import os 2 | os.environ["KMP_DUPLICATE_LIB_OK"]="TRUE" 3 | # os.environ['CUDA_VISIBLE_DEVICES'] = '-1' 4 | import numpy as np 5 | import ipdb as pdb 6 | import matplotlib.pyplot as plt 7 | 8 | 9 | def moving_average(a, n=10): 10 | ret = np.cumsum(a, dtype=float, axis=0) 11 | ret[n:] = ret[n:] - ret[:-n] 12 | return ret[n - 1:] / n 13 | 14 | def output_avg(dir): 15 | dir_path = dir 16 | fileList = os.listdir(dir_path) 17 | fileList = [name for name in fileList if '.npz' in name] 18 | avg_rs = [] 19 | for name in fileList[0:3]: 20 | path = dir_path + name 21 | np.load.__defaults__=(None, True, True, 'ASCII') 22 | res = np.load(path) 23 | np.load.__defaults__=(None, False, True, 'ASCII') 24 | temp_rs = np.array(res['arr_0']) 25 | avg_rs.append(temp_rs) 26 | avg_rs = moving_average(np.mean(avg_rs, axis=0, keepdims=True)[0],20) 27 | return avg_rs 28 | 29 | test_acc = output_avg('train/') 30 | test1 = test_acc.tolist() 31 | test = [i/1000 for i in test1] 32 | plt.plot(range(len(test1)), test1, label='reward', color='orange') 33 | #plt.legend() 34 | 35 | plt.grid(linestyle=':') 36 | plt.legend() 37 | plt.ylabel("reward") 38 | plt.xlabel("Episodes") 39 | plt.show() -------------------------------------------------------------------------------- /ps_quantizer.py: -------------------------------------------------------------------------------- 1 | import math 2 | import torch 3 | from identical_compressor import IdenticalCompressor 4 | 5 | 6 | class Quantizer(): 7 | def __init__(self, Compressor, parameters, args, level): 8 | self.parameters = list(parameters) # 模型权重 9 | self.num_layers = len(self.parameters) # 网络层数 10 | self.compressors = list() # 初始化量化器 11 | self.compressed_gradients = [list() for _ in range(self.num_layers)] # 压缩后的梯度 12 | self.args = args # 参数 13 | self.error_feedback = args.ef # False 14 | self.two_phase = self.args.two_phase # False 15 | for param in self.parameters: # 对每层的权重 16 | param_size = param.flatten().shape[0] # 权重降维后的尺寸 17 | self.compressors.append( 18 | Compressor(param_size, param.shape, args, level) if param_size > 1000 # 添加量化器 19 | else IdenticalCompressor() 20 | ) 21 | if self.error_feedback: 22 | param.error = [torch.zeros_like(param) 23 | for _ in range(args.num_users)] 24 | if self.error_feedback and self.two_phase: 25 | param.server_error = torch.zeros_like(param) 26 | 27 | def record(self, user, epoch): 28 | if self.args.scale == 'exp': 29 | scale = (2 / (math.exp(-epoch) + 1) - 1) 30 | else: 31 | scale = float(self.args.scale) 32 | 33 | for i, param in enumerate(self.parameters): 34 | if self.error_feedback: 35 | param.grad.data.add_(scale * param.error[user]) 36 | decompressed_g = self.compressors[i].decompress( 37 | self.compressors[i].compress(param.grad.data) 38 | ) 39 | param.error[user].data = param.grad.data - decompressed_g 40 | else: 41 | decompressed_g = self.compressors[i].decompress( 42 | self.compressors[i].compress(param.grad.data) 43 | ) 44 | self.compressed_gradients[i].append(decompressed_g) 45 | 46 | def apply(self): 47 | for i, param in enumerate(self.parameters): 48 | g = torch.stack(self.compressed_gradients[i], dim=0).mean(dim=0) 49 | 50 | # if compress gradient on two phase, i.e., 51 | # compress the sum of decompressed gradient 52 | if self.two_phase: 53 | if self.error_feedback: 54 | g.add_(param.server_error) 55 | decompressed_g = self.compressors[i].decompress( 56 | self.compressors[i].compress(g)) 57 | param.server_error = g - decompressed_g 58 | g = decompressed_g 59 | else: 60 | g = self.compressors[i].decompress( 61 | self.compressors[i].compress(g)) 62 | 63 | param.grad.data = g 64 | for compressed in self.compressed_gradients: 65 | compressed.clear() 66 | -------------------------------------------------------------------------------- /q_fl_dqn.py: -------------------------------------------------------------------------------- 1 | # FL相关 2 | import tflearn 3 | 4 | 5 | from qsgd_compressor import * 6 | from datetime import datetime 7 | from identical_compressor import * 8 | import torch.nn as nn 9 | import torch.optim as optim 10 | import time 11 | from fcn import FCN 12 | from resnet import * 13 | from dataloaders import mnist 14 | from ps_quantizer import * 15 | import os 16 | from logger import Logger 17 | 18 | from mec_env import MecTerm, MecTermDQN, MecSvrEnv 19 | 20 | from pe import * 21 | from helper import * 22 | from agent import * 23 | import tensorflow as tf 24 | import matplotlib 25 | import matplotlib.pyplot as plt 26 | 27 | from options import * 28 | 29 | 30 | # DDPG相关 31 | os.environ['CUDA_VISIBLE_DEVICES'] = '-1' 32 | 33 | # FL相关初始化 34 | os.environ['KMP_DUPLICATE_LIB_OK'] = 'True' 35 | 36 | # DQN相关设置 37 | tf.compat.v1.reset_default_graph() 38 | MAX_EPISODE = 2001 39 | MAX_EPISODE_LEN = 1001 40 | NUM_R = args.num_users #天线数 跟信道数有关 41 | 42 | SIGMA2 = 1e-9 # 噪声平方 10 -9 43 | 44 | config = {'state_dim': 4, 'action_dim': 2}; 45 | train_config = {'minibatch_size': 64, 46 | 'actor_lr': 0.0001, 47 | 'tau': 0.001, 48 | 'critic_lr': 0.05, # 0.001 49 | 'gamma': 0.99, 50 | 'buffer_size': 250000, # 250000 51 | 'random_seed': int(time.perf_counter() * 1000 % 1000), 52 | 'sigma2': SIGMA2, 53 | 'epsilon': 0.5} # 0.5 54 | 55 | IS_TRAIN = False 56 | 57 | res_path = 'train/' 58 | model_fold = 'model/' 59 | model_path = 'model/train_model-q' 60 | 61 | # 创建目录,用来保存dqn模型 62 | if not os.path.exists(res_path): 63 | os.mkdir(res_path) 64 | if not os.path.exists(model_fold): 65 | os.mkdir(model_fold) 66 | 67 | init_path = '' 68 | 69 | # choose the vehicle for training 70 | Train_vehicle_ID = 1 71 | 72 | # action_bound是需要后面调的 73 | user_config = [{'id': '1', 'model': 'AR', 'num_r': NUM_R, 'action_bound': 1}] 74 | 75 | # 0. initialize the session object 76 | sess = tf.compat.v1.Session() 77 | 78 | # 1. include all user in the system according to the user_config 79 | user_list = []; 80 | for info in user_config: 81 | info.update(config) 82 | info['model_path'] = model_path 83 | info['meta_path'] = info['model_path'] + '.meta' 84 | info['init_path'] = init_path 85 | info['action_level'] = 5 86 | user_list.append(MecTermDQN(sess, info, train_config)) 87 | print('Initialization OK!----> user ') 88 | 89 | # 2. create the simulation env 90 | env = MecSvrEnv(user_list, Train_vehicle_ID, SIGMA2, MAX_EPISODE_LEN) 91 | 92 | sess.run(tf.compat.v1.global_variables_initializer()) 93 | 94 | tflearn.config.is_training(is_training=IS_TRAIN, session=sess) 95 | 96 | env.init_target_network() 97 | 98 | res_r = [] # 每一步的平均奖励 99 | res_p = [] # 每个回合的平均动作 100 | 101 | args = args_parser() 102 | if args.logdir is None: 103 | assert False, "The logdir is not defined" 104 | LOGGER = Logger(args.logdir) 105 | 106 | args.no_cuda = args.no_cuda or not torch.cuda.is_available() 107 | 108 | torch.manual_seed(args.seed) 109 | device = torch.device("cpu" if args.no_cuda else "cuda") 110 | 111 | #train_loader, test_loader = mnist(args) # 235 10 112 | #model = FCN(num_classes=args.num_classes).to(device) 113 | #optimizer = optim.SGD(model.parameters(), lr=0.1, 114 | # momentum=args.momentum, weight_decay=args.weight_decay) 115 | ave_reward_ep = [] 116 | ave_ground_ep = [] 117 | ave_T_ep = [] 118 | ave_delay_ep = [] 119 | ave_error_ep = [] 120 | reward_step = [] 121 | train_loss, testing_accuracy = [], [] 122 | maxs = [] 123 | mins = [] 124 | # 开始训练DQN episode 125 | for episode in range(1, MAX_EPISODE): 126 | print(f'\n | Global Training Round/episode : {episode} |\n') 127 | #model.train() 128 | #batch_size = args.batch_size # 32 129 | #num_users = args.num_users # 8 130 | #train_data = list() 131 | #iteration = len(train_loader.dataset)//(num_users*batch_size) + \ 132 | # int(len(train_loader.dataset) % (num_users*batch_size) != 0) # 235 133 | # 记录间隔 # [23, 46, 69, 92, 115, 138, 161, 184, 207, 230] 134 | #log_interval = [iteration // args.log_epoch * (i+1) for i in range(args.log_epoch)] 135 | # Training 136 | 137 | tr_step_loss = [] 138 | tr_step_acc = [] 139 | val_acc_list, net_list = [], [] 140 | cv_loss, cv_acc = [], [] 141 | print_every = 2 142 | val_loss_pre, counter = 0, 0 143 | 144 | # DQN相关 145 | plt.ion() # 打开交互模式,画动态图 146 | cur_init_ds_ep = env.reset() # 环境重置 147 | cur_r_ep = 0 # 一个回合的总奖励 148 | cur_g_ep = 0 149 | cur_T_ep = 0 150 | cur_d_ep = 0 151 | cur_e_ep = 0 152 | step_cur_r_ep = [] 153 | count = 0 154 | # here the real batch size is (num_users * batch_size) 155 | # DQN时间步 & FL的一个通信轮次 156 | for epoch in range(1, MAX_EPISODE_LEN): 157 | i = Train_vehicle_ID - 1 158 | #if epoch == 1: 159 | #q_level1 = 1 160 | #if epoch % 5 == 0: 161 | actions = user_list[i].predict(True) 162 | print('step is : {}, q_level is: {}, power is: {}'.format(epoch,actions[0],actions[1])) 163 | #quantizer = Quantizer(QSGDCompressor, model.parameters(), args, actions[0]) 164 | #para = list(model.parameters()) 165 | #ini_para = para 166 | # FL本地训练迭代 167 | #for batch_idx, (data, target) in enumerate(train_loader): 168 | # user_batch_size = len(data) // num_users # 32 = 256//8 169 | # train_data.clear() 170 | # 给每个用户分配训练数据 171 | # for user_id in range(num_users-1): 172 | # train_data.append((data[user_id*user_batch_size:(user_id+1)*user_batch_size], 173 | # target[user_id*user_batch_size:(user_id+1)*user_batch_size])) 174 | # train_data.append((data[(num_users-1)*user_batch_size:], 175 | # target[(num_users-1)*user_batch_size:])) 176 | 177 | # 计算一次迭代的全局损失 178 | # loss = one_iter(model, device, LOSS_FUNC, optimizer, 179 | # quantizer, train_data, num_users, epoch=epoch) 180 | 181 | # 记录一个epoch的2个损失和测试精度 182 | # if (batch_idx+1) in log_interval: 183 | # train_loss.append(loss.item()) 184 | # test_accuracy = test(args, model, device, test_loader) 185 | # testing_accuracy.append(test_accuracy*100) 186 | #print('Train Episode: {} Train epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}\t Test Accuracy: {:.2f}%'.format( 187 | # episode, 188 | # epoch, 189 | # batch_idx * num_users * batch_size + len(data), 190 | # len(train_loader.dataset), 191 | # 100. * batch_idx / len(train_loader), 192 | # loss.item(), 193 | # test_accuracy*100)) 194 | 195 | # info = {'loss': loss.item(), 'accuracy(%)': test_accuracy*100} 196 | 197 | # for tag, value in info.items(): 198 | # LOGGER.scalar_summary( 199 | # tag, value, iteration*(epoch-1)+batch_idx) 200 | #para1 = list(model.parameters()) 201 | #maxx = max(para1[0].detach().numpy().reshape(1,200704).flatten()) 202 | #minn = min(abs(para1[0].detach().numpy().reshape(1,200704).flatten())) 203 | #maxs.append(maxx) 204 | #mins.append(minn) 205 | 206 | #print('Train Epoch: {} Done.\tLoss: {:.6f}'.format(epoch, loss.item())) 207 | 208 | reward = 0 209 | trs = 0 210 | deltas = 0 211 | diss = 0 212 | mini_ground = 0 213 | T_total = 0 214 | 215 | max_len = 1000 # MAX_EPISODE_LEN 216 | count += 1 217 | 218 | i = Train_vehicle_ID - 1 219 | # feedback the sinr to each user 220 | [reward, mini_ground, T_total, trs, deltas, diss, q_level, power, delay, q_error] = user_list[i].feedback(actions[0], actions[1]) 221 | print("reward is: {}, delay is: {}".format(reward,delay)) 222 | 223 | reward_step.append(reward) 224 | 225 | user_list[i].AgentUpdate(count >= max_len) # 训练数据个数逐渐增加 大于buffer的大小时 进行更新agent 226 | cur_r = reward 227 | cur_mg = mini_ground 228 | cur_T = T_total 229 | cur_q = q_level 230 | cur_d = delay 231 | cur_e = q_error 232 | done = count >= max_len # max_len即为MAX_EPISODE_LEN 233 | 234 | cur_r_ep += cur_r # 一个回合的总奖励 所有step的奖励之和 235 | cur_g_ep += cur_mg 236 | cur_T_ep += cur_T 237 | cur_d_ep += cur_d 238 | cur_e_ep += cur_e 239 | # for m in range(args.num_users): 240 | # cur_p_ep[m] += cur_p[m] 241 | 242 | # if done: # 一个episode结束 243 | # res_r.append(cur_r_ep / MAX_EPISODE_LEN) # 后面为了存储进模型 每一步的平均奖励 244 | 245 | # cur_p_ep1 = [0] * args.num_users 246 | # for m in range(args.num_users): 247 | # cur_p_ep1[m] = cur_p_ep[m] / MAX_EPISODE_LEN # 一个回合里平均每一个step的动作 248 | # res_p.append(cur_p_ep1) # 用来存储每个回合的平均动作 249 | 250 | print("episode = ", episode) 251 | print("r = ", cur_r_ep / MAX_EPISODE_LEN) 252 | print("delay = ", cur_d_ep / MAX_EPISODE_LEN) 253 | print("error = ", cur_e_ep / MAX_EPISODE_LEN) 254 | ave_reward_ep.append(cur_r_ep / MAX_EPISODE_LEN) 255 | ave_ground_ep.append(cur_g_ep / MAX_EPISODE_LEN) 256 | ave_T_ep.append(cur_T_ep / MAX_EPISODE_LEN) 257 | ave_delay_ep.append(cur_d_ep / MAX_EPISODE_LEN) 258 | ave_error_ep.append(cur_e_ep / MAX_EPISODE_LEN) 259 | # print("p_lambda = ", cur_p_ep1) # this is wrong 260 | # print("cur_r_ep = ", cur_r_ep) 261 | 262 | # line_reward = ax.plot(range(0, MAX_EPISODE), cur_r_ep, '#ff7f0e', label='reward', lw=1) 263 | # line_pro = ax2.plot(range(0, MAX_EPISODE), step_cur_r_ep[0], '#ff7f0e', label='第1辆车的the probility(选择每辆车的概率，即动作输出)', lw=1) 264 | plt.ioff() 265 | #upper = max(maxs) 266 | #lower = min(mins) 267 | #print("upper is", upper) 268 | #print("lower is", lower) 269 | # DDPG模型保存 270 | name = res_path + 'a1bathnormal_env_max_ep2000_tep_1000_buffersize250000_maxstep500000_prestep25000_lr0.05_plot_minig+T_tfacter0.5_num4' 271 | #np.savez(name, train_loss, testing_accuracy, ave_reward_ep, ave_delay_ep, ave_error_ep) # 保存平均每一步的q_level和奖励 # 为了后面的画图 272 | np.savez(name, ave_reward_ep, ave_ground_ep, ave_T_ep, ave_delay_ep, ave_error_ep) 273 | 274 | tflearn.config.is_training(is_training=False, session=sess) 275 | # Create a saver object which will save all the variables # 即保存模型参数 276 | saver = tf.compat.v1.train.Saver() 277 | saver.save(sess, model_path) 278 | sess.close() 279 | 280 | #if args.save_model: 281 | # filename = "saved_{}_{}.pt".format(args.network, datetime.now()) 282 | # torch.save(model.state_dict(), filename) 283 | 284 | matplotlib.use('TkAgg') 285 | os.environ["KMP_DUPLICATE_LIB_OK"]="TRUE" 286 | 287 | ''' 288 | plt.figure() 289 | plt.plot(range(len(ave_reward_ep)), ave_reward_ep, color='b') 290 | plt.ylabel('Training reward') 291 | plt.xlabel('Episode') 292 | plt.rcParams['axes.unicode_minus']=False 293 | plt.show() 294 | # plt.savefig('train_reward.png') 295 | 296 | plt.figure() 297 | # plt.title('Training loss') 298 | plt.plot(range(len(train_loss)), train_loss, color='b') 299 | plt.ylabel('Training loss') 300 | plt.xlabel('Episode') 301 | plt.show() 302 | # plt.savefig('train_loss.png') 303 | 304 | plt.figure() 305 | # plt.title('Training loss') 306 | plt.plot(range(len(train_loss)), train_loss, color='b') 307 | plt.ylabel('Training loss') 308 | plt.xlabel('Episode') 309 | plt.show() 310 | # plt.savefig('train_loss.png') 311 | 312 | plt.figure() 313 | # plt.title('Test Accuracy') 314 | plt.plot(range(len(ave_error_ep)), ave_error_ep, color='r') 315 | plt.ylabel('Average error') 316 | plt.xlabel('Episode') 317 | plt.show() 318 | 319 | plt.figure() 320 | # plt.title('Test Accuracy') 321 | plt.plot(range(len(ave_delay_ep)), ave_delay_ep, color='r') 322 | plt.ylabel('Average delay') 323 | plt.xlabel('Episode') 324 | plt.show() 325 | ''' 326 | 327 | ''' 328 | plt.figure() 329 | # plt.title('Test Accuracy') 330 | plt.plot(range(len(testing_accuracy)), testing_accuracy, color='r') 331 | plt.ylabel('Testing Accuracy') 332 | plt.xlabel('Episode') 333 | plt.show() 334 | # plt.savefig('train_accuracy.png') 335 | 336 | plt.figure() 337 | # plt.title('Test Accuracy') 338 | plt.plot(range(len(ave_delay_ep)), ave_delay_ep, color='r') 339 | plt.ylabel('Average delay') 340 | plt.xlabel('Episode') 341 | plt.show() 342 | 343 | plt.figure() 344 | # plt.title('Test Accuracy') 345 | plt.plot(range(len(ave_error_ep)), ave_error_ep, color='r') 346 | plt.ylabel('Average error') 347 | plt.xlabel('Episode') 348 | plt.show() 349 | ''' 350 | p_e() 351 | 352 | -------------------------------------------------------------------------------- /qsgd_compressor.py: -------------------------------------------------------------------------------- 1 | import torch 2 | 3 | 4 | class QSGDCompressor(object): 5 | def __init__(self, size, shape, args, q_level): 6 | self.random = args.random 7 | self.bit = q_level # 量化电平(动作) 8 | c_dim = args.c_dim # 桶? 9 | assert self.bit > 0 # 量化电平大于零 10 | 11 | self.cuda = not args.no_cuda # False 12 | self.s = 2 ** self.bit 13 | self.size = size # 权重的尺寸 14 | self.shape = shape # 权重的形状 15 | 16 | if c_dim == 0 or self.size < args.c_dim: 17 | self.dim = self.size 18 | else: 19 | self.dim = c_dim # 桶大小 20 | for i in range(0, 10): 21 | if size % self.dim != 0: 22 | self.dim = self.dim // 2 * 3 23 | 24 | if c_dim != self.dim: 25 | print("alternate dimension form" 26 | " {} to {}, size {} shape {}" 27 | .format(c_dim, self.dim, size, shape)) 28 | 29 | assert self.dim != 0, \ 30 | "0 sub dimension size {} " \ 31 | "c_dim {} self.dim {}"\ 32 | .format(size, c_dim, self.dim) 33 | assert size % self.dim == 0, \ 34 | "not divisible size {} " \ 35 | " c_dim {} self.dim {}"\ 36 | .format(size, c_dim, self.dim) 37 | 38 | self.M = size // self.dim 39 | self.code_dtype = torch.int32 40 | 41 | 42 | def compress(self, vec): 43 | """ 44 | :param vec: torch tensor 45 | :return: norm, signs, quantized_intervals 46 | """ 47 | vec = vec.view(-1, self.dim) 48 | # norm = torch.norm(vec, dim=1, keepdim=True) 49 | norm = torch.max(torch.abs(vec), dim=1, keepdim=True)[0] 50 | normalized_vec = vec / norm 51 | 52 | scaled_vec = torch.abs(normalized_vec) * self.s 53 | l = torch.clamp(scaled_vec, 0, self.s-1).type(self.code_dtype) 54 | 55 | if self.random: 56 | # l[i] <- l[i] + 1 with probability |v_i| / ||v|| * s - l 57 | probabilities = scaled_vec - l.type(torch.float32) 58 | r = torch.rand(l.size()) 59 | if self.cuda: 60 | r = r.cuda() 61 | l[:] += (probabilities > r).type(self.code_dtype) 62 | 63 | signs = torch.sign(vec) > 0 64 | return [norm, signs.view(self.shape), l.view(self.shape)] 65 | 66 | def decompress(self, signature): 67 | [norm, signs, l] = signature 68 | assert l.shape == signs.shape 69 | scaled_vec = l.type(torch.float32) * (2 * signs.type(torch.float32) - 1) 70 | compressed = (scaled_vec.view((-1, self.dim))) * norm / self.s 71 | return compressed.view(self.shape) 72 | -------------------------------------------------------------------------------- /resnet.py: -------------------------------------------------------------------------------- 1 | '''ResNet-18 Image classfication for cifar-10 with PyTorch 2 | 3 | Author 'Sun-qian'. 4 | https://blog.csdn.net/sunqiande88/article/details/80100891 5 | ''' 6 | import torch 7 | import torch.nn as nn 8 | import torch.nn.functional as F 9 | 10 | 11 | class ResidualBlock(nn.Module): 12 | expansion = 1 13 | 14 | def __init__(self, inchannel, outchannel, stride=1): 15 | super(ResidualBlock, self).__init__() 16 | self.left = nn.Sequential( 17 | nn.Conv2d(inchannel, outchannel, kernel_size=3, 18 | stride=stride, padding=1, bias=False), 19 | nn.BatchNorm2d(outchannel), 20 | nn.ReLU(inplace=True), 21 | nn.Conv2d(outchannel, outchannel, kernel_size=3, 22 | stride=1, padding=1, bias=False), 23 | nn.BatchNorm2d(outchannel) 24 | ) 25 | self.shortcut = nn.Sequential() 26 | if stride != 1 or inchannel != outchannel: 27 | self.shortcut = nn.Sequential( 28 | nn.Conv2d(inchannel, outchannel, kernel_size=1, 29 | stride=stride, bias=False), 30 | nn.BatchNorm2d(outchannel) 31 | ) 32 | 33 | def forward(self, x): 34 | out = self.left(x) 35 | out += self.shortcut(x) 36 | out = F.relu(out) 37 | return out 38 | 39 | 40 | class Bottleneck(nn.Module): 41 | expansion = 4 42 | 43 | def __init__(self, in_planes, planes, stride=1): 44 | super(Bottleneck, self).__init__() 45 | self.conv1 = nn.Conv2d(in_planes, planes, kernel_size=1, bias=False) 46 | self.bn1 = nn.BatchNorm2d(planes) 47 | self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride, padding=1, bias=False) 48 | self.bn2 = nn.BatchNorm2d(planes) 49 | self.conv3 = nn.Conv2d(planes, self.expansion*planes, kernel_size=1, bias=False) 50 | self.bn3 = nn.BatchNorm2d(self.expansion*planes) 51 | 52 | self.shortcut = nn.Sequential() 53 | if stride != 1 or in_planes != self.expansion*planes: 54 | self.shortcut = nn.Sequential( 55 | nn.Conv2d(in_planes, self.expansion*planes, kernel_size=1, stride=stride, bias=False), 56 | nn.BatchNorm2d(self.expansion*planes) 57 | ) 58 | 59 | def forward(self, x): 60 | out = F.relu(self.bn1(self.conv1(x))) 61 | out = F.relu(self.bn2(self.conv2(out))) 62 | out = self.bn3(self.conv3(out)) 63 | out += self.shortcut(x) 64 | out = F.relu(out) 65 | return out 66 | 67 | 68 | class ResNet(nn.Module): 69 | def __init__(self, block, num_blocks, num_classes=10): 70 | super(ResNet, self).__init__() 71 | self.inchannel = 64 72 | self.conv1 = nn.Sequential( 73 | nn.Conv2d(3, 64, kernel_size=3, stride=1, padding=1, bias=False), 74 | nn.BatchNorm2d(64), 75 | nn.ReLU(), 76 | ) 77 | self.layer1 = self.make_layer(block, 64, num_blocks[0], stride=1) 78 | self.layer2 = self.make_layer(block, 128, num_blocks[1], stride=2) 79 | self.layer3 = self.make_layer(block, 256, num_blocks[2], stride=2) 80 | self.layer4 = self.make_layer(block, 512, num_blocks[3], stride=2) 81 | self.fc = nn.Linear(512 * block.expansion, num_classes) # 512 82 | 83 | def make_layer(self, block, channels, num_blocks, stride): 84 | strides = [stride] + [1] * (num_blocks - 1) # strides=[1,1] 85 | layers = [] 86 | for stride in strides: 87 | layers.append(block(self.inchannel, channels, stride)) 88 | self.inchannel = channels * block.expansion 89 | return nn.Sequential(*layers) 90 | 91 | def forward(self, x): 92 | out = self.conv1(x) 93 | out = self.layer1(out) 94 | out = self.layer2(out) 95 | out = self.layer3(out) 96 | out = self.layer4(out) 97 | out = F.avg_pool2d(out, 4) # 4 98 | out = out.view(out.size(0), -1) 99 | out = self.fc(out) 100 | return out 101 | 102 | def ResNet18(**kwargs): 103 | return ResNet(ResidualBlock, [1,1,1,1], **kwargs) 104 | 105 | def ResNet34(**kwargs): 106 | return ResNet(ResidualBlock, [3,4,6,3], **kwargs) 107 | 108 | def ResNet50(**kwargs): 109 | return ResNet(Bottleneck, [3,4,6,3], **kwargs) 110 | 111 | def ResNet101(**kwargs): 112 | return ResNet(Bottleneck, [3,4,23,3], **kwargs) 113 | 114 | def ResNet152(**kwargs): 115 | return ResNet(Bottleneck, [3,8,36,3], **kwargs) 116 | -------------------------------------------------------------------------------- /test/dir/testNov_03_2022_01_11_51.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/test/dir/testNov_03_2022_01_11_51.npz -------------------------------------------------------------------------------- /test/dir/testNov_03_2022_17_30_04.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/test/dir/testNov_03_2022_17_30_04.npz -------------------------------------------------------------------------------- /test/step_result/dir/step_resultNov_02_2022_15_21_32.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/test/step_result/dir/step_resultNov_02_2022_15_21_32.npz -------------------------------------------------------------------------------- /test/step_result/dir/step_resultNov_02_2022_16_35_09.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/test/step_result/dir/step_resultNov_02_2022_16_35_09.npz -------------------------------------------------------------------------------- /test/step_result/dir/step_resultNov_02_2022_17_48_54.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/test/step_result/dir/step_resultNov_02_2022_17_48_54.npz -------------------------------------------------------------------------------- /test/step_result/dir/step_resultNov_02_2022_19_02_37.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/test/step_result/dir/step_resultNov_02_2022_19_02_37.npz -------------------------------------------------------------------------------- /test/step_result/dir/step_resultNov_02_2022_20_16_47.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/test/step_result/dir/step_resultNov_02_2022_20_16_47.npz -------------------------------------------------------------------------------- /test/step_result/dir/step_resultNov_02_2022_21_30_58.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/test/step_result/dir/step_resultNov_02_2022_21_30_58.npz -------------------------------------------------------------------------------- /test/step_result/dir/step_resultNov_02_2022_22_44_32.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/test/step_result/dir/step_resultNov_02_2022_22_44_32.npz -------------------------------------------------------------------------------- /test/step_result/dir/step_resultNov_02_2022_23_58_06.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/test/step_result/dir/step_resultNov_02_2022_23_58_06.npz -------------------------------------------------------------------------------- /test/step_result/dir/step_resultNov_03_2022_01_11_51.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/test/step_result/dir/step_resultNov_03_2022_01_11_51.npz -------------------------------------------------------------------------------- /test/step_result/dir/step_resultNov_03_2022_17_30_04.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/test/step_result/dir/step_resultNov_03_2022_17_30_04.npz -------------------------------------------------------------------------------- /test_q_fl_dqn.py: -------------------------------------------------------------------------------- 1 | # FL相关 2 | from qsgd_compressor import * 3 | from datetime import datetime 4 | from identical_compressor import * 5 | import torch.nn as nn 6 | import torch.optim as optim 7 | import time 8 | from fcn import FCN 9 | from resnet import * 10 | from dataloaders import mnist 11 | from ps_quantizer import * 12 | import os 13 | from logger import Logger 14 | 15 | from mec_env import * 16 | from helper import * 17 | from agent import * 18 | import tensorflow as tf 19 | import matplotlib 20 | import matplotlib.pyplot as plt 21 | 22 | from options import * 23 | 24 | # DDPG相关 25 | os.environ['CUDA_VISIBLE_DEVICES'] = '-1' 26 | 27 | # FL相关初始化 28 | os.environ['KMP_DUPLICATE_LIB_OK'] = 'True' 29 | 30 | # DQN相关设置 31 | tf.compat.v1.reset_default_graph() 32 | MAX_EPISODE = 2 33 | MAX_EPISODE_LEN = 501 34 | NUM_R = args.num_users # 天线数 跟信道数有关 35 | 36 | SIGMA2 = 1e-9 # 噪声平方 10 -9 37 | 38 | config = {'state_dim': 4, 'action_dim': 2}; 39 | train_config = {'minibatch_size': 64, 40 | 'actor_lr': 0.0001, 41 | 'tau': 0.001, 42 | 'critic_lr': 0.05, # 0.001 43 | 'gamma': 0.99, 44 | 'buffer_size': 250000, # 250000 45 | 'random_seed': int(time.perf_counter() * 1000 % 1000), 46 | 'sigma2': SIGMA2, 47 | 'epsilon': 0.5} # 0.5 48 | 49 | IS_TRAIN = False 50 | 51 | res_path = 'test/' 52 | step_result_path = res_path + 'step_result/' 53 | model_fold = 'model/' 54 | model_path = 'model/train_model-q' 55 | 56 | # 创建目录,用来保存dqn模型 57 | if not os.path.exists(res_path): 58 | os.mkdir(res_path) 59 | if not os.path.exists(step_result_path): 60 | os.mkdir(step_result_path) 61 | if not os.path.exists(model_fold): 62 | os.mkdir(model_fold) 63 | 64 | # meta_path = model_path + '.meta' 65 | init_path = '' 66 | # init_seqCnt = 40 67 | 68 | # choose the vehicle for training 69 | Train_vehicle_ID = 1 70 | 71 | # action_bound是需要后面调的 72 | user_config = [{'id': '1', 'model': 'AR', 'num_r': NUM_R, 'action_bound': 1}] 73 | 74 | # 0. initialize the session object 75 | sess = tf.compat.v1.Session() 76 | 77 | # 1. include all user in the system according to the user_config 78 | user_list = []; 79 | for info in user_config: 80 | info.update(config) 81 | info['model_path'] = model_path 82 | info['meta_path'] = info['model_path'] + '.meta' 83 | info['init_path'] = init_path 84 | info['action_level'] = 5 85 | user_list.append(MecTermDQN_LD(sess, info, train_config)) 86 | print('Initialization OK!----> user ') 87 | 88 | # 2. create the simulation env 89 | env = MecSvrEnv(user_list, Train_vehicle_ID, SIGMA2, MAX_EPISODE_LEN, mode='test') 90 | 91 | res_r = [] # 每一步的平均奖励 92 | res_a = [] # 每个回合的平均动作 93 | res_q = [] 94 | res_p = [] 95 | 96 | args = args_parser() 97 | 98 | args.no_cuda = args.no_cuda or not torch.cuda.is_available() 99 | 100 | torch.manual_seed(args.seed) 101 | device = torch.device("cpu" if args.no_cuda else "cuda") 102 | 103 | train_loader, test_loader = mnist(args) # 235 10 104 | model = FCN(num_classes=args.num_classes).to(device) 105 | optimizer = optim.SGD(model.parameters(), lr=0.1, 106 | momentum=args.momentum, weight_decay=args.weight_decay) 107 | ave_reward_ep = [] 108 | ave_delay_ep = [] 109 | ave_error_ep = [] 110 | #reward_step = [] 111 | train_loss, testing_accuracy = [], [] 112 | 113 | # 开始训练DQN episode 114 | for episode in range(1, MAX_EPISODE): 115 | print(f'\n | Global Training Round/episode : {episode} |\n') 116 | model.train() 117 | batch_size = args.batch_size # 32 118 | num_users = args.num_users # 8 119 | train_data = list() 120 | iteration = len(train_loader.dataset) // (num_users * batch_size) + \ 121 | int(len(train_loader.dataset) % (num_users * batch_size) != 0) # 235 122 | # 记录间隔 # [23, 46, 69, 92, 115, 138, 161, 184, 207, 230] 123 | log_interval = [iteration // args.log_epoch * (i + 1) for i in range(args.log_epoch)] 124 | # Training 125 | 126 | tr_step_loss = [] 127 | tr_step_acc = [] 128 | val_acc_list, net_list = [], [] 129 | cv_loss, cv_acc = [], [] 130 | print_every = 2 131 | val_loss_pre, counter = 0, 0 132 | step_r = [] 133 | step_mg = [] 134 | step_T = [] 135 | step_q = [] 136 | step_p = [] 137 | step_d = [] 138 | step_e = [] 139 | step_tr = [] 140 | step_delta = [] 141 | step_diss = [] 142 | 143 | # DQN相关 144 | # plt.ion() # 打开交互模式,画动态图 145 | cur_init_ds_ep = env.reset() # 环境重置 146 | cur_r_ep = 0 # 一个回合的总奖励 147 | cur_d_ep = 0 148 | cur_e_ep = 0 149 | step_cur_r_ep = [] 150 | count = 0 151 | # here the real batch size is (num_users * batch_size) 152 | # DQN时间步 & FL的一个通信轮次 153 | for epoch in range(1, MAX_EPISODE_LEN): 154 | i = Train_vehicle_ID - 1 155 | # if epoch == 1: 156 | # q_level1 = 1 157 | # if epoch % 5 == 0: 158 | actions = user_list[i].predict1(True) 159 | print('step is : {}, q_level is: {}, power is: {}'.format(epoch, actions[0], actions[1])) 160 | quantizer = Quantizer(QSGDCompressor, model.parameters(), args, actions[0]) 161 | # para = list(model.parameters()) 162 | # ini_para = para 163 | # FL本地训练迭代 164 | for batch_idx, (data, target) in enumerate(train_loader): 165 | user_batch_size = len(data) // num_users # 32 = 256//8 166 | train_data.clear() 167 | # 给每个用户分配训练数据 168 | for user_id in range(num_users - 1): 169 | train_data.append((data[user_id * user_batch_size:(user_id + 1) * user_batch_size], 170 | target[user_id * user_batch_size:(user_id + 1) * user_batch_size])) 171 | train_data.append((data[(num_users - 1) * user_batch_size:], 172 | target[(num_users - 1) * user_batch_size:])) 173 | 174 | # 计算一次迭代的全局损失 175 | loss = one_iter(model, device, LOSS_FUNC, optimizer, 176 | quantizer, train_data, num_users, epoch=epoch) 177 | 178 | # 记录一个epoch的2个损失和测试精度 179 | if (batch_idx + 1) in log_interval: 180 | train_loss.append(loss.item()) 181 | test_accuracy = test(args, model, device, test_loader) 182 | testing_accuracy.append(test_accuracy * 100) 183 | # print('Train Episode: {} Train epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}\t Test Accuracy: {:.2f}%'.format( 184 | # episode, 185 | # epoch, 186 | # batch_idx * num_users * batch_size + len(data), 187 | # len(train_loader.dataset), 188 | # 100. * batch_idx / len(train_loader), 189 | # loss.item(), 190 | # test_accuracy*100)) 191 | 192 | # info = {'loss': loss.item(), 'accuracy(%)': test_accuracy*100} 193 | 194 | # for tag, value in info.items(): 195 | # LOGGER.scalar_summary( 196 | # tag, value, iteration*(epoch-1)+batch_idx) 197 | # para1 = list(model.parameters()) 198 | # maxx = max(para1[0].detach().numpy().reshape(1,200704).flatten()) 199 | # minn = min(abs(para1[0].detach().numpy().reshape(1,200704).flatten())) 200 | # maxs.append(maxx) 201 | # mins.append(minn) 202 | 203 | print('Train Epoch: {} Done.\tLoss: {:.6f}'.format(epoch, loss.item())) 204 | 205 | reward = 0 206 | trs = 0 207 | deltas = 0 208 | diss = 0 209 | 210 | max_len = 500 # MAX_EPISODE_LEN 211 | count += 1 212 | 213 | i = Train_vehicle_ID - 1 214 | # feedback the sinr to each user 215 | [reward, mini_ground, T_total, trs, deltas, diss, q_level, power, delay, q_error] = user_list[i].feedback1(actions[0], actions[1]) 216 | print("reward is: {}, delay is: {}".format(reward, delay)) 217 | #reward_step.append(reward) 218 | 219 | # user_list[i].AgentUpdate(count >= max_len) # 训练数据个数逐渐增加 大于buffer的大小时 进行更新agent 220 | cur_r = reward 221 | cur_mg = mini_ground 222 | cur_T = T_total 223 | cur_q = q_level 224 | cur_p = power 225 | cur_d = delay 226 | cur_e = q_error 227 | cur_tr = trs 228 | cur_delta = deltas 229 | cur_diss = diss 230 | # done = count >= max_len # max_len即为MAX_EPISODE_LEN 231 | 232 | step_r.append(cur_r) 233 | step_mg.append(cur_mg) 234 | step_T.append(cur_T) 235 | step_q.append(cur_q) 236 | step_p.append(cur_p) 237 | step_d.append(cur_d) 238 | step_e.append(cur_e) 239 | step_tr.append(cur_tr) 240 | step_delta.append(cur_delta) 241 | step_diss.append(cur_diss) 242 | 243 | #cur_r_ep += cur_r # 一个回合的总奖励 所有step的奖励之和 244 | #cur_d_ep += cur_d 245 | #cur_e_ep += cur_e 246 | # for m in range(args.num_users): 247 | # cur_p_ep[m] += cur_p[m] 248 | 249 | # if done: # 一个episode结束 250 | # res_r.append(cur_r_ep / MAX_EPISODE_LEN) # 后面为了存储进模型 每一步的平均奖励 251 | 252 | # cur_p_ep1 = [0] * args.num_users 253 | # for m in range(args.num_users): 254 | # cur_p_ep1[m] = cur_p_ep[m] / MAX_EPISODE_LEN # 一个回合里平均每一个step的动作 255 | # res_p.append(cur_p_ep1) # 用来存储每个回合的平均动作 256 | 257 | #print("episode = ", episode) 258 | #print("r = ", cur_r_ep / MAX_EPISODE_LEN) 259 | #print("delay = ", cur_d_ep / MAX_EPISODE_LEN) 260 | #print("error = ", cur_e_ep / MAX_EPISODE_LEN) 261 | #ave_reward_ep.append(cur_r_ep / MAX_EPISODE_LEN) 262 | #ave_delay_ep.append(cur_d_ep / MAX_EPISODE_LEN) 263 | #ave_error_ep.append(cur_e_ep / MAX_EPISODE_LEN) 264 | 265 | step_result = step_result_path + 'step_result' + time.strftime("%b_%d_%Y_%H_%M_%S", time.localtime(time.time())) 266 | np.savez(step_result, step_r, step_mg, step_T, step_q, step_p, step_d, step_e, step_tr, step_delta, step_diss) 267 | # print("p_lambda = ", cur_p_ep1) # this is wrong 268 | # print("cur_r_ep = ", cur_r_ep) 269 | 270 | # line_reward = ax.plot(range(0, MAX_EPISODE), cur_r_ep, '#ff7f0e', label='reward', lw=1) 271 | # line_pro = ax2.plot(range(0, MAX_EPISODE), step_cur_r_ep[0], '#ff7f0e', label='第1辆车的the probility(选择每辆车的概率，即动作输出)', lw=1) 272 | # plt.ioff() 273 | 274 | # 模型保存 275 | name = res_path + 'test' + time.strftime("%b_%d_%Y_%H_%M_%S", time.localtime(time.time())) 276 | # np.savez(name, train_loss, testing_accuracy, ave_reward_ep, ave_delay_ep, ave_error_ep) # 保存平均每一步的q_level和奖励 # 为了后面的画图 277 | np.savez(name, train_loss, testing_accuracy) 278 | 279 | sess.close() -------------------------------------------------------------------------------- /test_results/acc/ada.txt: -------------------------------------------------------------------------------- 1 | 3.327250000000000485e-01 2 | 3.444800000000000084e-01 3 | 3.557500000000000107e-01 4 | 3.703450000000000353e-01 5 | 3.823300000000001142e-01 6 | 3.959450000000001024e-01 7 | 4.074550000000001226e-01 8 | 4.177650000000001640e-01 9 | 4.300150000000001471e-01 10 | 4.396550000000001290e-01 11 | 4.480200000000001959e-01 12 | 4.551000000000001711e-01 13 | 4.630350000000001964e-01 14 | 4.708150000000001500e-01 15 | 4.783400000000000984e-01 16 | 4.859100000000001751e-01 17 | 4.923900000000000499e-01 18 | 4.993800000000000461e-01 19 | 5.059750000000000636e-01 20 | 5.133550000000000058e-01 21 | 5.198650000000000215e-01 22 | 5.267199999999999660e-01 23 | 5.336250000000000160e-01 24 | 5.418800000000000283e-01 25 | 5.479799999999999116e-01 26 | 5.536349999999998772e-01 27 | 5.608799999999998231e-01 28 | 5.669699999999998630e-01 29 | 5.724299999999999944e-01 30 | 5.778600000000000403e-01 31 | 5.851699999999999680e-01 32 | 5.912549999999999750e-01 33 | 5.965599999999999792e-01 34 | 6.015700000000000491e-01 35 | 6.066050000000000608e-01 36 | 6.121100000000001540e-01 37 | 6.170050000000000257e-01 38 | 6.231650000000000800e-01 39 | 6.285950000000001259e-01 40 | 6.324650000000001659e-01 41 | 6.370800000000003127e-01 42 | 6.413050000000002360e-01 43 | 6.458150000000001389e-01 44 | 6.494600000000001483e-01 45 | 6.540250000000004116e-01 46 | 6.590150000000003505e-01 47 | 6.619450000000002277e-01 48 | 6.648400000000002086e-01 49 | 6.656250000000001332e-01 50 | 6.678650000000002640e-01 51 | 6.695100000000003826e-01 52 | 6.727800000000003777e-01 53 | 6.734200000000003516e-01 54 | 6.760150000000002546e-01 55 | 6.794900000000001494e-01 56 | 6.811250000000003135e-01 57 | 6.856850000000003220e-01 58 | 6.884200000000001429e-01 59 | 6.911749999999999838e-01 60 | 6.953149999999999054e-01 61 | 6.982899999999997442e-01 62 | 7.006599999999998385e-01 63 | 7.013199999999999434e-01 64 | 7.037599999999997191e-01 65 | 7.066899999999997073e-01 66 | 7.094449999999998813e-01 67 | 7.123599999999999932e-01 68 | 7.152699999999999614e-01 69 | 7.207399999999999363e-01 70 | 7.243449999999999056e-01 71 | 7.271899999999998920e-01 72 | 7.290499999999997538e-01 73 | 7.345849999999998214e-01 74 | 7.378750000000000586e-01 75 | 7.383050000000001001e-01 76 | 7.401399999999999091e-01 77 | 7.423749999999997851e-01 78 | 7.447399999999998466e-01 79 | 7.453800000000001535e-01 80 | 7.466950000000001086e-01 81 | 7.491150000000000864e-01 82 | 7.520800000000000818e-01 83 | 7.577750000000001984e-01 84 | 7.600200000000004730e-01 85 | 7.626500000000003832e-01 86 | 7.646700000000002939e-01 87 | 7.660100000000003018e-01 88 | 7.688600000000004320e-01 89 | 7.704700000000006543e-01 90 | 7.737150000000003747e-01 91 | 7.760400000000003962e-01 92 | 7.783450000000005087e-01 93 | 7.796900000000004383e-01 94 | 7.820550000000003887e-01 95 | 7.852950000000002984e-01 96 | 7.872250000000005077e-01 97 | 7.868800000000003569e-01 98 | 7.838550000000001905e-01 99 | 7.862200000000001410e-01 100 | 7.883700000000004593e-01 101 | 7.896100000000003671e-01 102 | 7.893200000000000216e-01 103 | 7.894899999999995810e-01 104 | 7.894649999999998613e-01 105 | 7.893499999999999961e-01 106 | 7.902400000000000535e-01 107 | 7.921600000000001973e-01 108 | 7.933950000000002944e-01 109 | 7.943750000000001643e-01 110 | 7.942700000000002536e-01 111 | 7.939799999999997970e-01 112 | 7.936849999999999739e-01 113 | 7.945250000000001478e-01 114 | 7.948850000000000637e-01 115 | 7.957199999999999829e-01 116 | 7.974699999999999012e-01 117 | 7.993650000000002143e-01 118 | 8.044100000000007356e-01 119 | 8.056150000000009692e-01 120 | 8.063000000000009049e-01 121 | 8.070800000000012409e-01 122 | 8.082050000000016166e-01 123 | 8.100050000000017514e-01 124 | 8.123950000000015326e-01 125 | 8.137550000000011163e-01 126 | 8.141000000000012671e-01 127 | 8.149150000000012772e-01 128 | 8.153200000000012659e-01 129 | 8.154450000000010856e-01 130 | 8.165550000000010300e-01 131 | 8.193600000000010875e-01 132 | 8.207600000000006002e-01 133 | 8.221050000000005298e-01 134 | 8.228650000000001796e-01 135 | 8.238200000000006629e-01 136 | 8.249700000000004252e-01 137 | 8.272899999999999698e-01 138 | 8.274349999999998095e-01 139 | 8.278949999999994924e-01 140 | 8.286549999999991423e-01 141 | 8.281199999999990791e-01 142 | 8.286999999999992150e-01 143 | 8.282699999999991736e-01 144 | 8.287099999999988365e-01 145 | 8.292899999999988614e-01 146 | 8.308999999999990838e-01 147 | 8.320749999999990099e-01 148 | 8.334399999999988484e-01 149 | 8.351949999999987995e-01 150 | 8.350549999999991035e-01 151 | 8.353649999999994691e-01 152 | 8.367249999999998300e-01 153 | 8.367249999999998300e-01 154 | 8.362200000000001854e-01 155 | 8.372849999999999460e-01 156 | 8.377850000000002240e-01 157 | 8.379450000000006060e-01 158 | 8.384850000000007020e-01 159 | 8.390500000000009617e-01 160 | 8.389450000000010510e-01 161 | 8.398400000000009191e-01 162 | 8.411250000000010107e-01 163 | 8.426650000000008855e-01 164 | 8.429550000000013421e-01 165 | 8.424250000000015337e-01 166 | 8.426650000000016627e-01 167 | 8.424000000000013699e-01 168 | 8.429150000000014131e-01 169 | 8.434350000000009340e-01 170 | 8.452400000000011016e-01 171 | 8.458050000000006952e-01 172 | 8.463799999999999102e-01 173 | 8.477150000000002183e-01 174 | 8.492950000000007993e-01 175 | 8.495200000000003859e-01 176 | 8.499549999999999050e-01 177 | 8.495249999999991974e-01 178 | 8.499349999999992189e-01 179 | 8.508049999999982571e-01 180 | 8.517499999999976756e-01 181 | 8.526249999999976348e-01 182 | 8.528049999999971487e-01 183 | 8.530999999999970829e-01 184 | 8.539849999999965524e-01 185 | 8.555099999999959959e-01 186 | 8.557299999999955498e-01 187 | 8.564749999999961005e-01 188 | 8.561099999999953747e-01 189 | 8.563949999999962426e-01 190 | 8.564649999999958130e-01 191 | 8.571499999999957486e-01 192 | 8.574199999999961852e-01 193 | 8.572699999999955356e-01 194 | 8.574899999999943123e-01 195 | 8.576999999999941338e-01 196 | 8.577699999999950364e-01 197 | 8.573299999999960397e-01 198 | 8.580849999999955457e-01 199 | 8.575799999999957901e-01 200 | 8.570249999999959289e-01 201 | 8.574199999999961852e-01 202 | 8.582549999999968815e-01 203 | 8.586799999999967792e-01 204 | 8.589449999999970720e-01 205 | 8.588549999999983697e-01 206 | 8.595749999999980906e-01 207 | 8.597049999999981651e-01 208 | 8.603299999999990400e-01 209 | 8.600749999999990347e-01 210 | 8.596849999999989222e-01 211 | 8.592299999999994942e-01 212 | 8.590299999999999603e-01 213 | 8.591200000000001058e-01 214 | 8.590200000000010050e-01 215 | 8.582000000000007400e-01 216 | 8.582050000000009948e-01 217 | 8.591700000000003223e-01 218 | 8.599100000000007293e-01 219 | 8.613450000000014706e-01 220 | 8.626350000000015950e-01 221 | 8.632950000000008117e-01 222 | 8.635950000000007787e-01 223 | 8.635600000000011045e-01 224 | 8.638800000000017576e-01 225 | 8.647950000000008686e-01 226 | 8.650350000000017747e-01 227 | 8.657700000000005947e-01 228 | 8.666099999999999914e-01 229 | 8.674149999999997140e-01 230 | 8.682500000000004103e-01 231 | 8.689599999999998436e-01 232 | 8.694549999999992007e-01 233 | 8.698200000000000376e-01 234 | 8.706999999999993634e-01 235 | 8.720799999999997443e-01 236 | 8.721049999999991309e-01 237 | 8.731600000000000472e-01 238 | 8.730099999999992866e-01 239 | 8.724949999999992434e-01 240 | 8.724499999999991706e-01 241 | 8.725549999999998585e-01 242 | 8.728299999999989955e-01 243 | 8.726599999999991031e-01 244 | 8.728399999999979508e-01 245 | 8.722599999999985920e-01 246 | 8.721099999999978314e-01 247 | 8.716399999999978609e-01 248 | 8.712099999999978195e-01 249 | 8.701649999999971907e-01 250 | 8.703099999999963643e-01 251 | 8.705449999999970156e-01 252 | 8.709649999999967696e-01 253 | 8.712449999999961614e-01 254 | 8.713849999999965235e-01 255 | 8.703849999999974107e-01 256 | 8.713699999999974244e-01 257 | 8.714149999999974971e-01 258 | 8.717699999999979354e-01 259 | 8.727599999999980929e-01 260 | 8.733049999999991098e-01 261 | 8.735149999999990422e-01 262 | 8.736849999999989347e-01 263 | 8.744749999999996692e-01 264 | 8.746650000000002478e-01 265 | 8.756500000000002615e-01 266 | 8.754900000000006566e-01 267 | 8.758150000000014535e-01 268 | 8.752550000000013375e-01 269 | 8.756850000000013790e-01 270 | 8.755000000000023874e-01 271 | 8.755550000000027477e-01 272 | 8.750050000000030304e-01 273 | 8.750700000000037893e-01 274 | 8.747100000000032072e-01 275 | 8.761000000000024324e-01 276 | 8.759350000000025727e-01 277 | 8.765000000000015001e-01 278 | 8.763000000000019662e-01 279 | 8.758750000000020686e-01 280 | 8.757900000000020668e-01 281 | 8.758850000000023561e-01 282 | 8.747300000000024500e-01 283 | 8.739100000000021851e-01 284 | 8.735450000000014592e-01 285 | 8.732800000000011664e-01 286 | 8.736250000000012061e-01 287 | 8.736900000000005218e-01 288 | 8.742900000000005667e-01 289 | 8.748100000000007537e-01 290 | 8.751450000000005058e-01 291 | 8.750850000000000017e-01 292 | 8.758949999999998681e-01 293 | 8.764799999999993707e-01 294 | 8.770799999999994156e-01 295 | 8.775300000000001432e-01 296 | 8.779849999999995713e-01 297 | 8.777450000000002195e-01 298 | 8.780500000000003302e-01 299 | 8.773399999999995647e-01 300 | 8.768149999999991229e-01 301 | 8.770849999999995594e-01 302 | 8.784300000000001551e-01 303 | 8.792699999999996630e-01 304 | 8.788300000000006662e-01 305 | 8.788200000000003786e-01 306 | 8.784900000000007703e-01 307 | 8.786400000000014199e-01 308 | 8.792650000000022947e-01 309 | 8.795150000000020452e-01 310 | 8.798800000000014387e-01 311 | 8.800950000000014040e-01 312 | 8.805050000000023136e-01 313 | 8.805850000000020605e-01 314 | 8.799700000000015843e-01 315 | 8.794500000000013973e-01 316 | 8.790250000000028319e-01 317 | 8.788800000000023260e-01 318 | 8.783800000000028252e-01 319 | 8.793700000000029826e-01 320 | 8.800200000000018008e-01 321 | 8.803800000000023829e-01 322 | 8.805100000000010141e-01 323 | 8.807750000000027502e-01 324 | 8.818250000000020794e-01 325 | 8.820350000000019008e-01 326 | 8.827550000000016217e-01 327 | 8.822250000000024794e-01 328 | 8.824850000000026284e-01 329 | 8.821200000000033459e-01 330 | 8.817750000000046384e-01 331 | 8.819750000000056156e-01 332 | 8.815800000000038050e-01 333 | 8.816250000000053211e-01 334 | 8.823850000000078575e-01 335 | 8.830600000000060623e-01 336 | 8.839750000000037300e-01 337 | 8.847900000000038512e-01 338 | 8.857000000000028184e-01 339 | 8.859950000000026415e-01 340 | 8.857000000000028184e-01 341 | 8.851700000000022328e-01 342 | 8.845900000000028740e-01 343 | 8.847150000000028047e-01 344 | 8.846300000000042463e-01 345 | 8.849300000000056565e-01 346 | 8.844800000000049289e-01 347 | 8.850550000000027007e-01 348 | 8.843500000000034111e-01 349 | 8.839950000000044161e-01 350 | 8.837600000000037648e-01 351 | 8.823500000000024102e-01 352 | 8.819550000000049295e-01 353 | 8.818750000000022959e-01 354 | 8.812800000000009515e-01 355 | 8.810650000000009863e-01 356 | 8.807150000000006917e-01 357 | 8.800600000000002865e-01 358 | 8.799100000000009691e-01 359 | 8.801100000000019463e-01 360 | 8.807950000000033253e-01 361 | 8.812950000000029371e-01 362 | 8.817950000000024380e-01 363 | 8.817350000000004906e-01 364 | 8.815550000000001996e-01 365 | 8.812249999999977046e-01 366 | 8.813599999999979229e-01 367 | 8.811299999999988586e-01 368 | 8.813949999999977081e-01 369 | 8.823399999999963494e-01 370 | 8.831599999999951711e-01 371 | 8.850949999999955242e-01 372 | 8.849699999999955935e-01 373 | 8.843799999999959471e-01 374 | 8.848649999999963489e-01 375 | 8.848749999999967475e-01 376 | 8.845799999999968133e-01 377 | 8.849849999999974681e-01 378 | 8.852649999999983033e-01 379 | 8.849299999999971078e-01 380 | 8.847149999999970316e-01 381 | 8.847299999999961306e-01 382 | 8.847349999999977177e-01 383 | 8.843899999999991213e-01 384 | 8.844249999999987955e-01 385 | 8.847550000000012904e-01 386 | 8.857400000000013041e-01 387 | 8.861800000000016331e-01 388 | 8.863150000000018514e-01 389 | 8.855900000000019867e-01 390 | 8.854150000000032827e-01 391 | 8.849750000000028427e-01 392 | 8.852550000000007913e-01 393 | 8.854000000000012971e-01 394 | 8.854050000000001086e-01 395 | 8.855700000000013006e-01 396 | 8.858850000000018099e-01 397 | 8.857400000000013041e-01 398 | 8.854700000000008675e-01 399 | 8.853400000000022363e-01 400 | 8.853600000000028114e-01 401 | 8.855100000000021288e-01 402 | 8.853900000000010095e-01 403 | 8.852200000000010061e-01 404 | 8.852900000000005765e-01 405 | 8.858249999999998625e-01 406 | 8.852399999999989166e-01 407 | 8.846899999999976449e-01 408 | 8.849999999999965672e-01 409 | 8.858049999999962898e-01 410 | 8.857299999999952433e-01 411 | 8.859199999999958219e-01 412 | 8.854099999999960335e-01 413 | 8.861199999999967991e-01 414 | 8.857749999999982027e-01 415 | 8.859949999999969794e-01 416 | 8.857699999999966156e-01 417 | 8.860199999999963660e-01 418 | 8.860149999999947790e-01 419 | 8.861599999999925092e-01 420 | 8.866049999999916498e-01 421 | 8.865049999999939923e-01 422 | 8.866799999999954718e-01 423 | 8.868949999999955480e-01 424 | 8.870149999999966672e-01 425 | 8.863849999999956486e-01 426 | 8.858849999999961478e-01 427 | 8.858849999999961478e-01 428 | 8.858499999999963626e-01 429 | 8.861949999999978456e-01 430 | 8.862649999999974160e-01 431 | 8.863849999999985352e-01 432 | 8.873149999999980775e-01 433 | 8.865599999999972392e-01 434 | 8.864749999999957941e-01 435 | 8.859699999999974818e-01 436 | 8.864049999999992213e-01 437 | 8.863900000000001222e-01 438 | 8.864699999999998692e-01 439 | 8.868100000000026517e-01 440 | 8.865700000000031888e-01 441 | 8.864400000000017821e-01 442 | 8.861250000000012728e-01 443 | 8.861299999999999732e-01 444 | 8.858199999999982754e-01 445 | 8.858649999999983482e-01 446 | 8.865649999999988262e-01 447 | 8.869249999999994083e-01 448 | 8.867150000000009191e-01 449 | 8.865949999999997999e-01 450 | 8.861299999999999732e-01 451 | 8.860549999999989268e-01 452 | 8.859600000000000808e-01 453 | 8.859900000000010545e-01 454 | 8.864550000000008811e-01 455 | 8.871900000000010333e-01 456 | 8.871149999999999869e-01 457 | 8.869049999999987222e-01 458 | 8.868850000000009226e-01 459 | 8.869249999999994083e-01 460 | 8.869149999999990097e-01 461 | 8.870199999999982543e-01 462 | 8.870599999999967400e-01 463 | 8.869599999999963069e-01 464 | 8.871999999999957698e-01 465 | 8.870749999999958391e-01 466 | 8.869599999999963069e-01 467 | 8.874549999999971073e-01 468 | 8.874199999999973221e-01 469 | 8.875949999999960260e-01 470 | 8.881499999999959982e-01 471 | 8.879999999999966809e-01 472 | 8.880399999999980531e-01 473 | 8.883899999999982366e-01 474 | 8.881849999999985590e-01 475 | 8.877199999999987323e-01 476 | 8.880650000000003264e-01 477 | 8.881200000000006867e-01 478 | 8.883250000000003643e-01 479 | 8.880700000000019134e-01 480 | 8.883050000000025648e-01 481 | 8.880350000000050148e-01 482 | 8.884000000000071839e-01 483 | 8.889700000000090307e-01 484 | 8.887100000000088817e-01 485 | 8.892600000000101534e-01 486 | 8.897100000000108810e-01 487 | 8.893300000000096128e-01 488 | 8.891200000000083481e-01 489 | 8.886150000000100357e-01 490 | 8.891900000000105830e-01 491 | 8.896100000000103369e-01 492 | 8.894750000000101187e-01 493 | 8.898500000000097998e-01 494 | 8.901650000000103091e-01 495 | 8.900850000000104512e-01 496 | 8.897950000000094395e-01 497 | 8.900900000000092627e-01 498 | 8.902200000000078939e-01 499 | 8.896600000000063346e-01 500 | 8.888500000000050250e-01 501 | -------------------------------------------------------------------------------- /test_results/acc/dqn_q.txt: -------------------------------------------------------------------------------- 1 | 2.907000000000000139e-01 2 | 4.256899999999999573e-01 3 | 4.530666666666668951e-01 4 | 4.707933333333332304e-01 5 | 5.036866666666665049e-01 6 | 5.363766666666666127e-01 7 | 5.557666666666661870e-01 8 | 5.655366666666665765e-01 9 | 5.739299999999999402e-01 10 | 5.822166666666664936e-01 11 | 5.954966666666664521e-01 12 | 6.054299999999999127e-01 13 | 6.130166666666665432e-01 14 | 6.201633333333330667e-01 15 | 6.256466666666665732e-01 16 | 6.313366666666664351e-01 17 | 6.381066666666665999e-01 18 | 6.429500000000000215e-01 19 | 6.480966666666666542e-01 20 | 6.527833333333332710e-01 21 | 6.591966666666666530e-01 22 | 6.639766666666666595e-01 23 | 6.682366666666667010e-01 24 | 6.725233333333332508e-01 25 | 6.777533333333333188e-01 26 | 6.816966666666666175e-01 27 | 6.851599999999999913e-01 28 | 6.902566666666667405e-01 29 | 6.943166666666668041e-01 30 | 6.997533333333334493e-01 31 | 7.035666666666668956e-01 32 | 7.064566666666668437e-01 33 | 7.095400000000002816e-01 34 | 7.118566666666670262e-01 35 | 7.148433333333337192e-01 36 | 7.176433333333336329e-01 37 | 7.209866666666671087e-01 38 | 7.224300000000005717e-01 39 | 7.261000000000005228e-01 40 | 7.290866666666672158e-01 41 | 7.332466666666671573e-01 42 | 7.363466666666670379e-01 43 | 7.385433333333337735e-01 44 | 7.410700000000005616e-01 45 | 7.447100000000005382e-01 46 | 7.471533333333338911e-01 47 | 7.491066666666672536e-01 48 | 7.512166666666671988e-01 49 | 7.542433333333338208e-01 50 | 7.561266666666672798e-01 51 | 7.588233333333339603e-01 52 | 7.606066666666674303e-01 53 | 7.626166666666674976e-01 54 | 7.639000000000008006e-01 55 | 7.658633333333340065e-01 56 | 7.678300000000006786e-01 57 | 7.693633333333340651e-01 58 | 7.709733333333339544e-01 59 | 7.726400000000005486e-01 60 | 7.744000000000004214e-01 61 | 7.758700000000005037e-01 62 | 7.788700000000002843e-01 63 | 7.808800000000002406e-01 64 | 7.827366666666667472e-01 65 | 7.850133333333334518e-01 66 | 7.853366666666669049e-01 67 | 7.867466666666668162e-01 68 | 7.893933333333333913e-01 69 | 7.897733333333332162e-01 70 | 7.911866666666665937e-01 71 | 7.928633333333332533e-01 72 | 7.942600000000000771e-01 73 | 7.957566666666664457e-01 74 | 7.968033333333330859e-01 75 | 7.980166666666664854e-01 76 | 7.995833333333329795e-01 77 | 8.009133333333328109e-01 78 | 8.025799999999994050e-01 79 | 8.031966666666660037e-01 80 | 8.052833333333324628e-01 81 | 8.054033333333324718e-01 82 | 8.060999999999988175e-01 83 | 8.074966666666651971e-01 84 | 8.088799999999987111e-01 85 | 8.101299999999987955e-01 86 | 8.108033333333319881e-01 87 | 8.116533333333317835e-01 88 | 8.127166666666648664e-01 89 | 8.139666666666651729e-01 90 | 8.144933333333319592e-01 91 | 8.154799999999985394e-01 92 | 8.147366666666653323e-01 93 | 8.153133333333322241e-01 94 | 8.161599999999989974e-01 95 | 8.170466666666655886e-01 96 | 8.190233333333319932e-01 97 | 8.200699999999988554e-01 98 | 8.206933333333320535e-01 99 | 8.220799999999987007e-01 100 | 8.230099999999987981e-01 101 | 8.221299999999990282e-01 102 | 8.226466666666657490e-01 103 | 8.233499999999992491e-01 104 | 8.246133333333323101e-01 105 | 8.241266666666656748e-01 106 | 8.245799999999988694e-01 107 | 8.251566666666657612e-01 108 | 8.252666666666660378e-01 109 | 8.252999999999995895e-01 110 | 8.254733333333329481e-01 111 | 8.263799999999993373e-01 112 | 8.266966666666661911e-01 113 | 8.272166666666663781e-01 114 | 8.277299999999996327e-01 115 | 8.284866666666663715e-01 116 | 8.292066666666665364e-01 117 | 8.304166666666669139e-01 118 | 8.311433333333340112e-01 119 | 8.313266666666673244e-01 120 | 8.321466666666670342e-01 121 | 8.321033333333337501e-01 122 | 8.338333333333338704e-01 123 | 8.342766666666672215e-01 124 | 8.348400000000002485e-01 125 | 8.348166666666670954e-01 126 | 8.353033333333338417e-01 127 | 8.360833333333336226e-01 128 | 8.369433333333338165e-01 129 | 8.371366666666674172e-01 130 | 8.375600000000005263e-01 131 | 8.390133333333338328e-01 132 | 8.389333333333335307e-01 133 | 8.389200000000002211e-01 134 | 8.386933333333336238e-01 135 | 8.398033333333335682e-01 136 | 8.402066666666669903e-01 137 | 8.413133333333334685e-01 138 | 8.415233333333332899e-01 139 | 8.430666666666667419e-01 140 | 8.438966666666668504e-01 141 | 8.444400000000001905e-01 142 | 8.454100000000001058e-01 143 | 8.458866666666670087e-01 144 | 8.464833333333335874e-01 145 | 8.469100000000001627e-01 146 | 8.477033333333333642e-01 147 | 8.484533333333327265e-01 148 | 8.490466666666658391e-01 149 | 8.499199999999992317e-01 150 | 8.504799999999997917e-01 151 | 8.511899999999996691e-01 152 | 8.515799999999994485e-01 153 | 8.522666666666661728e-01 154 | 8.526266666666661997e-01 155 | 8.532433333333325765e-01 156 | 8.535633333333326744e-01 157 | 8.536899999999992827e-01 158 | 8.540699999999991077e-01 159 | 8.547199999999989251e-01 160 | 8.546499999999994657e-01 161 | 8.550166666666664250e-01 162 | 8.557199999999994811e-01 163 | 8.564466666666665784e-01 164 | 8.568833333333329971e-01 165 | 8.570833333333335302e-01 166 | 8.571566666666664558e-01 167 | 8.570899999999994634e-01 168 | 8.579933333333324974e-01 169 | 8.583266666666660161e-01 170 | 8.586266666666659830e-01 171 | 8.588799999999991996e-01 172 | 8.594566666666660915e-01 173 | 8.601199999999996626e-01 174 | 8.603633333333330357e-01 175 | 8.599433333333327267e-01 176 | 8.603733333333328792e-01 177 | 8.601433333333337039e-01 178 | 8.605400000000003047e-01 179 | 8.606600000000005357e-01 180 | 8.607266666666665289e-01 181 | 8.611366666666668834e-01 182 | 8.614066666666673200e-01 183 | 8.617100000000003091e-01 184 | 8.624300000000004740e-01 185 | 8.628966666666675334e-01 186 | 8.631600000000010375e-01 187 | 8.629066666666678209e-01 188 | 8.631666666666679699e-01 189 | 8.633200000000016416e-01 190 | 8.637066666666678438e-01 191 | 8.646133333333343440e-01 192 | 8.653233333333351096e-01 193 | 8.657833333333343484e-01 194 | 8.666766666666679830e-01 195 | 8.670866666666673384e-01 196 | 8.677300000000012226e-01 197 | 8.680866666666683384e-01 198 | 8.684566666666683199e-01 199 | 8.689600000000012869e-01 200 | 8.692333333333341905e-01 201 | 8.695866666666679512e-01 202 | 8.694033333333341940e-01 203 | 8.691266666666668250e-01 204 | 8.690533333333330113e-01 205 | 8.695699999999997321e-01 206 | 8.695266666666668920e-01 207 | 8.697566666666659563e-01 208 | 8.698133333333326611e-01 209 | 8.698766666666651881e-01 210 | 8.703066666666653406e-01 211 | 8.700999999999984302e-01 212 | 8.702399999999983482e-01 213 | 8.707833333333316883e-01 214 | 8.710833333333312112e-01 215 | 8.713133333333312747e-01 216 | 8.715699999999979575e-01 217 | 8.724233333333312190e-01 218 | 8.728533333333312605e-01 219 | 8.733733333333304483e-01 220 | 8.739133333333304332e-01 221 | 8.739599999999967395e-01 222 | 8.737533333333298291e-01 223 | 8.740966666666631912e-01 224 | 8.738466666666634408e-01 225 | 8.735166666666638324e-01 226 | 8.731366666666635634e-01 227 | 8.731399999999970296e-01 228 | 8.732966666666641675e-01 229 | 8.733799999999973807e-01 230 | 8.734766666666644586e-01 231 | 8.731766666666639365e-01 232 | 8.738366666666640414e-01 233 | 8.743899999999977801e-01 234 | 8.752433333333319299e-01 235 | 8.757733333333325154e-01 236 | 8.760899999999982590e-01 237 | 8.764899999999983260e-01 238 | 8.765466666666649198e-01 239 | 8.768966666666652143e-01 240 | 8.766833333333323708e-01 241 | 8.769133333333324343e-01 242 | 8.776133333333320241e-01 243 | 8.773433333333324757e-01 244 | 8.774466666666663750e-01 245 | 8.770333333333326653e-01 246 | 8.767433333333324308e-01 247 | 8.761433333333324969e-01 248 | 8.763099999999989231e-01 249 | 8.763999999999990687e-01 250 | 8.762499999999988631e-01 251 | 8.760699999999995713e-01 252 | 8.762199999999997768e-01 253 | 8.762433333333338181e-01 254 | 8.767200000000002769e-01 255 | 8.775700000000000722e-01 256 | 8.783899999999997821e-01 257 | 8.788566666666668414e-01 258 | 8.789366666666665884e-01 259 | 8.789266666666663008e-01 260 | 8.792466666666665098e-01 261 | 8.800266666666668458e-01 262 | 8.797933333333333161e-01 263 | 8.796333333333327120e-01 264 | 8.797666666666656976e-01 265 | 8.799299999999987687e-01 266 | 8.800199999999989142e-01 267 | 8.800299999999993128e-01 268 | 8.802433333333330445e-01 269 | 8.805400000000001004e-01 270 | 8.811933333333333840e-01 271 | 8.817566666666664110e-01 272 | 8.811599999999998323e-01 273 | 8.813599999999998102e-01 274 | 8.815733333333326538e-01 275 | 8.824333333333328477e-01 276 | 8.831699999999993445e-01 277 | 8.842199999999991178e-01 278 | 8.843599999999990358e-01 279 | 8.843766666666662557e-01 280 | 8.847433333333327710e-01 281 | 8.852966666666655104e-01 282 | 8.855799999999988126e-01 283 | 8.859333333333315741e-01 284 | 8.859133333333317761e-01 285 | 8.860733333333314921e-01 286 | 8.861699999999984589e-01 287 | 8.862399999999979183e-01 288 | 8.862466666666648507e-01 289 | 8.863699999999984369e-01 290 | 8.866366666666645191e-01 291 | 8.866799999999973592e-01 292 | 8.865833333333312805e-01 293 | 8.869933333333316350e-01 294 | 8.866566666666642060e-01 295 | 8.867733333333309709e-01 296 | 8.869133333333308888e-01 297 | 8.870399999999970531e-01 298 | 8.873333333333306427e-01 299 | 8.873299999999962884e-01 300 | 8.872133333333295235e-01 301 | 8.873066666666640234e-01 302 | 8.876266666666642324e-01 303 | 8.879133333333300016e-01 304 | 8.876266666666642324e-01 305 | 8.876599999999986723e-01 306 | 8.872533333333326722e-01 307 | 8.868399999999998506e-01 308 | 8.866066666666664320e-01 309 | 8.864500000000001823e-01 310 | 8.867300000000000182e-01 311 | 8.868333333333330293e-01 312 | 8.867733333333328583e-01 313 | 8.866699999999989590e-01 314 | 8.868133333333313439e-01 315 | 8.868733333333315150e-01 316 | 8.866833333333318246e-01 317 | 8.864666666666655148e-01 318 | 8.868199999999991645e-01 319 | 8.871499999999987729e-01 320 | 8.870366666666654742e-01 321 | 8.876166666666658323e-01 322 | 8.883166666666663103e-01 323 | 8.885166666666672874e-01 324 | 8.886300000000005861e-01 325 | 8.886200000000011867e-01 326 | 8.885600000000011267e-01 327 | 8.887766666666683246e-01 328 | 8.885866666666667468e-01 329 | 8.885700000000004151e-01 330 | 8.886266666666661207e-01 331 | 8.887166666666643788e-01 332 | 8.889299999999972224e-01 333 | 8.888133333333314567e-01 334 | 8.888233333333308561e-01 335 | 8.887333333333307106e-01 336 | 8.891499999999969983e-01 337 | 8.896299999999958130e-01 338 | 8.897499999999960441e-01 339 | 8.899466666666626669e-01 340 | 8.892733333333296963e-01 341 | 8.890833333333300059e-01 342 | 8.893633333333298419e-01 343 | 8.891399999999975989e-01 344 | 8.890533333333318078e-01 345 | 8.892566666666652520e-01 346 | 8.893999999999986361e-01 347 | 8.894799999999974949e-01 348 | 8.892766666666640507e-01 349 | 8.892099999999970583e-01 350 | 8.891899999999982596e-01 351 | 8.887266666666656656e-01 352 | 8.883133333333319559e-01 353 | 8.881933333333317249e-01 354 | 8.880899999999997130e-01 355 | 8.878599999999986503e-01 356 | 8.881099999999985117e-01 357 | 8.881633333333316394e-01 358 | 8.884399999999990083e-01 359 | 8.886333333333330531e-01 360 | 8.886366666666674075e-01 361 | 8.887400000000014177e-01 362 | 8.890100000000018543e-01 363 | 8.891933333333346123e-01 364 | 8.893966666666680565e-01 365 | 8.896433333333334526e-01 366 | 8.899166666666663561e-01 367 | 8.897633333333335726e-01 368 | 8.899999999999995692e-01 369 | 8.902166666666649908e-01 370 | 8.910433333333325212e-01 371 | 8.912233333333328122e-01 372 | 8.912966666666667370e-01 373 | 8.915066666666670026e-01 374 | 8.914233333333337894e-01 375 | 8.912433333333334984e-01 376 | 8.914866666666664274e-01 377 | 8.918933333333333158e-01 378 | 8.918599999999988759e-01 379 | 8.919333333333326896e-01 380 | 8.918099999999981042e-01 381 | 8.918466666666650111e-01 382 | 8.923599999999983767e-01 383 | 8.925333333333317354e-01 384 | 8.928533333333310562e-01 385 | 8.935033333333308736e-01 386 | 8.937933333333318853e-01 387 | 8.938899999999989632e-01 388 | 8.940866666666654750e-01 389 | 8.937866666666650639e-01 390 | 8.938666666666639227e-01 391 | 8.936233333333309936e-01 392 | 8.935799999999971543e-01 393 | 8.936299999999979260e-01 394 | 8.940199999999985936e-01 395 | 8.944333333333323033e-01 396 | 8.941599999999993997e-01 397 | 8.943900000000003514e-01 398 | 8.943900000000003514e-01 399 | 8.946266666666683465e-01 400 | 8.947833333333334860e-01 401 | 8.950900000000009404e-01 402 | 8.949899999999995082e-01 403 | 8.949299999999993371e-01 404 | 8.951233333333316056e-01 405 | 8.951899999999984869e-01 406 | 8.952266666666655048e-01 407 | 8.952599999999999447e-01 408 | 8.954300000000008364e-01 409 | 8.949966666666682169e-01 410 | 8.952900000000018066e-01 411 | 8.954833333333340750e-01 412 | 8.952933333333343846e-01 413 | 8.953433333333331579e-01 414 | 8.955533333333335344e-01 415 | 8.953366666666682239e-01 416 | 8.951533333333353548e-01 417 | 8.948466666666680114e-01 418 | 8.945733333333351078e-01 419 | 8.948200000000023913e-01 420 | 8.949433333333369767e-01 421 | 8.953866666666707719e-01 422 | 8.953033333333375587e-01 423 | 8.954166666666708574e-01 424 | 8.953366666666701112e-01 425 | 8.951066666666709359e-01 426 | 8.953366666666701112e-01 427 | 8.954033333333352163e-01 428 | 8.952700000000031189e-01 429 | 8.951233333333353803e-01 430 | 8.950533333333358099e-01 431 | 8.947566666666678659e-01 432 | 8.948966666666687830e-01 433 | 8.950666666666677873e-01 434 | 8.954133333333345046e-01 435 | 8.954400000000021231e-01 436 | 8.953333333333356459e-01 437 | 8.953500000000019776e-01 438 | 8.957233333333344261e-01 439 | 8.963466666666666249e-01 440 | 8.966633333333333677e-01 441 | 8.967933333333329982e-01 442 | 8.969466666666657817e-01 443 | 8.967633333333329126e-01 444 | 8.966166666666651741e-01 445 | 8.965433333333313604e-01 446 | 8.964833333333311893e-01 447 | 8.965133333333312748e-01 448 | 8.965599999999975811e-01 449 | 8.965833333333307342e-01 450 | 8.965599999999975811e-01 451 | 8.963566666666641369e-01 452 | 8.963899999999985768e-01 453 | 8.961999999999988864e-01 454 | 8.963266666666659388e-01 455 | 8.962966666666659643e-01 456 | 8.961200000000000276e-01 457 | 8.961266666666669600e-01 458 | 8.960800000000006538e-01 459 | 8.961100000000006283e-01 460 | 8.959366666666672696e-01 461 | 8.960366666666687019e-01 462 | 8.959900000000023956e-01 463 | 8.960866666666693625e-01 464 | 8.960100000000029707e-01 465 | 8.959833333333354632e-01 466 | 8.958933333333353177e-01 467 | 8.957633333333357983e-01 468 | 8.957500000000038209e-01 469 | 8.956900000000037609e-01 470 | 8.955733333333379953e-01 471 | 8.956400000000049877e-01 472 | 8.955400000000054428e-01 473 | 8.957633333333395731e-01 474 | 8.958900000000066255e-01 475 | 8.959866666666737034e-01 476 | 8.958666666666734724e-01 477 | 8.957933333333414350e-01 478 | 8.955333333333422852e-01 479 | 8.954100000000076998e-01 480 | 8.955000000000078453e-01 481 | 8.956633333333400282e-01 482 | 8.954700000000059834e-01 483 | 8.955666666666710629e-01 484 | 8.954700000000040960e-01 485 | 8.955300000000041560e-01 486 | 8.956333333333380553e-01 487 | 8.958100000000058794e-01 488 | 8.960166666666717905e-01 489 | 8.959833333333392380e-01 490 | 8.961733333333389284e-01 491 | 8.958800000000053387e-01 492 | 8.957000000000050477e-01 493 | 8.956900000000056483e-01 494 | 8.952866666666731144e-01 495 | 8.952866666666731144e-01 496 | 8.954800000000072702e-01 497 | 8.955933333333405688e-01 498 | 8.954400000000058979e-01 499 | 8.954900000000065585e-01 500 | 8.954700000000059834e-01 501 | -------------------------------------------------------------------------------- /test_results/ada_time.txt: -------------------------------------------------------------------------------- 1 | 1.716699999999999893e+00 2 | 1.509800000000000031e+00 3 | 1.260499999999999954e+00 4 | 1.044300000000000006e+00 5 | 8.445000000000000284e-01 6 | -------------------------------------------------------------------------------- /test_results/dqn_q_time.txt: -------------------------------------------------------------------------------- 1 | 1.665075149999999837e+00 2 | 1.358924959999999960e+00 3 | 1.120074510000000023e+00 4 | 9.069251000000000396e-01 5 | 7.625851999999999631e-01 6 | -------------------------------------------------------------------------------- /test_results/error10.txt: -------------------------------------------------------------------------------- 1 | 2.900000000000000147e-02 2 | 2.900000000000000147e-02 3 | 2.900000000000000147e-02 4 | 2.900000000000000147e-02 5 | 2.900000000000000147e-02 6 | -------------------------------------------------------------------------------- /test_results/error6.txt: -------------------------------------------------------------------------------- 1 | 6.189000000000000057e-01 2 | 6.189000000000000057e-01 3 | 6.189000000000000057e-01 4 | 6.189000000000000057e-01 5 | 6.189000000000000057e-01 6 | -------------------------------------------------------------------------------- /test_results/error_ada.txt: -------------------------------------------------------------------------------- 1 | 8.538000000000000034e-01 2 | 8.538000000000000034e-01 3 | 8.538000000000000034e-01 4 | 8.538000000000000034e-01 5 | 8.538000000000000034e-01 6 | -------------------------------------------------------------------------------- /test_results/error_dqn.txt: -------------------------------------------------------------------------------- 1 | 2.650000000000000133e-01 2 | 2.650000000000000133e-01 3 | 2.650000000000000133e-01 4 | 2.650000000000000133e-01 5 | 2.650000000000000133e-01 6 | -------------------------------------------------------------------------------- /test_results/fix_10_time.txt: -------------------------------------------------------------------------------- 1 | 1.698046170000000021e+00 2 | 1.379543599999999870e+00 3 | 1.140423699999999929e+00 4 | 9.348789000000000682e-01 5 | 7.856487600000000571e-01 6 | -------------------------------------------------------------------------------- /test_results/fix_6_time.txt: -------------------------------------------------------------------------------- 1 | 1.758999999999999897e+00 2 | 1.553600000000000092e+00 3 | 1.301700000000000079e+00 4 | 1.085599999999999898e+00 5 | 8.842999999999999750e-01 6 | -------------------------------------------------------------------------------- /test_results/longtermreward1.txt: -------------------------------------------------------------------------------- 1 | -5902783.858212 2 | -6471831.556546 3 | -6260678.041225 4 | -5578671.158516 -------------------------------------------------------------------------------- /test_results/loss/ada.txt: -------------------------------------------------------------------------------- 1 | 2.051600000000000090e+00 2 | 1.686513594985008302e+00 3 | 1.530175370971361692e+00 4 | 1.450017154415448539e+00 5 | 1.350965719421704625e+00 6 | 1.327339027325312282e+00 7 | 1.308400927980740924e+00 8 | 1.288746838768323411e+00 9 | 1.257105358242988524e+00 10 | 1.225406175653139806e+00 11 | 1.203822907308737422e+00 12 | 1.183217766483624755e+00 13 | 1.157868157823880528e+00 14 | 1.129216341773668919e+00 15 | 1.107656049629052397e+00 16 | 1.070876245299975160e+00 17 | 1.047954266270001655e+00 18 | 1.034770912925402353e+00 19 | 1.019999846816062927e+00 20 | 1.003867184122403389e+00 21 | 9.903584341208140351e-01 22 | 9.781746506690979226e-01 23 | 9.644709388415019236e-01 24 | 9.517207791407903406e-01 25 | 9.395480136076609368e-01 26 | 9.268739412228266250e-01 27 | 9.149598767360050955e-01 28 | 9.011994709571202744e-01 29 | 8.898176232973734612e-01 30 | 8.798926353454590288e-01 31 | 8.677270034948985078e-01 32 | 8.569826225439707290e-01 33 | 8.470284044742584229e-01 34 | 8.379410525163014656e-01 35 | 8.277714222669600996e-01 36 | 8.169908195734023604e-01 37 | 8.068642646074295488e-01 38 | 7.968440453211466101e-01 39 | 7.865810861190160042e-01 40 | 7.783985058466593721e-01 41 | 7.691858698924382010e-01 42 | 7.603778252998988352e-01 43 | 7.521530618270237945e-01 44 | 7.441862463951111062e-01 45 | 7.368027627468108909e-01 46 | 7.291725496451060273e-01 47 | 7.212782482306162590e-01 48 | 7.125724375247954878e-01 49 | 7.054014374812443977e-01 50 | 6.993739972511927583e-01 51 | 6.925436218579610337e-01 52 | 6.855780909458796257e-01 53 | 6.793333699305852402e-01 54 | 6.715086499849954871e-01 55 | 6.653781155745188691e-01 56 | 6.586717496315638298e-01 57 | 6.520009865363438628e-01 58 | 6.459190179904301665e-01 59 | 6.394263088703155962e-01 60 | 6.316001385450362715e-01 61 | 6.261798232793808205e-01 62 | 6.200910439093907378e-01 63 | 6.147708922624588457e-01 64 | 6.073580344518025864e-01 65 | 6.018880099058151467e-01 66 | 5.961972147226333174e-01 67 | 5.894597172737121582e-01 68 | 5.860466261704763147e-01 69 | 5.817652463912963423e-01 70 | 5.777445177237192642e-01 71 | 5.713467096288998848e-01 72 | 5.672589913010597229e-01 73 | 5.631658335526784631e-01 74 | 5.592002302408218828e-01 75 | 5.551209380229313872e-01 76 | 5.507563506563504241e-01 77 | 5.456574742992719163e-01 78 | 5.388550534844398499e-01 79 | 5.343684022625286856e-01 80 | 5.317593619227409807e-01 81 | 5.285693724950154548e-01 82 | 5.247900187969207764e-01 83 | 5.214969297250111602e-01 84 | 5.174278944730759111e-01 85 | 5.138515561819076982e-01 86 | 5.101238489151000977e-01 87 | 5.057111779848734168e-01 88 | 5.030523757139842012e-01 89 | 4.993179758389790779e-01 90 | 4.950885961453119766e-01 91 | 4.921341588099797382e-01 92 | 4.884493360916773441e-01 93 | 4.838543126980463516e-01 94 | 4.799290498097737445e-01 95 | 4.764034807682037131e-01 96 | 4.738873109221458657e-01 97 | 4.718339060743649838e-01 98 | 4.692564373215039608e-01 99 | 4.655210668841997523e-01 100 | 4.622021848956743839e-01 101 | 4.582482407490412468e-01 102 | 4.549833203355471145e-01 103 | 4.516083811720212338e-01 104 | 4.492009515563646915e-01 105 | 4.460236077507336860e-01 106 | 4.429577618837356456e-01 107 | 4.407865802447001324e-01 108 | 4.388509690761566162e-01 109 | 4.351472596327463638e-01 110 | 4.311698709925015915e-01 111 | 4.291979297995567322e-01 112 | 4.261326119303703308e-01 113 | 4.228003963828086964e-01 114 | 4.195651660362879620e-01 115 | 4.169872442881266128e-01 116 | 4.142409349481264713e-01 117 | 4.126651595036188724e-01 118 | 4.087345853447914013e-01 119 | 4.054015244046846878e-01 120 | 4.028334711988766914e-01 121 | 3.998472496867179760e-01 122 | 3.977863525350888718e-01 123 | 3.954349602262178798e-01 124 | 3.939392983913421742e-01 125 | 3.909833709398905510e-01 126 | 3.897033830483754291e-01 127 | 3.874145671725273132e-01 128 | 3.839526092012723324e-01 129 | 3.811179225643475665e-01 130 | 3.782054965694745530e-01 131 | 3.767887686689694871e-01 132 | 3.741197804609934230e-01 133 | 3.719082728028297313e-01 134 | 3.700152804454167610e-01 135 | 3.677845612168312295e-01 136 | 3.660747145613034714e-01 137 | 3.640744924545288308e-01 138 | 3.634119609991709465e-01 139 | 3.611372619867324607e-01 140 | 3.589702496925989972e-01 141 | 3.562891329328218837e-01 142 | 3.549001330733299042e-01 143 | 3.527428557078043370e-01 144 | 3.509874050915241517e-01 145 | 3.491554388602574610e-01 146 | 3.484775605698426371e-01 147 | 3.468758795062700795e-01 148 | 3.450053149064382096e-01 149 | 3.438933728535968348e-01 150 | 3.423168560065825705e-01 151 | 3.412562814950942935e-01 152 | 3.405603127876917768e-01 153 | 3.382524402843912203e-01 154 | 3.371019167085488877e-01 155 | 3.360832933962344948e-01 156 | 3.350590397660297981e-01 157 | 3.343475822979609102e-01 158 | 3.324629426350196315e-01 159 | 3.321939085572958028e-01 160 | 3.317528365623950704e-01 161 | 3.284877835231025944e-01 162 | 3.283696298268238833e-01 163 | 3.273750924270351637e-01 164 | 3.262932959450284831e-01 165 | 3.252021728375554255e-01 166 | 3.231470614995161639e-01 167 | 3.220989261909326107e-01 168 | 3.202764294515053622e-01 169 | 3.194653940439224349e-01 170 | 3.188165755467712859e-01 171 | 3.178045882637302033e-01 172 | 3.167341069641709250e-01 173 | 3.158202315185288378e-01 174 | 3.148271263814181320e-01 175 | 3.135935089028477618e-01 176 | 3.120256491446152247e-01 177 | 3.129813443517257543e-01 178 | 3.115245633958528271e-01 179 | 3.102584339141498115e-01 180 | 3.108909322259699182e-01 181 | 3.093692479085674019e-01 182 | 3.084591375449051709e-01 183 | 3.064402645083765253e-01 184 | 3.074281266735494067e-01 185 | 3.064187644222627349e-01 186 | 3.054085765008380249e-01 187 | 3.038547300919890448e-01 188 | 3.038836313754320062e-01 189 | 3.023982291911045950e-01 190 | 3.014180883870025696e-01 191 | 3.004079023394733428e-01 192 | 2.993950060185789908e-01 193 | 2.987628516430655967e-01 194 | 2.996244384713471209e-01 195 | 2.975373415095359242e-01 196 | 2.964149407739440356e-01 197 | 2.952703349195420768e-01 198 | 2.933688568534950636e-01 199 | 2.934679162430266497e-01 200 | 2.923256658763339066e-01 201 | 2.917605725859602206e-01 202 | 2.929544189876814864e-01 203 | 2.927363497861474873e-01 204 | 2.912626331502075061e-01 205 | 2.903621991691490156e-01 206 | 2.890160687106102522e-01 207 | 2.880601860389113700e-01 208 | 2.895973307440678490e-01 209 | 2.903859005586306163e-01 210 | 2.910757214126487580e-01 211 | 2.901855412017802505e-01 212 | 2.892654230633378010e-01 213 | 2.895275586446126037e-01 214 | 2.880852317801863016e-01 215 | 2.879506724936266582e-01 216 | 2.867917280028263738e-01 217 | 2.865485236570238792e-01 218 | 2.884694410959879329e-01 219 | 2.876146699749926872e-01 220 | 2.884046100345253882e-01 221 | 2.862944681015362480e-01 222 | 2.874844560078282929e-01 223 | 2.865435517246524566e-01 224 | 2.854284461413820750e-01 225 | 2.842416143074631418e-01 226 | 2.830207526393731521e-01 227 | 2.830069802597860451e-01 228 | 2.825937594865759461e-01 229 | 2.838420372456312291e-01 230 | 2.810720239922404051e-01 231 | 2.805600561648607383e-01 232 | 2.790596492091814707e-01 233 | 2.785834623257318898e-01 234 | 2.763173476407925211e-01 235 | 2.751031150619188681e-01 236 | 2.750039036815364879e-01 237 | 2.741805372754732573e-01 238 | 2.730121983587741608e-01 239 | 2.726382370505730512e-01 240 | 2.704949858556190967e-01 241 | 2.702844216848412917e-01 242 | 2.690736977934837237e-01 243 | 2.686167030025521929e-01 244 | 2.672620063001910906e-01 245 | 2.651350217773020268e-01 246 | 2.645442193925380825e-01 247 | 2.633972671131292742e-01 248 | 2.623436733583609515e-01 249 | 2.612644051164388914e-01 250 | 2.601611920992533133e-01 251 | 2.601219069262345474e-01 252 | 2.590573909580707324e-01 253 | 2.585977504069606470e-01 254 | 2.575913481727242216e-01 255 | 2.564757572839657662e-01 256 | 2.543918333214521543e-01 257 | 2.532858044079939752e-01 258 | 2.527537208482623088e-01 259 | 2.522468013773361828e-01 260 | 2.516438804815213093e-01 261 | 2.504298098911841852e-01 262 | 2.493349372396866381e-01 263 | 2.512422307729720861e-01 264 | 2.520946938395500214e-01 265 | 2.519263764222462454e-01 266 | 2.508111644387245320e-01 267 | 2.559558717409769479e-01 268 | 2.548185922851164964e-01 269 | 2.546220603734254961e-01 270 | 2.546177541365226005e-01 271 | 2.536995755682389109e-01 272 | 2.536346810112397043e-01 273 | 2.525773251752058690e-01 274 | 2.505400756746530289e-01 275 | 2.515143516014019531e-01 276 | 2.490518586312731042e-01 277 | 2.484125307748715206e-01 278 | 2.484279916236797892e-01 279 | 2.483673794120550282e-01 280 | 2.472476772914330156e-01 281 | 2.461891261090835004e-01 282 | 2.441207458972931010e-01 283 | 2.420424999992052684e-01 284 | 2.419950199152032522e-01 285 | 2.409173704783121872e-01 286 | 2.398823759456475690e-01 287 | 2.389815576101342876e-01 288 | 2.379724962313969905e-01 289 | 2.377687170108159498e-01 290 | 2.369713764339685391e-01 291 | 2.369700912162661655e-01 292 | 2.356548804144064646e-01 293 | 2.336570890148480795e-01 294 | 2.311556335161129572e-01 295 | 2.318946360444029131e-01 296 | 2.307938175755242516e-01 297 | 2.316927974464992601e-01 298 | 2.326165901119510437e-01 299 | 2.315109190081556523e-01 300 | 2.324904615042110156e-01 301 | 2.339170530935128500e-01 302 | 2.302889452782770074e-01 303 | 2.313890997014939888e-01 304 | 2.293437719146410592e-01 305 | 2.313006780227025438e-01 306 | 2.302975954953332682e-01 307 | 2.312870381961266153e-01 308 | 2.293264068749050200e-01 309 | 2.298614065845807353e-01 310 | 2.318568593834837155e-01 311 | 2.317473487198352944e-01 312 | 2.308844532390435522e-01 313 | 2.296840441840390379e-01 314 | 2.338335118070244689e-01 315 | 2.323837560800214586e-01 316 | 2.303683782200763586e-01 317 | 2.324036136075357528e-01 318 | 2.313829681113362402e-01 319 | 2.293239238510529077e-01 320 | 2.296210047235091589e-01 321 | 2.310203850925763580e-01 322 | 2.328172178789973124e-01 323 | 2.313137621631224938e-01 324 | 2.335869421561546089e-01 325 | 2.328115658089518480e-01 326 | 2.328039588431517204e-01 327 | 2.337930953949689805e-01 328 | 2.327983623271187019e-01 329 | 2.318004704241951319e-01 330 | 2.300432097613811511e-01 331 | 2.302313689986864587e-01 332 | 2.298916694382826531e-01 333 | 2.287852755412459471e-01 334 | 2.285235995650290453e-01 335 | 2.278265806535879623e-01 336 | 2.267716398735841188e-01 337 | 2.257809040745099305e-01 338 | 2.256811331510543694e-01 339 | 2.256666487008333333e-01 340 | 2.266368608574072518e-01 341 | 2.252176711483548088e-01 342 | 2.261875006059805571e-01 343 | 2.276652713865041644e-01 344 | 2.261775610347588616e-01 345 | 2.259048648178577368e-01 346 | 2.253094149132569557e-01 347 | 2.245711746315161295e-01 348 | 2.252483357588450086e-01 349 | 2.257341715544462224e-01 350 | 2.267298488641778520e-01 351 | 2.272350492576758196e-01 352 | 2.272212414195140251e-01 353 | 2.272167467325925949e-01 354 | 2.287267259334524461e-01 355 | 2.282708037743965690e-01 356 | 2.279851339727640147e-01 357 | 2.287212220033009724e-01 358 | 2.272318270305792587e-01 359 | 2.271599676460027695e-01 360 | 2.261028391520182312e-01 361 | 2.262746943036715030e-01 362 | 2.270546697080135234e-01 363 | 2.269380653649568647e-01 364 | 2.264241891304651921e-01 365 | 2.261460231939951548e-01 366 | 2.263200084169705584e-01 367 | 2.256896831045548113e-01 368 | 2.253116423587004369e-01 369 | 2.267676104853550700e-01 370 | 2.256726028919220084e-01 371 | 2.267485366761684273e-01 372 | 2.276732349296410884e-01 373 | 2.256683456897735640e-01 374 | 2.266505449016888996e-01 375 | 2.256623746131857233e-01 376 | 2.255809606760740271e-01 377 | 2.260377211471398529e-01 378 | 2.256448609009385209e-01 379 | 2.246475707193215654e-01 380 | 2.256400836954514100e-01 381 | 2.246359697530667110e-01 382 | 2.243026499748229929e-01 383 | 2.245273912613590661e-01 384 | 2.250626275102297347e-01 385 | 2.246224114199479316e-01 386 | 2.236253522460659449e-01 387 | 2.232626125104725245e-01 388 | 2.230619617546598099e-01 389 | 2.226190287222464814e-01 390 | 2.230949265211820687e-01 391 | 2.225373785595099185e-01 392 | 2.222979405889908555e-01 393 | 2.220962653433283251e-01 394 | 2.218590150487919632e-01 395 | 2.214584506603578806e-01 396 | 2.216573534620304986e-01 397 | 2.215664444615443596e-01 398 | 2.210566044134398278e-01 399 | 2.208563267752528259e-01 400 | 2.206560070601602530e-01 401 | 2.202556734773019886e-01 402 | 2.201556459816793598e-01 403 | 2.200545128288368402e-01 404 | 2.200425421769420242e-01 405 | 2.214083702713251223e-01 406 | 2.218542323542137817e-01 407 | 2.207543566651642419e-01 408 | 2.194042258113622634e-01 409 | 2.193530862927436931e-01 410 | 2.195431432624657953e-01 411 | 2.205439198017120350e-01 412 | 2.208541629783809246e-01 413 | 2.217537927063802927e-01 414 | 2.205318021650115590e-01 415 | 2.214531481049955008e-01 416 | 2.202528062805533537e-01 417 | 2.208900605936845174e-01 418 | 2.201530048775176274e-01 419 | 2.205329263905684289e-01 420 | 2.208526878473659472e-01 421 | 2.211527822176615476e-01 422 | 2.205307267159223594e-01 423 | 2.209535742419461435e-01 424 | 2.205307657370964569e-01 425 | 2.198528112858533778e-01 426 | 2.201527925729751534e-01 427 | 2.205221670443812954e-01 428 | 2.212093862245480269e-01 429 | 2.205219698746999080e-01 430 | 2.202012711018323798e-01 431 | 2.219245284299055931e-01 432 | 2.209377390543619812e-01 433 | 2.202461636563142200e-01 434 | 2.213103092312812825e-01 435 | 2.209361739754676923e-01 436 | 2.203123781830072514e-01 437 | 2.213133579343557478e-01 438 | 2.200638657510280705e-01 439 | 2.215235802754759731e-01 440 | 2.205288855507969992e-01 441 | 2.195253098532557456e-01 442 | 2.204522815863291518e-01 443 | 2.212473235130310023e-01 444 | 2.209520797533293668e-01 445 | 2.207521668665111125e-01 446 | 2.203513581423709733e-01 447 | 2.198509751504907905e-01 448 | 2.195031189384559767e-01 449 | 2.204962987961868459e-01 450 | 2.209457712098956139e-01 451 | 2.214527808626491068e-01 452 | 2.203504714875171577e-01 453 | 2.200501981868098200e-01 454 | 2.196504472468048252e-01 455 | 2.190272798165678880e-01 456 | 2.207501902357985601e-01 457 | 2.204988721075157310e-01 458 | 2.205031855838994193e-01 459 | 2.215008446089923311e-01 460 | 2.219640488003691015e-01 461 | 2.213489196264495362e-01 462 | 2.206487053825209599e-01 463 | 2.211661993761857847e-01 464 | 2.204884266865750064e-01 465 | 2.206485853044937162e-01 466 | 2.193479939827074621e-01 467 | 2.194763181135058472e-01 468 | 2.197472877815365710e-01 469 | 2.199470820250610525e-01 470 | 2.206460729305942947e-01 471 | 2.204557685976227255e-01 472 | 2.218449288137257225e-01 473 | 2.202442819091181003e-01 474 | 2.214419479618469910e-01 475 | 2.204437473069876530e-01 476 | 2.209437671540925852e-01 477 | 2.200436608223865498e-01 478 | 2.203426281653344532e-01 479 | 2.201427992716431525e-01 480 | 2.208433921312292492e-01 481 | 2.214281846707066015e-01 482 | 2.191105449398358662e-01 483 | 2.198413246322423187e-01 484 | 2.201154690956076132e-01 485 | 2.211298090790708715e-01 486 | 2.217417300503700950e-01 487 | 2.214419630096604308e-01 488 | 2.216425370453546373e-01 489 | 2.214428290875007643e-01 490 | 2.203433409476031890e-01 491 | 2.201433973919600384e-01 492 | 2.199433108025540817e-01 493 | 2.197430786533902114e-01 494 | 2.204279232211411033e-01 495 | 2.212383999750018038e-01 496 | 2.207416920772443114e-01 497 | 2.214420514664302286e-01 498 | 2.203421730626374531e-01 499 | 2.198422212236871287e-01 500 | 2.207415379883100603e-01 501 | -------------------------------------------------------------------------------- /test_results/qerror1.txt: -------------------------------------------------------------------------------- 1 | 0.029 2 | 0.6189 3 | 0.8538 4 | 0.265 -------------------------------------------------------------------------------- /test_results/reward10.txt: -------------------------------------------------------------------------------- 1 | -8.435230849999999947e+00 2 | -7.147718000000000238e+00 3 | -5.802118499999999734e+00 4 | -4.774394499999999653e+00 5 | -3.923243800000000281e+00 6 | -------------------------------------------------------------------------------- /test_results/reward6.txt: -------------------------------------------------------------------------------- 1 | -8.845000000000000639e+00 2 | -7.767999999999999794e+00 3 | -6.508499999999999730e+00 4 | -5.427999999999999936e+00 5 | -4.416500000000000092e+00 6 | -------------------------------------------------------------------------------- /test_results/rewardada.txt: -------------------------------------------------------------------------------- 1 | -8.643499999999999517e+00 2 | -7.549000000000000377e+00 3 | -6.260678000000000409e+00 4 | -5.221499999999999808e+00 5 | -4.217500000000000249e+00 6 | -------------------------------------------------------------------------------- /test_results/rewarddqn.txt: -------------------------------------------------------------------------------- 1 | -8.220375749999998760e+00 2 | -6.794624800000000242e+00 3 | -5.578671150000000800e+00 4 | -4.534625499999999754e+00 5 | -3.807926000000000144e+00 6 | -------------------------------------------------------------------------------- /test_results/traintime1.txt: -------------------------------------------------------------------------------- 1 | 11805.5677 2 | 12943.663 3 | 12521.356 4 | 11157.3423 5 | -------------------------------------------------------------------------------- /train/a1bathnormal_env_max_ep2000_tep_1000_buffersize250000_maxstep500000_prestep25000_lr0.05_plot_minig+T_tfacter0.5_num4.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/train/a1bathnormal_env_max_ep2000_tep_1000_buffersize250000_maxstep500000_prestep25000_lr0.05_plot_minig+T_tfacter0.5_num4.npz -------------------------------------------------------------------------------- /train/a2bathnormal_env_max_ep2000_tep_1000_buffersize250000_maxstep500000_prestep25000_lr0.05_plot_minig+T_tfacter0.5_num4.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/train/a2bathnormal_env_max_ep2000_tep_1000_buffersize250000_maxstep500000_prestep25000_lr0.05_plot_minig+T_tfacter0.5_num4.npz -------------------------------------------------------------------------------- /train/a3bathnormal_env_max_ep2000_tep_1000_buffersize250000_maxstep500000_prestep25000_lr0.05_plot_minig+T_tfacter0.5_num4.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qiongwu86/Distributed-DRL-Based-Gradient-Quantization-for-Federated-Learning-Enabled-VEC/21594d7067058dd8bce7886581bf5f50c1709cc7/train/a3bathnormal_env_max_ep2000_tep_1000_buffersize250000_maxstep500000_prestep25000_lr0.05_plot_minig+T_tfacter0.5_num4.npz --------------------------------------------------------------------------------