├── README.md ├── part_1_feature_difference ├── README ├── model_d.py ├── model_d_2ch.py ├── model_d_map32.py ├── model_d_map32_bn_diff.py ├── model_d_map32_diff.py ├── model_d_map32_no.py ├── model_d_tensor.py ├── model_d_tensor_no.py ├── model_gd.py ├── read_data.py ├── read_data_map.py ├── train_nature.py ├── train_nature_2ch.py ├── train_nature_map32.py ├── train_nature_map32_diff.py ├── train_nature_map32_no.py ├── train_nature_tensor.py └── train_nature_tensor_no.py ├── part_2_element-wise_feature_relation ├── README ├── model_d_map32_diff_res.py ├── model_d_map32_diff_res_pro.py ├── model_d_map32_res.py ├── read_data.py ├── read_data_map.py ├── train_nature_map32_diff_res.py ├── train_nature_map32_diff_res_pro.py └── train_nature_map32_res.py └── part_3_GAN_transform ├── README ├── model_d_map32.py ├── model_d_map32_diff.py ├── model_d_tensor.py ├── model_gd_diff.py ├── model_gd_diff_pro.py ├── model_gd_nir.py ├── read_data.py ├── read_data_map.py ├── train_gd_gray2nir.py ├── train_gd_gray2nir_pro.py ├── train_gd_nir2nir.py ├── train_nature_map_gd.py └── train_nature_map_gd_pro.py /README.md: -------------------------------------------------------------------------------- 1 | ## AFD-Net_Aggregated_Feature_Difference_Learning 2 | **（1）paper：Element-wise Feature Relation Learning Network for Cross-Spectral Image Patch Matching（IEEE Transactions on Neural Networks and Learning Systems 2021.02.05已发表）** [中科院一区期刊论文地址](https://ieeexplore.ieee.org/abstract/document/9349201) 3 | **（2）该期刊论文的核心创新点是本人研究生毕业论文《*基于特征残差学习和图像转换的异源图像块匹配方法研究*》的第四章内容。** [研究生毕业论文地址](https://cdmd.cnki.com.cn/Article/CDMD-10701-1020004804.htm) 4 | **（3）该创新点在2019年被ICCV（计算机视觉方向的三大顶级会议之一）录用为口头报告(Oral)论文，而ICCV 2019的Oral录用比例仅为4.62%。** [ICCV录用论文地址](https://openaccess.thecvf.com/content_ICCV_2019/html/Quan_AFD-Net_Aggregated_Feature_Difference_Learning_for_Cross-Spectral_Image_Patch_Matching_ICCV_2019_paper.html) 5 | 6 | ## 论文创新点： 7 | （1）基于特征图差值融合的异源图像块匹配方法。针对异源图像提取的特征信息表达能力不足的问题，通过特征图融合的方式将**空间邻域信息引入到图像特征描述子中**，充分提取利用图像的有效信息。为解决特征描述子中存在的鉴别性差异信息容易被忽视的问题，本文在特征图融合方法的基础上进行改进，利用**特征图组求差值的方式得到残差特征图，之后进行匹配度量**。该策略不仅提高了网络对重要匹配特征的关注力度，而且增加了网络自身的稀疏性，提高了匹配准确率和网络健壮性。 8 | （2）基于特征残差学习的异源图像块匹配方法。通过对基于特征图差值融合方法的分析发现，差异特征信息在图像匹配中十分重要。针对图像的高层残差特征中会缺失重要细节信息的问题，通过增加副分支卷积网络的方式，对主干网络中不同卷积层得到的多个尺度残差特征图组进行特征学习。该策略将**底层的细节残差特征与高层的语义残差特征进行结合，充分挖掘图像中有利于匹配的重要特征信息**。在此基础上，本文进一步将主干网络与副分支网络得到的高层残差特征进行组合学习，利用三次度量的方式来提高匹配预测的准确率和泛化性能。 9 | （3）基于图像转换的异源图像块匹配方法。针对异源图像之间存在的表征差异性问题，利用生成对抗网络中的**生成器将异源图像转换为同源图像，之后对同源图像进行相似性度量**，以此降低匹配难度。为改善生成的转换图像质量及多样性，通过预训练的**卷积自编码器来正则约束判别器网络的训练**，从而间接强化生成器的稳定训练。图像转换策略克服了图像之间由于表征差异性大导致的匹配困难问题，为异源图像匹配提供了全新的解决思路。 10 | 11 | ## 概述： 12 | **1、** *背景：* 13 | 图像匹配一直是计算机视觉领域中的研究重点，在图像检索、图像配准、目标跟踪等诸多领域有着广泛的应用。异源图像包含的丰富互补信息，对于目标的全面、准确认识十分重要。但是**由于不同的成像机理和条件导致异源图像之间存在明显的表征差异性，使得同源图像的匹配方法不能适用于异源图像**，因此针对异源图像的匹配方法研究具有非常重要的意义。随着深度学习的飞速发展，基于卷积神经网络的图像匹配方法不断涌现出来。论文在总结现有图像匹配方法的基础上，将**深度卷积网络和生成对抗网络**应用于异源图像的匹配中。 14 | 15 | **2、** *主要工作总结：* 16 | （1）基于特征图差值融合的异源图像块匹配方法。基于特征图融合的方法是在特征提取阶段，通过特征图融合的方式有效保留了图像对的空间特征信息，显著提高了匹配预测的准确率。之后，在基于特征图融合方法的基础上，利用图像对的残差特征图作为匹配度量网络的输入数据，并以此判断图像对的匹配程度，提出了基于特征图差值融合的方法。该方法最大的创新点是，使用的**特征图差值融合策略不仅保留了图像的空间特征信息，而且提高了匹配网络对重要差异特征的关注力度**。最后，通过多组对比实验证明了，改进方法在增加网络自身稀疏性的同时，提高了匹配的准确率和网络的泛化性。 17 | （2）基于特征残差学习的异源图像块匹配方法。该创新点是从统计学的梯度提升树和深度学习的残差网络中获得灵感，将**残差学习**的思想引入到之前提出的基于特征图差值融合的异源图像匹配方法中，通过**增加一路副分支网络来专门学习主干网络中不同卷积层得到的多个尺度残差特征图的信息**，目的是将底层的细节残差特征与高层的语义残差特征进行结合，重点关注图像中具有鉴别性的差异特征信息，从而提出了基于特征残差学习的异源图像匹配方法。另外，论文在改进方法的基础上，将主干网络提取的残差特征信息与副分支网络经过多层学习得到的特征图残差信息进行融合学习，提出了**基于三次度量的匹配方法**，进一步提升了异源图像匹配的准确率。 18 | （3）基于图像转换的异源图像块匹配方法。从不同角度出发来思考解决异源图像的匹配问题，**利用生成器将表征差异较大的异源图像转换为同源图像后进行匹配度量，图像转换思想是最大的一个创新点**。该方法利用生成对抗网络中的生成器将异源图像转换为同源图像，之后对同源图像进行匹配预测，提出了基于图像转换的匹配方法。该方法通过图像转换策略降低了匹配难度，为异源图像匹配问题提供了全新的解决思路。由于生成的转换图像质量及多样性将直接影响之后匹配网络的正确判断，因此本章通过预训练的卷积自编码器来正则约束判别器网络的训练，从而间接强化生成器的稳定训练，提出了**基于正则生成对抗网络的匹配方法**，最后通过对比实验验证了图像转换策略以及正则约束策略在异源图像匹配上的有效性。 19 | 20 | **3、** *主要展望：* 21 | 本文主要是研究基于卷积神经网络和生成对抗网络的异源图像匹配方法，在特征提取、匹配度量和图像块转换等方面进行改进，提出了创新的解决方案，并取得了一些研究成果。但是，仍然存在很多需要改进和提升的方面。 22 | （1）本文的改进方法可以显著提高异源图像匹配的准确率，然而在网络的复杂度和计算量方面没有明显的改善。因此可以考虑在保证匹配准确率的前提下，简化网络结构，提升异源图像匹配预测的效率。 23 | （2）本文的创新方法在不同测试集上的表现不尽相同，这主要是由测试集与训练集数据分布的一致性程度决定的。因此，如何提高匹配方法的泛化能力，在多种测试集上都取得较好的匹配效果是接下来需要研究的重要问题。 24 | （3）本文初步验证了图像转换策略在异源图像匹配上的有效性。但由于转换图像的质量直接影响之后图像匹配的正确判断，因此如何设计良好的生成对抗网络结构，优化生成器的综合性能来提高生成的转换图像质量及多样性，是未来需要重点研究和解决的问题。 25 | -------------------------------------------------------------------------------- /part_1_feature_difference/README: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /part_1_feature_difference/model_d.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on Sat Mar 3 11:03:33 2018 4 | 5 | @author: Administrator 6 | """ 7 | 8 | import tensorflow as tf 9 | import numpy as np 10 | 11 | EPS = 1e-12 12 | 13 | def conv(batch_input, out_channels, filter_size, pad, stride=1): 14 | conv = tf.layers.conv2d(batch_input, out_channels,filter_size,strides=(stride,stride), padding=pad) 15 | return conv 16 | 17 | def batchnorm(inputs): 18 | normalized = tf.layers.batch_normalization(inputs, axis=-1) 19 | return normalized 20 | 21 | def evaluation(logits, labels): 22 | correct_prediction = tf.equal(logits, labels) 23 | correct_batch = tf.reduce_mean(tf.cast(correct_prediction, tf.int32), 1) 24 | return tf.reduce_sum(tf.cast(correct_batch, tf.float32)), correct_batch 25 | 26 | def features(img, name, reuse=False): 27 | with tf.variable_scope(name,reuse=reuse): 28 | layer1 = conv(img, 32, 3, pad='SAME') 29 | layer1 = tf.nn.relu(layer1) 30 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 31 | 32 | layer2 = conv(pool1, 64, 3, pad='SAME') 33 | layer2 = tf.nn.relu(layer2) 34 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 35 | 36 | layer3 = conv(pool2, 128, 3, pad='SAME') 37 | layer3 = tf.nn.relu(layer3) 38 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 39 | 40 | layer4 = conv(pool3, 256, 3, pad='SAME') 41 | layer4 = tf.nn.relu(layer4) 42 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 43 | # 4*4*256=4096 44 | # batch_size1 = int(pool4.get_shape()[0]) 45 | # dense = tf.reshape(pool4, [batch_size1,-1]) 46 | 47 | return [layer1, pool1, pool2, pool3, pool4], pool4 48 | 49 | # 4conv + 3fc 50 | def matching(f1,f2,keep_prob): 51 | with tf.variable_scope('match_layers'): 52 | all_feature = tf.concat([f1, f2], axis=3) # 两个特征连接 4*4*512 53 | layer5 = conv(all_feature, 512, 3, pad='SAME') 54 | layer5 = tf.nn.relu(layer5) 55 | pool5 = tf.nn.max_pool(layer5,[1,2,2,1],[1,2,2,1],padding='SAME') 56 | 57 | batch_size1 = int(pool5.get_shape()[0]) 58 | dense = tf.reshape(pool5, [batch_size1,-1]) 59 | dense1 = tf.layers.dense(dense, 1024) 60 | dense1 = tf.nn.relu(dense1) 61 | # dropout layers 62 | dense1 = tf.nn.dropout(dense1, keep_prob) 63 | 64 | dense2 = tf.layers.dense(dense1, 256) 65 | dense2 = tf.nn.relu(dense2) 66 | dense2 = tf.nn.dropout(dense2, keep_prob) 67 | 68 | dense3 = tf.layers.dense(dense2, 1) 69 | output = tf.sigmoid(dense3) 70 | return output, [pool5,dense1,dense2] 71 | 72 | # 双分支特征向量, tensor = 3conv + 2fc 73 | def features_matching(inputs_p1, inputs_p2,keep_prob): 74 | with tf.variable_scope("matching"): 75 | layers_1,f1 = features(inputs_p1, 'inputs_p') # 4*4*256 76 | layers_2,f2 = features(inputs_p2, 'inputs_p',reuse=True) # 4*4*256 77 | output,denses = matching(f1,f2,keep_prob) 78 | return output ,[layers_1,f1,layers_2,f2,denses] 79 | 80 | # 3conv + 5fc 81 | def matching_res(layer1_res, pool1_res, pool2_res, pool3_res, pool4_res,keep_prob): 82 | 83 | layer1 = conv(layer1_res, 32, 1, pad='SAME') 84 | # layer1 = batchnorm(layer1) 85 | layer1 = tf.nn.relu(layer1) 86 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 87 | 88 | pool1_con = tf.concat([pool1, pool1_res], axis=3) 89 | layer2 = conv(pool1_con, 64, 1, pad='SAME') 90 | # layer2 = batchnorm(layer2) 91 | layer2 = tf.nn.relu(layer2) 92 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 93 | 94 | pool2_con = tf.concat([pool2, pool2_res], axis=3) 95 | layer3 = conv(pool2_con, 128, 1, pad='SAME') 96 | # layer3 = batchnorm(layer3) 97 | layer3 = tf.nn.relu(layer3) 98 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 99 | 100 | pool3_con = tf.concat([pool3, pool3_res], axis=3) 101 | layer4 = conv(pool3_con, 256, 1, pad='SAME') 102 | # layer3 = batchnorm(layer3) 103 | layer4 = tf.nn.relu(layer4) 104 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 105 | 106 | pool4_con = tf.concat([pool4, pool4_res], axis=3) 107 | layer5 = conv(pool4_con, 512, 1, pad='SAME') 108 | layer5 = tf.nn.relu(layer5) 109 | pool5 = tf.nn.max_pool(layer5,[1,2,2,1],[1,2,2,1],padding='SAME') # 2*2*512 110 | 111 | # 2*2*512=2048 112 | batch_size1 = int(pool5.get_shape()[0]) 113 | dense = tf.reshape(pool5, [batch_size1,-1]) 114 | 115 | dense1 = tf.layers.dense(dense, 1024) 116 | dense1 = tf.nn.relu(dense1) 117 | # dropout layers 118 | dense1 = tf.nn.dropout(dense1, keep_prob) 119 | 120 | dense2 = tf.layers.dense(dense1, 256) 121 | dense2 = tf.nn.relu(dense2) 122 | dense2 = tf.nn.dropout(dense2, keep_prob) 123 | 124 | dense3 = tf.layers.dense(dense2, 1) 125 | output = tf.sigmoid(dense3) 126 | return output,[pool1, pool2, pool3, pool4, pool5, dense1,dense2] 127 | 128 | # 双分支特征向量,参差损失, tensor = 3conv + 2fc 129 | def features_matching_res(inputs_p1, inputs_p2, keep_prob): 130 | with tf.variable_scope("matching"): 131 | layers_1,f1 = features(inputs_p1, 'inputs_p') # 1024 132 | layers_2,f2 = features(inputs_p2, 'inputs_p',reuse=True) # 1024 133 | output ,denses = matching(f1,f2,keep_prob) 134 | with tf.variable_scope('match_res'): 135 | output_res, res_features = matching_res(abs(layers_1[0]-layers_2[0]), abs(layers_1[1]-layers_2[1]), abs(layers_1[2]-layers_2[2]), 136 | abs(layers_1[3]-layers_2[3]), abs(layers_1[4]-layers_2[4]),keep_prob) 137 | return output, output_res, [layers_1, f1, layers_2, f2, denses, res_features] 138 | 139 | # 双分支特征向量,共享参数 140 | def matchLoss(inputs_p1, inputs_p2, label_m, keep_prob): 141 | # 匹配损失 142 | m_output, all_features = features_matching(inputs_p1, inputs_p2,keep_prob) 143 | match_loss = tf.reduce_mean(-(label_m * tf.log(m_output + EPS) + (1 - label_m) * tf.log(1 - m_output + EPS))) 144 | return match_loss, m_output, all_features 145 | 146 | # 双分支特征向量,共享参数 147 | def matchLoss_res(inputs_p1, inputs_p2, label_m, keep_prob): 148 | # 匹配损失 149 | m_output, m_output_res, all_features = features_matching_res(inputs_p1, inputs_p2, keep_prob) 150 | match_loss = tf.reduce_mean(-(label_m * tf.log(m_output + EPS) + (1 - label_m) * tf.log(1 - m_output + EPS))) 151 | match_loss_res = tf.reduce_mean(-(label_m * tf.log(m_output_res + EPS) + (1 - label_m) * tf.log(1 - m_output_res + EPS))) 152 | return match_loss,match_loss_res, m_output, m_output_res, all_features 153 | 154 | 155 | 156 | 157 | 158 | 159 | def join_model(inputs_p1, inputs_p2, label_m, inputs_p, label_c, points_num): 160 | # 匹配损失 161 | m_output, f1, f2 = features_matching(inputs_p1, inputs_p2) 162 | match_loss = tf.reduce_mean(-(label_m * tf.log(m_output + EPS) + (1 - label_m) * tf.log(1 - m_output + EPS))) 163 | 164 | # 分类损失 165 | c_output, f = classifier(inputs_p, points_num) 166 | 167 | class_loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=label_c,logits=c_output)) 168 | 169 | return match_loss, class_loss, m_output 170 | 171 | 172 | 173 | 174 | 175 | 176 | 177 | -------------------------------------------------------------------------------- /part_1_feature_difference/model_d_2ch.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on 2018.11.9 11:03:33 4 | 5 | @author: Administrator 6 | 7 | 2 channel + 交叉熵损失函数 + 参数共享 8 | 9 | """ 10 | 11 | import tensorflow as tf 12 | import numpy as np 13 | 14 | EPS = 1e-12 15 | 16 | def conv(batch_input, out_channels, filter_size, pad, stride=1): 17 | conv = tf.layers.conv2d(batch_input, out_channels,filter_size,strides=(stride,stride), padding=pad) 18 | return conv 19 | 20 | def batchnorm(inputs): 21 | normalized = tf.layers.batch_normalization(inputs, axis=-1) 22 | return normalized 23 | 24 | def evaluation(logits, labels): 25 | correct_prediction = tf.equal(logits, labels) 26 | correct_batch = tf.reduce_mean(tf.cast(correct_prediction, tf.int32), 1) 27 | return tf.reduce_sum(tf.cast(correct_batch, tf.float32)), correct_batch 28 | 29 | 30 | ''' 31 | network model 32 | ''' 33 | def features_matching(inputs_p1, inputs_p2): 34 | with tf.variable_scope("matching_network"): 35 | inputs = tf.concat([inputs_p1, inputs_p2], axis=3) # 64*64*2 36 | layer1 = conv(inputs, 32, 3, pad='SAME') # 64*64*32 37 | layer1 = tf.nn.relu(layer1) 38 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 39 | 40 | layer2 = conv(pool1, 64, 3, pad='SAME') # 32*32*64 41 | layer2 = tf.nn.relu(layer2) 42 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 43 | 44 | layer3 = conv(pool2, 128, 3, pad='SAME') # 16*16*128 45 | layer3 = tf.nn.relu(layer3) 46 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 47 | 48 | layer4 = conv(pool3, 256, 3, pad='SAME') # 8*8*256 49 | layer4 = tf.nn.relu(layer4) 50 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 51 | 52 | layer5 = conv(pool4, 512, 3, pad='SAME') 53 | layer5 = tf.nn.relu(layer5) 54 | pool5 = tf.nn.max_pool(layer5,[1,2,2,1],[1,2,2,1],padding='SAME') # 2*2*512 55 | 56 | batch_size = int(pool5.get_shape()[0]) 57 | dense = tf.reshape(pool5, [batch_size,-1]) # reshape as a tensor 2048 58 | dense1 = tf.layers.dense(dense, 1024) 59 | dense1 = tf.nn.relu(dense1) 60 | dense2 = tf.layers.dense(dense1, 256) 61 | dense2 = tf.nn.relu(dense2) 62 | dense3 = tf.layers.dense(dense2, 1) 63 | output = tf.sigmoid(dense3) 64 | 65 | return output, [pool1, pool2, pool3, pool4, pool5, dense1, dense2] 66 | 67 | ''' 68 | cross loss function 69 | ''' 70 | 71 | def match_network(inputs_p1, inputs_p2, label_m): 72 | 73 | match_output, all_features = features_matching(inputs_p1, inputs_p2) 74 | match_loss = tf.reduce_mean(-(label_m * tf.log(match_output + EPS) + (1 - label_m) * tf.log(1 - match_output + EPS))) 75 | return match_loss, match_output, all_features 76 | 77 | 78 | 79 | 80 | 81 | 82 | -------------------------------------------------------------------------------- /part_1_feature_difference/model_d_map32.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on 2018.11.9 11:03:33 4 | 5 | @author: Administrator 6 | 7 | 双分支 + 特征map + 交叉熵损失函数 + 参数共享 8 | 9 | """ 10 | 11 | import tensorflow as tf 12 | import numpy as np 13 | 14 | EPS = 1e-12 15 | 16 | def conv(batch_input, out_channels, filter_size, pad, stride=1): 17 | conv = tf.layers.conv2d(batch_input, out_channels,filter_size,strides=(stride,stride), padding=pad) 18 | return conv 19 | 20 | def batchnorm(inputs): 21 | normalized = tf.layers.batch_normalization(inputs, axis=-1) 22 | return normalized 23 | 24 | def evaluation(logits, labels): 25 | correct_prediction = tf.equal(logits, labels) 26 | correct_batch = tf.reduce_mean(tf.cast(correct_prediction, tf.int32), 1) 27 | return tf.reduce_sum(tf.cast(correct_batch, tf.float32)), correct_batch 28 | 29 | ''' 30 | # image patch features abstract layers 31 | update: 2018.11.9 32 | 5*conv + fc 33 | 34 | ''' 35 | def features(img, name, reuse=False): 36 | with tf.variable_scope(name,reuse=reuse): 37 | layer1 = conv(img, 32, 3, pad='SAME') # 64*64*32 38 | layer1 = tf.nn.relu(layer1) 39 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 40 | 41 | layer2 = conv(pool1, 64, 3, pad='SAME') # 32*32*64 42 | layer2 = tf.nn.relu(layer2) 43 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 44 | 45 | layer3 = conv(pool2, 128, 3, pad='SAME') # 16*16*128 46 | layer3 = tf.nn.relu(layer3) 47 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 48 | 49 | layer4 = conv(pool3, 256, 3, pad='SAME') # 8*8*256 50 | layer4 = tf.nn.relu(layer4) 51 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 52 | # 4*4*256 53 | return [layer1, pool1, pool2, pool3, pool4], pool4 54 | 55 | ''' 56 | matching layers 57 | update: 2018.11.9 58 | 59 | ''' 60 | def matching(feature_map_1,feature_map_2): 61 | with tf.variable_scope('match_layers'): 62 | all_feature = tf.concat([feature_map_1,feature_map_2], axis=3) # 两个特征连接 4*4*512 63 | layer5 = conv(all_feature, 512, 3, pad='SAME') 64 | layer5 = tf.nn.relu(layer5) 65 | pool5 = tf.nn.max_pool(layer5,[1,2,2,1],[1,2,2,1],padding='SAME') # 2*2*512 66 | 67 | batch_size = int(pool5.get_shape()[0]) 68 | dense = tf.reshape(pool5, [batch_size,-1]) # reshape as a tensor 2048 69 | dense1 = tf.layers.dense(dense, 1024) 70 | dense1 = tf.nn.relu(dense1) 71 | dense2 = tf.layers.dense(dense1, 256) 72 | dense2 = tf.nn.relu(dense2) 73 | dense3 = tf.layers.dense(dense2, 1) 74 | output = tf.sigmoid(dense3) 75 | return output, [pool5, dense1, dense2] 76 | 77 | ''' 78 | network model 79 | ''' 80 | def features_matching(inputs_p1, inputs_p2): 81 | with tf.variable_scope("matching"): 82 | layers_1,feature_map_1 = features(inputs_p1, 'feature_layers') # 4*4*256 83 | layers_2,feature_map_2 = features(inputs_p2, 'feature_layers',reuse=True) # 4*4*256 84 | output,feature_maps = matching(feature_map_1,feature_map_2) 85 | return output ,[layers_1,layers_2,feature_maps] 86 | 87 | ''' 88 | cross loss function 89 | ''' 90 | 91 | def match_network(inputs_p1, inputs_p2, label_m): 92 | 93 | match_output, all_features = features_matching(inputs_p1, inputs_p2) 94 | match_loss = tf.reduce_mean(-(label_m * tf.log(match_output + EPS) + (1 - label_m) * tf.log(1 - match_output + EPS))) 95 | return match_loss, match_output, all_features 96 | 97 | 98 | 99 | 100 | 101 | 102 | -------------------------------------------------------------------------------- /part_1_feature_difference/model_d_map32_bn_diff.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on 2018.11.9 11:03:33 4 | 5 | @author: Administrator 6 | 7 | 双分支 + 特征map + 交叉熵损失函数 + 参数共享 + diffrent + bn only in diff 8 | 9 | """ 10 | 11 | import tensorflow as tf 12 | import numpy as np 13 | 14 | EPS = 1e-12 15 | 16 | def conv(batch_input, out_channels, filter_size, pad, stride=1): 17 | conv = tf.layers.conv2d(batch_input, out_channels,filter_size,strides=(stride,stride), padding=pad) 18 | return conv 19 | 20 | def batchnorm(inputs): 21 | normalized = tf.layers.batch_normalization(inputs, axis=-1) 22 | return normalized 23 | 24 | def evaluation(logits, labels): 25 | correct_prediction = tf.equal(logits, labels) 26 | correct_batch = tf.reduce_mean(tf.cast(correct_prediction, tf.int32), 1) 27 | return tf.reduce_sum(tf.cast(correct_batch, tf.float32)), correct_batch 28 | 29 | ''' 30 | # image patch features abstract layers 31 | update: 2018.11.9 32 | 5*conv + fc 33 | 34 | ''' 35 | def features(img, name, reuse=False): 36 | with tf.variable_scope(name,reuse=reuse): 37 | layer1 = conv(img, 32, 3, pad='SAME') # 64*64*32 38 | layer1 = batchnorm(layer1) 39 | layer1 = tf.nn.relu(layer1) 40 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 41 | 42 | layer2 = conv(pool1, 64, 3, pad='SAME') # 32*32*64 43 | layer2 = batchnorm(layer2) 44 | layer2 = tf.nn.relu(layer2) 45 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 46 | 47 | layer3 = conv(pool2, 128, 3, pad='SAME') # 16*16*128 48 | layer3 = batchnorm(layer3) 49 | layer3 = tf.nn.relu(layer3) 50 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 51 | 52 | layer4 = conv(pool3, 256, 3, pad='SAME') # 8*8*256 53 | layer4 = batchnorm(layer4) 54 | layer4 = tf.nn.relu(layer4) 55 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 56 | # 4*4*256 57 | return [layer1, pool1, pool2, pool3, pool4], pool4 58 | 59 | ''' 60 | matching layers 61 | update: 2018.11.9 62 | 63 | ''' 64 | def matching(feature_map_1,feature_map_2): 65 | with tf.variable_scope('match_layers'): 66 | # all_feature = tf.concat([feature_map_1,feature_map_2], axis=3) # 两个特征连接 4*4*512 67 | diff_feature = tf.abs(feature_map_1 - feature_map_2) 68 | layer5 = conv(diff_feature, 512, 3, pad='SAME') 69 | layer5 = batchnorm(layer5) 70 | layer5 = tf.nn.relu(layer5) 71 | pool5 = tf.nn.max_pool(layer5,[1,2,2,1],[1,2,2,1],padding='SAME') # 2*2*512 72 | 73 | batch_size = int(pool5.get_shape()[0]) 74 | dense = tf.reshape(pool5, [batch_size,-1]) # reshape as a tensor 2048 75 | dense1 = tf.layers.dense(dense, 1024) 76 | dense1 = tf.nn.relu(dense1) 77 | dense2 = tf.layers.dense(dense1, 256) 78 | dense2 = tf.nn.relu(dense2) 79 | dense3 = tf.layers.dense(dense2, 1) 80 | output = tf.sigmoid(dense3) 81 | return output, [pool5, dense1, dense2] 82 | 83 | ''' 84 | network model 85 | ''' 86 | def features_matching(inputs_p1, inputs_p2): 87 | with tf.variable_scope("matching"): 88 | layers_1,feature_map_1 = features(inputs_p1, 'feature_layers') # 4*4*256 89 | layers_2,feature_map_2 = features(inputs_p2, 'feature_layers',reuse=True) # 4*4*256 90 | output,feature_maps = matching(feature_map_1,feature_map_2) 91 | return output ,[layers_1,layers_2,feature_maps] 92 | 93 | ''' 94 | cross loss function 95 | ''' 96 | 97 | def match_network(inputs_p1, inputs_p2, label_m): 98 | 99 | match_output, all_features = features_matching(inputs_p1, inputs_p2) 100 | match_loss = tf.reduce_mean(-(label_m * tf.log(match_output + EPS) + (1 - label_m) * tf.log(1 - match_output + EPS))) 101 | return match_loss, match_output, all_features 102 | 103 | 104 | 105 | 106 | 107 | 108 | -------------------------------------------------------------------------------- /part_1_feature_difference/model_d_map32_diff.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on 2018.11.9 11:03:33 4 | 5 | @author: Administrator 6 | 7 | 双分支 + 特征map + 交叉熵损失函数 + 参数共享 + diffrent 8 | 9 | """ 10 | 11 | import tensorflow as tf 12 | import numpy as np 13 | 14 | EPS = 1e-12 15 | 16 | def conv(batch_input, out_channels, filter_size, pad, stride=1): 17 | conv = tf.layers.conv2d(batch_input, out_channels,filter_size,strides=(stride,stride), padding=pad) 18 | return conv 19 | 20 | def batchnorm(inputs): 21 | normalized = tf.layers.batch_normalization(inputs, axis=-1) 22 | return normalized 23 | 24 | def evaluation(logits, labels): 25 | correct_prediction = tf.equal(logits, labels) 26 | correct_batch = tf.reduce_mean(tf.cast(correct_prediction, tf.int32), 1) 27 | return tf.reduce_sum(tf.cast(correct_batch, tf.float32)), correct_batch 28 | 29 | ''' 30 | # image patch features abstract layers 31 | update: 2018.11.9 32 | 5*conv + fc 33 | 34 | ''' 35 | def features(img, name, reuse=False): 36 | with tf.variable_scope(name,reuse=reuse): 37 | layer1 = conv(img, 32, 3, pad='SAME') # 64*64*32 38 | layer1 = tf.nn.relu(layer1) 39 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 40 | 41 | layer2 = conv(pool1, 64, 3, pad='SAME') # 32*32*64 42 | layer2 = tf.nn.relu(layer2) 43 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 44 | 45 | layer3 = conv(pool2, 128, 3, pad='SAME') # 16*16*128 46 | layer3 = tf.nn.relu(layer3) 47 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 48 | 49 | layer4 = conv(pool3, 256, 3, pad='SAME') # 8*8*256 50 | layer4 = tf.nn.relu(layer4) 51 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 52 | # 4*4*256 53 | return [layer1, pool1, pool2, pool3, pool4], pool4 54 | 55 | ''' 56 | matching layers 57 | update: 2018.11.9 58 | 59 | ''' 60 | def matching(feature_map_1,feature_map_2): 61 | with tf.variable_scope('match_layers'): 62 | # all_feature = tf.concat([feature_map_1,feature_map_2], axis=3) # 两个特征连接 4*4*512 63 | diff_feature = tf.abs(feature_map_1 - feature_map_2) 64 | layer5 = conv(diff_feature, 512, 3, pad='SAME') 65 | layer5 = tf.nn.relu(layer5) 66 | pool5 = tf.nn.max_pool(layer5,[1,2,2,1],[1,2,2,1],padding='SAME') # 2*2*512 67 | 68 | batch_size = int(pool5.get_shape()[0]) 69 | dense = tf.reshape(pool5, [batch_size,-1]) # reshape as a tensor 2048 70 | dense1 = tf.layers.dense(dense, 1024) 71 | dense1 = tf.nn.relu(dense1) 72 | dense2 = tf.layers.dense(dense1, 256) 73 | dense2 = tf.nn.relu(dense2) 74 | dense3 = tf.layers.dense(dense2, 1) 75 | output = tf.sigmoid(dense3) 76 | return output, [pool5, dense1, dense2] 77 | 78 | ''' 79 | network model 80 | ''' 81 | def features_matching(inputs_p1, inputs_p2): 82 | with tf.variable_scope("matching"): 83 | layers_1,feature_map_1 = features(inputs_p1, 'feature_layers') # 4*4*256 84 | layers_2,feature_map_2 = features(inputs_p2, 'feature_layers',reuse=True) # 4*4*256 85 | output,feature_maps = matching(feature_map_1,feature_map_2) 86 | return output ,[layers_1,layers_2,feature_maps] 87 | 88 | ''' 89 | cross loss function 90 | ''' 91 | 92 | def match_network(inputs_p1, inputs_p2, label_m): 93 | 94 | match_output, all_features = features_matching(inputs_p1, inputs_p2) 95 | match_loss = tf.reduce_mean(-(label_m * tf.log(match_output + EPS) + (1 - label_m) * tf.log(1 - match_output + EPS))) 96 | return match_loss, match_output, all_features 97 | 98 | 99 | 100 | 101 | 102 | 103 | -------------------------------------------------------------------------------- /part_1_feature_difference/model_d_map32_no.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on 2018.11.9 11:03:33 4 | 5 | @author: Administrator 6 | 7 | 双分支 + 特征map + 交叉熵损失函数 + 参数共享 8 | 9 | """ 10 | 11 | import tensorflow as tf 12 | import numpy as np 13 | 14 | EPS = 1e-12 15 | 16 | def conv(batch_input, out_channels, filter_size, pad, stride=1): 17 | conv = tf.layers.conv2d(batch_input, out_channels,filter_size,strides=(stride,stride), padding=pad) 18 | return conv 19 | 20 | def batchnorm(inputs): 21 | normalized = tf.layers.batch_normalization(inputs, axis=-1) 22 | return normalized 23 | 24 | def evaluation(logits, labels): 25 | correct_prediction = tf.equal(logits, labels) 26 | correct_batch = tf.reduce_mean(tf.cast(correct_prediction, tf.int32), 1) 27 | return tf.reduce_sum(tf.cast(correct_batch, tf.float32)), correct_batch 28 | 29 | ''' 30 | # image patch features abstract layers 31 | update: 2018.11.9 32 | 5*conv + fc 33 | 34 | ''' 35 | def features(img, name, reuse=False): 36 | with tf.variable_scope(name,reuse=reuse): 37 | layer1 = conv(img, 32, 3, pad='SAME') # 64*64*32 38 | layer1 = tf.nn.relu(layer1) 39 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 40 | 41 | layer2 = conv(pool1, 64, 3, pad='SAME') # 32*32*64 42 | layer2 = tf.nn.relu(layer2) 43 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 44 | 45 | layer3 = conv(pool2, 128, 3, pad='SAME') # 16*16*128 46 | layer3 = tf.nn.relu(layer3) 47 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 48 | 49 | layer4 = conv(pool3, 256, 3, pad='SAME') # 8*8*256 50 | layer4 = tf.nn.relu(layer4) 51 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 52 | # 4*4*256 53 | return [layer1, pool1, pool2, pool3, pool4], pool4 54 | 55 | ''' 56 | matching layers 57 | update: 2018.11.9 58 | 59 | ''' 60 | def matching(feature_map_1,feature_map_2): 61 | with tf.variable_scope('match_layers'): 62 | all_feature = tf.concat([feature_map_1,feature_map_2], axis=3) # 两个特征连接 4*4*512 63 | layer5 = conv(all_feature, 512, 3, pad='SAME') 64 | layer5 = tf.nn.relu(layer5) 65 | pool5 = tf.nn.max_pool(layer5,[1,2,2,1],[1,2,2,1],padding='SAME') # 2*2*512 66 | 67 | batch_size = int(pool5.get_shape()[0]) 68 | dense = tf.reshape(pool5, [batch_size,-1]) # reshape as a tensor 2048 69 | dense1 = tf.layers.dense(dense, 1024) 70 | dense1 = tf.nn.relu(dense1) 71 | dense2 = tf.layers.dense(dense1, 256) 72 | dense2 = tf.nn.relu(dense2) 73 | dense3 = tf.layers.dense(dense2, 1) 74 | output = tf.sigmoid(dense3) 75 | return output, [pool5, dense1, dense2] 76 | 77 | ''' 78 | network model 79 | ''' 80 | def features_matching(inputs_p1, inputs_p2): 81 | with tf.variable_scope("matching"): 82 | layers_1,feature_map_1 = features(inputs_p1, 'feature_layers_1') # 4*4*256 83 | layers_2,feature_map_2 = features(inputs_p2, 'feature_layers_2') # 4*4*256 84 | output,feature_maps = matching(feature_map_1,feature_map_2) 85 | return output ,[layers_1,layers_2,feature_maps] 86 | 87 | ''' 88 | cross loss function 89 | ''' 90 | 91 | def match_network(inputs_p1, inputs_p2, label_m): 92 | 93 | match_output, all_features = features_matching(inputs_p1, inputs_p2) 94 | match_loss = tf.reduce_mean(-(label_m * tf.log(match_output + EPS) + (1 - label_m) * tf.log(1 - match_output + EPS))) 95 | return match_loss, match_output, all_features 96 | 97 | 98 | 99 | 100 | 101 | 102 | -------------------------------------------------------------------------------- /part_1_feature_difference/model_d_tensor.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on 2018.11.9 11:03:33 4 | 5 | @author: Administrator 6 | 7 | 双分支 + 特征向量 + 交叉熵损失函数 + 参数共享 8 | 9 | """ 10 | 11 | import tensorflow as tf 12 | import numpy as np 13 | 14 | EPS = 1e-12 15 | 16 | def conv(batch_input, out_channels, filter_size, pad, stride=1): 17 | conv = tf.layers.conv2d(batch_input, out_channels,filter_size,strides=(stride,stride), padding=pad) 18 | return conv 19 | 20 | def batchnorm(inputs): 21 | normalized = tf.layers.batch_normalization(inputs, axis=-1) 22 | return normalized 23 | 24 | def evaluation(logits, labels): 25 | correct_prediction = tf.equal(logits, labels) 26 | correct_batch = tf.reduce_mean(tf.cast(correct_prediction, tf.int32), 1) 27 | return tf.reduce_sum(tf.cast(correct_batch, tf.float32)), correct_batch 28 | 29 | ''' 30 | # image patch features abstract layers 31 | update: 2018.11.9 32 | 5*conv + fc 33 | 34 | ''' 35 | def features(img, name, reuse=False): 36 | with tf.variable_scope(name,reuse=reuse): 37 | layer1 = conv(img, 32, 3, pad='SAME') # 64*64*32 38 | layer1 = tf.nn.relu(layer1) 39 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 40 | 41 | layer2 = conv(pool1, 64, 3, pad='SAME') # 32*32*64 42 | layer2 = tf.nn.relu(layer2) 43 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 44 | 45 | layer3 = conv(pool2, 128, 3, pad='SAME') # 16*16*128 46 | layer3 = tf.nn.relu(layer3) 47 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 48 | 49 | layer4 = conv(pool3, 256, 3, pad='SAME') # 8*8*256 50 | layer4 = tf.nn.relu(layer4) 51 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 52 | 53 | # layer5 = conv(pool4, 256, 3, pad='SAME') # 4*4*256 54 | # layer5 = tf.nn.relu(layer5) 55 | pool5 = tf.nn.max_pool(pool4,[1,2,2,1],[1,2,2,1],padding='SAME') 56 | 57 | # 2*2*256=1024 58 | batch_size = int(pool5.get_shape()[0]) 59 | dense = tf.reshape(pool5, [batch_size,-1]) 60 | 61 | return [pool1, pool2, pool3, pool4, pool5, dense], dense 62 | 63 | ''' 64 | matching layers 65 | update: 2018.11.9 66 | 67 | ''' 68 | def matching(dense_1,dense_2): 69 | with tf.variable_scope('match_layers'): 70 | all_feature = tf.concat([dense_1, dense_2], axis=1) # 两个特征连接 1024*2=2048 71 | dense1 = tf.layers.dense(all_feature, 1024) 72 | dense1 = tf.nn.relu(dense1) 73 | 74 | dense2 = tf.layers.dense(dense1, 256) 75 | dense2 = tf.nn.relu(dense2) 76 | 77 | dense3 = tf.layers.dense(dense2, 1) 78 | output = tf.sigmoid(dense3) 79 | return output, [dense1,dense2] 80 | 81 | ''' 82 | network model 83 | ''' 84 | def features_matching(inputs_p1, inputs_p2): 85 | with tf.variable_scope("matching"): 86 | layers_1,dense_1 = features(inputs_p1, 'feature_layers') # 4*4*256 87 | layers_2,dense_2 = features(inputs_p2, 'feature_layers',reuse=True) # 4*4*256 88 | output,denses = matching(dense_1,dense_2) 89 | return output ,[layers_1,layers_2,denses] 90 | 91 | ''' 92 | cross loss function 93 | ''' 94 | 95 | def match_network(inputs_p1, inputs_p2, label_m): 96 | 97 | match_output, all_features = features_matching(inputs_p1, inputs_p2) 98 | match_loss = tf.reduce_mean(-(label_m * tf.log(match_output + EPS) + (1 - label_m) * tf.log(1 - match_output + EPS))) 99 | return match_loss, match_output, all_features 100 | 101 | 102 | 103 | 104 | 105 | 106 | -------------------------------------------------------------------------------- /part_1_feature_difference/model_d_tensor_no.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on 2018.11.9 11:03:33 4 | 5 | @author: Administrator 6 | 7 | 双分支 + 特征向量 + 交叉熵损失函数 + 参数fei共享 8 | 9 | """ 10 | 11 | import tensorflow as tf 12 | import numpy as np 13 | 14 | EPS = 1e-12 15 | 16 | def conv(batch_input, out_channels, filter_size, pad, stride=1): 17 | conv = tf.layers.conv2d(batch_input, out_channels,filter_size,strides=(stride,stride), padding=pad) 18 | return conv 19 | 20 | def batchnorm(inputs): 21 | normalized = tf.layers.batch_normalization(inputs, axis=-1) 22 | return normalized 23 | 24 | def evaluation(logits, labels): 25 | correct_prediction = tf.equal(logits, labels) 26 | correct_batch = tf.reduce_mean(tf.cast(correct_prediction, tf.int32), 1) 27 | return tf.reduce_sum(tf.cast(correct_batch, tf.float32)), correct_batch 28 | 29 | ''' 30 | # image patch features abstract layers 31 | update: 2018.11.9 32 | 5*conv + fc 33 | 34 | ''' 35 | def features(img, name, reuse=False): 36 | with tf.variable_scope(name,reuse=reuse): 37 | layer1 = conv(img, 32, 3, pad='SAME') # 64*64*32 38 | layer1 = tf.nn.relu(layer1) 39 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 40 | 41 | layer2 = conv(pool1, 64, 3, pad='SAME') # 32*32*64 42 | layer2 = tf.nn.relu(layer2) 43 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 44 | 45 | layer3 = conv(pool2, 128, 3, pad='SAME') # 16*16*128 46 | layer3 = tf.nn.relu(layer3) 47 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 48 | 49 | layer4 = conv(pool3, 256, 3, pad='SAME') # 8*8*256 50 | layer4 = tf.nn.relu(layer4) 51 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 52 | # 53 | # layer5 = conv(pool4, 256, 3, pad='SAME') # 4*4*256 54 | # layer5 = tf.nn.relu(layer5) 55 | pool5 = tf.nn.max_pool(pool4,[1,2,2,1],[1,2,2,1],padding='SAME') 56 | 57 | # 2*2*256=1024 58 | batch_size = int(pool5.get_shape()[0]) 59 | dense = tf.reshape(pool5, [batch_size,-1]) 60 | 61 | return [pool1, pool2, pool3, pool4, pool5, dense], dense 62 | 63 | ''' 64 | matching layers 65 | update: 2018.11.9 66 | 67 | ''' 68 | def matching(dense_1,dense_2): 69 | with tf.variable_scope('match_layers'): 70 | all_feature = tf.concat([dense_1, dense_2], axis=1) # 两个特征连接 1024*2=2048 71 | dense1 = tf.layers.dense(all_feature, 1024) 72 | dense1 = tf.nn.relu(dense1) 73 | 74 | dense2 = tf.layers.dense(dense1, 256) 75 | dense2 = tf.nn.relu(dense2) 76 | 77 | dense3 = tf.layers.dense(dense2, 1) 78 | output = tf.sigmoid(dense3) 79 | return output, [dense1,dense2] 80 | 81 | ''' 82 | network model 83 | ''' 84 | def features_matching(inputs_p1, inputs_p2): 85 | with tf.variable_scope("matching"): 86 | layers_1,dense_1 = features(inputs_p1, 'feature_layers_1') # 4*4*256 87 | layers_2,dense_2 = features(inputs_p2, 'feature_layers_2') # 4*4*256 88 | output,denses = matching(dense_1,dense_2) 89 | return output ,[layers_1,layers_2,denses] 90 | 91 | ''' 92 | cross loss function 93 | ''' 94 | 95 | def match_network(inputs_p1, inputs_p2, label_m): 96 | 97 | match_output, all_features = features_matching(inputs_p1, inputs_p2) 98 | match_loss = tf.reduce_mean(-(label_m * tf.log(match_output + EPS) + (1 - label_m) * tf.log(1 - match_output + EPS))) 99 | return match_loss, match_output, all_features 100 | 101 | 102 | 103 | 104 | 105 | 106 | -------------------------------------------------------------------------------- /part_1_feature_difference/model_gd.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on Sat Mar 3 11:03:33 2018 4 | 5 | @author: Administrator 6 | """ 7 | 8 | import tensorflow as tf 9 | import numpy as np 10 | import argparse 11 | 12 | parser = argparse.ArgumentParser() 13 | parser.add_argument("--l1_weight", type=float, default=100.0, help="weight on L1 term for generator gradient") 14 | parser.add_argument("--gan_weight", type=float, default=1.0, help="weight on GAN term for generator gradient") 15 | a = parser.parse_args() 16 | 17 | EPS = 1e-12 18 | 19 | def conv(batch_input, out_channels, filter_size, pad, stride=1): 20 | conv = tf.layers.conv2d(batch_input, out_channels,filter_size,strides=(stride,stride), padding=pad) 21 | return conv 22 | 23 | def deconv(batch_input, out_channels, filter_size, pad): 24 | conv = tf.layers.conv2d_transpose(batch_input, out_channels, filter_size, strides=(2,2), padding=pad ,kernel_initializer=tf.random_normal_initializer(0, 0.02)) 25 | return conv 26 | 27 | def evaluation(logits, labels): 28 | correct_prediction = tf.equal(logits, labels) 29 | correct_batch = tf.reduce_mean(tf.cast(correct_prediction, tf.int32), 1) 30 | return tf.reduce_sum(tf.cast(correct_batch, tf.float32)), correct_batch 31 | 32 | def create_generator(generator_inputs, generator_outputs_channels, name="generator", reuse=False): 33 | # U_Net结构 34 | with tf.variable_scope(name,reuse=reuse): 35 | layers = [] 36 | with tf.variable_scope("encoder_1"): 37 | # encoder_1: [batch, 64, 64, 1 ] => [batch, 32, 32, 32] 38 | convolved = conv(generator_inputs, 32, 3, pad='SAME') 39 | convolved = tf.nn.relu(convolved) 40 | output = tf.nn.max_pool(convolved,[1,2,2,1],[1,2,2,1],padding='SAME') 41 | 42 | layers.append(output) 43 | 44 | layer_specs = [ 45 | 64, # encoder_2: [batch, 32, 32, 32 ] => [batch, 16, 16, 64] 46 | 128, # encoder_3: [batch, 16, 16, 64] => [batch, 8, 8, 128] 47 | 256, # encoder_3: [batch, 8, 8, 128] => [batch, 4, 4, 256] 48 | ] 49 | 50 | for out_channels in layer_specs: 51 | with tf.variable_scope("encoder_%d" % (len(layers) + 1)): 52 | # [batch, in_height, in_width, in_channels] => [batch, in_height/2, in_width/2, out_channels] 53 | convolved = conv(output, out_channels, 3, pad='SAME') 54 | convolved = tf.nn.relu(convolved) 55 | output = tf.nn.max_pool(convolved,[1,2,2,1],[1,2,2,1],padding='SAME') 56 | layers.append(output) 57 | 58 | layer_specs = [ 59 | (128, 0.0), # decoder_2: [batch, 4, 4, 256] => [batch, 8, 8, 128] 60 | (64, 0.0), # decoder_3: [batch, 8, 8, 128] => [batch, 16, 16, 64] 61 | (32, 0.0), # decoder_3: [batch, 16, 16, 64] => [batch, 32, 32, 32] 62 | ] 63 | num_encoder_layers = len(layers) 64 | for decoder_layer, (out_channels, dropout) in enumerate(layer_specs): 65 | skip_layer = num_encoder_layers - decoder_layer - 1 66 | with tf.variable_scope("decoder_%d" % (skip_layer + 1)): 67 | if decoder_layer == 0: 68 | # first decoder layer doesn't have skip connections 69 | # since it is directly connected to the skip_layer 70 | inputs = layers[-1] 71 | else: 72 | inputs = tf.concat([layers[-1], layers[skip_layer]], axis=3) #跨层连接 73 | # [batch, in_height, in_width, in_channels] => [batch, in_height*2, in_width*2, out_channels] 74 | output = deconv(inputs, out_channels, 3, pad='SAME') 75 | output = tf.nn.relu(output) 76 | 77 | if dropout > 0.0: 78 | output = tf.nn.dropout(output, keep_prob=1 - dropout) 79 | 80 | layers.append(output) 81 | 82 | # decoder_4: [batch, 32, 32, ngf ] => [batch, 64, 64, generator_outputs_channels] 83 | with tf.variable_scope("decoder_1"): 84 | inputs = tf.concat([layers[-1], layers[0]], axis=3) 85 | output = deconv(inputs, generator_outputs_channels,3,pad='SAME') 86 | output = tf.tanh(output) 87 | layers.append(output) 88 | return layers[-1] 89 | 90 | 91 | # 判别器网络 92 | def create_discriminator(inputs_o, inputs_r, reuse=False): 93 | with tf.variable_scope("discriminator", reuse=reuse): 94 | # 2x [batch, height, width, in_channels] => [batch, height, width, in_channels * 2] 95 | inputs = tf.concat([inputs_o, inputs_r], axis=3) # 64*64*2 96 | layer1 = conv(inputs, 32, 3, pad='SAME') # 64*64*2 --> 64*64*32 97 | layer1 = tf.nn.relu(layer1) 98 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 99 | 100 | layer2 = conv(pool1, 64, 3, pad='SAME') # 32*32*32 --> 32*32*64 101 | layer2 = tf.nn.relu(layer2) 102 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 103 | 104 | layer3 = conv(pool2, 128, 3, pad='SAME') # 16*16*64 --> 16*16*128 105 | layer3 = tf.nn.relu(layer3) 106 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 107 | 108 | layer4 = conv(pool3, 256, 3, pad='SAME') # 8*8*128 --> 8*8*256 109 | layer4 = tf.nn.relu(layer4) 110 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') # 4*4*256 111 | 112 | layer5 = conv(pool4, 512, 3, pad='SAME') # 4*4*256 --> 4*4*512 113 | layer5 = tf.nn.relu(layer5) 114 | pool5 = tf.nn.max_pool(layer5,[1,2,2,1],[1,2,2,1],padding='SAME') # 2*2*512 115 | 116 | batch_size = int(pool5.get_shape()[0]) 117 | dense = tf.reshape(pool5, [batch_size,-1]) 118 | dense1 = tf.layers.dense(inputs=dense, units=1024) 119 | dense1 = tf.nn.relu(dense1) 120 | dense2 = tf.layers.dense(inputs=dense1, units=256) 121 | dense2 = tf.nn.relu(dense2) 122 | dense3 = tf.layers.dense(inputs=dense2, units=1) 123 | output = tf.sigmoid(dense3) 124 | return output 125 | 126 | 127 | def gd_model_o2r(inputs_o, inputs_r): 128 | # opt --> nir 129 | # with tf.variable_scope("sar2opt"): 130 | out_channels = int(inputs_r.get_shape()[-1]) 131 | g_outputs = create_generator(inputs_o, out_channels) # fake_optical 132 | # create two copies of discriminator, one for real pairs and one for fake pairs 133 | # they share the same underlying variables 134 | with tf.name_scope("real_discriminator"): 135 | # 2x [batch, height, width, channels] => [batch, 1] 136 | predict_real= create_discriminator(inputs_o, inputs_r) 137 | with tf.name_scope("fake_discriminator"): 138 | # 2x [batch, height, width, channels] => [batch, 1] 139 | predict_fake = create_discriminator(inputs_o, g_outputs, reuse=True) 140 | 141 | with tf.name_scope("discriminator_loss"): 142 | # minimizing -tf.log will try to get inputs to 1 143 | # predict_real => 1 144 | # predict_fake => 0 145 | discrim_loss = tf.reduce_mean(-(tf.log(predict_real + EPS) + tf.log(1 - predict_fake + EPS))) 146 | 147 | with tf.name_scope("generator_loss"): 148 | # predict_fake => 1 149 | # abs(targets - outputs) => 0 150 | gen_loss_GAN = tf.reduce_mean(-tf.log(predict_fake + EPS)) 151 | gen_loss_L1 = tf.reduce_mean(tf.abs(inputs_r - g_outputs)) 152 | gen_loss = gen_loss_GAN * a.gan_weight + gen_loss_L1 * a.l1_weight # 为什么权重相差这么大 153 | 154 | return gen_loss, discrim_loss, g_outputs, [gen_loss_GAN, gen_loss_L1] 155 | 156 | 157 | 158 | 159 | 160 | -------------------------------------------------------------------------------- /part_1_feature_difference/read_data.py: -------------------------------------------------------------------------------- 1 | 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Created on Thu Feb 1 13:56:02 2018 5 | 6 | @author: DELL 7 | """ 8 | 9 | from __future__ import absolute_import 10 | from __future__ import division 11 | from __future__ import print_function 12 | 13 | import os.path 14 | import tensorflow as tf 15 | import scipy.misc as misc 16 | import numpy as np 17 | 18 | SAVA_PATH = 'country' 19 | TRAIN_FILE = 'country.tfrecord' 20 | BATCH_SIZE = 100 21 | img_size = 64 22 | 23 | def read_and_decode_train(filename_queue): 24 | # create a reader from file queue 25 | reader = tf.TFRecordReader() 26 | #reader从 TFRecord 读取内容并保存到 serialized_example 中 27 | _, serialized_example = reader.read(filename_queue) 28 | 29 | features = tf.parse_single_example( # 读取 serialized_example 的格式 30 | serialized_example, 31 | features={ 32 | 'label_raw': tf.FixedLenFeature([], tf.string), 33 | 'image_raw': tf.FixedLenFeature([], tf.string) 34 | } 35 | ) # 解析从 serialized_example 读取到的内容 36 | img = tf.decode_raw(features['image_raw'], tf.uint8) 37 | img = tf.reshape(img, [img_size, img_size*2]) 38 | img = tf.cast(img, tf.float32) * (1. / 255.0) #[0,1] 39 | label = tf.decode_raw(features['label_raw'], tf.int32) 40 | label = tf.reshape(tf.cast(label, tf.float32),shape=(1,)) 41 | return img,label 42 | # return tf.cast(resize_image, tf.float32), tf.cast(reshape_label, tf.float32) 43 | 44 | def read_and_decode_test(filename_queue): 45 | # create a reader from file queue 46 | reader = tf.TFRecordReader() 47 | #reader从 TFRecord 读取内容并保存到 serialized_example 中 48 | _, serialized_example = reader.read(filename_queue) 49 | 50 | features = tf.parse_single_example( # 读取 serialized_example 的格式 51 | serialized_example, 52 | features={ 53 | 'label_raw': tf.FixedLenFeature([], tf.string), 54 | 'image_raw': tf.FixedLenFeature([], tf.string) 55 | } 56 | ) # 解析从 serialized_example 读取到的内容 57 | img = tf.decode_raw(features['image_raw'], tf.uint8) 58 | img = tf.reshape(img, [img_size, img_size*2]) 59 | img = tf.cast(img, tf.float32) * (1. / 255.0) #[0,1] 60 | label = tf.decode_raw(features['label_raw'], tf.int32) 61 | label = tf.reshape(tf.cast(label, tf.float32),shape=(1,)) 62 | return img,label 63 | 64 | def batch_inputs(filename_queue, train, batch_size): 65 | #创建文件队列,不限读取的数量 66 | with tf.name_scope('input'): 67 | if train: 68 | image, label = read_and_decode_train(filename_queue) 69 | images, labels = tf.train.shuffle_batch([image, label], 70 | batch_size=batch_size, 71 | num_threads=6, 72 | capacity=2000 + 3 * batch_size, 73 | min_after_dequeue=2000) 74 | else: 75 | image, label = read_and_decode_test(filename_queue) 76 | images, labels = tf.train.batch([image, label], 77 | batch_size=batch_size, 78 | num_threads=6, 79 | capacity=2000 + 3 * batch_size) 80 | return images, labels 81 | 82 | if __name__ == '__main__': 83 | 84 | init = tf.global_variables_initializer() 85 | # filename = os.path.join(SAVA_PATH, TRAIN_FILE) 86 | filename = '/home/ws/文档/wrj/data_all/country/country.tfrecord' 87 | filename_queue = tf.train.string_input_producer([filename],num_epochs=20, shuffle=True) 88 | img_batch, label_batch = batch_inputs(filename_queue, train = True, batch_size = BATCH_SIZE) 89 | with tf.Session() as sess: 90 | # sess.run(init) 91 | sess.run(tf.global_variables_initializer()) 92 | sess.run(tf.local_variables_initializer()) 93 | 94 | coord = tf.train.Coordinator() #创建一个协调器，管理线程 95 | #启动QueueRunner, 此时文件名队列已经进队。 96 | threads=tf.train.start_queue_runners(sess=sess,coord=coord) 97 | try: 98 | for i in range(200): 99 | img, label = sess.run([img_batch, label_batch]) 100 | for j in range(img.shape[0]): 101 | misc.imsave(str(j)+'.png', np.squeeze(img[j])) 102 | except tf.errors.OutOfRangeError: 103 | print('Done reading') 104 | finally: 105 | coord.request_stop() 106 | coord.request_stop() 107 | coord.join(threads) 108 | 109 | 110 | -------------------------------------------------------------------------------- /part_1_feature_difference/read_data_map.py: -------------------------------------------------------------------------------- 1 | 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Created on Thu Feb 1 13:56:02 2018 5 | 6 | @author: DELL 7 | """ 8 | 9 | from __future__ import absolute_import 10 | from __future__ import division 11 | from __future__ import print_function 12 | 13 | import os.path 14 | import tensorflow as tf 15 | import scipy.misc as misc 16 | import numpy as np 17 | 18 | 19 | SAVA_PATH = 'mapping_data' 20 | TRAIN_FILE = 'map_country.tfrecord' 21 | BATCH_SIZE = 10 22 | img_size = 64 23 | 24 | 25 | 26 | def read_and_decode_train(filename_queue): 27 | # create a reader from file queue 28 | reader = tf.TFRecordReader() 29 | #reader从 TFRecord 读取内容并保存到 serialized_example 中 30 | _, serialized_example = reader.read(filename_queue) 31 | 32 | features = tf.parse_single_example( # 读取 serialized_example 的格式 33 | serialized_example, 34 | features={ 35 | 'label_raw': tf.FixedLenFeature([], tf.string), 36 | 'image_raw': tf.FixedLenFeature([], tf.string) 37 | } 38 | ) # 解析从 serialized_example 读取到的内容 39 | img = tf.decode_raw(features['image_raw'], tf.uint8) 40 | img = tf.reshape(img, [img_size, img_size*2]) 41 | # img_gray = tf.image.rgb_to_grayscale(img) 42 | img = tf.cast(img, tf.float32) * (1. / 255.0) #[0,1] 43 | label = tf.decode_raw(features['label_raw'], tf.int32) 44 | label = tf.reshape(tf.cast(label, tf.float32),shape=(1,)) 45 | return img,label 46 | # return tf.cast(resize_image, tf.float32), tf.cast(reshape_label, tf.float32) 47 | 48 | def read_and_decode_test(filename_queue): 49 | # create a reader from file queue 50 | reader = tf.TFRecordReader() 51 | #reader从 TFRecord 读取内容并保存到 serialized_example 中 52 | _, serialized_example = reader.read(filename_queue) 53 | 54 | features = tf.parse_single_example( # 读取 serialized_example 的格式 55 | serialized_example, 56 | features={ 57 | 'label_raw': tf.FixedLenFeature([], tf.string), 58 | 'image_raw': tf.FixedLenFeature([], tf.string) 59 | } 60 | ) # 解析从 serialized_example 读取到的内容 61 | img = tf.decode_raw(features['image_raw'], tf.uint8) 62 | img = tf.reshape(img, [img_size, img_size*2]) 63 | # img_gray = tf.image.rgb_to_grayscale(img) 64 | img = tf.cast(img, tf.float32) * (1. / 255.0) #[0,1] 65 | label = tf.decode_raw(features['label_raw'], tf.int32) 66 | label = tf.reshape(tf.cast(label, tf.float32),shape=(1,)) 67 | return img,label 68 | 69 | def batch_inputs(filename_queue, train, batch_size): 70 | #创建文件队列,不限读取的数量 71 | with tf.name_scope('input'): 72 | if train: 73 | image, label = read_and_decode_train(filename_queue) 74 | images, labels = tf.train.shuffle_batch([image, label], 75 | batch_size=batch_size, 76 | num_threads=6, 77 | capacity=2000 + 3 * batch_size, 78 | min_after_dequeue=2000) 79 | else: 80 | image, label = read_and_decode_test(filename_queue) 81 | images, labels = tf.train.batch([image, label], 82 | batch_size=batch_size, 83 | num_threads=6, 84 | capacity=2000 + 3 * batch_size) 85 | return images, labels 86 | 87 | if __name__ == '__main__': 88 | 89 | init = tf.global_variables_initializer() 90 | # filename = os.path.join(SAVA_PATH, TRAIN_FILE) 91 | filename = '/home/ws/文档/wrj/mapping_data_all/mapping_data/map_country.tfrecord' 92 | filename_queue = tf.train.string_input_producer([filename],num_epochs=10 , shuffle=True) 93 | img_batch, label_batch = batch_inputs(filename_queue, train = True, batch_size = BATCH_SIZE) 94 | with tf.Session() as sess: 95 | # sess.run(init) 96 | sess.run(tf.global_variables_initializer()) 97 | sess.run(tf.local_variables_initializer()) 98 | 99 | coord = tf.train.Coordinator() #创建一个协调器，管理线程 100 | #启动QueueRunner, 此时文件名队列已经进队。 101 | threads=tf.train.start_queue_runners(sess=sess,coord=coord) 102 | try: 103 | for i in range(100): 104 | img, label = sess.run([img_batch, label_batch]) 105 | for j in range(img.shape[0]): 106 | misc.imsave(str(j)+'_read.png', np.squeeze(img[j])) 107 | except tf.errors.OutOfRangeError: 108 | print('Done reading') 109 | finally: 110 | coord.request_stop() 111 | coord.request_stop() 112 | coord.join(threads) 113 | 114 | 115 | -------------------------------------------------------------------------------- /part_1_feature_difference/train_nature.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Created on Mon Dec 4 19:37:06 2017 5 | 6 | @author: wrj 7 | """ 8 | ''' 9 | 双分支参数共享 + 分类损失 10 | 参差损失 11 | ''' 12 | 13 | import numpy as np 14 | import math 15 | import time 16 | import model_d as model 17 | import read_data 18 | import tensorflow as tf 19 | import os 20 | from datetime import datetime 21 | import logging 22 | from sklearn.metrics import roc_curve, auc 23 | 24 | BATCH_SIZE_matching=100 25 | BATCH_SIZE_iden=200 26 | epoch = 41 27 | learning_rate = 2e-4 28 | image_width = 64 29 | image_height = 64 30 | KEEP_prob = 1.0 31 | 32 | checkpoint_dir = 'ckpt_country' 33 | checkpoint_dir_g = 'ckpt_country' 34 | checkpoint_file = os.path.join(checkpoint_dir, 'model.ckpt') 35 | checkpoint_file_g = os.path.join(checkpoint_dir_g, 'model.ckpt') 36 | train_dir='summary_country' 37 | 38 | def initLogging(logFilename='record_country.log'): 39 | """Init for logging 40 | """ 41 | logging.basicConfig( 42 | level = logging.DEBUG, 43 | format='%(asctime)s-%(levelname)s-%(message)s', 44 | datefmt = '%y-%m-%d %H:%M', 45 | filename = logFilename, 46 | filemode = 'w'); 47 | console = logging.StreamHandler() 48 | console.setLevel(logging.INFO) 49 | formatter = logging.Formatter('%(asctime)s-%(levelname)s-%(message)s') 50 | console.setFormatter(formatter) 51 | logging.getLogger('').addHandler(console) 52 | 53 | 54 | def get_one_hot(targets, nb_classes): 55 | return np.eye(nb_classes)[np.array(targets, dtype=np.int32).reshape(-1)] 56 | 57 | def norm(patch_1): 58 | np.seterr(divide='ignore', invalid='ignore') 59 | patch_1 = np.reshape(patch_1,(patch_1.shape[0],-1)) 60 | # patch_max = np.max(patch_1, (1)) 61 | # patch_max = patch_max[:,np.newaxis] 62 | # patch_min = np.min(patch_1, (1)) 63 | # patch_min = patch_min[:,np.newaxis] 64 | # patch_dif = patch_max - patch_min 65 | # patch_1 = (patch_1 - patch_min)/patch_dif 66 | patch_1 = patch_1 /255.0 67 | patch_1 = np.reshape(patch_1,(patch_1.shape[0],64,64)) 68 | patch_1 = patch_1[:,:,:,np.newaxis] 69 | return patch_1 70 | 71 | def mc_train(): 72 | initLogging() 73 | current_time = datetime.now().strftime('%Y%m%d-%H%M') 74 | checkpoints_dir = 'checkpoints/{}'.format(current_time) 75 | try: 76 | os.makedirs(checkpoint_dir) 77 | os.makedirs(checkpoints_dir) 78 | except os.error: 79 | pass 80 | 81 | data1 = np.load('data_all/country.npz') 82 | train_matching_y = data1['arr_1'][:,np.newaxis] 83 | numbers_train = train_matching_y.shape[0] #训练集总数 84 | epoch_steps = np.int(numbers_train / BATCH_SIZE_matching) +1 # 一个epoch有多少个steps 85 | 86 | all_loss = np.array([]) 87 | graph = tf.Graph() 88 | with graph.as_default(): 89 | inputs_p1 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width,1], name='inputs_p1') 90 | inputs_p2 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width,1], name='inputs_p2') 91 | label_m = tf.placeholder(tf.float32, [BATCH_SIZE_matching, 1], name='label_m') 92 | keep_prob = tf.placeholder(tf.float32, name='keep_prob') 93 | # 训练 M 94 | match_loss, m_output, all_features = model.matchLoss(inputs_p1, inputs_p2, label_m,keep_prob) 95 | match_out = tf.round(m_output) 96 | m_correct,m_numbers = model.evaluation(match_out, label_m) 97 | 98 | m_train_opt = tf.train.AdamOptimizer(learning_rate).minimize(match_loss) 99 | 100 | filename = 'data_all/country/country.tfrecord' 101 | filename_queue = tf.train.string_input_producer([filename],num_epochs=epoch,shuffle=True) 102 | img_batch, label_batch = read_data.batch_inputs(filename_queue, train = True, batch_size = BATCH_SIZE_matching) 103 | tf.summary.scalar('mathing_loss', match_loss) 104 | 105 | summary = tf.summary.merge_all() 106 | saver = tf.train.Saver(max_to_keep=20) 107 | init = tf.global_variables_initializer() 108 | with tf.Session() as sess: 109 | summary_writer = tf.summary.FileWriter(train_dir, sess.graph) 110 | sess.run(tf.local_variables_initializer()) 111 | sess.run(init) 112 | 113 | try: 114 | coord = tf.train.Coordinator() 115 | threads = tf.train.start_queue_runners(sess=sess, coord=coord) 116 | step = 0 117 | 118 | while not coord.should_stop(): 119 | start_time = time.time() 120 | step = step + 1 121 | batch, l_batch = sess.run([img_batch, label_batch]) 122 | l_batch = l_batch 123 | x_batch = batch[:,:,:64,np.newaxis] 124 | y_batch = batch[:,:,64:,np.newaxis] 125 | feed_dict = {inputs_p1:x_batch, inputs_p2:y_batch, label_m:l_batch, keep_prob:KEEP_prob } 126 | _, m_loss, m_output_= sess.run([m_train_opt,match_loss,m_output], feed_dict = feed_dict) 127 | 128 | if step % 10 ==0: 129 | loss_write = np.array([[step, m_loss]]) 130 | if step ==10: 131 | all_loss = loss_write 132 | else: 133 | all_loss = np.concatenate((all_loss, loss_write)) 134 | 135 | if step % 100 == 0: 136 | duration = time.time() - start_time 137 | summary_str = sess.run(summary, feed_dict=feed_dict) 138 | summary_writer.add_summary(summary_str, step) 139 | summary_writer.flush() 140 | 141 | logging.info('>> Step %d run_train: matching_loss = %.3f (%.3f sec)' 142 | % (step, m_loss, duration)) 143 | 144 | if (step % epoch_steps == 0) and ((step / epoch_steps)%5 == 0): 145 | logging.info('>> %s Saving in %s' % (datetime.now(), checkpoint_dir)) 146 | saver.save(sess, checkpoint_file, global_step=step) 147 | 148 | except KeyboardInterrupt: 149 | print('INTERRUPTED') 150 | coord.request_stop() 151 | except Exception as e: 152 | coord.request_stop(e) 153 | 154 | finally: 155 | saver.save(sess, checkpoint_file, global_step=step) 156 | np.save(os.path.join(checkpoint_dir,'ckpt_country'), all_loss) 157 | print('Model saved in file :%s'%checkpoint_dir) 158 | # When done, ask the threads to stop. 159 | coord.request_stop() 160 | coord.join(threads) 161 | 162 | #''' 163 | def gfm_test(file_name): 164 | data1 = np.load('data_all/'+file_name+'.npz') 165 | train_matching_y = data1['arr_1'][:,np.newaxis] 166 | numbers_train = train_matching_y.shape[0] #训练集总数 167 | # epoch_steps = np.int(numbers_train / BATCH_SIZE_matching) +1 # 一个epoch有多少个steps 168 | graph = tf.Graph() 169 | with graph.as_default(): 170 | inputs_p1 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width, 1], name='inputs_p1') 171 | inputs_p2 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width, 1], name='inputs_p2') 172 | label_m = tf.placeholder(tf.float32, [BATCH_SIZE_matching, 1], name='label_m') 173 | keep_prob = tf.placeholder(tf.float32, name='keep_prob') 174 | # 训练 M 175 | match_loss, m_output, all_features = model.matchLoss(inputs_p1, inputs_p2, label_m,keep_prob,) 176 | match_out = tf.round(m_output) 177 | m_correct,m_numbers = model.evaluation(match_out, label_m) 178 | 179 | filename = 'data_all/country/'+file_name+'.tfrecord' 180 | filename_queue = tf.train.string_input_producer([filename],num_epochs=1,shuffle=False) 181 | img_batch, label_batch = read_data.batch_inputs(filename_queue, train = False, batch_size = BATCH_SIZE_matching) 182 | 183 | saver = tf.train.Saver() 184 | with tf.Session() as sess: 185 | sess.run(tf.local_variables_initializer()) 186 | saver.restore(sess, tf.train.latest_checkpoint('ckpt_country')) 187 | # saver.restore(sess, 'ckpt_mc_ottawa/model.ckpt-4000') 188 | m_count = 0 # Counts the number of correct predictions. 189 | num = numbers_train 190 | matching_out = np.array([]) 191 | matching_label = np.array([]) 192 | 193 | try: 194 | coord = tf.train.Coordinator() 195 | threads = tf.train.start_queue_runners(sess=sess, coord=coord) 196 | step_test = 0 197 | try: 198 | while not coord.should_stop(): 199 | step_test = step_test + 1 200 | batch, l_batch = sess.run([img_batch, label_batch]) 201 | l_batch = l_batch 202 | x_batch = batch[:,:,:64,np.newaxis] 203 | y_batch = batch[:,:,64:,np.newaxis] 204 | feed_dict = {inputs_p1:x_batch, inputs_p2:y_batch, label_m:l_batch,keep_prob:KEEP_prob} 205 | m_correct_, m_output_ = sess.run([m_correct,m_output], feed_dict = feed_dict) 206 | 207 | if step_test == 1: 208 | matching_out = m_output_ 209 | matching_label = l_batch 210 | elif(l_batch.size == BATCH_SIZE_matching): 211 | matching_out = np.concatenate((matching_out, m_output_)) 212 | matching_label = np.concatenate((matching_label, l_batch)) 213 | 214 | m_count = m_count + m_correct_.astype(int) 215 | if step_test % 100 == 0: 216 | print('Step %d run_test: batch_precision = %.2f '% (step_test, m_correct_/BATCH_SIZE_matching)) 217 | except Exception as e: 218 | coord.request_stop(e) 219 | m_precision = float(m_count) / num 220 | 221 | print(' Num examples: %d Num correct: %d match Precision : %0.04f ' %(num, m_count, m_precision)) 222 | save_file = open('test_nature.txt','a') 223 | save_file.write(file_name + '\n'+'match Precision : '+str(m_precision)) 224 | save_file.write('\n') 225 | 226 | # 绘制ROC曲线 227 | fpr,tpr,threshold = roc_curve(matching_label, matching_out) ###计算真正率和假正率 228 | roc_auc = auc(fpr,tpr) ###计算auc的值 229 | save_file.write('match roc_auc : '+str(roc_auc)) 230 | save_file.write('\n') 231 | save_file.write('\n') 232 | save_file.close() 233 | try: 234 | os.makedirs('plot_curve/roc/nature') 235 | except os.error: 236 | pass 237 | np.savez('plot_curve/roc/nature/match_'+file_name, fpr,tpr) 238 | except KeyboardInterrupt: 239 | print('INTERRUPTED') 240 | coord.request_stop() 241 | finally: 242 | # When done, ask the threads to stop. 243 | coord.request_stop() 244 | coord.join(threads) 245 | 246 | if __name__ == '__main__': 247 | 248 | # mc_train() 249 | 250 | test_data = ['field', 'forest','indoor','mountain','oldbuilding','street','urban','water'] 251 | for i in range(8): 252 | gfm_test(test_data[i]) 253 | 254 | 255 | -------------------------------------------------------------------------------- /part_1_feature_difference/train_nature_2ch.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Created on 2018.11.9 5 | 6 | @author: wrj 7 | """ 8 | ''' 9 | 2 channel + 交叉熵损失函数 + 参数共享 10 | ''' 11 | 12 | import numpy as np 13 | import math 14 | import time 15 | import model_d_2ch as model 16 | import read_data 17 | import tensorflow as tf 18 | import os 19 | from datetime import datetime 20 | import logging 21 | from sklearn.metrics import roc_curve, auc 22 | 23 | os.environ['CUDA_VISIBLE_DEVICES']='0' 24 | 25 | BATCH_SIZE_matching=100 26 | epoch = 41 27 | learning_rate = 2e-4 28 | image_width = 64 29 | image_height = 64 30 | 31 | checkpoint_dir = 'ckpt_2ch' 32 | checkpoint_file = os.path.join(checkpoint_dir, 'model.ckpt') 33 | train_dir='summary' 34 | 35 | def initLogging(logFilename='record_2ch.log'): 36 | """Init for logging 37 | """ 38 | logging.basicConfig( 39 | level = logging.DEBUG, 40 | format='%(asctime)s-%(levelname)s-%(message)s', 41 | datefmt = '%y-%m-%d %H:%M', 42 | filename = logFilename, 43 | filemode = 'w'); 44 | console = logging.StreamHandler() 45 | console.setLevel(logging.INFO) 46 | formatter = logging.Formatter('%(asctime)s-%(levelname)s-%(message)s') 47 | console.setFormatter(formatter) 48 | logging.getLogger('').addHandler(console) 49 | 50 | 51 | def mc_train(): 52 | initLogging() 53 | current_time = datetime.now().strftime('%Y-%m-%d') 54 | try: 55 | os.makedirs(checkpoint_dir) 56 | except os.error: 57 | pass 58 | 59 | data1 = np.load('/home/ws/文档/wrj/data_all/country.npz') 60 | train_matching_y = data1['arr_1'][:,np.newaxis] 61 | numbers_train = train_matching_y.shape[0] #训练集总数 62 | epoch_steps = np.int(numbers_train / BATCH_SIZE_matching) +1 # 一个epoch有多少个steps 63 | 64 | all_loss = np.array([]) 65 | graph = tf.Graph() 66 | with graph.as_default(): 67 | inputs_p1 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width,1], name='inputs_p1') 68 | inputs_p2 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width,1], name='inputs_p2') 69 | label_m = tf.placeholder(tf.float32, [BATCH_SIZE_matching, 1], name='label_m') 70 | 71 | # 训练 M 72 | match_loss, match_output, all_features = model.match_network(inputs_p1, inputs_p2, label_m) 73 | match_out = tf.round(match_output) 74 | m_correct,m_numbers = model.evaluation(match_out, label_m) 75 | 76 | m_train_opt = tf.train.AdamOptimizer(learning_rate).minimize(match_loss) 77 | 78 | filename = '/home/ws/文档/wrj/data_all/country/country.tfrecord' 79 | filename_queue = tf.train.string_input_producer([filename],num_epochs=epoch,shuffle=True) 80 | img_batch, label_batch = read_data.batch_inputs(filename_queue, train = True, batch_size = BATCH_SIZE_matching) 81 | tf.summary.scalar('mathing_loss', match_loss) 82 | 83 | summary = tf.summary.merge_all() 84 | saver = tf.train.Saver(max_to_keep=20) 85 | init = tf.global_variables_initializer() 86 | 87 | gpu_options = tf.GPUOptions(allow_growth=True) 88 | sess_config = tf.ConfigProto(gpu_options=gpu_options) 89 | 90 | with tf.Session(config=sess_config) as sess: 91 | summary_writer = tf.summary.FileWriter(train_dir, sess.graph) 92 | sess.run(tf.local_variables_initializer()) 93 | sess.run(init) 94 | 95 | try: 96 | coord = tf.train.Coordinator() 97 | threads = tf.train.start_queue_runners(sess=sess, coord=coord) 98 | step = 0 99 | 100 | while not coord.should_stop(): 101 | start_time = time.time() 102 | step = step + 1 103 | batch, l_batch = sess.run([img_batch, label_batch]) 104 | l_batch = l_batch 105 | x_batch = batch[:,:,:64,np.newaxis] 106 | y_batch = batch[:,:,64:,np.newaxis] 107 | feed_dict = {inputs_p1:x_batch, inputs_p2:y_batch, label_m:l_batch} 108 | _, m_loss, m_output_= sess.run([m_train_opt,match_loss,match_output], feed_dict = feed_dict) 109 | 110 | if step % 10 ==0: 111 | loss_write = np.array([[step, m_loss]]) 112 | if step ==10: 113 | all_loss = loss_write 114 | else: 115 | all_loss = np.concatenate((all_loss, loss_write)) 116 | 117 | if step % 100 == 0: 118 | duration = time.time() - start_time 119 | summary_str = sess.run(summary, feed_dict=feed_dict) 120 | summary_writer.add_summary(summary_str, step) 121 | summary_writer.flush() 122 | 123 | logging.info('>> Step %d run_train: matching_loss = %.3f (%.3f sec)' 124 | % (step, m_loss, duration)) 125 | 126 | if (step % epoch_steps == 0) and ((step / epoch_steps)%3 == 0): 127 | current_epoch = int(step / epoch_steps) 128 | 129 | if current_epoch > 15: 130 | logging.info('>> %s Saving in %s' % (datetime.now(), checkpoint_dir)) 131 | saver.save(sess, checkpoint_file, global_step=current_epoch) 132 | mc_test_all(current_epoch) 133 | 134 | except KeyboardInterrupt: 135 | print('INTERRUPTED') 136 | coord.request_stop() 137 | except Exception as e: 138 | coord.request_stop(e) 139 | 140 | finally: 141 | saver.save(sess, checkpoint_file, global_step=step) 142 | np.save(os.path.join(checkpoint_dir,'ckpt_2ch'), all_loss) 143 | print('Model saved in file :%s'%checkpoint_dir) 144 | # When done, ask the threads to stop. 145 | coord.request_stop() 146 | coord.join(threads) 147 | 148 | ''' 149 | test the network 150 | ''' 151 | def mc_test(file_name, number): 152 | data = np.load('/home/ws/文档/wrj/data_all/'+file_name+'.npz') 153 | train_matching_y = data['arr_1'][:,np.newaxis] 154 | numbers_train = train_matching_y.shape[0] #训练集总数 155 | 156 | graph = tf.Graph() 157 | with graph.as_default(): 158 | inputs_p1 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width, 1], name='inputs_p1') 159 | inputs_p2 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width, 1], name='inputs_p2') 160 | label_m = tf.placeholder(tf.float32, [BATCH_SIZE_matching, 1], name='label_m') 161 | 162 | match_loss, match_output, all_features = model.match_network(inputs_p1, inputs_p2, label_m) 163 | match_out = tf.round(match_output) 164 | m_correct,m_numbers = model.evaluation(match_out, label_m) 165 | 166 | filename = '/home/ws/文档/wrj/data_all/country/'+file_name+'.tfrecord' 167 | filename_queue = tf.train.string_input_producer([filename],num_epochs=1,shuffle=False) 168 | img_batch, label_batch = read_data.batch_inputs(filename_queue, train = False, batch_size = BATCH_SIZE_matching) 169 | 170 | saver = tf.train.Saver() 171 | 172 | gpu_options = tf.GPUOptions(allow_growth=True) 173 | sess_config = tf.ConfigProto(gpu_options=gpu_options) 174 | 175 | with tf.Session(config=sess_config) as sess: 176 | sess.run(tf.local_variables_initializer()) 177 | # saver.restore(sess, tf.train.latest_checkpoint('2_ch')) 178 | saver.restore(sess, '2_ch/model.ckpt-33') 179 | m_count = 0 # Counts the number of correct predictions. 180 | num = numbers_train 181 | matching_out = np.array([]) 182 | matching_label = np.array([]) 183 | 184 | try: 185 | coord = tf.train.Coordinator() 186 | threads = tf.train.start_queue_runners(sess=sess, coord=coord) 187 | step_test = 0 188 | try: 189 | while not coord.should_stop(): 190 | step_test = step_test + 1 191 | batch, l_batch = sess.run([img_batch, label_batch]) 192 | l_batch = l_batch 193 | x_batch = batch[:,:,:64,np.newaxis] 194 | y_batch = batch[:,:,64:,np.newaxis] 195 | feed_dict = {inputs_p1:x_batch, inputs_p2:y_batch, label_m:l_batch} 196 | m_correct_, m_output_ = sess.run([m_correct,match_output], feed_dict = feed_dict) 197 | 198 | if step_test == 1: 199 | matching_out = m_output_ 200 | matching_label = l_batch 201 | elif(l_batch.size == BATCH_SIZE_matching): 202 | matching_out = np.concatenate((matching_out, m_output_)) 203 | matching_label = np.concatenate((matching_label, l_batch)) 204 | 205 | m_count = m_count + m_correct_.astype(int) 206 | if step_test % 100 == 0: 207 | print('Step %d/%d run_test: batch_precision = %.2f '% (step_test, num/100, m_correct_/BATCH_SIZE_matching)) 208 | except Exception as e: 209 | coord.request_stop(e) 210 | m_precision = float(m_count) / num 211 | 212 | print('Num examples: %d Num correct: %d match Precision : %0.04f ' %(num, m_count, m_precision)) 213 | save_file = open('test_2ch.txt','a') 214 | save_file.write(file_name + ' epoch: ' + str(number) +'\n'+'num correct: '+str(m_count)+'/'+str(num)+' match precision : '+str(m_precision)) 215 | save_file.write(' ') 216 | 217 | # 绘制ROC曲线 218 | fpr,tpr,threshold = roc_curve(matching_label, matching_out) ###计算真正率和假正率 219 | 220 | q = np.where(0.95<=tpr ) 221 | q_value = q[0][0] 222 | fpr95 = fpr[q_value] 223 | 224 | save_file.write('match fpr95 : '+str(fpr95*100)) 225 | save_file.write('\n') 226 | save_file.write('\n') 227 | save_file.close() 228 | roc_dir = 'plot_curve/nature_2ch/epoch_'+str(number) 229 | try: 230 | os.makedirs(roc_dir) 231 | except os.error: 232 | pass 233 | np.savez(roc_dir+'/match_2ch_'+file_name, fpr,tpr) 234 | except KeyboardInterrupt: 235 | print('INTERRUPTED') 236 | coord.request_stop() 237 | finally: 238 | # When done, ask the threads to stop. 239 | coord.request_stop() 240 | coord.join(threads) 241 | 242 | ''' 243 | test all datasets 244 | ''' 245 | def mc_test_all(number): 246 | test_data = ['field', 'forest','indoor','mountain','oldbuilding','street','urban','water'] 247 | for i in range(8): 248 | mc_test(test_data[i],number) 249 | 250 | if __name__ == '__main__': 251 | 252 | # mc_train() 253 | # mc_test('field',1) 254 | mc_test_all(2) 255 | 256 | 257 | 258 | -------------------------------------------------------------------------------- /part_1_feature_difference/train_nature_map32.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Created on 2018.11.9 5 | 6 | @author: wrj 7 | """ 8 | ''' 9 | 双分支 + 特征map32 + 交叉熵损失函数 + 参数共享 10 | ''' 11 | 12 | import numpy as np 13 | import math 14 | import time 15 | import model_d_map32 as model 16 | import read_data 17 | import tensorflow as tf 18 | import os 19 | from datetime import datetime 20 | import logging 21 | from sklearn.metrics import roc_curve, auc 22 | 23 | os.environ['CUDA_VISIBLE_DEVICES']='0' 24 | 25 | BATCH_SIZE_matching=100 26 | epoch = 40 27 | learning_rate = 2e-4 28 | image_width = 64 29 | image_height = 64 30 | 31 | checkpoint_dir = 'ckpt_map32' 32 | checkpoint_file = os.path.join(checkpoint_dir, 'model.ckpt') 33 | train_dir='summary' 34 | 35 | def initLogging(logFilename='record_map32.log'): 36 | """Init for logging 37 | """ 38 | logging.basicConfig( 39 | level = logging.DEBUG, 40 | format='%(asctime)s-%(levelname)s-%(message)s', 41 | datefmt = '%y-%m-%d %H:%M', 42 | filename = logFilename, 43 | filemode = 'w'); 44 | console = logging.StreamHandler() 45 | console.setLevel(logging.INFO) 46 | formatter = logging.Formatter('%(asctime)s-%(levelname)s-%(message)s') 47 | console.setFormatter(formatter) 48 | logging.getLogger('').addHandler(console) 49 | 50 | 51 | def mc_train(): 52 | initLogging() 53 | current_time = datetime.now().strftime('%Y-%m-%d') 54 | try: 55 | os.makedirs(checkpoint_dir) 56 | except os.error: 57 | pass 58 | 59 | data1 = np.load('/home/ws/文档/wrj/data_all/country.npz') 60 | train_matching_y = data1['arr_1'][:,np.newaxis] 61 | numbers_train = train_matching_y.shape[0] #训练集总数 62 | epoch_steps = np.int(numbers_train / BATCH_SIZE_matching) +1 # 一个epoch有多少个steps 63 | 64 | all_loss = np.array([]) 65 | graph = tf.Graph() 66 | with graph.as_default(): 67 | inputs_p1 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width,1], name='inputs_p1') 68 | inputs_p2 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width,1], name='inputs_p2') 69 | label_m = tf.placeholder(tf.float32, [BATCH_SIZE_matching, 1], name='label_m') 70 | 71 | # 训练 M 72 | match_loss, match_output, all_features = model.match_network(inputs_p1, inputs_p2, label_m) 73 | match_out = tf.round(match_output) 74 | m_correct,m_numbers = model.evaluation(match_out, label_m) 75 | 76 | m_train_opt = tf.train.AdamOptimizer(learning_rate).minimize(match_loss) 77 | 78 | filename = '/home/ws/文档/wrj/data_all/country/country.tfrecord' 79 | filename_queue = tf.train.string_input_producer([filename],num_epochs=epoch,shuffle=True) 80 | img_batch, label_batch = read_data.batch_inputs(filename_queue, train = True, batch_size = BATCH_SIZE_matching) 81 | tf.summary.scalar('mathing_loss', match_loss) 82 | 83 | summary = tf.summary.merge_all() 84 | saver = tf.train.Saver(max_to_keep=20) 85 | init = tf.global_variables_initializer() 86 | 87 | gpu_options = tf.GPUOptions(allow_growth=True) 88 | sess_config = tf.ConfigProto(gpu_options=gpu_options) 89 | 90 | with tf.Session(config=sess_config) as sess: 91 | summary_writer = tf.summary.FileWriter(train_dir, sess.graph) 92 | sess.run(tf.local_variables_initializer()) 93 | sess.run(init) 94 | 95 | try: 96 | coord = tf.train.Coordinator() 97 | threads = tf.train.start_queue_runners(sess=sess, coord=coord) 98 | step = 0 99 | 100 | while not coord.should_stop(): 101 | start_time = time.time() 102 | step = step + 1 103 | batch, l_batch = sess.run([img_batch, label_batch]) 104 | l_batch = l_batch 105 | x_batch = batch[:,:,:64,np.newaxis] 106 | y_batch = batch[:,:,64:,np.newaxis] 107 | feed_dict = {inputs_p1:x_batch, inputs_p2:y_batch, label_m:l_batch} 108 | _, m_loss, m_output_= sess.run([m_train_opt,match_loss,match_output], feed_dict = feed_dict) 109 | 110 | if step % 10 ==0: 111 | loss_write = np.array([[step, m_loss]]) 112 | if step ==10: 113 | all_loss = loss_write 114 | else: 115 | all_loss = np.concatenate((all_loss, loss_write)) 116 | 117 | if step % 100 == 0: 118 | duration = time.time() - start_time 119 | summary_str = sess.run(summary, feed_dict=feed_dict) 120 | summary_writer.add_summary(summary_str, step) 121 | summary_writer.flush() 122 | 123 | logging.info('>> Step %d run_train: matching_loss = %.3f (%.3f sec)' 124 | % (step, m_loss, duration)) 125 | 126 | if (step % epoch_steps == 0) and ((step / epoch_steps)%3 == 0): 127 | current_epoch = int(step / epoch_steps) 128 | 129 | if current_epoch > 20: 130 | logging.info('>> %s Saving in %s' % (datetime.now(), checkpoint_dir)) 131 | saver.save(sess, checkpoint_file, global_step=current_epoch) 132 | mc_test_all(current_epoch) 133 | 134 | except KeyboardInterrupt: 135 | print('INTERRUPTED') 136 | coord.request_stop() 137 | except Exception as e: 138 | coord.request_stop(e) 139 | 140 | finally: 141 | saver.save(sess, checkpoint_file, global_step=step) 142 | np.save(os.path.join(checkpoint_dir,'ckpt_map32'), all_loss) 143 | print('Model saved in file :%s'%checkpoint_dir) 144 | # When done, ask the threads to stop. 145 | coord.request_stop() 146 | coord.join(threads) 147 | 148 | ''' 149 | test the network 150 | ''' 151 | def mc_test(file_name, number): 152 | data = np.load('/home/ws/文档/wrj/data_all/'+file_name+'.npz') 153 | train_matching_y = data['arr_1'][:,np.newaxis] 154 | numbers_train = train_matching_y.shape[0] #训练集总数 155 | 156 | graph = tf.Graph() 157 | with graph.as_default(): 158 | inputs_p1 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width, 1], name='inputs_p1') 159 | inputs_p2 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width, 1], name='inputs_p2') 160 | label_m = tf.placeholder(tf.float32, [BATCH_SIZE_matching, 1], name='label_m') 161 | 162 | match_loss, match_output, all_features = model.match_network(inputs_p1, inputs_p2, label_m) 163 | match_out = tf.round(match_output) 164 | m_correct,m_numbers = model.evaluation(match_out, label_m) 165 | 166 | filename = '/home/ws/文档/wrj/data_all/country/'+file_name+'.tfrecord' 167 | filename_queue = tf.train.string_input_producer([filename],num_epochs=1,shuffle=False) 168 | img_batch, label_batch = read_data.batch_inputs(filename_queue, train = False, batch_size = BATCH_SIZE_matching) 169 | 170 | saver = tf.train.Saver() 171 | 172 | gpu_options = tf.GPUOptions(allow_growth=True) 173 | sess_config = tf.ConfigProto(gpu_options=gpu_options) 174 | 175 | with tf.Session(config=sess_config) as sess: 176 | sess.run(tf.local_variables_initializer()) 177 | # saver.restore(sess, tf.train.latest_checkpoint('ckpt_map32')) 178 | saver.restore(sess, 'sia_map/model.ckpt-36') 179 | m_count = 0 # Counts the number of correct predictions. 180 | num = numbers_train 181 | matching_out = np.array([]) 182 | matching_label = np.array([]) 183 | 184 | try: 185 | coord = tf.train.Coordinator() 186 | threads = tf.train.start_queue_runners(sess=sess, coord=coord) 187 | step_test = 0 188 | try: 189 | while not coord.should_stop(): 190 | step_test = step_test + 1 191 | batch, l_batch = sess.run([img_batch, label_batch]) 192 | l_batch = l_batch 193 | x_batch = batch[:,:,:64,np.newaxis] 194 | y_batch = batch[:,:,64:,np.newaxis] 195 | feed_dict = {inputs_p1:x_batch, inputs_p2:y_batch, label_m:l_batch} 196 | m_correct_, m_output_ = sess.run([m_correct,match_output], feed_dict = feed_dict) 197 | 198 | if step_test == 1: 199 | matching_out = m_output_ 200 | matching_label = l_batch 201 | elif(l_batch.size == BATCH_SIZE_matching): 202 | matching_out = np.concatenate((matching_out, m_output_)) 203 | matching_label = np.concatenate((matching_label, l_batch)) 204 | 205 | m_count = m_count + m_correct_.astype(int) 206 | if step_test % 100 == 0: 207 | print('Step %d/%d run_test: batch_precision = %.2f '% (step_test, num/100, m_correct_/BATCH_SIZE_matching)) 208 | except Exception as e: 209 | coord.request_stop(e) 210 | m_precision = float(m_count) / num 211 | 212 | print('Num examples: %d Num correct: %d match Precision : %0.04f ' %(num, m_count, m_precision)) 213 | save_file = open('test_map32.txt','a') 214 | save_file.write(file_name + ' epoch: ' + str(number) +'\n'+'num correct: '+str(m_count)+'/'+str(num)+' match precision : '+str(m_precision)) 215 | save_file.write(' ') 216 | 217 | # 绘制ROC曲线 218 | fpr,tpr,threshold = roc_curve(matching_label, matching_out) ###计算真正率和假正率 219 | 220 | q = np.where(0.95<=tpr ) 221 | q_value = q[0][0] 222 | fpr95 = fpr[q_value] 223 | 224 | save_file.write('match fpr95 : '+str(fpr95*100)) 225 | save_file.write('\n') 226 | save_file.write('\n') 227 | save_file.close() 228 | roc_dir = 'plot_curve/nature_map32/epoch_'+str(number) 229 | try: 230 | os.makedirs(roc_dir) 231 | except os.error: 232 | pass 233 | np.savez(roc_dir+'/map32_'+file_name, fpr,tpr) 234 | except KeyboardInterrupt: 235 | print('INTERRUPTED') 236 | coord.request_stop() 237 | finally: 238 | # When done, ask the threads to stop. 239 | coord.request_stop() 240 | coord.join(threads) 241 | 242 | ''' 243 | test all datasets 244 | ''' 245 | def mc_test_all(number): 246 | test_data = ['field', 'forest','indoor','mountain','oldbuilding','street','urban','water'] 247 | for i in range(8): 248 | mc_test(test_data[i],number) 249 | 250 | if __name__ == '__main__': 251 | 252 | # mc_train() 253 | # mc_test('field',1) 254 | mc_test_all(1) 255 | 256 | 257 | 258 | -------------------------------------------------------------------------------- /part_1_feature_difference/train_nature_map32_diff.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Created on 2018.11.9 5 | 6 | @author: wrj 7 | """ 8 | ''' 9 | 双分支 + 特征map32 + 交叉熵损失函数 + 参数共享 10 | ''' 11 | 12 | import numpy as np 13 | import math 14 | import time 15 | import model_d_map32_diff as model 16 | import read_data 17 | import tensorflow as tf 18 | import os 19 | from datetime import datetime 20 | import logging 21 | from sklearn.metrics import roc_curve, auc 22 | 23 | os.environ['CUDA_VISIBLE_DEVICES']='0' 24 | 25 | BATCH_SIZE_matching=100 26 | epoch = 40 27 | learning_rate = 2e-4 28 | image_width = 64 29 | image_height = 64 30 | 31 | checkpoint_dir = 'ckpt_map32_diff' 32 | checkpoint_file = os.path.join(checkpoint_dir, 'model.ckpt') 33 | train_dir='summary' 34 | 35 | def initLogging(logFilename='record_map32_diff.log'): 36 | """Init for logging 37 | """ 38 | logging.basicConfig( 39 | level = logging.DEBUG, 40 | format='%(asctime)s-%(levelname)s-%(message)s', 41 | datefmt = '%y-%m-%d %H:%M', 42 | filename = logFilename, 43 | filemode = 'w'); 44 | console = logging.StreamHandler() 45 | console.setLevel(logging.INFO) 46 | formatter = logging.Formatter('%(asctime)s-%(levelname)s-%(message)s') 47 | console.setFormatter(formatter) 48 | logging.getLogger('').addHandler(console) 49 | 50 | 51 | def mc_train(): 52 | initLogging() 53 | current_time = datetime.now().strftime('%Y-%m-%d') 54 | try: 55 | os.makedirs(checkpoint_dir) 56 | except os.error: 57 | pass 58 | 59 | data1 = np.load('/home/ws/文档/wrj/data_all/country.npz') 60 | train_matching_y = data1['arr_1'][:,np.newaxis] 61 | numbers_train = train_matching_y.shape[0] #训练集总数 62 | epoch_steps = np.int(numbers_train / BATCH_SIZE_matching) +1 # 一个epoch有多少个steps 63 | 64 | all_loss = np.array([]) 65 | graph = tf.Graph() 66 | with graph.as_default(): 67 | inputs_p1 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width,1], name='inputs_p1') 68 | inputs_p2 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width,1], name='inputs_p2') 69 | label_m = tf.placeholder(tf.float32, [BATCH_SIZE_matching, 1], name='label_m') 70 | 71 | # 训练 M 72 | match_loss, match_output, all_features = model.match_network(inputs_p1, inputs_p2, label_m) 73 | match_out = tf.round(match_output) 74 | m_correct,m_numbers = model.evaluation(match_out, label_m) 75 | 76 | m_train_opt = tf.train.AdamOptimizer(learning_rate).minimize(match_loss) 77 | 78 | filename = '/home/ws/文档/wrj/data_all/country/country.tfrecord' 79 | filename_queue = tf.train.string_input_producer([filename],num_epochs=epoch,shuffle=True) 80 | img_batch, label_batch = read_data.batch_inputs(filename_queue, train = True, batch_size = BATCH_SIZE_matching) 81 | tf.summary.scalar('mathing_loss', match_loss) 82 | 83 | summary = tf.summary.merge_all() 84 | saver = tf.train.Saver(max_to_keep=20) 85 | init = tf.global_variables_initializer() 86 | 87 | gpu_options = tf.GPUOptions(allow_growth=True) 88 | sess_config = tf.ConfigProto(gpu_options=gpu_options) 89 | 90 | with tf.Session(config=sess_config) as sess: 91 | summary_writer = tf.summary.FileWriter(train_dir, sess.graph) 92 | sess.run(tf.local_variables_initializer()) 93 | sess.run(init) 94 | 95 | try: 96 | coord = tf.train.Coordinator() 97 | threads = tf.train.start_queue_runners(sess=sess, coord=coord) 98 | step = 0 99 | 100 | while not coord.should_stop(): 101 | start_time = time.time() 102 | step = step + 1 103 | batch, l_batch = sess.run([img_batch, label_batch]) 104 | l_batch = l_batch 105 | x_batch = batch[:,:,:64,np.newaxis] 106 | y_batch = batch[:,:,64:,np.newaxis] 107 | feed_dict = {inputs_p1:x_batch, inputs_p2:y_batch, label_m:l_batch} 108 | _, m_loss, m_output_= sess.run([m_train_opt,match_loss,match_output], feed_dict = feed_dict) 109 | 110 | if step % 10 ==0: 111 | loss_write = np.array([[step, m_loss]]) 112 | if step ==10: 113 | all_loss = loss_write 114 | else: 115 | all_loss = np.concatenate((all_loss, loss_write)) 116 | 117 | if step % 100 == 0: 118 | duration = time.time() - start_time 119 | summary_str = sess.run(summary, feed_dict=feed_dict) 120 | summary_writer.add_summary(summary_str, step) 121 | summary_writer.flush() 122 | 123 | logging.info('>> Step %d run_train: matching_loss = %.3f (%.3f sec)' 124 | % (step, m_loss, duration)) 125 | 126 | if (step % epoch_steps == 0) and ((step / epoch_steps)%3 == 0): 127 | current_epoch = int(step / epoch_steps) 128 | 129 | if current_epoch > 15: 130 | logging.info('>> %s Saving in %s' % (datetime.now(), checkpoint_dir)) 131 | saver.save(sess, checkpoint_file, global_step=current_epoch) 132 | mc_test_all(current_epoch) 133 | 134 | except KeyboardInterrupt: 135 | print('INTERRUPTED') 136 | coord.request_stop() 137 | except Exception as e: 138 | coord.request_stop(e) 139 | 140 | finally: 141 | saver.save(sess, checkpoint_file, global_step=step) 142 | np.save(os.path.join(checkpoint_dir,'ckpt_map32_diff'), all_loss) 143 | print('Model saved in file :%s'%checkpoint_dir) 144 | # When done, ask the threads to stop. 145 | coord.request_stop() 146 | coord.join(threads) 147 | 148 | ''' 149 | test the network 150 | ''' 151 | def mc_test(file_name, number): 152 | data = np.load('/home/ws/文档/wrj/data_all/'+file_name+'.npz') 153 | train_matching_y = data['arr_1'][:,np.newaxis] 154 | numbers_train = train_matching_y.shape[0] #训练集总数 155 | 156 | graph = tf.Graph() 157 | with graph.as_default(): 158 | inputs_p1 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width, 1], name='inputs_p1') 159 | inputs_p2 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width, 1], name='inputs_p2') 160 | label_m = tf.placeholder(tf.float32, [BATCH_SIZE_matching, 1], name='label_m') 161 | 162 | match_loss, match_output, all_features = model.match_network(inputs_p1, inputs_p2, label_m) 163 | match_out = tf.round(match_output) 164 | m_correct,m_numbers = model.evaluation(match_out, label_m) 165 | 166 | filename = '/home/ws/文档/wrj/data_all/country/'+file_name+'.tfrecord' 167 | filename_queue = tf.train.string_input_producer([filename],num_epochs=1,shuffle=False) 168 | img_batch, label_batch = read_data.batch_inputs(filename_queue, train = False, batch_size = BATCH_SIZE_matching) 169 | 170 | saver = tf.train.Saver() 171 | 172 | gpu_options = tf.GPUOptions(allow_growth=True) 173 | sess_config = tf.ConfigProto(gpu_options=gpu_options) 174 | 175 | with tf.Session(config=sess_config) as sess: 176 | sess.run(tf.local_variables_initializer()) 177 | # saver.restore(sess, tf.train.latest_checkpoint('ckpt_map32_diff')) 178 | saver.restore(sess, 'sia_map_diff/model.ckpt-39') 179 | m_count = 0 # Counts the number of correct predictions. 180 | num = numbers_train 181 | matching_out = np.array([]) 182 | matching_label = np.array([]) 183 | 184 | try: 185 | coord = tf.train.Coordinator() 186 | threads = tf.train.start_queue_runners(sess=sess, coord=coord) 187 | step_test = 0 188 | try: 189 | while not coord.should_stop(): 190 | step_test = step_test + 1 191 | batch, l_batch = sess.run([img_batch, label_batch]) 192 | l_batch = l_batch 193 | x_batch = batch[:,:,:64,np.newaxis] 194 | y_batch = batch[:,:,64:,np.newaxis] 195 | feed_dict = {inputs_p1:x_batch, inputs_p2:y_batch, label_m:l_batch} 196 | m_correct_, m_output_ = sess.run([m_correct,match_output], feed_dict = feed_dict) 197 | 198 | if step_test == 1: 199 | matching_out = m_output_ 200 | matching_label = l_batch 201 | elif(l_batch.size == BATCH_SIZE_matching): 202 | matching_out = np.concatenate((matching_out, m_output_)) 203 | matching_label = np.concatenate((matching_label, l_batch)) 204 | 205 | m_count = m_count + m_correct_.astype(int) 206 | if step_test % 100 == 0: 207 | print('Step %d/%d run_test: batch_precision = %.2f '% (step_test, num/100, m_correct_/BATCH_SIZE_matching)) 208 | except Exception as e: 209 | coord.request_stop(e) 210 | m_precision = float(m_count) / num 211 | 212 | print('Num examples: %d Num correct: %d match Precision : %0.04f ' %(num, m_count, m_precision)) 213 | save_file = open('test_map32_diff.txt','a') 214 | save_file.write(file_name + ' epoch: ' + str(number) +'\n'+'num correct: '+str(m_count)+'/'+str(num)+' match precision : '+str(m_precision)) 215 | save_file.write(' ') 216 | 217 | # 绘制ROC曲线 218 | fpr,tpr,threshold = roc_curve(matching_label, matching_out) ###计算真正率和假正率 219 | 220 | q = np.where(0.95<=tpr ) 221 | q_value = q[0][0] 222 | fpr95 = fpr[q_value] 223 | 224 | save_file.write('match fpr95 : '+str(fpr95*100)) 225 | save_file.write('\n') 226 | save_file.write('\n') 227 | save_file.close() 228 | roc_dir = 'plot_curve/nature_map32_diff/epoch_'+str(number) 229 | try: 230 | os.makedirs(roc_dir) 231 | except os.error: 232 | pass 233 | np.savez(roc_dir+'/match_map32_diff_'+file_name, fpr,tpr) 234 | except KeyboardInterrupt: 235 | print('INTERRUPTED') 236 | coord.request_stop() 237 | finally: 238 | # When done, ask the threads to stop. 239 | coord.request_stop() 240 | coord.join(threads) 241 | 242 | ''' 243 | test all datasets 244 | ''' 245 | def mc_test_all(number): 246 | test_data = ['field', 'forest','indoor','mountain','oldbuilding','street','urban','water'] 247 | for i in range(8): 248 | mc_test(test_data[i],number) 249 | 250 | if __name__ == '__main__': 251 | 252 | # mc_train() 253 | # mc_test('field',1) 254 | mc_test_all(1) 255 | 256 | 257 | 258 | -------------------------------------------------------------------------------- /part_1_feature_difference/train_nature_map32_no.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Created on 2018.11.9 5 | 6 | @author: wrj 7 | """ 8 | ''' 9 | 双分支 + 特征map32 + 交叉熵损失函数 + 参数共享 10 | ''' 11 | 12 | import numpy as np 13 | import math 14 | import time 15 | import model_d_map32_no as model 16 | import read_data 17 | import tensorflow as tf 18 | import os 19 | from datetime import datetime 20 | import logging 21 | from sklearn.metrics import roc_curve, auc 22 | 23 | os.environ['CUDA_VISIBLE_DEVICES']='0' 24 | 25 | BATCH_SIZE_matching=100 26 | epoch = 40 27 | learning_rate = 2e-4 28 | image_width = 64 29 | image_height = 64 30 | 31 | checkpoint_dir = 'ckpt_map32_no' 32 | checkpoint_file = os.path.join(checkpoint_dir, 'model.ckpt') 33 | train_dir='summary' 34 | 35 | def initLogging(logFilename='record_map32_no.log'): 36 | """Init for logging 37 | """ 38 | logging.basicConfig( 39 | level = logging.DEBUG, 40 | format='%(asctime)s-%(levelname)s-%(message)s', 41 | datefmt = '%y-%m-%d %H:%M', 42 | filename = logFilename, 43 | filemode = 'w'); 44 | console = logging.StreamHandler() 45 | console.setLevel(logging.INFO) 46 | formatter = logging.Formatter('%(asctime)s-%(levelname)s-%(message)s') 47 | console.setFormatter(formatter) 48 | logging.getLogger('').addHandler(console) 49 | 50 | 51 | def mc_train(): 52 | initLogging() 53 | current_time = datetime.now().strftime('%Y-%m-%d') 54 | try: 55 | os.makedirs(checkpoint_dir) 56 | except os.error: 57 | pass 58 | 59 | data1 = np.load('/home/ws/文档/wrj/data_all/country.npz') 60 | train_matching_y = data1['arr_1'][:,np.newaxis] 61 | numbers_train = train_matching_y.shape[0] #训练集总数 62 | epoch_steps = np.int(numbers_train / BATCH_SIZE_matching) +1 # 一个epoch有多少个steps 63 | 64 | all_loss = np.array([]) 65 | graph = tf.Graph() 66 | with graph.as_default(): 67 | inputs_p1 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width,1], name='inputs_p1') 68 | inputs_p2 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width,1], name='inputs_p2') 69 | label_m = tf.placeholder(tf.float32, [BATCH_SIZE_matching, 1], name='label_m') 70 | 71 | # 训练 M 72 | match_loss, match_output, all_features = model.match_network(inputs_p1, inputs_p2, label_m) 73 | match_out = tf.round(match_output) 74 | m_correct,m_numbers = model.evaluation(match_out, label_m) 75 | 76 | m_train_opt = tf.train.AdamOptimizer(learning_rate).minimize(match_loss) 77 | 78 | filename = '/home/ws/文档/wrj/data_all/country/country.tfrecord' 79 | filename_queue = tf.train.string_input_producer([filename],num_epochs=epoch,shuffle=True) 80 | img_batch, label_batch = read_data.batch_inputs(filename_queue, train = True, batch_size = BATCH_SIZE_matching) 81 | tf.summary.scalar('mathing_loss', match_loss) 82 | 83 | summary = tf.summary.merge_all() 84 | saver = tf.train.Saver(max_to_keep=20) 85 | init = tf.global_variables_initializer() 86 | 87 | gpu_options = tf.GPUOptions(allow_growth=True) 88 | sess_config = tf.ConfigProto(gpu_options=gpu_options) 89 | 90 | with tf.Session(config=sess_config) as sess: 91 | summary_writer = tf.summary.FileWriter(train_dir, sess.graph) 92 | sess.run(tf.local_variables_initializer()) 93 | sess.run(init) 94 | 95 | try: 96 | coord = tf.train.Coordinator() 97 | threads = tf.train.start_queue_runners(sess=sess, coord=coord) 98 | step = 0 99 | 100 | while not coord.should_stop(): 101 | start_time = time.time() 102 | step = step + 1 103 | batch, l_batch = sess.run([img_batch, label_batch]) 104 | l_batch = l_batch 105 | x_batch = batch[:,:,:64,np.newaxis] 106 | y_batch = batch[:,:,64:,np.newaxis] 107 | feed_dict = {inputs_p1:x_batch, inputs_p2:y_batch, label_m:l_batch} 108 | _, m_loss, m_output_= sess.run([m_train_opt,match_loss,match_output], feed_dict = feed_dict) 109 | 110 | if step % 10 ==0: 111 | loss_write = np.array([[step, m_loss]]) 112 | if step ==10: 113 | all_loss = loss_write 114 | else: 115 | all_loss = np.concatenate((all_loss, loss_write)) 116 | 117 | if step % 100 == 0: 118 | duration = time.time() - start_time 119 | summary_str = sess.run(summary, feed_dict=feed_dict) 120 | summary_writer.add_summary(summary_str, step) 121 | summary_writer.flush() 122 | 123 | logging.info('>> Step %d run_train: matching_loss = %.3f (%.3f sec)' 124 | % (step, m_loss, duration)) 125 | 126 | if (step % epoch_steps == 0) and ((step / epoch_steps)%3 == 0): 127 | current_epoch = int(step / epoch_steps) 128 | 129 | if current_epoch > 15: 130 | logging.info('>> %s Saving in %s' % (datetime.now(), checkpoint_dir)) 131 | saver.save(sess, checkpoint_file, global_step=current_epoch) 132 | mc_test_all(current_epoch) 133 | 134 | except KeyboardInterrupt: 135 | print('INTERRUPTED') 136 | coord.request_stop() 137 | except Exception as e: 138 | coord.request_stop(e) 139 | 140 | finally: 141 | saver.save(sess, checkpoint_file, global_step=step) 142 | np.save(os.path.join(checkpoint_dir,'ckpt_map32_no'), all_loss) 143 | print('Model saved in file :%s'%checkpoint_dir) 144 | # When done, ask the threads to stop. 145 | coord.request_stop() 146 | coord.join(threads) 147 | 148 | ''' 149 | test the network 150 | ''' 151 | def mc_test(file_name, number): 152 | data = np.load('/home/ws/文档/wrj/data_all/'+file_name+'.npz') 153 | train_matching_y = data['arr_1'][:,np.newaxis] 154 | numbers_train = train_matching_y.shape[0] #训练集总数 155 | 156 | graph = tf.Graph() 157 | with graph.as_default(): 158 | inputs_p1 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width, 1], name='inputs_p1') 159 | inputs_p2 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width, 1], name='inputs_p2') 160 | label_m = tf.placeholder(tf.float32, [BATCH_SIZE_matching, 1], name='label_m') 161 | 162 | match_loss, match_output, all_features = model.match_network(inputs_p1, inputs_p2, label_m) 163 | match_out = tf.round(match_output) 164 | m_correct,m_numbers = model.evaluation(match_out, label_m) 165 | 166 | filename = '/home/ws/文档/wrj/data_all/country/'+file_name+'.tfrecord' 167 | filename_queue = tf.train.string_input_producer([filename],num_epochs=1,shuffle=False) 168 | img_batch, label_batch = read_data.batch_inputs(filename_queue, train = False, batch_size = BATCH_SIZE_matching) 169 | 170 | saver = tf.train.Saver() 171 | 172 | gpu_options = tf.GPUOptions(allow_growth=True) 173 | sess_config = tf.ConfigProto(gpu_options=gpu_options) 174 | 175 | with tf.Session(config=sess_config) as sess: 176 | sess.run(tf.local_variables_initializer()) 177 | # saver.restore(sess, tf.train.latest_checkpoint('ckpt_map32_no')) 178 | saver.restore(sess, 'pse_sia_map/model.ckpt-27') 179 | m_count = 0 # Counts the number of correct predictions. 180 | num = numbers_train 181 | matching_out = np.array([]) 182 | matching_label = np.array([]) 183 | 184 | try: 185 | coord = tf.train.Coordinator() 186 | threads = tf.train.start_queue_runners(sess=sess, coord=coord) 187 | step_test = 0 188 | try: 189 | while not coord.should_stop(): 190 | step_test = step_test + 1 191 | batch, l_batch = sess.run([img_batch, label_batch]) 192 | l_batch = l_batch 193 | x_batch = batch[:,:,:64,np.newaxis] 194 | y_batch = batch[:,:,64:,np.newaxis] 195 | feed_dict = {inputs_p1:x_batch, inputs_p2:y_batch, label_m:l_batch} 196 | m_correct_, m_output_ = sess.run([m_correct,match_output], feed_dict = feed_dict) 197 | 198 | if step_test == 1: 199 | matching_out = m_output_ 200 | matching_label = l_batch 201 | elif(l_batch.size == BATCH_SIZE_matching): 202 | matching_out = np.concatenate((matching_out, m_output_)) 203 | matching_label = np.concatenate((matching_label, l_batch)) 204 | 205 | m_count = m_count + m_correct_.astype(int) 206 | if step_test % 100 == 0: 207 | print('Step %d/%d run_test: batch_precision = %.2f '% (step_test, num/100, m_correct_/BATCH_SIZE_matching)) 208 | except Exception as e: 209 | coord.request_stop(e) 210 | m_precision = float(m_count) / num 211 | 212 | print('Num examples: %d Num correct: %d match Precision : %0.04f ' %(num, m_count, m_precision)) 213 | save_file = open('test_map32_no.txt','a') 214 | save_file.write(file_name + ' epoch: ' + str(number) +'\n'+'num correct: '+str(m_count)+'/'+str(num)+' match precision : '+str(m_precision)) 215 | save_file.write(' ') 216 | 217 | # 绘制ROC曲线 218 | fpr,tpr,threshold = roc_curve(matching_label, matching_out) ###计算真正率和假正率 219 | 220 | q = np.where(0.95<=tpr ) 221 | q_value = q[0][0] 222 | fpr95 = fpr[q_value] 223 | 224 | save_file.write('match fpr95 : '+str(fpr95*100)) 225 | save_file.write('\n') 226 | save_file.write('\n') 227 | save_file.close() 228 | roc_dir = 'plot_curve/nature_map32_no/epoch_'+str(number) 229 | try: 230 | os.makedirs(roc_dir) 231 | except os.error: 232 | pass 233 | np.savez(roc_dir+'/match_map32_no_'+file_name, fpr,tpr) 234 | except KeyboardInterrupt: 235 | print('INTERRUPTED') 236 | coord.request_stop() 237 | finally: 238 | # When done, ask the threads to stop. 239 | coord.request_stop() 240 | coord.join(threads) 241 | 242 | ''' 243 | test all datasets 244 | ''' 245 | def mc_test_all(number): 246 | test_data = ['field', 'forest','indoor','mountain','oldbuilding','street','urban','water'] 247 | for i in range(8): 248 | mc_test(test_data[i],number) 249 | 250 | if __name__ == '__main__': 251 | 252 | # mc_train() 253 | # mc_test_all(36) 254 | # mc_test('field',1) 255 | mc_test_all(1) 256 | 257 | 258 | 259 | -------------------------------------------------------------------------------- /part_1_feature_difference/train_nature_tensor.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Created on 2018.11.9 5 | 6 | @author: wrj 7 | """ 8 | ''' 9 | 双分支 + 特征向量 + 交叉熵损失函数 + 参数共享 10 | ''' 11 | 12 | import numpy as np 13 | import math 14 | import time 15 | import model_d_tensor as model 16 | import read_data 17 | import tensorflow as tf 18 | import os 19 | from datetime import datetime 20 | import logging 21 | from sklearn.metrics import roc_curve, auc 22 | 23 | os.environ['CUDA_VISIBLE_DEVICES']='0' 24 | 25 | BATCH_SIZE_matching=100 26 | epoch = 41 27 | learning_rate = 2e-4 28 | image_width = 64 29 | image_height = 64 30 | 31 | checkpoint_dir = 'ckpt_tensor' 32 | checkpoint_file = os.path.join(checkpoint_dir, 'model.ckpt') 33 | train_dir='summary' 34 | 35 | def initLogging(logFilename='record_tensor.log'): 36 | """Init for logging 37 | """ 38 | logging.basicConfig( 39 | level = logging.DEBUG, 40 | format='%(asctime)s-%(levelname)s-%(message)s', 41 | datefmt = '%y-%m-%d %H:%M', 42 | filename = logFilename, 43 | filemode = 'w'); 44 | console = logging.StreamHandler() 45 | console.setLevel(logging.INFO) 46 | formatter = logging.Formatter('%(asctime)s-%(levelname)s-%(message)s') 47 | console.setFormatter(formatter) 48 | logging.getLogger('').addHandler(console) 49 | 50 | 51 | def mc_train(): 52 | initLogging() 53 | current_time = datetime.now().strftime('%Y-%m-%d') 54 | try: 55 | os.makedirs(checkpoint_dir) 56 | except os.error: 57 | pass 58 | 59 | data1 = np.load('/home/ws/文档/wrj/data_all/country.npz') 60 | train_matching_y = data1['arr_1'][:,np.newaxis] 61 | numbers_train = train_matching_y.shape[0] #训练集总数 62 | epoch_steps = np.int(numbers_train / BATCH_SIZE_matching) +1 # 一个epoch有多少个steps 63 | 64 | all_loss = np.array([]) 65 | graph = tf.Graph() 66 | with graph.as_default(): 67 | inputs_p1 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width,1], name='inputs_p1') 68 | inputs_p2 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width,1], name='inputs_p2') 69 | label_m = tf.placeholder(tf.float32, [BATCH_SIZE_matching, 1], name='label_m') 70 | 71 | # 训练 M 72 | match_loss, match_output, all_features = model.match_network(inputs_p1, inputs_p2, label_m) 73 | match_out = tf.round(match_output) 74 | m_correct,m_numbers = model.evaluation(match_out, label_m) 75 | 76 | m_train_opt = tf.train.AdamOptimizer(learning_rate).minimize(match_loss) 77 | 78 | filename = '/home/ws/文档/wrj/data_all/country/country.tfrecord' 79 | filename_queue = tf.train.string_input_producer([filename],num_epochs=epoch,shuffle=True) 80 | img_batch, label_batch = read_data.batch_inputs(filename_queue, train = True, batch_size = BATCH_SIZE_matching) 81 | tf.summary.scalar('mathing_loss', match_loss) 82 | 83 | summary = tf.summary.merge_all() 84 | saver = tf.train.Saver(max_to_keep=20) 85 | init = tf.global_variables_initializer() 86 | 87 | gpu_options = tf.GPUOptions(allow_growth=True) 88 | sess_config = tf.ConfigProto(gpu_options=gpu_options) 89 | 90 | with tf.Session(config=sess_config) as sess: 91 | summary_writer = tf.summary.FileWriter(train_dir, sess.graph) 92 | sess.run(tf.local_variables_initializer()) 93 | sess.run(init) 94 | 95 | try: 96 | coord = tf.train.Coordinator() 97 | threads = tf.train.start_queue_runners(sess=sess, coord=coord) 98 | step = 0 99 | 100 | while not coord.should_stop(): 101 | start_time = time.time() 102 | step = step + 1 103 | batch, l_batch = sess.run([img_batch, label_batch]) 104 | l_batch = l_batch 105 | x_batch = batch[:,:,:64,np.newaxis] 106 | y_batch = batch[:,:,64:,np.newaxis] 107 | feed_dict = {inputs_p1:x_batch, inputs_p2:y_batch, label_m:l_batch} 108 | _, m_loss, m_output_= sess.run([m_train_opt,match_loss,match_output], feed_dict = feed_dict) 109 | 110 | if step % 10 ==0: 111 | loss_write = np.array([[step, m_loss]]) 112 | if step ==10: 113 | all_loss = loss_write 114 | else: 115 | all_loss = np.concatenate((all_loss, loss_write)) 116 | 117 | if step % 100 == 0: 118 | duration = time.time() - start_time 119 | summary_str = sess.run(summary, feed_dict=feed_dict) 120 | summary_writer.add_summary(summary_str, step) 121 | summary_writer.flush() 122 | 123 | logging.info('>> Step %d run_train: matching_loss = %.3f (%.3f sec)' 124 | % (step, m_loss, duration)) 125 | 126 | if (step % epoch_steps == 0) and ((step / epoch_steps)%3 == 0): 127 | current_epoch = int(step / epoch_steps) 128 | 129 | if current_epoch > 20: 130 | logging.info('>> %s Saving in %s' % (datetime.now(), checkpoint_dir)) 131 | saver.save(sess, checkpoint_file, global_step=current_epoch) 132 | mc_test_all(current_epoch) 133 | 134 | except KeyboardInterrupt: 135 | print('INTERRUPTED') 136 | coord.request_stop() 137 | except Exception as e: 138 | coord.request_stop(e) 139 | 140 | finally: 141 | saver.save(sess, checkpoint_file, global_step=step) 142 | np.save(os.path.join(checkpoint_dir,'ckpt_tensor'), all_loss) 143 | print('Model saved in file :%s'%checkpoint_dir) 144 | # When done, ask the threads to stop. 145 | coord.request_stop() 146 | coord.join(threads) 147 | 148 | ''' 149 | test the network 150 | ''' 151 | def mc_test(file_name, number): 152 | data = np.load('/home/ws/文档/wrj/data_all/'+file_name+'.npz') 153 | train_matching_y = data['arr_1'][:,np.newaxis] 154 | numbers_train = train_matching_y.shape[0] #训练集总数 155 | 156 | graph = tf.Graph() 157 | with graph.as_default(): 158 | inputs_p1 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width, 1], name='inputs_p1') 159 | inputs_p2 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width, 1], name='inputs_p2') 160 | label_m = tf.placeholder(tf.float32, [BATCH_SIZE_matching, 1], name='label_m') 161 | 162 | match_loss, match_output, all_features = model.match_network(inputs_p1, inputs_p2, label_m) 163 | match_out = tf.round(match_output) 164 | m_correct,m_numbers = model.evaluation(match_out, label_m) 165 | 166 | filename = '/home/ws/文档/wrj/data_all/country/'+file_name+'.tfrecord' 167 | filename_queue = tf.train.string_input_producer([filename],num_epochs=1,shuffle=False) 168 | img_batch, label_batch = read_data.batch_inputs(filename_queue, train = False, batch_size = BATCH_SIZE_matching) 169 | 170 | saver = tf.train.Saver() 171 | 172 | gpu_options = tf.GPUOptions(allow_growth=True) 173 | sess_config = tf.ConfigProto(gpu_options=gpu_options) 174 | 175 | with tf.Session(config=sess_config) as sess: 176 | sess.run(tf.local_variables_initializer()) 177 | # saver.restore(sess, tf.train.latest_checkpoint('ckpt_tensor')) 178 | saver.restore(sess, 'sia_tensor/model.ckpt-35') 179 | m_count = 0 # Counts the number of correct predictions. 180 | num = numbers_train 181 | matching_out = np.array([]) 182 | matching_label = np.array([]) 183 | 184 | try: 185 | coord = tf.train.Coordinator() 186 | threads = tf.train.start_queue_runners(sess=sess, coord=coord) 187 | step_test = 0 188 | try: 189 | while not coord.should_stop(): 190 | step_test = step_test + 1 191 | batch, l_batch = sess.run([img_batch, label_batch]) 192 | l_batch = l_batch 193 | x_batch = batch[:,:,:64,np.newaxis] 194 | y_batch = batch[:,:,64:,np.newaxis] 195 | feed_dict = {inputs_p1:x_batch, inputs_p2:y_batch, label_m:l_batch} 196 | m_correct_, m_output_ = sess.run([m_correct,match_output], feed_dict = feed_dict) 197 | 198 | if step_test == 1: 199 | matching_out = m_output_ 200 | matching_label = l_batch 201 | elif(l_batch.size == BATCH_SIZE_matching): 202 | matching_out = np.concatenate((matching_out, m_output_)) 203 | matching_label = np.concatenate((matching_label, l_batch)) 204 | 205 | m_count = m_count + m_correct_.astype(int) 206 | if step_test % 100 == 0: 207 | print('Step %d/%d run_test: batch_precision = %.2f '% (step_test, num/100, m_correct_/BATCH_SIZE_matching)) 208 | except Exception as e: 209 | coord.request_stop(e) 210 | m_precision = float(m_count) / num 211 | 212 | print('Num examples: %d Num correct: %d match Precision : %0.04f ' %(num, m_count, m_precision)) 213 | save_file = open('test_tensor.txt','a') 214 | save_file.write(file_name + ' epoch: ' + str(number) +'\n'+'num correct: '+str(m_count)+'/'+str(num)+' match precision : '+str(m_precision)) 215 | save_file.write(' ') 216 | 217 | # 绘制ROC曲线 218 | fpr,tpr,threshold = roc_curve(matching_label, matching_out) ###计算真正率和假正率 219 | 220 | q = np.where(0.95<=tpr ) 221 | q_value = q[0][0] 222 | fpr95 = fpr[q_value] 223 | 224 | save_file.write('match fpr95 : '+str(fpr95*100)) 225 | save_file.write('\n') 226 | save_file.write('\n') 227 | save_file.close() 228 | roc_dir = 'plot_curve/nature_tensor/epoch_'+str(number) 229 | try: 230 | os.makedirs(roc_dir) 231 | except os.error: 232 | pass 233 | np.savez(roc_dir+'/match_tensor_'+file_name, fpr,tpr) 234 | except KeyboardInterrupt: 235 | print('INTERRUPTED') 236 | coord.request_stop() 237 | finally: 238 | # When done, ask the threads to stop. 239 | coord.request_stop() 240 | coord.join(threads) 241 | 242 | ''' 243 | test all datasets 244 | ''' 245 | def mc_test_all(number): 246 | test_data = ['field', 'forest','indoor','mountain','oldbuilding','street','urban','water'] 247 | for i in range(8): 248 | mc_test(test_data[i],number) 249 | 250 | if __name__ == '__main__': 251 | 252 | # mc_train() 253 | # mc_test('field',1) 254 | mc_test_all(1) 255 | 256 | 257 | 258 | -------------------------------------------------------------------------------- /part_1_feature_difference/train_nature_tensor_no.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Created on 2018.11.9 5 | 6 | @author: wrj 7 | """ 8 | ''' 9 | 双分支 + 特征向量 + 交叉熵损失函数 + 参数fei共享 10 | ''' 11 | 12 | import numpy as np 13 | import math 14 | import time 15 | import model_d_tensor_no as model 16 | import read_data 17 | import tensorflow as tf 18 | import os 19 | from datetime import datetime 20 | import logging 21 | from sklearn.metrics import roc_curve, auc 22 | 23 | os.environ['CUDA_VISIBLE_DEVICES']='0' 24 | 25 | BATCH_SIZE_matching=100 26 | epoch = 41 27 | learning_rate = 2e-4 28 | image_width = 64 29 | image_height = 64 30 | 31 | checkpoint_dir = 'ckpt_tensor_no' 32 | checkpoint_file = os.path.join(checkpoint_dir, 'model.ckpt') 33 | train_dir='summary' 34 | 35 | def initLogging(logFilename='record_tensor_no.log'): 36 | """Init for logging 37 | """ 38 | logging.basicConfig( 39 | level = logging.DEBUG, 40 | format='%(asctime)s-%(levelname)s-%(message)s', 41 | datefmt = '%y-%m-%d %H:%M', 42 | filename = logFilename, 43 | filemode = 'w'); 44 | console = logging.StreamHandler() 45 | console.setLevel(logging.INFO) 46 | formatter = logging.Formatter('%(asctime)s-%(levelname)s-%(message)s') 47 | console.setFormatter(formatter) 48 | logging.getLogger('').addHandler(console) 49 | 50 | 51 | def mc_train(): 52 | initLogging() 53 | current_time = datetime.now().strftime('%Y-%m-%d') 54 | try: 55 | os.makedirs(checkpoint_dir) 56 | except os.error: 57 | pass 58 | 59 | data1 = np.load('/home/ws/文档/wrj/data_all/country.npz') 60 | train_matching_y = data1['arr_1'][:,np.newaxis] 61 | numbers_train = train_matching_y.shape[0] #训练集总数 62 | epoch_steps = np.int(numbers_train / BATCH_SIZE_matching) +1 # 一个epoch有多少个steps 63 | 64 | all_loss = np.array([]) 65 | graph = tf.Graph() 66 | with graph.as_default(): 67 | inputs_p1 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width,1], name='inputs_p1') 68 | inputs_p2 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width,1], name='inputs_p2') 69 | label_m = tf.placeholder(tf.float32, [BATCH_SIZE_matching, 1], name='label_m') 70 | 71 | # 训练 M 72 | match_loss, match_output, all_features = model.match_network(inputs_p1, inputs_p2, label_m) 73 | match_out = tf.round(match_output) 74 | m_correct,m_numbers = model.evaluation(match_out, label_m) 75 | 76 | m_train_opt = tf.train.AdamOptimizer(learning_rate).minimize(match_loss) 77 | 78 | filename = '/home/ws/文档/wrj/data_all/country/country.tfrecord' 79 | filename_queue = tf.train.string_input_producer([filename],num_epochs=epoch,shuffle=True) 80 | img_batch, label_batch = read_data.batch_inputs(filename_queue, train = True, batch_size = BATCH_SIZE_matching) 81 | tf.summary.scalar('mathing_loss', match_loss) 82 | 83 | summary = tf.summary.merge_all() 84 | saver = tf.train.Saver(max_to_keep=20) 85 | init = tf.global_variables_initializer() 86 | 87 | gpu_options = tf.GPUOptions(allow_growth=True) 88 | sess_config = tf.ConfigProto(gpu_options=gpu_options) 89 | 90 | with tf.Session(config=sess_config) as sess: 91 | summary_writer = tf.summary.FileWriter(train_dir, sess.graph) 92 | sess.run(tf.local_variables_initializer()) 93 | sess.run(init) 94 | 95 | try: 96 | coord = tf.train.Coordinator() 97 | threads = tf.train.start_queue_runners(sess=sess, coord=coord) 98 | step = 0 99 | 100 | while not coord.should_stop(): 101 | start_time = time.time() 102 | step = step + 1 103 | batch, l_batch = sess.run([img_batch, label_batch]) 104 | l_batch = l_batch 105 | x_batch = batch[:,:,:64,np.newaxis] 106 | y_batch = batch[:,:,64:,np.newaxis] 107 | feed_dict = {inputs_p1:x_batch, inputs_p2:y_batch, label_m:l_batch} 108 | _, m_loss, m_output_= sess.run([m_train_opt,match_loss,match_output], feed_dict = feed_dict) 109 | 110 | if step % 10 ==0: 111 | loss_write = np.array([[step, m_loss]]) 112 | if step ==10: 113 | all_loss = loss_write 114 | else: 115 | all_loss = np.concatenate((all_loss, loss_write)) 116 | 117 | if step % 100 == 0: 118 | duration = time.time() - start_time 119 | summary_str = sess.run(summary, feed_dict=feed_dict) 120 | summary_writer.add_summary(summary_str, step) 121 | summary_writer.flush() 122 | 123 | logging.info('>> Step %d run_train: matching_loss = %.3f (%.3f sec)' 124 | % (step, m_loss, duration)) 125 | 126 | if (step % epoch_steps == 0) and ((step / epoch_steps)%3 == 0): 127 | current_epoch = int(step / epoch_steps) 128 | 129 | if current_epoch > 20: 130 | logging.info('>> %s Saving in %s' % (datetime.now(), checkpoint_dir)) 131 | saver.save(sess, checkpoint_file, global_step=current_epoch) 132 | mc_test_all(current_epoch) 133 | 134 | except KeyboardInterrupt: 135 | print('INTERRUPTED') 136 | coord.request_stop() 137 | except Exception as e: 138 | coord.request_stop(e) 139 | 140 | finally: 141 | saver.save(sess, checkpoint_file, global_step=step) 142 | np.save(os.path.join(checkpoint_dir,'ckpt_tensor_no'), all_loss) 143 | print('Model saved in file :%s'%checkpoint_dir) 144 | # When done, ask the threads to stop. 145 | coord.request_stop() 146 | coord.join(threads) 147 | 148 | ''' 149 | test the network 150 | ''' 151 | def mc_test(file_name, number): 152 | data = np.load('/home/ws/文档/wrj/data_all/'+file_name+'.npz') 153 | train_matching_y = data['arr_1'][:,np.newaxis] 154 | numbers_train = train_matching_y.shape[0] #训练集总数 155 | 156 | graph = tf.Graph() 157 | with graph.as_default(): 158 | inputs_p1 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width, 1], name='inputs_p1') 159 | inputs_p2 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width, 1], name='inputs_p2') 160 | label_m = tf.placeholder(tf.float32, [BATCH_SIZE_matching, 1], name='label_m') 161 | 162 | match_loss, match_output, all_features = model.match_network(inputs_p1, inputs_p2, label_m) 163 | match_out = tf.round(match_output) 164 | m_correct,m_numbers = model.evaluation(match_out, label_m) 165 | 166 | filename = '/home/ws/文档/wrj/data_all/country/'+file_name+'.tfrecord' 167 | filename_queue = tf.train.string_input_producer([filename],num_epochs=1,shuffle=False) 168 | img_batch, label_batch = read_data.batch_inputs(filename_queue, train = False, batch_size = BATCH_SIZE_matching) 169 | 170 | saver = tf.train.Saver() 171 | 172 | gpu_options = tf.GPUOptions(allow_growth=True) 173 | sess_config = tf.ConfigProto(gpu_options=gpu_options) 174 | 175 | with tf.Session(config=sess_config) as sess: 176 | sess.run(tf.local_variables_initializer()) 177 | # saver.restore(sess, tf.train.latest_checkpoint('ckpt_tensor_no')) 178 | saver.restore(sess, 'pse_sia_tensor/model.ckpt-36') 179 | m_count = 0 # Counts the number of correct predictions. 180 | num = numbers_train 181 | matching_out = np.array([]) 182 | matching_label = np.array([]) 183 | 184 | try: 185 | coord = tf.train.Coordinator() 186 | threads = tf.train.start_queue_runners(sess=sess, coord=coord) 187 | step_test = 0 188 | try: 189 | while not coord.should_stop(): 190 | step_test = step_test + 1 191 | batch, l_batch = sess.run([img_batch, label_batch]) 192 | l_batch = l_batch 193 | x_batch = batch[:,:,:64,np.newaxis] 194 | y_batch = batch[:,:,64:,np.newaxis] 195 | feed_dict = {inputs_p1:x_batch, inputs_p2:y_batch, label_m:l_batch} 196 | m_correct_, m_output_ = sess.run([m_correct,match_output], feed_dict = feed_dict) 197 | 198 | if step_test == 1: 199 | matching_out = m_output_ 200 | matching_label = l_batch 201 | elif(l_batch.size == BATCH_SIZE_matching): 202 | matching_out = np.concatenate((matching_out, m_output_)) 203 | matching_label = np.concatenate((matching_label, l_batch)) 204 | 205 | m_count = m_count + m_correct_.astype(int) 206 | if step_test % 100 == 0: 207 | print('Step %d/%d run_test: batch_precision = %.2f '% (step_test, num/100, m_correct_/BATCH_SIZE_matching)) 208 | except Exception as e: 209 | coord.request_stop(e) 210 | m_precision = float(m_count) / num 211 | 212 | print('Num examples: %d Num correct: %d match Precision : %0.04f ' %(num, m_count, m_precision)) 213 | save_file = open('test_tensor_no.txt','a') 214 | save_file.write(file_name + ' epoch: ' + str(number) +'\n'+'num correct: '+str(m_count)+'/'+str(num)+' match precision : '+str(m_precision)) 215 | save_file.write(' ') 216 | 217 | # 绘制ROC曲线 218 | fpr,tpr,threshold = roc_curve(matching_label, matching_out) ###计算真正率和假正率 219 | 220 | q = np.where(0.95<=tpr ) 221 | q_value = q[0][0] 222 | fpr95 = fpr[q_value] 223 | 224 | save_file.write('match fpr95 : '+str(fpr95*100)) 225 | save_file.write('\n') 226 | save_file.write('\n') 227 | save_file.close() 228 | roc_dir = 'plot_curve/nature_tensor_no/epoch_'+str(number) 229 | try: 230 | os.makedirs(roc_dir) 231 | except os.error: 232 | pass 233 | np.savez(roc_dir+'/match_tensor_no_'+file_name, fpr,tpr) 234 | except KeyboardInterrupt: 235 | print('INTERRUPTED') 236 | coord.request_stop() 237 | finally: 238 | # When done, ask the threads to stop. 239 | coord.request_stop() 240 | coord.join(threads) 241 | 242 | ''' 243 | test all datasets 244 | ''' 245 | def mc_test_all(number): 246 | test_data = ['field', 'forest','indoor','mountain','oldbuilding','street','urban','water'] 247 | for i in range(8): 248 | mc_test(test_data[i],number) 249 | 250 | if __name__ == '__main__': 251 | 252 | # mc_train() 253 | # mc_test('field',1) 254 | mc_test_all(1) 255 | 256 | 257 | 258 | -------------------------------------------------------------------------------- /part_2_element-wise_feature_relation/README: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /part_2_element-wise_feature_relation/model_d_map32_diff_res.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on 2018.11.9 11:03:33 4 | 5 | @author: Administrator 6 | 7 | 双分支 + 特征map + 交叉熵损失函数 + 参数共享 + 差值求绝对值 8 | 9 | """ 10 | 11 | import tensorflow as tf 12 | import numpy as np 13 | 14 | EPS = 1e-12 15 | 16 | def conv(batch_input, out_channels, filter_size, pad, stride=1): 17 | conv = tf.layers.conv2d(batch_input, out_channels,filter_size,strides=(stride,stride), padding=pad) 18 | return conv 19 | 20 | def batchnorm(inputs): 21 | normalized = tf.layers.batch_normalization(inputs, axis=-1) 22 | return normalized 23 | 24 | def evaluation(logits, labels): 25 | correct_prediction = tf.equal(logits, labels) 26 | correct_batch = tf.reduce_mean(tf.cast(correct_prediction, tf.int32), 1) 27 | return tf.reduce_sum(tf.cast(correct_batch, tf.float32)), correct_batch 28 | 29 | ''' 30 | # image patch features abstract layers 31 | update: 2018.11.9 32 | 5*conv + fc 33 | 34 | ''' 35 | def features(img, name, reuse=False): 36 | with tf.variable_scope(name,reuse=reuse): 37 | layer1 = conv(img, 32, 3, pad='SAME') # 64*64*32 38 | layer1 = tf.nn.relu(layer1) 39 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 40 | 41 | layer2 = conv(pool1, 64, 3, pad='SAME') # 32*32*64 42 | layer2 = tf.nn.relu(layer2) 43 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 44 | 45 | layer3 = conv(pool2, 128, 3, pad='SAME') # 16*16*128 46 | layer3 = tf.nn.relu(layer3) 47 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 48 | 49 | layer4 = conv(pool3, 256, 3, pad='SAME') # 8*8*256 50 | layer4 = tf.nn.relu(layer4) 51 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 52 | # 4*4*256 53 | return [layer1, pool1, pool2, pool3, pool4], pool4 54 | 55 | ''' 56 | matching layers 57 | update: 2018.11.9 58 | 59 | ''' 60 | 61 | def matching(feature_map_1,feature_map_2): 62 | with tf.variable_scope('match_layers'): 63 | # all_feature = tf.concat([feature_map_1,feature_map_2], axis=3) # 两个特征连接 4*4*512 64 | diff_feature = tf.abs(feature_map_1 - feature_map_2) 65 | layer5 = conv(diff_feature, 512, 3, pad='SAME') 66 | layer5 = tf.nn.relu(layer5) 67 | pool5 = tf.nn.max_pool(layer5,[1,2,2,1],[1,2,2,1],padding='SAME') # 2*2*512 68 | 69 | batch_size = int(pool5.get_shape()[0]) 70 | dense = tf.reshape(pool5, [batch_size,-1]) # reshape as a tensor 2048 71 | dense1 = tf.layers.dense(dense, 1024) 72 | dense1 = tf.nn.relu(dense1) 73 | dense2 = tf.layers.dense(dense1, 256) 74 | dense2 = tf.nn.relu(dense2) 75 | dense3 = tf.layers.dense(dense2, 1) 76 | output = tf.sigmoid(dense3) 77 | return output, [pool5, dense1, dense2] 78 | 79 | ''' 80 | add the resnet 81 | update: 2018.11.10 82 | 83 | ''' 84 | def matching_res(layer1_res, pool1_res, pool2_res, pool3_res, pool4_res): 85 | 86 | layer1 = conv(layer1_res, 32, 1, pad='SAME') 87 | layer1 = tf.nn.relu(layer1) 88 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 89 | 90 | pool1_con = tf.concat([pool1, pool1_res], axis=3) 91 | layer2 = conv(pool1_con, 64, 1, pad='SAME') 92 | layer2 = tf.nn.relu(layer2) 93 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 94 | 95 | pool2_con = tf.concat([pool2, pool2_res], axis=3) 96 | layer3 = conv(pool2_con, 128, 1, pad='SAME') 97 | layer3 = tf.nn.relu(layer3) 98 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 99 | 100 | pool3_con = tf.concat([pool3, pool3_res], axis=3) 101 | layer4 = conv(pool3_con, 256, 1, pad='SAME') 102 | layer4 = tf.nn.relu(layer4) 103 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 104 | 105 | pool4_con = tf.concat([pool4, pool4_res], axis=3) 106 | layer5 = conv(pool4_con, 512, 1, pad='SAME') 107 | layer5 = tf.nn.relu(layer5) 108 | pool5 = tf.nn.max_pool(layer5,[1,2,2,1],[1,2,2,1],padding='SAME') # 2*2*512 109 | 110 | # 2*2*512=2048 111 | batch_size1 = int(pool5.get_shape()[0]) 112 | dense = tf.reshape(pool5, [batch_size1,-1]) 113 | 114 | dense1 = tf.layers.dense(dense, 1024) 115 | dense1 = tf.nn.relu(dense1) 116 | 117 | dense2 = tf.layers.dense(dense1, 256) 118 | dense2 = tf.nn.relu(dense2) 119 | 120 | dense3 = tf.layers.dense(dense2, 1) 121 | output = tf.sigmoid(dense3) 122 | return output,[pool1, pool2, pool3, pool4, pool5, dense1, dense2] 123 | 124 | 125 | ''' 126 | network model 127 | ''' 128 | def features_matching(inputs_p1, inputs_p2): 129 | with tf.variable_scope("matching"): 130 | layers_1,feature_map_1 = features(inputs_p1, 'feature_layers') # 4*4*256 131 | layers_2,feature_map_2 = features(inputs_p2, 'feature_layers',reuse=True) # 4*4*256 132 | output,feature_maps = matching(feature_map_1,feature_map_2) 133 | return output ,[layers_1,layers_2,feature_maps] 134 | 135 | ''' 136 | features_matching_res network 137 | update: 2018.11.10 138 | 139 | ''' 140 | def features_matching_res(inputs_p1, inputs_p2): 141 | with tf.variable_scope("matching"): 142 | layers_1,feature_map_1 = features(inputs_p1, 'feature_layers') # 4*4*256 143 | layers_2,feature_map_2 = features(inputs_p2, 'feature_layers',reuse=True) # 4*4*256 144 | output,feature_maps = matching(feature_map_1,feature_map_2) 145 | with tf.variable_scope('match_res'): 146 | output_res, res_feature_maps = matching_res(abs(layers_1[0]-layers_2[0]), abs(layers_1[1]-layers_2[1]), abs(layers_1[2]-layers_2[2]), 147 | abs(layers_1[3]-layers_2[3]), abs(layers_1[4]-layers_2[4])) 148 | return output, output_res, [layers_1, layers_2, feature_maps, res_feature_maps] 149 | 150 | 151 | ''' 152 | cross loss function 153 | ''' 154 | 155 | def match_network(inputs_p1, inputs_p2, label_m): 156 | 157 | match_output, all_features = features_matching(inputs_p1, inputs_p2) 158 | match_loss = tf.reduce_mean(-(label_m * tf.log(match_output + EPS) + (1 - label_m) * tf.log(1 - match_output + EPS))) 159 | return match_loss, match_output, all_features 160 | 161 | def match_network_res(inputs_p1, inputs_p2, label_m): 162 | 163 | match_output, match_output_res, all_features = features_matching_res(inputs_p1, inputs_p2) 164 | match_loss = tf.reduce_mean(-(label_m * tf.log(match_output + EPS) + (1 - label_m) * tf.log(1 - match_output + EPS))) 165 | match_loss_res = tf.reduce_mean(-(label_m * tf.log(match_output_res + EPS) + (1 - label_m) * tf.log(1 - match_output_res + EPS))) 166 | return match_loss, match_loss_res, match_output, match_output_res, all_features 167 | 168 | 169 | 170 | 171 | -------------------------------------------------------------------------------- /part_2_element-wise_feature_relation/model_d_map32_diff_res_pro.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on 2018.11.9 11:03:33 4 | 5 | @author: Administrator 6 | 7 | 双分支 + 特征map + 交叉熵损失函数 + 参数共享 + 差值求绝对值 8 | 改进：将上下两个支路的特征向量结合起来，得到最终判别结果 9 | 10 | """ 11 | 12 | import tensorflow as tf 13 | import numpy as np 14 | 15 | EPS = 1e-12 16 | 17 | def conv(batch_input, out_channels, filter_size, pad, stride=1): 18 | conv = tf.layers.conv2d(batch_input, out_channels,filter_size,strides=(stride,stride), padding=pad) 19 | return conv 20 | 21 | def batchnorm(inputs): 22 | normalized = tf.layers.batch_normalization(inputs, axis=-1) 23 | return normalized 24 | 25 | def evaluation(logits, labels): 26 | correct_prediction = tf.equal(logits, labels) 27 | correct_batch = tf.reduce_mean(tf.cast(correct_prediction, tf.int32), 1) 28 | return tf.reduce_sum(tf.cast(correct_batch, tf.float32)), correct_batch 29 | 30 | ''' 31 | # image patch features abstract layers 32 | update: 2018.11.9 33 | 5*conv + fc 34 | 35 | ''' 36 | def features(img, name, reuse=False): 37 | with tf.variable_scope(name,reuse=reuse): 38 | layer1 = conv(img, 32, 3, pad='SAME') # 64*64*32 39 | layer1 = tf.nn.relu(layer1) 40 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 41 | 42 | layer2 = conv(pool1, 64, 3, pad='SAME') # 32*32*64 43 | layer2 = tf.nn.relu(layer2) 44 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 45 | 46 | layer3 = conv(pool2, 128, 3, pad='SAME') # 16*16*128 47 | layer3 = tf.nn.relu(layer3) 48 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 49 | 50 | layer4 = conv(pool3, 256, 3, pad='SAME') # 8*8*256 51 | layer4 = tf.nn.relu(layer4) 52 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 53 | # 4*4*256 54 | return [layer1, pool1, pool2, pool3, pool4], pool4 55 | 56 | ''' 57 | matching layers 58 | update: 2018.11.9 59 | 60 | ''' 61 | 62 | def matching(feature_map_1,feature_map_2): 63 | with tf.variable_scope('match_layers_1'): 64 | # all_feature = tf.concat([feature_map_1,feature_map_2], axis=3) # 两个特征连接 4*4*512 65 | diff_feature = tf.abs(feature_map_1 - feature_map_2) 66 | layer5 = conv(diff_feature, 512, 3, pad='SAME') 67 | layer5 = tf.nn.relu(layer5) 68 | pool5 = tf.nn.max_pool(layer5,[1,2,2,1],[1,2,2,1],padding='SAME') # 2*2*512 69 | 70 | batch_size = int(pool5.get_shape()[0]) 71 | dense = tf.reshape(pool5, [batch_size,-1]) # reshape as a tensor 2048 72 | dense1 = tf.layers.dense(dense, 1024) 73 | dense1 = tf.nn.relu(dense1) 74 | 75 | with tf.variable_scope('match_layers_2'): 76 | dense2 = tf.layers.dense(dense1, 256) 77 | dense2 = tf.nn.relu(dense2) 78 | dense3 = tf.layers.dense(dense2, 1) 79 | output = tf.sigmoid(dense3) 80 | return output, dense1, [pool5, dense1, dense2] 81 | 82 | ''' 83 | add the resnet 84 | update: 2018.11.10 85 | 86 | ''' 87 | def matching_res(layer1_res, pool1_res, pool2_res, pool3_res, pool4_res): 88 | with tf.variable_scope('match_res_1'): 89 | layer1 = conv(layer1_res, 32, 1, pad='SAME') 90 | layer1 = tf.nn.relu(layer1) 91 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 92 | 93 | pool1_con = tf.concat([pool1, pool1_res], axis=3) 94 | layer2 = conv(pool1_con, 64, 1, pad='SAME') 95 | layer2 = tf.nn.relu(layer2) 96 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 97 | 98 | pool2_con = tf.concat([pool2, pool2_res], axis=3) 99 | layer3 = conv(pool2_con, 128, 1, pad='SAME') 100 | layer3 = tf.nn.relu(layer3) 101 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 102 | 103 | pool3_con = tf.concat([pool3, pool3_res], axis=3) 104 | layer4 = conv(pool3_con, 256, 1, pad='SAME') 105 | layer4 = tf.nn.relu(layer4) 106 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 107 | 108 | pool4_con = tf.concat([pool4, pool4_res], axis=3) 109 | layer5 = conv(pool4_con, 512, 1, pad='SAME') 110 | layer5 = tf.nn.relu(layer5) 111 | pool5 = tf.nn.max_pool(layer5,[1,2,2,1],[1,2,2,1],padding='SAME') # 2*2*512 112 | 113 | # 2*2*512=2048 114 | batch_size1 = int(pool5.get_shape()[0]) 115 | dense = tf.reshape(pool5, [batch_size1,-1]) 116 | 117 | dense1 = tf.layers.dense(dense, 1024) 118 | dense1 = tf.nn.relu(dense1) 119 | 120 | with tf.variable_scope('match_res_2'): 121 | dense2 = tf.layers.dense(dense1, 256) 122 | dense2 = tf.nn.relu(dense2) 123 | 124 | dense3 = tf.layers.dense(dense2, 1) 125 | output = tf.sigmoid(dense3) 126 | return output, dense1, [pool1, pool2, pool3, pool4, pool5, dense1, dense2] 127 | 128 | ''' 129 | ''' 130 | def matching_res_pro(dense1, dense1_res): 131 | dense_con = tf.concat([dense1, dense1_res], axis=1) # 1024 + 1024 = 2048 132 | dense1 = tf.layers.dense(dense_con, 1024) 133 | dense1 = tf.nn.relu(dense1) 134 | dense2 = tf.layers.dense(dense1, 256) 135 | dense2 = tf.nn.relu(dense2) 136 | dense3 = tf.layers.dense(dense2, 1) 137 | output = tf.sigmoid(dense3) 138 | return output, [dense1, dense2, dense3] 139 | 140 | ''' 141 | network model 142 | ''' 143 | def features_matching(inputs_p1, inputs_p2): 144 | with tf.variable_scope("matching"): 145 | layers_1,feature_map_1 = features(inputs_p1, 'feature_layers') # 4*4*256 146 | layers_2,feature_map_2 = features(inputs_p2, 'feature_layers',reuse=True) # 4*4*256 147 | output, dense1, feature_maps = matching(feature_map_1,feature_map_2) 148 | return output, dense1, [layers_1,layers_2,feature_maps] 149 | 150 | ''' 151 | features_matching_res network 152 | update: 2018.11.10 153 | 154 | ''' 155 | def features_matching_res(inputs_p1, inputs_p2): 156 | with tf.variable_scope("matching"): 157 | layers_1,feature_map_1 = features(inputs_p1, 'feature_layers') # 4*4*256 158 | layers_2,feature_map_2 = features(inputs_p2, 'feature_layers',reuse=True) # 4*4*256 159 | output, dense1, feature_maps = matching(feature_map_1,feature_map_2) 160 | with tf.variable_scope('match_res'): 161 | output_res, dense1_res, res_feature_maps = matching_res(abs(layers_1[0]-layers_2[0]), abs(layers_1[1]-layers_2[1]), abs(layers_1[2]-layers_2[2]), 162 | abs(layers_1[3]-layers_2[3]), abs(layers_1[4]-layers_2[4])) 163 | with tf.variable_scope('match_pro'): 164 | output_pro, pro_feature_maps = matching_res_pro(dense1, dense1_res) 165 | 166 | return output, output_res, output_pro, [layers_1, layers_2, feature_maps, res_feature_maps, pro_feature_maps] 167 | 168 | 169 | ''' 170 | cross loss function 171 | ''' 172 | def match_network(inputs_p1, inputs_p2, label_m): 173 | 174 | match_output, all_features = features_matching(inputs_p1, inputs_p2) 175 | match_loss = tf.reduce_mean(-(label_m * tf.log(match_output + EPS) + (1 - label_m) * tf.log(1 - match_output + EPS))) 176 | return match_loss, match_output, all_features 177 | 178 | def match_network_res(inputs_p1, inputs_p2, label_m): 179 | 180 | match_output, match_output_res, match_output_pro, all_features = features_matching_res(inputs_p1, inputs_p2) 181 | match_loss = tf.reduce_mean(-(label_m * tf.log(match_output + EPS) + (1 - label_m) * tf.log(1 - match_output + EPS))) 182 | match_loss_res = tf.reduce_mean(-(label_m * tf.log(match_output_res + EPS) + (1 - label_m) * tf.log(1 - match_output_res + EPS))) 183 | match_loss_pro = tf.reduce_mean(-(label_m * tf.log(match_output_pro + EPS) + (1 - label_m) * tf.log(1 - match_output_pro + EPS))) 184 | 185 | return match_loss, match_loss_res, match_loss_pro, match_output, match_output_res, match_output_pro, all_features 186 | 187 | 188 | 189 | 190 | -------------------------------------------------------------------------------- /part_2_element-wise_feature_relation/model_d_map32_res.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on 2018.11.9 11:03:33 4 | 5 | @author: Administrator 6 | 7 | 双分支 + 特征map + 交叉熵损失函数 + 参数共享 8 | 9 | """ 10 | 11 | import tensorflow as tf 12 | import numpy as np 13 | 14 | EPS = 1e-12 15 | 16 | def conv(batch_input, out_channels, filter_size, pad, stride=1): 17 | conv = tf.layers.conv2d(batch_input, out_channels,filter_size,strides=(stride,stride), padding=pad) 18 | return conv 19 | 20 | def batchnorm(inputs): 21 | normalized = tf.layers.batch_normalization(inputs, axis=-1) 22 | return normalized 23 | 24 | def evaluation(logits, labels): 25 | correct_prediction = tf.equal(logits, labels) 26 | correct_batch = tf.reduce_mean(tf.cast(correct_prediction, tf.int32), 1) 27 | return tf.reduce_sum(tf.cast(correct_batch, tf.float32)), correct_batch 28 | 29 | ''' 30 | # image patch features abstract layers 31 | update: 2018.11.9 32 | 5*conv + fc 33 | 34 | ''' 35 | def features(img, name, reuse=False): 36 | with tf.variable_scope(name,reuse=reuse): 37 | layer1 = conv(img, 32, 3, pad='SAME') # 64*64*32 38 | layer1 = tf.nn.relu(layer1) 39 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 40 | 41 | layer2 = conv(pool1, 64, 3, pad='SAME') # 32*32*64 42 | layer2 = tf.nn.relu(layer2) 43 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 44 | 45 | layer3 = conv(pool2, 128, 3, pad='SAME') # 16*16*128 46 | layer3 = tf.nn.relu(layer3) 47 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 48 | 49 | layer4 = conv(pool3, 256, 3, pad='SAME') # 8*8*256 50 | layer4 = tf.nn.relu(layer4) 51 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 52 | # 4*4*256 53 | return [layer1, pool1, pool2, pool3, pool4], pool4 54 | 55 | ''' 56 | matching layers 57 | update: 2018.11.9 58 | 59 | ''' 60 | def matching(feature_map_1,feature_map_2): 61 | with tf.variable_scope('match_layers'): 62 | all_feature = tf.concat([feature_map_1,feature_map_2], axis=3) # 两个特征连接 4*4*512 63 | layer5 = conv(all_feature, 512, 3, pad='SAME') 64 | layer5 = tf.nn.relu(layer5) 65 | pool5 = tf.nn.max_pool(layer5,[1,2,2,1],[1,2,2,1],padding='SAME') # 2*2*512 66 | 67 | batch_size = int(pool5.get_shape()[0]) 68 | dense = tf.reshape(pool5, [batch_size,-1]) # reshape as a tensor 2048 69 | dense1 = tf.layers.dense(dense, 1024) 70 | dense1 = tf.nn.relu(dense1) 71 | dense2 = tf.layers.dense(dense1, 256) 72 | dense2 = tf.nn.relu(dense2) 73 | dense3 = tf.layers.dense(dense2, 1) 74 | output = tf.sigmoid(dense3) 75 | return output, [pool5, dense1, dense2] 76 | 77 | ''' 78 | add the resnet 79 | update: 2018.11.10 80 | 81 | ''' 82 | def matching_res(layer1_res, pool1_res, pool2_res, pool3_res, pool4_res): 83 | 84 | layer1 = conv(layer1_res, 32, 1, pad='SAME') 85 | layer1 = tf.nn.relu(layer1) 86 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 87 | 88 | pool1_con = tf.concat([pool1, pool1_res], axis=3) 89 | layer2 = conv(pool1_con, 64, 1, pad='SAME') 90 | layer2 = tf.nn.relu(layer2) 91 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 92 | 93 | pool2_con = tf.concat([pool2, pool2_res], axis=3) 94 | layer3 = conv(pool2_con, 128, 1, pad='SAME') 95 | layer3 = tf.nn.relu(layer3) 96 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 97 | 98 | pool3_con = tf.concat([pool3, pool3_res], axis=3) 99 | layer4 = conv(pool3_con, 256, 1, pad='SAME') 100 | layer4 = tf.nn.relu(layer4) 101 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 102 | 103 | pool4_con = tf.concat([pool4, pool4_res], axis=3) 104 | layer5 = conv(pool4_con, 512, 1, pad='SAME') 105 | layer5 = tf.nn.relu(layer5) 106 | pool5 = tf.nn.max_pool(layer5,[1,2,2,1],[1,2,2,1],padding='SAME') # 2*2*512 107 | 108 | # 2*2*512=2048 109 | batch_size1 = int(pool5.get_shape()[0]) 110 | dense = tf.reshape(pool5, [batch_size1,-1]) 111 | 112 | dense1 = tf.layers.dense(dense, 1024) 113 | dense1 = tf.nn.relu(dense1) 114 | 115 | dense2 = tf.layers.dense(dense1, 256) 116 | dense2 = tf.nn.relu(dense2) 117 | 118 | dense3 = tf.layers.dense(dense2, 1) 119 | output = tf.sigmoid(dense3) 120 | return output,[pool1, pool2, pool3, pool4, pool5, dense1, dense2] 121 | 122 | 123 | ''' 124 | network model 125 | ''' 126 | def features_matching(inputs_p1, inputs_p2): 127 | with tf.variable_scope("matching"): 128 | layers_1,feature_map_1 = features(inputs_p1, 'feature_layers') # 4*4*256 129 | layers_2,feature_map_2 = features(inputs_p2, 'feature_layers',reuse=True) # 4*4*256 130 | output,feature_maps = matching(feature_map_1,feature_map_2) 131 | return output ,[layers_1,layers_2,feature_maps] 132 | 133 | ''' 134 | features_matching_res network 135 | update: 2018.11.10 136 | 137 | ''' 138 | def features_matching_res(inputs_p1, inputs_p2): 139 | with tf.variable_scope("matching"): 140 | layers_1,feature_map_1 = features(inputs_p1, 'feature_layers') # 4*4*256 141 | layers_2,feature_map_2 = features(inputs_p2, 'feature_layers',reuse=True) # 4*4*256 142 | output,feature_maps = matching(feature_map_1,feature_map_2) 143 | with tf.variable_scope('match_res'): 144 | output_res, res_feature_maps = matching_res(abs(layers_1[0]-layers_2[0]), abs(layers_1[1]-layers_2[1]), abs(layers_1[2]-layers_2[2]), 145 | abs(layers_1[3]-layers_2[3]), abs(layers_1[4]-layers_2[4])) 146 | return output, output_res, [layers_1, layers_2, feature_maps, res_feature_maps] 147 | 148 | 149 | ''' 150 | cross loss function 151 | ''' 152 | 153 | def match_network(inputs_p1, inputs_p2, label_m): 154 | 155 | match_output, all_features = features_matching(inputs_p1, inputs_p2) 156 | match_loss = tf.reduce_mean(-(label_m * tf.log(match_output + EPS) + (1 - label_m) * tf.log(1 - match_output + EPS))) 157 | return match_loss, match_output, all_features 158 | 159 | def match_network_res(inputs_p1, inputs_p2, label_m): 160 | 161 | match_output, match_output_res, all_features = features_matching_res(inputs_p1, inputs_p2) 162 | match_loss = tf.reduce_mean(-(label_m * tf.log(match_output + EPS) + (1 - label_m) * tf.log(1 - match_output + EPS))) 163 | match_loss_res = tf.reduce_mean(-(label_m * tf.log(match_output_res + EPS) + (1 - label_m) * tf.log(1 - match_output_res + EPS))) 164 | return match_loss, match_loss_res, match_output, match_output_res, all_features 165 | 166 | 167 | 168 | 169 | -------------------------------------------------------------------------------- /part_2_element-wise_feature_relation/read_data.py: -------------------------------------------------------------------------------- 1 | 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Created on Thu Feb 1 13:56:02 2018 5 | 6 | @author: DELL 7 | """ 8 | 9 | from __future__ import absolute_import 10 | from __future__ import division 11 | from __future__ import print_function 12 | 13 | import os.path 14 | import tensorflow as tf 15 | import scipy.misc as misc 16 | import numpy as np 17 | 18 | SAVA_PATH = 'country' 19 | TRAIN_FILE = 'country.tfrecord' 20 | BATCH_SIZE = 100 21 | img_size = 64 22 | 23 | def read_and_decode_train(filename_queue): 24 | # create a reader from file queue 25 | reader = tf.TFRecordReader() 26 | #reader从 TFRecord 读取内容并保存到 serialized_example 中 27 | _, serialized_example = reader.read(filename_queue) 28 | 29 | features = tf.parse_single_example( # 读取 serialized_example 的格式 30 | serialized_example, 31 | features={ 32 | 'label_raw': tf.FixedLenFeature([], tf.string), 33 | 'image_raw': tf.FixedLenFeature([], tf.string) 34 | } 35 | ) # 解析从 serialized_example 读取到的内容 36 | img = tf.decode_raw(features['image_raw'], tf.uint8) 37 | img = tf.reshape(img, [img_size, img_size*2]) 38 | img = tf.cast(img, tf.float32) * (1. / 255.0) #[0,1] 39 | label = tf.decode_raw(features['label_raw'], tf.int32) 40 | label = tf.reshape(tf.cast(label, tf.float32),shape=(1,)) 41 | return img,label 42 | # return tf.cast(resize_image, tf.float32), tf.cast(reshape_label, tf.float32) 43 | 44 | def read_and_decode_test(filename_queue): 45 | # create a reader from file queue 46 | reader = tf.TFRecordReader() 47 | #reader从 TFRecord 读取内容并保存到 serialized_example 中 48 | _, serialized_example = reader.read(filename_queue) 49 | 50 | features = tf.parse_single_example( # 读取 serialized_example 的格式 51 | serialized_example, 52 | features={ 53 | 'label_raw': tf.FixedLenFeature([], tf.string), 54 | 'image_raw': tf.FixedLenFeature([], tf.string) 55 | } 56 | ) # 解析从 serialized_example 读取到的内容 57 | img = tf.decode_raw(features['image_raw'], tf.uint8) 58 | img = tf.reshape(img, [img_size, img_size*2]) 59 | img = tf.cast(img, tf.float32) * (1. / 255.0) #[0,1] 60 | label = tf.decode_raw(features['label_raw'], tf.int32) 61 | label = tf.reshape(tf.cast(label, tf.float32),shape=(1,)) 62 | return img,label 63 | 64 | def batch_inputs(filename_queue, train, batch_size): 65 | #创建文件队列,不限读取的数量 66 | with tf.name_scope('input'): 67 | if train: 68 | image, label = read_and_decode_train(filename_queue) 69 | images, labels = tf.train.shuffle_batch([image, label], 70 | batch_size=batch_size, 71 | num_threads=6, 72 | capacity=2000 + 3 * batch_size, 73 | min_after_dequeue=2000) 74 | else: 75 | image, label = read_and_decode_test(filename_queue) 76 | images, labels = tf.train.batch([image, label], 77 | batch_size=batch_size, 78 | num_threads=6, 79 | capacity=2000 + 3 * batch_size) 80 | return images, labels 81 | 82 | if __name__ == '__main__': 83 | 84 | init = tf.global_variables_initializer() 85 | # filename = os.path.join(SAVA_PATH, TRAIN_FILE) 86 | filename = '/home/ws/文档/wrj/data_all/country/country.tfrecord' 87 | filename_queue = tf.train.string_input_producer([filename],num_epochs=20, shuffle=True) 88 | img_batch, label_batch = batch_inputs(filename_queue, train = True, batch_size = BATCH_SIZE) 89 | with tf.Session() as sess: 90 | # sess.run(init) 91 | sess.run(tf.global_variables_initializer()) 92 | sess.run(tf.local_variables_initializer()) 93 | 94 | coord = tf.train.Coordinator() #创建一个协调器，管理线程 95 | #启动QueueRunner, 此时文件名队列已经进队。 96 | threads=tf.train.start_queue_runners(sess=sess,coord=coord) 97 | try: 98 | for i in range(200): 99 | img, label = sess.run([img_batch, label_batch]) 100 | for j in range(img.shape[0]): 101 | misc.imsave(str(j)+'.png', np.squeeze(img[j])) 102 | except tf.errors.OutOfRangeError: 103 | print('Done reading') 104 | finally: 105 | coord.request_stop() 106 | coord.request_stop() 107 | coord.join(threads) 108 | 109 | 110 | -------------------------------------------------------------------------------- /part_2_element-wise_feature_relation/read_data_map.py: -------------------------------------------------------------------------------- 1 | 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Created on Thu Feb 1 13:56:02 2018 5 | 6 | @author: DELL 7 | """ 8 | 9 | from __future__ import absolute_import 10 | from __future__ import division 11 | from __future__ import print_function 12 | 13 | import os.path 14 | import tensorflow as tf 15 | import scipy.misc as misc 16 | import numpy as np 17 | 18 | 19 | SAVA_PATH = 'mapping_data' 20 | TRAIN_FILE = 'map_country.tfrecord' 21 | BATCH_SIZE = 10 22 | img_size = 64 23 | 24 | 25 | 26 | def read_and_decode_train(filename_queue): 27 | # create a reader from file queue 28 | reader = tf.TFRecordReader() 29 | #reader从 TFRecord 读取内容并保存到 serialized_example 中 30 | _, serialized_example = reader.read(filename_queue) 31 | 32 | features = tf.parse_single_example( # 读取 serialized_example 的格式 33 | serialized_example, 34 | features={ 35 | 'label_raw': tf.FixedLenFeature([], tf.string), 36 | 'image_raw': tf.FixedLenFeature([], tf.string) 37 | } 38 | ) # 解析从 serialized_example 读取到的内容 39 | img = tf.decode_raw(features['image_raw'], tf.uint8) 40 | img = tf.reshape(img, [img_size, img_size*2]) 41 | # img_gray = tf.image.rgb_to_grayscale(img) 42 | img = tf.cast(img, tf.float32) * (1. / 255.0) #[0,1] 43 | label = tf.decode_raw(features['label_raw'], tf.int32) 44 | label = tf.reshape(tf.cast(label, tf.float32),shape=(1,)) 45 | return img,label 46 | # return tf.cast(resize_image, tf.float32), tf.cast(reshape_label, tf.float32) 47 | 48 | def read_and_decode_test(filename_queue): 49 | # create a reader from file queue 50 | reader = tf.TFRecordReader() 51 | #reader从 TFRecord 读取内容并保存到 serialized_example 中 52 | _, serialized_example = reader.read(filename_queue) 53 | 54 | features = tf.parse_single_example( # 读取 serialized_example 的格式 55 | serialized_example, 56 | features={ 57 | 'label_raw': tf.FixedLenFeature([], tf.string), 58 | 'image_raw': tf.FixedLenFeature([], tf.string) 59 | } 60 | ) # 解析从 serialized_example 读取到的内容 61 | img = tf.decode_raw(features['image_raw'], tf.uint8) 62 | img = tf.reshape(img, [img_size, img_size*2]) 63 | # img_gray = tf.image.rgb_to_grayscale(img) 64 | img = tf.cast(img, tf.float32) * (1. / 255.0) #[0,1] 65 | label = tf.decode_raw(features['label_raw'], tf.int32) 66 | label = tf.reshape(tf.cast(label, tf.float32),shape=(1,)) 67 | return img,label 68 | 69 | def batch_inputs(filename_queue, train, batch_size): 70 | #创建文件队列,不限读取的数量 71 | with tf.name_scope('input'): 72 | if train: 73 | image, label = read_and_decode_train(filename_queue) 74 | images, labels = tf.train.shuffle_batch([image, label], 75 | batch_size=batch_size, 76 | num_threads=6, 77 | capacity=2000 + 3 * batch_size, 78 | min_after_dequeue=2000) 79 | else: 80 | image, label = read_and_decode_test(filename_queue) 81 | images, labels = tf.train.batch([image, label], 82 | batch_size=batch_size, 83 | num_threads=6, 84 | capacity=2000 + 3 * batch_size) 85 | return images, labels 86 | 87 | if __name__ == '__main__': 88 | 89 | init = tf.global_variables_initializer() 90 | # filename = os.path.join(SAVA_PATH, TRAIN_FILE) 91 | filename = '/home/ws/文档/wrj/mapping_data_all/mapping_data/map_country.tfrecord' 92 | filename_queue = tf.train.string_input_producer([filename],num_epochs=10 , shuffle=True) 93 | img_batch, label_batch = batch_inputs(filename_queue, train = True, batch_size = BATCH_SIZE) 94 | with tf.Session() as sess: 95 | # sess.run(init) 96 | sess.run(tf.global_variables_initializer()) 97 | sess.run(tf.local_variables_initializer()) 98 | 99 | coord = tf.train.Coordinator() #创建一个协调器，管理线程 100 | #启动QueueRunner, 此时文件名队列已经进队。 101 | threads=tf.train.start_queue_runners(sess=sess,coord=coord) 102 | try: 103 | for i in range(100): 104 | img, label = sess.run([img_batch, label_batch]) 105 | for j in range(img.shape[0]): 106 | misc.imsave(str(j)+'_read.png', np.squeeze(img[j])) 107 | except tf.errors.OutOfRangeError: 108 | print('Done reading') 109 | finally: 110 | coord.request_stop() 111 | coord.request_stop() 112 | coord.join(threads) 113 | 114 | 115 | -------------------------------------------------------------------------------- /part_3_GAN_transform/README: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /part_3_GAN_transform/model_d_map32.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on 2018.11.9 11:03:33 4 | 5 | @author: Administrator 6 | 7 | 双分支 + 特征map + 交叉熵损失函数 + 参数共享 8 | 9 | """ 10 | 11 | import tensorflow as tf 12 | import numpy as np 13 | 14 | EPS = 1e-12 15 | 16 | def conv(batch_input, out_channels, filter_size, pad, stride=1): 17 | conv = tf.layers.conv2d(batch_input, out_channels,filter_size,strides=(stride,stride), padding=pad) 18 | return conv 19 | 20 | def batchnorm(inputs): 21 | normalized = tf.layers.batch_normalization(inputs, axis=-1) 22 | return normalized 23 | 24 | def evaluation(logits, labels): 25 | correct_prediction = tf.equal(logits, labels) 26 | correct_batch = tf.reduce_mean(tf.cast(correct_prediction, tf.int32), 1) 27 | return tf.reduce_sum(tf.cast(correct_batch, tf.float32)), correct_batch 28 | 29 | ''' 30 | # image patch features abstract layers 31 | update: 2018.11.9 32 | 5*conv + fc 33 | 34 | ''' 35 | def features(img, name, reuse=False): 36 | with tf.variable_scope(name,reuse=reuse): 37 | layer1 = conv(img, 32, 3, pad='SAME') # 64*64*32 38 | layer1 = tf.nn.relu(layer1) 39 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 40 | 41 | layer2 = conv(pool1, 64, 3, pad='SAME') # 32*32*64 42 | layer2 = tf.nn.relu(layer2) 43 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 44 | 45 | layer3 = conv(pool2, 128, 3, pad='SAME') # 16*16*128 46 | layer3 = tf.nn.relu(layer3) 47 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 48 | 49 | layer4 = conv(pool3, 256, 3, pad='SAME') # 8*8*256 50 | layer4 = tf.nn.relu(layer4) 51 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 52 | # 4*4*256 53 | return [layer1, pool1, pool2, pool3, pool4], pool4 54 | 55 | ''' 56 | matching layers 57 | update: 2018.11.9 58 | 59 | ''' 60 | def matching(feature_map_1,feature_map_2): 61 | with tf.variable_scope('match_layers'): 62 | all_feature = tf.concat([feature_map_1,feature_map_2], axis=3) # 两个特征连接 4*4*512 63 | layer5 = conv(all_feature, 512, 3, pad='SAME') 64 | layer5 = tf.nn.relu(layer5) 65 | pool5 = tf.nn.max_pool(layer5,[1,2,2,1],[1,2,2,1],padding='SAME') # 2*2*512 66 | 67 | batch_size = int(pool5.get_shape()[0]) 68 | dense = tf.reshape(pool5, [batch_size,-1]) # reshape as a tensor 2048 69 | dense1 = tf.layers.dense(dense, 1024) 70 | dense1 = tf.nn.relu(dense1) 71 | dense2 = tf.layers.dense(dense1, 256) 72 | dense2 = tf.nn.relu(dense2) 73 | dense3 = tf.layers.dense(dense2, 1) 74 | output = tf.sigmoid(dense3) 75 | return output, [pool5, dense1, dense2] 76 | 77 | ''' 78 | network model 79 | ''' 80 | def features_matching(inputs_p1, inputs_p2): 81 | with tf.variable_scope("matching"): 82 | layers_1,feature_map_1 = features(inputs_p1, 'feature_layers') # 4*4*256 83 | layers_2,feature_map_2 = features(inputs_p2, 'feature_layers',reuse=True) # 4*4*256 84 | output,feature_maps = matching(feature_map_1,feature_map_2) 85 | return output ,[layers_1,layers_2,feature_maps] 86 | 87 | ''' 88 | cross loss function 89 | ''' 90 | 91 | def match_network(inputs_p1, inputs_p2, label_m): 92 | 93 | match_output, all_features = features_matching(inputs_p1, inputs_p2) 94 | match_loss = tf.reduce_mean(-(label_m * tf.log(match_output + EPS) + (1 - label_m) * tf.log(1 - match_output + EPS))) 95 | return match_loss, match_output, all_features 96 | 97 | 98 | 99 | 100 | 101 | 102 | -------------------------------------------------------------------------------- /part_3_GAN_transform/model_d_map32_diff.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on 2018.11.9 11:03:33 4 | 5 | @author: Administrator 6 | 7 | 双分支 + 特征map + 交叉熵损失函数 + 参数共享 + diffrent 8 | 9 | """ 10 | 11 | import tensorflow as tf 12 | import numpy as np 13 | 14 | EPS = 1e-12 15 | 16 | def conv(batch_input, out_channels, filter_size, pad, stride=1): 17 | conv = tf.layers.conv2d(batch_input, out_channels,filter_size,strides=(stride,stride), padding=pad) 18 | return conv 19 | 20 | def batchnorm(inputs): 21 | normalized = tf.layers.batch_normalization(inputs, axis=-1) 22 | return normalized 23 | 24 | def evaluation(logits, labels): 25 | correct_prediction = tf.equal(logits, labels) 26 | correct_batch = tf.reduce_mean(tf.cast(correct_prediction, tf.int32), 1) 27 | return tf.reduce_sum(tf.cast(correct_batch, tf.float32)), correct_batch 28 | 29 | ''' 30 | # image patch features abstract layers 31 | update: 2018.11.9 32 | 5*conv + fc 33 | 34 | ''' 35 | def features(img, name, reuse=False): 36 | with tf.variable_scope(name,reuse=reuse): 37 | layer1 = conv(img, 32, 3, pad='SAME') # 64*64*32 38 | layer1 = tf.nn.relu(layer1) 39 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 40 | 41 | layer2 = conv(pool1, 64, 3, pad='SAME') # 32*32*64 42 | layer2 = tf.nn.relu(layer2) 43 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 44 | 45 | layer3 = conv(pool2, 128, 3, pad='SAME') # 16*16*128 46 | layer3 = tf.nn.relu(layer3) 47 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 48 | 49 | layer4 = conv(pool3, 256, 3, pad='SAME') # 8*8*256 50 | layer4 = tf.nn.relu(layer4) 51 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 52 | # 4*4*256 53 | return [layer1, pool1, pool2, pool3, pool4], pool4 54 | 55 | ''' 56 | matching layers 57 | update: 2018.11.9 58 | 59 | ''' 60 | def matching(feature_map_1,feature_map_2): 61 | with tf.variable_scope('match_layers'): 62 | # all_feature = tf.concat([feature_map_1,feature_map_2], axis=3) # 两个特征连接 4*4*512 63 | diff_feature = tf.abs(feature_map_1 - feature_map_2) # 2*2*256 64 | layer5 = conv(diff_feature, 512, 3, pad='SAME') 65 | layer5 = tf.nn.relu(layer5) 66 | pool5 = tf.nn.max_pool(layer5,[1,2,2,1],[1,2,2,1],padding='SAME') # 2*2*512 67 | 68 | batch_size = int(pool5.get_shape()[0]) 69 | dense = tf.reshape(pool5, [batch_size,-1]) # reshape as a tensor 2048 70 | dense1 = tf.layers.dense(dense, 1024) 71 | dense1 = tf.nn.relu(dense1) 72 | dense2 = tf.layers.dense(dense1, 256) 73 | dense2 = tf.nn.relu(dense2) 74 | dense3 = tf.layers.dense(dense2, 1) 75 | output = tf.sigmoid(dense3) 76 | return output, [pool5, dense1, dense2] 77 | 78 | ''' 79 | network model 80 | ''' 81 | def features_matching(inputs_p1, inputs_p2): 82 | with tf.variable_scope("matching"): 83 | layers_1,feature_map_1 = features(inputs_p1, 'feature_layers') # 4*4*256 84 | layers_2,feature_map_2 = features(inputs_p2, 'feature_layers',reuse=True) # 4*4*256 85 | output,feature_maps = matching(feature_map_1,feature_map_2) 86 | return output ,[layers_1,layers_2,feature_maps] 87 | 88 | ''' 89 | cross loss function 90 | ''' 91 | 92 | def match_network(inputs_p1, inputs_p2, label_m): 93 | 94 | match_output, all_features = features_matching(inputs_p1, inputs_p2) 95 | match_loss = tf.reduce_mean(-(label_m * tf.log(match_output + EPS) + (1 - label_m) * tf.log(1 - match_output + EPS))) 96 | return match_loss, match_output, all_features 97 | 98 | 99 | 100 | 101 | 102 | 103 | -------------------------------------------------------------------------------- /part_3_GAN_transform/model_d_tensor.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on 2018.11.9 11:03:33 4 | 5 | @author: Administrator 6 | 7 | 双分支 + 特征向量 + 交叉熵损失函数 + 参数共享 8 | 9 | """ 10 | 11 | import tensorflow as tf 12 | import numpy as np 13 | 14 | EPS = 1e-12 15 | 16 | def conv(batch_input, out_channels, filter_size, pad, stride=1): 17 | conv = tf.layers.conv2d(batch_input, out_channels,filter_size,strides=(stride,stride), padding=pad) 18 | return conv 19 | 20 | def batchnorm(inputs): 21 | normalized = tf.layers.batch_normalization(inputs, axis=-1) 22 | return normalized 23 | 24 | def evaluation(logits, labels): 25 | correct_prediction = tf.equal(logits, labels) 26 | correct_batch = tf.reduce_mean(tf.cast(correct_prediction, tf.int32), 1) 27 | return tf.reduce_sum(tf.cast(correct_batch, tf.float32)), correct_batch 28 | 29 | ''' 30 | # image patch features abstract layers 31 | update: 2018.11.9 32 | 5*conv + fc 33 | 34 | ''' 35 | def features(img, name, reuse=False): 36 | with tf.variable_scope(name,reuse=reuse): 37 | layer1 = conv(img, 32, 3, pad='SAME') # 64*64*32 38 | layer1 = tf.nn.relu(layer1) 39 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 40 | 41 | layer2 = conv(pool1, 64, 3, pad='SAME') # 32*32*64 42 | layer2 = tf.nn.relu(layer2) 43 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 44 | 45 | layer3 = conv(pool2, 128, 3, pad='SAME') # 16*16*128 46 | layer3 = tf.nn.relu(layer3) 47 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 48 | 49 | layer4 = conv(pool3, 256, 3, pad='SAME') # 8*8*256 50 | layer4 = tf.nn.relu(layer4) 51 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 52 | 53 | # layer5 = conv(pool4, 256, 3, pad='SAME') # 4*4*256 54 | # layer5 = tf.nn.relu(layer5) 55 | pool5 = tf.nn.max_pool(pool4,[1,2,2,1],[1,2,2,1],padding='SAME') 56 | 57 | # 2*2*256=1024 58 | batch_size = int(pool5.get_shape()[0]) 59 | dense = tf.reshape(pool5, [batch_size,-1]) 60 | 61 | return [pool1, pool2, pool3, pool4, pool5, dense], dense 62 | 63 | ''' 64 | matching layers 65 | update: 2018.11.9 66 | 67 | ''' 68 | def matching(dense_1,dense_2): 69 | with tf.variable_scope('match_layers'): 70 | all_feature = tf.concat([dense_1, dense_2], axis=1) # 两个特征连接 1024*2=2048 71 | dense1 = tf.layers.dense(all_feature, 1024) 72 | dense1 = tf.nn.relu(dense1) 73 | 74 | dense2 = tf.layers.dense(dense1, 256) 75 | dense2 = tf.nn.relu(dense2) 76 | 77 | dense3 = tf.layers.dense(dense2, 1) 78 | output = tf.sigmoid(dense3) 79 | return output, [dense1,dense2] 80 | 81 | ''' 82 | network model 83 | ''' 84 | def features_matching(inputs_p1, inputs_p2): 85 | with tf.variable_scope("matching"): 86 | layers_1,dense_1 = features(inputs_p1, 'feature_layers') # 4*4*256 87 | layers_2,dense_2 = features(inputs_p2, 'feature_layers',reuse=True) # 4*4*256 88 | output,denses = matching(dense_1,dense_2) 89 | return output ,[layers_1,layers_2,denses] 90 | 91 | ''' 92 | cross loss function 93 | ''' 94 | 95 | def match_network(inputs_p1, inputs_p2, label_m): 96 | 97 | match_output, all_features = features_matching(inputs_p1, inputs_p2) 98 | match_loss = tf.reduce_mean(-(label_m * tf.log(match_output + EPS) + (1 - label_m) * tf.log(1 - match_output + EPS))) 99 | return match_loss, match_output, all_features 100 | 101 | 102 | 103 | 104 | 105 | 106 | -------------------------------------------------------------------------------- /part_3_GAN_transform/model_gd_diff.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on Sat Mar 3 11:03:33 2018 4 | 5 | @author: Administrator 6 | """ 7 | 8 | import tensorflow as tf 9 | import numpy as np 10 | import argparse 11 | 12 | parser = argparse.ArgumentParser() 13 | parser.add_argument("--l1_weight", type=float, default=100.0, help="weight on L1 term for generator gradient") 14 | parser.add_argument("--gan_weight", type=float, default=1.0, help="weight on GAN term for generator gradient") 15 | a = parser.parse_args() 16 | 17 | EPS = 1e-12 18 | 19 | def conv(batch_input, out_channels, filter_size, pad, stride=1): 20 | conv = tf.layers.conv2d(batch_input, out_channels,filter_size,strides=(stride,stride), padding=pad) 21 | 22 | return conv 23 | 24 | def deconv(batch_input, out_channels, filter_size, pad): 25 | conv = tf.layers.conv2d_transpose(batch_input, out_channels, filter_size, strides=(2,2), padding=pad ,kernel_initializer=tf.random_normal_initializer(0, 0.02)) 26 | return conv 27 | 28 | def batchnorm(inputs): 29 | normalized = tf.layers.batch_normalization(inputs, axis=-1) 30 | return normalized 31 | 32 | def lrelu(x, a): 33 | with tf.name_scope("lrelu"): 34 | # adding these together creates the leak part and linear part 35 | # then cancels them out by subtracting/adding an absolute value term 36 | # leak: a*x/2 - a*abs(x)/2 37 | # linear: x/2 + abs(x)/2 38 | 39 | # this block looks like it has 2 inputs on the graph unless we do this 40 | x = tf.identity(x) 41 | return (0.5 * (1 + a)) * x + (0.5 * (1 - a)) * tf.abs(x) 42 | 43 | def preprocess(image): 44 | with tf.name_scope("preprocess"): 45 | # [0, 1] => [-1, 1] 46 | return tf.subtract(tf.multiply(image, 2.0),1.0) 47 | # return image * 2.0 - 1.0 48 | 49 | def deprocess(image): 50 | with tf.name_scope("deprocess"): 51 | # [-1, 1] => [0, 1] 52 | return tf.div(tf.add(image , 1.0),2.0) 53 | # return (image + 1.0) / 2.0 54 | 55 | def evaluation(logits, labels): 56 | correct_prediction = tf.equal(logits, labels) 57 | correct_batch = tf.reduce_mean(tf.cast(correct_prediction, tf.int32), 1) 58 | return tf.reduce_sum(tf.cast(correct_batch, tf.float32)), correct_batch 59 | 60 | # U_Net结构， 用于将 nir ---> gray 61 | def create_generator(generator_inputs, generator_outputs_channels, name="generator", reuse=False): 62 | with tf.variable_scope(name,reuse=reuse): 63 | layers = [] 64 | with tf.variable_scope("encoder_1"): 65 | # encoder_1: [batch, 64, 64, 1 ] => [batch, 32, 32, 32] 66 | convolved = conv(generator_inputs, 32, 3, pad='SAME') 67 | output = tf.nn.max_pool(convolved,[1,2,2,1],[1,2,2,1],padding='SAME') 68 | layers.append(output) 69 | 70 | layer_specs = [ 71 | 64, # encoder_2: [batch, 32, 32, 32 ] => [batch, 16, 16, 64] 72 | 128, # encoder_3: [batch, 16, 16, 64] => [batch, 8, 8, 128] 73 | 256, # encoder_3: [batch, 8, 8, 128] => [batch, 4, 4, 256] 74 | ] 75 | 76 | for out_channels in layer_specs: 77 | with tf.variable_scope("encoder_%d" % (len(layers) + 1)): 78 | # [batch, in_height, in_width, in_channels] => [batch, in_height/2, in_width/2, out_channels] 79 | output = lrelu(layers[-1], 0.2) 80 | convolved = conv(output, out_channels, 3, pad='SAME') 81 | output = tf.nn.max_pool(convolved,[1,2,2,1],[1,2,2,1],padding='SAME') 82 | output = batchnorm(output) 83 | layers.append(output) # 在每一层激活前，进行特征图的保存 84 | 85 | layer_specs = [ 86 | (128, 0.0), # decoder_2: [batch, 4, 4, 256] => [batch, 8, 8, 128] 87 | (64, 0.0), # decoder_3: [batch, 8, 8, 128] => [batch, 16, 16, 64] 88 | (32, 0.0), # decoder_3: [batch, 16, 16, 64] => [batch, 32, 32, 32] 89 | ] 90 | num_encoder_layers = len(layers) 91 | for decoder_layer, (out_channels, dropout) in enumerate(layer_specs): 92 | skip_layer = num_encoder_layers - decoder_layer - 1 93 | with tf.variable_scope("decoder_%d" % (skip_layer + 1)): 94 | if decoder_layer == 0: 95 | # first decoder layer doesn't have skip connections 96 | # since it is directly connected to the skip_layer 97 | inputs = layers[-1] 98 | else: 99 | inputs = tf.concat([layers[-1], layers[skip_layer]], axis=3) #跨层连接 100 | # [batch, in_height, in_width, in_channels] => [batch, in_height*2, in_width*2, out_channels] 101 | inputs = tf.nn.relu(inputs) 102 | output = deconv(inputs, out_channels, 3, pad='SAME') 103 | output = batchnorm(output) 104 | 105 | if dropout > 0.0: 106 | output = tf.nn.dropout(output, keep_prob=1 - dropout) 107 | 108 | layers.append(output) 109 | 110 | # decoder_4: [batch, 32, 32, ngf ] => [batch, 64, 64, generator_outputs_channels] 111 | 112 | with tf.variable_scope("decoder_1"): 113 | inputs = tf.concat([layers[-1], layers[0]], axis=3) 114 | inputs = tf.nn.relu(inputs) 115 | output = deconv(inputs, generator_outputs_channels,3,pad='SAME') 116 | output = tf.tanh(output) # output (-1,1) 117 | layers.append(output) 118 | return layers[-1] 119 | 120 | 121 | # 判别器网络，使用特征图相减求差，替代之前的双通道 122 | def features(img, name, reuse=False): 123 | with tf.variable_scope(name,reuse=reuse): 124 | layer1 = conv(img, 32, 3, pad='SAME') # 64*64*32 125 | layer1 = tf.nn.relu(layer1) 126 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 127 | 128 | layer2 = conv(pool1, 64, 3, pad='SAME') # 32*32*64 129 | layer2 = tf.nn.relu(layer2) 130 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 131 | 132 | layer3 = conv(pool2, 128, 3, pad='SAME') # 16*16*128 133 | layer3 = tf.nn.relu(layer3) 134 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 135 | 136 | layer4 = conv(pool3, 256, 3, pad='SAME') # 8*8*256 137 | layer4 = tf.nn.relu(layer4) 138 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 139 | # 4*4*256 140 | return pool4, [layer1, pool1, pool2, pool3, pool4] 141 | 142 | def matching(feature_map_1,feature_map_2): 143 | with tf.variable_scope('match_layers'): 144 | 145 | diff_feature = tf.abs(feature_map_1 - feature_map_2) # 2*2*256 146 | layer5 = conv(diff_feature, 512, 3, pad='SAME') 147 | layer5 = tf.nn.relu(layer5) 148 | pool5 = tf.nn.max_pool(layer5,[1,2,2,1],[1,2,2,1],padding='SAME') # 2*2*512 149 | 150 | batch_size = int(pool5.get_shape()[0]) 151 | dense = tf.reshape(pool5, [batch_size,-1]) # reshape as a tensor 2048 152 | dense1 = tf.layers.dense(dense, 1024) 153 | dense1 = tf.nn.relu(dense1) 154 | dense2 = tf.layers.dense(dense1, 256) 155 | dense2 = tf.nn.relu(dense2) 156 | dense3 = tf.layers.dense(dense2, 1) 157 | output = tf.sigmoid(dense3) 158 | return output 159 | 160 | # 判别器网络，p1：gray p2： nir 161 | def create_discriminator(inputs_p1, inputs_p2, name="discriminator", reuse=False): 162 | with tf.variable_scope(name, reuse=reuse): 163 | feature_map_1, layers_1 = features(inputs_p1, 'feature_layers') # 4*4*256 164 | feature_map_2, layers_2 = features(inputs_p2, 'feature_layers',reuse=True) # 4*4*256 165 | output = matching(feature_map_1, feature_map_2) 166 | return output 167 | 168 | def gd_model_g2r(inputs_g, inputs_r): 169 | # gray --> nir 170 | out_channels = int(inputs_r.get_shape()[-1]) 171 | g_outputs = create_generator(inputs_g, out_channels) # fake_optical 172 | # create two copies of discriminator, one for real pairs and one for fake pairs 173 | # they share the same underlying variables 174 | with tf.name_scope("real_discriminator"): 175 | # 2x [batch, height, width, channels] => [batch, 1] 176 | predict_real = create_discriminator(inputs_g, inputs_r) 177 | with tf.name_scope("fake_discriminator"): 178 | # 2x [batch, height, width, channels] => [batch, 1] 179 | predict_fake = create_discriminator(inputs_g, g_outputs, reuse=True) 180 | 181 | with tf.name_scope("discriminator_loss"): 182 | # minimizing -tf.log will try to get inputs to 1 183 | # predict_real => 1 184 | # predict_fake => 0 185 | discrim_loss = tf.reduce_mean(-(tf.log(predict_real + EPS) + tf.log(1 - predict_fake + EPS))) 186 | 187 | with tf.name_scope("generator_loss"): 188 | # predict_fake => 1 189 | # abs(targets - outputs) => 0 190 | gen_loss_GAN = tf.reduce_mean(-tf.log(predict_fake + EPS)) 191 | gen_loss_L1 = tf.reduce_mean(tf.abs(inputs_r - g_outputs)) 192 | gen_loss = gen_loss_GAN * a.gan_weight + gen_loss_L1 * a.l1_weight 193 | 194 | return gen_loss, discrim_loss, g_outputs, [gen_loss_GAN, gen_loss_L1] 195 | 196 | 197 | 198 | 199 | 200 | -------------------------------------------------------------------------------- /part_3_GAN_transform/model_gd_diff_pro.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on Sat Mar 3 11:03:33 2018 4 | 5 | @author: Administrator 6 | """ 7 | 8 | import tensorflow as tf 9 | import numpy as np 10 | import argparse 11 | 12 | parser = argparse.ArgumentParser() 13 | parser.add_argument("--l1_weight", type=float, default=100.0, help="weight on L1 term for generator gradient") 14 | parser.add_argument("--gan_weight", type=float, default=1.0, help="weight on GAN term for generator gradient") 15 | a = parser.parse_args() 16 | 17 | EPS = 1e-12 18 | 19 | def conv(batch_input, out_channels, filter_size, pad, stride=1): 20 | conv = tf.layers.conv2d(batch_input, out_channels,filter_size,strides=(stride,stride), padding=pad) 21 | 22 | return conv 23 | 24 | def deconv(batch_input, out_channels, filter_size, pad): 25 | conv = tf.layers.conv2d_transpose(batch_input, out_channels, filter_size, strides=(2,2), padding=pad ,kernel_initializer=tf.random_normal_initializer(0, 0.02)) 26 | return conv 27 | 28 | def batchnorm(inputs): 29 | normalized = tf.layers.batch_normalization(inputs, axis=-1) 30 | return normalized 31 | 32 | def lrelu(x, a): 33 | with tf.name_scope("lrelu"): 34 | # adding these together creates the leak part and linear part 35 | # then cancels them out by subtracting/adding an absolute value term 36 | # leak: a*x/2 - a*abs(x)/2 37 | # linear: x/2 + abs(x)/2 38 | 39 | # this block looks like it has 2 inputs on the graph unless we do this 40 | x = tf.identity(x) 41 | return (0.5 * (1 + a)) * x + (0.5 * (1 - a)) * tf.abs(x) 42 | 43 | def preprocess(image): 44 | with tf.name_scope("preprocess"): 45 | # [0, 1] => [-1, 1] 46 | return tf.subtract(tf.multiply(image, 2.0),1.0) 47 | # return image * 2.0 - 1.0 48 | 49 | def deprocess(image): 50 | with tf.name_scope("deprocess"): 51 | # [-1, 1] => [0, 1] 52 | return tf.div(tf.add(image , 1.0),2.0) 53 | # return (image + 1.0) / 2.0 54 | 55 | def evaluation(logits, labels): 56 | correct_prediction = tf.equal(logits, labels) 57 | correct_batch = tf.reduce_mean(tf.cast(correct_prediction, tf.int32), 1) 58 | return tf.reduce_sum(tf.cast(correct_batch, tf.float32)), correct_batch 59 | 60 | # U_Net结构， 用于将 nir ---> gray 61 | def create_generator(generator_inputs, generator_outputs_channels, name="generator", reuse=False): 62 | with tf.variable_scope(name,reuse=reuse): 63 | layers = [] 64 | with tf.variable_scope("encoder_1"): 65 | # encoder_1: [batch, 64, 64, 1 ] => [batch, 32, 32, 32] 66 | convolved = conv(generator_inputs, 32, 3, pad='SAME') 67 | output = tf.nn.max_pool(convolved,[1,2,2,1],[1,2,2,1],padding='SAME') 68 | layers.append(output) 69 | 70 | layer_specs = [ 71 | 64, # encoder_2: [batch, 32, 32, 32 ] => [batch, 16, 16, 64] 72 | 128, # encoder_3: [batch, 16, 16, 64] => [batch, 8, 8, 128] 73 | 256, # encoder_3: [batch, 8, 8, 128] => [batch, 4, 4, 256] 74 | ] 75 | 76 | for out_channels in layer_specs: 77 | with tf.variable_scope("encoder_%d" % (len(layers) + 1)): 78 | # [batch, in_height, in_width, in_channels] => [batch, in_height/2, in_width/2, out_channels] 79 | output = lrelu(layers[-1], 0.2) 80 | convolved = conv(output, out_channels, 3, pad='SAME') 81 | output = tf.nn.max_pool(convolved,[1,2,2,1],[1,2,2,1],padding='SAME') 82 | output = batchnorm(output) 83 | layers.append(output) # 在每一层激活前，进行特征图的保存 84 | 85 | layer_specs = [ 86 | (128, 0.0), # decoder_2: [batch, 4, 4, 256] => [batch, 8, 8, 128] 87 | (64, 0.0), # decoder_3: [batch, 8, 8, 128] => [batch, 16, 16, 64] 88 | (32, 0.0), # decoder_3: [batch, 16, 16, 64] => [batch, 32, 32, 32] 89 | ] 90 | num_encoder_layers = len(layers) 91 | for decoder_layer, (out_channels, dropout) in enumerate(layer_specs): 92 | skip_layer = num_encoder_layers - decoder_layer - 1 93 | with tf.variable_scope("decoder_%d" % (skip_layer + 1)): 94 | if decoder_layer == 0: 95 | # first decoder layer doesn't have skip connections 96 | # since it is directly connected to the skip_layer 97 | inputs = layers[-1] 98 | else: 99 | inputs = tf.concat([layers[-1], layers[skip_layer]], axis=3) #跨层连接 100 | # [batch, in_height, in_width, in_channels] => [batch, in_height*2, in_width*2, out_channels] 101 | inputs = tf.nn.relu(inputs) 102 | output = deconv(inputs, out_channels, 3, pad='SAME') 103 | output = batchnorm(output) 104 | 105 | if dropout > 0.0: 106 | output = tf.nn.dropout(output, keep_prob=1 - dropout) 107 | 108 | layers.append(output) 109 | 110 | # decoder_4: [batch, 32, 32, ngf ] => [batch, 64, 64, generator_outputs_channels] 111 | 112 | with tf.variable_scope("decoder_1"): 113 | inputs = tf.concat([layers[-1], layers[0]], axis=3) 114 | inputs = tf.nn.relu(inputs) 115 | output = deconv(inputs, generator_outputs_channels,3,pad='SAME') 116 | output = tf.tanh(output) # output (-1,1) 117 | layers.append(output) 118 | return layers[-1] 119 | 120 | # 自编码器，nir图像 121 | def nir_encode(generator_inputs, name="nir_encode", reuse=False): 122 | with tf.variable_scope(name,reuse=reuse): 123 | 124 | layer1 = conv(generator_inputs, 32, 3, pad='SAME') # 64*64*32 125 | layer1 = tf.nn.relu(layer1) 126 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 127 | 128 | layer2 = conv(pool1, 64, 3, pad='SAME') # 32*32*64 129 | layer2 = tf.nn.relu(layer2) 130 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 131 | 132 | layer3 = conv(pool2, 128, 3, pad='SAME') # 16*16*128 133 | layer3 = tf.nn.relu(layer3) 134 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 135 | 136 | layer4 = conv(pool3, 256, 3, pad='SAME') # 8*8*256 137 | layer4 = tf.nn.relu(layer4) 138 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 139 | 140 | return pool4, [pool1, pool2, pool3, pool4] 141 | 142 | # 判别器网络，使用特征图相减求差，替代之前的双通道 143 | def features(img, name, reuse=False): 144 | with tf.variable_scope(name,reuse=reuse): 145 | layer1 = conv(img, 32, 3, pad='SAME') # 64*64*32 146 | layer1 = tf.nn.relu(layer1) 147 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 148 | 149 | layer2 = conv(pool1, 64, 3, pad='SAME') # 32*32*64 150 | layer2 = tf.nn.relu(layer2) 151 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 152 | 153 | layer3 = conv(pool2, 128, 3, pad='SAME') # 16*16*128 154 | layer3 = tf.nn.relu(layer3) 155 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 156 | 157 | layer4 = conv(pool3, 256, 3, pad='SAME') # 8*8*256 158 | layer4 = tf.nn.relu(layer4) 159 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 160 | # 4*4*256 161 | return pool4, [layer1, pool1, pool2, pool3, pool4] 162 | 163 | def matching(feature_map_1, feature_map_2, feature_map_3): 164 | with tf.variable_scope('match_layers'): 165 | # 特征图求差值 166 | diff_feature_1 = tf.abs(feature_map_1 - feature_map_2) 167 | diff_feature_2 = tf.abs(feature_map_3 - feature_map_2) 168 | all_feature = tf.concat([diff_feature_1,diff_feature_2], axis=3) # 两个特征连接 4*4*512 169 | 170 | layer5 = conv(all_feature, 512, 3, pad='SAME') 171 | layer5 = tf.nn.relu(layer5) 172 | pool5 = tf.nn.max_pool(layer5,[1,2,2,1],[1,2,2,1],padding='SAME') # 2*2*512 173 | 174 | batch_size = int(pool5.get_shape()[0]) 175 | dense = tf.reshape(pool5, [batch_size,-1]) # reshape as a tensor 2048 176 | dense1 = tf.layers.dense(dense, 1024) 177 | dense1 = tf.nn.relu(dense1) 178 | dense2 = tf.layers.dense(dense1, 256) 179 | dense2 = tf.nn.relu(dense2) 180 | dense3 = tf.layers.dense(dense2, 1) 181 | output = tf.sigmoid(dense3) 182 | return output 183 | 184 | # 判别器网络，p1：gray p2： nir 185 | def create_discriminator(inputs_p1, inputs_p2, feature_map_3, name="discriminator", reuse=False): 186 | with tf.variable_scope(name, reuse=reuse): 187 | feature_map_1, layers_1 = features(inputs_p1, 'feature_layers') # 4*4*256 188 | feature_map_2, layers_2 = features(inputs_p2, 'feature_layers',reuse=True) # 4*4*256 189 | output = matching(feature_map_1, feature_map_2, feature_map_3) 190 | return output 191 | 192 | def gd_model_g2r(inputs_g, inputs_r): 193 | # gray --> nir 194 | out_channels = int(inputs_r.get_shape()[-1]) 195 | g_outputs = create_generator(inputs_g, out_channels) # fake_optical 196 | # create two copies of discriminator, one for real pairs and one for fake pairs 197 | # they share the same underlying variables 198 | with tf.name_scope("real_discriminator"): 199 | # 2x [batch, height, width, channels] => [batch, 1] 200 | feature_map_3, layers_3 = nir_encode(inputs_r) 201 | predict_real = create_discriminator(inputs_g, inputs_r, feature_map_3 ) 202 | with tf.name_scope("fake_discriminator"): 203 | # 2x [batch, height, width, channels] => [batch, 1] 204 | feature_map_3, layers_3 = nir_encode(g_outputs, reuse=True) 205 | predict_fake = create_discriminator(inputs_g, g_outputs, feature_map_3, reuse=True) 206 | 207 | with tf.name_scope("discriminator_loss"): 208 | # minimizing -tf.log will try to get inputs to 1 209 | # predict_real => 1 210 | # predict_fake => 0 211 | discrim_loss = tf.reduce_mean(-(tf.log(predict_real + EPS) + tf.log(1 - predict_fake + EPS))) 212 | 213 | with tf.name_scope("generator_loss"): 214 | # predict_fake => 1 215 | # abs(targets - outputs) => 0 216 | gen_loss_GAN = tf.reduce_mean(-tf.log(predict_fake + EPS)) 217 | gen_loss_L1 = tf.reduce_mean(tf.abs(inputs_r - g_outputs)) 218 | gen_loss = gen_loss_GAN * a.gan_weight + gen_loss_L1 * a.l1_weight 219 | 220 | return gen_loss, discrim_loss, g_outputs, [gen_loss_GAN, gen_loss_L1] 221 | 222 | 223 | 224 | 225 | 226 | -------------------------------------------------------------------------------- /part_3_GAN_transform/model_gd_nir.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on Sat Mar 3 11:03:33 2018 4 | 5 | @author: Administrator 6 | 7 | 自编码器，用于 nir 特征编码的学习 8 | """ 9 | 10 | import tensorflow as tf 11 | import numpy as np 12 | import argparse 13 | 14 | parser = argparse.ArgumentParser() 15 | parser.add_argument("--l1_weight", type=float, default=100.0, help="weight on L1 term for generator gradient") 16 | parser.add_argument("--gan_weight", type=float, default=1.0, help="weight on GAN term for generator gradient") 17 | a = parser.parse_args() 18 | 19 | EPS = 1e-12 20 | 21 | def conv(batch_input, out_channels, filter_size, pad, stride=1): 22 | conv = tf.layers.conv2d(batch_input, out_channels,filter_size,strides=(stride,stride), padding=pad) 23 | return conv 24 | 25 | def deconv(batch_input, out_channels, filter_size, pad): 26 | conv = tf.layers.conv2d_transpose(batch_input, out_channels, filter_size, strides=(2,2), padding=pad ,kernel_initializer=tf.random_normal_initializer(0, 0.02)) 27 | return conv 28 | 29 | def batchnorm(inputs): 30 | normalized = tf.layers.batch_normalization(inputs, axis=-1) 31 | return normalized 32 | 33 | def lrelu(x, a): 34 | with tf.name_scope("lrelu"): 35 | # adding these together creates the leak part and linear part 36 | # then cancels them out by subtracting/adding an absolute value term 37 | # leak: a*x/2 - a*abs(x)/2 38 | # linear: x/2 + abs(x)/2 39 | 40 | # this block looks like it has 2 inputs on the graph unless we do this 41 | x = tf.identity(x) 42 | return (0.5 * (1 + a)) * x + (0.5 * (1 - a)) * tf.abs(x) 43 | 44 | def preprocess(image): 45 | with tf.name_scope("preprocess"): 46 | # [0, 1] => [-1, 1] 47 | return tf.subtract(tf.multiply(image, 2.0),1.0) 48 | # return image * 2.0 - 1.0 49 | 50 | def deprocess(image): 51 | with tf.name_scope("deprocess"): 52 | # [-1, 1] => [0, 1] 53 | return tf.div(tf.add(image , 1.0),2.0) 54 | # return (image + 1.0) / 2.0 55 | 56 | def evaluation(logits, labels): 57 | correct_prediction = tf.equal(logits, labels) 58 | correct_batch = tf.reduce_mean(tf.cast(correct_prediction, tf.int32), 1) 59 | return tf.reduce_sum(tf.cast(correct_batch, tf.float32)), correct_batch 60 | 61 | # 自编码器，nir图像 62 | def nir_encode(generator_inputs, name="nir_encode", reuse=False): 63 | with tf.variable_scope(name,reuse=reuse): 64 | 65 | layer1 = conv(generator_inputs, 32, 3, pad='SAME') # 64*64*32 66 | layer1 = tf.nn.relu(layer1) 67 | pool1 = tf.nn.max_pool(layer1,[1,2,2,1],[1,2,2,1],padding='SAME') 68 | 69 | layer2 = conv(pool1, 64, 3, pad='SAME') # 32*32*64 70 | layer2 = tf.nn.relu(layer2) 71 | pool2 = tf.nn.max_pool(layer2,[1,2,2,1],[1,2,2,1],padding='SAME') 72 | 73 | layer3 = conv(pool2, 128, 3, pad='SAME') # 16*16*128 74 | layer3 = tf.nn.relu(layer3) 75 | pool3 = tf.nn.max_pool(layer3,[1,2,2,1],[1,2,2,1],padding='SAME') 76 | 77 | layer4 = conv(pool3, 256, 3, pad='SAME') # 8*8*256 78 | layer4 = tf.nn.relu(layer4) 79 | pool4 = tf.nn.max_pool(layer4,[1,2,2,1],[1,2,2,1],padding='SAME') 80 | 81 | return pool4, [pool1, pool2, pool3, pool4] 82 | 83 | 84 | def nir_decode( nir_encode_out,nir_encode_layers, generator_outputs_channels, name="nir_decode", reuse=False): 85 | with tf.variable_scope(name,reuse=reuse): 86 | 87 | layer1 = deconv(nir_encode_out, 128, 3, pad='SAME') # 8*8*128 88 | # layer1 = batchnorm(layer1) 89 | layer1_con = tf.concat([layer1, nir_encode_layers[2]], axis=3) 90 | layer1 = tf.nn.relu(layer1_con) 91 | 92 | layer2 = deconv(layer1, 64, 3, pad='SAME') # 16*16*64 93 | # layer2 = batchnorm(layer2) 94 | layer2_con = tf.concat([layer2, nir_encode_layers[1]], axis=3) 95 | layer2 = tf.nn.relu(layer2_con) 96 | 97 | layer3 = deconv(layer2, 32, 3, pad='SAME') # 32*32*32 98 | # layer3 = batchnorm(layer3) 99 | layer3_con = tf.concat([layer3, nir_encode_layers[0]], axis=3) 100 | layer3 = tf.nn.relu(layer3_con) 101 | 102 | layer4 = deconv(layer3, generator_outputs_channels, 3, pad='SAME') # 64*64*1 103 | output = tf.tanh(layer4) 104 | 105 | return output 106 | 107 | def create_generator(generator_inputs, generator_outputs_channels, name="generator", reuse=False): 108 | # with tf.variable_scope(name,reuse=reuse): 109 | nir_encode_out, nir_encode_layers = nir_encode(generator_inputs) 110 | output = nir_decode(nir_encode_out, nir_encode_layers, generator_outputs_channels) 111 | 112 | return output, nir_encode_layers 113 | 114 | # 训练 nir --> nir 的自编码器，主要是学习网络对 nir 的编码能力 115 | # 此处的输入和输出图像都是 nir 图像 116 | def gd_model_r2r(inputs_g): 117 | # nir --> nir 118 | # with tf.variable_scope("sar2opt"): 119 | 120 | out_channels = int(inputs_g.get_shape()[-1]) 121 | g_outputs, gray_layers = create_generator(inputs_g, out_channels) # fake_optical 122 | 123 | with tf.name_scope("generator_loss"): 124 | gen_loss_L1 = 0.5 * tf.reduce_sum(tf.pow(tf.subtract(inputs_g , g_outputs), 2)) 125 | 126 | # map_sub = tf.subtract(inputs_r , g_outputs) 127 | # gen_loss_L1 = 10 * tf.reduce_mean(tf.reduce_mean(map_sub, (1,2,3))) 128 | 129 | return gen_loss_L1, gray_layers 130 | 131 | 132 | 133 | 134 | 135 | -------------------------------------------------------------------------------- /part_3_GAN_transform/read_data.py: -------------------------------------------------------------------------------- 1 | 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Created on Thu Feb 1 13:56:02 2018 5 | 6 | @author: DELL 7 | """ 8 | 9 | from __future__ import absolute_import 10 | from __future__ import division 11 | from __future__ import print_function 12 | 13 | import os.path 14 | import tensorflow as tf 15 | import scipy.misc as misc 16 | import numpy as np 17 | 18 | SAVA_PATH = 'country' 19 | TRAIN_FILE = 'country.tfrecord' 20 | BATCH_SIZE = 100 21 | img_size = 64 22 | 23 | def read_and_decode_train(filename_queue): 24 | # create a reader from file queue 25 | reader = tf.TFRecordReader() 26 | #reader从 TFRecord 读取内容并保存到 serialized_example 中 27 | _, serialized_example = reader.read(filename_queue) 28 | 29 | features = tf.parse_single_example( # 读取 serialized_example 的格式 30 | serialized_example, 31 | features={ 32 | 'label_raw': tf.FixedLenFeature([], tf.string), 33 | 'image_raw': tf.FixedLenFeature([], tf.string) 34 | } 35 | ) # 解析从 serialized_example 读取到的内容 36 | img = tf.decode_raw(features['image_raw'], tf.uint8) 37 | img = tf.reshape(img, [img_size, img_size*2]) 38 | img = tf.cast(img, tf.float32) * (1. / 255.0) #[0,1] 39 | label = tf.decode_raw(features['label_raw'], tf.int32) 40 | label = tf.reshape(tf.cast(label, tf.float32),shape=(1,)) 41 | return img,label 42 | # return tf.cast(resize_image, tf.float32), tf.cast(reshape_label, tf.float32) 43 | 44 | def read_and_decode_test(filename_queue): 45 | # create a reader from file queue 46 | reader = tf.TFRecordReader() 47 | #reader从 TFRecord 读取内容并保存到 serialized_example 中 48 | _, serialized_example = reader.read(filename_queue) 49 | 50 | features = tf.parse_single_example( # 读取 serialized_example 的格式 51 | serialized_example, 52 | features={ 53 | 'label_raw': tf.FixedLenFeature([], tf.string), 54 | 'image_raw': tf.FixedLenFeature([], tf.string) 55 | } 56 | ) # 解析从 serialized_example 读取到的内容 57 | img = tf.decode_raw(features['image_raw'], tf.uint8) 58 | img = tf.reshape(img, [img_size, img_size*2]) 59 | img = tf.cast(img, tf.float32) * (1. / 255.0) #[0,1] 60 | label = tf.decode_raw(features['label_raw'], tf.int32) 61 | label = tf.reshape(tf.cast(label, tf.float32),shape=(1,)) 62 | return img,label 63 | 64 | def batch_inputs(filename_queue, train, batch_size): 65 | #创建文件队列,不限读取的数量 66 | with tf.name_scope('input'): 67 | if train: 68 | image, label = read_and_decode_train(filename_queue) 69 | images, labels = tf.train.shuffle_batch([image, label], 70 | batch_size=batch_size, 71 | num_threads=6, 72 | capacity=2000 + 3 * batch_size, 73 | min_after_dequeue=2000) 74 | else: 75 | image, label = read_and_decode_test(filename_queue) 76 | images, labels = tf.train.batch([image, label], 77 | batch_size=batch_size, 78 | num_threads=6, 79 | capacity=2000 + 3 * batch_size) 80 | return images, labels 81 | 82 | if __name__ == '__main__': 83 | 84 | init = tf.global_variables_initializer() 85 | # filename = os.path.join(SAVA_PATH, TRAIN_FILE) 86 | filename = '/home/ws/文档/wrj/data_all_test/test_data/test_field.tfrecord' 87 | filename_queue = tf.train.string_input_producer([filename],num_epochs=20, shuffle=True) 88 | img_batch, label_batch = batch_inputs(filename_queue, train = True, batch_size = BATCH_SIZE) 89 | with tf.Session() as sess: 90 | # sess.run(init) 91 | sess.run(tf.global_variables_initializer()) 92 | sess.run(tf.local_variables_initializer()) 93 | 94 | coord = tf.train.Coordinator() #创建一个协调器，管理线程 95 | #启动QueueRunner, 此时文件名队列已经进队。 96 | threads=tf.train.start_queue_runners(sess=sess,coord=coord) 97 | try: 98 | for i in range(200): 99 | img, label = sess.run([img_batch, label_batch]) 100 | for j in range(img.shape[0]): 101 | misc.imsave(str(j)+'.png', np.squeeze(img[j])) 102 | except tf.errors.OutOfRangeError: 103 | print('Done reading') 104 | finally: 105 | coord.request_stop() 106 | coord.request_stop() 107 | coord.join(threads) 108 | 109 | 110 | -------------------------------------------------------------------------------- /part_3_GAN_transform/read_data_map.py: -------------------------------------------------------------------------------- 1 | 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Created on Thu Feb 1 13:56:02 2018 5 | 6 | @author: DELL 7 | """ 8 | 9 | from __future__ import absolute_import 10 | from __future__ import division 11 | from __future__ import print_function 12 | 13 | import os.path 14 | import tensorflow as tf 15 | import scipy.misc as misc 16 | import numpy as np 17 | 18 | 19 | SAVA_PATH = 'mapping_gray_data' 20 | TRAIN_FILE = 'map_country.tfrecord' 21 | BATCH_SIZE = 10 22 | img_size = 64 23 | 24 | def random_rot(img): 25 | left,right = tf.split(img,2,axis=1) # 64*64*1 26 | 27 | reshape_image = tf.concat((left,right),axis=2) # 64*64*2 28 | reshape_image = tf.image.random_flip_left_right(reshape_image) 29 | reshape_image = tf.image.random_flip_up_down(reshape_image) 30 | left,right = tf.split(reshape_image,2,axis=2) 31 | 32 | distorted = tf.concat((left,right),axis=1) #[64,128,1] 33 | distorted = tf.reshape(distorted, [img_size, img_size*2]) 34 | return distorted 35 | 36 | def read_and_decode_train(filename_queue): 37 | # create a reader from file queue 38 | reader = tf.TFRecordReader() 39 | #reader从 TFRecord 读取内容并保存到 serialized_example 中 40 | _, serialized_example = reader.read(filename_queue) 41 | 42 | features = tf.parse_single_example( # 读取 serialized_example 的格式 43 | serialized_example, 44 | features={ 45 | 'label_raw': tf.FixedLenFeature([], tf.string), 46 | 'image_raw': tf.FixedLenFeature([], tf.string) 47 | } 48 | ) 49 | # 解析从 serialized_example 读取到的内容 50 | img = tf.decode_raw(features['image_raw'], tf.uint8) 51 | img = tf.reshape(img, [img_size, img_size*2,1]) # 64*128 52 | label = tf.decode_raw(features['label_raw'], tf.int32) 53 | label = tf.reshape(tf.cast(label, tf.float32),shape=(1,)) 54 | 55 | img = random_rot(img) # 数据扩充 56 | img = tf.cast(img, tf.float32) * (1. / 255.0) #[0,1] 57 | 58 | return img,label 59 | 60 | def read_and_decode_test(filename_queue): 61 | # create a reader from file queue 62 | reader = tf.TFRecordReader() 63 | #reader从 TFRecord 读取内容并保存到 serialized_example 中 64 | _, serialized_example = reader.read(filename_queue) 65 | 66 | features = tf.parse_single_example( # 读取 serialized_example 的格式 67 | serialized_example, 68 | features={ 69 | 'label_raw': tf.FixedLenFeature([], tf.string), 70 | 'image_raw': tf.FixedLenFeature([], tf.string) 71 | } 72 | ) # 解析从 serialized_example 读取到的内容 73 | img = tf.decode_raw(features['image_raw'], tf.uint8) 74 | img = tf.reshape(img, [img_size, img_size*2]) 75 | # img_gray = tf.image.rgb_to_grayscale(img) 76 | img = tf.cast(img, tf.float32) * (1. / 255.0) #[0,1] 77 | label = tf.decode_raw(features['label_raw'], tf.int32) 78 | label = tf.reshape(tf.cast(label, tf.float32),shape=(1,)) 79 | return img,label 80 | 81 | def batch_inputs(filename_queue, train, batch_size): 82 | #创建文件队列,不限读取的数量 83 | with tf.name_scope('input'): 84 | if train: 85 | image, label = read_and_decode_train(filename_queue) 86 | images, labels = tf.train.shuffle_batch([image, label], 87 | batch_size=batch_size, 88 | num_threads=6, 89 | capacity=2000 + 3 * batch_size, 90 | min_after_dequeue=2000) 91 | else: 92 | image, label = read_and_decode_test(filename_queue) 93 | images, labels = tf.train.batch([image, label], 94 | batch_size=batch_size, 95 | num_threads=6, 96 | capacity=2000 + 3 * batch_size) 97 | return images, labels 98 | 99 | if __name__ == '__main__': 100 | 101 | init = tf.global_variables_initializer() 102 | # filename = os.path.join(SAVA_PATH, TRAIN_FILE) 103 | filename = 'map_data/map_country.tfrecord' 104 | filename_queue = tf.train.string_input_producer([filename],num_epochs=10 , shuffle=True) 105 | img_batch, label_batch = batch_inputs(filename_queue, train = True, batch_size = BATCH_SIZE) 106 | with tf.Session() as sess: 107 | # sess.run(init) 108 | sess.run(tf.global_variables_initializer()) 109 | sess.run(tf.local_variables_initializer()) 110 | 111 | coord = tf.train.Coordinator() #创建一个协调器，管理线程 112 | #启动QueueRunner, 此时文件名队列已经进队。 113 | threads=tf.train.start_queue_runners(sess=sess,coord=coord) 114 | try: 115 | for i in range(100): 116 | img, label = sess.run([img_batch, label_batch]) 117 | for j in range(img.shape[0]): 118 | misc.imsave(str(j)+'_read_rgb.png', np.squeeze(img[j,:,:64])) 119 | misc.imsave(str(j)+'_read_nir.png', np.squeeze(img[j,:,64:])) 120 | except tf.errors.OutOfRangeError: 121 | print('Done reading') 122 | finally: 123 | coord.request_stop() 124 | coord.request_stop() 125 | coord.join(threads) 126 | 127 | 128 | -------------------------------------------------------------------------------- /part_3_GAN_transform/train_gd_gray2nir.py: -------------------------------------------------------------------------------- 1 | 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Created on 2018.11.18 5 | 6 | @author: DELL 7 | """ 8 | 9 | ''' 10 | 用于生成器的训练，将九个数据集各自训练一个生成器 11 | gray --> nir 12 | ''' 13 | 14 | import numpy as np 15 | import math 16 | import time 17 | import model_gd_diff as model 18 | import tensorflow as tf 19 | import read_data_map as read 20 | import os 21 | from datetime import datetime 22 | import logging 23 | import cv2 24 | 25 | BATCH_SIZE_matching = 100 26 | EPOCH = 81 27 | learning_rate = 2e-4 28 | image_width = 64 29 | image_height = 64 30 | #checkpoint_dir = 'ckpt_gd_o2r' 31 | #checkpoint_dir_g = 'ckpt_g_o2r' 32 | #output_dir = 'out_o2r' 33 | train_dir='summary_gd' 34 | 35 | os.environ['CUDA_VISIBLE_DEVICES']='0' 36 | 37 | def initLogging(logFilename='record_gd_g2r.log'): 38 | """Init for logging 39 | """ 40 | logging.basicConfig( 41 | level = logging.DEBUG, 42 | format='%(asctime)s-%(levelname)s-%(message)s', 43 | datefmt = '%y-%m-%d %H:%M', 44 | filename = logFilename, 45 | filemode = 'w'); 46 | console = logging.StreamHandler() 47 | console.setLevel(logging.INFO) 48 | formatter = logging.Formatter('%(asctime)s-%(levelname)s-%(message)s') 49 | console.setFormatter(formatter) 50 | logging.getLogger('').addHandler(console) 51 | 52 | def combine_images(generated_images): 53 | num = generated_images.shape[0] 54 | width = int(math.sqrt(num)) 55 | height = int(math.ceil(float(num)/width)) 56 | shape = generated_images.shape[1:3] 57 | image = np.zeros((height*shape[0], width*shape[1]), 58 | dtype=generated_images.dtype) 59 | for index, img in enumerate(generated_images): 60 | i = int(index/width) 61 | j = index % width 62 | image[i*shape[0]:(i+1)*shape[0], j*shape[1]:(j+1)*shape[1]] = img[:,:,0] 63 | return image 64 | 65 | def preprocess(image): 66 | # with tf.name_scope("preprocess"): 67 | # [0, 1] => [-1, 1] 68 | return image * 2.0 - 1.0 69 | 70 | def deprocess(image): 71 | # with tf.name_scope("deprocess"): 72 | # [-1, 1] => [0, 1] 73 | return (image + 1.0) / 2.0 74 | 75 | def gd_train(f_name): 76 | checkpoint_dir = 'ckpt_gd_g2r_'+f_name 77 | checkpoint_dir_g = 'ckpt_g_g2r_'+f_name 78 | output_dir = 'out_g2r_'+f_name 79 | checkpoint_file = os.path.join(checkpoint_dir, 'model.ckpt') 80 | checkpoint_file_g = os.path.join(checkpoint_dir_g, 'model.ckpt') 81 | initLogging('record_gd_g2r_'+f_name+'.log') 82 | # if FLAGS.load_model is not None: 83 | # checkpoints_dir = 'checkpoints/' + FLAGS.load_model 84 | # else: 85 | current_time = datetime.now().strftime('%Y%m%d-%H%M') 86 | try: 87 | os.makedirs(checkpoint_dir_g) 88 | os.makedirs(checkpoint_dir) 89 | except os.error: 90 | pass 91 | try: 92 | os.makedirs(output_dir) 93 | except os.error: 94 | pass 95 | # 96 | data1 = np.load('/home/ws/文档/wrj/mapping_data_gray/map_' + f_name + '.npz') 97 | train_matching_y = data1['arr_0'][:,np.newaxis] 98 | numbers_train = train_matching_y.shape[0] #训练集总数 99 | epoch_steps = np.int(numbers_train / BATCH_SIZE_matching) +1 # 一个epoch有多少个steps 100 | 101 | all_loss = np.array([]) 102 | graph = tf.Graph() 103 | with graph.as_default(): 104 | inputs_p1 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width,1], name='inputs_gray') 105 | inputs_p2 = tf.placeholder(tf.float32, [BATCH_SIZE_matching, image_height, image_width,1], name='inputs_nir') 106 | # test_opt = tf.placeholder(tf.float32, [None, image_height, image_width, 1], name='test_opt') 107 | 108 | # 训练 G 109 | # 将输入图像从 （0，1）变换到 （-1，1） 110 | inputs_p1_ = model.preprocess(inputs_p1) 111 | inputs_p2_ = model.preprocess(inputs_p2) 112 | gen_loss, dis_loss, _, _ = model.gd_model_g2r(inputs_p1_, inputs_p2_) 113 | discrim_tvars = [var for var in tf.trainable_variables() if var.name.startswith("discriminator")] 114 | d_train_opt = tf.train.AdamOptimizer(learning_rate).minimize(dis_loss,var_list=discrim_tvars) 115 | gen_tvars = [var for var in tf.trainable_variables() if var.name.startswith("generator")] 116 | g_train_opt = tf.train.AdamOptimizer(learning_rate).minimize(gen_loss,var_list=gen_tvars) 117 | 118 | # with tf.control_dependencies([g_train_opt, d_train_opt]): 119 | # gd_train_opt = tf.no_op(name='optimizers') 120 | 121 | # 122 | filename = '/home/ws/文档/wrj/mapping_data_gray/map_data/map_' + f_name + '.tfrecord' 123 | filename_queue = tf.train.string_input_producer([filename],num_epochs=EPOCH,shuffle=True) 124 | img_batch, label_batch = read.batch_inputs(filename_queue, train = True, batch_size = BATCH_SIZE_matching) 125 | 126 | saver = tf.train.Saver(max_to_keep=20) 127 | saver_g = tf.train.Saver(var_list=gen_tvars, max_to_keep=20) 128 | init = tf.global_variables_initializer() 129 | # 130 | gpu_options = tf.GPUOptions(allow_growth=True) 131 | sess_config = tf.ConfigProto(gpu_options=gpu_options) 132 | 133 | with tf.Session(config=sess_config) as sess: 134 | sess.run(tf.local_variables_initializer()) 135 | sess.run(init) 136 | 137 | try: 138 | coord = tf.train.Coordinator() 139 | threads = tf.train.start_queue_runners(sess=sess, coord=coord) 140 | step = 0 141 | while not coord.should_stop(): 142 | start_time = time.time() 143 | step = step + 1 144 | batch, l_batch = sess.run([img_batch, label_batch]) 145 | x_batch = batch[:,:,:64,np.newaxis] 146 | y_batch = batch[:,:,64:,np.newaxis] 147 | feed_dict = {inputs_p1:x_batch, inputs_p2:y_batch} 148 | _, _, g_loss, d_loss = sess.run([d_train_opt, g_train_opt, gen_loss, dis_loss], feed_dict = feed_dict) 149 | 150 | if step % 10 ==0: 151 | loss_write = np.array([[step, g_loss, d_loss]]) 152 | if step ==10: 153 | all_loss = loss_write 154 | else: 155 | all_loss = np.concatenate((all_loss, loss_write)) 156 | 157 | if step % 100 == 0: 158 | duration = time.time() - start_time 159 | logging.info('>> Step %d run_train: g_loss = %.2f d_loss = %.2f (%.3f sec)' 160 | % (step, g_loss, d_loss, duration)) 161 | 162 | if (step % epoch_steps == 0) and ((step / epoch_steps)%10 == 0): 163 | current_epoch = int(step / epoch_steps) 164 | 165 | if current_epoch >= 50: 166 | logging.info('>> %s Saving in %s' % (datetime.now(), checkpoint_dir)) 167 | saver.save(sess, checkpoint_file, global_step=current_epoch) 168 | saver_g.save(sess, checkpoint_file_g, global_step=current_epoch) 169 | mc_test_all(current_epoch) 170 | # if current_epoch >= 50: 171 | # mc_test_all(current_epoch) 172 | 173 | except KeyboardInterrupt: 174 | print('INTERRUPTED') 175 | coord.request_stop() 176 | except Exception as e: 177 | coord.request_stop(e) 178 | 179 | finally: 180 | saver.save(sess, checkpoint_file, global_step=step) 181 | saver_g.save(sess, checkpoint_file_g, global_step=step) 182 | np.save(os.path.join(checkpoint_dir,'ckpt_map_gd_'+f_name), all_loss) 183 | np.save(os.path.join(checkpoint_dir_g,'ckpt_map_gd_'+f_name), all_loss) 184 | print('Model saved in file :%s'%checkpoint_dir) 185 | # When done, ask the threads to stop. 186 | coord.request_stop() 187 | coord.join(threads) 188 | 189 | ''' 190 | test the network 191 | ''' 192 | def mc_test(file_name, number): 193 | 194 | # data = np.load('/home/ws/文档/wrj/mapping_data_all/map_rgb_'+file_name+'.npz') 195 | # train_matching_y = data['arr_0'][:] 196 | # numbers_train = train_matching_y.shape[0] #训练集总数 197 | BATCH_SIZE_map = 40 198 | 199 | graph = tf.Graph() 200 | with graph.as_default(): 201 | inputs_p1 = tf.placeholder(tf.float32, [BATCH_SIZE_map, image_height, image_width, 1], name='inputs_gray') 202 | inputs_p2 = tf.placeholder(tf.float32, [BATCH_SIZE_map, image_height, image_width, 1], name='inputs_nir') 203 | 204 | gen_loss, dis_loss, _, _ = model.gd_model_g2r(inputs_p1, inputs_p2) 205 | inputs_p1_ = model.preprocess(inputs_p1) 206 | gen = model.create_generator(inputs_p1_, 1, reuse=True) 207 | gen = model.deprocess(gen) 208 | 209 | filename = '/home/ws/文档/wrj/data_all/country/'+file_name+'.tfrecord' 210 | filename_queue = tf.train.string_input_producer([filename],num_epochs=1,shuffle=False) 211 | img_batch, label_batch = read.batch_inputs(filename_queue, train = False, batch_size = BATCH_SIZE_map) 212 | 213 | saver = tf.train.Saver() 214 | with tf.Session() as sess: 215 | sess.run(tf.local_variables_initializer()) 216 | # saver.restore(sess, tf.train.latest_checkpoint('ckpt_gd_g2r_all')) 217 | saver.restore(sess, 'ckpt_gd_g2r_all/model.ckpt-'+ str(number)) 218 | # num = 0 219 | 220 | try: 221 | coord = tf.train.Coordinator() 222 | threads = tf.train.start_queue_runners(sess=sess, coord=coord) 223 | step_test = 0 224 | while not coord.should_stop(): 225 | if step_test < 1: 226 | 227 | step_test = step_test + 1 228 | batch, l_batch = sess.run([img_batch, label_batch]) 229 | x_batch = batch[:,:,:64,np.newaxis] 230 | y_batch = batch[:,:,64:,np.newaxis] 231 | feed_dict = {inputs_p1:x_batch} 232 | gen_out = sess.run(gen, feed_dict = feed_dict) 233 | 234 | gen_out_dir = 'out_g2r_all/epoch_' + str(number) 235 | try: 236 | os.makedirs(gen_out_dir) 237 | except os.error: 238 | pass 239 | show_images = np.concatenate((y_batch, gen_out), axis=1) 240 | show_images = show_images*255 241 | for i in range(BATCH_SIZE_map): 242 | cv2.imwrite(gen_out_dir+'/{}.png'.format(file_name+'_'+str(i+1)+"_gray"), np.squeeze(x_batch[i,:,:,:])*255) 243 | cv2.imwrite(gen_out_dir+'/{}.png'.format(file_name+'_'+str(i+1)+"_nir"), np.squeeze(show_images[i,:64,:,:])) 244 | cv2.imwrite(gen_out_dir+'/{}.png'.format(file_name+'_'+str(i+1)+"_fnir"), np.squeeze(show_images[i,64:,:,:])) 245 | else: 246 | break 247 | except KeyboardInterrupt: 248 | print('INTERRUPTED') 249 | coord.request_stop() 250 | except Exception as e: 251 | coord.request_stop(e) 252 | finally: 253 | coord.request_stop() 254 | coord.join(threads) 255 | 256 | ''' 257 | test all datasets 258 | ''' 259 | def mc_test_all(number): 260 | 261 | test_data = ['country', 'field', 'forest','indoor','mountain','oldbuilding','street','urban','water'] 262 | for i in range(9): 263 | mc_test(test_data[i], number) 264 | 265 | def mc_train_all(): 266 | # 'country', 267 | train_data = ['country', 'field', 'forest','indoor','mountain','oldbuilding','street','urban','water'] 268 | for i in range(5): 269 | gd_train(train_data[i]) 270 | 271 | if __name__ == '__main__': 272 | # mc_train_all() 273 | 274 | # gd_train('all') 275 | 276 | # mc_test('country', 25) 277 | for i in [50,60,70,80]: 278 | mc_test_all(i) 279 | 280 | --------------------------------------------------------------------------------