├── README.md ├── data_process ├── Img_2_hd5.py └── readme.md ├── evaluation ├── accuracy │ ├── PCA.m │ ├── ROCcurve.m │ ├── lfw_eval.m │ ├── pair.label │ └── readme.md ├── feature_extraction │ ├── LCNN29.py │ ├── eva.py │ ├── layer.py │ ├── model.py │ └── readme.md └── features │ └── readme.md └── train ├── LCNN29.py ├── README.md ├── hd5_list_train.txt ├── hd5_list_val.txt ├── hd5_reader.py ├── layer.py └── model.py /README.md: -------------------------------------------------------------------------------- 1 | # Light CNN for Deep Face Recognition, in Tensorflow 2 | A Tensorflow implementation of [A Light CNN for Deep Face Representation with Noisy Labels](https://arxiv.org/abs/1511.02683) from the paper by Xiang Wu 3 | 4 | ## Updates 5 | - Jan 9, 2018 6 | - Add cleaned training list 10K and 70K. 7 | - Sep 20, 2017 8 | - Add model and evaluted code. 9 | - Add training code. 10 | - Sep 19, 2017 11 | - The repository was built. 12 | 13 | 14 | ## Datasets 15 | - Training data 16 | - Download face dataset [MS-Celeb-1M (Aligned)](http://www.msceleb.org/download/aligned). 17 | - All face images are RGB images and resize to **122x144** 18 | - Download MS-Celeb-1M cleaned image_list [10K](https://1drv.ms/t/s!AleP5K29t5x7ge87YS8Ue92h8JDDMw), [70K](https://1drv.ms/t/s!AleP5K29t5x7gfEu_3My1D3lgDhLlQ) 19 | - Testing data 20 | - Download aligned LFW (122*144) [images](https://1drv.ms/u/s!AleP5K29t5x7ge88rngfpitnvpkZbw) and [list](https://1drv.ms/t/s!AleP5K29t5x7ge9DV6jfHo392ONwCA) 21 | 22 | ## Training 23 | - Add 24 | 25 | ## Evaluation 26 | - Download [LCNN-29 model](https://1drv.ms/f/s!AleP5K29t5x7ge89GqB3Ue_Pe5rN3A), this model's performance on LFW:98.2% (100%-EER) 27 | - Download [LFW features](https://1drv.ms/u/s!AleP5K29t5x7ge9ElofW_tDzxCq5sw) 28 | 29 | ## Performance 30 | The Light CNN performance on lfw 6,000 pairs. 31 | 32 | | Model | traing data | method | Acc |100% - EER | TPR@FAR=1% | TPR@FAR=0.1%| TPR@FAR=0| 33 | | :------- | :----: | :----: | :----:| :----: | :---: | :---: |:---: | 34 | | LightCNN-29 (Wu Xiang)| 70K/- |Softmax| - |99.40% | 99.43% | 98.67% | 95.70% | 35 | | LightCNN-29 (Tensorflow)|10K/- |Softmax|98.36% |98.2% | 97.73% | 92.26% | 60.53% | 36 | | LightCNN-29 (Tensorflow)|10K/- |Softmax+L2+PCA|98.76% |98.66% |   98.36%   |   97% |   79.33% | 37 | | LightCNN-29 (Tensorflow)|10K/- |Softmax+L2+PCA+[b]|98.95% |98.8% |   98.76%   |   97.16% |   83.36% | 38 | | LightCNN-29 (Tensorflow)|10K/- |Softmax_enforce+L2+PCA+[b]|99.01% |98.96% |   98.96%   |   95.83% |   90.23% | 39 | 40 | | Model | traing data | method | Acc |100% - EER | TPR@FAR=1% | TPR@FAR=0.1%| TPR@FAR=0| 41 | | :------- | :----: | :----: | :----:| :----: | :---: | :---: |:---: | 42 | | LightCNN-29 (Wu Xiang)| 70K/- |Softmax| - |99.40% | 99.43% | 98.67% | 95.70% | 43 | | LightCNN-29 (Tensorflow)|70K/- |Softmax_enforce+L2+PCA|99.18% |98.9% |   98.86%   |   97.9% |   94.03% | 44 | | LightCNN-29 (Tensorflow)|70K/- |Softmax_enforce+L2+PCA+[a]|99.48% |99.43% |   99.56%   |   98.26% |   94.53% | 45 | 46 | 47 | Some improved solutions: 48 | - [a] It can be further improved by manaully aligned these images which are mis-algined in LFW 49 | - [b] It can be further improved by doing mutiple-crop, e.g. 25 crops for per image 50 | - [c] It can be further improved by ensemble different models 51 | - [d] It can be further improved by adding metric learning method for similarity caculation 52 | 53 | ## Referencs 54 | - [Original Light CNN implementation (caffe)](https://github.com/AlfredXiangWu/face_verification_experiment). 55 | -------------------------------------------------------------------------------- /data_process/Img_2_hd5.py: -------------------------------------------------------------------------------- 1 | import cv2 as cv 2 | import numpy as np 3 | import h5py 4 | import os 5 | import random 6 | 7 | f = open('imglist_10k.txt') 8 | A = [] 9 | for line in f: 10 | line = line.strip() 11 | A.append(line) 12 | f.close() 13 | 14 | random.shuffle(A) 15 | IMG = [] 16 | LAB = [] 17 | num = 0 18 | for i in range(len(A)): 19 | j = i+1 20 | line = A[i] 21 | # print (line) 22 | # input() 23 | path = line.split(' ')[0] 24 | label = int(line.split(' ')[-1]) 25 | 26 | img = cv.imread(path) 27 | img = cv.resize(img,(122,144)) 28 | M2 = np.float32([[1,0,11],[0,1,0]]) 29 | img = cv.warpAffine(img,M2,(144,144)) 30 | # cv.imshow('img', img) 31 | # cv.waitKey(0) 32 | 33 | IMG.append(img) 34 | LAB.append(label) 35 | 36 | if (j%5000 == 0 and i > 0): 37 | print ('convet image to hd5 file...') 38 | IMG = np.array(IMG) 39 | LAB =np.array(LAB) 40 | print (IMG.shape) 41 | print (IMG.shape) 42 | 43 | h5_path = os.path.join('./hd5/', 'train_'+str(num)+'.h5') 44 | # with h5py.File(h5_path, 'w') as f: 45 | # f['data'] = IMG 46 | # f['label'] = LAB 47 | f = h5py.File(h5_path, 'w') 48 | f.create_dataset('data', data=IMG, compression='gzip', compression_opts=4) 49 | f.create_dataset('label', data=LAB, compression='gzip', compression_opts=4) 50 | f.close() 51 | 52 | num += 1 53 | IMG = [] 54 | LAB = [] 55 | 56 | if (i%100 == 0): 57 | print (str(i) + '\t' + str(num)) 58 | 59 | if (len(IMG) != 0): 60 | IMG = np.array(IMG) 61 | LAB =np.array(LAB) 62 | 63 | h5_path = os.path.join('./hd5/', 'train_'+str(num)+'.h5') 64 | f = h5py.File(h5_path, 'w') 65 | f.create_dataset('data', data=IMG, compression='gzip', compression_opts=4) 66 | f.create_dataset('label', data=LAB, compression='gzip', compression_opts=4) 67 | f.close() 68 | -------------------------------------------------------------------------------- /data_process/readme.md: -------------------------------------------------------------------------------- 1 | ## Img_2_hd5.py 2 | - convert RGB images to hd5 files 3 | - RGB image size: **122*144**，pad 11 piexl for right and left size and normlize to **144*144** 4 | - Input: 10K image list 5 | - Output: hd5 files 6 | -------------------------------------------------------------------------------- /evaluation/accuracy/PCA.m: -------------------------------------------------------------------------------- 1 | function [eigvector, eigvalue, elapse,sampleMean] = PCA(data, ReducedDim) 2 | %PCA Principal Component Analysis 3 | % 4 | % Usage: 5 | % [eigvector, eigvalue] = PCA(data, ReducedDim) 6 | % [eigvector, eigvalue] = PCA(data) 7 | % 8 | % Input: 9 | % data - Data matrix. Each row vector of fea is a data point. 10 | % 11 | % ReducedDim - The dimensionality of the reduced subspace. If 0, 12 | % all the dimensions will be kept. 13 | % Default is 0. 14 | % 15 | % Output: 16 | % eigvector - Each column is an embedding function, for a new 17 | % data point (row vector) x, y = x*eigvector 18 | % will be the embedding result of x. 19 | % eigvalue - The sorted eigvalue of PCA eigen-problem. 20 | % 21 | % Examples: 22 | % fea = rand(7,10); 23 | % [eigvector,eigvalue] = PCA(fea,4); 24 | % Y = fea*eigvector; 25 | % 26 | % 27 | % version 2.1 --June/2007 28 | % version 2.0 --May/2007 29 | % version 1.1 --Feb/2006 30 | % version 1.0 --April/2004 31 | % 32 | % Written by Deng Cai (dengcai2 AT cs.uiuc.edu) 33 | % 34 | 35 | 36 | if (~exist('ReducedDim','var')) 37 | ReducedDim = 0; 38 | end 39 | 40 | [nSmp,nFea] = size(data); 41 | if (ReducedDim > nFea) | (ReducedDim <=0) 42 | ReducedDim = nFea; 43 | end 44 | 45 | tmp_T = cputime; 46 | 47 | 48 | if issparse(data) 49 | data = full(data); 50 | end 51 | sampleMean = mean(data,1); 52 | data = (data - repmat(sampleMean,nSmp,1)); 53 | 54 | 55 | 56 | if nFea/nSmp > 1.0713 57 | % This is an efficient method which computes the eigvectors of 58 | % of A*A^T (instead of A^T*A) first, and then convert them back to 59 | % the eigenvectors of A^T*A. 60 | ddata = data*data'; 61 | ddata = max(ddata, ddata'); 62 | 63 | dimMatrix = size(ddata,2); 64 | if dimMatrix > 1000 & ReducedDim < dimMatrix/10 % using eigs to speed up! 65 | option = struct('disp',0); 66 | [eigvector, eigvalue] = eigs(ddata,ReducedDim,'la',option); 67 | eigvalue = diag(eigvalue); 68 | else 69 | [eigvector, eigvalue] = eig(ddata); 70 | eigvalue = diag(eigvalue); 71 | 72 | [junk, index] = sort(-eigvalue); 73 | eigvalue = eigvalue(index); 74 | eigvector = eigvector(:, index); 75 | end 76 | 77 | clear ddata; 78 | 79 | maxEigValue = max(abs(eigvalue)); 80 | eigIdx = find(abs(eigvalue)/maxEigValue < 1e-12); 81 | eigvalue (eigIdx) = []; 82 | eigvector (:,eigIdx) = []; 83 | eigvector = data'*eigvector; % Eigenvectors of A^T*A 84 | eigvector = eigvector*diag(1./(sum(eigvector.^2).^0.5)); % Normalization 85 | else 86 | ddata = data'*data; 87 | ddata = max(ddata, ddata'); 88 | 89 | dimMatrix = size(ddata,2); 90 | if dimMatrix > 1000 & ReducedDim < dimMatrix/10 % using eigs to speed up! 91 | option = struct('disp',0); 92 | [eigvector, eigvalue] = eigs(ddata,eigvector_n,'la',option); 93 | eigvalue = diag(eigvalue); 94 | else 95 | [eigvector, eigvalue] = eig(ddata); 96 | eigvalue = diag(eigvalue); 97 | 98 | [junk, index] = sort(-eigvalue); 99 | eigvalue = eigvalue(index); 100 | eigvector = eigvector(:, index); 101 | end 102 | 103 | clear ddata; 104 | maxEigValue = max(abs(eigvalue)); 105 | eigIdx = find(abs(eigvalue)/maxEigValue < 1e-12); 106 | eigvalue (eigIdx) = []; 107 | eigvector (:,eigIdx) = []; 108 | end 109 | 110 | 111 | if ReducedDim < length(eigvalue) 112 | eigvalue = eigvalue(1:ReducedDim); 113 | eigvector = eigvector(:, 1:ReducedDim); 114 | end 115 | 116 | elapse = cputime - tmp_T; 117 | -------------------------------------------------------------------------------- /evaluation/accuracy/ROCcurve.m: -------------------------------------------------------------------------------- 1 | function [faRate, hitRate, AUC, EER, Acc] = ROCcurve(score, trueLabel, doPlot) 2 | %% Plot a receiver operating curve 3 | % function [faRate, hitRate, AUC, EER] = plotROCcurve(score, trueLabel, doPlot) 4 | % 5 | % score(i) = confidence in i'th detection (bigger means more confident) 6 | % trueLabel(i) = 0 if background or 1 if target 7 | % doPlot - optional (default 0) 8 | % 9 | % faRate(t) = false alarm rate at t'th threshold 10 | % hitRate(t) = detection rate at t'th threshold 11 | % AUC = area under curve 12 | % EER = equal error rate 13 | % Acc = Accuracy 14 | 15 | if nargin < 3, doPlot = 0; end 16 | 17 | class1 = find(trueLabel==1); 18 | class0 = find(trueLabel==0); 19 | 20 | thresh = sort(score); 21 | Nthresh = length(thresh); 22 | hitRate = zeros(1, Nthresh); 23 | faRate = zeros(1, Nthresh); 24 | rec = zeros(1, Nthresh); 25 | for thi=1:length(thresh) 26 | th = thresh(thi); 27 | % hit rate = TP/P 28 | hitRate(thi) = sum(score(class1) >= th) / length(class1); 29 | % fa rate = FP/N 30 | faRate(thi) = sum(score(class0) >= th) / length(class0); 31 | rec(thi) = (sum(score(class1) >= th) + sum(score(class0) < th)) / Nthresh; 32 | end 33 | 34 | % area under curve 35 | AUC = sum(abs(faRate(2:end) - faRate(1:end-1)) .* hitRate(2:end)); 36 | 37 | % equal error rate 38 | i1 = find(hitRate >= (1-faRate), 1, 'last' ) ; 39 | i2 = find((1-faRate) >= hitRate, 1, 'last' ) ; 40 | EER = 1 - max(1-faRate(i1), hitRate(i2)) ; 41 | 42 | % Accuracy 43 | Acc = max(rec); 44 | 45 | if ~doPlot, return; end 46 | 47 | plot(faRate, hitRate, '-'); 48 | 49 | 50 | xlabel('False Positiucve Rate') 51 | ylabel('True Positive Rate') 52 | grid on 53 | title(sprintf('AUC = %5.4f, EER = %5.4f', AUC, EER)) 54 | 55 | end -------------------------------------------------------------------------------- /evaluation/accuracy/lfw_eval.m: -------------------------------------------------------------------------------- 1 | clc; 2 | clear all; 3 | close all; 4 | 5 | % load feature you extracted 6 | % load('../CNNCaffeExtraction/lfwfeatures.mat'); 7 | 8 | % or the features extracted by myself 9 | % model downloaded from "https://github.com/ydwen/caffe-face" 10 | % load('features/lfwfeatures.mat'); 11 | 12 | % or some other features extracted by myself 13 | % 99.53% performance on LFW after doing a PCA on LFW 13233 faces 14 | load('LFW_feas.mat'); 15 | 16 | %feature = cell2mat(lfwfeatures); 17 | feature = data'; 18 | imglist = label; 19 | 20 | % load pair list and label 21 | % generated from original lfw view 2 file. 22 | fea_list = 'pair.label'; 23 | [label img1 img2]= textread(fea_list,'%d %s %s'); 24 | 25 | % PCA 26 | %[eigvec, ~, ~, sampleMean] = PCA(feature'); 27 | %feature = ( bsxfun(@minus, feature', sampleMean)* eigvec )'; 28 | 29 | % generate scores 30 | for i = 1:size(label,1) 31 | % find feature 1 32 | index1 = find(strcmp(imglist, img1{i}) == 1); 33 | fea1 = feature(:,index1); 34 | % find feature 2 35 | index2 = find(strcmp(imglist, img2{i}) == 1); 36 | fea2 = feature(:,index2); 37 | 38 | % cosine distance 39 | cos(i) = (fea1' * fea2)/(norm(fea1) * norm(fea2)); 40 | end 41 | 42 | % ROC and accuracy 43 | [fpr, tpr, auc, eer, acc] = ROCcurve(cos, label); 44 | tmp=sprintf('ACC: %f \nEER: %f \nAUC: %f',acc,eer,auc); 45 | disp(tmp); 46 | 47 | plot(fpr, tpr); 48 | axis([0,0.05,0.95,1]); 49 | legend('cos'); 50 | xlabel('False Positive Rate'); 51 | ylabel('True Positive Rate'); 52 | hold on; 53 | 54 | -------------------------------------------------------------------------------- /evaluation/accuracy/readme.md: -------------------------------------------------------------------------------- 1 | ## Steps 2 | - run lfw_eva.m to get the accuracy 3 | -------------------------------------------------------------------------------- /evaluation/feature_extraction/LCNN29.py: -------------------------------------------------------------------------------- 1 | import os 2 | os.environ["CUDA_VISIBLE_DEVICES"]="0" 3 | import tensorflow as tf 4 | import model as M 5 | import numpy as np 6 | 7 | def LCNN29(): 8 | with tf.name_scope('img_holder'): 9 | img_holder = tf.placeholder(tf.float32, [None, 128, 128, 3]) 10 | 11 | mod = M.Model(img_holder, [None, 128, 128, 3]) 12 | 13 | mod.conv_layer(5, 96, activation=1) 14 | mod.maxpooling_layer(2, 2) #pool1 15 | 16 | # 17 | a = mod.get_current_layer() 18 | mod.conv_layer(3, 96, activation=1) 19 | mod.conv_layer(3, 96, activation=1) 20 | # print (mod.get_shape()) 21 | 22 | mod.sum(a) 23 | mod.conv_layer(1, 96, activation=1) 24 | mod.conv_layer(3, 192, activation=1) 25 | mod.maxpooling_layer(2, 2) #pool2 26 | 27 | # 28 | b = mod.get_current_layer() 29 | mod.conv_layer(3, 96*2, activation=1) 30 | mod.conv_layer(3, 96*2, activation=1) 31 | 32 | mod.sum(b) 33 | mod.conv_layer(1, 96, activation=1) 34 | mod.conv_layer(3, 384, activation=1) 35 | mod.maxpooling_layer(2, 2) #pool3 36 | 37 | # 38 | c = mod.get_current_layer() 39 | mod.conv_layer(3, 192*2, activation=1) 40 | mod.conv_layer(3, 192*2, activation=1) 41 | 42 | mod.sum(c) 43 | mod.conv_layer(1, 384, activation=1) 44 | mod.conv_layer(3, 256, activation=1) 45 | 46 | # 47 | d = mod.get_current_layer() 48 | mod.conv_layer(3, 128*2, activation=1) 49 | mod.conv_layer(3, 128*2, activation=1) 50 | 51 | mod.sum(d) 52 | mod.conv_layer(1, 256, activation=1) 53 | mod.conv_layer(3, 256, activation=1) 54 | mod.maxpooling_layer(2, 2) #pool4 55 | 56 | mod.flatten() 57 | mod.fcnn_layer(512) 58 | feature_layer = mod.get_current_layer()[0] 59 | 60 | return feature_layer, img_holder 61 | 62 | with tf.variable_scope('LCNN29'): 63 | feature_layer, img_holder = LCNN29() 64 | 65 | # 66 | sess = tf.Session() 67 | model_path = './model/Epoc_49_Iter_663.cpkt' 68 | M.loadSess(model_path, sess) 69 | 70 | def eval(img): 71 | res = sess.run(feature_layer, feed_dict={img_holder:img}) 72 | return res 73 | 74 | def __exit__(): 75 | sess.close() 76 | -------------------------------------------------------------------------------- /evaluation/feature_extraction/eva.py: -------------------------------------------------------------------------------- 1 | import LCNN29 as LCNN29 2 | import numpy as np 3 | import cv2 4 | import scipy.io as sio 5 | import random 6 | 7 | res = [] 8 | # labs = [] 9 | f = open('lfw_list_part.txt', 'r') 10 | labs = np.empty([13233, 1], dtype=object) 11 | count = 0 12 | for line in f: 13 | name = [] 14 | line = line.strip() 15 | name.append(line.split('\\')[-2] + '/' + line.split('\\')[-1]) 16 | labs[count, 0] = name 17 | 18 | imgs = [] 19 | img = cv2.imread(line, 1) 20 | img = cv2.resize(img,(122,144)) 21 | M2 = np.float32([[1,0,11],[0,1,0]]) 22 | img = cv2.warpAffine(img,M2,(144,144)) 23 | 24 | for i in range(1): 25 | w = 8 26 | h = 8 27 | img2 = img[w:w+128, h:h+128]/255. 28 | img2 = np.float32(img2) 29 | imgs.append(img2) 30 | 31 | imgs = np.array(imgs) 32 | feas = LCNN29.eval(imgs) 33 | res.append(feas) 34 | count += 1 35 | # if count == 10: 36 | # break 37 | if count %10 == 0: 38 | print (count) 39 | res = np.array(res) 40 | res = np.reshape(res, [13233, 512]) 41 | print (res.shape) 42 | print (labs.shape) 43 | sio.savemat('LFW_feas.mat',{'data':res, 'label':labs}) 44 | f.close() 45 | 46 | 47 | -------------------------------------------------------------------------------- /evaluation/feature_extraction/layer.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | import numpy as np 3 | 4 | def weight(shape): 5 | return tf.get_variable('weight', shape, initializer=tf.contrib.layers.xavier_initializer()) 6 | 7 | def bias(shape, value=0.1): 8 | return tf.get_variable('bias', shape, initializer=tf.constant_initializer(value)) 9 | 10 | 11 | def Fcnn(x,insize,outsize,name,activation=None,nobias=False): 12 | with tf.variable_scope(name): 13 | if nobias: 14 | print('No biased fully connected layer is used!') 15 | W = weight([insize,outsize]) 16 | tf.summary.histogram(name+'/weight',W) 17 | if activation==None: 18 | return tf.matmul(x,W) 19 | return activation(tf.matmul(x,W)) 20 | else: 21 | W = weight([insize,outsize]) 22 | b = bias([outsize]) 23 | tf.summary.histogram(name+'/weight',W) 24 | tf.summary.histogram(name+'/bias',b) 25 | if activation==None: 26 | return tf.matmul(x,W)+b 27 | return activation(tf.matmul(x,W)+b) 28 | 29 | def conv2D(x,kernel_size,outchn,name,stride=1,pad='SAME', usebias=True): 30 | print('Conv_bias:',usebias) 31 | # with tf.variable_scope(name): 32 | # if isinstance(size,list): 33 | # kernel = size 34 | # else: 35 | kernel = [kernel_size, kernel_size] 36 | z = tf.layers.conv2d(x, outchn, kernel, strides=(stride, stride), padding=pad,\ 37 | kernel_initializer=tf.contrib.layers.xavier_initializer_conv2d(),\ 38 | use_bias=usebias,\ 39 | bias_initializer=tf.constant_initializer(0.1),name=name) 40 | # print ('z:', z.get_shape()) 41 | return z 42 | 43 | def maxpooling(x,size,stride,name,pad='SAME'): 44 | with tf.variable_scope(name): 45 | return tf.nn.max_pool(x,ksize=[1,size,size,1],strides=[1,stride,stride,1],padding=pad) 46 | 47 | def avgpooling(x,size,stride,name,pad='SAME'): 48 | with tf.variable_scope(name): 49 | return tf.nn.avg_pool(x,ksize=[1,size,size,1],strides=[1,stride,stride,1],padding=pad) 50 | 51 | 52 | def MFM(x,name): 53 | with tf.variable_scope(name): 54 | shape = x.get_shape().as_list() 55 | res = tf.reshape(x,[-1,shape[1],shape[2],2,shape[-1]//2]) 56 | res = tf.reduce_max(res,axis=[3]) 57 | return res 58 | 59 | def MFMfc(x,half,name): 60 | with tf.variable_scope(name): 61 | shape = x.get_shape().as_list() 62 | # print('fcshape:',shape) 63 | res = tf.reduce_max(tf.reshape(x,[-1,2,shape[-1]//2]),reduction_indices=[1]) 64 | return res 65 | 66 | def batch_norm(inp,name,training=True): 67 | print('BN training:',training) 68 | return tf.layers.batch_normalization(inp,training=training,name=name) 69 | 70 | def L2_norm(inp, dim): 71 | print ('L2 normlization...') 72 | return tf.nn.l2_norm(inp, dim) 73 | 74 | def lrelu(x,name,leaky=0.2): 75 | return tf.maximum(x,x*leaky,name=name) 76 | 77 | def relu(inp,name): 78 | return tf.nn.relu(inp,name=name) 79 | 80 | def tanh(inp,name): 81 | return tf.tanh(inp,name=name) 82 | 83 | def elu(inp,name): 84 | return tf.nn.elu(inp,name=name) 85 | 86 | def sparse_softmax_cross_entropy(inp,lab,name): 87 | with tf.name_scope(name): 88 | loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(labels=lab,logits=inp)) 89 | return loss 90 | 91 | def sigmoid(inp,name): 92 | return tf.sigmoid(inp,name=name) 93 | 94 | def resize_nn(inp,size,name): 95 | with tf.name_scope(name): 96 | return tf.image.resize_nearest_neighbor(inp,size=(int(size),int(size))) 97 | 98 | def accuracy(pred,y,name): 99 | with tf.variable_scope(name): 100 | correct = tf.equal(tf.cast(tf.argmax(pred,1),tf.int64),tf.cast(y,tf.int64)) 101 | acc = tf.reduce_mean(tf.cast(correct,tf.float32)) 102 | return acc 103 | 104 | def dropout(inp, keep_prob): 105 | return tf.nn.dropout(inp,keep_prob) 106 | -------------------------------------------------------------------------------- /evaluation/feature_extraction/model.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | import layer as L 3 | 4 | class Model(): 5 | def __init__(self, res, res_size): 6 | self.res = res 7 | self.res_size = res_size 8 | self.layer_num = 0 9 | self.batch_norm = True 10 | 11 | def get_current_layer_res(self): 12 | return self.res 13 | 14 | def get_shape(self): 15 | return self.res_size 16 | 17 | def get_current_layer(self): 18 | return [self.res, list(self.res_size)] 19 | 20 | def activate_layer(self, param): 21 | # res = self.res 22 | with tf.name_scope('activation_' + str(self.layer_num)): 23 | if param == 1: 24 | self.res = L.MFM(self.res, name='mfm_'+str(self.layer_num)) 25 | self.res_size = self.res.get_shape() 26 | else: 27 | self.res = self.res 28 | self.res_size = self.res_size 29 | return [self.res, self.res_size] 30 | 31 | def conv_layer(self, kernel_size, outchn, stride=1, pad='SAME', activation=-1, batch_norm=False, usebias=True): 32 | with tf.name_scope('conv_' + str(self.layer_num)): 33 | inp_size = self.res_size 34 | self.res = L.conv2D(self.res, kernel_size, outchn, 'conv_'+str(self.layer_num), stride=stride, pad=pad, usebias=usebias) 35 | self.res_size = self.res.get_shape() 36 | 37 | if batch_norm: 38 | self.res = batch_norm(self.res, 'batch_norm_'+str(self.layer_num), training=self.batch_norm) 39 | 40 | self.activate_layer(activation) 41 | self.layer_num += 1 42 | 43 | print ('conv_'+str(self.layer_num), kernel_size, inp_size, self.res_size) 44 | return [self.res, self.res_size] 45 | 46 | def maxpooling_layer(self, kernel_size, stride=1, pad='SAME'): 47 | with tf.name_scope('maxpooling_' + str(self.layer_num)): 48 | inp_size = self.res_size 49 | self.res = L.maxpooling(self.res, kernel_size, stride=stride, name='maxpooling_'+str(self.layer_num), pad=pad) 50 | self.res_size = self.res.get_shape() 51 | 52 | print ('maxpooling_'+str(self.layer_num), kernel_size, inp_size, self.res_size) 53 | return [self.res, self.res_size] 54 | 55 | def sum(self, layerin): 56 | layer1_size = self.res_size 57 | layer2_size = layerin[1] 58 | print (layer1_size[2], layer1_size[3]) 59 | print (layer2_size[2], layer2_size[3]) 60 | assert layer1_size[1] == layer2_size[1] 61 | assert layer1_size[2] == layer2_size[2] and layer1_size[3] == layer2_size[3] 62 | 63 | inp_size = self.res_size 64 | with tf.name_scope('sum_' + str(self.layer_num)): 65 | self.res = self.res + layerin[0] 66 | self.res_size = self.res.get_shape() 67 | 68 | print ('sum_'+str(self.layer_num), inp_size, self.res_size) 69 | return [self.res, self.res_size] 70 | 71 | def flatten(self): 72 | inp_size = self.res_size 73 | # print (inp_size[1], inp_size[2], inp_size[3]) 74 | with tf.name_scope('flatten_' + str(self.layer_num)): 75 | self.res = tf.reshape(self.res, [-1, int(inp_size[1])*int(inp_size[2])*int(inp_size[3])]) 76 | self.res_size = self.res.get_shape() 77 | 78 | print ('flatten_'+str(self.layer_num), inp_size, self.res_size) 79 | return [self.res, self.res_size] 80 | 81 | def fcnn_layer(self, outsize, nobias=False): 82 | inp_size = self.res.get_shape()[1] 83 | with tf.name_scope('Fcnn_' + str(self.layer_num)): 84 | self.res = L.Fcnn(self.res, inp_size, outsize, 'Fcnn_'+str(self.layer_num), activation=None, nobias=False) 85 | self.res_size = self.res.get_shape() 86 | 87 | self.layer_num += 1 88 | print ('Fcnn_'+str(self.layer_num), inp_size, self.res_size) 89 | return [self.res, self.res_size] 90 | 91 | def dropout(self, keep_prob): 92 | self.res = L.dropout(self.res, keep_prob) 93 | self.res_size = self.res.get_shape() 94 | 95 | return [self.res, self.res_size] 96 | 97 | def accuracy(self, label): 98 | self.res = L.accuracy(self.res, label, 'accuracy') 99 | return self.res 100 | 101 | 102 | def loadSess(model_path, sess): 103 | saver = tf.train.Saver() 104 | saver.restore(sess, model_path) 105 | return sess 106 | 107 | 108 | 109 | 110 | 111 | -------------------------------------------------------------------------------- /evaluation/feature_extraction/readme.md: -------------------------------------------------------------------------------- 1 | ## Steps 2 | - Download [LCNN-29 model](https://1drv.ms/f/s!AleP5K29t5x7ge89GqB3Ue_Pe5rN3A) 3 | - Download aligned LFW (122*144) [images](https://1drv.ms/u/s!AleP5K29t5x7ge88rngfpitnvpkZbw) and [list](https://1drv.ms/t/s!AleP5K29t5x7ge9DV6jfHo392ONwCA) 4 | - Run eva.py to get features of LFW 5 | -------------------------------------------------------------------------------- /evaluation/features/readme.md: -------------------------------------------------------------------------------- 1 | ## Steps 2 | - Download [lfw features](https://1drv.ms/u/s!AleP5K29t5x7ge9ElofW_tDzxCq5sw) 3 | -------------------------------------------------------------------------------- /train/LCNN29.py: -------------------------------------------------------------------------------- 1 | import os 2 | os.environ["CUDA_VISIBLE_DEVICES"]="2" 3 | import tensorflow as tf 4 | import model as M 5 | import numpy as np 6 | from hd5_reader import hd5_reader 7 | import shutil 8 | 9 | BSIZE = 400 10 | CLASS = 10000 11 | EPOC = 50 12 | 13 | def LCNN29(): 14 | with tf.name_scope('img_holder'): 15 | img_holder = tf.placeholder(tf.float32, [None, 128, 128, 3]) 16 | with tf.name_scope('lab_holder'): 17 | lab_holder = tf.placeholder(tf.int64, [None, CLASS]) 18 | 19 | mod = M.Model(img_holder, [None, 128, 128, 3]) 20 | 21 | mod.conv_layer(5, 96, activation=1) 22 | mod.maxpooling_layer(2, 2) #pool1 23 | 24 | # 25 | a = mod.get_current_layer() 26 | mod.conv_layer(3, 96, activation=1) 27 | mod.conv_layer(3, 96, activation=1) 28 | # print (mod.get_shape()) 29 | 30 | mod.sum(a) 31 | mod.conv_layer(1, 96, activation=1) 32 | mod.conv_layer(3, 192, activation=1) 33 | mod.maxpooling_layer(2, 2) #pool2 34 | 35 | # 36 | b = mod.get_current_layer() 37 | mod.conv_layer(3, 96*2, activation=1) 38 | mod.conv_layer(3, 96*2, activation=1) 39 | 40 | mod.sum(b) 41 | mod.conv_layer(1, 96, activation=1) 42 | mod.conv_layer(3, 384, activation=1) 43 | mod.maxpooling_layer(2, 2) #pool3 44 | 45 | # 46 | c = mod.get_current_layer() 47 | mod.conv_layer(3, 192*2, activation=1) 48 | mod.conv_layer(3, 192*2, activation=1) 49 | 50 | mod.sum(c) 51 | mod.conv_layer(1, 384, activation=1) 52 | mod.conv_layer(3, 256, activation=1) 53 | 54 | # 55 | d = mod.get_current_layer() 56 | mod.conv_layer(3, 128*2, activation=1) 57 | mod.conv_layer(3, 128*2, activation=1) 58 | 59 | mod.sum(d) 60 | mod.conv_layer(1, 256, activation=1) 61 | mod.conv_layer(3, 256, activation=1) 62 | mod.maxpooling_layer(2, 2) #pool4 63 | 64 | mod.flatten() 65 | mod.fcnn_layer(512) 66 | feature_layer = mod.get_current_layer()[0] 67 | 68 | mod.dropout(0.2) 69 | mod.fcnn_layer(CLASS) 70 | class_layer = mod.get_current_layer()[0] 71 | acc = mod.accuracy(tf.argmax(lab_holder,1)) 72 | 73 | with tf.name_scope('loss'): 74 | loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=lab_holder, logits=class_layer)) 75 | 76 | with tf.name_scope('accuracy'): 77 | accuracy = acc 78 | 79 | return loss, accuracy, img_holder, lab_holder 80 | 81 | with tf.variable_scope('LCNN29'): 82 | loss, acc, img_holder, lab_holder = LCNN29() 83 | 84 | with tf.name_scope('Optimizer'): 85 | train = tf.train.AdamOptimizer(0.00001).minimize(loss) 86 | # print (train) 87 | 88 | # 89 | model_path = './model/' 90 | log_path = './log/' 91 | list_train = 'hd5_list_train.txt' 92 | list_val = 'hd5_list_val.txt' 93 | f_log = open('./log/log.txt', 'a+') 94 | with tf.Session() as sess: 95 | 96 | writer = tf.summary.FileWriter(log_path, sess.graph) 97 | sess, epoc, iters = M.loadSess(model_path, sess) 98 | saver = tf.train.Saver() 99 | 100 | reader = hd5_reader(list_train, list_val, BSIZE, BSIZE) 101 | ITERS = reader.train_epoc 102 | count = 0 103 | for j in range(EPOC): 104 | for i in range(ITERS): 105 | x_train, y_train_ = reader.train_nextbatch() 106 | global BSIZE 107 | BSIZE = reader.train_bsize 108 | # print ('BSIZE:', BSIZE) 109 | y_train = np.zeros([BSIZE,CLASS],dtype=np.int64) 110 | for index in range(BSIZE): 111 | y_train[index][y_train_[index]] = 1 112 | 113 | _, ls, ac = sess.run([train, loss, acc], feed_dict={img_holder:x_train, lab_holder:y_train}) 114 | # str1 =' Epoc: ' + str(j) + '\t|Iter: ' + str(i) + '\t|Train_Loss: ' + str(ls) + '\t|Train_Acc: ' + str(ac) 115 | str1 =' Epoc: ' + str(j+epoc) + '\t|Iter: ' + str(i+iters) + '\t|Train_Loss: ' + str(ls) + '\t|Train_Acc: ' + str(ac) 116 | print(str1) 117 | f_log.write(str1 + '\n') 118 | 119 | # # 120 | if count%100 == 0: 121 | x_val, y_val_ = reader.val_nextbatch() 122 | global BSIZE 123 | BSIZE = reader.val_bsize 124 | y_val = np.zeros([BSIZE,CLASS],dtype=np.int64) 125 | for index in range(BSIZE): 126 | y_val[index][y_val_[index]] = 1 127 | 128 | ls_val = 0 129 | acc_val = 0 130 | for n in range(reader.val_data_ITERS): 131 | ls, ac = sess.run([loss, acc], feed_dict={img_holder:x_val, lab_holder:y_val}) 132 | ls_val += ls 133 | acc_val += ac 134 | ls_val = ls_val/reader.val_data_ITERS 135 | acc_val = acc_val/reader.val_data_ITERS 136 | # print('Epoc:',j,' |Iter:',i,' |Val_Loss1:',ls_val,' |Val_Acc:',acc_val) 137 | # str1 =' Epoc: ' + str(j) + '\t|Iter: ' + str(i) + '\t|Val_Loss: ' + str(ls_val) + '\t|Val_Acc: ' + str(acc_val) 138 | str1 =' Epoc: ' + str(j+epoc) + '\t|Iter: ' + str(i+iters) + '\t|Val_Loss: ' + str(ls_val) + '\t|Val_Acc: ' + str(acc_val) 139 | print(str1) 140 | f_log.write(str1 + '\n') 141 | 142 | if count%1 == 1000 and count > 0: 143 | save_path = model_path+'Epoc_'+ str(j+epoc) + '_' + 'Iter_' + str(i+iters) + '.cpkt' 144 | saver.save(sess, save_path) 145 | 146 | save_path2 = save_path + '.meta' 147 | save_path3 = save_path + '.index' 148 | save_path4 = save_path + '.data-00000-of-00001' 149 | save_path5 = model_path + 'checkpoint' 150 | 151 | shutil.copy(save_path2, save_path2.replace('./model/', './backup/')) 152 | shutil.copy(save_path3, save_path3.replace('./model/', './backup/')) 153 | shutil.copy(save_path4, save_path4.replace('./model/', './backup/')) 154 | shutil.copy(save_path5, save_path5.replace('./model/', './backup/')) 155 | 156 | count += 1 157 | writer.close() 158 | # writer = tf.summary.FileWrite(log_path, sess.graph) 159 | # merge = tf.summary.merge_all() 160 | -------------------------------------------------------------------------------- /train/README.md: -------------------------------------------------------------------------------- 1 | # 2 | -------------------------------------------------------------------------------- /train/hd5_list_train.txt: -------------------------------------------------------------------------------- 1 | /home/psl/XuYan/LCNN/hd5/train_0.h5 2 | /home/psl/XuYan/LCNN/hd5/train_1.h5 3 | /home/psl/XuYan/LCNN/hd5/train_10.h5 4 | /home/psl/XuYan/LCNN/hd5/train_100.h5 5 | /home/psl/XuYan/LCNN/hd5/train_101.h5 6 | /home/psl/XuYan/LCNN/hd5/train_102.h5 7 | /home/psl/XuYan/LCNN/hd5/train_103.h5 8 | /home/psl/XuYan/LCNN/hd5/train_104.h5 9 | /home/psl/XuYan/LCNN/hd5/train_105.h5 10 | /home/psl/XuYan/LCNN/hd5/train_106.h5 11 | /home/psl/XuYan/LCNN/hd5/train_107.h5 12 | /home/psl/XuYan/LCNN/hd5/train_108.h5 13 | /home/psl/XuYan/LCNN/hd5/train_109.h5 14 | /home/psl/XuYan/LCNN/hd5/train_11.h5 15 | /home/psl/XuYan/LCNN/hd5/train_110.h5 16 | /home/psl/XuYan/LCNN/hd5/train_111.h5 17 | /home/psl/XuYan/LCNN/hd5/train_112.h5 18 | /home/psl/XuYan/LCNN/hd5/train_113.h5 19 | /home/psl/XuYan/LCNN/hd5/train_114.h5 20 | /home/psl/XuYan/LCNN/hd5/train_115.h5 21 | /home/psl/XuYan/LCNN/hd5/train_116.h5 22 | /home/psl/XuYan/LCNN/hd5/train_117.h5 23 | /home/psl/XuYan/LCNN/hd5/train_118.h5 24 | /home/psl/XuYan/LCNN/hd5/train_119.h5 25 | /home/psl/XuYan/LCNN/hd5/train_12.h5 26 | /home/psl/XuYan/LCNN/hd5/train_120.h5 27 | /home/psl/XuYan/LCNN/hd5/train_121.h5 28 | /home/psl/XuYan/LCNN/hd5/train_122.h5 29 | /home/psl/XuYan/LCNN/hd5/train_123.h5 30 | /home/psl/XuYan/LCNN/hd5/train_124.h5 31 | /home/psl/XuYan/LCNN/hd5/train_83.h5 32 | /home/psl/XuYan/LCNN/hd5/train_84.h5 33 | /home/psl/XuYan/LCNN/hd5/train_85.h5 34 | /home/psl/XuYan/LCNN/hd5/train_86.h5 35 | /home/psl/XuYan/LCNN/hd5/train_87.h5 36 | /home/psl/XuYan/LCNN/hd5/train_88.h5 37 | /home/psl/XuYan/LCNN/hd5/train_89.h5 38 | /home/psl/XuYan/LCNN/hd5/train_9.h5 39 | /home/psl/XuYan/LCNN/hd5/train_90.h5 40 | /home/psl/XuYan/LCNN/hd5/train_91.h5 41 | /home/psl/XuYan/LCNN/hd5/train_92.h5 42 | /home/psl/XuYan/LCNN/hd5/train_93.h5 43 | /home/psl/XuYan/LCNN/hd5/train_94.h5 44 | /home/psl/XuYan/LCNN/hd5/train_95.h5 45 | /home/psl/XuYan/LCNN/hd5/train_96.h5 46 | /home/psl/XuYan/LCNN/hd5/train_97.h5 47 | /home/psl/XuYan/LCNN/hd5/train_98.h5 48 | /home/psl/XuYan/LCNN/hd5/train_99.h5 49 | /home/psl/XuYan/LCNN/hd5/train_66.h5 50 | /home/psl/XuYan/LCNN/hd5/train_67.h5 51 | /home/psl/XuYan/LCNN/hd5/train_68.h5 52 | /home/psl/XuYan/LCNN/hd5/train_69.h5 53 | /home/psl/XuYan/LCNN/hd5/train_7.h5 54 | /home/psl/XuYan/LCNN/hd5/train_70.h5 55 | /home/psl/XuYan/LCNN/hd5/train_71.h5 56 | /home/psl/XuYan/LCNN/hd5/train_72.h5 57 | /home/psl/XuYan/LCNN/hd5/train_73.h5 58 | /home/psl/XuYan/LCNN/hd5/train_74.h5 59 | /home/psl/XuYan/LCNN/hd5/train_75.h5 60 | /home/psl/XuYan/LCNN/hd5/train_76.h5 61 | /home/psl/XuYan/LCNN/hd5/train_77.h5 62 | /home/psl/XuYan/LCNN/hd5/train_78.h5 63 | /home/psl/XuYan/LCNN/hd5/train_79.h5 64 | /home/psl/XuYan/LCNN/hd5/train_8.h5 65 | /home/psl/XuYan/LCNN/hd5/train_80.h5 66 | /home/psl/XuYan/LCNN/hd5/train_81.h5 67 | /home/psl/XuYan/LCNN/hd5/train_49.h5 68 | /home/psl/XuYan/LCNN/hd5/train_5.h5 69 | /home/psl/XuYan/LCNN/hd5/train_50.h5 70 | /home/psl/XuYan/LCNN/hd5/train_51.h5 71 | /home/psl/XuYan/LCNN/hd5/train_52.h5 72 | /home/psl/XuYan/LCNN/hd5/train_53.h5 73 | /home/psl/XuYan/LCNN/hd5/train_54.h5 74 | /home/psl/XuYan/LCNN/hd5/train_55.h5 75 | /home/psl/XuYan/LCNN/hd5/train_56.h5 76 | /home/psl/XuYan/LCNN/hd5/train_57.h5 77 | /home/psl/XuYan/LCNN/hd5/train_58.h5 78 | /home/psl/XuYan/LCNN/hd5/train_59.h5 79 | /home/psl/XuYan/LCNN/hd5/train_6.h5 80 | /home/psl/XuYan/LCNN/hd5/train_60.h5 81 | /home/psl/XuYan/LCNN/hd5/train_61.h5 82 | /home/psl/XuYan/LCNN/hd5/train_62.h5 83 | /home/psl/XuYan/LCNN/hd5/train_63.h5 84 | /home/psl/XuYan/LCNN/hd5/train_64.h5 85 | /home/psl/XuYan/LCNN/hd5/train_31.h5 86 | /home/psl/XuYan/LCNN/hd5/train_32.h5 87 | /home/psl/XuYan/LCNN/hd5/train_33.h5 88 | /home/psl/XuYan/LCNN/hd5/train_34.h5 89 | /home/psl/XuYan/LCNN/hd5/train_35.h5 90 | /home/psl/XuYan/LCNN/hd5/train_36.h5 91 | /home/psl/XuYan/LCNN/hd5/train_37.h5 92 | /home/psl/XuYan/LCNN/hd5/train_38.h5 93 | /home/psl/XuYan/LCNN/hd5/train_39.h5 94 | /home/psl/XuYan/LCNN/hd5/train_4.h5 95 | /home/psl/XuYan/LCNN/hd5/train_40.h5 96 | /home/psl/XuYan/LCNN/hd5/train_41.h5 97 | /home/psl/XuYan/LCNN/hd5/train_42.h5 98 | /home/psl/XuYan/LCNN/hd5/train_43.h5 99 | /home/psl/XuYan/LCNN/hd5/train_44.h5 100 | /home/psl/XuYan/LCNN/hd5/train_45.h5 101 | /home/psl/XuYan/LCNN/hd5/train_46.h5 102 | /home/psl/XuYan/LCNN/hd5/train_47.h5 103 | /home/psl/XuYan/LCNN/hd5/train_13.h5 104 | /home/psl/XuYan/LCNN/hd5/train_30.h5 105 | /home/psl/XuYan/LCNN/hd5/train_48.h5 106 | /home/psl/XuYan/LCNN/hd5/train_65.h5 107 | /home/psl/XuYan/LCNN/hd5/train_82.h5 108 | /home/psl/XuYan/LCNN/hd5/train_14.h5 109 | /home/psl/XuYan/LCNN/hd5/train_15.h5 110 | /home/psl/XuYan/LCNN/hd5/train_16.h5 111 | /home/psl/XuYan/LCNN/hd5/train_17.h5 112 | /home/psl/XuYan/LCNN/hd5/train_18.h5 113 | /home/psl/XuYan/LCNN/hd5/train_19.h5 114 | /home/psl/XuYan/LCNN/hd5/train_2.h5 115 | /home/psl/XuYan/LCNN/hd5/train_20.h5 116 | /home/psl/XuYan/LCNN/hd5/train_21.h5 117 | /home/psl/XuYan/LCNN/hd5/train_22.h5 118 | /home/psl/XuYan/LCNN/hd5/train_23.h5 119 | /home/psl/XuYan/LCNN/hd5/train_24.h5 120 | /home/psl/XuYan/LCNN/hd5/train_25.h5 121 | /home/psl/XuYan/LCNN/hd5/train_26.h5 122 | /home/psl/XuYan/LCNN/hd5/train_27.h5 123 | /home/psl/XuYan/LCNN/hd5/train_28.h5 124 | -------------------------------------------------------------------------------- /train/hd5_list_val.txt: -------------------------------------------------------------------------------- 1 | /home/psl/XuYan/LCNN/hd5/train_29.h5 2 | /home/psl/XuYan/LCNN/hd5/train_3.h5 3 | -------------------------------------------------------------------------------- /train/hd5_reader.py: -------------------------------------------------------------------------------- 1 | import h5py 2 | import cv2 3 | import random 4 | import numpy as np 5 | 6 | class hd5_reader(): 7 | def __init__(self, train_list, val_list, train_bsize, val_bsize): 8 | print ('initial...') 9 | self.train_list = [] 10 | self.val_list = [] 11 | self.train_bsize = train_bsize 12 | self.val_bsize = val_bsize 13 | self.BSIZE = train_bsize 14 | self.train_data = [] 15 | self.train_label = [] 16 | self.val_data = [] 17 | self.val_label = [] 18 | self.train_data_iter = 0 19 | self.read_data_flag = 0 20 | # self.ITER_flag = True 21 | 22 | # train 23 | f = open(train_list) 24 | for line in f: 25 | line = line.strip() 26 | self.train_list.append(line) 27 | f.close() 28 | random.shuffle(self.train_list) 29 | print ('training list: ' + str(len(self.train_list))) 30 | 31 | f = h5py.File(self.train_list[0]) 32 | self.train_data = np.float32(f['data']) 33 | self.train_label = np.int32(f['label']) 34 | f.close() 35 | print ('train data: ' + str(self.train_data.shape)) 36 | print ('train data: ' + str(self.train_data[0].shape)) 37 | 38 | self.train_data_ITERS = float(len(self.train_data)/float(self.train_bsize)) 39 | if self.train_data_ITERS % 1 == 0: 40 | self.train_data_ITERS = int(self.train_data_ITERS) 41 | else: 42 | self.train_data_ITERS = int(self.train_data_ITERS) + 1 43 | print ('train data iters: ' + str(self.train_data_ITERS)) 44 | 45 | self.train_epoc = int(len(self.train_data)/self.train_bsize)*len(self.train_list) + len(self.train_list) 46 | self.train_list_num = 1 47 | print ('train data epoc: ' + str(self.train_epoc)) 48 | print ('train data list num: ' + str(self.train_list_num)) 49 | 50 | # validation 51 | self.val_data_iter = 0 52 | f = open(val_list) 53 | for line in f: 54 | line = line.strip() 55 | self.val_list.append(line) 56 | f.close() 57 | print ('val list: ' + str(len(self.val_list))) 58 | 59 | for i in range(len(self.val_list)): 60 | f = h5py.File(self.val_list[i]) 61 | self.val_data.extend(np.float32(f['data'])) 62 | self.val_label.extend(np.int32(f['label'])) 63 | f.close() 64 | print ('val data: ' + str(len(self.val_data))) 65 | print ('val data: ' + str(self.val_data[0].shape)) 66 | 67 | self.val_data_ITERS = float(len(self.val_data)/float(self.val_bsize)) 68 | if self.val_data_ITERS % 1 == 0: 69 | self.val_data_ITERS = int(self.val_data_ITERS) 70 | else: 71 | self.val_data_ITERS = int(self.val_data_ITERS) + 1 72 | print ('val data iters: ' + str(self.val_data_ITERS)) 73 | # raw_input() 74 | 75 | def read_train_data(self): 76 | print ('read next hd5 file...') 77 | self.train_data = [] 78 | self.train_label = [] 79 | self.train_data_iter = 1 80 | self.train_bsize = self.BSIZE 81 | self.read_data_flag = 0 82 | # self.ITER_flag = True 83 | 84 | if self.train_list_num < len(self.train_list): 85 | f = h5py.File(self.train_list[self.train_list_num]) 86 | self.train_data = np.float32(f['data']) 87 | self.train_label = np.int32(f['label']) 88 | f.close() 89 | print ('train data: ' + str(self.train_data.shape)) 90 | print ('train data: ' + str(self.train_data[0].shape)) 91 | 92 | self.train_data_ITERS = float(len(self.train_data)/float(self.train_bsize)) 93 | if self.train_data_ITERS % 1 == 0: 94 | self.train_data_ITERS = int(self.train_data_ITERS) 95 | else: 96 | self.train_data_ITERS = int(self.train_data_ITERS) + 1 97 | 98 | self.train_list_num += 1 99 | else: 100 | print ('shuffle data...') 101 | self.train_list_num = 0 102 | self.train_data = [] 103 | self.train_label = [] 104 | self.train_data_iter = 1 105 | self.train_bsize = self.BSIZE 106 | self.read_data_flag = 0 107 | # self.ITER_flag = False 108 | 109 | random.shuffle(self.train_list) 110 | f = h5py.File(self.train_list[self.train_list_num]) 111 | self.train_data = np.float32(f['data']) 112 | self.train_label = np.int32(f['label']) 113 | f.close() 114 | print ('train data: ' + str(self.train_data.shape)) 115 | print ('train data: ' + str(self.train_data[0].shape)) 116 | 117 | self.train_data_ITERS = float(len(self.train_data)/float(self.train_bsize)) 118 | if self.train_data_ITERS % 1 == 0: 119 | self.train_data_ITERS = int(self.train_data_ITERS) 120 | else: 121 | self.train_data_ITERS = int(self.train_data_ITERS) + 1 122 | print (self.train_data_ITERS) 123 | self.train_list_num += 1 124 | 125 | def train_nextbatch(self): 126 | self.train_data_iter += 1 127 | if self.read_data_flag == 1: 128 | self.read_train_data() 129 | 130 | train_data_batch = self.train_data[(self.train_data_iter-1)*self.train_bsize:self.train_data_iter*self.train_bsize] 131 | train_label_batch = self.train_label[(self.train_data_iter-1)*self.train_bsize:self.train_data_iter*self.train_bsize] 132 | 133 | elif self.train_data_iter == self.train_data_ITERS: 134 | train_data_batch = self.train_data[(self.train_data_iter-1)*self.train_bsize:] 135 | train_label_batch = self.train_label[(self.train_data_iter-1)*self.train_bsize:] 136 | self.train_bsize = len(train_data_batch) 137 | self.read_data_flag = 1 138 | else: 139 | train_data_batch = self.train_data[(self.train_data_iter-1)*self.train_bsize:self.train_data_iter*self.train_bsize] 140 | train_label_batch = self.train_label[(self.train_data_iter-1)*self.train_bsize:self.train_data_iter*self.train_bsize] 141 | 142 | w = random.randint(0, 16) 143 | h = random.randint(0, 16) 144 | train_data_batch = np.float32(train_data_batch)[:,w:w+128,h:h+128] 145 | train_data_batch = train_data_batch/255. 146 | train_label_batch = np.array(train_label_batch).reshape(-1) 147 | return train_data_batch, train_label_batch 148 | 149 | def val_nextbatch(self): 150 | self.val_data_iter += 1 151 | 152 | if self.val_data_iter == self.val_data_ITERS: 153 | val_data_batch = self.val_data[(self.val_data_iter-1)*self.val_bsize:] 154 | val_label_batch = self.val_label[(self.val_data_iter-1)*self.val_bsize:] 155 | self.val_bsize = len(val_data_batch) 156 | else: 157 | val_data_batch = self.val_data[(self.val_data_iter-1)*self.val_bsize:self.val_data_iter*self.val_bsize] 158 | val_label_batch = self.val_label[(self.val_data_iter-1)*self.val_bsize:self.val_data_iter*self.val_bsize] 159 | 160 | w = 8 161 | h = 8 162 | val_data_batch = np.float32(val_data_batch)[:,w:w+128,h:h+128] 163 | # val_data_batch = np.float32(val_data_batch)[:,8:136,8:136] 164 | val_data_batch = val_data_batch/255. 165 | val_label_batch = np.array(val_label_batch).reshape(-1) 166 | return val_data_batch, val_label_batch 167 | 168 | 169 | 170 | 171 | 172 | 173 | 174 | 175 | 176 | 177 | 178 | 179 | 180 | 181 | 182 | 183 | 184 | -------------------------------------------------------------------------------- /train/layer.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | import numpy as np 3 | 4 | def weight(shape): 5 | return tf.get_variable('weight', shape, initializer=tf.contrib.layers.xavier_initializer()) 6 | 7 | def bias(shape, value=0.1): 8 | return tf.get_variable('bias', shape, initializer=tf.constant_initializer(value)) 9 | 10 | 11 | def Fcnn(x,insize,outsize,name,activation=None,nobias=False): 12 | with tf.variable_scope(name): 13 | if nobias: 14 | print('No biased fully connected layer is used!') 15 | W = weight([insize,outsize]) 16 | tf.summary.histogram(name+'/weight',W) 17 | if activation==None: 18 | return tf.matmul(x,W) 19 | return activation(tf.matmul(x,W)) 20 | else: 21 | W = weight([insize,outsize]) 22 | b = bias([outsize]) 23 | tf.summary.histogram(name+'/weight',W) 24 | tf.summary.histogram(name+'/bias',b) 25 | if activation==None: 26 | return tf.matmul(x,W)+b 27 | return activation(tf.matmul(x,W)+b) 28 | 29 | def conv2D(x,kernel_size,outchn,name,stride=1,pad='SAME', usebias=True): 30 | print('Conv_bias:',usebias) 31 | # with tf.variable_scope(name): 32 | # if isinstance(size,list): 33 | # kernel = size 34 | # else: 35 | kernel = [kernel_size, kernel_size] 36 | z = tf.layers.conv2d(x, outchn, kernel, strides=(stride, stride), padding=pad,\ 37 | kernel_initializer=tf.contrib.layers.xavier_initializer_conv2d(),\ 38 | use_bias=usebias,\ 39 | bias_initializer=tf.constant_initializer(0.1),name=name) 40 | # print ('z:', z.get_shape()) 41 | return z 42 | 43 | def maxpooling(x,size,stride,name,pad='SAME'): 44 | with tf.variable_scope(name): 45 | return tf.nn.max_pool(x,ksize=[1,size,size,1],strides=[1,stride,stride,1],padding=pad) 46 | 47 | def avgpooling(x,size,stride,name,pad='SAME'): 48 | with tf.variable_scope(name): 49 | return tf.nn.avg_pool(x,ksize=[1,size,size,1],strides=[1,stride,stride,1],padding=pad) 50 | 51 | 52 | def MFM(x,name): 53 | with tf.variable_scope(name): 54 | #shape is in format [batchsize, x, y, channel] 55 | # shape = tf.shape(x) 56 | shape = x.get_shape().as_list() 57 | res = tf.reshape(x,[-1,shape[1],shape[2],2,shape[-1]//2]) 58 | # x2 = tf.reshape(x,[-1,2,shape[1]//2, shape[2], shape[3]]) 59 | res = tf.reduce_max(res,axis=[3]) 60 | # x2 = tf.reduce_max(x2,axis=[1]) 61 | # x3 = tf.reshape(x2,[-1,int(x2.get_shape()[3]), int(x2.get_shape()[2]), int(x2.get_shape()[1])]) 62 | return res 63 | 64 | def MFMfc(x,half,name): 65 | with tf.variable_scope(name): 66 | shape = x.get_shape().as_list() 67 | # print('fcshape:',shape) 68 | res = tf.reduce_max(tf.reshape(x,[-1,2,shape[-1]//2]),reduction_indices=[1]) 69 | return res 70 | 71 | def batch_norm(inp,name,training=True): 72 | print('BN training:',training) 73 | return tf.layers.batch_normalization(inp,training=training,name=name) 74 | 75 | def L2_norm(inp, dim): 76 | print ('L2 normlization...') 77 | return tf.nn.l2_norm(inp, dim) 78 | 79 | def lrelu(x,name,leaky=0.2): 80 | return tf.maximum(x,x*leaky,name=name) 81 | 82 | def relu(inp,name): 83 | return tf.nn.relu(inp,name=name) 84 | 85 | def tanh(inp,name): 86 | return tf.tanh(inp,name=name) 87 | 88 | def elu(inp,name): 89 | return tf.nn.elu(inp,name=name) 90 | 91 | def sparse_softmax_cross_entropy(inp,lab,name): 92 | with tf.name_scope(name): 93 | loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(labels=lab,logits=inp)) 94 | return loss 95 | 96 | def sigmoid(inp,name): 97 | return tf.sigmoid(inp,name=name) 98 | 99 | def resize_nn(inp,size,name): 100 | with tf.name_scope(name): 101 | return tf.image.resize_nearest_neighbor(inp,size=(int(size),int(size))) 102 | 103 | def accuracy(pred,y,name): 104 | with tf.variable_scope(name): 105 | # a = tf.cast(tf.argmax(pred,1),tf.int64) 106 | # b = tf.cast(tf.argmax(y, 1),tf.int64) 107 | # c = tf.argmax(pred,1) 108 | correct = tf.equal(tf.cast(tf.argmax(pred,1),tf.int64),tf.cast(y,tf.int64)) 109 | # correct = tf.equal(tf.cast(tf.argmax(pred,1),tf.int64),tf.cast(tf.argmax(y, 1),tf.int64)) 110 | acc = tf.reduce_mean(tf.cast(correct,tf.float32)) 111 | # #acc = tf.cast(correct,tf.float32) 112 | return acc 113 | 114 | def dropout(inp, keep_prob): 115 | return tf.nn.dropout(inp,keep_prob) 116 | -------------------------------------------------------------------------------- /train/model.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | import layer as L 3 | 4 | class Model(): 5 | def __init__(self, res, res_size): 6 | self.res = res 7 | self.res_size = res_size 8 | self.layer_num = 0 9 | self.batch_norm = True 10 | 11 | def get_current_layer_res(self): 12 | return self.res 13 | 14 | def get_shape(self): 15 | return self.res_size 16 | 17 | def get_current_layer(self): 18 | return [self.res, list(self.res_size)] 19 | 20 | def activate_layer(self, param): 21 | # res = self.res 22 | with tf.name_scope('activation_' + str(self.layer_num)): 23 | if param == 1: 24 | self.res = L.MFM(self.res, name='mfm_'+str(self.layer_num)) 25 | self.res_size = self.res.get_shape() 26 | else: 27 | self.res = self.res 28 | self.res_size = self.res_size 29 | return [self.res, self.res_size] 30 | 31 | def conv_layer(self, kernel_size, outchn, stride=1, pad='SAME', activation=-1, batch_norm=False, usebias=True): 32 | with tf.name_scope('conv_' + str(self.layer_num)): 33 | inp_size = self.res_size 34 | self.res = L.conv2D(self.res, kernel_size, outchn, 'conv_'+str(self.layer_num), stride=stride, pad=pad, usebias=usebias) 35 | self.res_size = self.res.get_shape() 36 | 37 | if batch_norm: 38 | self.res = batch_norm(self.res, 'batch_norm_'+str(self.layer_num), training=self.batch_norm) 39 | 40 | self.activate_layer(activation) 41 | self.layer_num += 1 42 | 43 | print ('conv_'+str(self.layer_num), kernel_size, inp_size, self.res_size) 44 | return [self.res, self.res_size] 45 | 46 | def maxpooling_layer(self, kernel_size, stride=1, pad='SAME'): 47 | with tf.name_scope('maxpooling_' + str(self.layer_num)): 48 | inp_size = self.res_size 49 | self.res = L.maxpooling(self.res, kernel_size, stride=stride, name='maxpooling_'+str(self.layer_num), pad=pad) 50 | self.res_size = self.res.get_shape() 51 | 52 | print ('maxpooling_'+str(self.layer_num), kernel_size, inp_size, self.res_size) 53 | return [self.res, self.res_size] 54 | 55 | def sum(self, layerin): 56 | layer1_size = self.res_size 57 | layer2_size = layerin[1] 58 | print (layer1_size[2], layer1_size[3]) 59 | print (layer2_size[2], layer2_size[3]) 60 | assert layer1_size[1] == layer2_size[1] 61 | assert layer1_size[2] == layer2_size[2] and layer1_size[3] == layer2_size[3] 62 | 63 | inp_size = self.res_size 64 | with tf.name_scope('sum_' + str(self.layer_num)): 65 | self.res = self.res + layerin[0] 66 | self.res_size = self.res.get_shape() 67 | 68 | print ('sum_'+str(self.layer_num), inp_size, self.res_size) 69 | return [self.res, self.res_size] 70 | 71 | def flatten(self): 72 | inp_size = self.res_size 73 | # print (inp_size[1], inp_size[2], inp_size[3]) 74 | with tf.name_scope('flatten_' + str(self.layer_num)): 75 | self.res = tf.reshape(self.res, [-1, int(inp_size[1])*int(inp_size[2])*int(inp_size[3])]) 76 | self.res_size = self.res.get_shape() 77 | 78 | print ('flatten_'+str(self.layer_num), inp_size, self.res_size) 79 | return [self.res, self.res_size] 80 | 81 | def fcnn_layer(self, outsize, nobias=False): 82 | # inp_size = int(self.res.get_shape()[1])*int(self.res.get_shape()[2])*int(self.res.get_shape()[3]) 83 | inp_size = self.res.get_shape()[1] 84 | # print ('inp_size', inp_size) 85 | # print ('outsize', outsize) 86 | with tf.name_scope('Fcnn_' + str(self.layer_num)): 87 | self.res = L.Fcnn(self.res, inp_size, outsize, 'Fcnn_'+str(self.layer_num), activation=None, nobias=False) 88 | self.res_size = self.res.get_shape() 89 | 90 | self.layer_num += 1 91 | print ('Fcnn_'+str(self.layer_num), inp_size, self.res_size) 92 | return [self.res, self.res_size] 93 | 94 | def dropout(self, keep_prob): 95 | self.res = L.dropout(self.res, keep_prob) 96 | self.res_size = self.res.get_shape() 97 | 98 | return [self.res, self.res_size] 99 | 100 | def accuracy(self, label): 101 | self.res = L.accuracy(self.res, label, 'accuracy') 102 | 103 | return self.res 104 | 105 | 106 | def loadSess(model_path, sess): 107 | 108 | epoc = 0 109 | iters = 0 110 | if model_path != None: 111 | ckpt = tf.train.get_checkpoint_state(model_path) 112 | if ckpt: 113 | model = ckpt.model_checkpoint_path 114 | epoc = model.split('_')[1] 115 | iters = model.split('_')[-1].replace('.cpkt', '') 116 | print ('loading from model:', model) 117 | 118 | saver = tf.train.Saver() 119 | saver.restore(sess, model) 120 | else: 121 | sess.run(tf.global_variables_initializer()) 122 | print ('No checkpoint in the folder...') 123 | 124 | return sess, int(epoc), int(iters) 125 | 126 | 127 | 128 | 129 | --------------------------------------------------------------------------------