├── .gitignore ├── README.md ├── __init__.py ├── cache ├── confusion_matrix.npy ├── fc2_mean.npy ├── fc2_std.npy ├── image_mean.npy ├── image_std.npy ├── mvcnn_test_accuracy_shuffled.npy ├── mvcnn_test_accuracy_unshuffled.npy ├── svm_top_k_1.pkl ├── svm_top_k_10.pkl ├── svm_top_k_10_confusion.npy ├── svm_top_k_10_random.pkl ├── svm_top_k_10_random_confusion.npy ├── svm_top_k_15.pkl ├── svm_top_k_15_confusion.npy ├── svm_top_k_15_random.pkl ├── svm_top_k_15_random_confusion.npy ├── svm_top_k_1_confusion.npy ├── svm_top_k_1_random.pkl ├── svm_top_k_1_random_confusion.npy ├── svm_top_k_20.pkl ├── svm_top_k_20_confusion.npy ├── svm_top_k_20_random.pkl ├── svm_top_k_20_random_confusion.npy ├── svm_top_k_25.pkl ├── svm_top_k_25_confusion.npy ├── svm_top_k_25_random.pkl ├── svm_top_k_25_random_confusion.npy ├── svm_top_k_3.pkl ├── svm_top_k_3_confusion.npy ├── svm_top_k_3_random.pkl ├── svm_top_k_3_random_confusion.npy ├── svm_top_k_5.pkl ├── svm_top_k_5_confusion.npy ├── svm_top_k_5_random.pkl ├── svm_top_k_5_random_confusion.npy ├── svm_top_k_7.pkl ├── svm_top_k_7_confusion.npy ├── svm_top_k_7_random.pkl └── svm_top_k_7_random_confusion.npy ├── classification ├── __init__.py └── mvcnn_FPS.py ├── data ├── saliency_labels_train.txt └── saliency_labels_valid.txt ├── dependencies.txt ├── generate_views ├── CMakeLists.txt ├── README.md ├── cmake │ └── FindLIBIGL.cmake ├── include │ ├── helpers │ │ └── RandomUtils.h │ └── render │ │ ├── OpenGLStuff.h │ │ ├── Phong.h │ │ └── Program.h └── src │ ├── helpers │ └── RandomUtils.cpp │ ├── main.cpp │ └── render │ ├── OpenGLStuff.cpp │ ├── Phong.cpp │ └── shaders │ ├── phong.frag │ ├── phong.geom │ └── phong.vert ├── globals.py ├── logs ├── 10_15_20_25.log ├── 1_3_5_7.log ├── saliency.log ├── saliency_log_4_27.log ├── top_1.log ├── top_10.log ├── top_15.log ├── top_20.log ├── top_25.log ├── top_3.log ├── top_5.log ├── top_7.log ├── training_04_13_08_08.log └── training_04_19_02_01.log ├── models ├── __init__.py ├── multiview_cnn.py ├── multiview_svm.py └── saliency.py ├── poster.pdf ├── requirements.txt ├── screenshots ├── confusion_matrix.png └── sample_data.png ├── scripts ├── generate_confusion_matrix.py ├── generate_saliency_data.py ├── plot_logs.py ├── precompute_fc_mean_std.py ├── precompute_image_mean_std.py ├── test_mvcnn.py ├── test_saliency.py ├── train_saliency.py ├── train_svm.py ├── view_confusion_matrix.py └── view_salient.py └── utils ├── __init__.py ├── custom_datagen.py └── helpers.py /.gitignore: -------------------------------------------------------------------------------- 1 | *.swo 2 | *.swp 3 | *.pyc 4 | __pycache__ 5 | .DS_Store 6 | **/build 7 | *.h5 8 | 9 | train_cnn/model_weights.* 10 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Setup 2 | 3 | Clone the repo, add the package to the python path, download python dependencies. 4 | 5 | ``` 6 | PYTHONPATH=$(pwd):$PYTHONPATH 7 | git clone https://github.com/bradyz/geometry_processing.git 8 | 9 | pip install -r requirements.txt 10 | or 11 | pip install --user -r requirements.txt 12 | ``` 13 | 14 | ## Data Dependencies 15 | 16 | Modelnet with 25 viewpoints each - https://drive.google.com/open?id=0B0d9M5p2RxBqN0IzOXpudjMyTDQ 17 | 18 | Our model's weights - https://drive.google.com/open?id=0B0d9M5p2RxBqMlNZOFg1YmlYR3c 19 | 20 | ## Contents 21 | 22 | 1. [View Generator](#view_gen) - take 2D projections of mesh files. 23 | 2. [Train CNN](#train_cnn) - fine tune a VGG-16 CNN on the new images. 24 | 3. [Classifier](#classify) - train a SVM on the CNN features. 25 | 3. [References](#references) - papers and resources used. 26 | 27 | ## View Generator 28 | 29 | Given a model and a list of viewpoints - .png image files that correspond to 2D projection will be generated. 30 | 31 | Preprocessing consists of centering the mesh, uniformly scaling the bounding box to a unit cube, and taking viewpoints that are centered at the centroid. 32 | 33 | Currently there are 25 viewpoints being generated that fall around the unit sphere from 5 different phis and 5 different thetas (spherical coordinates). 34 | 35 | 36 | 37 | ## Train CNN 38 | 39 | The model used in this project is a VGG-16 with pretrained weights (ImageNet), with two additional layers fc1 (2048), fc2 (1024). 40 | 41 | Training was done for 10 epochs on 100k training images (4000 meshes) over 10 labels of ModelNet10. The images were 224 x 224 rgb. Cross entropy loss was used in combination with a SGD optimizer with a batch size of 64. Training took approximately 5 hours a NVIDIA K40 gpu. 42 | 43 | After training, classification accuracy, given a single pose, is at 80% on a test set of 20k images. 44 | 45 | 46 | 47 | ## Classifier 48 | 49 | The question asked is - given a mesh and several viewpoints, does it help to use all of the viewpoints (MVCNN), or does a selected subset of size k give better accuracy? 50 | 51 | We use a one-vs-rest linear SVM, similar to MVCNN, to classify activation values of the final fc layer. 52 | 53 | The current methods consist of the using the following (currently unimplemented) - 54 | 55 | * Sort by minimized entropy 56 | * Random K 57 | * FPS (farthest point selection) on sorted 58 | 59 | ## References 60 | 61 | Multi-view Convolutional Neural Networks for 3D Shape Recognition - https://arxiv.org/pdf/1505.00880.pdf 62 | Princeton ModelNet - http://modelnet.cs.princeton.edu/ 63 | -------------------------------------------------------------------------------- /__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/__init__.py -------------------------------------------------------------------------------- /cache/confusion_matrix.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/confusion_matrix.npy -------------------------------------------------------------------------------- /cache/fc2_mean.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/fc2_mean.npy -------------------------------------------------------------------------------- /cache/fc2_std.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/fc2_std.npy -------------------------------------------------------------------------------- /cache/image_mean.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/image_mean.npy -------------------------------------------------------------------------------- /cache/image_std.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/image_std.npy -------------------------------------------------------------------------------- /cache/mvcnn_test_accuracy_shuffled.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/mvcnn_test_accuracy_shuffled.npy -------------------------------------------------------------------------------- /cache/mvcnn_test_accuracy_unshuffled.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/mvcnn_test_accuracy_unshuffled.npy -------------------------------------------------------------------------------- /cache/svm_top_k_1.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_1.pkl -------------------------------------------------------------------------------- /cache/svm_top_k_10.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_10.pkl -------------------------------------------------------------------------------- /cache/svm_top_k_10_confusion.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_10_confusion.npy -------------------------------------------------------------------------------- /cache/svm_top_k_10_random.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_10_random.pkl -------------------------------------------------------------------------------- /cache/svm_top_k_10_random_confusion.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_10_random_confusion.npy -------------------------------------------------------------------------------- /cache/svm_top_k_15.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_15.pkl -------------------------------------------------------------------------------- /cache/svm_top_k_15_confusion.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_15_confusion.npy -------------------------------------------------------------------------------- /cache/svm_top_k_15_random.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_15_random.pkl -------------------------------------------------------------------------------- /cache/svm_top_k_15_random_confusion.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_15_random_confusion.npy -------------------------------------------------------------------------------- /cache/svm_top_k_1_confusion.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_1_confusion.npy -------------------------------------------------------------------------------- /cache/svm_top_k_1_random.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_1_random.pkl -------------------------------------------------------------------------------- /cache/svm_top_k_1_random_confusion.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_1_random_confusion.npy -------------------------------------------------------------------------------- /cache/svm_top_k_20.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_20.pkl -------------------------------------------------------------------------------- /cache/svm_top_k_20_confusion.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_20_confusion.npy -------------------------------------------------------------------------------- /cache/svm_top_k_20_random.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_20_random.pkl -------------------------------------------------------------------------------- /cache/svm_top_k_20_random_confusion.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_20_random_confusion.npy -------------------------------------------------------------------------------- /cache/svm_top_k_25.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_25.pkl -------------------------------------------------------------------------------- /cache/svm_top_k_25_confusion.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_25_confusion.npy -------------------------------------------------------------------------------- /cache/svm_top_k_25_random.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_25_random.pkl -------------------------------------------------------------------------------- /cache/svm_top_k_25_random_confusion.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_25_random_confusion.npy -------------------------------------------------------------------------------- /cache/svm_top_k_3.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_3.pkl -------------------------------------------------------------------------------- /cache/svm_top_k_3_confusion.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_3_confusion.npy -------------------------------------------------------------------------------- /cache/svm_top_k_3_random.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_3_random.pkl -------------------------------------------------------------------------------- /cache/svm_top_k_3_random_confusion.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_3_random_confusion.npy -------------------------------------------------------------------------------- /cache/svm_top_k_5.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_5.pkl -------------------------------------------------------------------------------- /cache/svm_top_k_5_confusion.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_5_confusion.npy -------------------------------------------------------------------------------- /cache/svm_top_k_5_random.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_5_random.pkl -------------------------------------------------------------------------------- /cache/svm_top_k_5_random_confusion.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_5_random_confusion.npy -------------------------------------------------------------------------------- /cache/svm_top_k_7.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_7.pkl -------------------------------------------------------------------------------- /cache/svm_top_k_7_confusion.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_7_confusion.npy -------------------------------------------------------------------------------- /cache/svm_top_k_7_random.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_7_random.pkl -------------------------------------------------------------------------------- /cache/svm_top_k_7_random_confusion.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/cache/svm_top_k_7_random_confusion.npy -------------------------------------------------------------------------------- /classification/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/classification/__init__.py -------------------------------------------------------------------------------- /classification/mvcnn_FPS.py: -------------------------------------------------------------------------------- 1 | import pickle 2 | import os 3 | import cv2 4 | import sys 5 | import Image 6 | import numpy as np 7 | import random as rd 8 | from fnmatch import fnmatch 9 | from sklearn import linear_model 10 | from matplotlib import pyplot as plt 11 | %matplotlib inline 12 | 13 | import keras.layers 14 | from keras.models import Model, Sequential 15 | from keras.applications.vgg16 import VGG16 16 | from keras.layers import Dense, Flatten, Input 17 | from tensorflow.python.platform import gfile 18 | from keras.preprocessing.image import ImageDataGenerator 19 | from keras.callbacks import CSVLogger, ModelCheckpoint, ReduceLROnPlateau 20 | 21 | SAVE_WEIGHTS_FILE = '/your/weights/path/model_weights.h5' 22 | VALID_DIR = "/your/testimages/path/ModelNetViewpoints/test/" 23 | IMAGE_SIZE = 224 24 | NUM_CLASSES = 10 25 | 26 | def load_model_vgg(): 27 | img_input = Input(tensor=Input(shape=(IMAGE_SIZE, IMAGE_SIZE, 3))) 28 | base_model = VGG16(include_top=False, input_tensor=img_input) 29 | 30 | for layer in base_model.layers: 31 | layer.trainable = False 32 | 33 | x = base_model.output 34 | x = Flatten(name='flatten')(x) 35 | x = Dense(2048, activation='relu', name='fc1')(x) 36 | x = Dense(1024, activation='relu', name='fc2')(x) 37 | x = Dense(NUM_CLASSES, activation='softmax', name='predictions')(x) 38 | model = Model(input=img_input, output=x) 39 | model.load_weights(SAVE_WEIGHTS_FILE, by_name=True) 40 | #print('Model loaded with weights from %s.' % SAVE_WEIGHTS_FILE) 41 | 42 | return model 43 | 44 | class mvcnnclass: 45 | 46 | def __init__(self, 47 | model, 48 | featurelayer='fc1', 49 | numallview=25, 50 | numviewselection=7, 51 | data_path=VALID_DIR 52 | ): 53 | self.model = model 54 | self.featurelayer = featurelayer 55 | self.numallview = numallview 56 | self.numviewselection = numviewselection 57 | self.data_path = data_path 58 | 59 | def get_data(self, objname): 60 | viewimgfilepaths, objIDs = [],[] 61 | objlist = np.sort(os.listdir(self.data_path+objname+'/')) 62 | modelnum = objlist[::self.numallview] 63 | for i,name in enumerate(modelnum): 64 | name = name.replace('.off_1_1.png','') 65 | files = np.sort(gfile.Glob(self.data_path+objname+'/'+name+'*')) 66 | viewimgfilepaths.append(files) 67 | objIDs.append(name) 68 | print '%s views are loaded!' % objname 69 | return viewimgfilepaths, objIDs 70 | 71 | def image_from_path(self, image_path): 72 | image = cv2.imread(image_path, cv2.IMREAD_COLOR) 73 | try: 74 | image = np.reshape(image, (1, IMAGE_SIZE, IMAGE_SIZE, 3)) 75 | except: 76 | print('IMAGE LOADING FAILDED!!!') 77 | print('->path: %s' % image_path) 78 | raise 79 | return image 80 | 81 | def singleview_classification(self, image_path): 82 | image = self.image_from_path(image_path) 83 | prediction = self.model.predict(image) 84 | return prediction[0], np.argsort(prediction[0])[:-6:-1] 85 | 86 | def feature_extraction(self, imagepath, featurelayer): 87 | image = self.image_from_path(imagepath) 88 | intermediate_layer_model = Model(input=self.model.input, 89 | output=self.model.get_layer(featurelayer).output) 90 | return intermediate_layer_model.predict(image) 91 | 92 | def feat_distance(self, feat1, feat2): 93 | sim = spatial.distance.cosine(feat1, feat2) 94 | return 1-sim 95 | 96 | def output_entropy(self, prediction, eps=10e-4): 97 | return np.sum(-(prediction+eps)*np.log2(prediction+eps)) 98 | 99 | def view_score(self, feat_test, feat_ref, prediction, portion=1.0): 100 | entropy_score = self.output_entropy(prediction) 101 | fps_score = self.feat_distance(feat_test, feat_ref) 102 | return entropy_score+portion*fps_score 103 | 104 | def entropy_selection(self, viewimgfilepaths): 105 | filenames = [] 106 | entropies = [] 107 | for file in viewimgfilepaths: 108 | sys.stdout.write('.') 109 | prediction, _ = self.singleview_classification(file) 110 | entropies = np.append(entropies, self.output_entropy(prediction)) 111 | filenames = np.append(filenames, file) 112 | solution_feats, solution_filepath = [],[] 113 | argminentropy = np.argsort(entropies)[:self.numviewselection] 114 | solution_filepath = filenames[argminentropy] 115 | for file in solution_filepath: 116 | solution_feats.append(self.feature_extraction(file, self.featurelayer)) 117 | solution_feats = np.asarray(solution_feats) 118 | solution_feats = np.reshape(solution_feats, 119 | (solution_filepath.shape[0], 120 | self.model.get_layer(self.featurelayer).output.shape[1])) 121 | solution_filepath = np.asarray(solution_filepath) 122 | return solution_feats, solution_filepath 123 | 124 | def fps_selection(self, viewimgfilepaths): 125 | feats, filenames = [],[] 126 | for file in viewimgfilepaths: 127 | sys.stdout.write('.') 128 | feat = self.feature_extraction(file, self.featurelayer) 129 | filenames.append(file) 130 | feats.append(feat) 131 | solution_feats, solution_filepath = [],[] 132 | initindx = rd.randint(0, len(filenames)-1) 133 | 134 | solution_feats.append(feats.pop(initindx)) 135 | solution_filepath.append(filenames.pop(initindx)) 136 | 137 | for i in range(self.numviewselection-1): 138 | distances = [self.feat_distance(f, solution_feats[0]) for f in feats] 139 | for i, f in enumerate(feats): 140 | for j, s in enumerate(solution_feats): 141 | distances[i] = min(distances[i], self.feat_distance(f, s)) 142 | solution_feats.append(feats.pop(distances.index(max(distances)))) 143 | solution_filepath.append(filenames.pop(distances.index(max(distances)))) 144 | solution_feats = np.asarray(solution_feats) 145 | solution_feats = np.reshape(solution_feats, 146 | (len(solution_filepath), 147 | self.model.get_layer(self.featurelayer).output.shape[1])) 148 | solution_filepath = np.asarray(solution_filepath) 149 | sys.stdout.write('!\n') 150 | print "FPS selection done." 151 | return solution_feats, solution_filepath 152 | 153 | def feature_pooling(self, selected_feats): 154 | return np.amax(selected_feats, axis=0) 155 | 156 | def mvcnn_classification(self, objname): 157 | predictions, class_names = [],[] 158 | viewimgfilepaths, objID =mvcnn.get_data(objname) 159 | for i, objIDpaths in enumerate(viewimgfilepaths): 160 | print "Object ID-> %s" % objID[i] 161 | bestfeats, bestfilepath = self.entropy_selection(objIDpaths) 162 | agg_feat = self.feature_pooling(bestfeats) 163 | 164 | feat_input = Input(tensor=Input(shape=(agg_feat.shape))) 165 | if self.featurelayer=='fc1': 166 | x = self.model.get_layer('fc2')(feat_input) 167 | x = self.model.get_layer('predictions')(x) 168 | else: 169 | x = self.model.get_layer('predictions')(feat_input) 170 | cnn2_model = Model(input=feat_input, output=x) 171 | 172 | prediction = cnn2_model.predict(np.array([agg_feat])) 173 | class_name = np.sort(os.listdir(VALID_DIR))[np.argmax(prediction, axis=1)] 174 | sys.stdout.write('!\n') 175 | predictions.append(prediction) 176 | class_names.append(class_name) 177 | print "Classification Done." 178 | return prediction, class_name 179 | 180 | def singleview_analysis(self, savetopath): 181 | objlist = np.sort(os.listdir(self.data_path)) 182 | print objlist 183 | for _, objname in enumerate(objlist): 184 | viewimgfilepaths, objID =mvcnn.get_data(objname) 185 | for i, file in enumerate(viewimgfilepaths): 186 | print "Model ID : %s" % objID[i] 187 | fc1_feats, fc2_feats, fc1_fps, fc2_fps = [],[],[],[] 188 | filename, entropies, classindxs = [],[],[] 189 | recordlist = [] 190 | savefile = savetopath+objname+"/"+objID[i]+"_result.npy" 191 | if os.path.isfile(savefile): 192 | print "%s_result.npy file already exists!" % objID[i] 193 | else: 194 | for _, viewimg in enumerate(file): 195 | filename.append(viewimg.replace(self.data_path+objname+'/','')) 196 | prediction, classindx = self.singleview_classification(viewimg) 197 | entropies.append(self.output_entropy(prediction)) 198 | classindxs.append(classindx) 199 | fc1_feats.append(self.feature_extraction(viewimg, 'fc1')) 200 | fc2_feats.append(self.feature_extraction(viewimg, 'fc2')) 201 | agg_fc1_feat = np.asarray(fc1_feats) 202 | agg_fc1_feat = np.reshape(agg_fc1_feat,(len(file), fc1_feats[0].size)) 203 | agg_fc1_feat = self.feature_pooling(agg_fc1_feat) 204 | agg_fc2_feat = np.asarray(fc2_feats) 205 | agg_fc2_feat = np.reshape(agg_fc2_feat,(len(file), fc2_feats[0].size)) 206 | agg_fc2_feat = self.feature_pooling(agg_fc2_feat) 207 | for j, fc1_feat in enumerate(fc1_feats): 208 | fc1_fps.append(self.feat_distance(fc1_feat, agg_fc1_feat)) 209 | fc2_fps.append(self.feat_distance(fc2_feats[j], agg_fc2_feat)) 210 | 211 | record={"imgids": filename, 212 | "labels": classindxs, 213 | "entropy": entropies, 214 | "fc1_fps": fc1_fps, 215 | "fc1": fc1_feats, 216 | "fc1_global": agg_fc1_feat, 217 | "fc1_fps": fc2_fps, 218 | "fc2": fc2_feats, 219 | "fc2_global": agg_fc2_feat, 220 | } 221 | recordlist.append(record) 222 | np.save(savefile, recordlist) 223 | 224 | if __name__ == '__main__': 225 | vggmodel = load_model_vgg() 226 | mvcnn = mvcnnclass(vggmodel, featurelayer='fc1') 227 | pred, classname = mvcnn.mvcnn_classification('bed') 228 | mvcnn.singleview_analysis(ANALYSIS_DIR) 229 | 230 | -------------------------------------------------------------------------------- /dependencies.txt: -------------------------------------------------------------------------------- 1 | sudo apt-get install python3-tk 2 | -------------------------------------------------------------------------------- /generate_views/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | cmake_minimum_required(VERSION 2.8) 2 | project(GLSL) 3 | 4 | set(EXECUTABLE_OUTPUT_PATH ${PROJECT_BINARY_DIR}/bin) 5 | 6 | set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -g -lpng -O3" ) 7 | set(CMAKE_FIND_LIBRARY_SUFFIXES "${CMAKE_FIND_LIBRARY_SUFFIXES}") 8 | 9 | link_directories( 10 | "/usr/local/lib" 11 | ) 12 | 13 | include_directories( 14 | "/usr/local/include" 15 | "${GLSL_SOURCE_DIR}" 16 | "${PROJECT_SOURCE_DIR}/include" 17 | ) 18 | 19 | set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${CMAKE_CURRENT_SOURCE_DIR}/cmake) 20 | 21 | find_package(LIBIGL QUIET) 22 | 23 | if (NOT LIBIGL_FOUND) 24 | message(FATAL_ERROR "libigl not found") 25 | endif() 26 | 27 | option(LIBIGL_USE_STATIC_LIBRARY "Use LibIGL as static library" OFF) 28 | 29 | # add a customizable menu bar 30 | option(LIBIGL_WITH_NANOGUI "Use Nanogui menu" OFF) 31 | 32 | # libigl options: choose your dependencies (by default everything is OFF except opengl) 33 | option(LIBIGL_WITH_VIEWER "Use OpenGL viewer" ON) 34 | option(LIBIGL_WITH_OPENGL "Use OpenGL" ON) 35 | option(LIBIGL_WITH_OPENGL_GLFW "Use GLFW" ON) 36 | option(LIBIGL_WITH_BBW "Use BBW" OFF) 37 | option(LIBIGL_WITH_EMBREE "Use Embree" OFF) 38 | option(LIBIGL_WITH_PNG "Use PNG" OFF) 39 | option(LIBIGL_WITH_TETGEN "Use Tetgen" OFF) 40 | option(LIBIGL_WITH_TRIANGLE "Use Triangle" OFF) 41 | option(LIBIGL_WITH_XML "Use XML" OFF) 42 | option(LIBIGL_WITH_LIM "Use LIM" OFF) 43 | option(LIBIGL_WITH_COMISO "Use CoMiso" OFF) 44 | option(LIBIGL_WITH_MATLAB "Use Matlab" OFF) # This option is not supported yet 45 | option(LIBIGL_WITH_MOSEK "Use MOSEK" OFF) # This option is not supported yet 46 | option(LIBIGL_WITH_CGAL "Use CGAL" OFF) 47 | if(LIBIGL_WITH_CGAL) # Do not remove or move this block, the cgal build system fails without it 48 | find_package(CGAL REQUIRED) 49 | set(CGAL_DONT_OVERRIDE_CMAKE_FLAGS TRUE CACHE BOOL "CGAL's CMAKE Setup is super annoying ") 50 | include(${CGAL_USE_FILE}) 51 | endif() 52 | 53 | add_subdirectory("${LIBIGL_INCLUDE_DIR}/../shared/cmake" "libigl") 54 | 55 | # libigl information 56 | message("libigl includes: ${LIBIGL_INCLUDE_DIRS}") 57 | message("libigl libraries: ${LIBIGL_LIBRARIES}") 58 | message("libigl extra sources: ${LIBIGL_EXTRA_SOURCES}") 59 | message("libigl extra libraries: ${LIBIGL_EXTRA_LIBRARIES}") 60 | message("libigl definitions: ${LIBIGL_DEFINITIONS}") 61 | 62 | # Prepare the build environment 63 | include_directories(${LIBIGL_INCLUDE_DIRS}) 64 | add_definitions(${LIBIGL_DEFINITIONS}) 65 | 66 | find_package(OpenGL REQUIRED) 67 | include_directories(${OpenGL_INCLUDE_DIRS}) 68 | link_directories(${OpenGL_LIBRARY_DIRS}) 69 | add_definitions(${OpenGL_DEFINITIONS}) 70 | 71 | find_package(GLEW REQUIRED) 72 | include_directories(${GLEW_INCLUDE_DIRS}) 73 | link_libraries(${GLEW_LIBRARIES}) 74 | 75 | find_package(PkgConfig REQUIRED) 76 | pkg_search_module(GLFW REQUIRED glfw3) 77 | include_directories(${GLFW_INCLUDE_DIRS}) 78 | 79 | find_package(JPEG REQUIRED) 80 | include_directories(${JPEG_INCLUDE_DIR}) 81 | 82 | find_package(Eigen3 REQUIRED) 83 | include_directories(${EIGEN3_INCLUDE_DIR}) 84 | 85 | # Subdivision Algorithms Demo. 86 | FILE(GLOB SRC ./src/main.cpp ./src/**/*.cpp) 87 | add_executable(generate_views ${SRC} ${LIBIGL_EXTRA_SOURCES}) 88 | target_link_libraries(generate_views 89 | ${OPENGL_gl_LIBRARY} 90 | ${GLFW_LIBRARIES} 91 | ${GLEW_LIBRARY} 92 | ${JPEG_LIBRARIES} 93 | ${LDFLAGS} 94 | ${LIBIGL_LIBRARIES} 95 | ${LIBIGL_EXTRA_LIBRARIES}) 96 | -------------------------------------------------------------------------------- /generate_views/README.md: -------------------------------------------------------------------------------- 1 | ## Dependencies 2 | 3 | Libigl 4 | Png++ 5 | Eigen 6 | Glm 7 | Glfw 8 | OpenGL 4.1 9 | 10 | 11 | ## Usage 12 | 13 | ```bash 14 | mkdir build 15 | cd build 16 | cmake .. 17 | ``` 18 | 19 | At this point you should add the libigl include path (use ccmake). 20 | 21 | In the build directory - 22 | 23 | ```bash 24 | make -j4 25 | ``` 26 | 27 | In the main directory, supply the text file with all the paths to the data, and 28 | the output directory (must include the slash on the end). 29 | 30 | All the nested paths must be created, so if you have /path/to/model/mesh.off, 31 | $OUTPUT_DIR/path/to/model must be created before the script runs. 32 | 33 | ```bash 34 | ./build/bin/generate_views $INPUT_TXT $OUTPUT_DIR 35 | ``` 36 | -------------------------------------------------------------------------------- /generate_views/cmake/FindLIBIGL.cmake: -------------------------------------------------------------------------------- 1 | # - Try to find the LIBIGL library 2 | # Once done this will define 3 | # 4 | # LIBIGL_FOUND - system has LIBIGL 5 | # LIBIGL_INCLUDE_DIR - **the** LIBIGL include directory 6 | # LIBIGL_INCLUDE_DIRS - LIBIGL include directories 7 | # LIBIGL_SOURCES - the LIBIGL source files 8 | if(NOT LIBIGL_FOUND) 9 | 10 | FIND_PATH(LIBIGL_INCLUDE_DIR igl/readOBJ.h 11 | ${PROJECT_SOURCE_DIR}/../../include 12 | ${PROJECT_SOURCE_DIR}/../include 13 | ${PROJECT_SOURCE_DIR}/include 14 | ${PROJECT_SOURCE_DIR}/../libigl/include 15 | ${PROJECT_SOURCE_DIR}/../../libigl/include 16 | $ENV{LIBIGL}/include 17 | $ENV{LIBIGLROOT}/include 18 | $ENV{LIBIGL_ROOT}/include 19 | $ENV{LIBIGL_DIR}/include 20 | $ENV{LIBIGL_DIR}/inc 21 | /usr/include 22 | /usr/local/include 23 | /usr/local/igl/libigl/include 24 | ) 25 | 26 | 27 | if(LIBIGL_INCLUDE_DIR) 28 | set(LIBIGL_FOUND TRUE) 29 | set(LIBIGL_INCLUDE_DIRS ${LIBIGL_INCLUDE_DIR} ${LIBIGL_INCLUDE_DIR}/../external/Singular_Value_Decomposition) 30 | #set(LIBIGL_SOURCES 31 | # ${LIBIGL_INCLUDE_DIR}/igl/viewer/Viewer.cpp 32 | #) 33 | endif() 34 | 35 | endif() 36 | -------------------------------------------------------------------------------- /generate_views/include/helpers/RandomUtils.h: -------------------------------------------------------------------------------- 1 | #ifndef RANDOMUTILS_H 2 | #define RANDOMUTILS_H 3 | 4 | #include 5 | #include 6 | #include 7 | 8 | #include 9 | 10 | const glm::vec4 GREEN = glm::vec4(0.0f, 1.0f, 0.0f, 1.0f); 11 | const glm::vec4 BLUE = glm::vec4(0.0f, 0.0f, 1.0f, 1.0f); 12 | const glm::vec4 WHITE = glm::vec4(1.0f, 1.0f, 1.0f, 1.0f); 13 | const glm::vec4 CYAN = glm::vec4(0.0f, 1.0f, 1.0f, 1.0f); 14 | const glm::vec4 RED = glm::vec4(1.0f, 0.0f, 0.0f, 1.0f); 15 | const glm::vec4 BROWN = glm::vec4(0.72f, 0.60f, 0.41f, 1.0f); 16 | const glm::vec4 BLACK = glm::vec4(0.0f, 0.0f, 0.0f, 1.0f); 17 | 18 | void LoadOBJ(const std::string& file, 19 | std::vector& vertices, 20 | std::vector& faces, 21 | std::vector& normals); 22 | 23 | void LoadOBJWithNormals(const std::string& file, 24 | std::vector& vertices, 25 | std::vector& faces, 26 | std::vector& normals); 27 | 28 | std::string loadShader(const std::string& file); 29 | 30 | std::vector getVertexNormals (const std::vector& vertices, 31 | const std::vector& faces); 32 | 33 | void fixDuplicateVertices (std::vector& v, 34 | std::vector& f); 35 | 36 | namespace glm { 37 | std::ostream& operator<<(std::ostream& os, const glm::vec2& v); 38 | std::ostream& operator<<(std::ostream& os, const glm::vec3& v); 39 | std::ostream& operator<<(std::ostream& os, const glm::vec4& v); 40 | std::ostream& operator<<(std::ostream& os, const glm::mat3& v); 41 | std::ostream& operator<<(std::ostream& os, const glm::mat4& v); 42 | } 43 | 44 | #endif 45 | -------------------------------------------------------------------------------- /generate_views/include/render/OpenGLStuff.h: -------------------------------------------------------------------------------- 1 | #ifndef OPENGLSTUFF_H 2 | #define OPENGLSTUFF_H 3 | 4 | #include 5 | #include 6 | 7 | #include 8 | #include 9 | 10 | #include 11 | #include 12 | 13 | extern glm::vec4 LIGHT_POSITION; 14 | 15 | extern const float kNear; 16 | extern const float kFar; 17 | extern const float kFov; 18 | 19 | extern float camera_distance; 20 | 21 | extern glm::vec3 up; 22 | extern glm::vec3 look; 23 | extern glm::vec3 tangent; 24 | extern glm::mat3 orientation; 25 | 26 | extern glm::vec3 eye; 27 | extern glm::vec3 center; 28 | 29 | extern glm::mat4 view_matrix; 30 | extern glm::mat4 projection_matrix; 31 | 32 | extern const std::string window_title; 33 | 34 | extern int window_width; 35 | extern int window_height; 36 | 37 | extern const int WIDTH; 38 | extern const int HEIGHT; 39 | 40 | extern bool do_action; 41 | 42 | const std::string PHONG_VERT = "./src/render/shaders/phong.vert"; 43 | const std::string PHONG_GEOM = "./src/render/shaders/phong.geom"; 44 | const std::string PHONG_FRAG = "./src/render/shaders/phong.frag"; 45 | 46 | enum { 47 | kVertexBuffer, 48 | kIndexBuffer, 49 | kVertexNormalBuffer, 50 | kNumVbos 51 | }; 52 | 53 | enum { 54 | kPhongVao, 55 | kNumVaos 56 | }; 57 | 58 | extern GLuint array_objects[kNumVaos]; 59 | extern GLuint buffer_objects[kNumVaos][kNumVbos]; 60 | 61 | void initOpenGL(); 62 | bool keepLoopingOpenGL(glm::vec3 eye_pos); 63 | void cleanupOpenGL(); 64 | void endLoopOpenGL(); 65 | 66 | GLuint setupShader (const char* shaderName, GLenum shaderType); 67 | 68 | extern const char* OpenGlErrorToString(GLenum error); 69 | 70 | #define CHECK_SUCCESS(x) \ 71 | if (!(x)) { \ 72 | glfwTerminate(); \ 73 | exit(EXIT_FAILURE); \ 74 | } 75 | 76 | #define CHECK_GL_SHADER_ERROR(id) \ 77 | { \ 78 | GLint status = 0; \ 79 | GLint length = 0; \ 80 | glGetShaderiv(id, GL_COMPILE_STATUS, &status); \ 81 | glGetShaderiv(id, GL_INFO_LOG_LENGTH, &length); \ 82 | if (!status) { \ 83 | string log(length, 0); \ 84 | glGetShaderInfoLog(id, length, nullptr, &log[0]); \ 85 | cerr << "Line :" << __LINE__ << " OpenGL Shader Error: Log = \n" \ 86 | << &log[0]; \ 87 | glfwTerminate(); \ 88 | exit(EXIT_FAILURE); \ 89 | } \ 90 | } 91 | 92 | #define CHECK_GL_PROGRAM_ERROR(id) \ 93 | { \ 94 | GLint status = 0; \ 95 | GLint length = 0; \ 96 | glGetProgramiv(id, GL_LINK_STATUS, &status); \ 97 | glGetProgramiv(id, GL_INFO_LOG_LENGTH, &length); \ 98 | if (!status) { \ 99 | string log(length, 0); \ 100 | glGetProgramInfoLog(id, length, nullptr, &log[0]); \ 101 | cerr << "Line :" << __LINE__ << " OpenGL Program Error: Log = \n" \ 102 | << &log[0]; \ 103 | glfwTerminate(); \ 104 | exit(EXIT_FAILURE); \ 105 | } \ 106 | } 107 | 108 | #define CHECK_GL_ERROR(statement) \ 109 | { \ 110 | { statement; } \ 111 | GLenum error = GL_NO_ERROR; \ 112 | if ((error = glGetError()) != GL_NO_ERROR) { \ 113 | cerr << "Line :" << __LINE__ << " OpenGL Error: code = " << error \ 114 | << " description = " << OpenGlErrorToString(error); \ 115 | glfwTerminate(); \ 116 | exit(EXIT_FAILURE); \ 117 | } \ 118 | } 119 | 120 | #endif 121 | -------------------------------------------------------------------------------- /generate_views/include/render/Phong.h: -------------------------------------------------------------------------------- 1 | #ifndef PHONG_H 2 | #define PHONG_H 3 | 4 | #include 5 | 6 | #include 7 | 8 | #include "helpers/RandomUtils.h" 9 | #include "render/Program.h" 10 | 11 | struct PhongProgram: public Program { 12 | GLint obj_color_location; 13 | GLint eye_location; 14 | 15 | PhongProgram (glm::mat4* view_p, glm::mat4* proj_p) : 16 | Program(view_p, proj_p) { 17 | } 18 | 19 | virtual void setup(); 20 | void draw (const std::vector& vertices, 21 | const std::vector& faces, 22 | const std::vector& normals, 23 | const glm::mat4& model, const glm::vec4& color, 24 | const glm::vec4& eye); 25 | }; 26 | 27 | #endif 28 | -------------------------------------------------------------------------------- /generate_views/include/render/Program.h: -------------------------------------------------------------------------------- 1 | #ifndef PROGRAM_H 2 | #define PROGRAM_H 3 | 4 | #include 5 | 6 | #include 7 | 8 | struct Program { 9 | int vaoIndex; 10 | 11 | GLuint programId; 12 | 13 | GLint projection_matrix_location; 14 | GLint model_matrix_location; 15 | GLint view_matrix_location; 16 | 17 | const glm::mat4& view; 18 | const glm::mat4& proj; 19 | 20 | GLint light_position_location; 21 | 22 | Program (glm::mat4* view_p, glm::mat4* proj_p) : 23 | view(*view_p), proj(*proj_p) { } 24 | 25 | virtual void setup() = 0; 26 | }; 27 | 28 | #endif 29 | -------------------------------------------------------------------------------- /generate_views/src/helpers/RandomUtils.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | 9 | #include 10 | #include // scale 11 | #include // to_string 12 | 13 | 14 | using namespace std; 15 | using namespace glm; 16 | 17 | namespace glm { 18 | std::ostream& operator<<(std::ostream& os, const glm::vec2& v) { 19 | os << glm::to_string(v); 20 | return os; 21 | } 22 | 23 | std::ostream& operator<<(std::ostream& os, const glm::vec3& v) { 24 | os << glm::to_string(v); 25 | return os; 26 | } 27 | 28 | std::ostream& operator<<(std::ostream& os, const glm::vec4& v) { 29 | os << glm::to_string(v); 30 | return os; 31 | } 32 | 33 | std::ostream& operator<<(std::ostream& os, const glm::mat4& v) { 34 | for (int i = 0; i < 4; ++i) { 35 | for (int j = 0; j < 4; ++j) 36 | os << std::setprecision(3) << v[j][i] << "\t"; 37 | os << std::endl; 38 | } 39 | return os; 40 | } 41 | 42 | std::ostream& operator<<(std::ostream& os, const glm::mat3& v) { 43 | os << glm::to_string(v); 44 | return os; 45 | } 46 | } // namespace glm 47 | 48 | vector getVertexNormals (const vector& vertices, 49 | const vector& faces) { 50 | vector normals(vertices.size()); 51 | for (const uvec3& face: faces) { 52 | int v1 = face[0]; 53 | int v2 = face[1]; 54 | int v3 = face[2]; 55 | vec3 a = vec3(vertices[v1]); 56 | vec3 b = vec3(vertices[v2]); 57 | vec3 c = vec3(vertices[v3]); 58 | vec3 u = normalize(b - a); 59 | vec3 v = normalize(c - a); 60 | vec4 n = vec4(normalize(cross(u, v)), 0.0f); 61 | normals[v1] += n; 62 | normals[v2] += n; 63 | normals[v3] += n; 64 | } 65 | for (int i = 0; i < normals.size(); ++i) { 66 | normals[i] = normalize(normals[i]); 67 | normals[i][3] = 0.0f; 68 | } 69 | return normals; 70 | } 71 | 72 | void LoadOBJ(const string& file, 73 | vector& vertices, 74 | vector& faces, 75 | vector& normals) { 76 | vertices.clear(); 77 | faces.clear(); 78 | normals.clear(); 79 | fstream ifs(file); 80 | string buffer; 81 | while (getline(ifs, buffer)) { 82 | stringstream ss(buffer); 83 | char type; 84 | ss >> type; 85 | if (type == 'f') { 86 | int i, j, k; 87 | ss >> i >> j >> k; 88 | faces.push_back(uvec3(i-1, j-1, k-1)); 89 | } 90 | else if (type == 'v') { 91 | float x, y, z; 92 | ss >> x >> y >> z; 93 | vertices.push_back(vec4(x, y, z, 1.0)); 94 | } 95 | } 96 | normals = getVertexNormals(vertices, faces); 97 | cout << "Loaded " << file << " successfully." << endl; 98 | cout << "\t" << vertices.size() << " vertices." << endl; 99 | cout << "\t" << faces.size() << " faces." << endl; 100 | } 101 | 102 | void LoadOBJWithNormals(const string& file, 103 | vector& vertices, 104 | vector& faces, 105 | vector& normals) { 106 | vertices.clear(); 107 | faces.clear(); 108 | normals.clear(); 109 | ifstream ifs(file); 110 | string buffer; 111 | while (getline(ifs, buffer)) { 112 | string type; 113 | stringstream ss(buffer); 114 | ss >> type; 115 | if (type == "v") { 116 | double x, y, z; 117 | ss >> x >> y >> z; 118 | vertices.push_back(vec4(x, y, z, 1.0)); 119 | } 120 | else if (type == "vn"){ 121 | double x, y, z; 122 | ss >> x >> y >> z; 123 | normals.push_back(vec4(x, y, z, 1.0)); 124 | } 125 | else if (type == "f") { 126 | // of the form 22283//22283 127 | uvec3 idx; 128 | for (int i = 0; i < 3; ++i) { 129 | string tmp; 130 | ss >> tmp; 131 | for (int j = 0; j < tmp.size() && tmp[j] != '/'; ++j) { 132 | idx[i] = idx[i] * 10 + int(tmp[j]) - '0'; 133 | } 134 | idx[i] -= 1; 135 | } 136 | faces.push_back(idx); 137 | } 138 | } 139 | cout << "Loaded " << file << " successfully." << endl; 140 | cout << "\t" << vertices.size() << " vertices." << endl; 141 | cout << "\t" << faces.size() << " faces." << endl; 142 | } 143 | 144 | string loadShader (const string& filename) { 145 | cout << "Loading shader: " << filename; 146 | ifstream file(filename); 147 | cout << (file.fail() ? " failed." : " succeeded.") << endl; 148 | stringstream buffer; 149 | buffer << file.rdbuf(); 150 | return buffer.str(); 151 | } 152 | 153 | void fixSphereVertices (vector& sphere_vertices) { 154 | mat4 T = translate(vec3(0.0, -1.0, 0.0)); 155 | mat4 S = scale(vec3(10.0, 10.0, 10.0)); 156 | for (vec4& vertex: sphere_vertices) 157 | vertex = T * S * vertex; 158 | } 159 | 160 | struct vec4_sort { 161 | bool operator() (const vec4& a, const vec4& b) const { 162 | for (int i = 0; i < 4; ++i) { 163 | if (a[i] == b[i]) 164 | continue; 165 | return a[i] < b[i]; 166 | } 167 | return false; 168 | } 169 | }; 170 | 171 | void fixDuplicateVertices (vector& vertices, vector& faces) { 172 | vector v; 173 | vector f; 174 | 175 | map dupeVertCheck; 176 | for (const vec4& vert : vertices) { 177 | if (dupeVertCheck.find(vert) == dupeVertCheck.end()) { 178 | dupeVertCheck[vert] = dupeVertCheck.size(); 179 | v.push_back(vert); 180 | } 181 | } 182 | 183 | for (const uvec3& face : faces) { 184 | uvec3 new_face; 185 | for (int i = 0; i < 3; ++i) 186 | new_face[i] = dupeVertCheck[vertices[face[i]]]; 187 | f.push_back(new_face); 188 | } 189 | 190 | cout << "Removed: " << (vertices.size() - v.size()) << " vertices." << endl; 191 | 192 | vertices = v; 193 | faces = f; 194 | } 195 | -------------------------------------------------------------------------------- /generate_views/src/main.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | 6 | #include 7 | 8 | #include 9 | 10 | #include 11 | 12 | #include 13 | 14 | #include 15 | #include 16 | #include 17 | 18 | #include "helpers/RandomUtils.h" 19 | 20 | #include "render/OpenGLStuff.h" 21 | #include "render/Phong.h" 22 | 23 | using namespace std; 24 | using namespace glm; 25 | using namespace Eigen; 26 | 27 | string FILES_TO_PROCESS = "first argument"; 28 | string OUT_DIR = "second argument"; 29 | 30 | PhongProgram phongP(&view_matrix, &projection_matrix); 31 | 32 | vector object_vertices; 33 | vector object_faces; 34 | vector object_normals; 35 | 36 | void translate(vector &points, const vec4 &translation) { 37 | for (vec4 &point : points) 38 | point += translation; 39 | } 40 | 41 | void scale(vector &points, float factor) { 42 | for (vec4 &point : points) { 43 | for (int i = 0; i < 3; i++) 44 | point[i] *= factor; 45 | } 46 | } 47 | 48 | vector min_bounds(const vector &points) { 49 | vector result(3); 50 | for (int i = 0; i < 3; i++) 51 | result[i] = points[0][i]; 52 | for (const vec4 &point : points) { 53 | for (int i = 0; i < 3; i++) 54 | result[i] = std::min(result[i], point[i]); 55 | } 56 | return result; 57 | } 58 | 59 | vector max_bounds(const vector &points) { 60 | vector result(3); 61 | for (int i = 0; i < 3; i++) 62 | result[i] = points[0][i]; 63 | for (const vec4 &point : points) { 64 | for (int i = 0; i < 3; i++) 65 | result[i] = std::max(result[i], point[i]); 66 | } 67 | return result; 68 | } 69 | 70 | // Resizes the bounding box of points into bounding box with max length 1.0. 71 | void expand(vector &points) { 72 | vector min_vals = min_bounds(points); 73 | vector max_vals = max_bounds(points); 74 | int axis = 0; 75 | for (int i = 0; i < 3; i++) { 76 | if (max_vals[i] - min_vals[i] > max_vals[axis] - min_vals[axis]) 77 | axis = i; 78 | } 79 | float factor = 1.0 / (max_vals[axis] - min_vals[axis]); 80 | scale(points, factor); 81 | } 82 | 83 | void preprocess(vector &points) { 84 | // Modelnet10 faces from the top. Rotate by 90 degrees. 85 | for (vec4 &point : points) { 86 | float y = point[1]; 87 | float z = point[2]; 88 | point[1] = z; 89 | point[2] = y; 90 | } 91 | 92 | // Translate bounding box corner to origin. 93 | vector min_vals = min_bounds(points); 94 | vec4 translation(-min_vals[0], -min_vals[1], -min_vals[2], 0.0f); 95 | translate(points, translation); 96 | 97 | // Make the bounding box touch the unit cube. 98 | expand(points); 99 | 100 | // Translate so bounding box center is at the origin. 101 | translate(points, vec4(-0.5f, -0.5f, -0.5f, 0.0f)); 102 | } 103 | 104 | void writePNG(vector &in, const char *fn, int w, int h) { 105 | png::image< png::rgb_pixel > image(w, h); 106 | int offset = 0; 107 | for (size_t y = 0; y < image.get_height(); ++y) { 108 | for (size_t x = 0; x < image.get_width(); ++x) { 109 | image[h-y-1][x] = png::rgb_pixel(in[offset+0], 110 | in[offset+1], 111 | in[offset+2]); 112 | offset += 3; 113 | } 114 | } 115 | image.write(fn); 116 | } 117 | 118 | void generate(string &filename, float r, int theta_samples, int phi_samples) { 119 | cout << filename << endl; 120 | 121 | float d_phi = (M_PI) / (float) (phi_samples + 1); 122 | float d_theta = (2 * M_PI) / (float) (theta_samples); 123 | 124 | for (int i = 1; i <= phi_samples; i++) { 125 | for (int j = 1; j <= theta_samples; j++) { 126 | float phi = i * d_phi; 127 | float theta = j * d_theta; 128 | 129 | vec3 eye_pos = vec3(r * sin(phi) * cos(theta), 130 | r * cos(phi), 131 | r * sin(phi) * sin(theta)); 132 | 133 | // Pass in the new eye position. 134 | if (!keepLoopingOpenGL(eye_pos)) 135 | return; 136 | 137 | phongP.draw(object_vertices, object_faces, object_normals, mat4(), 138 | RED, vec4(eye, 1.0f)); 139 | 140 | endLoopOpenGL(); 141 | 142 | // Must be done after buffers are swapped. 143 | vector pixels(3 * window_width * window_height); 144 | glReadPixels(0, 0, window_width, window_height, GL_RGB, 145 | GL_UNSIGNED_BYTE, &pixels[0]); 146 | 147 | stringstream ss; 148 | ss << OUT_DIR << filename << "_" << i << "_" << j << ".png"; 149 | writePNG(pixels, ss.str().c_str(), window_width, window_height); 150 | } 151 | } 152 | } 153 | 154 | // Uses libigl to read into an Eigen matrix then converts back to glm. 155 | void readMesh(string filename, bool is_obj) { 156 | MatrixXd vertices; 157 | MatrixXi faces; 158 | MatrixXd normals; 159 | 160 | if (is_obj) 161 | igl::readOBJ(filename, vertices, faces); 162 | else 163 | igl::readOFF(filename, vertices, faces); 164 | 165 | igl::per_vertex_normals(vertices, faces, normals); 166 | 167 | object_vertices.clear(); 168 | object_faces.clear(); 169 | object_normals.clear(); 170 | 171 | for (int i = 0; i < vertices.rows(); i++) { 172 | vec4 tmp; 173 | for (int j = 0; j < 3; j++) 174 | tmp[j] = vertices(i, j); 175 | tmp[3] = 1.0f; 176 | object_vertices.push_back(tmp); 177 | } 178 | 179 | for (int i = 0; i < faces.rows(); i++) { 180 | vec3 tmp; 181 | for (int j = 0; j < 3; j++) 182 | tmp[j] = faces(i, j); 183 | object_faces.push_back(tmp); 184 | } 185 | 186 | for (int i = 0; i < normals.rows(); i++) { 187 | vec4 tmp; 188 | for (int j = 0; j < 3; j++) 189 | tmp[j] = normals(i, j); 190 | object_normals.push_back(tmp); 191 | } 192 | } 193 | 194 | int main (int argc, char* argv[]) { 195 | FILES_TO_PROCESS = argv[1]; 196 | OUT_DIR = argv[2]; 197 | 198 | ifstream process_file(FILES_TO_PROCESS); 199 | 200 | // OpenGL Setup. 201 | initOpenGL(); 202 | phongP.setup(); 203 | 204 | string buffer; 205 | while (getline(process_file, buffer)) { 206 | stringstream ss(buffer); 207 | string content = ss.str(); 208 | 209 | // Set boolean to false if the mesh is an off file. 210 | readMesh(content, false); 211 | preprocess(object_vertices); 212 | 213 | // Generate all viewpoints. 214 | generate(content, 3.0f, 5, 5); 215 | } 216 | 217 | cleanupOpenGL(); 218 | } 219 | -------------------------------------------------------------------------------- /generate_views/src/render/OpenGLStuff.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | 5 | #include 6 | 7 | #include 8 | #include 9 | #include 10 | 11 | #include "render/Program.h" 12 | #include "render/OpenGLStuff.h" 13 | #include "helpers/RandomUtils.h" 14 | 15 | using namespace std; 16 | 17 | enum { 18 | kMouseModeCamera, 19 | kNumMouseModes 20 | }; 21 | 22 | const std::string window_title = "Collision Detection"; 23 | 24 | GLuint array_objects[kNumVaos]; 25 | GLuint buffer_objects[kNumVaos][kNumVbos]; 26 | 27 | glm::vec4 LIGHT_POSITION = glm::vec4(10.0f, 10.0f, 10.0f, 1.0f); 28 | 29 | const float kNear = 1.0f; 30 | const float kFar = 100.0f; 31 | const float kFov = 45.0f; 32 | float camera_distance = 2.0f; 33 | 34 | glm::vec3 eye = glm::vec3(5.0f, 5.0f, 5.0f); 35 | glm::vec3 center = glm::vec3(0.0f, 0.0f, 0.0f); 36 | 37 | glm::vec3 up = glm::vec3(0.0f, 1.0f, 0.0f); 38 | glm::vec3 look = eye - center; 39 | glm::vec3 tangent = glm::cross(up, look); 40 | glm::mat3 orientation = glm::mat3(tangent, up, look); 41 | 42 | glm::mat4 view_matrix; 43 | glm::mat4 projection_matrix; 44 | 45 | const int WIDTH = 112; 46 | const int HEIGHT = 112; 47 | int window_width = WIDTH; 48 | int window_height = HEIGHT; 49 | 50 | bool do_action = false; 51 | 52 | GLFWwindow* window; 53 | 54 | const char* OpenGlErrorToString(GLenum error) { 55 | switch (error) { 56 | case GL_NO_ERROR: 57 | return "GL_NO_ERROR"; 58 | break; 59 | case GL_INVALID_ENUM: 60 | return "GL_INVALID_ENUM"; 61 | break; 62 | case GL_INVALID_VALUE: 63 | return "GL_INVALID_VALUE"; 64 | break; 65 | case GL_INVALID_OPERATION: 66 | return "GL_INVALID_OPERATION"; 67 | break; 68 | case GL_OUT_OF_MEMORY: 69 | return "GL_OUT_OF_MEMORY"; 70 | break; 71 | default: 72 | return "Unknown Error"; 73 | break; 74 | } 75 | return "Unknown Error"; 76 | } 77 | 78 | GLuint setupShader (const char* shaderName, GLenum shaderType) { 79 | GLuint shaderId = 0; 80 | CHECK_GL_ERROR(shaderId = glCreateShader(shaderType)); 81 | CHECK_GL_ERROR(glShaderSource(shaderId, 1, &shaderName, nullptr)); 82 | glCompileShader(shaderId); 83 | CHECK_GL_SHADER_ERROR(shaderId); 84 | return shaderId; 85 | } 86 | 87 | void ErrorCallback (int error, const char* description) { 88 | cerr << "GLFW Error: " << description << "\n"; 89 | } 90 | 91 | void KeyCallback (GLFWwindow* window, int key, int scancode, int action, int mods) { 92 | do_action = false; 93 | if (key == GLFW_KEY_ESCAPE && action == GLFW_PRESS) 94 | glfwSetWindowShouldClose(window, GL_TRUE); 95 | else if (action == GLFW_PRESS && key == GLFW_KEY_SPACE) { 96 | do_action = true; 97 | } 98 | } 99 | 100 | void initOpenGL () { 101 | if (!glfwInit()) 102 | exit(EXIT_FAILURE); 103 | 104 | glfwSetErrorCallback(ErrorCallback); 105 | glfwWindowHint(GLFW_CONTEXT_VERSION_MAJOR, 3); 106 | glfwWindowHint(GLFW_CONTEXT_VERSION_MINOR, 3); 107 | glfwWindowHint(GLFW_OPENGL_FORWARD_COMPAT, GL_TRUE); 108 | glfwWindowHint(GLFW_OPENGL_PROFILE, GLFW_OPENGL_CORE_PROFILE); 109 | glfwWindowHint(GLFW_SAMPLES, 4); 110 | 111 | window = glfwCreateWindow(window_width, window_height, 112 | &window_title[0], nullptr, nullptr); 113 | 114 | CHECK_SUCCESS(window != nullptr); 115 | 116 | glfwMakeContextCurrent(window); 117 | CHECK_SUCCESS(glewInit() == GLEW_OK); 118 | glGetError(); 119 | 120 | glfwSetKeyCallback(window, KeyCallback); 121 | glfwSwapInterval(1); 122 | 123 | cout << "Renderer: " << glGetString(GL_RENDERER) << "\n"; 124 | cout << "OpenGL version supported:" << glGetString(GL_VERSION) << "\n"; 125 | 126 | // Setup our VAOs. 127 | CHECK_GL_ERROR(glGenVertexArrays(kNumVaos, array_objects)); 128 | 129 | // Generate buffer objects 130 | for (int i = 0; i < kNumVaos; ++i) 131 | CHECK_GL_ERROR(glGenBuffers(kNumVbos, &buffer_objects[i][0])); 132 | 133 | glClearColor(1.0f, 1.0f, 1.0f, 1.0f); 134 | 135 | glEnable(GL_DEPTH_TEST); 136 | glEnable(GL_MULTISAMPLE); 137 | glEnable(GL_BLEND); 138 | glEnable(GL_LINE_SMOOTH); 139 | glEnable(GL_POLYGON_SMOOTH); 140 | 141 | glDepthFunc(GL_LESS); 142 | glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA); 143 | } 144 | 145 | bool keepLoopingOpenGL (glm::vec3 eye_pos) { 146 | if (glfwWindowShouldClose(window)) 147 | return false; 148 | 149 | glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); 150 | glfwGetFramebufferSize(window, &window_width, &window_height); 151 | glViewport(0, 0, window_width, window_height); 152 | 153 | float fovy = static_cast(kFov * (M_PI / 180.0f)); 154 | float aspect = static_cast(window_width) / window_height; 155 | 156 | LIGHT_POSITION = glm::vec4(eye_pos * 2.0f, 1.0f); 157 | 158 | view_matrix = glm::lookAt(eye_pos, center, up); 159 | projection_matrix = glm::perspective(fovy, aspect, kNear, kFar); 160 | 161 | return true; 162 | } 163 | 164 | void endLoopOpenGL () { 165 | glfwPollEvents(); 166 | glfwSwapBuffers(window); 167 | } 168 | 169 | void cleanupOpenGL () { 170 | glfwDestroyWindow(window); 171 | glfwTerminate(); 172 | exit(EXIT_SUCCESS); 173 | } 174 | -------------------------------------------------------------------------------- /generate_views/src/render/Phong.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | 4 | #include 5 | 6 | #include 7 | 8 | #include "helpers/RandomUtils.h" 9 | #include "render/OpenGLStuff.h" 10 | #include "render/Phong.h" 11 | 12 | using namespace std; 13 | 14 | void PhongProgram::setup () { 15 | this->vaoIndex = kPhongVao; 16 | 17 | // shaders 18 | string vertex_shader = loadShader(PHONG_VERT); 19 | GLuint vertex_shader_id = setupShader(vertex_shader.c_str(), GL_VERTEX_SHADER); 20 | 21 | string geometry_shader = loadShader(PHONG_GEOM); 22 | GLuint geometry_shader_id = setupShader(geometry_shader.c_str(), GL_GEOMETRY_SHADER); 23 | 24 | string fragment_shader = loadShader(PHONG_FRAG); 25 | GLuint fragment_shader_id = setupShader(fragment_shader.c_str(), GL_FRAGMENT_SHADER); 26 | 27 | // program 28 | GLuint& program_id = this->programId; 29 | CHECK_GL_ERROR(program_id = glCreateProgram()); 30 | CHECK_GL_ERROR(glAttachShader(program_id, vertex_shader_id)); 31 | CHECK_GL_ERROR(glAttachShader(program_id, geometry_shader_id)); 32 | CHECK_GL_ERROR(glAttachShader(program_id, fragment_shader_id)); 33 | 34 | // attributes 35 | CHECK_GL_ERROR(glBindAttribLocation(program_id, 0, "vertex_position")); 36 | CHECK_GL_ERROR(glBindAttribLocation(program_id, 1, "vertex_normal")); 37 | CHECK_GL_ERROR(glBindFragDataLocation(program_id, 0, "fragment_color")); 38 | 39 | glLinkProgram(program_id); 40 | CHECK_GL_PROGRAM_ERROR(program_id); 41 | 42 | // uniforms 43 | GLint& projection_matrix_location = this->projection_matrix_location; 44 | CHECK_GL_ERROR(projection_matrix_location = glGetUniformLocation(program_id, "projection")); 45 | GLint& model_matrix_location = this->model_matrix_location; 46 | CHECK_GL_ERROR(model_matrix_location = glGetUniformLocation(program_id, "model")); 47 | GLint& view_matrix_location = this->view_matrix_location; 48 | CHECK_GL_ERROR(view_matrix_location = glGetUniformLocation(program_id, "view")); 49 | 50 | GLint& light_position_location = this->light_position_location; 51 | CHECK_GL_ERROR(light_position_location = glGetUniformLocation(program_id, "light_position")); 52 | 53 | GLint& obj_color_location = this->obj_color_location; 54 | CHECK_GL_ERROR(obj_color_location = glGetUniformLocation(program_id, "obj_color")); 55 | 56 | GLint& eye_location = this->eye_location; 57 | CHECK_GL_ERROR(eye_location = glGetUniformLocation(program_id, "eye")); 58 | } 59 | 60 | void PhongProgram::draw (const vector& vertices, 61 | const vector& faces, 62 | const vector& normals, 63 | const glm::mat4& model, const glm::vec4& color, 64 | const glm::vec4& eye) { 65 | CHECK_GL_ERROR(glUseProgram(this->programId)); 66 | 67 | CHECK_GL_ERROR(glUniformMatrix4fv(this->model_matrix_location, 1, GL_FALSE, 68 | &model[0][0])); 69 | CHECK_GL_ERROR(glUniformMatrix4fv(this->view_matrix_location, 1, GL_FALSE, 70 | &this->view[0][0])); 71 | CHECK_GL_ERROR(glUniformMatrix4fv(this->projection_matrix_location, 1, 72 | GL_FALSE, &this->proj[0][0])); 73 | 74 | CHECK_GL_ERROR(glUniform4fv(this->light_position_location, 1, 75 | &LIGHT_POSITION[0])); 76 | 77 | CHECK_GL_ERROR(glUniform4fv(this->obj_color_location, 1, &color[0])); 78 | CHECK_GL_ERROR(glUniform4fv(this->eye_location, 1, &eye[0])); 79 | 80 | CHECK_GL_ERROR(glBindVertexArray(array_objects[this->vaoIndex])); 81 | 82 | CHECK_GL_ERROR(glBindBuffer(GL_ARRAY_BUFFER, 83 | buffer_objects[this->vaoIndex][kVertexBuffer])); 84 | CHECK_GL_ERROR(glBufferData(GL_ARRAY_BUFFER, 85 | sizeof(float) * vertices.size() * 4, 86 | &vertices[0], GL_STATIC_DRAW)); 87 | 88 | CHECK_GL_ERROR(glVertexAttribPointer(0, 4, GL_FLOAT, GL_FALSE, 0, 0)); 89 | CHECK_GL_ERROR(glEnableVertexAttribArray(0)); 90 | 91 | CHECK_GL_ERROR(glBindBuffer(GL_ARRAY_BUFFER, 92 | buffer_objects[this->vaoIndex][kVertexNormalBuffer])); 93 | CHECK_GL_ERROR(glBufferData(GL_ARRAY_BUFFER, 94 | sizeof(float) * normals.size() * 4, 95 | &normals[0], GL_STATIC_DRAW)); 96 | 97 | CHECK_GL_ERROR(glVertexAttribPointer(1, 4, GL_FLOAT, GL_FALSE, 0, 0)); 98 | CHECK_GL_ERROR(glEnableVertexAttribArray(1)); 99 | 100 | CHECK_GL_ERROR(glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, 101 | buffer_objects[this->vaoIndex][kIndexBuffer])); 102 | CHECK_GL_ERROR(glBufferData(GL_ELEMENT_ARRAY_BUFFER, 103 | sizeof(uint32_t) * faces.size() * 3, 104 | &faces[0], GL_STATIC_DRAW)); 105 | 106 | CHECK_GL_ERROR(glDrawElements(GL_TRIANGLES, faces.size() * 3, 107 | GL_UNSIGNED_INT, 0)); 108 | } 109 | -------------------------------------------------------------------------------- /generate_views/src/render/shaders/phong.frag: -------------------------------------------------------------------------------- 1 | #version 330 core 2 | 3 | uniform vec4 obj_color; 4 | uniform vec4 eye; 5 | 6 | in vec4 world_pos; 7 | in vec4 normal; 8 | in vec4 light; 9 | 10 | out vec4 fragment_color; 11 | 12 | void main() { 13 | vec4 to_eye = normalize(eye - world_pos); 14 | float dot_nl = dot(normalize(light), normalize(normal)); 15 | dot_nl = max(dot_nl, -dot_nl); 16 | 17 | vec4 H = normalize(normalize(light) + to_eye); 18 | 19 | float KS = pow(clamp(dot(normalize(normal), H), 0.0f, 1.0f), 100); 20 | float KD = clamp(dot_nl, 0.0f, 1.0f); 21 | 22 | fragment_color = clamp(vec4((KD + KS) * obj_color.xyz, obj_color.w), 23 | 0.0f, 1.0f); 24 | } 25 | -------------------------------------------------------------------------------- /generate_views/src/render/shaders/phong.geom: -------------------------------------------------------------------------------- 1 | #version 330 core 2 | 3 | layout (triangles) in; 4 | layout (triangle_strip, max_vertices = 3) out; 5 | 6 | uniform mat4 projection; 7 | uniform mat4 model; 8 | uniform mat4 view; 9 | 10 | in vec4 vs_light_direction[]; 11 | in vec4 vs_vertex_normal[]; 12 | in vec4 vs_world_pos[]; 13 | 14 | out vec4 normal; 15 | out vec4 light; 16 | out vec4 world_pos; 17 | 18 | void main() { 19 | for (int i = 0; i < gl_in.length(); i++) { 20 | normal = model * vs_vertex_normal[i]; 21 | light = vs_light_direction[i]; 22 | world_pos = vs_world_pos[i]; 23 | gl_Position = projection * view * model * gl_in[i].gl_Position; 24 | EmitVertex(); 25 | } 26 | EndPrimitive(); 27 | } 28 | -------------------------------------------------------------------------------- /generate_views/src/render/shaders/phong.vert: -------------------------------------------------------------------------------- 1 | #version 330 core 2 | 3 | uniform vec4 light_position; 4 | 5 | in vec4 vertex_position; 6 | in vec4 vertex_normal; 7 | 8 | out vec4 vs_light_direction; 9 | out vec4 vs_vertex_normal; 10 | out vec4 vs_world_pos; 11 | 12 | void main() { 13 | gl_Position = vertex_position; 14 | vs_light_direction = light_position - gl_Position; 15 | vs_vertex_normal = vertex_normal; 16 | vs_world_pos = vertex_position; 17 | } 18 | -------------------------------------------------------------------------------- /globals.py: -------------------------------------------------------------------------------- 1 | import os 2 | import time 3 | import numpy as np 4 | 5 | import geometry_processing 6 | 7 | 8 | PACKAGE_PATH = os.path.dirname(geometry_processing.__file__) 9 | 10 | IMAGE_SIZE = 224 11 | BATCH = 64 12 | NUM_CLASSES = 10 13 | 14 | IMAGE_MEAN = np.load(os.path.join(PACKAGE_PATH, "cache/image_mean.npy")) 15 | IMAGE_STD = np.load(os.path.join(PACKAGE_PATH, "cache/image_std.npy")) 16 | 17 | FC2_MEAN = np.load(os.path.join(PACKAGE_PATH, "cache/fc2_mean.npy")) 18 | FC2_STD = np.load(os.path.join(PACKAGE_PATH, "cache/fc2_std.npy")) 19 | 20 | MODEL_WEIGHTS = os.path.join(PACKAGE_PATH, "cache", "model_weights_V2.h5") 21 | SALIENCY_MODEL = os.path.join(PACKAGE_PATH, "cache", "saliency_model.h5") 22 | 23 | LOG_FILE = os.path.join(PACKAGE_PATH, "logs", 24 | "training_%s.log" % time.strftime("%m_%d_%H_%M")) 25 | 26 | SALIENCY_DATA_TRAIN = os.path.join(PACKAGE_PATH, "data", "saliency_labels_train.txt") 27 | SALIENCY_DATA_VALID = os.path.join(PACKAGE_PATH, "data", "saliency_labels_valid.txt") 28 | 29 | ################################################################################ 30 | # Machine Specific Paths. 31 | ################################################################################ 32 | # Local mac machine paths. 33 | TRAIN_DIR = "/Users/bradyzhou/code/data/ModelNetViewpoints/train/" 34 | VALID_DIR = "/Users/bradyzhou/code/data/ModelNetViewpoints/test/" 35 | 36 | # Local unix machine paths. 37 | # TRAIN_DIR = "/home/brady/code/data/ModelNetViewpoints/train/" 38 | # VALID_DIR = "/home/brady/code/data/ModelNetViewpoints/test/" 39 | 40 | # TACC supercomputer information. 41 | # TRAIN_DIR = "/home/04365/bradyz/data/ModelNetViewpoints/train/" 42 | # VALID_DIR = "/home/04365/bradyz/data/ModelNetViewpoints/test/" 43 | ################################################################################ 44 | -------------------------------------------------------------------------------- /logs/10_15_20_25.log: -------------------------------------------------------------------------------- 1 | k: 10 2 | Samples Processed 0/1000 3 | Samples Processed 64/1000 4 | Samples Processed 128/1000 5 | Samples Processed 192/1000 6 | Samples Processed 256/1000 7 | Samples Processed 320/1000 8 | Samples Processed 384/1000 9 | Samples Processed 448/1000 10 | Samples Processed 512/1000 11 | Samples Processed 576/1000 12 | Samples Processed 640/1000 13 | Samples Processed 704/1000 14 | Samples Processed 768/1000 15 | Samples Processed 832/1000 16 | Samples Processed 896/1000 17 | Samples Processed 960/1000 18 | Saving to /home/04365/bradyz/code/geometry_processing/cache/svm_top_k_10_confusion.npy. 19 | Accuracy 0.9004 20 | k: 15 21 | Samples Processed 0/1000 22 | Samples Processed 64/1000 23 | Samples Processed 128/1000 24 | Samples Processed 192/1000 25 | Samples Processed 256/1000 26 | Samples Processed 320/1000 27 | Samples Processed 384/1000 28 | Samples Processed 448/1000 29 | Samples Processed 512/1000 30 | Samples Processed 576/1000 31 | Samples Processed 640/1000 32 | Samples Processed 704/1000 33 | Samples Processed 768/1000 34 | Samples Processed 832/1000 35 | Samples Processed 896/1000 36 | Samples Processed 960/1000 37 | Saving to /home/04365/bradyz/code/geometry_processing/cache/svm_top_k_15_confusion.npy. 38 | Accuracy 0.8760 39 | k: 20 40 | Samples Processed 0/1000 41 | Samples Processed 64/1000 42 | Samples Processed 128/1000 43 | Samples Processed 192/1000 44 | Samples Processed 256/1000 45 | Samples Processed 320/1000 46 | Samples Processed 384/1000 47 | Samples Processed 448/1000 48 | Samples Processed 512/1000 49 | Samples Processed 576/1000 50 | Samples Processed 640/1000 51 | Samples Processed 704/1000 52 | Samples Processed 768/1000 53 | Samples Processed 832/1000 54 | Samples Processed 896/1000 55 | Samples Processed 960/1000 56 | Saving to /home/04365/bradyz/code/geometry_processing/cache/svm_top_k_20_confusion.npy. 57 | Accuracy 0.8916 58 | k: 25 59 | Samples Processed 0/1000 60 | Samples Processed 64/1000 61 | Samples Processed 128/1000 62 | Samples Processed 192/1000 63 | Samples Processed 256/1000 64 | Samples Processed 320/1000 65 | Samples Processed 384/1000 66 | Samples Processed 448/1000 67 | Samples Processed 512/1000 68 | Samples Processed 576/1000 69 | Samples Processed 640/1000 70 | Samples Processed 704/1000 71 | Samples Processed 768/1000 72 | Samples Processed 832/1000 73 | Samples Processed 896/1000 74 | Samples Processed 960/1000 75 | Saving to /home/04365/bradyz/code/geometry_processing/cache/svm_top_k_25_confusion.npy. 76 | Accuracy 0.8154 77 | -------------------------------------------------------------------------------- /logs/1_3_5_7.log: -------------------------------------------------------------------------------- 1 | k: 1 2 | Samples Processed 0/1000 3 | Samples Processed 64/1000 4 | Samples Processed 128/1000 5 | Samples Processed 192/1000 6 | Samples Processed 256/1000 7 | Samples Processed 320/1000 8 | Samples Processed 384/1000 9 | Samples Processed 448/1000 10 | Samples Processed 512/1000 11 | Samples Processed 576/1000 12 | Samples Processed 640/1000 13 | Samples Processed 704/1000 14 | Samples Processed 768/1000 15 | Samples Processed 832/1000 16 | Samples Processed 896/1000 17 | Samples Processed 960/1000 18 | Saving to /home/04365/bradyz/code/geometry_processing/cache/svm_top_k_1_confusion.npy. 19 | Accuracy 0.8643 20 | k: 3 21 | Samples Processed 0/1000 22 | Samples Processed 64/1000 23 | Samples Processed 128/1000 24 | Samples Processed 192/1000 25 | Samples Processed 256/1000 26 | Samples Processed 320/1000 27 | Samples Processed 384/1000 28 | Samples Processed 448/1000 29 | Samples Processed 512/1000 30 | Samples Processed 576/1000 31 | Samples Processed 640/1000 32 | Samples Processed 704/1000 33 | Samples Processed 768/1000 34 | Samples Processed 832/1000 35 | Samples Processed 896/1000 36 | Samples Processed 960/1000 37 | Saving to /home/04365/bradyz/code/geometry_processing/cache/svm_top_k_3_confusion.npy. 38 | Accuracy 0.8828 39 | k: 5 40 | Samples Processed 0/1000 41 | Samples Processed 64/1000 42 | Samples Processed 128/1000 43 | Samples Processed 192/1000 44 | Samples Processed 256/1000 45 | Samples Processed 320/1000 46 | Samples Processed 384/1000 47 | Samples Processed 448/1000 48 | Samples Processed 512/1000 49 | Samples Processed 576/1000 50 | Samples Processed 640/1000 51 | Samples Processed 704/1000 52 | Samples Processed 768/1000 53 | Samples Processed 832/1000 54 | Samples Processed 896/1000 55 | Samples Processed 960/1000 56 | Saving to /home/04365/bradyz/code/geometry_processing/cache/svm_top_k_5_confusion.npy. 57 | Accuracy 0.8701 58 | k: 7 59 | Samples Processed 0/1000 60 | Samples Processed 64/1000 61 | Samples Processed 128/1000 62 | Samples Processed 192/1000 63 | Samples Processed 256/1000 64 | Samples Processed 320/1000 65 | Samples Processed 384/1000 66 | Samples Processed 448/1000 67 | Samples Processed 512/1000 68 | Samples Processed 576/1000 69 | Samples Processed 640/1000 70 | Samples Processed 704/1000 71 | Samples Processed 768/1000 72 | Samples Processed 832/1000 73 | Samples Processed 896/1000 74 | Samples Processed 960/1000 75 | Saving to /home/04365/bradyz/code/geometry_processing/cache/svm_top_k_7_confusion.npy. 76 | Accuracy 0.8896 77 | -------------------------------------------------------------------------------- /logs/saliency.log: -------------------------------------------------------------------------------- 1 | epoch,acc,loss,val_acc,val_loss 2 | 0,0.830933087291,9.02100773555,0.857291666667,6.61597630183 3 | 1,0.871700497433,4.95146221756,0.842708333333,3.69922366142 4 | 2,0.86737804878,2.79606470248,0.840625,2.16024055481 5 | 3,0.866114409499,1.64134978965,0.845833333333,1.32317822774 6 | -------------------------------------------------------------------------------- /logs/saliency_log_4_27.log: -------------------------------------------------------------------------------- 1 | epoch,acc,loss,val_acc,val_loss 2 | 0,0.877908376123,5.03884567131,0.277083333333,5.07790384293 3 | 1,0.882652037869,2.82252467597,0.270833333333,3.58448673884 4 | 2,0.886563302311,1.64146175931,0.302083333333,2.86326626142 5 | 3,0.890414393453,1.01145571023,0.31875,2.28957502047 6 | 4,0.892751123235,0.671413788613,0.295833333333,2.14315556685 7 | 5,0.895298459564,0.486767710231,0.3125,2.08609606425 8 | -------------------------------------------------------------------------------- /logs/top_1.log: -------------------------------------------------------------------------------- 1 | batch,train_acc,val_acc 2 | 0,0.9375,0.7500 3 | 1,0.9062,0.8125 4 | 2,0.9688,0.7812 5 | 3,0.9375,0.7500 6 | 4,1.0000,0.8750 7 | 5,1.0000,0.7812 8 | 6,1.0000,0.7188 9 | 7,1.0000,0.7812 10 | 8,1.0000,0.9062 11 | 9,0.9375,0.7812 12 | 10,0.9375,0.8750 13 | 11,1.0000,0.8750 14 | 12,0.9375,0.8125 15 | 13,0.9688,0.8438 16 | 14,1.0000,0.7812 17 | 15,0.9688,0.8438 18 | 16,0.9688,0.7500 19 | 17,1.0000,0.8125 20 | 18,0.9062,0.9688 21 | 19,0.9375,0.8750 22 | 20,0.9375,0.7812 23 | 21,0.9688,0.8438 24 | 22,1.0000,0.8125 25 | 23,1.0000,0.8125 26 | 24,0.9688,0.9062 27 | 25,1.0000,0.7812 28 | 26,1.0000,0.7812 29 | 27,0.9375,0.7500 30 | 28,1.0000,0.8750 31 | 29,0.9688,0.8125 32 | 30,0.9375,0.9688 33 | 31,0.9062,0.9062 34 | 32,1.0000,0.8750 35 | 33,1.0000,0.9688 36 | 34,0.9688,0.8438 37 | 35,1.0000,0.8750 38 | 36,0.9688,0.7500 39 | 37,1.0000,0.7812 40 | 38,1.0000,0.9062 41 | 39,1.0000,0.8125 42 | 40,1.0000,0.8125 43 | 41,1.0000,0.9062 44 | 42,1.0000,0.8125 45 | 43,1.0000,0.8750 46 | 44,0.9688,0.8125 47 | 45,0.9375,0.8750 48 | 46,0.9688,0.7188 49 | 47,1.0000,0.7812 50 | 48,0.9375,0.8125 51 | 49,0.9688,0.7812 52 | 50,1.0000,0.8125 53 | 51,1.0000,0.8125 54 | 52,0.9688,0.9062 55 | 53,1.0000,0.9688 56 | 54,0.9688,0.8750 57 | 55,0.9688,0.8438 58 | 56,0.9688,0.9062 59 | 57,1.0000,0.8750 60 | 58,1.0000,0.8750 61 | 59,0.8438,0.8438 62 | 60,1.0000,0.8438 63 | 61,1.0000,0.8438 64 | 62,0.9375,0.8750 65 | 63,0.9688,0.8750 66 | 64,0.9688,1.0000 67 | 65,1.0000,0.8750 68 | 66,0.9688,0.8750 69 | 67,0.9688,0.8125 70 | 68,0.9375,0.7812 71 | 69,1.0000,0.8438 72 | 70,1.0000,0.9062 73 | 71,1.0000,0.9375 74 | 72,1.0000,0.6875 75 | 73,1.0000,0.7812 76 | 74,0.9688,0.8125 77 | 75,0.9688,0.9062 78 | 76,0.9688,0.7812 79 | 77,0.9688,0.8125 80 | 78,1.0000,0.8438 81 | 79,0.9375,0.8125 82 | 80,1.0000,0.8750 83 | 81,0.9375,0.7188 84 | 82,0.9375,0.9062 85 | 83,0.9375,0.8438 86 | 84,1.0000,0.8125 87 | 85,0.9688,0.8438 88 | 86,0.9688,0.8125 89 | 87,0.9375,0.6875 90 | 88,1.0000,0.7188 91 | 89,1.0000,0.8750 92 | 90,0.9688,0.8438 93 | 91,1.0000,0.7812 94 | 92,1.0000,0.7500 95 | 93,1.0000,0.8125 96 | 94,1.0000,0.8438 97 | 95,1.0000,0.8438 98 | 96,1.0000,0.8438 99 | 97,1.0000,0.8125 100 | 98,0.9688,0.9688 101 | 99,1.0000,0.9062 102 | 100,1.0000,0.9062 103 | 101,0.9375,0.8750 104 | 102,1.0000,0.6875 105 | 103,1.0000,0.9062 106 | 104,0.9688,0.8438 107 | 105,0.9688,0.8438 108 | 106,1.0000,0.8125 109 | 107,0.9688,0.8750 110 | 108,1.0000,1.0000 111 | 109,0.9688,0.8750 112 | 110,1.0000,0.7812 113 | 111,1.0000,0.8438 114 | 112,1.0000,0.9062 115 | 113,0.9375,0.8125 116 | 114,1.0000,0.8750 117 | 115,1.0000,0.8438 118 | 116,0.9688,0.8438 119 | 117,0.9375,0.9375 120 | 118,1.0000,0.9375 121 | 119,1.0000,0.8438 122 | 120,0.9688,0.9375 123 | 121,0.9688,0.7500 124 | 122,1.0000,0.8750 125 | 123,1.0000,0.9062 126 | 124,1.0000,0.8750 127 | 125,1.0000,0.9062 128 | 126,0.9688,0.9062 129 | 127,1.0000,0.9062 130 | 128,0.9688,0.7188 131 | 129,1.0000,0.8125 132 | 130,1.0000,0.9375 133 | 131,1.0000,0.7500 134 | 132,1.0000,0.8750 135 | 133,1.0000,0.8125 136 | 134,0.9688,0.7812 137 | 135,1.0000,0.7812 138 | 136,0.9688,0.8438 139 | 137,1.0000,0.9062 140 | 138,1.0000,0.8750 141 | 139,1.0000,0.9062 142 | 140,1.0000,0.9062 143 | 141,1.0000,0.6875 144 | 142,1.0000,0.8438 145 | 143,0.9375,0.8438 146 | 144,1.0000,0.8125 147 | 145,1.0000,0.9062 148 | 146,1.0000,0.7188 149 | 147,1.0000,0.9062 150 | 148,0.9688,0.8438 151 | 149,1.0000,0.8438 152 | -------------------------------------------------------------------------------- /logs/top_10.log: -------------------------------------------------------------------------------- 1 | batch,train_acc,val_acc 2 | 0,0.9375,0.6875 3 | 1,0.8750,0.6875 4 | 2,1.0000,0.8125 5 | 3,0.9688,0.7812 6 | 4,0.9062,0.8438 7 | 5,0.9688,0.7812 8 | 6,1.0000,0.7812 9 | 7,0.9688,0.7812 10 | 8,0.9688,0.7500 11 | 9,0.8750,0.8750 12 | 10,1.0000,0.9688 13 | 11,1.0000,0.9062 14 | 12,1.0000,0.8438 15 | 13,1.0000,0.9062 16 | 14,0.9688,0.9062 17 | 15,0.9375,0.9375 18 | 16,0.9375,0.7812 19 | 17,1.0000,0.8750 20 | 18,0.9688,0.8750 21 | 19,0.9375,0.7500 22 | 20,0.9688,0.8125 23 | 21,1.0000,0.9375 24 | 22,0.9062,0.8438 25 | 23,1.0000,0.8750 26 | 24,1.0000,0.9375 27 | 25,1.0000,0.8750 28 | 26,1.0000,0.8750 29 | 27,0.9688,0.8438 30 | 28,0.9688,0.8750 31 | 29,0.9688,0.9062 32 | 30,1.0000,0.9375 33 | 31,0.9375,0.8750 34 | 32,1.0000,0.9062 35 | 33,1.0000,0.9688 36 | 34,0.9688,0.8438 37 | 35,1.0000,0.8125 38 | 36,0.9688,0.7812 39 | 37,0.9375,0.7812 40 | 38,1.0000,0.8438 41 | 39,1.0000,0.9062 42 | 40,1.0000,0.8438 43 | 41,1.0000,0.7812 44 | 42,0.9688,0.8750 45 | 43,1.0000,0.8438 46 | 44,0.9688,0.7812 47 | 45,0.9062,0.8438 48 | 46,0.9688,0.8750 49 | 47,0.9375,0.8438 50 | 48,1.0000,0.9688 51 | 49,1.0000,0.7812 52 | 50,0.9688,0.8125 53 | 51,1.0000,0.8750 54 | 52,0.9688,0.9375 55 | 53,1.0000,0.8438 56 | 54,1.0000,0.8750 57 | 55,1.0000,0.8750 58 | 56,1.0000,0.8438 59 | 57,0.9375,0.8750 60 | 58,1.0000,0.8750 61 | 59,1.0000,0.9375 62 | 60,0.8750,0.8125 63 | 61,1.0000,0.8438 64 | 62,1.0000,0.9375 65 | 63,1.0000,0.8750 66 | 64,1.0000,0.8750 67 | 65,1.0000,0.9375 68 | 66,1.0000,0.9062 69 | 67,1.0000,0.8438 70 | 68,1.0000,0.9688 71 | 69,0.9688,0.8750 72 | 70,1.0000,0.8750 73 | 71,0.9688,0.9062 74 | 72,0.9688,0.8750 75 | 73,0.9688,0.9375 76 | 74,1.0000,0.9062 77 | 75,0.9688,0.8438 78 | 76,1.0000,0.9375 79 | 77,1.0000,0.8125 80 | 78,1.0000,0.9375 81 | 79,1.0000,0.9062 82 | 80,1.0000,0.9062 83 | 81,1.0000,0.8438 84 | 82,1.0000,0.8750 85 | 83,1.0000,0.9062 86 | 84,1.0000,0.9062 87 | 85,1.0000,0.8125 88 | 86,0.9688,1.0000 89 | 87,1.0000,0.9375 90 | 88,0.9688,0.8125 91 | 89,0.9688,0.8125 92 | 90,0.9688,0.9688 93 | 91,0.9688,0.8125 94 | 92,1.0000,0.9062 95 | 93,0.9688,0.8125 96 | 94,1.0000,0.8438 97 | 95,0.8750,0.9062 98 | 96,0.9688,0.8125 99 | 97,1.0000,0.9062 100 | 98,0.9375,0.8438 101 | 99,1.0000,0.9062 102 | 100,1.0000,0.8750 103 | 101,1.0000,0.9062 104 | 102,1.0000,0.9062 105 | 103,0.9375,0.9375 106 | 104,0.9688,0.7812 107 | 105,1.0000,0.8438 108 | 106,1.0000,0.9375 109 | 107,1.0000,0.9062 110 | 108,1.0000,0.9375 111 | 109,0.9688,0.8750 112 | 110,1.0000,0.9062 113 | 111,0.9688,0.9062 114 | 112,0.9688,0.9062 115 | 113,0.9688,0.9375 116 | 114,1.0000,0.9062 117 | 115,1.0000,0.8125 118 | 116,1.0000,0.8438 119 | 117,1.0000,0.7812 120 | 118,1.0000,0.9062 121 | 119,1.0000,0.9375 122 | 120,1.0000,0.9375 123 | 121,1.0000,0.9375 124 | 122,1.0000,0.8438 125 | 123,0.9688,0.9062 126 | 124,1.0000,0.9062 127 | 125,1.0000,0.9688 128 | 126,1.0000,0.8438 129 | 127,1.0000,0.8750 130 | 128,1.0000,0.7500 131 | 129,1.0000,0.9375 132 | 130,1.0000,0.8438 133 | 131,0.9375,0.9375 134 | 132,0.9375,0.7812 135 | 133,1.0000,0.8438 136 | 134,1.0000,0.9375 137 | 135,1.0000,0.8438 138 | 136,1.0000,0.9062 139 | 137,0.9688,0.9375 140 | 138,1.0000,0.9062 141 | 139,1.0000,0.9375 142 | 140,0.9688,0.8438 143 | 141,1.0000,0.9062 144 | 142,1.0000,1.0000 145 | 143,0.9688,0.9688 146 | 144,1.0000,0.7500 147 | 145,1.0000,0.9375 148 | 146,1.0000,0.8438 149 | 147,1.0000,0.8438 150 | 148,0.9688,0.8438 151 | 149,1.0000,0.7812 152 | -------------------------------------------------------------------------------- /logs/top_15.log: -------------------------------------------------------------------------------- 1 | batch,train_acc,val_acc 2 | 0,0.9062,0.5625 3 | 1,1.0000,0.5625 4 | 2,0.9375,0.8125 5 | 3,0.9062,0.6562 6 | 4,1.0000,0.8438 7 | 5,0.9688,0.8125 8 | 6,0.9688,0.7500 9 | 7,0.9062,0.7500 10 | 8,0.9688,0.8438 11 | 9,0.9688,0.8125 12 | 10,1.0000,0.7500 13 | 11,0.9688,0.8125 14 | 12,1.0000,0.8438 15 | 13,0.9375,0.8438 16 | 14,0.9375,0.6875 17 | 15,1.0000,0.8750 18 | 16,0.9688,0.8750 19 | 17,0.9688,0.7812 20 | 18,0.9375,0.8125 21 | 19,0.9688,0.8750 22 | 20,1.0000,0.9062 23 | 21,0.9375,0.8438 24 | 22,0.9688,0.7500 25 | 23,1.0000,0.9375 26 | 24,0.9688,0.9062 27 | 25,0.9688,0.7500 28 | 26,1.0000,0.8125 29 | 27,0.9688,0.9062 30 | 28,0.9688,0.8125 31 | 29,0.9688,0.8750 32 | 30,1.0000,0.9062 33 | 31,1.0000,0.9062 34 | 32,1.0000,0.9375 35 | 33,1.0000,0.8750 36 | 34,0.9688,0.9375 37 | 35,0.9688,0.8750 38 | 36,0.9688,0.9375 39 | 37,1.0000,0.9688 40 | 38,0.9688,0.8125 41 | 39,1.0000,0.9062 42 | 40,0.9375,0.7188 43 | 41,0.9375,0.8750 44 | 42,0.9688,0.8750 45 | 43,0.9688,0.9062 46 | 44,1.0000,0.7812 47 | 45,0.9688,0.8750 48 | 46,0.9688,0.7812 49 | 47,1.0000,0.8125 50 | 48,1.0000,0.9062 51 | 49,1.0000,0.7812 52 | 50,1.0000,0.9688 53 | 51,1.0000,0.9062 54 | 52,0.9688,0.9062 55 | 53,1.0000,0.8750 56 | 54,0.9688,0.8750 57 | 55,0.9688,0.7812 58 | 56,1.0000,0.7500 59 | 57,1.0000,0.9375 60 | 58,1.0000,0.7812 61 | 59,1.0000,0.7500 62 | 60,1.0000,0.9062 63 | 61,0.9688,0.9375 64 | 62,1.0000,0.8438 65 | 63,0.9688,0.9062 66 | 64,1.0000,0.9375 67 | 65,0.9375,0.8438 68 | 66,1.0000,0.8438 69 | 67,0.9688,0.9375 70 | 68,1.0000,0.8125 71 | 69,1.0000,0.9062 72 | 70,1.0000,0.9062 73 | 71,0.9375,0.9375 74 | 72,0.8750,0.9062 75 | 73,1.0000,0.8438 76 | 74,0.9375,0.9375 77 | 75,1.0000,0.9062 78 | 76,1.0000,0.7812 79 | 77,1.0000,0.9375 80 | 78,1.0000,0.8750 81 | 79,1.0000,0.9062 82 | 80,0.9688,0.9062 83 | 81,0.9062,0.7812 84 | 82,0.9688,0.9688 85 | 83,1.0000,0.9062 86 | 84,1.0000,0.8125 87 | 85,1.0000,0.7500 88 | 86,0.9688,0.7812 89 | 87,0.9688,0.8125 90 | 88,1.0000,0.8750 91 | 89,1.0000,0.8125 92 | 90,0.9688,0.9375 93 | 91,1.0000,0.9375 94 | 92,1.0000,0.7500 95 | 93,1.0000,0.9062 96 | 94,0.9688,0.8438 97 | 95,1.0000,0.9688 98 | 96,0.9688,0.9375 99 | 97,1.0000,0.9375 100 | 98,1.0000,0.9375 101 | 99,1.0000,0.8125 102 | 100,0.9688,0.9375 103 | 101,1.0000,0.8438 104 | 102,1.0000,0.9062 105 | 103,1.0000,0.9375 106 | 104,1.0000,0.8125 107 | 105,1.0000,0.8750 108 | 106,0.9688,0.9062 109 | 107,1.0000,0.9375 110 | 108,1.0000,0.9062 111 | 109,1.0000,0.8125 112 | 110,1.0000,0.7500 113 | 111,1.0000,0.7188 114 | 112,1.0000,0.8750 115 | 113,1.0000,0.9688 116 | 114,0.9688,0.8125 117 | 115,1.0000,0.8750 118 | 116,1.0000,0.8750 119 | 117,0.9688,0.8125 120 | 118,1.0000,0.9375 121 | 119,1.0000,0.9062 122 | 120,1.0000,0.7500 123 | 121,1.0000,0.8750 124 | 122,1.0000,0.9688 125 | 123,1.0000,0.7812 126 | 124,1.0000,0.9688 127 | 125,1.0000,0.8750 128 | 126,1.0000,0.8438 129 | 127,1.0000,0.7500 130 | 128,0.9688,0.6562 131 | 129,1.0000,0.9688 132 | 130,1.0000,0.8438 133 | 131,1.0000,0.7812 134 | 132,1.0000,0.8750 135 | 133,1.0000,0.7812 136 | 134,0.9688,0.8750 137 | 135,1.0000,0.8750 138 | 136,1.0000,0.7812 139 | 137,0.9688,0.8750 140 | 138,1.0000,0.9375 141 | 139,1.0000,0.9375 142 | 140,1.0000,0.9062 143 | 141,1.0000,0.9688 144 | 142,0.9688,0.8750 145 | 143,1.0000,0.7812 146 | 144,1.0000,0.8750 147 | 145,1.0000,0.8750 148 | 146,1.0000,1.0000 149 | 147,1.0000,0.9375 150 | 148,0.9375,0.8750 151 | 149,1.0000,0.8750 152 | -------------------------------------------------------------------------------- /logs/top_20.log: -------------------------------------------------------------------------------- 1 | batch,train_acc,val_acc 2 | 0,0.9062,0.4688 3 | 1,0.9688,0.7812 4 | 2,1.0000,0.6875 5 | 3,0.9375,0.7188 6 | 4,0.9688,0.8125 7 | 5,0.9062,0.7500 8 | 6,1.0000,0.7188 9 | 7,1.0000,0.8125 10 | 8,1.0000,0.7812 11 | 9,1.0000,0.7812 12 | 10,0.9688,0.8438 13 | 11,0.9688,0.8438 14 | 12,1.0000,0.9062 15 | 13,1.0000,0.7188 16 | 14,1.0000,0.7500 17 | 15,1.0000,0.9688 18 | 16,0.9688,0.9375 19 | 17,0.9375,0.8125 20 | 18,1.0000,0.8750 21 | 19,0.9688,0.8750 22 | 20,1.0000,0.8438 23 | 21,1.0000,0.8750 24 | 22,0.9688,0.9375 25 | 23,1.0000,0.8125 26 | 24,0.9688,0.8750 27 | 25,1.0000,0.8750 28 | 26,1.0000,0.9062 29 | 27,1.0000,0.7500 30 | 28,1.0000,0.9375 31 | 29,0.9688,0.9375 32 | 30,1.0000,0.8438 33 | 31,1.0000,0.7812 34 | 32,1.0000,0.8438 35 | 33,1.0000,0.8438 36 | 34,1.0000,0.9062 37 | 35,0.9688,0.8125 38 | 36,0.9688,0.9062 39 | 37,1.0000,0.9688 40 | 38,0.9375,0.7188 41 | 39,1.0000,0.8750 42 | 40,0.9688,0.9062 43 | 41,0.9375,0.8750 44 | 42,1.0000,0.8125 45 | 43,1.0000,0.9062 46 | 44,0.9688,0.8750 47 | 45,0.9062,0.9062 48 | 46,1.0000,0.7812 49 | 47,0.9688,0.9375 50 | 48,1.0000,0.8750 51 | 49,0.9375,0.8125 52 | 50,1.0000,0.8750 53 | 51,1.0000,0.8750 54 | 52,1.0000,0.8750 55 | 53,1.0000,0.8125 56 | 54,0.9688,0.8125 57 | 55,0.9375,0.7812 58 | 56,1.0000,0.8125 59 | 57,1.0000,0.9062 60 | 58,0.9688,0.8438 61 | 59,0.9688,0.7500 62 | 60,0.9688,0.8750 63 | 61,0.9375,0.8750 64 | 62,1.0000,0.9375 65 | 63,1.0000,0.8750 66 | 64,1.0000,0.8438 67 | 65,1.0000,0.9688 68 | 66,1.0000,0.9375 69 | 67,1.0000,0.9062 70 | 68,1.0000,0.9062 71 | 69,1.0000,0.9375 72 | 70,1.0000,0.9062 73 | 71,1.0000,0.9375 74 | 72,1.0000,0.7812 75 | 73,1.0000,0.7500 76 | 74,0.9062,0.7812 77 | 75,1.0000,0.8438 78 | 76,1.0000,0.8750 79 | 77,0.9688,0.9062 80 | 78,1.0000,0.8750 81 | 79,1.0000,0.9062 82 | 80,1.0000,0.9062 83 | 81,1.0000,0.8750 84 | 82,1.0000,0.8438 85 | 83,1.0000,0.8750 86 | 84,1.0000,0.9062 87 | 85,1.0000,0.9062 88 | 86,1.0000,0.7188 89 | 87,1.0000,0.8750 90 | 88,1.0000,0.8750 91 | 89,0.9688,0.7812 92 | 90,0.9688,0.9062 93 | 91,1.0000,0.9062 94 | 92,0.9688,0.9688 95 | 93,1.0000,0.8750 96 | 94,1.0000,0.9688 97 | 95,0.9688,0.8125 98 | 96,0.9688,0.9375 99 | 97,1.0000,0.8125 100 | 98,1.0000,0.9375 101 | 99,1.0000,0.8438 102 | 100,0.9688,0.8438 103 | 101,1.0000,0.8125 104 | 102,1.0000,0.8125 105 | 103,1.0000,0.7188 106 | 104,0.9688,0.7812 107 | 105,1.0000,0.9375 108 | 106,1.0000,0.9375 109 | 107,0.9688,0.8438 110 | 108,1.0000,0.8750 111 | 109,1.0000,0.9375 112 | 110,1.0000,0.8125 113 | 111,1.0000,0.9688 114 | 112,1.0000,0.9062 115 | 113,1.0000,0.9062 116 | 114,0.9688,0.8750 117 | 115,0.9688,0.7812 118 | 116,1.0000,0.8750 119 | 117,1.0000,0.8125 120 | 118,1.0000,0.7812 121 | 119,1.0000,0.9062 122 | 120,0.9688,0.8125 123 | 121,1.0000,0.8438 124 | 122,1.0000,0.9062 125 | 123,1.0000,0.9062 126 | 124,0.9688,0.8125 127 | 125,1.0000,0.7500 128 | 126,1.0000,0.8750 129 | 127,1.0000,0.9375 130 | 128,1.0000,0.9062 131 | 129,1.0000,0.9062 132 | 130,0.9688,0.8125 133 | 131,0.9688,0.8125 134 | 132,1.0000,0.8438 135 | 133,0.9688,0.8750 136 | 134,1.0000,0.8750 137 | 135,1.0000,0.9062 138 | 136,1.0000,0.8438 139 | 137,0.9688,0.9062 140 | 138,1.0000,0.7812 141 | 139,0.9688,0.8750 142 | 140,1.0000,0.9062 143 | 141,1.0000,0.8125 144 | 142,0.9688,0.9062 145 | 143,0.9688,0.7812 146 | 144,1.0000,0.9375 147 | 145,1.0000,0.8438 148 | 146,0.9688,0.9688 149 | 147,0.9688,0.9375 150 | 148,0.9688,0.9375 151 | 149,0.9688,0.8750 152 | -------------------------------------------------------------------------------- /logs/top_25.log: -------------------------------------------------------------------------------- 1 | batch,train_acc,val_acc 2 | 0,1.0000,0.7500 3 | 1,0.9375,0.6562 4 | 2,0.9375,0.7188 5 | 3,0.9688,0.8750 6 | 4,0.9688,0.7500 7 | 5,1.0000,0.8125 8 | 6,0.9688,0.8750 9 | 7,0.9688,0.9062 10 | 8,0.9375,0.8438 11 | 9,1.0000,0.7500 12 | 10,1.0000,0.9062 13 | 11,0.9688,0.8750 14 | 12,0.9688,0.8438 15 | 13,0.8750,0.6875 16 | 14,1.0000,0.8750 17 | 15,1.0000,0.7500 18 | 16,1.0000,0.7500 19 | 17,1.0000,0.9375 20 | 18,1.0000,0.9375 21 | 19,0.9688,0.7500 22 | 20,1.0000,0.6562 23 | 21,0.9688,0.7812 24 | 22,0.9688,0.7812 25 | 23,0.9375,0.8750 26 | 24,1.0000,0.9062 27 | 25,0.9375,0.7812 28 | 26,0.9375,0.7188 29 | 27,0.9688,0.7812 30 | 28,1.0000,0.9375 31 | 29,1.0000,0.8750 32 | 30,1.0000,0.9062 33 | 31,0.9688,0.9375 34 | 32,0.9375,0.8438 35 | 33,1.0000,0.8438 36 | 34,1.0000,0.8125 37 | 35,0.9375,0.9375 38 | 36,0.9375,0.8750 39 | 37,1.0000,0.9062 40 | 38,1.0000,0.9688 41 | -------------------------------------------------------------------------------- /logs/top_3.log: -------------------------------------------------------------------------------- 1 | batch,train_acc,val_acc 2 | 0,0.9688,0.6875 3 | 1,0.9062,0.8438 4 | 2,0.9688,0.6875 5 | 3,1.0000,0.7812 6 | 4,0.9688,0.7812 7 | 5,0.9062,0.8125 8 | 6,0.9688,0.8750 9 | 7,1.0000,0.8750 10 | 8,0.9375,0.9062 11 | 9,0.9375,0.9688 12 | 10,1.0000,0.8125 13 | 11,1.0000,0.8750 14 | 12,1.0000,0.8750 15 | 13,0.9688,0.7188 16 | 14,1.0000,0.9375 17 | 15,0.9375,0.8438 18 | 16,1.0000,0.9062 19 | 17,1.0000,0.8750 20 | 18,1.0000,0.9688 21 | 19,1.0000,0.7500 22 | 20,1.0000,0.7812 23 | 21,0.9688,0.7812 24 | 22,1.0000,0.8125 25 | 23,1.0000,0.7812 26 | 24,1.0000,0.8750 27 | 25,1.0000,0.9062 28 | 26,1.0000,0.9375 29 | 27,0.9375,0.8438 30 | 28,0.9375,0.7188 31 | 29,1.0000,0.8125 32 | 30,1.0000,0.8125 33 | 31,1.0000,0.8750 34 | 32,1.0000,0.8125 35 | 33,1.0000,0.8750 36 | 34,1.0000,0.9062 37 | 35,1.0000,0.9688 38 | 36,1.0000,0.9375 39 | 37,1.0000,0.9688 40 | 38,1.0000,0.8125 41 | 39,1.0000,0.8438 42 | 40,1.0000,0.9062 43 | 41,1.0000,0.8750 44 | 42,1.0000,0.9375 45 | 43,0.9688,0.8750 46 | 44,0.9688,0.8438 47 | 45,1.0000,1.0000 48 | 46,1.0000,0.8125 49 | 47,0.9062,0.7812 50 | 48,1.0000,0.9062 51 | 49,1.0000,0.8438 52 | 50,0.9375,0.8438 53 | 51,1.0000,0.9062 54 | 52,1.0000,0.9062 55 | 53,0.8750,0.7500 56 | 54,0.9688,0.9375 57 | 55,1.0000,0.6875 58 | 56,1.0000,0.9062 59 | 57,0.9375,0.8438 60 | 58,1.0000,0.8125 61 | 59,0.9688,0.8438 62 | 60,1.0000,0.6875 63 | 61,0.9375,0.8438 64 | 62,0.9688,0.7812 65 | 63,1.0000,0.9688 66 | 64,1.0000,0.9062 67 | 65,1.0000,0.8125 68 | 66,0.9688,0.9062 69 | 67,0.9688,0.8438 70 | 68,1.0000,0.8438 71 | 69,0.9688,0.9688 72 | 70,0.9688,0.8750 73 | 71,1.0000,0.9688 74 | 72,0.9688,0.8438 75 | 73,1.0000,0.9688 76 | 74,1.0000,0.7812 77 | 75,1.0000,0.9062 78 | 76,1.0000,0.9062 79 | 77,1.0000,0.9688 80 | 78,1.0000,0.9375 81 | 79,1.0000,0.9375 82 | 80,1.0000,0.9062 83 | 81,1.0000,0.8438 84 | 82,1.0000,0.8438 85 | 83,1.0000,0.9062 86 | 84,1.0000,0.8125 87 | 85,0.9688,0.8438 88 | 86,1.0000,0.9062 89 | 87,0.9688,0.8125 90 | 88,0.9688,0.8750 91 | 89,1.0000,0.8750 92 | 90,1.0000,0.9688 93 | 91,1.0000,0.8750 94 | 92,1.0000,0.8125 95 | 93,1.0000,0.8438 96 | 94,1.0000,1.0000 97 | 95,0.9688,0.8438 98 | 96,0.9688,0.8750 99 | 97,1.0000,0.9062 100 | 98,1.0000,0.8750 101 | 99,1.0000,0.8125 102 | 100,1.0000,0.9062 103 | 101,1.0000,0.8438 104 | 102,1.0000,0.9062 105 | 103,1.0000,0.9062 106 | 104,1.0000,0.8438 107 | 105,1.0000,0.9688 108 | 106,0.9688,0.9062 109 | 107,0.9688,0.9062 110 | 108,1.0000,0.9375 111 | 109,1.0000,0.8438 112 | 110,0.9688,0.9062 113 | 111,1.0000,0.8125 114 | 112,1.0000,0.9375 115 | 113,1.0000,0.9062 116 | 114,0.9375,0.7812 117 | 115,0.9688,0.9062 118 | 116,1.0000,0.8438 119 | 117,1.0000,0.9688 120 | 118,1.0000,0.9062 121 | 119,0.9688,0.8438 122 | 120,0.9688,0.8750 123 | 121,1.0000,0.9062 124 | 122,1.0000,0.8750 125 | 123,0.9688,0.8438 126 | 124,1.0000,0.9062 127 | 125,1.0000,0.7188 128 | 126,0.9688,0.9375 129 | 127,1.0000,0.8438 130 | 128,1.0000,0.8750 131 | 129,1.0000,0.9062 132 | 130,1.0000,0.8438 133 | 131,1.0000,0.8125 134 | 132,1.0000,0.8750 135 | 133,1.0000,0.9688 136 | 134,1.0000,0.9688 137 | 135,1.0000,0.9375 138 | 136,0.9688,0.8438 139 | 137,0.9688,0.8125 140 | 138,0.9688,0.8438 141 | 139,1.0000,0.9062 142 | 140,0.9688,0.8750 143 | 141,0.9375,0.8750 144 | 142,1.0000,0.8750 145 | 143,1.0000,0.8750 146 | 144,1.0000,0.9062 147 | 145,1.0000,0.7812 148 | 146,1.0000,0.9375 149 | 147,1.0000,0.9062 150 | 148,1.0000,0.9375 151 | 149,0.9375,0.8125 152 | -------------------------------------------------------------------------------- /logs/top_5.log: -------------------------------------------------------------------------------- 1 | batch,train_acc,val_acc 2 | 0,0.9375,0.6250 3 | 1,0.9375,0.7500 4 | 2,0.9375,0.9062 5 | 3,1.0000,0.8125 6 | 4,1.0000,0.7812 7 | 5,0.9688,0.8750 8 | 6,0.9375,0.8438 9 | 7,0.9688,0.8750 10 | 8,0.9688,0.9062 11 | 9,0.9688,0.9062 12 | 10,1.0000,0.8438 13 | 11,0.9062,0.8438 14 | 12,0.9688,0.8750 15 | 13,1.0000,0.8125 16 | 14,0.9688,0.7812 17 | 15,0.9688,0.8438 18 | 16,0.9688,0.8750 19 | 17,1.0000,0.8750 20 | 18,1.0000,0.9062 21 | 19,1.0000,0.8438 22 | 20,1.0000,0.8750 23 | 21,0.9688,0.8125 24 | 22,1.0000,0.9062 25 | 23,0.9688,0.8750 26 | 24,1.0000,0.8750 27 | 25,1.0000,0.9062 28 | 26,1.0000,0.8750 29 | 27,1.0000,0.8750 30 | 28,1.0000,0.8438 31 | 29,1.0000,0.8750 32 | 30,0.9688,0.9375 33 | 31,1.0000,0.9062 34 | 32,1.0000,0.8750 35 | 33,1.0000,0.9062 36 | 34,1.0000,0.9375 37 | 35,1.0000,0.9062 38 | 36,0.9375,0.9062 39 | 37,1.0000,0.8438 40 | 38,1.0000,0.8750 41 | 39,1.0000,0.9688 42 | 40,1.0000,0.8438 43 | 41,1.0000,0.8750 44 | 42,1.0000,0.7500 45 | 43,0.9375,0.8750 46 | 44,1.0000,0.9688 47 | 45,1.0000,0.8750 48 | 46,1.0000,0.8125 49 | 47,0.9688,0.8438 50 | 48,0.9688,0.9062 51 | 49,1.0000,0.9375 52 | 50,0.9688,0.8125 53 | 51,1.0000,0.8750 54 | 52,1.0000,0.8750 55 | 53,1.0000,0.9375 56 | 54,1.0000,0.8438 57 | 55,0.9688,0.8125 58 | 56,1.0000,0.8438 59 | 57,1.0000,0.8125 60 | 58,1.0000,0.8125 61 | 59,0.9688,0.9375 62 | 60,1.0000,0.8750 63 | 61,0.9688,0.9062 64 | 62,0.9688,0.8438 65 | 63,1.0000,0.9062 66 | 64,1.0000,0.7500 67 | 65,0.9688,0.8438 68 | 66,1.0000,0.8750 69 | 67,0.9375,0.8438 70 | 68,0.9688,0.9062 71 | 69,1.0000,0.7500 72 | 70,1.0000,0.8125 73 | 71,1.0000,0.9688 74 | 72,1.0000,0.7812 75 | 73,1.0000,0.8125 76 | 74,1.0000,0.9375 77 | 75,1.0000,0.9062 78 | 76,1.0000,0.9375 79 | 77,1.0000,0.8438 80 | 78,0.9688,0.9375 81 | 79,1.0000,0.7812 82 | 80,1.0000,0.8125 83 | 81,1.0000,0.9688 84 | 82,1.0000,0.8750 85 | 83,1.0000,0.8438 86 | 84,1.0000,0.9375 87 | 85,0.9688,0.9062 88 | 86,1.0000,0.9375 89 | 87,1.0000,0.8438 90 | 88,0.9688,0.9062 91 | 89,1.0000,0.9688 92 | 90,0.9688,0.8750 93 | 91,1.0000,0.9688 94 | 92,1.0000,0.9375 95 | 93,1.0000,0.9375 96 | 94,1.0000,0.8125 97 | 95,0.9688,0.9062 98 | 96,1.0000,0.7812 99 | 97,0.9688,0.7188 100 | 98,1.0000,0.9062 101 | 99,1.0000,0.9062 102 | 100,1.0000,0.9375 103 | 101,1.0000,0.8750 104 | 102,1.0000,0.9375 105 | 103,1.0000,0.7500 106 | 104,0.9688,0.9375 107 | 105,1.0000,0.9688 108 | 106,0.9688,0.8438 109 | 107,0.9688,0.9688 110 | 108,1.0000,0.9062 111 | 109,1.0000,0.8438 112 | 110,1.0000,0.9062 113 | 111,1.0000,0.8438 114 | 112,1.0000,0.9062 115 | 113,0.9688,0.8750 116 | 114,0.9688,0.9062 117 | 115,1.0000,0.8125 118 | 116,1.0000,0.9375 119 | 117,1.0000,0.9375 120 | 118,1.0000,0.9375 121 | 119,0.9688,0.8438 122 | 120,1.0000,0.9375 123 | 121,1.0000,0.9062 124 | 122,0.9375,0.8750 125 | 123,1.0000,0.9375 126 | 124,1.0000,0.9062 127 | 125,1.0000,0.8438 128 | 126,0.9375,0.8438 129 | 127,1.0000,0.8438 130 | 128,1.0000,0.9375 131 | 129,1.0000,0.9062 132 | 130,1.0000,0.9375 133 | 131,1.0000,0.8438 134 | 132,1.0000,0.8750 135 | 133,1.0000,0.8438 136 | 134,1.0000,0.8438 137 | 135,1.0000,0.9375 138 | 136,1.0000,0.9062 139 | 137,1.0000,0.8438 140 | 138,0.9688,0.9375 141 | 139,1.0000,0.7500 142 | 140,0.9688,0.9375 143 | 141,1.0000,0.9375 144 | 142,1.0000,0.9688 145 | 143,1.0000,0.8125 146 | 144,0.9375,0.9375 147 | 145,0.9688,0.9062 148 | 146,1.0000,0.9688 149 | 147,1.0000,0.8750 150 | 148,0.9688,0.8750 151 | 149,0.9688,0.8750 152 | -------------------------------------------------------------------------------- /logs/top_7.log: -------------------------------------------------------------------------------- 1 | batch,train_acc,val_acc 2 | 0,0.9375,0.6250 3 | 1,0.7812,0.5938 4 | 2,1.0000,0.7812 5 | 3,0.9375,0.7500 6 | 4,0.9688,0.8750 7 | 5,0.8750,0.8125 8 | 6,1.0000,0.8438 9 | 7,1.0000,0.8438 10 | 8,0.9688,0.7812 11 | 9,0.9375,0.7812 12 | 10,0.9688,0.6875 13 | 11,1.0000,0.8750 14 | 12,0.9688,0.8438 15 | 13,0.9688,0.9375 16 | 14,1.0000,0.7500 17 | 15,0.9688,0.8750 18 | 16,0.9375,0.9375 19 | 17,1.0000,0.9062 20 | 18,1.0000,0.7188 21 | 19,0.9375,0.7812 22 | 20,1.0000,0.7812 23 | 21,1.0000,0.8125 24 | 22,0.9688,0.9062 25 | 23,0.9688,0.7188 26 | 24,1.0000,0.8750 27 | 25,0.9688,0.7500 28 | 26,1.0000,0.8125 29 | 27,0.9688,0.8438 30 | 28,0.8125,0.8125 31 | 29,1.0000,0.8125 32 | 30,0.9688,0.9062 33 | 31,0.9688,0.9062 34 | 32,0.9375,0.8438 35 | 33,1.0000,0.7812 36 | 34,0.9688,0.8750 37 | -------------------------------------------------------------------------------- /logs/training_04_13_08_08.log: -------------------------------------------------------------------------------- 1 | epoch,acc,loss,val_acc,val_loss 2 | 0,0.84090202955,0.454722055137,0.798828125,0.526426492725 3 | 1,0.854562766216,0.414594654183,0.8466796875,0.473556938116 4 | -------------------------------------------------------------------------------- /logs/training_04_19_02_01.log: -------------------------------------------------------------------------------- 1 | epoch,acc,loss,val_acc,val_loss 2 | 0,0.817542909849,0.542034926401,0.809255784873,0.546623258929 3 | 1,0.853954663467,0.423497933521,0.82066916823,0.539905698796 4 | 2,0.86359945058,0.394509454416,0.834605757204,0.468954528516 5 | -------------------------------------------------------------------------------- /models/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/models/__init__.py -------------------------------------------------------------------------------- /models/multiview_cnn.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | from sklearn.metrics import confusion_matrix 4 | 5 | from keras.applications.vgg16 import VGG16 6 | from keras.callbacks import CSVLogger, ModelCheckpoint, ReduceLROnPlateau 7 | from keras.layers import Dense, Flatten, Input, Dropout 8 | from keras.optimizers import SGD 9 | from keras.models import Model 10 | 11 | from geometry_processing.globals import (TRAIN_DIR, VALID_DIR, MODEL_WEIGHTS, 12 | LOG_FILE, IMAGE_SIZE, NUM_CLASSES, IMAGE_MEAN, IMAGE_STD) 13 | from geometry_processing.utils.helpers import (get_data, samplewise_normalize, 14 | load_weights) 15 | 16 | 17 | # Set to 2 on supercomputer during training. 18 | VERBOSE = 1 19 | 20 | 21 | def train(model, save_to=''): 22 | # Center and normalize each sample. 23 | normalize = samplewise_normalize(IMAGE_MEAN, IMAGE_STD) 24 | 25 | # Get streaming data. 26 | train_generator = get_data(TRAIN_DIR, preprocess=normalize) 27 | valid_generator = get_data(VALID_DIR, preprocess=normalize) 28 | 29 | print('%d training samples.' % train_generator.n) 30 | print('%d validation samples.' % valid_generator.n) 31 | 32 | model.compile(loss='categorical_crossentropy', 33 | optimizer=SGD(lr=1e-3, momentum=0.9), 34 | metrics=['accuracy']) 35 | 36 | callbacks = list() 37 | 38 | callbacks.append(CSVLogger(LOG_FILE)) 39 | callbacks.append(ReduceLROnPlateau(monitor='val_loss', factor=0.1, 40 | patience=0, min_lr=0.0001)) 41 | 42 | if save_to: 43 | callbacks.append(ModelCheckpoint(filepath=save_to, verbose=1)) 44 | 45 | model.fit_generator(generator=train_generator, 46 | samples_per_epoch=train_generator.n, 47 | nb_epoch=2, 48 | validation_data=valid_generator, 49 | nb_val_samples=1000, 50 | callbacks=callbacks, 51 | verbose=VERBOSE) 52 | 53 | # Save the weights on completion. 54 | if save_to: 55 | model.save_weights(save_to) 56 | 57 | 58 | def load_model(input_tensor=None, include_top=True): 59 | if input_tensor is None: 60 | input_tensor = Input(shape=(IMAGE_SIZE, IMAGE_SIZE, 3)) 61 | 62 | # Don't include VGG fc layers. 63 | base_model = VGG16(include_top=False, input_tensor=input_tensor) 64 | 65 | # Freeze all layers in pretrained network. 66 | for layer in base_model.layers: 67 | layer.trainable = False 68 | 69 | x = base_model.output 70 | x = Flatten(name='flatten')(x) 71 | x = Dense(4096, activation='relu', name='fc1')(x) 72 | x = Dropout(0.5)(x) 73 | x = Dense(2048, activation='relu', name='fc2')(x) 74 | x = Dropout(0.5)(x) 75 | 76 | if include_top: 77 | x = Dense(NUM_CLASSES, activation='softmax', name='predictions')(x) 78 | 79 | return Model(inputs=input_tensor, outputs=x) 80 | 81 | 82 | def test(model, nb_batch=32, nb_worker=2): 83 | # Optimizer is unused. 84 | model.compile(loss='categorical_crossentropy', optimizer='sgd', 85 | metrics=['accuracy']) 86 | 87 | # Center and normalize each sample. 88 | normalize = samplewise_normalize(IMAGE_MEAN, IMAGE_STD) 89 | 90 | # Get streaming data. 91 | test_generator = get_data(VALID_DIR, batch=nb_batch, shuffle=True, 92 | preprocess=normalize) 93 | 94 | matrix = np.zeros((NUM_CLASSES, NUM_CLASSES)) 95 | 96 | # Flag that batch_index at 0 has been seen. 97 | start = False 98 | 99 | while not start or test_generator.batch_index != 0: 100 | start = True 101 | 102 | # Grab the next batch. 103 | x, y_true = test_generator.next() 104 | 105 | # Convert probabilities to predictions. 106 | y_true = np.argmax(y_true, axis=1) 107 | y_pred = np.argmax(model.predict_on_batch(x), axis=1) 108 | 109 | matrix += confusion_matrix(y_true, y_pred, labels=range(NUM_CLASSES)) 110 | 111 | return matrix 112 | 113 | 114 | if __name__ == '__main__': 115 | mvcnn = load_model() 116 | load_weights(mvcnn, MODEL_WEIGHTS) 117 | 118 | print("Log file: %s" % LOG_FILE) 119 | train(mvcnn, save_to=MODEL_WEIGHTS) 120 | -------------------------------------------------------------------------------- /models/multiview_svm.py: -------------------------------------------------------------------------------- 1 | import random 2 | import pickle 3 | 4 | import keras.backend as K 5 | import numpy as np 6 | 7 | from sklearn import linear_model 8 | 9 | from geometry_processing.utils.helpers import entropy 10 | 11 | 12 | # Hack for enums. 13 | GREEDY = 0 14 | RANDOM = 1 15 | MODES = [GREEDY, RANDOM] 16 | 17 | 18 | class MultiviewModel: 19 | def __init__(self, input_shape, feature_layer, softmax_layer, k, nb_classes, 20 | svm_path=None, preprocess=None, sort_mode=GREEDY): 21 | # Used to simultaneously get activations of multiple layers. 22 | self.functor = K.function([input_shape] + [K.learning_phase()], 23 | [feature_layer, softmax_layer]) 24 | self.k = k 25 | self.nb_classes = nb_classes 26 | self.sort_mode = sort_mode 27 | self.preprocess = preprocess 28 | 29 | if svm_path: 30 | print('Loading SVM from %s.' % svm_path) 31 | with open(svm_path, 'rb') as fd: 32 | self.svm = pickle.load(fd) 33 | else: 34 | self.svm = linear_model.SGDClassifier(penalty='l2', alpha=0.0002, 35 | loss='hinge') 36 | 37 | def get_top_k_features(self, x): 38 | # Sanity checks. 39 | n = x.shape[0] 40 | assert n >= self.k, 'Not enough views. n: %d < k: %d' % (n, self.k) 41 | 42 | features, softmax = self.functor([x, 0.]) 43 | 44 | # Sort by minimum entropy. 45 | entropy_index = [(entropy(softmax[i]), i) for i in range(n)] 46 | 47 | # Pick K features based on sort scheme. 48 | if self.sort_mode == GREEDY: 49 | entropy_index.sort(key=lambda x: x[0]) 50 | elif self.sort_mode == RANDOM: 51 | random.shuffle(entropy_index) 52 | 53 | # Create a chunk using only the K most confident views. 54 | top_k_features = np.zeros((self.k, features.shape[1])) 55 | 56 | # No need to vectorize - GPU transfer time is the dominant time. 57 | for i in range(self.k): 58 | ith_index = entropy_index[i][1] 59 | top_k_features[i] = features[ith_index] 60 | 61 | return top_k_features 62 | 63 | def aggregated_features(self, batch): 64 | batch_size = batch.shape[0] 65 | 66 | # TODO: don't hardcode this. 67 | examples = np.zeros((batch_size, 2048)) 68 | 69 | # No need to vectorize - GPU memory is not large enough. 70 | for i in range(batch_size): 71 | top_k_features = self.get_top_k_features(batch[i]) 72 | 73 | # The feature vector is a element-wise max over the top k. 74 | examples[i] = np.max(top_k_features, axis=0) 75 | 76 | if self.preprocess is not None: 77 | examples = np.apply_along_axis(self.preprocess, 1, examples) 78 | 79 | return examples 80 | 81 | def fit(self, x, y): 82 | x_aggregated = self.aggregated_features(x) 83 | self.svm.partial_fit(x_aggregated, y, classes=range(self.nb_classes)) 84 | return self.svm.score(x_aggregated, y) 85 | 86 | def predict(self, x): 87 | """ 88 | Arguments: 89 | x: (n, k, m) np.ndarray, 90 | n the number of examples, 91 | k the number of samples to pool, 92 | m the feature size. 93 | Returns: 94 | (n, 1) np.ndarray, 95 | n the number of examples, containing class predictions. 96 | """ 97 | return self.svm.predict(self.aggregated_features(x)) 98 | 99 | def score(self, x, y): 100 | return self.svm.score(self.aggregated_features(x), y) 101 | 102 | def save(self, file_path): 103 | print('Saving to %s.' % file_path) 104 | with open(file_path, 'wb') as fd: 105 | pickle.dump(self.svm, fd) 106 | print('Success.') 107 | -------------------------------------------------------------------------------- /models/saliency.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | from sklearn.metrics import confusion_matrix 4 | 5 | from keras import regularizers 6 | from keras.callbacks import CSVLogger, ModelCheckpoint, ReduceLROnPlateau 7 | from keras.layers import Dense, Input, Dropout 8 | from keras.optimizers import SGD 9 | from keras.models import Model 10 | 11 | from geometry_processing.globals import (TRAIN_DIR, VALID_DIR, 12 | SALIENCY_DATA_TRAIN, SALIENCY_DATA_VALID, 13 | IMAGE_SIZE, IMAGE_MEAN, IMAGE_STD, NUM_CLASSES) 14 | from geometry_processing.utils.helpers import samplewise_normalize 15 | from geometry_processing.utils.custom_datagen import SaliencyDataGenerator 16 | from geometry_processing.models import multiview_cnn 17 | 18 | 19 | def build_model(input_tensor=None): 20 | if input_tensor is None: 21 | input_tensor = Input(shape=(IMAGE_SIZE, IMAGE_SIZE, 3)) 22 | 23 | mvcnn = multiview_cnn.load_model(input_tensor=input_tensor, include_top=False) 24 | 25 | x = mvcnn.output 26 | x = Dense(256, activation='relu', 27 | kernel_regularizer=regularizers.l2(0.01), 28 | bias_regularizer=regularizers.l2(0.01), name='fc3')(x) 29 | x = Dropout(0.5)(x) 30 | x = Dense(128, activation='relu', 31 | kernel_regularizer=regularizers.l2(0.01), 32 | bias_regularizer=regularizers.l2(0.01), name='fc4')(x) 33 | x = Dropout(0.5)(x) 34 | x = Dense(2, activation='softmax', name='saliency')(x) 35 | 36 | # Saliency predictor. 37 | return Model(inputs=input_tensor, outputs=x) 38 | 39 | 40 | def train(model, save_path, nb_epoch=10, nb_val_samples=1000, batch_size=64, 41 | log_file=None, verbose=1): 42 | model.compile(loss='binary_crossentropy', 43 | optimizer=SGD(lr=1e-3, momentum=0.9), 44 | metrics=['accuracy']) 45 | 46 | # Center and normalize each sample. 47 | normalize = samplewise_normalize(IMAGE_MEAN, IMAGE_STD) 48 | 49 | # Get streaming data. 50 | train_generator = SaliencyDataGenerator(TRAIN_DIR, SALIENCY_DATA_TRAIN, 51 | preprocess=normalize, batch_size=batch_size) 52 | valid_generator = SaliencyDataGenerator(VALID_DIR, SALIENCY_DATA_VALID, 53 | preprocess=normalize, batch_size=batch_size) 54 | 55 | print('%d training samples.' % train_generator.nb_data) 56 | print('%d validation samples.' % valid_generator.nb_data) 57 | 58 | # Various routines to run. 59 | callbacks = list() 60 | 61 | if log_file: 62 | callbacks.append(CSVLogger(log_file)) 63 | 64 | if save_path: 65 | callbacks.append(ModelCheckpoint(filepath=save_path, verbose=1)) 66 | 67 | callbacks.append(ReduceLROnPlateau(monitor='val_loss', factor=0.5, 68 | patience=0, min_lr=1e-5)) 69 | 70 | # Train. 71 | model.fit_generator(generator=train_generator.generate(), 72 | steps_per_epoch=train_generator.nb_data // batch_size, 73 | epochs=nb_epoch, 74 | validation_data=valid_generator.generate(), 75 | validation_steps=nb_val_samples // batch_size, 76 | callbacks=callbacks, 77 | verbose=verbose) 78 | 79 | 80 | def test(model, batch_size=32): 81 | # Optimizer is unused. 82 | model.compile(loss='categorical_crossentropy', optimizer='sgd', 83 | metrics=['accuracy']) 84 | 85 | # Center and normalize each sample. 86 | normalize = samplewise_normalize(IMAGE_MEAN, IMAGE_STD) 87 | 88 | test_generator = SaliencyDataGenerator(VALID_DIR, SALIENCY_DATA_VALID, 89 | preprocess=normalize, batch_size=batch_size) 90 | 91 | print('%d validation samples.' % test_generator.nb_data) 92 | 93 | # Either right or wrong. 94 | matrix = np.zeros((2, 2)) 95 | 96 | for x, y_true in test_generator.generate(): 97 | if test_generator.epochs_seen == 1: 98 | break 99 | 100 | # Convert probabilities to predictions. 101 | y_true = np.argmax(y_true, axis=1) 102 | y_pred = np.argmax(model.predict_on_batch(x), axis=1) 103 | 104 | matrix += confusion_matrix(y_true, y_pred, labels=range(2)) 105 | 106 | return matrix 107 | -------------------------------------------------------------------------------- /poster.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/poster.pdf -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | Keras==1.2.2 2 | h5py==2.6.0 3 | matplotlib==1.5.3 4 | opencv-python==3.2.0.6 5 | scikit-learn==0.18.1 6 | tensorflow==0.12.1 7 | six==1.10.0 8 | Pillow==4.0.0 9 | -------------------------------------------------------------------------------- /screenshots/confusion_matrix.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/screenshots/confusion_matrix.png -------------------------------------------------------------------------------- /screenshots/sample_data.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/screenshots/sample_data.png -------------------------------------------------------------------------------- /scripts/generate_confusion_matrix.py: -------------------------------------------------------------------------------- 1 | """ 2 | Sample runs (from the package directory). 3 | 4 | Fresh training: 5 | python3 scripts/generate_confusion_matrix.py \ 6 | --k_features=5 \ 7 | --svm_path=cache/svm_top_k_5_new.pkl 8 | --matrix_path=cache/svm_top_k_5 9 | """ 10 | import time 11 | import argparse 12 | 13 | import numpy as np 14 | 15 | from sklearn.metrics import confusion_matrix 16 | 17 | from geometry_processing.globals import (VALID_DIR, NUM_CLASSES, 18 | IMAGE_MEAN, IMAGE_STD, MODEL_WEIGHTS, FC2_MEAN, FC2_STD) 19 | from geometry_processing.utils.helpers import samplewise_normalize 20 | from geometry_processing.utils.custom_datagen import GroupedDatagen 21 | from geometry_processing.models.multiview_cnn import load_model 22 | from geometry_processing.models.multiview_svm import MultiviewModel 23 | 24 | 25 | # Command line arguments. 26 | parser = argparse.ArgumentParser(description='Generate a confusion matrix.') 27 | 28 | parser.add_argument('--k_features', required=True, type=int, 29 | help='Number of features to consider.') 30 | parser.add_argument('--svm_path', required=True, type=str, 31 | help='Path to the pickled SVM.') 32 | parser.add_argument('--matrix_path', required=False, type=str, default='', 33 | help='Path (without extension) to save the matrix.') 34 | parser.add_argument('--sort_mode', required=False, type=int, default=0, 35 | help='Scheme to pick top k (0 - greedy, 1 - random).') 36 | 37 | args = parser.parse_args() 38 | k_features = args.k_features 39 | svm_path = args.svm_path 40 | matrix_path = args.matrix_path 41 | sort_mode = args.sort_mode 42 | 43 | 44 | def evaluate_loop(mv_model, valid_group, batch_size=64, nb_epoch=100, 45 | save_file=None): 46 | # Start time. 47 | tic = time.time() 48 | 49 | matrix = np.zeros((NUM_CLASSES, NUM_CLASSES)) 50 | 51 | for t, batch in enumerate(valid_group.generate(batch_size=batch_size)): 52 | if t >= nb_epoch: 53 | break 54 | # Batch start time. 55 | toc = time.time() 56 | 57 | x = batch[0] 58 | y_true = np.argmax(batch[1], axis=2)[:, 0] 59 | y_pred = mv_model.predict(x) 60 | 61 | matrix += confusion_matrix(y_true, y_pred, labels=range(NUM_CLASSES)) 62 | 63 | print('Batch took %.4f seconds.' % (time.time() - toc)) 64 | 65 | print('Total time elapsed - %.4f seconds.' % (time.time() - tic)) 66 | print('Accuracy %.4f' % np.mean(np.diag(matrix) / np.sum(matrix, axis=1))) 67 | 68 | # Save matrix to disk. 69 | if save_file: 70 | print('Saving to %s.' % save_file) 71 | np.save(save_file, matrix) 72 | print('Success.') 73 | 74 | 75 | if __name__ == '__main__': 76 | # Data source. 77 | img_normalize = samplewise_normalize(IMAGE_MEAN, IMAGE_STD) 78 | fc2_normalize = samplewise_normalize(FC2_MEAN, FC2_STD) 79 | valid_group = GroupedDatagen(VALID_DIR, preprocess=img_normalize) 80 | 81 | # Use the fc activations as features. 82 | model = load_model(MODEL_WEIGHTS) 83 | fc2_layer = model.get_layer('fc2').output 84 | softmax_layer = model.get_layer('predictions').output 85 | 86 | # Low budget version of pickle.load(). 87 | multiview = MultiviewModel(model.layers[0].input, fc2_layer, softmax_layer, 88 | k_features, NUM_CLASSES, preprocess=fc2_normalize, svm_path=svm_path, 89 | sort_mode=sort_mode) 90 | 91 | evaluate_loop(multiview, valid_group, save_file=matrix_path) 92 | -------------------------------------------------------------------------------- /scripts/generate_saliency_data.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | 3 | import keras.backend as K 4 | 5 | from geometry_processing.globals import (TRAIN_DIR, VALID_DIR, 6 | IMAGE_MEAN, IMAGE_STD, MODEL_WEIGHTS) 7 | from geometry_processing.utils.helpers import (samplewise_normalize, 8 | entropy, load_weights) 9 | from geometry_processing.utils.custom_datagen import FilenameImageDatagen 10 | from geometry_processing.models.multiview_cnn import load_model 11 | 12 | 13 | parser = argparse.ArgumentParser(description='Generate saliency data.') 14 | 15 | parser.add_argument('--confidence_threshold', required=False, type=float, 16 | default=0.7, help='Value from [0, inf]. Lower is more confident.') 17 | parser.add_argument('--pick_top', required=False, type=int, 18 | default=-1, help='Value from [-1, num_views].') 19 | parser.add_argument('--generate_dataset', required=True, type=str, 20 | help='train/test') 21 | 22 | args = parser.parse_args() 23 | confidence_threshold = args.confidence_threshold 24 | pick_top = args.pick_top 25 | generate_dataset = args.generate_dataset 26 | 27 | 28 | # Set pick_top to -1 if thresholding by entropy value. 29 | def generate(datagen, functor): 30 | # Generate training data. 25 to ensure viewpoints are batched (hack). 31 | for full_paths, images in datagen.generate(25): 32 | # 0 means test mode (turn off dropout). 33 | predictions = functor([images, 0])[0] 34 | 35 | if pick_top > -1: 36 | paths_predictions = list(zip(full_paths, predictions)) 37 | paths_predictions.sort(key=lambda x: entropy(x[1])) 38 | 39 | for i, path_prediction in enumerate(paths_predictions): 40 | if i < pick_top: 41 | print(path_prediction[0], 1.0) 42 | else: 43 | print(path_prediction[0], 0.0) 44 | else: 45 | for i in range(predictions.shape[0]): 46 | if entropy(predictions[i]) <= confidence_threshold: 47 | print(full_paths[i], 1.0) 48 | else: 49 | print(full_paths[i], 0.0) 50 | 51 | 52 | if __name__ == '__main__': 53 | # Data source and image normalization. 54 | img_normalize = samplewise_normalize(IMAGE_MEAN, IMAGE_STD) 55 | 56 | # Directory of images. 57 | if generate_dataset == 'train': 58 | datagen = FilenameImageDatagen(TRAIN_DIR, preprocess=img_normalize) 59 | elif generate_dataset == 'test': 60 | datagen = FilenameImageDatagen(VALID_DIR, preprocess=img_normalize) 61 | else: 62 | raise ValueError('Invalid input for --generate_dataset.') 63 | 64 | # # Use the fc activations as features. 65 | model = load_model() 66 | load_weights(model, MODEL_WEIGHTS) 67 | 68 | # Wrapper around Tensorflow run operation. 69 | functor = K.function([model.layers[0].input, K.learning_phase()], 70 | [model.get_layer('predictions').output]) 71 | 72 | generate(datagen, functor) 73 | -------------------------------------------------------------------------------- /scripts/plot_logs.py: -------------------------------------------------------------------------------- 1 | """ 2 | Sample usage: 3 | 4 | python3 scripts/plot_logs.py --log_file=logs/saliency.log --labels=acc,val_acc,loss,val_loss 5 | """ 6 | import argparse 7 | import csv 8 | 9 | from matplotlib import pyplot as plt 10 | 11 | 12 | # Command line arguments. 13 | parser = argparse.ArgumentParser(description='Train a top K SVM.') 14 | 15 | parser.add_argument('--log_file', required=True, type=str, 16 | help='Path to log file.') 17 | parser.add_argument('--labels', required=False, type=str, default='', 18 | help='Comma delimited column labels. Default is all') 19 | 20 | args = parser.parse_args() 21 | log_file = args.log_file 22 | labels = args.labels 23 | 24 | 25 | def show_graph(to_plot): 26 | plt.ylim([0.0, 1.0]) 27 | 28 | for label, graph in to_plot.items(): 29 | plt.plot(list(range(len(graph))), graph, "o-", label=label) 30 | 31 | plt.legend(loc='best') 32 | plt.show() 33 | 34 | 35 | if __name__ == "__main__": 36 | to_plot = dict() 37 | 38 | if labels: 39 | to_plot = {label: list() for label in labels.split(',')} 40 | 41 | with open(log_file) as fd: 42 | for i, row in enumerate(csv.DictReader(fd)): 43 | if i == 0: 44 | if not labels: 45 | to_plot = {label: list() for label in row} 46 | continue 47 | 48 | if labels: 49 | for label in labels.split(','): 50 | to_plot[label].append(row[label]) 51 | else: 52 | for label, value in row.items(): 53 | to_plot[label].append(value) 54 | 55 | show_graph(to_plot) 56 | -------------------------------------------------------------------------------- /scripts/precompute_fc_mean_std.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | from geometry_processing.globals import (TRAIN_DIR, IMAGE_MEAN, 4 | IMAGE_STD, SAVE_FILE) 5 | from geometry_processing.utils.helpers import (get_data, samplewise_normalize, 6 | extract_layer) 7 | from geometry_processing.models.multiview_cnn import load_model 8 | 9 | 10 | # Number of samples to use for std and mean calculations. 11 | NUM_SAMPLES = 45000 12 | 13 | 14 | def get_mean_std(layer, datagen, num_samples): 15 | samples = np.zeros((num_samples, layer.output_shape[1])) 16 | 17 | index = 0 18 | for x, _ in datagen: 19 | if index >= num_samples: 20 | break 21 | print(index) 22 | 23 | activations = layer.predict(x) 24 | 25 | offset = min(num_samples - index, activations.shape[0]) 26 | samples[index:index+offset] = activations[:offset] 27 | index += offset 28 | 29 | print(samples[-1]) 30 | 31 | return np.mean(samples, axis=0), np.std(samples, axis=0) 32 | 33 | 34 | if __name__ == '__main__': 35 | # Use the fc activations as features. 36 | model = load_model(SAVE_FILE) 37 | fc2 = extract_layer(model, 'fc2') 38 | 39 | # Normalize the image. 40 | normalize = samplewise_normalize(IMAGE_MEAN, IMAGE_STD) 41 | 42 | # Initialize data generators. 43 | train_datagen = get_data(TRAIN_DIR, 64, preprocess=normalize) 44 | 45 | # Precompute fc2 center and std. 46 | mean, std = get_mean_std(fc2, train_datagen, NUM_SAMPLES) 47 | 48 | # Cache for future use. 49 | with open("fc2_mean.npy", "wb") as fd: 50 | np.save(fd, mean) 51 | with open("fc2_std.npy", "wb") as fd: 52 | np.save(fd, std) 53 | 54 | print(mean, std) 55 | -------------------------------------------------------------------------------- /scripts/precompute_image_mean_std.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | from geometry_processing.globals import TRAIN_DIR 4 | from geometry_processing.utils.helpers import get_precomputed_statistics 5 | 6 | 7 | mean, std = get_precomputed_statistics(TRAIN_DIR, 10000) 8 | 9 | with open("mean.npy", "wb") as fd: 10 | np.save(fd, mean) 11 | with open("std.npy", "wb") as fd: 12 | np.save(fd, std) 13 | -------------------------------------------------------------------------------- /scripts/test_mvcnn.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | 3 | import numpy as np 4 | 5 | from geometry_processing.globals import MODEL_WEIGHTS 6 | from geometry_processing.utils.helpers import load_weights 7 | from geometry_processing.models.multiview_cnn import load_model, test 8 | 9 | 10 | parser = argparse.ArgumentParser(description='Test MVCNN classification.') 11 | 12 | parser.add_argument('--matrix_path', required=True, type=str, 13 | help='Path to save the confusion matrix.') 14 | 15 | args = parser.parse_args() 16 | matrix_path = args.matrix_path 17 | 18 | 19 | if __name__ == '__main__': 20 | # Initialize model. 21 | mvcnn = load_model() 22 | load_weights(mvcnn, MODEL_WEIGHTS) 23 | 24 | # Run through entire test dataset. 25 | matrix = test(mvcnn) 26 | print('Accuracy %.4f' % np.mean(np.diag(matrix) / np.sum(matrix, axis=1))) 27 | 28 | # Save matrix to disk. 29 | if matrix_path: 30 | print('Saving to %s.' % matrix_path) 31 | np.save(matrix_path, matrix) 32 | -------------------------------------------------------------------------------- /scripts/test_saliency.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | 3 | import numpy as np 4 | 5 | from geometry_processing.globals import SALIENCY_MODEL 6 | from geometry_processing.utils.helpers import load_weights 7 | from geometry_processing.models.saliency import build_model, test 8 | 9 | 10 | parser = argparse.ArgumentParser(description='Test saliency classification.') 11 | 12 | parser.add_argument('--matrix_path', required=True, type=str, 13 | help='Path to save the confusion matrix.') 14 | 15 | args = parser.parse_args() 16 | matrix_path = args.matrix_path 17 | 18 | 19 | if __name__ == '__main__': 20 | # Initialize model. 21 | saliency_cnn = build_model() 22 | load_weights(saliency_cnn, SALIENCY_MODEL) 23 | 24 | # Run through entire test dataset. 25 | matrix = test(saliency_cnn) 26 | print('Per Class Accuracy %s' % (np.diag(matrix) / np.sum(matrix, axis=1))) 27 | print('MAP: %.4f' % np.mean(np.diag(matrix) / np.sum(matrix, axis=1))) 28 | 29 | # Save matrix to disk. 30 | if matrix_path: 31 | print('Saving to %s.' % matrix_path) 32 | np.save(matrix_path, matrix) 33 | -------------------------------------------------------------------------------- /scripts/train_saliency.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | 3 | from geometry_processing.globals import SALIENCY_MODEL, MODEL_WEIGHTS 4 | from geometry_processing.utils.helpers import load_weights 5 | from geometry_processing.models.saliency import build_model, train 6 | 7 | 8 | parser = argparse.ArgumentParser(description='Train a saliency NN.') 9 | 10 | parser.add_argument('--verbose', required=False, type=int, 11 | default=1, help='[1] for ncurses, [2] for per epoch.') 12 | parser.add_argument('--log_file', required=False, type=str, 13 | default='', help='File to log training, validation loss and accuracy.') 14 | 15 | args = parser.parse_args() 16 | verbose = args.verbose 17 | log_file = args.log_file 18 | 19 | 20 | if __name__ == '__main__': 21 | # Build and load cached weights. 22 | saliency_cnn = build_model() 23 | load_weights(saliency_cnn, MODEL_WEIGHTS) 24 | 25 | # Update model. 26 | train(saliency_cnn, save_path=SALIENCY_MODEL) 27 | -------------------------------------------------------------------------------- /scripts/train_svm.py: -------------------------------------------------------------------------------- 1 | """ 2 | Sample runs (from the package directory). 3 | 4 | Fresh training: 5 | python3 scripts/train_svm.py \ 6 | --k_features=5 \ 7 | --save_path=cache/svm_top_k_5_new.pkl 8 | 9 | Fresh training, using random sort: 10 | python3 scripts/train_svm.py \ 11 | --k_features=5 \ 12 | --save_path=cache/svm_top_k_5_new.pkl \ 13 | --sort_mode=1 14 | 15 | Continued training from a previous run: 16 | python3 scripts/train_svm.py \ 17 | --k_features=5 \ 18 | --svm_path=cache/svm_top_k_5.pkl \ 19 | --save_path=cache/svm_top_k_5_new.pkl 20 | """ 21 | import time 22 | import argparse 23 | 24 | import numpy as np 25 | 26 | from geometry_processing.globals import (TRAIN_DIR, VALID_DIR, NUM_CLASSES, 27 | IMAGE_MEAN, IMAGE_STD, MODEL_WEIGHTS, FC2_MEAN, FC2_STD) 28 | from geometry_processing.utils.helpers import samplewise_normalize 29 | from geometry_processing.utils.custom_datagen import GroupedDatagen 30 | from geometry_processing.models.multiview_cnn import load_model 31 | from geometry_processing.models.multiview_svm import MultiviewModel 32 | 33 | 34 | # Command line arguments. 35 | parser = argparse.ArgumentParser(description='Train a top K SVM.') 36 | 37 | parser.add_argument('--k_features', required=True, type=int, 38 | help='Number of features to consider.') 39 | parser.add_argument('--svm_path', required=False, type=str, default='', 40 | help='Path to the pre-trained SVM (leave blank if fresh start).') 41 | parser.add_argument('--save_path', required=False, type=str, default='', 42 | help='Path to save the trained SVM.') 43 | parser.add_argument('--sort_mode', required=False, type=int, default=0, 44 | help='Scheme to pick top k (0 - greedy, 1 - random).') 45 | 46 | args = parser.parse_args() 47 | k_features = args.k_features 48 | svm_path = args.svm_path 49 | save_path = args.save_path 50 | sort_mode = args.sort_mode 51 | 52 | 53 | def train_loop(mv_model, train_group, valid_group, batch=64, nb_batches=50, 54 | save_file=None): 55 | # Start time. 56 | tic = time.time() 57 | 58 | for t, batch in enumerate(zip(train_group.generate(batch_size=batch), 59 | valid_group.generate(batch_size=batch // 4))): 60 | # Have seen enough batches. Terminate. 61 | if t >= nb_batches: 62 | break 63 | # Batch start time. 64 | toc = time.time() 65 | 66 | # Both train and valid are tuples. 67 | train, valid = batch 68 | 69 | # Sample train: (32, 25, 224, 224, 3), (32, 25, 10). 70 | train_x = train[0] 71 | train_y = np.argmax(train[1], axis=2)[:, 0] 72 | 73 | valid_x = valid[0] 74 | valid_y = np.argmax(valid[1], axis=2)[:, 0] 75 | 76 | score = mv_model.fit(train_x, train_y) 77 | 78 | print('Training accuracy %.4f' % score) 79 | print('Validation accuracy %.4f' % mv_model.score(valid_x, valid_y)) 80 | print('Batch training took %.4f seconds.' % (time.time() - toc)) 81 | 82 | print('Total time elapsed - %.4f seconds.' % (time.time() - tic)) 83 | 84 | # Keep the model's weights. 85 | if save_file: 86 | mv_model.save(save_file) 87 | 88 | 89 | if __name__ == '__main__': 90 | # Data source. 91 | img_normalize = samplewise_normalize(IMAGE_MEAN, IMAGE_STD) 92 | fc2_normalize = samplewise_normalize(FC2_MEAN, FC2_STD) 93 | train_group = GroupedDatagen(TRAIN_DIR, preprocess=img_normalize) 94 | valid_group = GroupedDatagen(VALID_DIR, preprocess=img_normalize) 95 | 96 | # Use the fc activations as features. 97 | model = load_model(MODEL_WEIGHTS) 98 | fc2_layer = model.get_layer('fc2').output 99 | softmax_layer = model.get_layer('predictions').output 100 | 101 | # Training. 102 | multiview = MultiviewModel(model.layers[0].input, fc2_layer, softmax_layer, 103 | k_features, NUM_CLASSES, preprocess=fc2_normalize, svm_path=svm_path, 104 | sort_mode=sort_mode) 105 | 106 | train_loop(multiview, train_group, valid_group, save_file=save_path) 107 | -------------------------------------------------------------------------------- /scripts/view_confusion_matrix.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | 3 | import numpy as np 4 | 5 | from geometry_processing.globals import VALID_DIR 6 | from geometry_processing.utils.helpers import plot_confusion_matrix, get_data 7 | 8 | 9 | # Command line arguments. 10 | parser = argparse.ArgumentParser(description='View a confusion matrix.') 11 | 12 | parser.add_argument('--matrix_path', required=True, type=str, 13 | help='Path to the pickled (via numpy) matrix.') 14 | 15 | args = parser.parse_args() 16 | matrix_path = args.matrix_path 17 | 18 | 19 | def get_class_labels(data_generator): 20 | # Something like ['bathtub', 'desk', ...]. 21 | class_labels = data_generator.class_indices 22 | class_labels = list(sorted(class_labels, key=lambda x: class_labels[x])) 23 | return class_labels 24 | 25 | 26 | if __name__ == '__main__': 27 | data_generator = get_data(VALID_DIR) 28 | 29 | matrix = np.load(matrix_path) 30 | plot_confusion_matrix(matrix, get_class_labels(data_generator), 31 | normalize=True, title="Confusion") 32 | -------------------------------------------------------------------------------- /scripts/view_salient.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from matplotlib import pyplot as plt 3 | 4 | from geometry_processing.globals import (VALID_DIR, MODEL_WEIGHTS, 5 | SALIENCY_DATA_VALID, SALIENCY_MODEL, IMAGE_MEAN, IMAGE_STD) 6 | from geometry_processing.utils.helpers import samplewise_normalize 7 | from geometry_processing.utils.custom_datagen import SaliencyDataGenerator 8 | from geometry_processing.models.saliency import build_model 9 | 10 | 11 | TO_SHOW = 3 12 | GRID_FORMAT = '{}{}%s'.format(TO_SHOW, TO_SHOW) 13 | 14 | 15 | def denormalize(img): 16 | return np.uint8(np.clip(img * IMAGE_STD + IMAGE_MEAN, 0.0, 255.0)) 17 | 18 | 19 | def run(model, datagen): 20 | for imgs, labels in datagen.generate(): 21 | prediction = model.predict(imgs) 22 | 23 | for i in range(TO_SHOW): 24 | for j in range(TO_SHOW): 25 | index = i * TO_SHOW + j 26 | 27 | plt.subplot(GRID_FORMAT % (index + 1)) 28 | plt.title('%.3f/%.3f ' % (prediction[index][1], labels[index][1])) 29 | plt.imshow(denormalize(imgs[index])) 30 | plt.xticks([]) 31 | plt.yticks([]) 32 | 33 | plt.show() 34 | 35 | 36 | if __name__ == '__main__': 37 | # Center and normalize each sample. 38 | normalize = samplewise_normalize(IMAGE_MEAN, IMAGE_STD) 39 | 40 | # Get streaming data. 41 | valid_generator = SaliencyDataGenerator(VALID_DIR, SALIENCY_DATA_VALID, 42 | preprocess=normalize, batch_size=TO_SHOW ** 2) 43 | 44 | print('%d samples.' % valid_generator.nb_data) 45 | 46 | saliency_cnn = build_model(MODEL_WEIGHTS, SALIENCY_MODEL) 47 | 48 | run(saliency_cnn, valid_generator) 49 | -------------------------------------------------------------------------------- /utils/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradyz/geometry_processing/946729f379ad6d70d6937aec459c46e6d840a3cb/utils/__init__.py -------------------------------------------------------------------------------- /utils/custom_datagen.py: -------------------------------------------------------------------------------- 1 | import os 2 | 3 | import numpy as np 4 | 5 | from keras.preprocessing import image 6 | import keras.backend as K 7 | 8 | from geometry_processing.globals import VALID_DIR 9 | 10 | 11 | # Helper to see if two images were generated by the same mesh. 12 | def get_prefix(filename): 13 | return filename[:filename.index(".")] 14 | 15 | 16 | class GroupedDatagen: 17 | def __init__(self, data_dir, shape=(224, 224, 3), 18 | preprocess=None, nb_class=10): 19 | self.data_dir = data_dir 20 | self.nb_class = nb_class 21 | self.preprocess = preprocess 22 | self.shape = shape 23 | self.class_mapping = dict() 24 | self.data_pairs = list() 25 | self.index = 0 26 | 27 | # Populate data pairs. 28 | self._setup() 29 | 30 | def _add_bin(self, classname, root, relative_paths): 31 | # Change to full paths. 32 | for j in range(len(relative_paths)): 33 | relative_paths[j] = os.path.join(root, relative_paths[j]) 34 | self.data_pairs.append((relative_paths, classname)) 35 | 36 | def _setup(self): 37 | # The files in this directory should be names of classes. 38 | for i, label in enumerate(sorted(os.listdir(self.data_dir))): 39 | self.class_mapping[label] = i 40 | 41 | # All the images associated with the class. 42 | for root, _, files in os.walk(os.path.join(self.data_dir, label)): 43 | if not files: 44 | continue 45 | 46 | files.sort() 47 | current_bin = [files[0]] 48 | 49 | for j in range(1, len(files)): 50 | # Generated from the same model. 51 | if get_prefix(files[j]) == get_prefix(current_bin[-1]): 52 | current_bin.append(files[j]) 53 | else: 54 | self._add_bin(label, root, current_bin) 55 | current_bin = [files[j]] 56 | 57 | # Don't forget the last bin. 58 | self._add_bin(label, root, current_bin) 59 | 60 | def _get_pairs(self, samples, shuffle): 61 | while True: 62 | if self.index == 0 and shuffle: 63 | np.random.shuffle(self.data_pairs) 64 | 65 | # All n samples of this batch belong to a single mesh. 66 | full_paths, label = self.data_pairs[self.index] 67 | 68 | assert samples <= len(full_paths) 69 | 70 | # Initialize batch. 71 | batch_x = np.zeros((samples,) + self.shape, dtype=K.floatx()) 72 | batch_y = np.zeros((samples, self.nb_class), dtype=K.floatx()) 73 | 74 | # Get some samples. 75 | for i, full_path in enumerate(full_paths): 76 | if i >= samples: 77 | break 78 | 79 | img = image.load_img(full_path, target_size=self.shape) 80 | x = image.img_to_array(img) 81 | if self.preprocess: 82 | x = self.preprocess(x) 83 | 84 | # Add sample to batch. 85 | batch_x[i] = x 86 | batch_y[i][self.class_mapping[label]] = 1.0 87 | 88 | # Mark this pair as used. 89 | self.index = (self.index + 1) % len(self.data_pairs) 90 | 91 | yield batch_x, batch_y 92 | 93 | def generate(self, samples=25, batch_size=64, shuffle=True): 94 | # Infinite generator. 95 | while True: 96 | x = np.zeros((batch_size, samples,) + self.shape, dtype=K.floatx()) 97 | y = np.zeros((batch_size, samples, self.nb_class), dtype=K.floatx()) 98 | 99 | for i, batch in enumerate(self._get_pairs(samples=samples, 100 | shuffle=shuffle)): 101 | if i >= batch_size: 102 | break 103 | 104 | batch_x, batch_y = batch 105 | x[i] = batch_x 106 | y[i] = batch_y 107 | 108 | yield x, y 109 | 110 | 111 | class FilenameImageDatagen: 112 | def __init__(self, data_dir, shape=(224, 224, 3), preprocess=None): 113 | self.data_dir = data_dir 114 | self.preprocess = preprocess 115 | self.shape = shape 116 | 117 | def generate_single(self): 118 | # The files in this directory should be names of classes. 119 | for _, label in enumerate(sorted(os.listdir(self.data_dir))): 120 | # All the images associated with the class. 121 | for root, _, files in os.walk(os.path.join(self.data_dir, label)): 122 | for filename in sorted(files): 123 | full_path = os.path.join(root, filename) 124 | 125 | img = image.load_img(full_path, target_size=self.shape) 126 | img = image.img_to_array(img) 127 | 128 | if self.preprocess: 129 | img = self.preprocess(img) 130 | 131 | yield full_path, img 132 | 133 | def generate(self, batch_size=16): 134 | """Generates batchs of size batch_size except for the last batch.""" 135 | paths = np.zeros((batch_size), dtype='object') 136 | images = np.zeros((batch_size,) + self.shape, dtype=K.floatx()) 137 | 138 | next_spot = 0 139 | 140 | for _, path_image in enumerate(self.generate_single()): 141 | paths[next_spot] = path_image[0] 142 | images[next_spot] = path_image[1] 143 | next_spot += 1 144 | 145 | # Filled up entire batch, yield it. 146 | if next_spot == batch_size: 147 | yield paths, images 148 | 149 | # Overwrite the previous data. 150 | next_spot = 0 151 | 152 | # Last batch not filled, yield what remains. 153 | if next_spot != 0: 154 | yield paths[:next_spot], images[:next_spot] 155 | 156 | 157 | class SaliencyDataGenerator: 158 | def __init__(self, image_dir, labels_path, batch_size=32, 159 | shape=(224, 224, 3), preprocess=None, shuffle=True): 160 | self.image_dir = image_dir 161 | self.labels_path = labels_path 162 | self.batch_size = batch_size 163 | self.shape = shape 164 | self.preprocess = preprocess 165 | self.shuffle = shuffle 166 | 167 | # Holds tuples of (path, label). 168 | self.data = list() 169 | 170 | with open(self.labels_path, 'r') as data: 171 | for line in data: 172 | path, label = line.split(' ') 173 | 174 | path = os.path.join(self.image_dir, path) 175 | label = int(float(label)) 176 | 177 | # Add the training pair. 178 | self.data.append((path, label)) 179 | 180 | self.nb_data = len(self.data) 181 | self.epochs_seen = 0 182 | 183 | 184 | def generate(self): 185 | x = np.zeros((self.batch_size,) + self.shape, dtype=K.floatx()) 186 | y = np.zeros((self.batch_size, 2), dtype=K.floatx()) 187 | 188 | next_spot = 0 189 | 190 | while True: 191 | for i in range(self.batch_size): 192 | if next_spot == 0 and self.shuffle: 193 | np.random.shuffle(self.data) 194 | 195 | image_path, label = self.data[next_spot] 196 | 197 | img = image.load_img(image_path, target_size=self.shape) 198 | img = image.img_to_array(img) 199 | 200 | if self.preprocess: 201 | img = self.preprocess(img) 202 | 203 | x[i] = img 204 | 205 | # One hot encoding. 206 | y[i][0] = 0.0 207 | y[i][1] = 0.0 208 | y[i][label] = 1.0 209 | 210 | next_spot = (next_spot + 1) % self.nb_data 211 | 212 | # Completed an epoch. 213 | if next_spot == 0: 214 | self.epochs_seen += 1 215 | 216 | yield x, y 217 | 218 | 219 | if __name__ == "__main__": 220 | # Sample usage. 221 | grouped_datagen = GroupedDatagen(VALID_DIR) 222 | 223 | # Loops indefinitely. 224 | for examples, labels in grouped_datagen.generate(): 225 | pass 226 | 227 | # Sample usage. 228 | filename_datagen = FilenameImageDatagen(VALID_DIR) 229 | 230 | # Loops until directory is exhausted. 231 | for filename, image in filename_datagen.generate(): 232 | pass 233 | -------------------------------------------------------------------------------- /utils/helpers.py: -------------------------------------------------------------------------------- 1 | """ 2 | This file assumes keras ordering is set to 'tf'. 3 | """ 4 | 5 | import itertools 6 | 7 | import cv2 8 | 9 | from matplotlib import pyplot as plt 10 | 11 | import numpy as np 12 | 13 | from keras.models import Model 14 | from keras.callbacks import Callback 15 | from keras.preprocessing.image import ImageDataGenerator 16 | import keras.backend as K 17 | 18 | from geometry_processing.globals import IMAGE_SIZE 19 | 20 | 21 | class ManualInspection(Callback): 22 | def __init__(self, model): 23 | self.model = model 24 | 25 | def on_train_begin(self, logs={}): 26 | pass 27 | 28 | def on_train_end(self, logs={}): 29 | pass 30 | 31 | def on_batch_begin(self, batch, logs={}): 32 | pass 33 | 34 | def on_batch_end(self, batch, logs={}): 35 | pass 36 | 37 | 38 | def show(image): 39 | plt.imshow(image, interpolation='none') 40 | plt.show() 41 | return 42 | 43 | 44 | def view_filters(weights, number_to_show=10, axis=0): 45 | filter_size_x, filter_size_y, dimensions, number = weights.shape 46 | print("Filter size: %d by %d by %d" % (filter_size_x, filter_size_y, 47 | dimensions)) 48 | 49 | assert axis < dimensions, "Invalid axis." 50 | 51 | for i in range(number_to_show): 52 | show(weights[:, :, axis, i]) 53 | 54 | 55 | def to_rgb(greyscale_image): 56 | return cv2.cvtColor(greyscale_image, cv2.COLOR_GRAY2RGB) 57 | 58 | 59 | def convert_greyscale_to_rgb(data): 60 | samples, image_x, image_y, channels = data.shape 61 | 62 | assert channels == 1, "Data must be greyscale." 63 | 64 | result = np.empty([samples, image_x, image_y, 3]) 65 | for i in range(samples): 66 | result[i] = to_rgb(data[i]) 67 | return result 68 | 69 | 70 | def resize_dataset(data, output_x, output_y): 71 | samples, image_x, image_y, channels = data.shape 72 | 73 | result = np.empty([samples, output_x, output_y, channels]) 74 | for i in range(samples): 75 | result[i] = cv2.resize(data[i], (output_x, output_y)) 76 | return result 77 | 78 | 79 | def get_data(data_path, batch=32, preprocess=None, shuffle=True): 80 | data_datagen = ImageDataGenerator(preprocessing_function=preprocess) 81 | return data_datagen.flow_from_directory(data_path, 82 | target_size=(IMAGE_SIZE, IMAGE_SIZE), 83 | batch_size=batch, 84 | shuffle=shuffle) 85 | 86 | 87 | def plot_confusion_matrix(cm, classes, 88 | normalize=False, 89 | title='Confusion matrix', 90 | cmap=plt.cm.Blues): 91 | plt.imshow(cm, interpolation='nearest', cmap=cmap) 92 | plt.title(title) 93 | plt.colorbar() 94 | tick_marks = np.arange(len(classes)) 95 | plt.xticks(tick_marks, classes, rotation=45) 96 | plt.yticks(tick_marks, classes) 97 | 98 | if normalize: 99 | cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis] 100 | print('Normalized confusion matrix') 101 | else: 102 | print('Confusion matrix, without normalization') 103 | 104 | thresh = cm.max() / 2.0 105 | for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])): 106 | plt.text(j, i, format(cm[i, j], '.2f'), 107 | horizontalalignment='center', 108 | color='white' if cm[i, j] > thresh else 'black') 109 | 110 | plt.tight_layout() 111 | plt.ylabel('True label') 112 | plt.xlabel('Predicted label') 113 | plt.show() 114 | 115 | 116 | def test_from_path(model, image_path): 117 | image = cv2.imread(image_path, cv2.IMREAD_COLOR) 118 | image = np.reshape(image, (1, 224, 224, 3)) 119 | return test_from_image(model, image) 120 | 121 | 122 | # Image must be of size (1, 224, 224, 3). 123 | def test_from_image(model, image): 124 | prediction = model.predict(image) 125 | class_name = np.argmax(prediction, axis=1) 126 | return prediction, class_name 127 | 128 | 129 | def flow_from_directory_statistics(dirname, batch_size=64, num_samples=1000): 130 | """ 131 | Iteratively calculate mean and std from a dataset too large to fit 132 | in memory. Uses Welford's Method - 133 | https://www.johndcook.com/blog/standard_deviation/ 134 | NOTE: there is a bug in the std calculation. 135 | 136 | Args: 137 | dirname (string) - directory with class labels and data. 138 | batch_size (int) - number of samples to process in a batch. 139 | num_samples (int) - terminate early after seeing x batches. 140 | Returns: 141 | (tuple), corresponding mean RGB values. 142 | """ 143 | datagen = get_data(dirname, batch_size) 144 | 145 | mean = np.zeros((3,), dtype=K.floatx()) 146 | running = np.zeros((3,), dtype=K.floatx()) 147 | seen = 0 148 | 149 | for x, _ in datagen: 150 | # Terminate if datagen is exhausted or sampled sufficiently. 151 | if datagen.batch_index == 0 and seen != 0: 152 | break 153 | elif seen >= num_samples: 154 | break 155 | 156 | for i in range(x.shape[0]): 157 | if seen >= num_samples: 158 | break 159 | seen += 1 160 | sample = np.sum(x[i], axis=(0, 1)) / (IMAGE_SIZE ** 2) 161 | 162 | delta = sample - mean 163 | mean = mean + delta / seen 164 | running = running + delta * (sample - mean) 165 | 166 | return mean, running / (seen - 1) 167 | 168 | 169 | def get_precomputed_statistics(directory, num_samples=50): 170 | # Keras generators use float32 which gives weird values. 171 | K.set_floatx('float64') 172 | 173 | # Get a clean datagen. 174 | vanilla_datagen = get_data(directory) 175 | 176 | # Collect a bunch of samples. 177 | x = np.zeros((num_samples, IMAGE_SIZE, IMAGE_SIZE, 3)) 178 | 179 | index = 0 180 | for x_, _ in vanilla_datagen: 181 | if index >= num_samples: 182 | break 183 | 184 | offset = min(num_samples - index, x_.shape[0]) 185 | x[index:index+offset] = x_[:offset] 186 | index += offset 187 | 188 | # Actually fit the data and compute statistics. 189 | statistics_datagen = ImageDataGenerator( 190 | featurewise_std_normalization=True, 191 | featurewise_center=True) 192 | statistics_datagen.fit(x) 193 | 194 | print("Dataset path: %s" % directory) 195 | print("Sample mean: %s" % statistics_datagen.mean) 196 | print("Sample standard deviation: %s" % statistics_datagen.std) 197 | 198 | return statistics_datagen.mean, statistics_datagen.std 199 | 200 | 201 | def samplewise_normalize(mean, std): 202 | return lambda x: (x - mean) / (std + 1e-7) 203 | 204 | 205 | def extract_layer(full_model, layer): 206 | intermediate_layer_model = Model(input=full_model.input, 207 | output=full_model.get_layer(layer).output) 208 | return intermediate_layer_model 209 | 210 | 211 | def entropy(x): 212 | return -np.sum(x * np.log(x + 1e-7)) 213 | 214 | 215 | def load_weights(model, weights_file): 216 | try: 217 | print('Loading weights from %s.' % weights_file) 218 | model.load_weights(weights_file, by_name=True) 219 | except Exception as e: 220 | print(e) 221 | print('Loading failed. Starting from scratch.') 222 | --------------------------------------------------------------------------------