├── .gitignore ├── LICENSE ├── README.md └── snn_object └── snn_object ├── S2_reconstructions ├── train_20imgs_50ms_63px_scales_fs3_3prots │ ├── train_20imgs_50ms_63px_scales_fs3_3prots_0010_images.png │ └── train_20imgs_50ms_63px_scales_fs3_3prots_0020_images.png └── train_30imgs_50ms_63px_scales_fs3_3prots │ ├── train_30imgs_50ms_63px_scales_fs3_3prots_0010_images.png │ ├── train_30imgs_50ms_63px_scales_fs3_3prots_0020_images.png │ └── train_30imgs_50ms_63px_scales_fs3_3prots_0030_images.png ├── S2_weights ├── train_20imgs_50ms_63px_scales_fs3_3prots.bin └── train_30imgs_50ms_63px_scales_fs3_3prots.bin ├── airplanes_10_6 └── 1.jpg ├── auto_jpg_resize.py ├── c1-single-spikes-from-file-test.py ├── classify-images-one-shot.py ├── classify-images.py ├── common.py ├── design_1_wrapper.bin ├── dump-blanked-c1-spikes.py ├── dump-c1-spikes.py ├── dump-c2-spikes.py ├── dump-single-c1-spikes.py ├── dvs-test.py ├── features ├── backslash_10x10.png ├── horiz_10x10.png ├── slash_10x10.png └── vert_10x10.png ├── features_blurred ├── backslash_10x10_blurred.png ├── horiz_10x10_blurred.png ├── slash_10x10_blurred.png └── vert_10x10_blurred.png ├── features_gabor ├── backslash.png ├── horiz_backslash.png ├── horiz_slash.png └── slash.png ├── features_halved_blurred ├── backslash_5x5_blurred.png ├── horiz_5x5_blurred.png ├── slash_5x5_blurred.png └── vert_5x5_blurred.png ├── gabor-image-test.py ├── iaf_psc_exp.bin ├── image-test.py ├── learn-features.py ├── learn-features_1.py ├── machinefile ├── network.py ├── network_sur.py ├── result ├── dump-c1-spikes.py ├── result_0.35backslash.txt ├── result_0.35horiz_backslash.txt ├── result_0.35horiz_slash.txt ├── result_0.35slash.txt ├── result_0.5backslash.txt ├── result_0.5horiz_backslash.txt ├── result_0.5horiz_slash.txt ├── result_0.5slash.txt ├── result_0.71backslash.txt ├── result_0.71horiz_backslash.txt ├── result_0.71horiz_slash.txt ├── result_0.71slash.txt ├── result_1.0backslash.txt ├── result_1.0horiz_backslash.txt ├── result_1.0horiz_slash.txt └── result_1.0slash.txt ├── result_right ├── result_0.35backslash.txt ├── result_0.35horiz_backslash.txt ├── result_0.35horiz_slash.txt ├── result_0.35slash.txt ├── result_0.5backslash.txt ├── result_0.5horiz_backslash.txt ├── result_0.5horiz_slash.txt ├── result_0.5slash.txt ├── result_0.71backslash.txt ├── result_0.71horiz_backslash.txt ├── result_0.71horiz_slash.txt ├── result_0.71slash.txt ├── result_1.0backslash.txt ├── result_1.0horiz_backslash.txt ├── result_1.0horiz_slash.txt └── result_1.0slash.txt ├── samples ├── combined_airplanes_mo_fa.png ├── fa_1.png ├── fa_2.png ├── fa_3.png ├── fa_4.png ├── mo_1.png ├── mo_2.png ├── mo_3.png └── mo_4.png ├── stream.py ├── train.txt ├── validation.txt ├── video-test.py ├── video ├── airplanes-motorbikes-pianos.avi └── motorbikes-faces.avi ├── visualization.py └── visualizer.py /.gitignore: -------------------------------------------------------------------------------- 1 | # Prerequisites 2 | *.d 3 | 4 | # Compiled Object files 5 | *.slo 6 | *.lo 7 | *.o 8 | *.obj 9 | 10 | # Precompiled Headers 11 | *.gch 12 | *.pch 13 | 14 | # Compiled Dynamic libraries 15 | *.so 16 | *.dylib 17 | *.dll 18 | 19 | # Fortran module files 20 | *.mod 21 | *.smod 22 | 23 | # Compiled Static libraries 24 | *.lai 25 | *.la 26 | *.a 27 | *.lib 28 | 29 | # Executables 30 | *.exe 31 | *.out 32 | *.app 33 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 clancylea 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # SNN-simulator-NEST_14.0-xilinx_FPGA_cluster 2 | A Spiking neural network simulator NEST base on FPGA‘s cluster（LIF NEURON） 3 | 4 | ## NEST 5 | NEST Simulation:http://www.nest-simulator.org 6 | NEST-github：https://github.com/nest/nest-simulator 7 | NEST-14.0 github：https://github.com/nest/nest-simulator/releases/tag/v2.14.0 8 | 9 | ## pyNN 10 | A Python package for simulator-independent specification of neuronal network models. 11 | PyNN :http://neuralensemble.org/PyNN/ 12 | 13 | ## NEST base on GPU 14 | Implemention Spiking neural network simulator NEST on multi-GPU and distributed GPU 15 | https://github.com/pnquanganh/opencl-nest 16 | Paper:A.Podobas,S.Matsuoka,Luk Wayne.Designing and accelerating spiking neural networks using OpenCL for FPGAs[C].//International Conference on Field Programmable Technology (ICFPT).Melbourne,VIC, Australia: IEEE,2017. 17 | 18 | ## snn_object_recognition on PYNN-NEST 19 | Describes a new biologically plausible mechanism for generating intermediate-level visual representations using an unsupervised learning scheme. 20 | https://github.com/roberthangu/snn_object_recognition 21 | Paper:Masquelier, Timothée, Thorpe S J. Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity[J].PLoS Computational Biology, 2007, 3(2):e31. 22 | 23 | ## others 24 | Paper：Schuman C D , Potok T E , Patton R M , et al. A Survey of Neuromorphic Computing and Neural Networks in Hardware 25 | -------------------------------------------------------------------------------- /snn_object/snn_object/S2_reconstructions/train_20imgs_50ms_63px_scales_fs3_3prots/train_20imgs_50ms_63px_scales_fs3_3prots_0010_images.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/S2_reconstructions/train_20imgs_50ms_63px_scales_fs3_3prots/train_20imgs_50ms_63px_scales_fs3_3prots_0010_images.png -------------------------------------------------------------------------------- /snn_object/snn_object/S2_reconstructions/train_20imgs_50ms_63px_scales_fs3_3prots/train_20imgs_50ms_63px_scales_fs3_3prots_0020_images.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/S2_reconstructions/train_20imgs_50ms_63px_scales_fs3_3prots/train_20imgs_50ms_63px_scales_fs3_3prots_0020_images.png -------------------------------------------------------------------------------- /snn_object/snn_object/S2_reconstructions/train_30imgs_50ms_63px_scales_fs3_3prots/train_30imgs_50ms_63px_scales_fs3_3prots_0010_images.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/S2_reconstructions/train_30imgs_50ms_63px_scales_fs3_3prots/train_30imgs_50ms_63px_scales_fs3_3prots_0010_images.png -------------------------------------------------------------------------------- /snn_object/snn_object/S2_reconstructions/train_30imgs_50ms_63px_scales_fs3_3prots/train_30imgs_50ms_63px_scales_fs3_3prots_0020_images.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/S2_reconstructions/train_30imgs_50ms_63px_scales_fs3_3prots/train_30imgs_50ms_63px_scales_fs3_3prots_0020_images.png -------------------------------------------------------------------------------- /snn_object/snn_object/S2_reconstructions/train_30imgs_50ms_63px_scales_fs3_3prots/train_30imgs_50ms_63px_scales_fs3_3prots_0030_images.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/S2_reconstructions/train_30imgs_50ms_63px_scales_fs3_3prots/train_30imgs_50ms_63px_scales_fs3_3prots_0030_images.png -------------------------------------------------------------------------------- /snn_object/snn_object/S2_weights/train_20imgs_50ms_63px_scales_fs3_3prots.bin: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/S2_weights/train_20imgs_50ms_63px_scales_fs3_3prots.bin -------------------------------------------------------------------------------- /snn_object/snn_object/S2_weights/train_30imgs_50ms_63px_scales_fs3_3prots.bin: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/S2_weights/train_30imgs_50ms_63px_scales_fs3_3prots.bin -------------------------------------------------------------------------------- /snn_object/snn_object/airplanes_10_6/1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/airplanes_10_6/1.jpg -------------------------------------------------------------------------------- /snn_object/snn_object/auto_jpg_resize.py: -------------------------------------------------------------------------------- 1 | "author:youngkun date:20180615 function:change the size of pictures in one folder" 2 | import cv2 3 | import os 4 | 5 | image_size=224 #设定尺寸 6 | source_path="./airplanes/" #源文件路径 7 | target_path="./airplanes_10*6/" #输出目标文件路径 8 | 9 | if not os.path.exists(target_path): 10 | os.makedirs(target_path) 11 | 12 | image_list=os.listdir(source_path) #获得文件名 13 | 14 | i=0 15 | for file in image_list: 16 | i=i+1 17 | image_source=cv2.imread(source_path+file)#读取图片 18 | image = cv2.resize(image_source, (100, 63),0,0, cv2.INTER_LINEAR)#修改尺寸 19 | cv2.imwrite(target_path+str(i)+".jpg",image) #重命名并且保存 20 | print("批量处理完成") 21 | 22 | -------------------------------------------------------------------------------- /snn_object/snn_object/c1-single-spikes-from-file-test.py: -------------------------------------------------------------------------------- 1 | #!/bin/ipython 2 | 3 | # ---LICENSE-BEGIN - DO NOT CHANGE OR MOVE THIS HEADER 4 | # This file is part of the Neurorobotics Platform software 5 | # Copyright (C) 2014,2015,2016,2017 Human Brain Project 6 | # 7 | # This program is free software; you can redistribute it and/or 8 | # modify it under the terms of the GNU General Public License 9 | # as published by the Free Software Foundation; either version 2 10 | # of the License, or (at your option) any later version. 11 | # 12 | # This program is distributed in the hope that it will be useful, 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 15 | # GNU General Public License for more details. 16 | # 17 | # You should have received a copy of the GNU General Public License 18 | # along with this program; if not, write to the Free Software 19 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 20 | # ---LICENSE-END 21 | 22 | import numpy as np 23 | import cv2 24 | import sys 25 | import pyNN.nest as sim 26 | import pathlib as plb 27 | import time 28 | import pickle 29 | import argparse as ap 30 | 31 | import common as cm 32 | import network as nw 33 | import visualization as vis 34 | import time 35 | 36 | parser = ap.ArgumentParser('./c1-spikes-from-file-test.py --') 37 | parser.add_argument('--c1-dumpfile', type=str, required=True, 38 | help='The output file to contain the C1 spiketrains') 39 | parser.add_argument('--dataset-label', type=str, required=True, 40 | help='The name of the dataset which was used for\ 41 | training') 42 | parser.add_argument('--plot-c1-spikes', action='store_true', 43 | help='Plot the spike trains of the C1 layers') 44 | parser.add_argument('--plot-s2-spikes', action='store_true', 45 | help='Plot the spike trains of the S2 layers') 46 | parser.add_argument('--refrac-s2', type=float, default=.1, metavar='MS', 47 | help='The refractory period of neurons in the S2 layer in ms') 48 | parser.add_argument('--sim-time', default=50, type=float, metavar='50', 49 | help='Simulation time') 50 | parser.add_argument('--threads', default=1, type=int) 51 | parser.add_argument('--target-name', type=str, 52 | help='The name of the already edge-filtered image to be\ 53 | recognized') 54 | args = parser.parse_args() 55 | 56 | sim.setup(threads=args.threads) 57 | # 58 | # Read the gabor features for reconstruction 59 | feature_imgs_dict = {} # feature string -> image 60 | for filepath in plb.Path('features_gabor').iterdir(): 61 | feature_imgs_dict[filepath.stem] = cv2.imread(filepath.as_posix(), 62 | cv2.CV_8UC1) 63 | 64 | layer_collection = {} 65 | 66 | print('Create C1 layers') 67 | t1 = time.clock() 68 | dumpfile = open(args.c1_dumpfile, 'rb') 69 | ddict = pickle.load(dumpfile) 70 | layer_collection['C1'] = {} 71 | for size, layers_as_dicts in ddict.items(): 72 | layer_list = [] 73 | for layer_as_dict in layers_as_dicts: 74 | n, m = layer_as_dict['shape'] 75 | spiketrains = layer_as_dict['segment'].spiketrains 76 | dimensionless_sts = [[s for s in st] for st in spiketrains] 77 | new_layer = nw.Layer(sim.Population(n * m, 78 | sim.SpikeSourceArray(spike_times=dimensionless_sts), 79 | label=layer_as_dict['label']), (n, m)) 80 | layer_list.append(new_layer) 81 | layer_collection['C1'][size] = layer_list 82 | print('C1 creation took {} s'.format(time.clock() - t1)) 83 | 84 | print('Creating S2 layers') 85 | t1 = time.clock() 86 | layer_collection['S2'] = nw.create_S2_layers(layer_collection['C1'], args) 87 | print('S2 creation took {} s'.format(time.clock() - t1)) 88 | 89 | for layers in layer_collection['C1'].values(): 90 | for layer in layers: 91 | layer.population.record('spikes') 92 | for layer in layer_collection['S2'].values(): 93 | layer.population.record(['spikes', 'v']) 94 | 95 | print('========= Start simulation =========') 96 | start_time = time.clock() 97 | sim.run(args.sim_time) 98 | end_time = time.clock() 99 | print('========= Stop simulation =========') 100 | print('Simulation took', end_time - start_time, 's') 101 | 102 | if args.target_name != None: 103 | outname = plb.Path(args.target_name).stem 104 | else: 105 | outname = plb.Path(args.c1_dumpfile).stem 106 | for size, layer in layer_collection['S2'].items(): 107 | for i in range(len(list(layer.projections.values())[0])): 108 | print('Reconstructing S2 features for size', size, 'feature', i) 109 | reconstruction = vis.reconstruct_S2_features(\ 110 | dict([(label, projections[i].get('weight', 'array'))\ 111 | for label, projections in layer.projections.items()]), 112 | feature_imgs_dict) 113 | cv2.imwrite('S2_reconstructions/{}_{}_{}.png'.format(outname, size, i), 114 | reconstruction) 115 | 116 | t1 = time.clock() 117 | if args.plot_c1_spikes: 118 | print('Plotting C1 spikes') 119 | vis.plot_C1_spikes(layer_collection['C1'], outname) 120 | print('Plotting spiketrains took {} s'.format(time.clock() - t1)) 121 | 122 | if args.plot_s2_spikes: 123 | print('Plotting S2 spikes') 124 | vis.plot_S2_spikes(layer_collection['S2'], outname) 125 | print('Plotting spiketrains took {} s'.format(time.clock() - t1)) 126 | 127 | sim.end() 128 | -------------------------------------------------------------------------------- /snn_object/snn_object/classify-images.py: -------------------------------------------------------------------------------- 1 | #!/bin/ipython 2 | 3 | # ---LICENSE-BEGIN - DO NOT CHANGE OR MOVE THIS HEADER 4 | # This file is part of the Neurorobotics Platform software 5 | # Copyright (C) 2014,2015,2016,2017 Human Brain Project 6 | # 7 | # This program is free software; you can redistribute it and/or 8 | # modify it under the terms of the GNU General Public License 9 | # as published by the Free Software Foundation; either version 2 10 | # of the License, or (at your option) any later version. 11 | # 12 | # This program is distributed in the hope that it will be useful, 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 15 | # GNU General Public License for more details. 16 | # 17 | # You should have received a copy of the GNU General Public License 18 | # along with this program; if not, write to the Free Software 19 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 20 | # ---LICENSE-END 21 | 22 | import numpy as np 23 | import pyNN.nest as sim 24 | import pathlib as plb 25 | import time 26 | import pickle 27 | import argparse as ap 28 | import re 29 | from sklearn import svm, metrics 30 | 31 | import network as nw 32 | import visualization as vis 33 | 34 | parser = ap.ArgumentParser('./classify-images.py --') 35 | 36 | parser.add_argument('--training-c1-dumpfile', type=str, required=True, 37 | help='The output file to contain the C1 spiketrains for\ 38 | training') 39 | parser.add_argument('--validation-c1-dumpfile', type=str, required=True, 40 | help='The output file to contain the C1 spiketrains for\ 41 | validation') 42 | 43 | parser.add_argument('--training-labels', type=str, required=True, 44 | help='Text file which contains the labels of the training\ 45 | dataset') 46 | parser.add_argument('--validation-labels', type=str, required=True, 47 | help='Text file which contains the labels of the validation\ 48 | dataset') 49 | parser.add_argument('--threads', default=1, type=int) 50 | parser.add_argument('--weights-from', type=str, required=True, 51 | help='Dumpfile of the S2 weight array') 52 | args = parser.parse_args() 53 | 54 | sim.setup(threads=args.threads, min_delay=.1) 55 | 56 | layer_collection = {} 57 | 58 | # Extracting meta-information about the simulation from the filename 59 | training_dumpfile_name = plb.Path(args.training_c1_dumpfile).stem 60 | validation_dumpfile_name = plb.Path(args.validation_c1_dumpfile).stem 61 | training_image_count = int(re.search('\d*imgs', 62 | training_dumpfile_name).group()[:-4]) 63 | validation_image_count = int(re.search('\d*imgs', 64 | validation_dumpfile_name).group()[:-4]) 65 | sim_time = float(re.search('\d*ms', validation_dumpfile_name).group()[:-2]) 66 | #sim_time=float(50) 67 | blanktime = 0 68 | occurrence=None 69 | #occurrence = re.search('\d+\.\d+blank', training_dumpfile_name) 70 | if occurrence is not None: 71 | blanktime = float(occurrence.group()[:-5]) 72 | occurrence=None 73 | #occurrence = re.search('\d+\.\d+blank', validation_dumpfile_name) 74 | if occurrence is not None: 75 | blanktime = float(occurrence.group()[:-5]) 76 | 77 | print('Create C1 layers') 78 | t1 = time.clock() 79 | training_ddict = pickle.load(open(args.training_c1_dumpfile, 'rb')) 80 | validation_ddict = pickle.load(open(args.validation_c1_dumpfile, 'rb')) 81 | layer_collection['C1'] = {} 82 | for size, layers_as_dicts in training_ddict.items(): 83 | layer_list = [] 84 | for layer_as_dict in layers_as_dicts: 85 | n, m = layer_as_dict['shape'] 86 | new_layer = nw.Layer(sim.Population(n * m, 87 | sim.SpikeSourceArray(), 88 | label=layer_as_dict['label']), (n, m)) 89 | layer_list.append(new_layer) 90 | layer_collection['C1'][size] = layer_list 91 | print('C1 creation took {} s'.format(time.clock() - t1)) 92 | t1 = time.clock() 93 | print('Creating S2 layers and reading the epoch weights') 94 | epoch_weights_list = pickle.load(open(args.weights_from, 'rb')) 95 | epoch = epoch_weights_list[-1][0] 96 | weights_dict_list = epoch_weights_list[-1][1] 97 | f_s = int(np.sqrt(list(weights_dict_list[0].values())[0].shape[0])) 98 | s2_prototype_cells = len(weights_dict_list) 99 | layer_collection['S2'] = nw.create_S2_layers(layer_collection['C1'], f_s, 100 | s2_prototype_cells, refrac_s2=.1, 101 | stdp=False, inhibition=False) 102 | print('s2 creation took {} s'.format(time.clock() - t1)) 103 | print('Creating C2 layers') 104 | t1 = time.clock() 105 | layer_collection['C2'] = nw.create_C2_layers(layer_collection['S2'], 106 | s2_prototype_cells) 107 | print('C2 creation took {} s'.format(time.clock() - t1)) 108 | 109 | for pop in layer_collection['C2']: 110 | pop.record('spikes') 111 | 112 | def set_c1_spiketrains(ddict): 113 | for size, layers_as_dicts in ddict.items(): 114 | for layer_as_dict in layers_as_dicts: 115 | spiketrains = layer_as_dict['segment'].spiketrains 116 | dimensionless_sts = [[s for s in st] for st in spiketrains] 117 | the_layer_iter = filter(lambda layer: layer.population.label\ 118 | == layer_as_dict['label'], layer_collection['C1'][size]) 119 | the_layer_iter.__next__().population.set(spike_times=dimensionless_sts) 120 | 121 | training_labels = open(args.training_labels, 'r').read().splitlines() 122 | validation_labels = open(args.validation_labels, 'r').read().splitlines() 123 | 124 | def clear_data(C2_populations): 125 | for pop in C2_populations: 126 | pop.get_data(clear=True) 127 | 128 | def extract_data_samples(image_count): 129 | samples = [] 130 | print('========= Start simulation =========') 131 | for i in range(image_count): 132 | print('Simulating for image number', i) 133 | sim.run(sim_time + blanktime) 134 | spikes =\ 135 | [list(layer_collection['C2'][prot].get_spike_counts().values())[0]\ 136 | for prot in range(s2_prototype_cells)] 137 | samples.append(spikes) 138 | clear_data(layer_collection['C2']) 139 | 140 | print('========= Stop simulation =========') 141 | return samples 142 | 143 | #### Temporary reduction to every third epoch! ############# 144 | epoch_weights_list = [index_pair[1] for index_pair in\ 145 | filter(lambda index_pair: index_pair[0] % 3 == 0, 146 | zip(range(len(epoch_weights_list)), 147 | epoch_weights_list))] 148 | ############################################################ 149 | start = time.clock() 150 | for epoch, weights_dict_list in epoch_weights_list: 151 | 152 | # Set the S2 weights to those from the file 153 | print('Setting S2 weights to epoch', epoch) 154 | for prototype in range(s2_prototype_cells): 155 | nw.set_s2_weights(layer_collection['S2'], prototype, 156 | weights_dict_list=weights_dict_list) 157 | t1 = time.clock() 158 | training_samples = [] 159 | validation_samples = [] 160 | start=time.clock() 161 | print('Setting C1 spike trains to the training dataset') 162 | set_c1_spiketrains(training_ddict) 163 | # Let the simulation run to "fill" the layer pipeline with spikes 164 | sim.run(40) 165 | clear_data(layer_collection['C2']) 166 | print('>>>>>>>>> Extracting data samples for fitting <<<<<<<<<') 167 | training_samples = extract_data_samples(training_image_count) 168 | sim.reset() 169 | print('svm_training total creation took {} s'.format(time.clock() - t1)) 170 | 171 | t1 = time.clock() 172 | print('Setting C1 spike trains to the validation dataset') 173 | set_c1_spiketrains(validation_ddict) 174 | # Let the simulation run to "fill" the layer pipeline with spikes 175 | sim.run(40) 176 | clear_data(layer_collection['C2']) 177 | print('>>>>>>>>> Extracting data samples for validation <<<<<<<<<') 178 | validation_samples = extract_data_samples(validation_image_count) 179 | sim.reset() 180 | print('svm_validation total creation took {} s'.format(time.clock() - t1)) 181 | 182 | print('Fitting SVM model onto the training samples') 183 | t1 = time.clock() 184 | clf = svm.SVC(kernel='linear') 185 | print("////////training_labels",len(training_labels)) 186 | clf.fit(training_samples, training_labels)#train svm 187 | logfile = open('log_final/{}.log'.format(plb.Path(args.weights_from).stem), 'a') 188 | print('Predicting the categories of the validation samples') 189 | predicted_labels = clf.predict(validation_samples)#validation predict 190 | print('svm_validation total creation took {} s'.format(time.clock() - t1)) 191 | print('============================================================', 192 | file=logfile) 193 | print('Epoch', epoch, file=logfile) 194 | print("///////predicted_labels:",predicted_labels) 195 | print("///////validation_labels:",validation_labels) 196 | #clf_report = metrics.classification_report(validation_labels, predicted_labels) 197 | #conf_matrix = metrics.confusion_matrix(validation_labels, predicted_labels) 198 | #print(clf_report, file=logfile) 199 | #print(clf_report) 200 | #print(conf_matrix, file=logfile) 201 | #print(conf_matrix) 202 | logfile.close() 203 | print('svm total creation took {} s'.format(time.clock() - start)) 204 | #labels：分类报告中显示的类标签的索引列表 205 | 206 | #target_names：显示与labels对应的名称 207 | 208 | #digits：指定输出格式的精确度 209 | 210 | #精度(precision) = 正确预测的个数(TP)/被预测正确的个数(TP+FP) 211 | 212 | #召回率(recall)=正确预测的个数(TP)/预测个数(TP+FN) 213 | 214 | 215 | sim.end() 216 | -------------------------------------------------------------------------------- /snn_object/snn_object/common.py: -------------------------------------------------------------------------------- 1 | 2 | # ---LICENSE-BEGIN - DO NOT CHANGE OR MOVE THIS HEADER 3 | # This file is part of the Neurorobotics Platform software 4 | # Copyright (C) 2014,2015,2016,2017 Human Brain Project 5 | # 6 | # This program is free software; you can redistribute it and/or 7 | # modify it under the terms of the GNU General Public License 8 | # as published by the Free Software Foundation; either version 2 9 | # of the License, or (at your option) any later version. 10 | # 11 | # This program is distributed in the hope that it will be useful, 12 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14 | # GNU General Public License for more details. 15 | # 16 | # You should have received a copy of the GNU General Public License 17 | # along with this program; if not, write to the Free Software 18 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 19 | # ---LICENSE-END 20 | 21 | import argparse as ap 22 | import cv2 23 | import numpy as np 24 | import pickle 25 | import time 26 | from typing import Dict 27 | 28 | def parse_args(): 29 | """ 30 | Defines the valid commandline options and the variables they are linked to. 31 | 32 | Returns: 33 | An object which contains the variables which correspond to the 34 | commandline options. 35 | """ 36 | dflt_move=4 37 | parser = ap.ArgumentParser(description='SNN feature detector') 38 | parser.add_argument('--c1-output', type=str, default='C1_reconstructions/', 39 | help='The toplevel output directory for C1\ 40 | reconstructions') 41 | parser.add_argument('--delta', metavar='vert', default=dflt_move, type=int, 42 | help='The horizontal and vertical distance between the\ 43 | basic recognizers') 44 | parser.add_argument('--feature-dir', type=str, 45 | help='A directory where the features are stored as images') 46 | parser.add_argument('--filter', choices=['canny', 'sobel', 'none'], 47 | default='none', help='Sets the edge filter to be used.\ 48 | Defaults to \'none\'') 49 | parser.add_argument('--frames', default=10, type=int, 50 | help='The number of video frames to be processed') 51 | parser.add_argument('--no-c1', action='store_true', 52 | help='Disables the creation of C1 layers') 53 | parser.add_argument('--plot-spikes', action='store_true', 54 | help='Plot the spike trains of all layers') 55 | parser.add_argument('--plot-weights', action='store_true', 56 | help='Plots the learned feature weights and exits') 57 | parser.add_argument('--refrac-s1', type=float, default=.1, metavar='MS', 58 | help='The refractory period of neurons in the S1 layer in ms') 59 | parser.add_argument('--refrac-s2', type=float, default=.1, metavar='MS', 60 | help='The refractory period of neurons in the S2 layer in ms') 61 | parser.add_argument('--refrac-c1', type=float, default=.1, metavar='MS', 62 | help='The refractory period of neurons in the C1 layer in ms') 63 | parser.add_argument('--reconstruct-s1-img', action='store_true', 64 | help='If set, draws a reconstruction of the recognized\ 65 | features from S1') 66 | parser.add_argument('--reconstruct-c1-img', action='store_true', 67 | help='If set, draws a reconstruction of the recognized\ 68 | features from C1') 69 | parser.add_argument('--scales', default=[1.0, 0.71, 0.5, 0.35, 0.25], 70 | nargs='+', type=float, 71 | help='A list of image scales for which to create\ 72 | layers. Defaults to [1, 0.71, 0.5, 0.35, 0.25]') 73 | parser.add_argument('--sim-time', default=100, type=float, help='Simulation time') 74 | parser.add_argument('--target-name', type=str, 75 | help='The name of the already edge-filtered image to\ 76 | be recognized') 77 | args = parser.parse_args() 78 | print(args) 79 | return args 80 | 81 | def filter_img(target_img, filter_type): 82 | """ 83 | Performs the given edge detector on the given image 84 | 85 | Arguments: 86 | `target_img`: The image to detect edges from 87 | 88 | `filter_type`: The filter to be applied to the target image. Can be one 89 | of 'canny', 'sobel' or 'none', if the image is to be 90 | used as-is. 91 | 92 | Returns: 93 | An image containing the edges of the target image 94 | """ 95 | blurred_img = cv2.GaussianBlur(target_img, (5, 5), 1.4) 96 | filtered_img = None 97 | if filter_type == 'none': 98 | return target_img 99 | if filter_type == 'canny': 100 | filtered_img = cv2.Canny(blurred_img, 70, 210) 101 | else: 102 | dx = cv2.Sobel(blurred_img, cv2.CV_64F, 1, 0) 103 | dy = cv2.Sobel(blurred_img, cv2.CV_64F, 0, 1) 104 | edge_detected = cv2.sqrt(dx * dx + dy * dy) 105 | filtered_img = cv2.convertScaleAbs(edge_detected) 106 | return filtered_img 107 | 108 | def get_gabor_feature_names(): 109 | """ 110 | Returns the feature names of the gabor filtered images 111 | """ 112 | return ['slash', 'horiz_slash', 'horiz_backslash', 'backslash'] 113 | 114 | def get_gabor_edges(target_img) -> Dict[str, np.array]: 115 | """ 116 | Computes the gabor filtered images for four orientations for the given 117 | unfiltered image 118 | 119 | Parameters: 120 | `target_img`: The original target image 121 | 122 | Returns: 123 | A dictionary which contains for each name the corresponding filtered 124 | image 125 | """ 126 | angles = [np.pi / 8, np.pi / 4 + np.pi / 8, np.pi / 2 + np.pi / 8, 127 | 3 * np.pi / 4 + np.pi / 8] 128 | feature_names = get_gabor_feature_names() 129 | blurred_img = cv2.GaussianBlur(target_img, (5, 5), 5)#gao si filter pinghua 130 | return dict([(name, 131 | cv2.convertScaleAbs(\ 132 | cv2.filter2D(blurred_img, cv2.CV_64F, 133 | cv2.getGaborKernel((5, 5), 1.4, angle, 5, 1))))\ 134 | for name, angle in zip(feature_names, angles)])#2dfilter 64 double 135 | 136 | def read_and_prepare_img(target_name, filter_type): 137 | """ 138 | Reads the input image and performs the edge detector of the passed 139 | commandline arguments on it 140 | 141 | Arguments: 142 | `target_name`: The name of the image to be read 143 | 144 | `filter_type`: The filter to be applied to the target image. Can be one 145 | of 'canny', 'sobel' or 'none', if the image is to be 146 | used as-is. 147 | 148 | Returns: 149 | An image containing the edges of the target image 150 | """ 151 | target_img = cv2.imread(target_name, cv2.CV_8U) 152 | # Optionally resize the image to 300 pixels (or less) in height 153 | return filter_img(target_img, filter_type) 154 | 155 | def float_to_fourcc_string(x): 156 | """ 157 | Converns a float to its fourcc number as a string. 158 | 159 | Parameters: 160 | `x`: The float as returned by cv2.VideoCapture.get(cv2.CAP_PROP_FOURCC) 161 | 162 | Returns: 163 | The used encoder extension as a string 164 | """ 165 | x = int(x) 166 | c1 = chr(x & 0xFF) 167 | c2 = chr((x & 0xFF00) >> 8) 168 | c3 = chr((x & 0xFF0000) >> 16) 169 | c4 = chr((x & 0xFF000000) >> 24) 170 | return c1 + c2 + c3 + c4 171 | 172 | def fourcc_string_to_int(s): 173 | """ 174 | Converns a fourcc string to a float 175 | 176 | Parameters: 177 | `s`: The fourcc string to be converted 178 | 179 | Returns: 180 | A float representing the code for the given codec string 181 | """ 182 | n1 = ord(s[0]) 183 | n2 = ord(s[1]) 184 | n3 = ord(s[2]) 185 | n4 = ord(s[3]) 186 | return (n4 << 24) + (n3 << 16) + (n2 << 8) + n1 187 | -------------------------------------------------------------------------------- /snn_object/snn_object/design_1_wrapper.bin: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/design_1_wrapper.bin -------------------------------------------------------------------------------- /snn_object/snn_object/dump-blanked-c1-spikes.py: -------------------------------------------------------------------------------- 1 | #!/bin/ipython 2 | 3 | # ---LICENSE-BEGIN - DO NOT CHANGE OR MOVE THIS HEADER 4 | # This file is part of the Neurorobotics Platform software 5 | # Copyright (C) 2014,2015,2016,2017 Human Brain Project 6 | # 7 | # This program is free software; you can redistribute it and/or 8 | # modify it under the terms of the GNU General Public License 9 | # as published by the Free Software Foundation; either version 2 10 | # of the License, or (at your option) any later version. 11 | # 12 | # This program is distributed in the hope that it will be useful, 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 15 | # GNU General Public License for more details. 16 | # 17 | # You should have received a copy of the GNU General Public License 18 | # along with this program; if not, write to the Free Software 19 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 20 | # ---LICENSE-END 21 | 22 | import numpy as np 23 | import cv2 24 | import sys 25 | import pyNN.nest as sim 26 | import pathlib as plb 27 | import time 28 | import pickle 29 | import argparse as ap 30 | 31 | import network as nw 32 | import visualization as vis 33 | 34 | parser = ap.ArgumentParser('./dump-blanked-c1-spikes.py --') 35 | parser.add_argument('--blanktime', type=float, default=120, 36 | help='The blank time between the image spikes. Defaults to\ 37 | 120 ms') 38 | parser.add_argument('--dataset-label', type=str, required=True, 39 | help='The name of the dataset which was used for\ 40 | training') 41 | parser.add_argument('--training-dir', type=str, required=True, 42 | help='The directory with the training images') 43 | parser.add_argument('--refrac-c1', type=float, default=.1, metavar='0.1', 44 | help='The refractory period of neurons in the C1 layer in\ 45 | ms') 46 | parser.add_argument('--sim-time', default=50, type=float, help='Simulation time', 47 | metavar='50') 48 | parser.add_argument('--scales', default=[1.0, 0.71, 0.5, 0.35], 49 | nargs='+', type=float, 50 | help='A list of image scales for which to create\ 51 | layers. Defaults to [1, 0.71, 0.5, 0.35]') 52 | parser.add_argument('--threads', default=1, type=int) 53 | args = parser.parse_args() 54 | 55 | training_path = plb.Path(args.training_dir) 56 | imgs = [(filename.stem, cv2.imread(filename.as_posix(), cv2.CV_8UC1))\ 57 | for filename in sorted(training_path.glob('*.png'))] 58 | 59 | sim.setup(threads=args.threads) 60 | 61 | layer_collection = {} 62 | 63 | print('Create S1 layers') 64 | t1 = time.clock() 65 | layer_collection['S1'] =\ 66 | nw.create_gabor_input_layers_for_scales(imgs[0][1], args.scales) 67 | nw.create_cross_layer_inhibition(layer_collection['S1']) 68 | print('S1 layer creation took {} s'.format(time.clock() - t1)) 69 | 70 | print('Create C1 layers') 71 | t1 = time.clock() 72 | layer_collection['C1'] = nw.create_C1_layers(layer_collection['S1'], 73 | args.refrac_c1) 74 | nw.create_local_inhibition(layer_collection['C1']) 75 | print('C1 creation took {} s'.format(time.clock() - t1)) 76 | 77 | for layer_name in ['C1']: 78 | if layer_name in layer_collection: 79 | for layers in layer_collection[layer_name].values(): 80 | for layer in layers: 81 | layer.population.record('spikes') 82 | 83 | # Simulate for a certain time to allow the whole layer pipeline to "get filled" 84 | sim.run(40) 85 | 86 | print('========= Start simulation =========') 87 | start_time = time.clock() 88 | count = 0 89 | for filename, target_img in imgs: 90 | t1 = time.clock() 91 | print('Simulating for', filename, 'number', count) 92 | count += 1 93 | nw.set_i_offsets_for_all_scales_to(layer_collection['S1'], target_img) 94 | sim.run(args.sim_time) 95 | nw.set_blank_i_offsets(layer_collection['S1']) 96 | sim.run(args.blanktime) 97 | print('Took', time.clock() - t1, 'seconds') 98 | end_time = time.clock() 99 | print('========= Stop simulation =========') 100 | print('Simulation took', end_time - start_time, 's') 101 | 102 | ddict = {} 103 | dataset_label = '{}_{}imgs_{}ms_{}px_scales'.format(args.dataset_label, 104 | len(imgs), args.sim_time, 105 | imgs[0][1].shape[0]) 106 | for size, layers in layer_collection['C1'].items(): 107 | ddict[size] = [{'segment': layer.population.get_data().segments[0], 108 | 'shape': layer.shape, 109 | 'label': layer.population.label } for layer in layers] 110 | dataset_label += '_{}'.format(size) 111 | dataset_label += '_{}blank'.format(float(args.blanktime)) 112 | 113 | dumpname = 'C1_spikes/{}.bin'.format(dataset_label) 114 | print('Dumping spikes for all scales and layers to file', dumpname) 115 | dumpfile = open(dumpname, 'wb') 116 | pickle.dump(ddict, dumpfile, protocol=4) 117 | dumpfile.close() 118 | 119 | sim.end() 120 | -------------------------------------------------------------------------------- /snn_object/snn_object/dump-c1-spikes.py: -------------------------------------------------------------------------------- 1 | #!/bin/ipython 2 | 3 | # ---LICENSE-BEGIN - DO NOT CHANGE OR MOVE THIS HEADER 4 | # This file is part of the Neurorobotics Platform software 5 | # Copyright (C) 2014,2015,2016,2017 Human Brain Project 6 | # 7 | # This program is free software; you can redistribute it and/or 8 | # modify it under the terms of the GNU General Public License 9 | # as published by the Free Software Foundation; either version 2 10 | # of the License, or (at your option) any later version. 11 | # 12 | # This program is distributed in the hope that it will be useful, 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 15 | # GNU General Public License for more details. 16 | # 17 | # You should have received a copy of the GNU General Public License 18 | # along with this program; if not, write to the Free Software 19 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 20 | # ---LICENSE-END 21 | 22 | import numpy as np 23 | import cv2 24 | import sys 25 | import pyNN.nest as sim 26 | import pathlib as plb 27 | import time 28 | import pickle 29 | import argparse as ap 30 | 31 | import network as nw 32 | import visualization as vis 33 | from pyNN.utility.plotting import Figure, Panel 34 | parser = ap.ArgumentParser('./dump-c1-spikes.py --') 35 | parser.add_argument('--dataset-label', type=str, required=True, 36 | help='The name of the dataset which was used for\ 37 | training') 38 | parser.add_argument('--training-dir', type=str, required=True, 39 | help='The directory with the training images') 40 | parser.add_argument('--refrac-c1', type=float, default=0.01, metavar='0.01', 41 | help='The refractory period of neurons in the C1 layer in\ 42 | ms') 43 | parser.add_argument('--sim-time', default=50, type=float, help='Simulation time', 44 | metavar='50') 45 | parser.add_argument('--scales', default=[1.0, 0.71, 0.5, 0.35], 46 | nargs='+', type=float, 47 | help='A list of image scales for which to create\ 48 | layers. Defaults to [1, 0.71, 0.5, 0.35]') 49 | parser.add_argument('--threads', default=1, type=int) 50 | args = parser.parse_args() 51 | 52 | training_path = plb.Path(args.training_dir) 53 | 54 | imgs = [(filename.stem, cv2.imread(filename.as_posix(), cv2.CV_8UC1))\ 55 | for filename in sorted(training_path.glob('*'+'.jpg'))] 56 | 57 | sim.setup(threads=args.threads) 58 | 59 | layer_collection = {} 60 | 61 | print('Create S1 layers') 62 | t1 = time.time() 63 | layer_collection['S1'] =\ 64 | nw.create_gabor_input_layers_for_scales(imgs[0][1], args.scales) 65 | print('S1 layer creation gabor_input_layers took {} s'.format(time.time() - t1)) 66 | t2 = time.time() 67 | nw.create_cross_layer_inhibition(layer_collection['S1']) 68 | print('S1 layer creation layer_inhibition took {} s'.format(time.time() - t2)) 69 | 70 | 71 | print('Create C1 layers') 72 | t1 = time.time() 73 | layer_collection['C1'] = nw.create_C1_layers(layer_collection['S1'], 74 | args.refrac_c1) 75 | 76 | print("Projection.get('weight')::",layer_collection['C1'].get('weight')) 77 | print('C1 creation took {} s'.format(time.time() - t1)) 78 | t2 = time.time() 79 | nw.create_local_inhibition(layer_collection['C1']) 80 | print('C1 creation create_local_inhibition took {} s'.format(time.time() - t2)) 81 | 82 | for layer_name in ['C1']: 83 | if layer_name in layer_collection: 84 | for layers in layer_collection[layer_name].values(): 85 | for layer in layers: 86 | layer.population.record('spikes') 87 | 88 | # Simulate for a certain time to allow the whole layer pipeline to "get filled" 89 | #sim.run(2) 90 | dataset_label = '{}_{}imgs_{}ms_{}px_scales'.format(args.dataset_label, 91 | len(imgs), args.sim_time, 92 | imgs[0][1].shape[0]) 93 | 94 | c1_plots_dir_path = plb.Path('plots/C1/' + dataset_label) 95 | if not c1_plots_dir_path.exists(): 96 | c1_plots_dir_path.mkdir(parents=True) 97 | 98 | 99 | print('========= Start simulation =========') 100 | start_time = time.time() 101 | count = 0 102 | for filename, target_img in imgs: 103 | 104 | print('Simulating for', filename, 'number', count) 105 | count += 1 106 | t2=time.time() 107 | nw.set_i_offsets_for_all_scales_to(layer_collection['S1'], target_img) 108 | print('set_i_offsets_for_all_scales_to', time.time() - t2, 'seconds') 109 | t1 = time.time() 110 | sim.run(args.sim_time) 111 | 112 | #vis.plot_C1_spikes(layer_collection['C1'], 113 | # '{}_image_{}'.format(dataset_label, count), 114 | # out_dir_name=c1_plots_dir_path.as_posix()) 115 | #sim.run(1) 116 | print('Took', time.time() - t1, 'seconds') 117 | end_time = time.time() 118 | print('========= Stop simulation =========') 119 | print('Simulation took', end_time - start_time, 's') 120 | 121 | ddict = {} 122 | 123 | i=0 124 | for size, layers in layer_collection['C1'].items(): 125 | ddict[size] = [{'segment': layer.population.get_data().segments[0], 126 | 'shape': layer.shape, 127 | 'label': layer.population.label } for layer in layers] 128 | for layer in layers: 129 | f=open('result/result_'+str(size)+str(layer.population.label)+'.txt','w+') 130 | f.write(str(layer.population.get_data().segments[0].spiketrains)) 131 | f.close() 132 | """ 133 | for layer in layers: 134 | data = layer.population.get_data().segments[0] 135 | print(data) 136 | #vm = data.filter(name="v")[0] 137 | #spike = data.filter(name="spike")[0] 138 | i=i+1 139 | Figure( 140 | Panel(data.spiketrains, xlabel="Time (ms)", xticks=True) 141 | ).save("simulation_results"+str(i)+".png") 142 | dataset_label += '_{}'.format(size) 143 | """ 144 | 145 | #dumpname = 'C1_spikes/{}.txt'.format(dataset_label) 146 | #print('Dumping spikes for all scales and layers to file', dumpname) 147 | #dumpfile = open(dumpname, 'wb') 148 | #pickle.dump(ddict, dumpfile, protocol=4) 149 | #dumpfile.close() 150 | 151 | sim.end() 152 | -------------------------------------------------------------------------------- /snn_object/snn_object/dump-c2-spikes.py: -------------------------------------------------------------------------------- 1 | #!/bin/ipython 2 | 3 | # ---LICENSE-BEGIN - DO NOT CHANGE OR MOVE THIS HEADER 4 | # This file is part of the Neurorobotics Platform software 5 | # Copyright (C) 2014,2015,2016,2017 Human Brain Project 6 | # 7 | # This program is free software; you can redistribute it and/or 8 | # modify it under the terms of the GNU General Public License 9 | # as published by the Free Software Foundation; either version 2 10 | # of the License, or (at your option) any later version. 11 | # 12 | # This program is distributed in the hope that it will be useful, 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 15 | # GNU General Public License for more details. 16 | # 17 | # You should have received a copy of the GNU General Public License 18 | # along with this program; if not, write to the Free Software 19 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 20 | # ---LICENSE-END 21 | 22 | import numpy as np 23 | import cv2 24 | import pyNN.nest as sim 25 | import pathlib as plb 26 | import time 27 | import pickle 28 | import argparse as ap 29 | import re 30 | 31 | import network as nw 32 | 33 | parser = ap.ArgumentParser('./dump-c2-spikes.py --') 34 | parser.add_argument('--training-c1-dumpfile', type=str, required=True, 35 | help='The output file to contain the C1 spiketrains for\ 36 | training') 37 | parser.add_argument('--validation-c1-dumpfile', type=str, required=True, 38 | help='The output file to contain the C1 spiketrains for\ 39 | validation') 40 | parser.add_argument('--threads', default=4, type=int) 41 | parser.add_argument('--weights-from', type=str, required=True, 42 | help='Dumpfile of the S2 weight array') 43 | args = parser.parse_args() 44 | 45 | sim.setup(threads=args.threads, min_delay=.1) 46 | 47 | layer_collection = {} 48 | 49 | # Extracting meta-information about the simulation from the filename 50 | training_dumpfile_name = plb.Path(args.training_c1_dumpfile).stem 51 | validation_dumpfile_name = plb.Path(args.validation_c1_dumpfile).stem 52 | training_image_count = int(re.search('\d*imgs', 53 | training_dumpfile_name).group()[:-4]) 54 | validation_image_count = int(re.search('\d*imgs', 55 | validation_dumpfile_name).group()[:-4]) 56 | #training_sim_time = float(re.search('\d+\.\d+ms', 57 | # training_dumpfile_name).group()[:-2]) 58 | 59 | #validation_sim_time = float(re.search('\d+\.\d+ms', 60 | # validation_dumpfile_name).group()[:-2]) 61 | training_sim_time=float(50) 62 | validation_sim_time=float(50) 63 | blanktime = 0 64 | occurrence = re.search('\d+\.\d+blank', training_dumpfile_name) 65 | if occurrence is not None: 66 | blanktime = float(occurrence.group()[:-5]) 67 | 68 | print('Create C1 layers') 69 | t1 = time.clock() 70 | training_ddict = pickle.load(open(args.training_c1_dumpfile, 'rb')) 71 | validation_ddict = pickle.load(open(args.validation_c1_dumpfile, 'rb')) 72 | layer_collection['C1'] = {} 73 | for size, layers_as_dicts in training_ddict.items(): 74 | layer_list = [] 75 | for layer_as_dict in layers_as_dicts: 76 | n, m = layer_as_dict['shape'] 77 | new_layer = nw.Layer(sim.Population(n * m, 78 | sim.SpikeSourceArray(), 79 | label=layer_as_dict['label']), (n, m)) 80 | layer_list.append(new_layer) 81 | layer_collection['C1'][size] = layer_list 82 | print('C1 creation took {} s'.format(time.clock() - t1)) 83 | t1 = time.clock() 84 | print('Creating S2 layers and reading the epoch weights') 85 | epoch_weights_list = pickle.load(open(args.weights_from, 'rb')) 86 | epoch = epoch_weights_list[-1][0] 87 | weights_dict_list = epoch_weights_list[-1][1] 88 | f_s = int(np.sqrt(list(weights_dict_list[0].values())[0].shape[0])) 89 | s2_prototype_cells = len(weights_dict_list) 90 | print(s2_prototype_cells) 91 | layer_collection['S2'] = nw.create_S2_layers(layer_collection['C1'], f_s, 92 | s2_prototype_cells, refrac_s2=.1, 93 | stdp=False, inhibition=True) 94 | print('S2 creation took {} s'.format(time.clock() - t1)) 95 | 96 | print('Creating C2 layers') 97 | t1 = time.clock() 98 | layer_collection['C2'] = nw.create_C2_layers(layer_collection['S2'], 99 | s2_prototype_cells) 100 | print("/////////////////s2_prototype_cells",s2_prototype_cells) 101 | print('C2 creation took {} s'.format(time.clock() - t1)) 102 | 103 | for pop in layer_collection['C2']: 104 | pop.record('spikes') 105 | 106 | def set_c1_spiketrains(ddict): 107 | for size, layers_as_dicts in ddict.items(): 108 | for layer_as_dict in layers_as_dicts: 109 | spiketrains = layer_as_dict['segment'].spiketrains 110 | dimensionless_sts = [[s for s in st] for st in spiketrains] 111 | the_layer_iter = filter(lambda layer: layer.population.label\ 112 | == layer_as_dict['label'], layer_collection['C1'][size]) 113 | the_layer_iter.__next__().population.set(spike_times=dimensionless_sts) 114 | 115 | def extract_spiketrains(image_count, sim_time): 116 | print('========= Start simulation =========') 117 | print('Simulating for', image_count, 'images') 118 | sim.run((sim_time + blanktime) * image_count) 119 | print('========= Stop simulation =========') 120 | return [layer_collection['C2'][prot].get_data(clear=True).segments[0]\ 121 | .spiketrains[0] for prot in range(s2_prototype_cells)] 122 | 123 | c2_training_spikes = [] 124 | c2_validation_spikes = [] 125 | t1 = time.clock() 126 | for epoch, weights_dict_list in epoch_weights_list: 127 | print("1111111111") 128 | # Set the S2 weights to those from the file 129 | print('Setting S2 weights to epoch', epoch) 130 | for prototype in range(s2_prototype_cells): 131 | nw.set_s2_weights(layer_collection['S2'], prototype, 132 | weights_dict_list=weights_dict_list) 133 | 134 | print('Setting C1 spike trains to the training dataset') 135 | set_c1_spiketrains(training_ddict) 136 | # Let the simulation run to "fill" the layer pipeline with spikes 137 | sim.run(40) 138 | print('>>>>>>>>> Extracting spike trains for learning <<<<<<<<<') 139 | c2_training_spikes.append((epoch, 140 | extract_spiketrains(training_image_count, training_sim_time))) 141 | sim.reset() 142 | print('C2 train took {} s'.format(time.clock() - t1)) 143 | t1 = time.clock() 144 | for epoch, weights_dict_list in epoch_weights_list: 145 | print('Setting C1 spike trains to the validation dataset') 146 | set_c1_spiketrains(validation_ddict) 147 | # Let the simulation run to "fill" the layer pipeline with spikes 148 | sim.run(40) 149 | print('>>>>>>>>> Extracting spike trains for validation <<<<<<<<<') 150 | c2_validation_spikes.append((epoch, 151 | extract_spiketrains(validation_image_count, validation_sim_time))) 152 | sim.reset() 153 | print('C2 validation took {} s'.format(time.clock() - t1)) 154 | 155 | c2_training_dumpfile_name = 'C2_spikes/{}_fs{}_{}prots.bin'\ 156 | .format(training_dumpfile_name, f_s, s2_prototype_cells) 157 | c2_validation_dumpfile_name = 'C2_spikes/{}_fs{}_{}prots.bin'\ 158 | .format(validation_dumpfile_name, f_s, s2_prototype_cells) 159 | c2_training_dumpfile = open(c2_training_dumpfile_name, 'wb') 160 | c2_validation_dumpfile = open(c2_validation_dumpfile_name, 'wb') 161 | print('Dumping C2 training spikes to file', c2_training_dumpfile_name) 162 | print('Dumping C2 validation spikes to file', c2_validation_dumpfile_name) 163 | pickle.dump(c2_training_spikes, c2_training_dumpfile, protocol=4) 164 | pickle.dump(c2_validation_spikes, c2_validation_dumpfile, protocol=4) 165 | 166 | sim.end() 167 | -------------------------------------------------------------------------------- /snn_object/snn_object/dump-single-c1-spikes.py: -------------------------------------------------------------------------------- 1 | #!/bin/ipython 2 | 3 | # ---LICENSE-BEGIN - DO NOT CHANGE OR MOVE THIS HEADER 4 | # This file is part of the Neurorobotics Platform software 5 | # Copyright (C) 2014,2015,2016,2017 Human Brain Project 6 | # 7 | # This program is free software; you can redistribute it and/or 8 | # modify it under the terms of the GNU General Public License 9 | # as published by the Free Software Foundation; either version 2 10 | # of the License, or (at your option) any later version. 11 | # 12 | # This program is distributed in the hope that it will be useful, 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 15 | # GNU General Public License for more details. 16 | # 17 | # You should have received a copy of the GNU General Public License 18 | # along with this program; if not, write to the Free Software 19 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 20 | # ---LICENSE-END 21 | 22 | import numpy as np 23 | import cv2 24 | import sys 25 | import pyNN.nest as sim 26 | import pathlib as plb 27 | import time 28 | import pickle 29 | import argparse as ap 30 | 31 | import network as nw 32 | import visualization as vis 33 | 34 | try: 35 | from mpi4py import MPI 36 | except ImportError: 37 | raise Exception("Trying to gather data without MPI installed. If you are\ 38 | not running a distributed simulation, this is a bug in PyNN.") 39 | 40 | parser = ap.ArgumentParser('./dump-single-c1-spikes.py --') 41 | parser.add_argument('--refrac-c1', type=float, default=.1, metavar='0.1', 42 | help='The refractory period of neurons in the C1 layer in\ 43 | ms') 44 | parser.add_argument('--sim-time', default=50, type=float, help='Simulation time', 45 | metavar='50') 46 | parser.add_argument('--scales', default=[1.0, 0.71, 0.5, 0.35, 0.25], 47 | nargs='+', type=float, 48 | help='A list of image scales for which to create\ 49 | layers. Defaults to [1, 0.71, 0.5, 0.35, 0.25]') 50 | parser.add_argument('--target-name', type=str, 51 | help='The name of the already edge-filtered image to be\ 52 | recognized') 53 | parser.add_argument('--threads', default=1, type=int) 54 | args = parser.parse_args() 55 | 56 | MPI_ROOT = 0 57 | 58 | def is_root(): 59 | return MPI.COMM_WORLD.rank == MPI_ROOT 60 | 61 | target_path = plb.Path(args.target_name) 62 | target_img = cv2.imread(target_path.as_posix(), cv2.CV_8UC1) 63 | 64 | sim.setup(threads=args.threads) 65 | 66 | layer_collection = {} 67 | 68 | print('Create S1 layers') 69 | t1 = time.clock() 70 | layer_collection['S1'] =\ 71 | nw.create_gabor_input_layers_for_scales(target_img, args.scales) 72 | nw.create_cross_layer_inhibition(layer_collection['S1']) 73 | print('S1 layer creation took {} s'.format(time.clock() - t1)) 74 | 75 | print('Create C1 layers') 76 | t1 = time.clock() 77 | layer_collection['C1'] = nw.create_C1_layers(layer_collection['S1'], 78 | args.refrac_c1) 79 | nw.create_local_inhibition(layer_collection['C1']) 80 | print('C1 creation took {} s'.format(time.clock() - t1)) 81 | 82 | for layers in layer_collection['C1'].values(): 83 | for layer in layers: 84 | layer.population.record('spikes') 85 | 86 | print('========= Start simulation =========') 87 | start_time = time.clock() 88 | sim.run(args.sim_time) 89 | end_time = time.clock() 90 | print('========= Stop simulation =========') 91 | print('Simulation took', end_time - start_time, 's') 92 | 93 | print('Dumping spikes for all scales and layers') 94 | ddict = {} 95 | filename = 'C1_spike_data/' + target_path.stem 96 | for size, layers in layer_collection['C1'].items(): 97 | ddict[size] = [{'segment': layer.population.get_data().segments[0], 98 | 'shape': layer.shape, 99 | 'label': layer.population.label } for layer in layers] 100 | filename += '_{}'.format(size) 101 | if is_root(): 102 | dumpfile = open('{}_{}ms_norefrac.bin'.format(filename, args.sim_time), 'wb') 103 | pickle.dump(ddict, dumpfile, protocol=4) 104 | dumpfile.close() 105 | 106 | sim.end() 107 | -------------------------------------------------------------------------------- /snn_object/snn_object/dvs-test.py: -------------------------------------------------------------------------------- 1 | 2 | # ---LICENSE-BEGIN - DO NOT CHANGE OR MOVE THIS HEADER 3 | # This file is part of the Neurorobotics Platform software 4 | # Copyright (C) 2014,2015,2016,2017 Human Brain Project 5 | # 6 | # This program is free software; you can redistribute it and/or 7 | # modify it under the terms of the GNU General Public License 8 | # as published by the Free Software Foundation; either version 2 9 | # of the License, or (at your option) any later version. 10 | # 11 | # This program is distributed in the hope that it will be useful, 12 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14 | # GNU General Public License for more details. 15 | # 16 | # You should have received a copy of the GNU General Public License 17 | # along with this program; if not, write to the Free Software 18 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 19 | # ---LICENSE-END 20 | 21 | from mpl_toolkits.mplot3d import Axes3D 22 | import pathlib as plb 23 | import matplotlib.pyplot as plt 24 | import pickle 25 | import numpy as np 26 | import stream 27 | import pyNN.nest as sim 28 | import visualization as vis 29 | import cv2 30 | import pyNN.utility.plotting as pynnplt 31 | 32 | import common as cm 33 | import network as nw 34 | 35 | 36 | args = cm.parse_args() 37 | weights_dict, feature_imgs_dict = nw.train_weights(args.feature_dir) 38 | 39 | # The training part is done. Go on with the "actual" simulation 40 | sim.setup(threads=4) 41 | 42 | file_extension = plb.Path(args.target_name).suffix 43 | filename = plb.Path(args.target_name).stem 44 | 45 | target = stream.read_stream(args.target_name) 46 | size = 1 47 | S1_layers = nw.create_S1_layers(target, weights_dict, [size], args, 48 | is_bag=True) 49 | 50 | # NOTE: Since in your original code you're using only size 1 in creating the 51 | # corner layers, I don't create an extra dictionary to store only that one 52 | # layer. Hence the `corner_layer` variable. 53 | corner_layer = nw.create_corner_layer_for(S1_layers[size]) 54 | corner_layer_wrapped = { size: [corner_layer] } 55 | 56 | layer_collection = {'S1' : S1_layers, 57 | 'corner': corner_layer_wrapped} 58 | 59 | for layer_dict in layer_collection.values(): 60 | for layers in layer_dict.values(): 61 | for layer in layers: 62 | layer.population.record('spikes') 63 | 64 | stimuli_duration = 0 65 | if file_extension == '.bag': 66 | stimuli_duration = target.duration 67 | 68 | print('========= Start simulation: {} ========='.format(sim.get_current_time())) 69 | sim.run(stimuli_duration + 300) 70 | print('========= Stop simulation: {} ========='.format(sim.get_current_time())) 71 | 72 | 73 | # visualize spatiotemporal spiketrain 74 | def extract_spatiotemporal_spiketrain(size, layer_name, spiketrain, shape): 75 | x = [] 76 | y = [] 77 | times = [] 78 | for populationIdx, neuron in enumerate(spiketrain): 79 | imageIdx = [populationIdx / shape[0], populationIdx % shape[0]] 80 | for spike in neuron: 81 | x.append(imageIdx[0]) 82 | y.append(imageIdx[1]) 83 | times.append(spike) 84 | return [x, y, times] 85 | 86 | allSpatioTemporal = [] 87 | for size, layers in S1_layers.items(): 88 | for layer in layers: 89 | out_data = layer.population.get_data().segments[0] 90 | allSpatioTemporal.append(extract_spatiotemporal_spiketrain(size, layer.population.label, 91 | out_data.spiketrains, 92 | target.shape)) 93 | pickle.dump(allSpatioTemporal, open("results/spatiotemporal_{}.p".format(filename), "wb")) 94 | 95 | max_spike_rate = 60. / 300. # mHz 96 | max_firing = max_spike_rate * (stimuli_duration + 300.) 97 | if args.reconstruct_s1_img: 98 | vis_img = np.zeros(target.shape) 99 | vis_parts = vis.visualization_parts(target.shape, 100 | layer_collection['S1'], 101 | feature_imgs_dict, 102 | args.delta_i, args.delta_j) 103 | for size, img_pairs in vis_parts.items(): 104 | for img, feature_label in img_pairs: 105 | vis_img += img 106 | cv2.imwrite('{}_S1_reconstruction.png'.format(filename), 107 | vis_img) 108 | 109 | # Plot the spike trains of both neuron layers 110 | for layer_name, layer_dict in layer_collection.items(): 111 | for size, layers in layer_dict.items(): 112 | spike_panels = [] 113 | for layer in layers: 114 | out_data = layer.population.get_data().segments[0] 115 | dump_filename = 'results/spiketrain_{}/{}_{}_scale.p'.format(\ 116 | filename, 117 | layer.population.label, 118 | size) 119 | try: 120 | plb.Path(dump_filename).parent.mkdir(parents=True) 121 | except OSError as exc: # Python >2.5 122 | pass 123 | pickle.dump(out_data.spiketrains,\ 124 | open(dump_filename, 'wb')) 125 | spike_panels.append(pynnplt.Panel(out_data.spiketrains,# xlabel='Time (ms)', 126 | xticks=True, yticks=True, 127 | xlabel='{}, {} scale layer'.format(\ 128 | layer.population.label, size))) 129 | pynnplt.Figure(*spike_panels).save('plots/{}_{}_{}_scale.png'.format(\ 130 | layer_name, 131 | filename, 132 | size)) 133 | sim.end() 134 | -------------------------------------------------------------------------------- /snn_object/snn_object/features/backslash_10x10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/features/backslash_10x10.png -------------------------------------------------------------------------------- /snn_object/snn_object/features/horiz_10x10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/features/horiz_10x10.png -------------------------------------------------------------------------------- /snn_object/snn_object/features/slash_10x10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/features/slash_10x10.png -------------------------------------------------------------------------------- /snn_object/snn_object/features/vert_10x10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/features/vert_10x10.png -------------------------------------------------------------------------------- /snn_object/snn_object/features_blurred/backslash_10x10_blurred.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/features_blurred/backslash_10x10_blurred.png -------------------------------------------------------------------------------- /snn_object/snn_object/features_blurred/horiz_10x10_blurred.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/features_blurred/horiz_10x10_blurred.png -------------------------------------------------------------------------------- /snn_object/snn_object/features_blurred/slash_10x10_blurred.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/features_blurred/slash_10x10_blurred.png -------------------------------------------------------------------------------- /snn_object/snn_object/features_blurred/vert_10x10_blurred.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/features_blurred/vert_10x10_blurred.png -------------------------------------------------------------------------------- /snn_object/snn_object/features_gabor/backslash.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/features_gabor/backslash.png -------------------------------------------------------------------------------- /snn_object/snn_object/features_gabor/horiz_backslash.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/features_gabor/horiz_backslash.png -------------------------------------------------------------------------------- /snn_object/snn_object/features_gabor/horiz_slash.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/features_gabor/horiz_slash.png -------------------------------------------------------------------------------- /snn_object/snn_object/features_gabor/slash.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/features_gabor/slash.png -------------------------------------------------------------------------------- /snn_object/snn_object/features_halved_blurred/backslash_5x5_blurred.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/features_halved_blurred/backslash_5x5_blurred.png -------------------------------------------------------------------------------- /snn_object/snn_object/features_halved_blurred/horiz_5x5_blurred.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/features_halved_blurred/horiz_5x5_blurred.png -------------------------------------------------------------------------------- /snn_object/snn_object/features_halved_blurred/slash_5x5_blurred.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/features_halved_blurred/slash_5x5_blurred.png -------------------------------------------------------------------------------- /snn_object/snn_object/features_halved_blurred/vert_5x5_blurred.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/features_halved_blurred/vert_5x5_blurred.png -------------------------------------------------------------------------------- /snn_object/snn_object/gabor-image-test.py: -------------------------------------------------------------------------------- 1 | #!/bin/ipython 2 | 3 | # ---LICENSE-BEGIN - DO NOT CHANGE OR MOVE THIS HEADER 4 | # This file is part of the Neurorobotics Platform software 5 | # Copyright (C) 2014,2015,2016,2017 Human Brain Project 6 | # 7 | # This program is free software; you can redistribute it and/or 8 | # modify it under the terms of the GNU General Public License 9 | # as published by the Free Software Foundation; either version 2 10 | # of the License, or (at your option) any later version. 11 | # 12 | # This program is distributed in the hope that it will be useful, 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 15 | # GNU General Public License for more details. 16 | # 17 | # You should have received a copy of the GNU General Public License 18 | # along with this program; if not, write to the Free Software 19 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 20 | # ---LICENSE-END 21 | 22 | import numpy as np 23 | import cv2 24 | import sys 25 | import pyNN.nest as sim 26 | import pathlib as plb 27 | import time 28 | import argparse as ap 29 | 30 | import common as cm 31 | import network as nw 32 | import visualization as vis 33 | import time 34 | 35 | parser = ap.ArgumentParser(description='SNN feature detector') 36 | parser.add_argument('--refrac-c1', type=float, default=.1, metavar='MS', 37 | help='The refractory period of neurons in the C1 layer in ms') 38 | parser.add_argument('--refrac-s1', type=float, default=.1, metavar='MS', 39 | help='The refractory period of neurons in the S1 layer in ms') 40 | parser.add_argument('--refrac-s2', type=float, default=.1, metavar='MS', 41 | help='The refractory period of neurons in the S2 layer in ms') 42 | parser.add_argument('--scales', default=[1.0, 0.71, 0.5, 0.35, 0.25], 43 | nargs='+', type=float, 44 | help='A list of image scales for which to create layers.\ 45 | Defaults to [1, 0.71, 0.5, 0.35, 0.25]') 46 | parser.add_argument('--plot-c1-spikes', action='store_true', 47 | help='Plot the spike trains of the C1 layers') 48 | parser.add_argument('--plot-s2-spikes', action='store_true', 49 | help='Plot the spike trains of the S2 layers') 50 | parser.add_argument('--sim-time', default=100, type=float, help='Simulation time') 51 | parser.add_argument('--target-name', type=str, 52 | help='The name of the already edge-filtered image to be\ 53 | recognized') 54 | args = parser.parse_args() 55 | 56 | sim.setup(threads=4) 57 | 58 | layer_collection = {} 59 | 60 | target_img = cv2.imread(args.target_name, cv2.CV_8UC1) 61 | print('Create S1 layers') 62 | t1 = time.clock() 63 | layer_collection['S1'] =\ 64 | nw.create_gabor_input_layers_for_scales(target_img, args.scales) 65 | nw.create_cross_layer_inhibition(layer_collection['S1']) 66 | print('S1 layer creation took {} s'.format(time.clock() - t1)) 67 | 68 | print('Create C1 layers') 69 | t1 = time.clock() 70 | layer_collection['C1'] = nw.create_C1_layers(layer_collection['S1'], 71 | args.refrac_c1) 72 | nw.create_local_inhibition(layer_collection['C1']) 73 | print('C1 creation took {} s'.format(time.clock() - t1)) 74 | 75 | print('Creating S2 layers') 76 | t1 = time.clock() 77 | layer_collection['S2'] = nw.create_S2_layers(layer_collection['C1'], args) 78 | print('S2 creation took {} s'.format(time.clock() - t1)) 79 | 80 | for layer_name in ['C1']: 81 | if layer_name in layer_collection: 82 | for layers in layer_collection[layer_name].values(): 83 | for layer in layers: 84 | layer.population.record('spikes') 85 | for layer in layer_collection['S2'].values(): 86 | layer.population.record(['spikes', 'v']) 87 | 88 | print('========= Start simulation =========') 89 | start_time = time.clock() 90 | sim.run(args.sim_time) 91 | end_time = time.clock() 92 | print('========= Stop simulation =========') 93 | print('Simulation took', end_time - start_time, 's') 94 | 95 | t1 = time.clock() 96 | if args.plot_c1_spikes: 97 | print('Plotting C1 spikes') 98 | vis.plot_C1_spikes(layer_collection['C1'], plb.Path(args.target_name).stem) 99 | print('Plotting spiketrains took {} s'.format(time.clock() - t1)) 100 | 101 | if args.plot_s2_spikes: 102 | print('Plotting S2 spikes') 103 | vis.plot_S2_spikes(layer_collection['S2'], plb.Path(args.target_name).stem) 104 | print('Plotting spiketrains took {} s'.format(time.clock() - t1)) 105 | 106 | sim.end() 107 | -------------------------------------------------------------------------------- /snn_object/snn_object/iaf_psc_exp.bin: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/iaf_psc_exp.bin -------------------------------------------------------------------------------- /snn_object/snn_object/image-test.py: -------------------------------------------------------------------------------- 1 | #!/bin/ipython 2 | 3 | # ---LICENSE-BEGIN - DO NOT CHANGE OR MOVE THIS HEADER 4 | # This file is part of the Neurorobotics Platform software 5 | # Copyright (C) 2014,2015,2016,2017 Human Brain Project 6 | # 7 | # This program is free software; you can redistribute it and/or 8 | # modify it under the terms of the GNU General Public License 9 | # as published by the Free Software Foundation; either version 2 10 | # of the License, or (at your option) any later version. 11 | # 12 | # This program is distributed in the hope that it will be useful, 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 15 | # GNU General Public License for more details. 16 | # 17 | # You should have received a copy of the GNU General Public License 18 | # along with this program; if not, write to the Free Software 19 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 20 | # ---LICENSE-END 21 | 22 | import numpy as np 23 | import cv2 24 | import sys 25 | import pyNN.nest as sim 26 | import pathlib as plb 27 | import time 28 | 29 | import common as cm 30 | import network as nw 31 | import visualization as vis 32 | import time 33 | 34 | args = cm.parse_args() 35 | 36 | t1 = time.clock() 37 | weights_dict, feature_imgs_dict = nw.train_weights(args.feature_dir) 38 | print('Training weights took {} s'.format(time.clock() - t1)) 39 | 40 | if args.plot_weights: 41 | vis.plot_weights(weights_dict) 42 | sys.exit(0) 43 | 44 | # The training part is done. Go on with the "actual" simulation 45 | sim.setup(threads=4) 46 | 47 | target_img = cm.read_and_prepare_img(args.target_name, args.filter) 48 | if args.filter != 'none': 49 | filename = 'edges/{}_{}_edges.png'.format(plb.Path(args.target_name).stem, 50 | args.filter) 51 | if not plb.Path(filename).exists(): 52 | cv2.imwrite(filename, target_img) 53 | 54 | layer_collection = {} # layer name -> dict of S1 layers of type 55 | # 'scale -> layer list' 56 | layer_collection['input'] = nw.create_input_layers_for_scales(target_img, 57 | args.scales) 58 | t1 = time.clock() 59 | layer_collection['S1'] = nw.create_S1_layers(layer_collection['input'], 60 | weights_dict, args) 61 | nw.create_cross_layer_inhibition(layer_collection['S1']) 62 | print('S1 creation took {} s'.format(time.clock() - t1)) 63 | 64 | if not args.no_c1: 65 | t1 = time.clock() 66 | print('Create C1 layers') 67 | layer_collection['C1'] = nw.create_C1_layers(layer_collection['S1'], 68 | args.refrac_c1) 69 | nw.create_local_inhibition(layer_collection['C1']) 70 | print('C1 creation took {} s'.format(time.clock() - t1)) 71 | 72 | for layer_name in ['S1', 'C1']: 73 | if layer_name in layer_collection: 74 | for layers in layer_collection[layer_name].values(): 75 | for layer in layers: 76 | layer.population.record('spikes') 77 | 78 | print('========= Start simulation =========') 79 | start_time = time.clock() 80 | sim.run(args.sim_time) 81 | end_time = time.clock() 82 | print('========= Stop simulation =========') 83 | print('Simulation took', end_time - start_time, 's') 84 | 85 | for layer_name in ['S1', 'C1']: 86 | if layer_name in layer_collection: 87 | for layers in layer_collection[layer_name].values(): 88 | for layer in layers: 89 | layer.update_spike_counts() 90 | 91 | t1 = time.clock() 92 | if args.reconstruct_s1_img: 93 | vis.reconstruct_S1_features(target_img, layer_collection, feature_imgs_dict, 94 | args) 95 | print('S1 visualization took {} s'.format(time.clock() - t1)) 96 | 97 | t1 = time.clock() 98 | if args.reconstruct_c1_img and not args.no_c1: 99 | vis.reconstruct_C1_features(target_img, layer_collection, feature_imgs_dict, 100 | args) 101 | print('C1 visualization took {} s'.format(time.clock() - t1)) 102 | 103 | 104 | t1 = time.clock() 105 | if args.plot_spikes: 106 | print('Plotting spikes') 107 | vis.plot_spikes(layer_collection, args) 108 | print('Plotting spiketrains took {} s'.format(time.clock() - t1)) 109 | 110 | sim.end() 111 | -------------------------------------------------------------------------------- /snn_object/snn_object/learn-features.py: -------------------------------------------------------------------------------- 1 | #!/bin/ipython 2 | 3 | # ---LICENSE-BEGIN - DO NOT CHANGE OR MOVE THIS HEADER 4 | # This file is part of the Neurorobotics Platform software 5 | # Copyright (C) 2014,2015,2016,2017 Human Brain Project 6 | # 7 | # This program is free software; you can redistribute it and/or 8 | # modify it under the terms of the GNU General Public License 9 | # as published by the Free Software Foundation; either version 2 10 | # of the License, or (at your option) any later version. 11 | # 12 | # This program is distributed in the hope that it will be useful, 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 15 | # GNU General Public License for more details. 16 | # 17 | # You should have received a copy of the GNU General Public License 18 | # along with this program; if not, write to the Free Software 19 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 20 | # ---LICENSE-END 21 | 22 | import numpy as np 23 | import cv2 24 | import sys 25 | import pyNN.nest as sim 26 | import pathlib as plb 27 | import time 28 | import pickle 29 | import argparse as ap 30 | import re 31 | 32 | import common as cm 33 | import network as nw 34 | import visualization as vis 35 | import time 36 | 37 | parser = ap.ArgumentParser('./learn-features.py --') 38 | parser.add_argument('--c1-dumpfile', type=str, required=True, 39 | help='The output file to contain the C1 spiketrains') 40 | parser.add_argument('--epoch-size', type=int, default=30, 41 | help='The lenght of an epoch') 42 | parser.add_argument('--feature-size', type=int, default=3, 43 | help='The size of the features to be learnt') 44 | parser.add_argument('--s2-prototype-cells', type=int, default=3, 45 | help='The number of S2 features to compute') 46 | parser.add_argument('--plot-c1-spikes', action='store_true', 47 | help='Plot the spike trains of the C1 layers') 48 | parser.add_argument('--plot-s2-spikes', action='store_true', 49 | help='Plot the spike trains of the S2 layers') 50 | #parser.add_argument('--refrac-s2', type=float, default=.1, metavar='.1', 51 | # help='The refractory period of neurons in the S2 layer in\ 52 | # ms') 53 | parser.add_argument('--threads', default=1, type=int) 54 | args = parser.parse_args() 55 | 56 | sim.setup(threads=args.threads,min_delay=0.1) 57 | 58 | layer_collection = {} 59 | 60 | # Read the gabor features for reconstruction 61 | feature_imgs_dict = {} # feature string -> image 62 | for filepath in plb.Path('features_gabor').iterdir(): 63 | feature_imgs_dict[filepath.stem] = cv2.imread(filepath.as_posix(), 64 | cv2.CV_8UC1) 65 | 66 | # Extracting meta-information about the simulation from the filename 67 | c1_dumpfile_name = plb.Path(args.c1_dumpfile).stem 68 | image_count = int(re.search('\d*imgs', c1_dumpfile_name).group()[:-4]) 69 | sim_time = float(re.search('\d*ms', c1_dumpfile_name).group()[:-2]) 70 | #sim_time=float(50) 71 | print(sim_time) 72 | dataset_label = '{}_fs{}_{}prots'.format(c1_dumpfile_name, args.feature_size, 73 | args.s2_prototype_cells) 74 | 75 | print('Create C1 layers') 76 | t1 = time.clock() 77 | dumpfile = open(args.c1_dumpfile, 'rb') 78 | ddict = pickle.load(dumpfile) 79 | layer_collection['C1'] = {} 80 | for size, layers_as_dicts in ddict.items(): 81 | layer_list = [] 82 | for layer_as_dict in layers_as_dicts: 83 | n, m = layer_as_dict['shape'] 84 | spiketrains = layer_as_dict['segment'].spiketrains 85 | dimensionless_sts = [[s for s in st] for st in spiketrains] 86 | new_layer = nw.Layer(sim.Population(n * m, 87 | sim.SpikeSourceArray(spike_times=dimensionless_sts), 88 | label=layer_as_dict['label']), (n, m)) 89 | layer_list.append(new_layer) 90 | layer_collection['C1'][size] = layer_list 91 | print('C1 creation took {} s'.format(time.clock() - t1)) 92 | 93 | print('Creating S2 layers') 94 | t1 = time.clock() 95 | layer_collection['S2'] = nw.create_S2_layers(layer_collection['C1'], 96 | args.feature_size, 97 | args.s2_prototype_cells, 98 | refrac_s2=6) 99 | print('S2 creation took {} s'.format(time.clock() - t1)) 100 | 101 | #for layers in layer_collection['C1'].values(): 102 | # for layer in layers: 103 | # layer.population.record('spikes') 104 | #for layer_list in layer_collection['S2'].values(): 105 | # for layer in layer_list: 106 | # layer.population.record(['spikes', 'v']) 107 | 108 | reconstructions_dir_dataset_path = plb.Path('S2_reconstructions/' + dataset_label) 109 | if not reconstructions_dir_dataset_path.exists(): 110 | reconstructions_dir_dataset_path.mkdir(parents=True) 111 | if args.plot_c1_spikes: 112 | c1_plots_dir_path = plb.Path('plots/C1/' + dataset_label) 113 | if not c1_plots_dir_path.exists(): 114 | c1_plots_dir_path.mkdir(parents=True) 115 | if args.plot_s2_spikes: 116 | s2_plots_dataset_dir = plb.Path('plots/S2/' + dataset_label) 117 | for i in range(args.s2_prototype_cells): 118 | s2_plots_dir_path = s2_plots_dataset_dir / str(i) 119 | if not s2_plots_dir_path.exists(): 120 | s2_plots_dir_path.mkdir(parents=True) 121 | 122 | epoch_weights = [] # type: List[Tuple[int, List[Dict[str, np.array]]]] 123 | 124 | # Let the simulation run to "fill" the layer pipeline with spikes 125 | sim.run(40) 126 | 127 | print('========= Start simulation =========') 128 | start_time = time.clock() 129 | for i in range(image_count): 130 | print('Simulating for image number', i) 131 | sim.run(sim_time) 132 | if args.plot_c1_spikes: 133 | vis.plot_C1_spikes(layer_collection['C1'], 134 | '{}_image_{}'.format(dataset_label, i), 135 | out_dir_name=c1_plots_dir_path.as_posix()) 136 | if args.plot_s2_spikes: 137 | vis.plot_S2_spikes(layer_collection['S2'], 138 | '{}_image_{}'.format(dataset_label, i), 139 | args.s2_prototype_cells, 140 | out_dir_name=s2_plots_dataset_dir.as_posix()) 141 | if (i + 1) % 10 == 0: 142 | current_weights = nw.get_current_weights(layer_collection['S2'], 143 | args.s2_prototype_cells) 144 | cv2.imwrite('{}/{}_{:0>4}_images.png'.format(\ 145 | reconstructions_dir_dataset_path.as_posix(), 146 | dataset_label, i + 1), 147 | vis.reconstruct_S2_features(current_weights, 148 | feature_imgs_dict, 149 | args.feature_size)) 150 | if (i + 1) % args.epoch_size == 0: 151 | current_weights = nw.get_current_weights(layer_collection['S2'], 152 | args.s2_prototype_cells) 153 | epoch_weights.append((i + 1, current_weights)) 154 | end_time = time.clock() 155 | print('========= Stop simulation =========') 156 | print('Simulation took', end_time - start_time, 's') 157 | 158 | 159 | # Reconstruct the last image 160 | cv2.imwrite('{}/{}_{:0>4}_images.png'.format(\ 161 | reconstructions_dir_dataset_path.as_posix(), 162 | dataset_label, i + 1), 163 | vis.reconstruct_S2_features(current_weights, 164 | feature_imgs_dict, 165 | args.feature_size)) 166 | # Also add the weights of the last iteration to the dumpfile 167 | if image_count % args.epoch_size != 0: 168 | epoch_weights.append((image_count, current_weights)) 169 | 170 | dumpfile_name = 'S2_weights/{}.bin'.format(dataset_label) 171 | out_dumpfile = open(dumpfile_name, 'wb') 172 | print('Dumping weights for the selected epochs to file', dumpfile_name) 173 | pickle.dump(epoch_weights, out_dumpfile, protocol=4) 174 | out_dumpfile.close() 175 | 176 | sim.end() 177 | -------------------------------------------------------------------------------- /snn_object/snn_object/learn-features_1.py: -------------------------------------------------------------------------------- 1 | #!/bin/ipython 2 | 3 | # ---LICENSE-BEGIN - DO NOT CHANGE OR MOVE THIS HEADER 4 | # This file is part of the Neurorobotics Platform software 5 | # Copyright (C) 2014,2015,2016,2017 Human Brain Project 6 | # 7 | # This program is free software; you can redistribute it and/or 8 | # modify it under the terms of the GNU General Public License 9 | # as published by the Free Software Foundation; either version 2 10 | # of the License, or (at your option) any later version. 11 | # 12 | # This program is distributed in the hope that it will be useful, 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 15 | # GNU General Public License for more details. 16 | # 17 | # You should have received a copy of the GNU General Public License 18 | # along with this program; if not, write to the Free Software 19 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 20 | # ---LICENSE-END 21 | 22 | import numpy as np 23 | import cv2 24 | import sys 25 | import pyNN.nest as sim 26 | import pathlib as plb 27 | import time 28 | import pickle 29 | import argparse as ap 30 | import re 31 | 32 | import common as cm 33 | import network as nw 34 | import visualization as vis 35 | import time 36 | 37 | parser = ap.ArgumentParser('./learn-features.py --') 38 | parser.add_argument('--c1-dumpfile', type=str, required=True, 39 | help='The output file to contain the C1 spiketrains') 40 | parser.add_argument('--epoch-size', type=int, default=30, 41 | help='The lenght of an epoch') 42 | parser.add_argument('--feature-size', type=int, default=3, 43 | help='The size of the features to be learnt') 44 | parser.add_argument('--s2-prototype-cells', type=int, default=3, 45 | help='The number of S2 features to compute') 46 | parser.add_argument('--plot-c1-spikes', action='store_true', 47 | help='Plot the spike trains of the C1 layers') 48 | parser.add_argument('--plot-s2-spikes', action='store_true', 49 | help='Plot the spike trains of the S2 layers') 50 | #parser.add_argument('--refrac-s2', type=float, default=.1, metavar='.1', 51 | # help='The refractory period of neurons in the S2 layer in\ 52 | # ms') 53 | parser.add_argument('--threads', default=1, type=int) 54 | args = parser.parse_args() 55 | 56 | sim.setup(threads=args.threads, min_delay=.1) 57 | 58 | layer_collection = {} 59 | 60 | # Read the gabor features for reconstruction 61 | feature_imgs_dict = {} # feature string -> image 62 | for filepath in plb.Path('features_gabor').iterdir(): 63 | feature_imgs_dict[filepath.stem] = cv2.imread(filepath.as_posix(), 64 | cv2.CV_8UC1) 65 | 66 | # Extracting meta-information about the simulation from the filename 67 | c1_dumpfile_name = plb.Path(args.c1_dumpfile).stem 68 | image_count = int(re.search('\d*imgs', c1_dumpfile_name).group()[:-4]) 69 | #sim_time = float(re.search('\d+\.\d+ms', c1_dumpfile_name).group()[:-2]) 70 | sim_time = float(50) 71 | dataset_label = '{}_fs{}_{}prots'.format(c1_dumpfile_name, args.feature_size, 72 | args.s2_prototype_cells) 73 | 74 | print('Create C1 layers') 75 | t1 = time.clock() 76 | dumpfile = open(args.c1_dumpfile, 'rb') 77 | ddict = pickle.load(dumpfile) 78 | layer_collection['C1'] = {} 79 | for size, layers_as_dicts in ddict.items(): 80 | layer_list = [] 81 | for layer_as_dict in layers_as_dicts: 82 | n, m = layer_as_dict['shape'] 83 | spiketrains = layer_as_dict['segment'].spiketrains 84 | dimensionless_sts = [[s for s in st] for st in spiketrains] 85 | new_layer = nw.Layer(sim.Population(n * m, 86 | sim.SpikeSourceArray(spike_times=dimensionless_sts), 87 | label=layer_as_dict['label']), (n, m)) 88 | layer_list.append(new_layer) 89 | layer_collection['C1'][size] = layer_list 90 | print('C1 creation took {} s'.format(time.clock() - t1)) 91 | 92 | print('Creating S2 layers') 93 | t1 = time.clock() 94 | layer_collection['S2'] = nw.create_S2_layers(layer_collection['C1'], 95 | args.feature_size, 96 | args.s2_prototype_cells, 97 | refrac_s2=6) 98 | print('S2 creation took {} s'.format(time.clock() - t1)) 99 | 100 | #for layers in layer_collection['C1'].values(): 101 | # for layer in layers: 102 | # layer.population.record('spikes') 103 | #for layer_list in layer_collection['S2'].values(): 104 | # for layer in layer_list: 105 | # layer.population.record(['spikes', 'v']) 106 | 107 | reconstructions_dir_dataset_path = plb.Path('S2_reconstructions/' + dataset_label) 108 | if not reconstructions_dir_dataset_path.exists(): 109 | reconstructions_dir_dataset_path.mkdir(parents=True) 110 | if args.plot_c1_spikes: 111 | c1_plots_dir_path = plb.Path('plots/C1/' + dataset_label) 112 | if not c1_plots_dir_path.exists(): 113 | c1_plots_dir_path.mkdir(parents=True) 114 | if args.plot_s2_spikes: 115 | s2_plots_dataset_dir = plb.Path('plots/S2/' + dataset_label) 116 | for i in range(args.s2_prototype_cells): 117 | s2_plots_dir_path = s2_plots_dataset_dir / str(i) 118 | if not s2_plots_dir_path.exists(): 119 | s2_plots_dir_path.mkdir(parents=True) 120 | 121 | epoch_weights = [] # type: List[Tuple[int, List[Dict[str, np.array]]]] 122 | 123 | # Let the simulation run to "fill" the layer pipeline with spikes 124 | sim.run(40) 125 | 126 | print('========= Start simulation =========') 127 | start_time = time.clock() 128 | for i in range(image_count): 129 | print('Simulating for image number', i) 130 | sim.run(sim_time) 131 | if args.plot_c1_spikes: 132 | vis.plot_C1_spikes(layer_collection['C1'], 133 | '{}_image_{}'.format(dataset_label, i), 134 | out_dir_name=c1_plots_dir_path.as_posix()) 135 | if args.plot_s2_spikes: 136 | vis.plot_S2_spikes(layer_collection['S2'], 137 | '{}_image_{}'.format(dataset_label, i), 138 | args.s2_prototype_cells, 139 | out_dir_name=s2_plots_dataset_dir.as_posix()) 140 | if (i + 1) % 10 == 0: 141 | current_weights = nw.get_current_weights(layer_collection['S2'], 142 | args.s2_prototype_cells) 143 | cv2.imwrite('{}/{}_{:0>4}_images.png'.format(\ 144 | reconstructions_dir_dataset_path.as_posix(), 145 | dataset_label, i + 1), 146 | vis.reconstruct_S2_features(current_weights, 147 | feature_imgs_dict, 148 | args.feature_size)) 149 | if (i + 1) % args.epoch_size == 0: 150 | current_weights = nw.get_current_weights(layer_collection['S2'], 151 | args.s2_prototype_cells) 152 | epoch_weights.append((i + 1, current_weights)) 153 | end_time = time.clock() 154 | print('========= Stop simulation =========') 155 | print('Simulation took', end_time - start_time, 's') 156 | # Reconstruct the last image 157 | cv2.imwrite('{}/{}_{:0>4}_images.png'.format(\ 158 | reconstructions_dir_dataset_path.as_posix(), 159 | dataset_label, i + 1), 160 | vis.reconstruct_S2_features(current_weights, 161 | feature_imgs_dict, 162 | args.feature_size)) 163 | # Also add the weights of the last iteration to the dumpfile 164 | if image_count % args.epoch_size != 0: 165 | epoch_weights.append((image_count, current_weights)) 166 | 167 | dumpfile_name = 'S2_weights/{}.bin'.format(dataset_label) 168 | out_dumpfile = open(dumpfile_name, 'wb') 169 | print('Dumping weights for the selected epochs to file', dumpfile_name) 170 | pickle.dump(epoch_weights, out_dumpfile, protocol=4) 171 | out_dumpfile.close() 172 | 173 | sim.end() 174 | -------------------------------------------------------------------------------- /snn_object/snn_object/machinefile: -------------------------------------------------------------------------------- 1 | 192.168.150.101 2 | -------------------------------------------------------------------------------- /snn_object/snn_object/result/dump-c1-spikes.py: -------------------------------------------------------------------------------- 1 | #!/bin/ipython 2 | 3 | # ---LICENSE-BEGIN - DO NOT CHANGE OR MOVE THIS HEADER 4 | # This file is part of the Neurorobotics Platform software 5 | # Copyright (C) 2014,2015,2016,2017 Human Brain Project 6 | # 7 | # This program is free software; you can redistribute it and/or 8 | # modify it under the terms of the GNU General Public License 9 | # as published by the Free Software Foundation; either version 2 10 | # of the License, or (at your option) any later version. 11 | # 12 | # This program is distributed in the hope that it will be useful, 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 15 | # GNU General Public License for more details. 16 | # 17 | # You should have received a copy of the GNU General Public License 18 | # along with this program; if not, write to the Free Software 19 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 20 | # ---LICENSE-END 21 | 22 | import numpy as np 23 | import cv2 24 | import sys 25 | import pyNN.nest as sim 26 | import pathlib as plb 27 | import time 28 | import pickle 29 | import argparse as ap 30 | 31 | import network as nw 32 | import visualization as vis 33 | from pyNN.utility.plotting import Figure, Panel 34 | parser = ap.ArgumentParser('./dump-c1-spikes.py --') 35 | parser.add_argument('--dataset-label', type=str, required=True, 36 | help='The name of the dataset which was used for\ 37 | training') 38 | parser.add_argument('--training-dir', type=str, required=True, 39 | help='The directory with the training images') 40 | parser.add_argument('--refrac-c1', type=float, default=0.01, metavar='0.01', 41 | help='The refractory period of neurons in the C1 layer in\ 42 | ms') 43 | parser.add_argument('--sim-time', default=50, type=float, help='Simulation time', 44 | metavar='50') 45 | parser.add_argument('--scales', default=[1.0, 0.71, 0.5, 0.35], 46 | nargs='+', type=float, 47 | help='A list of image scales for which to create\ 48 | layers. Defaults to [1, 0.71, 0.5, 0.35]') 49 | parser.add_argument('--threads', default=1, type=int) 50 | args = parser.parse_args() 51 | 52 | training_path = plb.Path(args.training_dir) 53 | 54 | imgs = [(filename.stem, cv2.imread(filename.as_posix(), cv2.CV_8UC1))\ 55 | for filename in sorted(training_path.glob('*'+'.jpg'))] 56 | 57 | sim.setup(threads=args.threads) 58 | 59 | layer_collection = {} 60 | 61 | print('Create S1 layers') 62 | t1 = time.clock() 63 | layer_collection['S1'] =\ 64 | nw.create_gabor_input_layers_for_scales(imgs[0][1], args.scales) 65 | print('S1 layer creation gabor_input_layers took {} s'.format(time.clock() - t1)) 66 | t2 = time.clock() 67 | nw.create_cross_layer_inhibition(layer_collection['S1']) 68 | print('S1 layer creation layer_inhibition took {} s'.format(time.clock() - t2)) 69 | 70 | 71 | print('Create C1 layers') 72 | t1 = time.clock() 73 | layer_collection['C1'] = nw.create_C1_layers(layer_collection['S1'], 74 | args.refrac_c1) 75 | 76 | print("Projection.get('weight')::",layer_collection['C1'].get('weight')) 77 | print('C1 creation took {} s'.format(time.clock() - t1)) 78 | t2 = time.clock() 79 | nw.create_local_inhibition(layer_collection['C1']) 80 | print('C1 creation create_local_inhibition took {} s'.format(time.clock() - t2)) 81 | 82 | for layer_name in ['C1']: 83 | if layer_name in layer_collection: 84 | for layers in layer_collection[layer_name].values(): 85 | for layer in layers: 86 | layer.population.record('spikes') 87 | 88 | # Simulate for a certain time to allow the whole layer pipeline to "get filled" 89 | #sim.run(2) 90 | dataset_label = '{}_{}imgs_{}ms_{}px_scales'.format(args.dataset_label, 91 | len(imgs), args.sim_time, 92 | imgs[0][1].shape[0]) 93 | 94 | c1_plots_dir_path = plb.Path('plots/C1/' + dataset_label) 95 | if not c1_plots_dir_path.exists(): 96 | c1_plots_dir_path.mkdir(parents=True) 97 | 98 | 99 | print('========= Start simulation =========') 100 | start_time = time.clock() 101 | count = 0 102 | for filename, target_img in imgs: 103 | 104 | print('Simulating for', filename, 'number', count) 105 | count += 1 106 | t2=time.clock() 107 | nw.set_i_offsets_for_all_scales_to(layer_collection['S1'], target_img) 108 | print('set_i_offsets_for_all_scales_to', time.clock() - t2, 'seconds') 109 | t1 = time.clock() 110 | sim.run(args.sim_time) 111 | 112 | #vis.plot_C1_spikes(layer_collection['C1'], 113 | # '{}_image_{}'.format(dataset_label, count), 114 | # out_dir_name=c1_plots_dir_path.as_posix()) 115 | #sim.run(1) 116 | print('Took', time.clock() - t1, 'seconds') 117 | end_time = time.clock() 118 | print('========= Stop simulation =========') 119 | print('Simulation took', end_time - start_time, 's') 120 | 121 | ddict = {} 122 | 123 | i=0 124 | for size, layers in layer_collection['C1'].items(): 125 | ddict[size] = [{'segment': layer.population.get_data().segments[0], 126 | 'shape': layer.shape, 127 | 'label': layer.population.label } for layer in layers] 128 | for layer in layers: 129 | f=open('result/result_'+str(size)+str(layer.population.label)+'.txt','w+') 130 | f.write(str(layer.population.get_data().segments[0].spiketrains)) 131 | f.close() 132 | """ 133 | for layer in layers: 134 | data = layer.population.get_data().segments[0] 135 | print(data) 136 | #vm = data.filter(name="v")[0] 137 | #spike = data.filter(name="spike")[0] 138 | i=i+1 139 | Figure( 140 | Panel(data.spiketrains, xlabel="Time (ms)", xticks=True) 141 | ).save("simulation_results"+str(i)+".png") 142 | dataset_label += '_{}'.format(size) 143 | """ 144 | 145 | #dumpname = 'C1_spikes/{}.txt'.format(dataset_label) 146 | #print('Dumping spikes for all scales and layers to file', dumpname) 147 | #dumpfile = open(dumpname, 'wb') 148 | #pickle.dump(ddict, dumpfile, protocol=4) 149 | #dumpfile.close() 150 | 151 | sim.end() 152 | -------------------------------------------------------------------------------- /snn_object/snn_object/result/result_0.35backslash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result/result_0.35horiz_backslash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result/result_0.35horiz_slash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result/result_0.35slash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result/result_0.5backslash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result/result_0.5horiz_backslash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result/result_0.5horiz_slash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result/result_0.5slash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result/result_0.71backslash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result/result_0.71horiz_backslash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result/result_0.71horiz_slash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result/result_0.71slash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result/result_1.0backslash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result/result_1.0horiz_slash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result_right/result_0.35backslash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result_right/result_0.35horiz_backslash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result_right/result_0.35horiz_slash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result_right/result_0.35slash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result_right/result_0.5backslash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result_right/result_0.5horiz_backslash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result_right/result_0.5horiz_slash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result_right/result_0.5slash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result_right/result_0.71backslash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result_right/result_0.71horiz_backslash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result_right/result_0.71horiz_slash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/result_right/result_0.71slash.txt: -------------------------------------------------------------------------------- 1 | [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] -------------------------------------------------------------------------------- /snn_object/snn_object/samples/combined_airplanes_mo_fa.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/samples/combined_airplanes_mo_fa.png -------------------------------------------------------------------------------- /snn_object/snn_object/samples/fa_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/samples/fa_1.png -------------------------------------------------------------------------------- /snn_object/snn_object/samples/fa_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/samples/fa_2.png -------------------------------------------------------------------------------- /snn_object/snn_object/samples/fa_3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/samples/fa_3.png -------------------------------------------------------------------------------- /snn_object/snn_object/samples/fa_4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/samples/fa_4.png -------------------------------------------------------------------------------- /snn_object/snn_object/samples/mo_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/samples/mo_1.png -------------------------------------------------------------------------------- /snn_object/snn_object/samples/mo_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/samples/mo_2.png -------------------------------------------------------------------------------- /snn_object/snn_object/samples/mo_3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/samples/mo_3.png -------------------------------------------------------------------------------- /snn_object/snn_object/samples/mo_4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/samples/mo_4.png -------------------------------------------------------------------------------- /snn_object/snn_object/stream.py: -------------------------------------------------------------------------------- 1 | 2 | # ---LICENSE-BEGIN - DO NOT CHANGE OR MOVE THIS HEADER 3 | # This file is part of the Neurorobotics Platform software 4 | # Copyright (C) 2014,2015,2016,2017 Human Brain Project 5 | # 6 | # This program is free software; you can redistribute it and/or 7 | # modify it under the terms of the GNU General Public License 8 | # as published by the Free Software Foundation; either version 2 9 | # of the License, or (at your option) any later version. 10 | # 11 | # This program is distributed in the hope that it will be useful, 12 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14 | # GNU General Public License for more details. 15 | # 16 | # You should have received a copy of the GNU General Public License 17 | # along with this program; if not, write to the Free Software 18 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 19 | # ---LICENSE-END 20 | 21 | import numpy as np 22 | import rosbag 23 | import pathlib as plb 24 | 25 | class Stream: 26 | def __init__(self, events, shape, duration): 27 | self.events = events 28 | self.shape = shape 29 | self.duration = duration 30 | 31 | def resize_stream(stream, size): 32 | # no interpolation so far 33 | resized_shape = np.ceil(np.multiply(stream.shape, size)).astype(int) 34 | resized_events = np.copy(stream.events) 35 | for event in resized_events: 36 | event.x = int(np.floor(event.x * size)) 37 | event.y = int(np.floor(event.y * size)) 38 | return Stream(resized_events, resized_shape, stream.duration) 39 | 40 | def read_stream(filename): 41 | bag = rosbag.Bag(filename) 42 | allEvents = [] 43 | initial_time = None 44 | last_time = 0 45 | for topic, msg, t in bag.read_messages(topics=['/dvs/events']): 46 | if not initial_time and msg.events: 47 | # we want the first event to happen at 1ms 48 | initial_time = int(msg.events[0].ts.to_sec() * 1000) - 1 49 | for event in msg.events: 50 | event.ts = int(event.ts.to_sec() * 1000) - initial_time 51 | allEvents = np.append(allEvents, msg.events) 52 | last_time = t.to_sec() * 1000 53 | # NOTE: I forgot to specify in the Layer class that the shape is 54 | # specified in matrix notation like (rows, cols). So here maybe 55 | # (msg.height, msg.width) could be more appropriate? 56 | shape = [msg.width, msg.height] 57 | bag.close() 58 | 59 | return Stream(allEvents, shape, last_time - initial_time) 60 | -------------------------------------------------------------------------------- /snn_object/snn_object/train.txt: -------------------------------------------------------------------------------- 1 | 1 2 | 1 3 | 1 4 | 1 5 | 1 6 | 1 7 | 1 8 | 1 9 | 1 10 | 1 11 | 1 12 | 1 13 | 1 14 | 1 15 | 1 16 | 1 17 | 1 18 | 0 19 | 1 20 | 1 21 | 1 22 | 1 23 | 1 24 | 1 25 | 1 26 | 1 27 | 1 28 | 1 29 | 1 30 | 1 31 | -------------------------------------------------------------------------------- /snn_object/snn_object/validation.txt: -------------------------------------------------------------------------------- 1 | 1 2 | 1 3 | 1 4 | 1 5 | 1 6 | 1 7 | 1 8 | 1 9 | 1 10 | 1 11 | 1 12 | 1 13 | 1 14 | 1 15 | 1 16 | 1 17 | 1 18 | 1 19 | 1 20 | 1 21 | 1 22 | 1 23 | 1 24 | 1 25 | 1 26 | 1 27 | 1 28 | 1 29 | 1 30 | 1 31 | 32 | -------------------------------------------------------------------------------- /snn_object/snn_object/video-test.py: -------------------------------------------------------------------------------- 1 | #!/bin/ipython 2 | 3 | # ---LICENSE-BEGIN - DO NOT CHANGE OR MOVE THIS HEADER 4 | # This file is part of the Neurorobotics Platform software 5 | # Copyright (C) 2014,2015,2016,2017 Human Brain Project 6 | # 7 | # This program is free software; you can redistribute it and/or 8 | # modify it under the terms of the GNU General Public License 9 | # as published by the Free Software Foundation; either version 2 10 | # of the License, or (at your option) any later version. 11 | # 12 | # This program is distributed in the hope that it will be useful, 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 15 | # GNU General Public License for more details. 16 | # 17 | # You should have received a copy of the GNU General Public License 18 | # along with this program; if not, write to the Free Software 19 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 20 | # ---LICENSE-END 21 | 22 | import numpy as np 23 | import cv2 24 | import pyNN.nest as sim 25 | import pathlib as plb 26 | import time 27 | 28 | import common as cm 29 | import network as nw 30 | import visualization as vis 31 | 32 | args = cm.parse_args() 33 | 34 | # Train weights 35 | weights_dict, feature_imgs_dict = nw.train_weights(args.feature_dir) 36 | 37 | # Open the video capture and writer objects 38 | cap = cv2.VideoCapture(args.target_name) 39 | cap_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)) 40 | cap_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)) 41 | writer = cv2.VideoWriter('video_S1_reconstructions/{}.avi'.format(\ 42 | plb.Path(args.target_name).stem), 43 | int(cap.get(cv2.CAP_PROP_FOURCC)), 44 | cap.get(cv2.CAP_PROP_FPS), (cap_width, cap_height), 45 | isColor=False) 46 | 47 | sim.setup(threads=4, spike_precision='on_grid') 48 | 49 | # Set up the network 50 | layer_collection = {} # layer name -> dict of S1 layers of type 51 | # 'scale -> layer list' 52 | 53 | layer_collection['input'] = nw.create_input_layers_for_scales(\ 54 | np.zeros((cap_height, cap_width)), args.scales) 55 | layer_collection['S1'] = nw.create_S1_layers(layer_collection['input'], 56 | weights_dict, args) 57 | nw.create_cross_layer_inhibition(layer_collection['S1']) 58 | # We build only the S1 layer for the moment, to speed up the simulation time 59 | 60 | for layers in layer_collection['S1'].values(): 61 | for layer in layers: 62 | layer.population.record('spikes') 63 | 64 | # The actual frame-by-frame simulation and input neuron updating 65 | t1 = time.clock() 66 | #for i in range(int(cap.get(cv2.CAP_PROP_FRAME_COUNT))): 67 | for i in range(args.frames): 68 | t2 = time.clock() 69 | img = cap.read()[1] 70 | img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) 71 | 72 | # It's assumed that the image is already filtered, so no filtering is 73 | # required 74 | nw.change_rates_for_scales(layer_collection['input'], img) 75 | sim.run(50) 76 | 77 | # Refresh the current spike counts 78 | for layers in layer_collection['S1'].values(): 79 | for layer in layers: 80 | layer.update_spike_counts() 81 | 82 | reconstructed_img = vis.create_S1_feature_image(img, layer_collection, 83 | feature_imgs_dict, args)[1] 84 | reconstructed_img = cv2.convertScaleAbs(reconstructed_img) 85 | writer.write(reconstructed_img) 86 | print('Frame', i, 'took', time.clock() - t2, 's to finish') 87 | print('Processing', args.frames, 'frames took', time.clock() - t1, 's') 88 | 89 | cap.release() 90 | writer.release() 91 | 92 | sim.end() 93 | -------------------------------------------------------------------------------- /snn_object/snn_object/video/airplanes-motorbikes-pianos.avi: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/video/airplanes-motorbikes-pianos.avi -------------------------------------------------------------------------------- /snn_object/snn_object/video/motorbikes-faces.avi: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clancylea/SNN-simulator-NEST_14.0-xilinx_FPGA_cluster/dea66f1dad3874140114b04ad97af5184e533863/snn_object/snn_object/video/motorbikes-faces.avi -------------------------------------------------------------------------------- /snn_object/snn_object/visualizer.py: -------------------------------------------------------------------------------- 1 | 2 | # ---LICENSE-BEGIN - DO NOT CHANGE OR MOVE THIS HEADER 3 | # This file is part of the Neurorobotics Platform software 4 | # Copyright (C) 2014,2015,2016,2017 Human Brain Project 5 | # 6 | # This program is free software; you can redistribute it and/or 7 | # modify it under the terms of the GNU General Public License 8 | # as published by the Free Software Foundation; either version 2 9 | # of the License, or (at your option) any later version. 10 | # 11 | # This program is distributed in the hope that it will be useful, 12 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14 | # GNU General Public License for more details. 15 | # 16 | # You should have received a copy of the GNU General Public License 17 | # along with this program; if not, write to the Free Software 18 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 19 | # ---LICENSE-END 20 | 21 | from mpl_toolkits.mplot3d.axes3d import Axes3D 22 | import matplotlib.pyplot as plt 23 | import numpy as np 24 | import pylab 25 | import pickle 26 | from cycler import cycler 27 | from os import listdir 28 | from os.path import isfile, join 29 | 30 | pylab.ion() 31 | color_list = ['r', 'g', 'b', 'y', 'c', 'm', 'y', 'k'] 32 | marker_list = [ '+', 'x', 'o', 'v', '^', '<', '>' ] 33 | 34 | def plot_3d_spatiotemporal(): 35 | allSpikesPerLayer = pickle.load(open('results/spatiotemporal_dvs-page2-30s_2016-06-24-18-15-21.p', 'r')) 36 | 37 | fig = plt.figure(figsize=(40,5)) 38 | # `ax` is a 3D-aware axis instance, because of the projection='3d' keyword argument to add_subplot 39 | ax = fig.add_subplot(1, 2, 1, projection='3d') 40 | ax.set_xlabel('X') 41 | ax.set_zlabel('Y') 42 | ax.set_ylabel('times') 43 | 44 | for i, layer in enumerate(allSpikesPerLayer): 45 | ax.scatter(layer[0], layer[2], layer[1], c=color_list[i]) 46 | 47 | fig.show() 48 | 49 | def plot_2d_spiketrains(pathToPickles): 50 | fig = plt.figure(figsize=(40,5)) 51 | ax = fig.add_subplot(1, 1, 1) 52 | 53 | pickleFilenames = listdir(pathToPickles) 54 | allPickles = [join(pathToPickles, f) for f in pickleFilenames if isfile(join(pathToPickles, f))] 55 | allSpiketrains = [ pickle.load(open(pickleFile, 'r')) for pickleFile in allPickles ] 56 | for featureMapIdx, spiketrain in enumerate(allSpiketrains): 57 | populationName = pickleFilenames[featureMapIdx] 58 | x = [] 59 | y = [] 60 | for i, neuron in enumerate(spiketrain): 61 | for spike in neuron: 62 | x.append(spike) 63 | y.append(i) 64 | if populationName.startswith("corner"): 65 | markersize = 10 66 | alpha = 1 67 | else: 68 | markersize = 1 69 | alpha = 0.1 70 | 71 | ax.scatter(x, y, marker=marker_list[featureMapIdx], c=color_list[featureMapIdx], label=populationName, linewidths=markersize, alpha = alpha) 72 | ax.legend() 73 | 74 | # plot_2d_spiketrains('results/spiketrain_dvs-page4_2016-06-23-14-58-02') 75 | plot_2d_spiketrains('results/spiketrain_dvs-page2-30s_2016-06-24-18-15-21') 76 | 77 | raw_input("Press Enter to continue...") 78 | --------------------------------------------------------------------------------