└── README.md /README.md: -------------------------------------------------------------------------------- 1 | ## Tools to Design or Visualize Architecture of Neural Network 2 | 3 | 1. [**Net2Vis**](https://viscom.net2vis.uni-ulm.de/OG1Br2BAkYSwwrV6CADl4X5EfErFjUzvuUwXWDdLbdsIXNhb9L): Net2Vis automatically generates abstract visualizations for convolutional neural networks from Keras code. 4 | 5 | ![](https://storage.googleapis.com/groundai-web-prod/media/users/user_299833/project_401171/images/application.png) 6 | 7 | 2. [**visualkeras**](https://github.com/paulgavrikov/visualkeras/) : Visualkeras is a Python package to help visualize Keras (either standalone or included in tensorflow) neural network architectures. It allows easy styling to fit most needs. As of now it supports layered style architecture generation which is great for CNNs (Convolutional Neural Networks) and a grap style architecture. 8 | 9 | ```python 10 | import visualkeras 11 | 12 | model = ... 13 | 14 | visualkeras.layered_view(model).show() # display using your system viewer 15 | visualkeras.layered_view(model, to_file='output.png') # write to disk 16 | visualkeras.layered_view(model, to_file='output.png').show() # write and show 17 | 18 | visualkeras.layered_view(model) 19 | ``` 20 | 21 | ![](https://github.com/paulgavrikov/visualkeras/raw/master/figures/vgg16.png) 22 | 23 | 3. [**draw_convnet**](https://github.com/gwding/draw_convnet) : Python script for illustrating Convolutional Neural Network (ConvNet) 24 | 25 | ![img](https://raw.githubusercontent.com/gwding/draw_convnet/master/convnet_fig.png) 26 | 27 | 4. [**NNSVG**](http://alexlenail.me/NN-SVG/LeNet.html) 28 | 29 | ![AlexNet style](https://i.stack.imgur.com/Q0xOe.png) 30 | 31 | ![enter image description here](https://i.stack.imgur.com/K9lmg.png) 32 | 33 | ![enter image description here](https://i.stack.imgur.com/DlJ8J.png) 34 | 35 | 5. **[PlotNeuralNet](https://github.com/HarisIqbal88/PlotNeuralNet)** : Latex code for drawing neural networks for reports and presentation. Have a look into examples to see how they are made. Additionally, lets consolidate any improvements that you make and fix any bugs to help more people with this code. 36 | 37 | ![img](https://user-images.githubusercontent.com/17570785/50308846-c2231880-049c-11e9-8763-3daa1024de78.png) 38 | 39 | ![img](https://user-images.githubusercontent.com/17570785/50308873-e2eb6e00-049c-11e9-9587-9da6bdec011b.png) 40 | 41 | 6. **[Tensorboard](https://www.tensorflow.org/tensorboard/graphs)** - TensorBoard’s **Graphs dashboard** is a powerful tool for examining your TensorFlow model. 42 | 43 | ![enter image description here](https://i.stack.imgur.com/zJHpV.png) 44 | 45 | 7. **[Caffe](https://github.com/BVLC/caffe/blob/master/python/caffe/draw.py)** - In Caffe you can use [caffe/draw.py](https://github.com/BVLC/caffe/blob/master/python/caffe/draw.py) to draw the NetParameter protobuffer: 46 | 47 | ![enter image description here](https://i.stack.imgur.com/5Z1Cb.png) 48 | 49 | 8. **[Matlab](http://www.mathworks.com/help/nnet/ref/view.html)** 50 | 51 | ![enter image description here](https://i.stack.imgur.com/rPpfa.png) 52 | 53 | 9. [**Keras.js**](https://transcranial.github.io/keras-js/#/inception-v3) 54 | 55 | ![enter image description here](https://i.stack.imgur.com/vEfTv.png) 56 | 57 | 9. **[keras-sequential-ascii](https://github.com/stared/keras-sequential-ascii/)** - A library for [Keras](https://keras.io/) for investigating architectures and parameters of sequential models. 58 | 59 | **VGG 16 Architecture** 60 | 61 | ``` 62 | OPERATION DATA DIMENSIONS WEIGHTS(N) WEIGHTS(%) 63 | 64 | Input ##### 3 224 224 65 | InputLayer | ------------------- 0 0.0% 66 | ##### 3 224 224 67 | Convolution2D \|/ ------------------- 1792 0.0% 68 | relu ##### 64 224 224 69 | Convolution2D \|/ ------------------- 36928 0.0% 70 | relu ##### 64 224 224 71 | MaxPooling2D Y max ------------------- 0 0.0% 72 | ##### 64 112 112 73 | Convolution2D \|/ ------------------- 73856 0.1% 74 | relu ##### 128 112 112 75 | Convolution2D \|/ ------------------- 147584 0.1% 76 | relu ##### 128 112 112 77 | MaxPooling2D Y max ------------------- 0 0.0% 78 | ##### 128 56 56 79 | Convolution2D \|/ ------------------- 295168 0.2% 80 | relu ##### 256 56 56 81 | Convolution2D \|/ ------------------- 590080 0.4% 82 | relu ##### 256 56 56 83 | Convolution2D \|/ ------------------- 590080 0.4% 84 | relu ##### 256 56 56 85 | MaxPooling2D Y max ------------------- 0 0.0% 86 | ##### 256 28 28 87 | Convolution2D \|/ ------------------- 1180160 0.9% 88 | relu ##### 512 28 28 89 | Convolution2D \|/ ------------------- 2359808 1.7% 90 | relu ##### 512 28 28 91 | Convolution2D \|/ ------------------- 2359808 1.7% 92 | relu ##### 512 28 28 93 | MaxPooling2D Y max ------------------- 0 0.0% 94 | ##### 512 14 14 95 | Convolution2D \|/ ------------------- 2359808 1.7% 96 | relu ##### 512 14 14 97 | Convolution2D \|/ ------------------- 2359808 1.7% 98 | relu ##### 512 14 14 99 | Convolution2D \|/ ------------------- 2359808 1.7% 100 | relu ##### 512 14 14 101 | MaxPooling2D Y max ------------------- 0 0.0% 102 | ##### 512 7 7 103 | Flatten ||||| ------------------- 0 0.0% 104 | ##### 25088 105 | Dense XXXXX ------------------- 102764544 74.3% 106 | relu ##### 4096 107 | Dense XXXXX ------------------- 16781312 12.1% 108 | relu ##### 4096 109 | Dense XXXXX ------------------- 4097000 3.0% 110 | softmax ##### 1000 111 | ``` 112 | 113 | 10. **[ Netron ](https://github.com/lutzroeder/Netron)** 114 | 115 | ![screenshot.png](https://github.com/lutzroeder/netron/raw/main/.github/screenshot.png) 116 | 117 | 11. **[DotNet](https://github.com/martisak/dotnets)** 118 | 119 | ![Simple net](https://github.com/martisak/dotnets/raw/master/test.png) 120 | 121 | 12. [**Graphviz**](http://www.graphviz.org/) : **[Tutorial](https://tgmstat.wordpress.com/2013/06/12/draw-neural-network-diagrams-graphviz/)** 122 | 123 | ![img](https://tgmstat.files.wordpress.com/2013/05/multiclass_neural_network_example.png) 124 | 125 | 13. **[Keras Visualization](https://keras.io/visualization/)** - The [keras.utils.vis_utils module](https://keras.io/visualization/) provides utility functions to plot a Keras model (using graphviz) 126 | 127 | ![enter image description here](https://i.stack.imgur.com/o17GY.png) 128 | 129 | 14. **[Conx](https://conx.readthedocs.io/en/latest/index.html)** - The Python package `conx` can visualize networks with activations with the function `net.picture()` to produce SVG, PNG, or PIL Images like this: 130 | 131 | ![enter image description here](https://i.stack.imgur.com/nhHjO.png) 132 | 133 | 15. **[ENNUI](https://math.mit.edu/ennui/)** - Working on a drag-and-drop neural network visualizer (and more). Here's an example of a visualization for a LeNet-like architecture. 134 | 135 | ![A visualization of a LeNet-like architecture](https://i.stack.imgur.com/pRLeG.png) 136 | 137 | 16. **NNet - R Package** - **[Tutorial](https://beckmw.wordpress.com/2013/03/04/visualizing-neural-networks-from-the-nnet-package/)** 138 | 139 | ``` 140 | data(infert, package="datasets") 141 | plot(neuralnet(case~parity+induced+spontaneous, infert)) 142 | ``` 143 | 144 | [![neuralnet](https://i.stack.imgur.com/yyftd.png)](https:// 145 | 146 | 17. **[GraphCore](https://www.graphcore.ai/posts/what-does-machine-learning-look-like)** - These approaches are more oriented towards visualizing neural network operation, however, NN architecture is also somewhat visible on the resulting diagrams. 147 | 148 | **AlexNet** 149 | 150 | ![alexnet_label logo.jpg](https://www.graphcore.ai/hubfs/images/alexnet_label%20logo.jpg) 151 | 152 | **ResNet50**![resnet50_label_logo.jpg](https://www.graphcore.ai/hubfs/images/resnet50_label_logo.jpg) 153 | 154 | 18. [**Neataptic**](https://wagenaartje.github.io/neataptic/ ) 155 | 156 | Neataptic offers flexible neural networks; neurons and synapses can be removed with a single line of code. No fixed architecture is required for neural networks to function at all. This flexibility allows networks to be shaped for your dataset through neuro-evolution, which is done using multiple threads. 157 | 158 | ![img](https://camo.githubusercontent.com/4326c3f603b828b61fd63d927acca2cfc152773f/68747470733a2f2f692e6779617a6f2e636f6d2f66353636643233363461663433646433613738633839323665643230346135312e706e67) 159 | 160 | 19. **[TensorSpace](https://tensorspace.org/)** : TensorSpace is a neural network 3D visualization framework built by TensorFlow.js, Three.js and Tween.js. TensorSpace provides Layer APIs to build deep learning layers, load pre-trained models, and generate a 3D visualization in the browser. By applying TensorSpace API, it is more intuitive to visualize and understand any pre-trained models built by TensorFlow, Keras, TensorFlow.js, etc. 161 | 162 | **[Tutorial](https://www.freecodecamp.org/news/tensorspace-js-a-way-to-3d-visualize-neural-networks-in-browsers-2c0afd7648a8/)** 163 | 164 | ![enter image description here](https://i.stack.imgur.com/ekF5v.png) 165 | 166 | ​ 167 | 168 | 20. **[Netscope CNN Analyzer](http://dgschwend.github.io/netscope/quickstart.html)** 169 | 170 | ![enter image description here](https://i.stack.imgur.com/VVDsg.png) 171 | 172 | 21. **[Monial](https://github.com/mlajtos/moniel)** 173 | 174 | Interactive Notation for Computational Graphs https://mlajtos.github.io/moniel/ 175 | 176 | ![img](https://miro.medium.com/max/819/1*u6uIQF4xTVe-ylJnAPoIDg.png) 177 | 178 | 22. [**Texample**](http://www.texample.net/tikz/examples/neural-network/) 179 | 180 | ![Neural Network](https://texample.net/media/tikz/examples/PNG/neural-network.png) 181 | 182 | 183 | ``` 184 | \documentclass{article} 185 | 186 | \usepackage{tikz} 187 | \begin{document} 188 | \pagestyle{empty} 189 | 190 | \def\layersep{2.5cm} 191 | 192 | \begin{tikzpicture}[shorten >=1pt,->,draw=black!50, node distance=\layersep] 193 | \tikzstyle{every pin edge}=[<-,shorten <=1pt] 194 | \tikzstyle{neuron}=[circle,fill=black!25,minimum size=17pt,inner sep=0pt] 195 | \tikzstyle{input neuron}=[neuron, fill=green!50]; 196 | \tikzstyle{output neuron}=[neuron, fill=red!50]; 197 | \tikzstyle{hidden neuron}=[neuron, fill=blue!50]; 198 | \tikzstyle{annot} = [text width=4em, text centered] 199 | 200 | % Draw the input layer nodes 201 | \foreach \name / \y in {1,...,4} 202 | % This is the same as writing \foreach \name / \y in {1/1,2/2,3/3,4/4} 203 | \node[input neuron, pin=left:Input \#\y] (I-\name) at (0,-\y) {}; 204 | 205 | % Draw the hidden layer nodes 206 | \foreach \name / \y in {1,...,5} 207 | \path[yshift=0.5cm] 208 | node[hidden neuron] (H-\name) at (\layersep,-\y cm) {}; 209 | 210 | % Draw the output layer node 211 | \node[output neuron,pin={[pin edge={->}]right:Output}, right of=H-3] (O) {}; 212 | 213 | % Connect every node in the input layer with every node in the 214 | % hidden layer. 215 | \foreach \source in {1,...,4} 216 | \foreach \dest in {1,...,5} 217 | \path (I-\source) edge (H-\dest); 218 | 219 | % Connect every node in the hidden layer with the output layer 220 | \foreach \source in {1,...,5} 221 | \path (H-\source) edge (O); 222 | 223 | % Annotate the layers 224 | \node[annot,above of=H-1, node distance=1cm] (hl) {Hidden layer}; 225 | \node[annot,left of=hl] {Input layer}; 226 | \node[annot,right of=hl] {Output layer}; 227 | \end{tikzpicture} 228 | % End of code 229 | \end{document} 230 | ``` 231 | 232 | 23. [**Quiver**](https://github.com/keplr-io/quiver) 233 | 234 | ![gzqll3](https://cloud.githubusercontent.com/assets/5866348/20253975/f3d56f14-a9e4-11e6-9693-9873a18df5d3.gif) 235 | 236 | **References :** 237 | 238 | 1) https://datascience.stackexchange.com/questions/12851/how-do-you-visualize-neural-network-architectures 239 | 240 | 2) https://datascience.stackexchange.com/questions/2670/visualizing-deep-neural-network-training 241 | --------------------------------------------------------------------------------