├── .gitignore ├── INSTALL.md ├── LICENSE ├── README.md ├── categories ├── ilsvrc_imagenet_categories.txt ├── places205_categories.txt └── places365_categories.txt ├── examples ├── inputs │ ├── brad_pitt.jpg │ ├── escher_sphere.jpg │ ├── frida_kahlo.jpg │ ├── golden_gate.jpg │ ├── hoovertowernight.jpg │ ├── picasso_selfport1907.jpg │ ├── seated-nude.jpg │ ├── shipwreck.jpg │ ├── starry_night.jpg │ ├── starry_night_crop.png │ ├── starry_night_google.jpg │ ├── the_scream.jpg │ ├── tubingen.jpg │ └── woman-with-hat-matisse.jpg └── outputs │ ├── goldengate_3a_5x5_reduce.png │ ├── goldengate_4d_3x3_reduce_avg10_lp4.png │ ├── goldengate_4d_5x5_s10.png │ ├── pitt_nin_cudnn.png │ ├── places205_4b_pool_proj.png │ ├── places365_big.png │ ├── places365_inception_4a_pool_proj.png │ ├── tubingen_4d_3x3_reduce_call.png │ ├── tubingen_4e_3x3_reduce_all.png │ ├── tubingen_avg10.png │ ├── tubingen_c1.png │ ├── tubingen_c108.png │ ├── tubingen_c119.png │ ├── tubingen_c25.png │ ├── tubingen_c25_108.png │ ├── tubingen_c25_119.png │ ├── tubingen_c29.png │ ├── tubingen_s10.png │ └── tubingen_w10.png ├── models └── download_models.py ├── neural_dream.py └── neural_dream ├── CaffeLoader.py ├── __init__.py ├── dream_auto.py ├── dream_experimental.py ├── dream_image.py ├── dream_model.py ├── dream_tile.py ├── dream_utils.py ├── helper_layers.py ├── loss_layers.py └── models ├── __init__.py ├── googlenet ├── __init__.py ├── bvlc_googlenet.py ├── googlenet_cars.py ├── googlenet_layer_names.py ├── googlenet_sos.py └── googlenetplaces.py ├── inception ├── __init__.py ├── inception5h.py ├── inception_layer_names.py └── inceptionv3_keras.py └── resnet ├── __init__.py ├── resnet_50_1by2_nsfw.py └── resnet_layer_names.py /.gitignore: -------------------------------------------------------------------------------- 1 | *.swp 2 | out*.png 3 | *.png 4 | *.jpg 5 | **.pyc 6 | **.pth 7 | models/ 8 | !models/download_models.py 9 | **__pycache__/ -------------------------------------------------------------------------------- /INSTALL.md: -------------------------------------------------------------------------------- 1 | # neural-dream Installation 2 | 3 | This guide will walk you through multiple ways to setup `neural-dream` on Ubuntu and Windows. If you wish to install PyTorch and neural-dream on a different operating system like MacOS, installation guides can be found [here](https://pytorch.org). 4 | 5 | Note that in order to reduce their size, the pre-packaged binary releases (pip, Conda, etc...) have removed support for some older GPUs, and thus you will have to install from source in order to use these GPUs. 6 | 7 | 8 | # Ubuntu: 9 | 10 | ## With A Package Manager: 11 | 12 | The pip and Conda packages ship with CUDA and cuDNN already built in, so after you have installed PyTorch with pip or Conda, you can skip to [installing neural-dream](https://github.com/ProGamerGov/neural-dream/blob/master/INSTALL.md#install-neural-dream). 13 | 14 | ### pip: 15 | 16 | The neural-dream PyPI page can be found here: https://pypi.org/project/neural-dream/ 17 | 18 | If you wish to install neural-dream as a pip package, then use the following command: 19 | 20 | ``` 21 | # in a terminal, run the command 22 | pip install neural-dream 23 | ``` 24 | 25 | Or: 26 | 27 | 28 | ``` 29 | # in a terminal, run the command 30 | pip3 install neural-dream 31 | ``` 32 | 33 | Next download the models with: 34 | 35 | 36 | ``` 37 | neural-dream -download_models 38 | ``` 39 | 40 | By default the models are downloaded to your home directory, but you can specify a download location with: 41 | 42 | ``` 43 | neural-dream -download_models -download_path 44 | ``` 45 | 46 | To download specific models or specific groups of models, you can use a comma separated list of models like this: 47 | 48 | ``` 49 | neural-dream -download_models all-caffe-googlenet,caffe-vgg19 50 | ``` 51 | 52 | To print all the models available for download, run the following command: 53 | 54 | ``` 55 | neural-dream -download_models print-all 56 | ``` 57 | 58 | #### Github and pip: 59 | 60 | Following the pip installation instructions 61 | [here](http://pytorch.org), you can install PyTorch with the following commands: 62 | 63 | ``` 64 | # in a terminal, run the commands 65 | cd ~/ 66 | pip install torch torchvision 67 | ``` 68 | 69 | Or: 70 | 71 | ``` 72 | cd ~/ 73 | pip3 install torch torchvision 74 | ``` 75 | 76 | Now continue on to [installing neural-dream](https://github.com/ProGamerGov/neural-dream/blob/master/INSTALL.md#install-neural-dream) to install neural-dream. 77 | 78 | ### Conda: 79 | 80 | Following the Conda installation instructions 81 | [here](http://pytorch.org), you can install PyTorch with the following command: 82 | 83 | ``` 84 | conda install pytorch torchvision -c pytorch 85 | ``` 86 | 87 | Now continue on to [installing neural-dream](https://github.com/ProGamerGov/neural-dream/blob/master/INSTALL.md#install-neural-dream) to install neural-dream. 88 | 89 | ## From Source: 90 | 91 | ### (Optional) Step 1: Install CUDA 92 | 93 | If you have a [CUDA-capable GPU from NVIDIA](https://developer.nvidia.com/cuda-gpus) then you can 94 | speed up `neural-dream` with CUDA. 95 | 96 | Instructions for downloading and installing the latest CUDA version on all supported operating systems, can be found [here](https://developer.nvidia.com/cuda-downloads). 97 | 98 | 99 | ### (Optional) Step 2: Install cuDNN 100 | 101 | cuDNN is a library from NVIDIA that efficiently implements many of the operations (like convolutions and pooling) 102 | that are commonly used in deep learning. 103 | 104 | After registering as a developer with NVIDIA, you can [download cuDNN here](https://developer.nvidia.com/cudnn). Make sure that you use the appropriate version of cuDNN for your version of CUDA. 105 | 106 | Follow the download instructions on Nvidia's site to install cuDNN correctly. 107 | 108 | Note that the cuDNN backend can only be used for GPU mode. 109 | 110 | ### (Optional) Steps 1-3: Install PyTorch with support for AMD GPUs using Radeon Open Compute Stack (ROCm) 111 | 112 | 113 | It is recommended that if you wish to use PyTorch with an AMD GPU, you install it via the official ROCm dockerfile: 114 | https://rocm.github.io/pytorch.html 115 | 116 | - Supported AMD GPUs for the dockerfile are: Vega10 / gfx900 generation discrete graphics cards (Vega56, Vega64, or MI25). 117 | 118 | PyTorch does not officially provide support for compilation on the host with AMD GPUs, but [a user guide posted here](https://github.com/ROCmSoftwarePlatform/pytorch/issues/337#issuecomment-467220107) apparently works well. 119 | 120 | ROCm utilizes a CUDA porting tool called HIP, which automatically converts CUDA code into HIP code. HIP code can run on both AMD and Nvidia GPUs. 121 | 122 | 123 | ### Step 3: Install PyTorch 124 | 125 | To install PyTorch [from source](https://github.com/pytorch/pytorch#from-source) on Ubuntu (Instructions may be different if you are using a different OS): 126 | 127 | ``` 128 | cd ~/ 129 | git clone --recursive https://github.com/pytorch/pytorch 130 | cd pytorch 131 | python setup.py install 132 | 133 | cd ~/ 134 | git clone --recursive https://github.com/pytorch/vision 135 | cd vision 136 | python setup.py install 137 | ``` 138 | 139 | To check that your torch installation is working, run the command `python` or `python3` to enter the Python interpreter. Then type `import torch` and hit enter. 140 | 141 | You can then type `print(torch.version.cuda)` and `print(torch.backends.cudnn.version())` to confirm that you are using the desired versions of CUDA and cuDNN. 142 | 143 | To quit just type `exit()` or use Ctrl-D. 144 | 145 | Now continue on to [installing neural-dream](https://github.com/ProGamerGov/neural-dream/blob/master/INSTALL.md#install-neural-dream) to install neural-dream. 146 | 147 | 148 | # Windows Installation 149 | 150 | If you wish to install PyTorch on Windows From Source or via Conda, you can find instructions on the PyTorch website: https://pytorch.org/ 151 | 152 | 153 | ### Github and pip 154 | 155 | First, you will need to download Python 3 and install it: https://www.python.org/downloads/windows/. I recommend using the executable installer for the latest version of Python 3. 156 | 157 | Then using https://pytorch.org/, get the correct pip command, paste it into the Command Prompt (CMD) and hit enter: 158 | 159 | 160 | ``` 161 | pip install torch===1.4.0 torchvision===0.5.0 -f https://download.pytorch.org/whl/torch_stable.html 162 | ``` 163 | 164 | 165 | After installing PyTorch, download the neural-dream Github repository and extract/unzip it to the desired location. 166 | 167 | Then copy the file path to your neural-dream folder, and paste it into the Command Prompt, with `cd` in front of it and then hit enter. 168 | 169 | In the example below, the neural-dream folder was placed on the desktop: 170 | 171 | ``` 172 | cd C:\Users\\Desktop\neural-dream-master 173 | ``` 174 | 175 | You can now continue on to [installing neural-dream](https://github.com/ProGamerGov/neural-dream/blob/master/INSTALL.md#install-neural-dream), skipping the `git clone` step. 176 | 177 | # Install neural-dream 178 | 179 | First we clone `neural-dream` from GitHub: 180 | 181 | ``` 182 | cd ~/ 183 | git clone https://github.com/ProGamerGov/neural-dream.git 184 | cd neural-dream 185 | ``` 186 | 187 | Next we need to download the pretrained neural network models: 188 | 189 | ``` 190 | python models/download_models.py 191 | ``` 192 | 193 | You should now be able to run `neural-dream` in CPU mode like this: 194 | 195 | ``` 196 | python neural_dream.py -gpu c -print_octave_iter 2 197 | ``` 198 | 199 | If you installed PyTorch with support for CUDA, then should now be able to run `neural-dream` in GPU mode like this: 200 | 201 | ``` 202 | python neural_dream.py -gpu 0 -print_octave_iter 5 203 | ``` 204 | 205 | If you installed PyTorch with support for cuDNN, then you should now be able to run `neural-dream` with the `cudnn` backend like this: 206 | 207 | ``` 208 | python neural_dream.py -gpu 0 -backend cudnn -print_octave_iter 5 209 | ``` 210 | 211 | If everything is working properly you should see output like this: 212 | 213 | ``` 214 | Octave iter 1 iteration 25 / 50 215 | DeepDream 1 loss: 19534752.0 216 | Octave iter 1 iteration 50 / 50 217 | DeepDream 1 loss: 23289720.0 218 | Octave iter 2 iteration 25 / 50 219 | DeepDream 1 loss: 38870436.0 220 | Octave iter 2 iteration 50 / 50 221 | DeepDream 1 loss: 47514664.0 222 | Iteration 1 / 10 223 | DeepDream 1 loss: 71727704.0 224 | Total loss: 2767866014.0 225 | Octave iter 1 iteration 25 / 50 226 | DeepDream 1 loss: 27209894.0 227 | Octave iter 1 iteration 50 / 50 228 | DeepDream 1 loss: 31386542.0 229 | Octave iter 2 iteration 25 / 50 230 | DeepDream 1 loss: 47773244.0 231 | Octave iter 2 iteration 50 / 50 232 | DeepDream 1 loss: 51204812.0 233 | Iteration 2 / 10 234 | DeepDream 1 loss: 87182300.0 235 | Total loss: 3758961954.0 236 | ``` -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2020 ProGamerGov 4 | 5 | Copyright (c) 2015 Justin Johnson 6 | 7 | Permission is hereby granted, free of charge, to any person obtaining a copy 8 | of this software and associated documentation files (the "Software"), to deal 9 | in the Software without restriction, including without limitation the rights 10 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 11 | copies of the Software, and to permit persons to whom the Software is 12 | furnished to do so, subject to the following conditions: 13 | 14 | The above copyright notice and this permission notice shall be included in all 15 | copies or substantial portions of the Software. 16 | 17 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 18 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 19 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 20 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 21 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 22 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 23 | SOFTWARE. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # neural-dream 2 | 3 | This is a PyTorch implementation of DeepDream. The code is based on [neural-style-pt](https://github.com/ProGamerGov/neural-style-pt). 4 | 5 |
6 | 7 |
8 | 9 | Here we DeepDream a photograph of the Golden Gate Bridge with a variety of settings: 10 | 11 |
12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 |
21 | 22 | 23 | ### Specific Channel Selection 24 | 25 | You can select individual or specific combinations of channels. 26 | 27 | Clockwise from upper left: 119, 1, 29, and all channels of the `inception_4d_3x3_reduce` layer 28 | 29 |
30 | 31 | 32 | 33 | 34 | 35 | 36 |
37 | 38 | Clockwise from upper left: 25, 108, 25 & 108, and 25 & 119 from the `inception_4d_3x3_reduce` layer 39 | 40 |
41 | 42 | 43 | 44 | 45 |
46 | 47 | ### Channel Selection Based On Activation Strength 48 | 49 | You can select channels automatically based on their activation strength. 50 | 51 | Clockwise from upper left: The top 10 weakest channels, the 10 most average channels, 52 | the top 10 strongest channels, and all channels of the `inception_4e_3x3_reduce` layer 53 | 54 | 55 |
56 | 57 | 58 | 59 | 60 | 61 | 62 |
63 | 64 | 65 | ## Setup: 66 | 67 | Dependencies: 68 | * [PyTorch](http://pytorch.org/) 69 | 70 | Optional dependencies: 71 | * For CUDA backend: 72 | * CUDA 7.5 or above 73 | * For cuDNN backend: 74 | * cuDNN v6 or above 75 | * For ROCm backend: 76 | * ROCm 2.1 or above 77 | * For MKL backend: 78 | * MKL 2019 or above 79 | * For OpenMP backend: 80 | * OpenMP 5.0 or above 81 | 82 | After installing the dependencies, you'll need to run the following script to download the BVLC GoogleNet model: 83 | ``` 84 | python models/download_models.py 85 | ``` 86 | This will download the original [BVLC GoogleNet model](https://github.com/BVLC/caffe/tree/master/models/bvlc_googlenet). 87 | 88 | If you have a smaller memory GPU then using the NIN Imagenet model could be an alternative to the BVLC GoogleNet model, though it's DeepDream quality is nowhere near that of the other models. You can get the details on the model from [BVLC Caffe ModelZoo](https://github.com/BVLC/caffe/wiki/Model-Zoo). The NIN model is downloaded when you run the `download_models.py` script with default parameters. 89 | 90 | To download most of the compatible models, run the `download_models.py` script with following parameters: 91 | 92 | ``` 93 | python models/download_models.py -models all 94 | ``` 95 | 96 | You can find detailed installation instructions for Ubuntu and Windows in the [installation guide](INSTALL.md). 97 | 98 | ## Usage 99 | Basic usage: 100 | ``` 101 | python neural_dream.py -content_image 102 | ``` 103 | 104 | cuDNN usage with NIN Model: 105 | ``` 106 | python neural_dream.py -content_image examples/inputs/brad_pitt.jpg -output_image pitt_nin_cudnn.png -model_file models/nin_imagenet.pth -gpu 0 -backend cudnn -num_iterations 10 -seed 876 -dream_layers relu0,relu3,relu7,relu12 -dream_weight 10 -image_size 512 -optimizer adam -learning_rate 0.1 107 | ``` 108 | 109 | ![cuDNN NIN Model Picasso Brad Pitt](https://raw.githubusercontent.com/ProGamerGov/neural-dream/master/examples/outputs/pitt_nin_cudnn.png) 110 | 111 | 112 | Note that paths to images should not contain the `~` character to represent your home directory; you should instead use a relative 113 | path or a full absolute path. 114 | 115 | **Options**: 116 | * `-image_size`: Maximum side length (in pixels) of the generated image. Default is 512. 117 | * `-gpu`: Zero-indexed ID of the GPU to use; for CPU mode set `-gpu` to `c`. 118 | 119 | **Optimization options**: 120 | * `-dream_weight`: How much to weight DeepDream. Default is `1e3`. 121 | * `-tv_weight`: Weight of total-variation (TV) regularization; this helps to smooth the image. 122 | Default is set to `0` to disable total-variation (TV) regularization. 123 | * `-l2_weight`: Weight of latent state regularization. 124 | Default is set to `0` to disable latent state regularization. 125 | * `-num_iterations`: Default is `10`. 126 | * `-init`: Method for generating the generated image; one of `random` or `image`. 127 | Default is `image` which initializes with the content image; `random` uses random noise to initialize the input image. 128 | * `-jitter`: Apply jitter to image. Default is `32`. Set to `0` to disable jitter. 129 | * `-layer_sigma`: Apply gaussian blur to image. Default is set to `0` to disable the gaussian blur layer. 130 | * `-optimizer`: The optimization algorithm to use; either `lbfgs` or `adam`; default is `adam`. 131 | Adam tends to perform the best for DeepDream. L-BFGS tends to give worse results and it uses more memory; when using L-BFGS you will probably need to play with other parameters to get good results, especially the learning rate. 132 | * `-learning_rate`: Learning rate to use with the ADAM and L-BFGS optimizers. Default is `1.5`. On other DeepDream projects this parameter is commonly called 'step size'. 133 | * `-normalize_weights`: If this flag is present, dream weights will be divided by the number of channels for each layer. Idea from [PytorchNeuralStyleTransfer](https://github.com/leongatys/PytorchNeuralStyleTransfer). 134 | * `-loss_mode`: The DeepDream loss mode; `bce`, `mse`, `mean`, `norm`, or `l2`; default is `l2`. 135 | 136 | **Output options**: 137 | * `-output_image`: Name of the output image. Default is `out.png`. 138 | * `-output_start_num`: The number to start output image names at. Default is set to `1`. 139 | * `-leading_zeros`: The number of leading zeros to use for output image names. Default is set to `0` to disable leading zeros. 140 | * `-print_iter`: Print progress every `print_iter` iterations. Set to `0` to disable printing. 141 | * `-print_octave_iter`: Print octave progress every `print_octave_iter` iterations. Default is set to `0` to disable printing. If tiling is enabled, then octave progress will be printed every `print_octave_iter` octaves. 142 | * `-save_iter`: Save the image every `save_iter` iterations. Set to `0` to disable saving intermediate results. 143 | * `-save_octave_iter`: Save the image every `save_octave_iter` iterations. Default is set to `0` to disable saving intermediate results. If tiling is enabled, then octaves will be saved every `save_octave_iter` octaves. 144 | 145 | **Layer options**: 146 | * `-dream_layers`: Comma-separated list of layer names to use for DeepDream reconstruction. 147 | 148 | **Channel options:** 149 | * `-channels`: Comma-separated list of channels to use for DeepDream. If `-channel_mode` is set to a value other than `all` or `ignore`, only the first value in the list will be used. 150 | * `-channel_mode`: The DeepDream channel selection mode; `all`, `strong`, `avg`, `weak`, or `ignore`; default is `all`. The `strong` option will select the strongest channels, while `weak` will do the same with the weakest channels. The `avg` option will select the most average channels instead of the strongest or weakest. The number of channels selected by `strong`, `avg`, or `weak` is based on the first value for the `-channels` parameter. The `ignore` option will omit any specified channels. 151 | * `-channel_capture`: How often to select channels based on activation strength; either `once` or `octave_iter`; default is `once`. The `once` option will select channels once at the start, while the `octave_iter` will select potentially new channels every octave iteration. This parameter only comes into play if `-channel_mode` is not set to `all` or `ignore`. 152 | 153 | **Octave options:** 154 | * `-num_octaves`: Number of octaves per iteration. Default is `4`. 155 | * `-octave_scale`: Value for resizing the image by. Default is `0.6`. 156 | * `-octave_iter`: Number of iterations per octave. Default is `50`. On other DeepDream projects this parameter is commonly called 'steps'. 157 | * `-octave_mode`: The octave size calculation mode; `normal`, `advanced`, `manual_max`, `manual_min`, or `manual`. Default is `normal`. If set to `manual_max` or `manual_min`, then `-octave_scale` takes a comma separated list of image sizes for the largest or smallest image dimension for `num_octaves` minus 1 octaves. If set `manual` then `-octave_scale` takes a comma separated list of image size pairs for `num_octaves` minus 1 octaves, in the form of `,`. 158 | 159 | **Laplacian Pyramid options:** 160 | * `-lap_scale`: The number of layers in a layer's laplacian pyramid. Default is set to `0` to disable laplacian pyramids. 161 | * `-sigma`: The strength of gaussian blur to use in laplacian pyramids. Default is `1`. By default, unless a second sigma value is provided with a comma to separate it from the first, the high gaussian layers will use sigma `sigma` * `lap_scale`. 162 | 163 | **Zoom options:** 164 | * `-zoom`: The amount to zoom in on the image. 165 | * `-zoom_mode`: Whether to read the zoom value as a percentage or pixel value; one of `percentage` or `pixel`. Default is `percentage`. 166 | 167 | **FFT options:** 168 | * `-use_fft`: Whether to enable Fast Fourier transform (FFT) decorrelation. 169 | * `-fft_block`: The size of your FFT frequency filtering block. Default is `25`. 170 | 171 | **Tiling options:** 172 | * `-tile_size`: The desired tile size to use. Default is set to `0` to disable tiling. 173 | * `-overlap_percent`: The percentage of overlap to use for the tiles. Default is `50`. 174 | * `-print_tile`: Print the current tile being processed every `print_tile` tiles without any other information. Default is set to `0` to disable printing. 175 | * `-print_tile_iter`: Print tile progress every `print_tile_iter` iterations. Default is set to `0` to disable printing. 176 | * `-image_capture_size`: The image size to use for the initial full image capture and optional `-classify` parameter. Default is set to `512`. Set to `0` disable it and `image_size` is used instead. 177 | 178 | **GIF options:** 179 | * `-create_gif`: Whether to create a GIF from the output images after all iterations have been completed. 180 | * `-frame_duration`: The duration for each GIF frame in milliseconds. Default is `100`. 181 | 182 | **Help options:** 183 | * `-print_layers`: Pass this flag to print the names of all usable layers for the selected model. 184 | * `-print_channels`: Pass this flag to print all the selected channels. 185 | 186 | **Other options**: 187 | * `-original_colors`: If you set this to `1`, then the output image will keep the colors of the content image. 188 | * `-model_file`: Path to the `.pth` file for the VGG Caffe model. Default is the original VGG-19 model; you can also try the original VGG-16 model. 189 | * `-model_type`: Whether the model was trained using Caffe, PyTorch, or Keras preprocessing; `caffe`, `pytorch`, `keras`, or `auto`; default is `auto`. 190 | * `-model_mean`: A comma separated list of 3 numbers for the model's mean; default is `auto`. 191 | * `-pooling`: The type of pooling layers to use for VGG and NIN models; one of `max` or `avg`. Default is `max`. VGG models seem to create better results with average pooling. 192 | * `-seed`: An integer value that you can specify for repeatable results. By default this value is random for each run. 193 | * `-multidevice_strategy`: A comma-separated list of layer indices at which to split the network when using multiple devices. See [Multi-GPU scaling](https://github.com/ProGamerGov/neural-dream#multi-gpu-scaling) for more details. Currently this feature only works for VGG and NIN models. 194 | * `-backend`: `nn`, `cudnn`, `openmp`, or `mkl`. Default is `nn`. `mkl` requires Intel's MKL backend. 195 | * `-cudnn_autotune`: When using the cuDNN backend, pass this flag to use the built-in cuDNN autotuner to select 196 | the best convolution algorithms for your architecture. This will make the first iteration a bit slower and can 197 | take a bit more memory, but may significantly speed up the cuDNN backend. 198 | * `-clamp`: If this flag is enabled, every iteration will clamp the output image so that it is within the model's input range. 199 | * `-adjust_contrast`: A value between `0` and `100.0` for altering the image's contrast (ex: `99.98`). Default is set to 0 to disable contrast adjustments. 200 | * `-label_file`: Path to the `.txt` category list file for classification and channel selection. 201 | * `-random_transforms`: Whether to use random transforms on the image; either `none`, `rotate`, `flip`, or `all`; default is `none`. 202 | * `-classify`: Display what the model thinks an image contains. Integer for the number of choices ranked by how likely each is. 203 | 204 | 205 | ## Frequently Asked Questions 206 | 207 | **Problem:** The program runs out of memory and dies 208 | 209 | **Solution:** Try reducing the image size: `-image_size 512` (or lower). Note that different image sizes will likely 210 | require non-default values for `-octave_scale` and `-num_octaves` for optimal results. 211 | If you are running on a GPU, you can also try running with `-backend cudnn` to reduce memory usage. 212 | 213 | **Problem:** `-backend cudnn` is slower than default NN backend 214 | 215 | **Solution:** Add the flag `-cudnn_autotune`; this will use the built-in cuDNN autotuner to select the best convolution algorithms. 216 | 217 | **Problem:** Get the following error message: 218 | 219 | `Missing key(s) in state_dict: "classifier.0.bias", "classifier.0.weight", "classifier.3.bias", "classifier.3.weight". 220 | Unexpected key(s) in state_dict: "classifier.1.weight", "classifier.1.bias", "classifier.4.weight", "classifier.4.bias".` 221 | 222 | **Solution:** Due to a mix up with layer locations, older models require a fix to be compatible with newer versions of PyTorch. The included [`donwload_models.py`](https://github.com/ProGamerGov/neural-dream/blob/master/models/download_models.py) script will automatically perform these fixes after downloading the models. 223 | 224 | **Problem:** Get the following error message: 225 | 226 | `Given input size: (...). Calculated output size: (...). Output size is too small` 227 | 228 | **Solution:** Use a larger `-image_size` value and/or adjust the octave parameters so that the smallest octave size is larger. 229 | 230 | ## Memory Usage 231 | By default, `neural-dream` uses the `nn` backend for convolutions and Adam for optimization. These give good results, but can both use a lot of memory. You can reduce memory usage with the following: 232 | 233 | * **Use cuDNN**: Add the flag `-backend cudnn` to use the cuDNN backend. This will only work in GPU mode. 234 | * **Reduce image size**: You can reduce the size of the generated image to lower memory usage; 235 | pass the flag `-image_size 256` to generate an image at half the default size. 236 | 237 | With the default settings, neural-dream uses about 1.3 GB of GPU memory on my system; switching to cuDNN reduces the GPU memory footprint to about 1 GB. 238 | 239 | 240 | ## Multi-GPU scaling 241 | You can use multiple CPU and GPU devices to process images at higher resolutions; different layers of the network will be 242 | computed on different devices. You can control which GPU and CPU devices are used with the `-gpu` flag, and you can control 243 | how to split layers across devices using the `-multidevice_strategy` flag. 244 | 245 | For example in a server with four GPUs, you can give the flag `-gpu 0,1,2,3` to process on GPUs 0, 1, 2, and 3 in that order; by also giving the flag `-multidevice_strategy 3,6,12` you indicate that the first two layers should be computed on GPU 0, layers 3 to 5 should be computed on GPU 1, layers 6 to 11 should be computed on GPU 2, and the remaining layers should be computed on GPU 3. You will need to tune the `-multidevice_strategy` for your setup in order to achieve maximal resolution. 246 | 247 | We can achieve very high quality results at high resolution by combining multi-GPU processing with multiscale 248 | generation as described in the paper 249 | **Controlling Perceptual Factors in Neural Style Transfer** by Leon A. Gatys, 250 | Alexander S. Ecker, Matthias Bethge, Aaron Hertzmann and Eli Shechtman. 251 | -------------------------------------------------------------------------------- /categories/ilsvrc_imagenet_categories.txt: -------------------------------------------------------------------------------- 1 | tench, Tinca tinca 2 | goldfish, Carassius auratus 3 | great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias 4 | tiger shark, Galeocerdo cuvieri 5 | hammerhead, hammerhead shark 6 | electric ray, crampfish, numbfish, torpedo 7 | stingray 8 | cock 9 | hen 10 | ostrich, Struthio camelus 11 | brambling, Fringilla montifringilla 12 | goldfinch, Carduelis carduelis 13 | house finch, linnet, Carpodacus mexicanus 14 | junco, snowbird 15 | indigo bunting, indigo finch, indigo bird, Passerina cyanea 16 | robin, American robin, Turdus migratorius 17 | bulbul 18 | jay 19 | magpie 20 | chickadee 21 | water ouzel, dipper 22 | kite 23 | bald eagle, American eagle, Haliaeetus leucocephalus 24 | vulture 25 | great grey owl, great gray owl, Strix nebulosa 26 | European fire salamander, Salamandra salamandra 27 | common newt, Triturus vulgaris 28 | eft 29 | spotted salamander, Ambystoma maculatum 30 | axolotl, mud puppy, Ambystoma mexicanum 31 | bullfrog, Rana catesbeiana 32 | tree frog, tree-frog 33 | tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui 34 | loggerhead, loggerhead turtle, Caretta caretta 35 | leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea 36 | mud turtle 37 | terrapin 38 | box turtle, box tortoise 39 | banded gecko 40 | common iguana, iguana, Iguana iguana 41 | American chameleon, anole, Anolis carolinensis 42 | whiptail, whiptail lizard 43 | agama 44 | frilled lizard, Chlamydosaurus kingi 45 | alligator lizard 46 | Gila monster, Heloderma suspectum 47 | green lizard, Lacerta viridis 48 | African chameleon, Chamaeleo chamaeleon 49 | Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis 50 | African crocodile, Nile crocodile, Crocodylus niloticus 51 | American alligator, Alligator mississipiensis 52 | triceratops 53 | thunder snake, worm snake, Carphophis amoenus 54 | ringneck snake, ring-necked snake, ring snake 55 | hognose snake, puff adder, sand viper 56 | green snake, grass snake 57 | king snake, kingsnake 58 | garter snake, grass snake 59 | water snake 60 | vine snake 61 | night snake, Hypsiglena torquata 62 | boa constrictor, Constrictor constrictor 63 | rock python, rock snake, Python sebae 64 | Indian cobra, Naja naja 65 | green mamba 66 | sea snake 67 | horned viper, cerastes, sand viper, horned asp, Cerastes cornutus 68 | diamondback, diamondback rattlesnake, Crotalus adamanteus 69 | sidewinder, horned rattlesnake, Crotalus cerastes 70 | trilobite 71 | harvestman, daddy longlegs, Phalangium opilio 72 | scorpion 73 | black and gold garden spider, Argiope aurantia 74 | barn spider, Araneus cavaticus 75 | garden spider, Aranea diademata 76 | black widow, Latrodectus mactans 77 | tarantula 78 | wolf spider, hunting spider 79 | tick 80 | centipede 81 | black grouse 82 | ptarmigan 83 | ruffed grouse, partridge, Bonasa umbellus 84 | prairie chicken, prairie grouse, prairie fowl 85 | peacock 86 | quail 87 | partridge 88 | African grey, African gray, Psittacus erithacus 89 | macaw 90 | sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita 91 | lorikeet 92 | coucal 93 | bee eater 94 | hornbill 95 | hummingbird 96 | jacamar 97 | toucan 98 | drake 99 | red-breasted merganser, Mergus serrator 100 | goose 101 | black swan, Cygnus atratus 102 | tusker 103 | echidna, spiny anteater, anteater 104 | platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus 105 | wallaby, brush kangaroo 106 | koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus 107 | wombat 108 | jellyfish 109 | sea anemone, anemone 110 | brain coral 111 | flatworm, platyhelminth 112 | nematode, nematode worm, roundworm 113 | conch 114 | snail 115 | slug 116 | sea slug, nudibranch 117 | chiton, coat-of-mail shell, sea cradle, polyplacophore 118 | chambered nautilus, pearly nautilus, nautilus 119 | Dungeness crab, Cancer magister 120 | rock crab, Cancer irroratus 121 | fiddler crab 122 | king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica 123 | American lobster, Northern lobster, Maine lobster, Homarus americanus 124 | spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish 125 | crayfish, crawfish, crawdad, crawdaddy 126 | hermit crab 127 | isopod 128 | white stork, Ciconia ciconia 129 | black stork, Ciconia nigra 130 | spoonbill 131 | flamingo 132 | little blue heron, Egretta caerulea 133 | American egret, great white heron, Egretta albus 134 | bittern 135 | crane 136 | limpkin, Aramus pictus 137 | European gallinule, Porphyrio porphyrio 138 | American coot, marsh hen, mud hen, water hen, Fulica americana 139 | bustard 140 | ruddy turnstone, Arenaria interpres 141 | red-backed sandpiper, dunlin, Erolia alpina 142 | redshank, Tringa totanus 143 | dowitcher 144 | oystercatcher, oyster catcher 145 | pelican 146 | king penguin, Aptenodytes patagonica 147 | albatross, mollymawk 148 | grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus 149 | killer whale, killer, orca, grampus, sea wolf, Orcinus orca 150 | dugong, Dugong dugon 151 | sea lion 152 | Chihuahua 153 | Japanese spaniel 154 | Maltese dog, Maltese terrier, Maltese 155 | Pekinese, Pekingese, Peke 156 | Shih-Tzu 157 | Blenheim spaniel 158 | papillon 159 | toy terrier 160 | Rhodesian ridgeback 161 | Afghan hound, Afghan 162 | basset, basset hound 163 | beagle 164 | bloodhound, sleuthhound 165 | bluetick 166 | black-and-tan coonhound 167 | Walker hound, Walker foxhound 168 | English foxhound 169 | redbone 170 | borzoi, Russian wolfhound 171 | Irish wolfhound 172 | Italian greyhound 173 | whippet 174 | Ibizan hound, Ibizan Podenco 175 | Norwegian elkhound, elkhound 176 | otterhound, otter hound 177 | Saluki, gazelle hound 178 | Scottish deerhound, deerhound 179 | Weimaraner 180 | Staffordshire bullterrier, Staffordshire bull terrier 181 | American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier 182 | Bedlington terrier 183 | Border terrier 184 | Kerry blue terrier 185 | Irish terrier 186 | Norfolk terrier 187 | Norwich terrier 188 | Yorkshire terrier 189 | wire-haired fox terrier 190 | Lakeland terrier 191 | Sealyham terrier, Sealyham 192 | Airedale, Airedale terrier 193 | cairn, cairn terrier 194 | Australian terrier 195 | Dandie Dinmont, Dandie Dinmont terrier 196 | Boston bull, Boston terrier 197 | miniature schnauzer 198 | giant schnauzer 199 | standard schnauzer 200 | Scotch terrier, Scottish terrier, Scottie 201 | Tibetan terrier, chrysanthemum dog 202 | silky terrier, Sydney silky 203 | soft-coated wheaten terrier 204 | West Highland white terrier 205 | Lhasa, Lhasa apso 206 | flat-coated retriever 207 | curly-coated retriever 208 | golden retriever 209 | Labrador retriever 210 | Chesapeake Bay retriever 211 | German short-haired pointer 212 | vizsla, Hungarian pointer 213 | English setter 214 | Irish setter, red setter 215 | Gordon setter 216 | Brittany spaniel 217 | clumber, clumber spaniel 218 | English springer, English springer spaniel 219 | Welsh springer spaniel 220 | cocker spaniel, English cocker spaniel, cocker 221 | Sussex spaniel 222 | Irish water spaniel 223 | kuvasz 224 | schipperke 225 | groenendael 226 | malinois 227 | briard 228 | kelpie 229 | komondor 230 | Old English sheepdog, bobtail 231 | Shetland sheepdog, Shetland sheep dog, Shetland 232 | collie 233 | Border collie 234 | Bouvier des Flandres, Bouviers des Flandres 235 | Rottweiler 236 | German shepherd, German shepherd dog, German police dog, alsatian 237 | Doberman, Doberman pinscher 238 | miniature pinscher 239 | Greater Swiss Mountain dog 240 | Bernese mountain dog 241 | Appenzeller 242 | EntleBucher 243 | boxer 244 | bull mastiff 245 | Tibetan mastiff 246 | French bulldog 247 | Great Dane 248 | Saint Bernard, St Bernard 249 | Eskimo dog, husky 250 | malamute, malemute, Alaskan malamute 251 | Siberian husky 252 | dalmatian, coach dog, carriage dog 253 | affenpinscher, monkey pinscher, monkey dog 254 | basenji 255 | pug, pug-dog 256 | Leonberg 257 | Newfoundland, Newfoundland dog 258 | Great Pyrenees 259 | Samoyed, Samoyede 260 | Pomeranian 261 | chow, chow chow 262 | keeshond 263 | Brabancon griffon 264 | Pembroke, Pembroke Welsh corgi 265 | Cardigan, Cardigan Welsh corgi 266 | toy poodle 267 | miniature poodle 268 | standard poodle 269 | Mexican hairless 270 | timber wolf, grey wolf, gray wolf, Canis lupus 271 | white wolf, Arctic wolf, Canis lupus tundrarum 272 | red wolf, maned wolf, Canis rufus, Canis niger 273 | coyote, prairie wolf, brush wolf, Canis latrans 274 | dingo, warrigal, warragal, Canis dingo 275 | dhole, Cuon alpinus 276 | African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus 277 | hyena, hyaena 278 | red fox, Vulpes vulpes 279 | kit fox, Vulpes macrotis 280 | Arctic fox, white fox, Alopex lagopus 281 | grey fox, gray fox, Urocyon cinereoargenteus 282 | tabby, tabby cat 283 | tiger cat 284 | Persian cat 285 | Siamese cat, Siamese 286 | Egyptian cat 287 | cougar, puma, catamount, mountain lion, painter, panther, Felis concolor 288 | lynx, catamount 289 | leopard, Panthera pardus 290 | snow leopard, ounce, Panthera uncia 291 | jaguar, panther, Panthera onca, Felis onca 292 | lion, king of beasts, Panthera leo 293 | tiger, Panthera tigris 294 | cheetah, chetah, Acinonyx jubatus 295 | brown bear, bruin, Ursus arctos 296 | American black bear, black bear, Ursus americanus, Euarctos americanus 297 | ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus 298 | sloth bear, Melursus ursinus, Ursus ursinus 299 | mongoose 300 | meerkat, mierkat 301 | tiger beetle 302 | ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle 303 | ground beetle, carabid beetle 304 | long-horned beetle, longicorn, longicorn beetle 305 | leaf beetle, chrysomelid 306 | dung beetle 307 | rhinoceros beetle 308 | weevil 309 | fly 310 | bee 311 | ant, emmet, pismire 312 | grasshopper, hopper 313 | cricket 314 | walking stick, walkingstick, stick insect 315 | cockroach, roach 316 | mantis, mantid 317 | cicada, cicala 318 | leafhopper 319 | lacewing, lacewing fly 320 | dragonfly, darning needle, devil's darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk 321 | damselfly 322 | admiral 323 | ringlet, ringlet butterfly 324 | monarch, monarch butterfly, milkweed butterfly, Danaus plexippus 325 | cabbage butterfly 326 | sulphur butterfly, sulfur butterfly 327 | lycaenid, lycaenid butterfly 328 | starfish, sea star 329 | sea urchin 330 | sea cucumber, holothurian 331 | wood rabbit, cottontail, cottontail rabbit 332 | hare 333 | Angora, Angora rabbit 334 | hamster 335 | porcupine, hedgehog 336 | fox squirrel, eastern fox squirrel, Sciurus niger 337 | marmot 338 | beaver 339 | guinea pig, Cavia cobaya 340 | sorrel 341 | zebra 342 | hog, pig, grunter, squealer, Sus scrofa 343 | wild boar, boar, Sus scrofa 344 | warthog 345 | hippopotamus, hippo, river horse, Hippopotamus amphibius 346 | ox 347 | water buffalo, water ox, Asiatic buffalo, Bubalus bubalis 348 | bison 349 | ram, tup 350 | bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis 351 | ibex, Capra ibex 352 | hartebeest 353 | impala, Aepyceros melampus 354 | gazelle 355 | Arabian camel, dromedary, Camelus dromedarius 356 | llama 357 | weasel 358 | mink 359 | polecat, fitch, foulmart, foumart, Mustela putorius 360 | black-footed ferret, ferret, Mustela nigripes 361 | otter 362 | skunk, polecat, wood pussy 363 | badger 364 | armadillo 365 | three-toed sloth, ai, Bradypus tridactylus 366 | orangutan, orang, orangutang, Pongo pygmaeus 367 | gorilla, Gorilla gorilla 368 | chimpanzee, chimp, Pan troglodytes 369 | gibbon, Hylobates lar 370 | siamang, Hylobates syndactylus, Symphalangus syndactylus 371 | guenon, guenon monkey 372 | patas, hussar monkey, Erythrocebus patas 373 | baboon 374 | macaque 375 | langur 376 | colobus, colobus monkey 377 | proboscis monkey, Nasalis larvatus 378 | marmoset 379 | capuchin, ringtail, Cebus capucinus 380 | howler monkey, howler 381 | titi, titi monkey 382 | spider monkey, Ateles geoffroyi 383 | squirrel monkey, Saimiri sciureus 384 | Madagascar cat, ring-tailed lemur, Lemur catta 385 | indri, indris, Indri indri, Indri brevicaudatus 386 | Indian elephant, Elephas maximus 387 | African elephant, Loxodonta africana 388 | lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens 389 | giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca 390 | barracouta, snoek 391 | eel 392 | coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch 393 | rock beauty, Holocanthus tricolor 394 | anemone fish 395 | sturgeon 396 | gar, garfish, garpike, billfish, Lepisosteus osseus 397 | lionfish 398 | puffer, pufferfish, blowfish, globefish 399 | abacus 400 | abaya 401 | academic gown, academic robe, judge's robe 402 | accordion, piano accordion, squeeze box 403 | acoustic guitar 404 | aircraft carrier, carrier, flattop, attack aircraft carrier 405 | airliner 406 | airship, dirigible 407 | altar 408 | ambulance 409 | amphibian, amphibious vehicle 410 | analog clock 411 | apiary, bee house 412 | apron 413 | ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin,, trash barrel, trash bin 414 | assault rifle, assault gun 415 | backpack, back pack, knapsack, packsack, rucksack, haversack 416 | bakery, bakeshop, bakehouse 417 | balance beam, beam 418 | balloon 419 | ballpoint, ballpoint pen, ballpen, Biro 420 | Band Aid 421 | banjo 422 | bannister, banister, balustrade, balusters, handrail 423 | barbell 424 | barber chair 425 | barbershop 426 | barn 427 | barometer 428 | barrel, cask 429 | barrow, garden cart, lawn cart, wheelbarrow 430 | baseball 431 | basketball 432 | bassinet 433 | bassoon 434 | bathing cap, swimming cap 435 | bath towel 436 | bathtub, bathing tub, bath, tub 437 | beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon 438 | beacon, lighthouse, beacon light, pharos 439 | beaker 440 | bearskin, busby, shako 441 | beer bottle 442 | beer glass 443 | bell cote, bell cot 444 | bib 445 | bicycle-built-for-two, tandem bicycle, tandem 446 | bikini, two-piece 447 | binder, ring-binder 448 | binoculars, field glasses, opera glasses 449 | birdhouse 450 | boathouse 451 | bobsled, bobsleigh, bob 452 | bolo tie, bolo, bola tie, bola 453 | bonnet, poke bonnet 454 | bookcase 455 | bookshop, bookstore, bookstall 456 | bottlecap 457 | bow 458 | bow tie, bow-tie, bowtie 459 | brass, memorial tablet, plaque 460 | brassiere, bra, bandeau 461 | breakwater, groin, groyne, mole, bulwark, seawall, jetty 462 | breastplate, aegis, egis 463 | broom 464 | bucket, pail 465 | buckle 466 | bulletproof vest 467 | bullet train, bullet 468 | butcher shop, meat market 469 | cab, hack, taxi, taxicab 470 | caldron, cauldron 471 | candle, taper, wax light 472 | cannon 473 | canoe 474 | can opener, tin opener 475 | cardigan 476 | car mirror 477 | carousel, carrousel, merry-go-round, roundabout, whirligig 478 | carpenter's kit, tool kit 479 | carton 480 | car wheel 481 | cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM 482 | cassette 483 | cassette player 484 | castle 485 | catamaran 486 | CD player 487 | cello, violoncello 488 | cellular telephone, cellular phone, cellphone, cell, mobile phone 489 | chain 490 | chainlink fence 491 | chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour 492 | chain saw, chainsaw 493 | chest 494 | chiffonier, commode 495 | chime, bell, gong 496 | china cabinet, china closet 497 | Christmas stocking 498 | church, church building 499 | cinema, movie theater, movie theatre, movie house, picture palace 500 | cleaver, meat cleaver, chopper 501 | cliff dwelling 502 | cloak 503 | clog, geta, patten, sabot 504 | cocktail shaker 505 | coffee mug 506 | coffeepot 507 | coil, spiral, volute, whorl, helix 508 | combination lock 509 | computer keyboard, keypad 510 | confectionery, confectionary, candy store 511 | container ship, containership, container vessel 512 | convertible 513 | corkscrew, bottle screw 514 | cornet, horn, trumpet, trump 515 | cowboy boot 516 | cowboy hat, ten-gallon hat 517 | cradle 518 | crane 519 | crash helmet 520 | crate 521 | crib, cot 522 | Crock Pot 523 | croquet ball 524 | crutch 525 | cuirass 526 | dam, dike, dyke 527 | desk 528 | desktop computer 529 | dial telephone, dial phone 530 | diaper, nappy, napkin 531 | digital clock 532 | digital watch 533 | dining table, board 534 | dishrag, dishcloth 535 | dishwasher, dish washer, dishwashing machine 536 | disk brake, disc brake 537 | dock, dockage, docking facility 538 | dogsled, dog sled, dog sleigh 539 | dome 540 | doormat, welcome mat 541 | drilling platform, offshore rig 542 | drum, membranophone, tympan 543 | drumstick 544 | dumbbell 545 | Dutch oven 546 | electric fan, blower 547 | electric guitar 548 | electric locomotive 549 | entertainment center 550 | envelope 551 | espresso maker 552 | face powder 553 | feather boa, boa 554 | file, file cabinet, filing cabinet 555 | fireboat 556 | fire engine, fire truck 557 | fire screen, fireguard 558 | flagpole, flagstaff 559 | flute, transverse flute 560 | folding chair 561 | football helmet 562 | forklift 563 | fountain 564 | fountain pen 565 | four-poster 566 | freight car 567 | French horn, horn 568 | frying pan, frypan, skillet 569 | fur coat 570 | garbage truck, dustcart 571 | gasmask, respirator, gas helmet 572 | gas pump, gasoline pump, petrol pump, island dispenser 573 | goblet 574 | go-kart 575 | golf ball 576 | golfcart, golf cart 577 | gondola 578 | gong, tam-tam 579 | gown 580 | grand piano, grand 581 | greenhouse, nursery, glasshouse 582 | grille, radiator grille 583 | grocery store, grocery, food market, market 584 | guillotine 585 | hair slide 586 | hair spray 587 | half track 588 | hammer 589 | hamper 590 | hand blower, blow dryer, blow drier, hair dryer, hair drier 591 | hand-held computer, hand-held microcomputer 592 | handkerchief, hankie, hanky, hankey 593 | hard disc, hard disk, fixed disk 594 | harmonica, mouth organ, harp, mouth harp 595 | harp 596 | harvester, reaper 597 | hatchet 598 | holster 599 | home theater, home theatre 600 | honeycomb 601 | hook, claw 602 | hoopskirt, crinoline 603 | horizontal bar, high bar 604 | horse cart, horse-cart 605 | hourglass 606 | iPod 607 | iron, smoothing iron 608 | jack-o'-lantern 609 | jean, blue jean, denim 610 | jeep, landrover 611 | jersey, T-shirt, tee shirt 612 | jigsaw puzzle 613 | jinrikisha, ricksha, rickshaw 614 | joystick 615 | kimono 616 | knee pad 617 | knot 618 | lab coat, laboratory coat 619 | ladle 620 | lampshade, lamp shade 621 | laptop, laptop computer 622 | lawn mower, mower 623 | lens cap, lens cover 624 | letter opener, paper knife, paperknife 625 | library 626 | lifeboat 627 | lighter, light, igniter, ignitor 628 | limousine, limo 629 | liner, ocean liner 630 | lipstick, lip rouge 631 | Loafer 632 | lotion 633 | loudspeaker, speaker, speaker unit, loudspeaker system, speaker system 634 | loupe, jeweler's loupe 635 | lumbermill, sawmill 636 | magnetic compass 637 | mailbag, postbag 638 | mailbox, letter box 639 | maillot 640 | maillot, tank suit 641 | manhole cover 642 | maraca 643 | marimba, xylophone 644 | mask 645 | matchstick 646 | maypole 647 | maze, labyrinth 648 | measuring cup 649 | medicine chest, medicine cabinet 650 | megalith, megalithic structure 651 | microphone, mike 652 | microwave, microwave oven 653 | military uniform 654 | milk can 655 | minibus 656 | miniskirt, mini 657 | minivan 658 | missile 659 | mitten 660 | mixing bowl 661 | mobile home, manufactured home 662 | Model T 663 | modem 664 | monastery 665 | monitor 666 | moped 667 | mortar 668 | mortarboard 669 | mosque 670 | mosquito net 671 | motor scooter, scooter 672 | mountain bike, all-terrain bike, off-roader 673 | mountain tent 674 | mouse, computer mouse 675 | mousetrap 676 | moving van 677 | muzzle 678 | nail 679 | neck brace 680 | necklace 681 | nipple 682 | notebook, notebook computer 683 | obelisk 684 | oboe, hautboy, hautbois 685 | ocarina, sweet potato 686 | odometer, hodometer, mileometer, milometer 687 | oil filter 688 | organ, pipe organ 689 | oscilloscope, scope, cathode-ray oscilloscope, CRO 690 | overskirt 691 | oxcart 692 | oxygen mask 693 | packet 694 | paddle, boat paddle 695 | paddlewheel, paddle wheel 696 | padlock 697 | paintbrush 698 | pajama, pyjama, pj's, jammies 699 | palace 700 | panpipe, pandean pipe, syrinx 701 | paper towel 702 | parachute, chute 703 | parallel bars, bars 704 | park bench 705 | parking meter 706 | passenger car, coach, carriage 707 | patio, terrace 708 | pay-phone, pay-station 709 | pedestal, plinth, footstall 710 | pencil box, pencil case 711 | pencil sharpener 712 | perfume, essence 713 | Petri dish 714 | photocopier 715 | pick, plectrum, plectron 716 | pickelhaube 717 | picket fence, paling 718 | pickup, pickup truck 719 | pier 720 | piggy bank, penny bank 721 | pill bottle 722 | pillow 723 | ping-pong ball 724 | pinwheel 725 | pirate, pirate ship 726 | pitcher, ewer 727 | plane, carpenter's plane, woodworking plane 728 | planetarium 729 | plastic bag 730 | plate rack 731 | plow, plough 732 | plunger, plumber's helper 733 | Polaroid camera, Polaroid Land camera 734 | pole 735 | police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria 736 | poncho 737 | pool table, billiard table, snooker table 738 | pop bottle, soda bottle 739 | pot, flowerpot 740 | potter's wheel 741 | power drill 742 | prayer rug, prayer mat 743 | printer 744 | prison, prison house 745 | projectile, missile 746 | projector 747 | puck, hockey puck 748 | punching bag, punch bag, punching ball, punchball 749 | purse 750 | quill, quill pen 751 | quilt, comforter, comfort, puff 752 | racer, race car, racing car 753 | racket, racquet 754 | radiator 755 | radio, wireless 756 | radio telescope, radio reflector 757 | rain barrel 758 | recreational vehicle, RV, R.V. 759 | reel 760 | reflex camera 761 | refrigerator, icebox 762 | remote control, remote 763 | restaurant, eating house, eating place, eatery 764 | revolver, six-gun, six-shooter 765 | rifle 766 | rocking chair, rocker 767 | rotisserie 768 | rubber eraser, rubber, pencil eraser 769 | rugby ball 770 | rule, ruler 771 | running shoe 772 | safe 773 | safety pin 774 | saltshaker, salt shaker 775 | sandal 776 | sarong 777 | sax, saxophone 778 | scabbard 779 | scale, weighing machine 780 | school bus 781 | schooner 782 | scoreboard 783 | screen, CRT screen 784 | screw 785 | screwdriver 786 | seat belt, seatbelt 787 | sewing machine 788 | shield, buckler 789 | shoe shop, shoe-shop, shoe store 790 | shoji 791 | shopping basket 792 | shopping cart 793 | shovel 794 | shower cap 795 | shower curtain 796 | ski 797 | ski mask 798 | sleeping bag 799 | slide rule, slipstick 800 | sliding door 801 | slot, one-armed bandit 802 | snorkel 803 | snowmobile 804 | snowplow, snowplough 805 | soap dispenser 806 | soccer ball 807 | sock 808 | solar dish, solar collector, solar furnace 809 | sombrero 810 | soup bowl 811 | space bar 812 | space heater 813 | space shuttle 814 | spatula 815 | speedboat 816 | spider web, spider's web 817 | spindle 818 | sports car, sport car 819 | spotlight, spot 820 | stage 821 | steam locomotive 822 | steel arch bridge 823 | steel drum 824 | stethoscope 825 | stole 826 | stone wall 827 | stopwatch, stop watch 828 | stove 829 | strainer 830 | streetcar, tram, tramcar, trolley, trolley car 831 | stretcher 832 | studio couch, day bed 833 | stupa, tope 834 | submarine, pigboat, sub, U-boat 835 | suit, suit of clothes 836 | sundial 837 | sunglass 838 | sunglasses, dark glasses, shades 839 | sunscreen, sunblock, sun blocker 840 | suspension bridge 841 | swab, swob, mop 842 | sweatshirt 843 | swimming trunks, bathing trunks 844 | swing 845 | switch, electric switch, electrical switch 846 | syringe 847 | table lamp 848 | tank, army tank, armored combat vehicle, armoured combat vehicle 849 | tape player 850 | teapot 851 | teddy, teddy bear 852 | television, television system 853 | tennis ball 854 | thatch, thatched roof 855 | theater curtain, theatre curtain 856 | thimble 857 | thresher, thrasher, threshing machine 858 | throne 859 | tile roof 860 | toaster 861 | tobacco shop, tobacconist shop, tobacconist 862 | toilet seat 863 | torch 864 | totem pole 865 | tow truck, tow car, wrecker 866 | toyshop 867 | tractor 868 | trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi 869 | tray 870 | trench coat 871 | tricycle, trike, velocipede 872 | trimaran 873 | tripod 874 | triumphal arch 875 | trolleybus, trolley coach, trackless trolley 876 | trombone 877 | tub, vat 878 | turnstile 879 | typewriter keyboard 880 | umbrella 881 | unicycle, monocycle 882 | upright, upright piano 883 | vacuum, vacuum cleaner 884 | vase 885 | vault 886 | velvet 887 | vending machine 888 | vestment 889 | viaduct 890 | violin, fiddle 891 | volleyball 892 | waffle iron 893 | wall clock 894 | wallet, billfold, notecase, pocketbook 895 | wardrobe, closet, press 896 | warplane, military plane 897 | washbasin, handbasin, washbowl, lavabo, wash-hand basin 898 | washer, automatic washer, washing machine 899 | water bottle 900 | water jug 901 | water tower 902 | whiskey jug 903 | whistle 904 | wig 905 | window screen 906 | window shade 907 | Windsor tie 908 | wine bottle 909 | wing 910 | wok 911 | wooden spoon 912 | wool, woolen, woollen 913 | worm fence, snake fence, snake-rail fence, Virginia fence 914 | wreck 915 | yawl 916 | yurt 917 | web site, website, internet site, site 918 | comic book 919 | crossword puzzle, crossword 920 | street sign 921 | traffic light, traffic signal, stoplight 922 | book jacket, dust cover, dust jacket, dust wrapper 923 | menu 924 | plate 925 | guacamole 926 | consomme 927 | hot pot, hotpot 928 | trifle 929 | ice cream, icecream 930 | ice lolly, lolly, lollipop, popsicle 931 | French loaf 932 | bagel, beigel 933 | pretzel 934 | cheeseburger 935 | hotdog, hot dog, red hot 936 | mashed potato 937 | head cabbage 938 | broccoli 939 | cauliflower 940 | zucchini, courgette 941 | spaghetti squash 942 | acorn squash 943 | butternut squash 944 | cucumber, cuke 945 | artichoke, globe artichoke 946 | bell pepper 947 | cardoon 948 | mushroom 949 | Granny Smith 950 | strawberry 951 | orange 952 | lemon 953 | fig 954 | pineapple, ananas 955 | banana 956 | jackfruit, jak, jack 957 | custard apple 958 | pomegranate 959 | hay 960 | carbonara 961 | chocolate sauce, chocolate syrup 962 | dough 963 | meat loaf, meatloaf 964 | pizza, pizza pie 965 | potpie 966 | burrito 967 | red wine 968 | espresso 969 | cup 970 | eggnog 971 | alp 972 | bubble 973 | cliff, drop, drop-off 974 | coral reef 975 | geyser 976 | lakeside, lakeshore 977 | promontory, headland, head, foreland 978 | sandbar, sand bar 979 | seashore, coast, seacoast, sea-coast 980 | valley, vale 981 | volcano 982 | ballplayer, baseball player 983 | groom, bridegroom 984 | scuba diver 985 | rapeseed 986 | daisy 987 | yellow lady's slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum 988 | corn 989 | acorn 990 | hip, rose hip, rosehip 991 | buckeye, horse chestnut, conker 992 | coral fungus 993 | agaric 994 | gyromitra 995 | stinkhorn, carrion fungus 996 | earthstar 997 | hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa 998 | bolete 999 | ear, spike, capitulum 1000 | toilet tissue, toilet paper, bathroom tissue 1001 | -------------------------------------------------------------------------------- /categories/places205_categories.txt: -------------------------------------------------------------------------------- 1 | abbey 2 | airport_terminal 3 | alley 4 | amphitheater 5 | amusement_park 6 | aquarium 7 | aqueduct 8 | arch 9 | art_gallery 10 | art_studio 11 | assembly_line 12 | attic 13 | auditorium 14 | apartment_building/outdoor 15 | badlands 16 | ballroom 17 | bamboo_forest 18 | banquet_hall 19 | bar 20 | baseball_field 21 | basement 22 | basilica 23 | bayou 24 | beauty_salon 25 | bedroom 26 | boardwalk 27 | boat_deck 28 | bookstore 29 | botanical_garden 30 | bowling_alley 31 | boxing_ring 32 | bridge 33 | building_facade 34 | bus_interior 35 | butchers_shop 36 | butte 37 | bakery/shop 38 | cafeteria 39 | campsite 40 | candy_store 41 | canyon 42 | castle 43 | cemetery 44 | chalet 45 | classroom 46 | closet 47 | clothing_store 48 | coast 49 | cockpit 50 | coffee_shop 51 | conference_center 52 | conference_room 53 | construction_site 54 | corn_field 55 | corridor 56 | cottage_garden 57 | courthouse 58 | courtyard 59 | creek 60 | crevasse 61 | crosswalk 62 | cathedral/outdoor 63 | church/outdoor 64 | dam 65 | dining_room 66 | dock 67 | dorm_room 68 | driveway 69 | desert/sand 70 | desert/vegetation 71 | dinette/home 72 | doorway/outdoor 73 | engine_room 74 | excavation 75 | fairway 76 | fire_escape 77 | fire_station 78 | food_court 79 | forest_path 80 | forest_road 81 | formal_garden 82 | fountain 83 | field/cultivated 84 | field/wild 85 | galley 86 | game_room 87 | garbage_dump 88 | gas_station 89 | gift_shop 90 | golf_course 91 | harbor 92 | herb_garden 93 | highway 94 | home_office 95 | hospital 96 | hospital_room 97 | hot_spring 98 | hotel_room 99 | hotel/outdoor 100 | ice_cream_parlor 101 | iceberg 102 | igloo 103 | islet 104 | ice_skating_rink/outdoor 105 | inn/outdoor 106 | jail_cell 107 | kasbah 108 | kindergarden_classroom 109 | kitchen 110 | kitchenette 111 | laundromat 112 | lighthouse 113 | living_room 114 | lobby 115 | locker_room 116 | mansion 117 | marsh 118 | martial_arts_gym 119 | mausoleum 120 | medina 121 | motel 122 | mountain 123 | mountain_snowy 124 | music_studio 125 | market/outdoor 126 | monastery/outdoor 127 | museum/indoor 128 | nursery 129 | ocean 130 | office 131 | office_building 132 | orchard 133 | pagoda 134 | palace 135 | pantry 136 | parking_lot 137 | parlor 138 | pasture 139 | patio 140 | pavilion 141 | phone_booth 142 | picnic_area 143 | playground 144 | plaza 145 | pond 146 | pulpit 147 | racecourse 148 | raft 149 | railroad_track 150 | rainforest 151 | reception 152 | residential_neighborhood 153 | restaurant 154 | restaurant_kitchen 155 | restaurant_patio 156 | rice_paddy 157 | river 158 | rock_arch 159 | rope_bridge 160 | ruin 161 | runway 162 | sandbar 163 | schoolhouse 164 | sea_cliff 165 | shed 166 | shoe_shop 167 | shopfront 168 | shower 169 | ski_resort 170 | ski_slope 171 | sky 172 | skyscraper 173 | slum 174 | snowfield 175 | staircase 176 | supermarket 177 | swamp 178 | stadium/baseball 179 | stadium/football 180 | stage/indoor 181 | subway_station/platform 182 | swimming_pool/outdoor 183 | television_studio 184 | topiary_garden 185 | tower 186 | train_railway 187 | tree_farm 188 | trench 189 | temple/east_asia 190 | temple/south_asia 191 | track/outdoor 192 | train_station/platform 193 | underwater/coral_reef 194 | valley 195 | vegetable_garden 196 | veranda 197 | viaduct 198 | volcano 199 | waiting_room 200 | water_tower 201 | watering_hole 202 | wheat_field 203 | wind_farm 204 | windmill 205 | yard -------------------------------------------------------------------------------- /categories/places365_categories.txt: -------------------------------------------------------------------------------- 1 | airfield 2 | airplane_cabin 3 | airport_terminal 4 | alcove 5 | alley 6 | amphitheater 7 | amusement_arcade 8 | amusement_park 9 | apartment_building/outdoor 10 | aquarium 11 | aqueduct 12 | arcade 13 | arch 14 | archaelogical_excavation 15 | archive 16 | arena/hockey 17 | arena/performance 18 | arena/rodeo 19 | army_base 20 | art_gallery 21 | art_school 22 | art_studio 23 | artists_loft 24 | assembly_line 25 | athletic_field/outdoor 26 | atrium/public 27 | attic 28 | auditorium 29 | auto_factory 30 | auto_showroom 31 | badlands 32 | bakery/shop 33 | balcony/exterior 34 | balcony/interior 35 | ball_pit 36 | ballroom 37 | bamboo_forest 38 | bank_vault 39 | banquet_hall 40 | bar 41 | barn 42 | barndoor 43 | baseball_field 44 | basement 45 | basketball_court/indoor 46 | bathroom 47 | bazaar/indoor 48 | bazaar/outdoor 49 | beach 50 | beach_house 51 | beauty_salon 52 | bedchamber 53 | bedroom 54 | beer_garden 55 | beer_hall 56 | berth 57 | biology_laboratory 58 | boardwalk 59 | boat_deck 60 | boathouse 61 | bookstore 62 | booth/indoor 63 | botanical_garden 64 | bow_window/indoor 65 | bowling_alley 66 | boxing_ring 67 | bridge 68 | building_facade 69 | bullring 70 | burial_chamber 71 | bus_interior 72 | bus_station/indoor 73 | butchers_shop 74 | butte 75 | cabin/outdoor 76 | cafeteria 77 | campsite 78 | campus 79 | canal/natural 80 | canal/urban 81 | candy_store 82 | canyon 83 | car_interior 84 | carrousel 85 | castle 86 | catacomb 87 | cemetery 88 | chalet 89 | chemistry_lab 90 | childs_room 91 | church/indoor 92 | church/outdoor 93 | classroom 94 | clean_room 95 | cliff 96 | closet 97 | clothing_store 98 | coast 99 | cockpit 100 | coffee_shop 101 | computer_room 102 | conference_center 103 | conference_room 104 | construction_site 105 | corn_field 106 | corral 107 | corridor 108 | cottage 109 | courthouse 110 | courtyard 111 | creek 112 | crevasse 113 | crosswalk 114 | dam 115 | delicatessen 116 | department_store 117 | desert/sand 118 | desert/vegetation 119 | desert_road 120 | diner/outdoor 121 | dining_hall 122 | dining_room 123 | discotheque 124 | doorway/outdoor 125 | dorm_room 126 | downtown 127 | dressing_room 128 | driveway 129 | drugstore 130 | elevator/door 131 | elevator_lobby 132 | elevator_shaft 133 | embassy 134 | engine_room 135 | entrance_hall 136 | escalator/indoor 137 | excavation 138 | fabric_store 139 | farm 140 | fastfood_restaurant 141 | field/cultivated 142 | field/wild 143 | field_road 144 | fire_escape 145 | fire_station 146 | fishpond 147 | flea_market/indoor 148 | florist_shop/indoor 149 | food_court 150 | football_field 151 | forest/broadleaf 152 | forest_path 153 | forest_road 154 | formal_garden 155 | fountain 156 | galley 157 | garage/indoor 158 | garage/outdoor 159 | gas_station 160 | gazebo/exterior 161 | general_store/indoor 162 | general_store/outdoor 163 | gift_shop 164 | glacier 165 | golf_course 166 | greenhouse/indoor 167 | greenhouse/outdoor 168 | grotto 169 | gymnasium/indoor 170 | hangar/indoor 171 | hangar/outdoor 172 | harbor 173 | hardware_store 174 | hayfield 175 | heliport 176 | highway 177 | home_office 178 | home_theater 179 | hospital 180 | hospital_room 181 | hot_spring 182 | hotel/outdoor 183 | hotel_room 184 | house 185 | hunting_lodge/outdoor 186 | ice_cream_parlor 187 | ice_floe 188 | ice_shelf 189 | ice_skating_rink/indoor 190 | ice_skating_rink/outdoor 191 | iceberg 192 | igloo 193 | industrial_area 194 | inn/outdoor 195 | islet 196 | jacuzzi/indoor 197 | jail_cell 198 | japanese_garden 199 | jewelry_shop 200 | junkyard 201 | kasbah 202 | kennel/outdoor 203 | kindergarden_classroom 204 | kitchen 205 | lagoon 206 | lake/natural 207 | landfill 208 | landing_deck 209 | laundromat 210 | lawn 211 | lecture_room 212 | legislative_chamber 213 | library/indoor 214 | library/outdoor 215 | lighthouse 216 | living_room 217 | loading_dock 218 | lobby 219 | lock_chamber 220 | locker_room 221 | mansion 222 | manufactured_home 223 | market/indoor 224 | market/outdoor 225 | marsh 226 | martial_arts_gym 227 | mausoleum 228 | medina 229 | mezzanine 230 | moat/water 231 | mosque/outdoor 232 | motel 233 | mountain 234 | mountain_path 235 | mountain_snowy 236 | movie_theater/indoor 237 | museum/indoor 238 | museum/outdoor 239 | music_studio 240 | natural_history_museum 241 | nursery 242 | nursing_home 243 | oast_house 244 | ocean 245 | office 246 | office_building 247 | office_cubicles 248 | oilrig 249 | operating_room 250 | orchard 251 | orchestra_pit 252 | pagoda 253 | palace 254 | pantry 255 | park 256 | parking_garage/indoor 257 | parking_garage/outdoor 258 | parking_lot 259 | pasture 260 | patio 261 | pavilion 262 | pet_shop 263 | pharmacy 264 | phone_booth 265 | physics_laboratory 266 | picnic_area 267 | pier 268 | pizzeria 269 | playground 270 | playroom 271 | plaza 272 | pond 273 | porch 274 | promenade 275 | pub/indoor 276 | racecourse 277 | raceway 278 | raft 279 | railroad_track 280 | rainforest 281 | reception 282 | recreation_room 283 | repair_shop 284 | residential_neighborhood 285 | restaurant 286 | restaurant_kitchen 287 | restaurant_patio 288 | rice_paddy 289 | river 290 | rock_arch 291 | roof_garden 292 | rope_bridge 293 | ruin 294 | runway 295 | sandbox 296 | sauna 297 | schoolhouse 298 | science_museum 299 | server_room 300 | shed 301 | shoe_shop 302 | shopfront 303 | shopping_mall/indoor 304 | shower 305 | ski_resort 306 | ski_slope 307 | sky 308 | skyscraper 309 | slum 310 | snowfield 311 | soccer_field 312 | stable 313 | stadium/baseball 314 | stadium/football 315 | stadium/soccer 316 | stage/indoor 317 | stage/outdoor 318 | staircase 319 | storage_room 320 | street 321 | subway_station/platform 322 | supermarket 323 | sushi_bar 324 | swamp 325 | swimming_hole 326 | swimming_pool/indoor 327 | swimming_pool/outdoor 328 | synagogue/outdoor 329 | television_room 330 | television_studio 331 | temple/asia 332 | throne_room 333 | ticket_booth 334 | topiary_garden 335 | tower 336 | toyshop 337 | train_interior 338 | train_station/platform 339 | tree_farm 340 | tree_house 341 | trench 342 | tundra 343 | underwater/ocean_deep 344 | utility_room 345 | valley 346 | vegetable_garden 347 | veterinarians_office 348 | viaduct 349 | village 350 | vineyard 351 | volcano 352 | volleyball_court/outdoor 353 | waiting_room 354 | water_park 355 | water_tower 356 | waterfall 357 | watering_hole 358 | wave 359 | wet_bar 360 | wheat_field 361 | wind_farm 362 | windmill 363 | yard 364 | youth_hostel 365 | zen_garden -------------------------------------------------------------------------------- /examples/inputs/brad_pitt.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/inputs/brad_pitt.jpg -------------------------------------------------------------------------------- /examples/inputs/escher_sphere.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/inputs/escher_sphere.jpg -------------------------------------------------------------------------------- /examples/inputs/frida_kahlo.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/inputs/frida_kahlo.jpg -------------------------------------------------------------------------------- /examples/inputs/golden_gate.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/inputs/golden_gate.jpg -------------------------------------------------------------------------------- /examples/inputs/hoovertowernight.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/inputs/hoovertowernight.jpg -------------------------------------------------------------------------------- /examples/inputs/picasso_selfport1907.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/inputs/picasso_selfport1907.jpg -------------------------------------------------------------------------------- /examples/inputs/seated-nude.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/inputs/seated-nude.jpg -------------------------------------------------------------------------------- /examples/inputs/shipwreck.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/inputs/shipwreck.jpg -------------------------------------------------------------------------------- /examples/inputs/starry_night.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/inputs/starry_night.jpg -------------------------------------------------------------------------------- /examples/inputs/starry_night_crop.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/inputs/starry_night_crop.png -------------------------------------------------------------------------------- /examples/inputs/starry_night_google.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/inputs/starry_night_google.jpg -------------------------------------------------------------------------------- /examples/inputs/the_scream.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/inputs/the_scream.jpg -------------------------------------------------------------------------------- /examples/inputs/tubingen.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/inputs/tubingen.jpg -------------------------------------------------------------------------------- /examples/inputs/woman-with-hat-matisse.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/inputs/woman-with-hat-matisse.jpg -------------------------------------------------------------------------------- /examples/outputs/goldengate_3a_5x5_reduce.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/outputs/goldengate_3a_5x5_reduce.png -------------------------------------------------------------------------------- /examples/outputs/goldengate_4d_3x3_reduce_avg10_lp4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/outputs/goldengate_4d_3x3_reduce_avg10_lp4.png -------------------------------------------------------------------------------- /examples/outputs/goldengate_4d_5x5_s10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/outputs/goldengate_4d_5x5_s10.png -------------------------------------------------------------------------------- /examples/outputs/pitt_nin_cudnn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/outputs/pitt_nin_cudnn.png -------------------------------------------------------------------------------- /examples/outputs/places205_4b_pool_proj.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/outputs/places205_4b_pool_proj.png -------------------------------------------------------------------------------- /examples/outputs/places365_big.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/outputs/places365_big.png -------------------------------------------------------------------------------- /examples/outputs/places365_inception_4a_pool_proj.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/outputs/places365_inception_4a_pool_proj.png -------------------------------------------------------------------------------- /examples/outputs/tubingen_4d_3x3_reduce_call.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/outputs/tubingen_4d_3x3_reduce_call.png -------------------------------------------------------------------------------- /examples/outputs/tubingen_4e_3x3_reduce_all.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/outputs/tubingen_4e_3x3_reduce_all.png -------------------------------------------------------------------------------- /examples/outputs/tubingen_avg10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/outputs/tubingen_avg10.png -------------------------------------------------------------------------------- /examples/outputs/tubingen_c1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/outputs/tubingen_c1.png -------------------------------------------------------------------------------- /examples/outputs/tubingen_c108.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/outputs/tubingen_c108.png -------------------------------------------------------------------------------- /examples/outputs/tubingen_c119.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/outputs/tubingen_c119.png -------------------------------------------------------------------------------- /examples/outputs/tubingen_c25.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/outputs/tubingen_c25.png -------------------------------------------------------------------------------- /examples/outputs/tubingen_c25_108.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/outputs/tubingen_c25_108.png -------------------------------------------------------------------------------- /examples/outputs/tubingen_c25_119.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/outputs/tubingen_c25_119.png -------------------------------------------------------------------------------- /examples/outputs/tubingen_c29.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/outputs/tubingen_c29.png -------------------------------------------------------------------------------- /examples/outputs/tubingen_s10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/outputs/tubingen_s10.png -------------------------------------------------------------------------------- /examples/outputs/tubingen_w10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/examples/outputs/tubingen_w10.png -------------------------------------------------------------------------------- /models/download_models.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import argparse 3 | from os import path 4 | from sys import version_info 5 | from collections import OrderedDict 6 | from torch.utils.model_zoo import load_url 7 | 8 | if version_info[0] < 3: 9 | import urllib 10 | else: 11 | import urllib.request 12 | 13 | 14 | options_list = ['all', 'caffe-vgg16', 'caffe-vgg19', 'caffe-nin', 'caffe-googlenet-places205', 'caffe-googlenet-places365', 'caffe-googlenet-bvlc', 'caffe-googlenet-cars', 'caffe-googlenet-sos', \ 15 | 'caffe-resnet-opennsfw', 'pytorch-vgg16', 'pytorch-vgg19', 'pytorch-googlenet', 'pytorch-inceptionv3', 'tensorflow-inception5h', 'keras-inceptionv3', 'all-caffe', 'all-caffe-googlenet'] 16 | 17 | 18 | def main(): 19 | params = params_list() 20 | if params.models == 'all': 21 | params.models = options_list[1:16] 22 | elif 'all-caffe' in params.models and 'all-caffe-googlenet' not in params.models: 23 | params.models = options_list[1:10] + params.models.split(',') 24 | elif 'all-caffe-googlenet' in params.models: 25 | params.models = options_list[4:9] + params.models.split(',') 26 | else: 27 | params.models = params.models.split(',') 28 | 29 | if 'caffe-vgg19' in params.models: 30 | # Download the VGG-19 ILSVRC model and fix the layer names 31 | print("Downloading the VGG-19 ILSVRC model") 32 | sd = load_url("https://web.eecs.umich.edu/~justincj/models/vgg19-d01eb7cb.pth") 33 | map = {'classifier.1.weight':u'classifier.0.weight', 'classifier.1.bias':u'classifier.0.bias', 'classifier.4.weight':u'classifier.3.weight', 'classifier.4.bias':u'classifier.3.bias'} 34 | sd = OrderedDict([(map[k] if k in map else k,v) for k,v in sd.items()]) 35 | torch.save(sd, path.join(params.download_path, "vgg19-d01eb7cb.pth")) 36 | 37 | if 'caffe-vgg16' in params.models: 38 | # Download the VGG-16 ILSVRC model and fix the layer names 39 | print("Downloading the VGG-16 ILSVRC model") 40 | sd = load_url("https://web.eecs.umich.edu/~justincj/models/vgg16-00b39a1b.pth") 41 | map = {'classifier.1.weight':u'classifier.0.weight', 'classifier.1.bias':u'classifier.0.bias', 'classifier.4.weight':u'classifier.3.weight', 'classifier.4.bias':u'classifier.3.bias'} 42 | sd = OrderedDict([(map[k] if k in map else k,v) for k,v in sd.items()]) 43 | torch.save(sd, path.join(params.download_path, "vgg16-00b39a1b.pth")) 44 | 45 | if 'caffe-nin' in params.models: 46 | # Download the NIN model 47 | print("Downloading the NIN model") 48 | fileurl = "https://raw.githubusercontent.com/ProGamerGov/pytorch-nin/master/nin_imagenet.pth" 49 | name = "nin_imagenet.pth" 50 | download_file(fileurl, name, params.download_path) 51 | 52 | if 'caffe-googlenet-places205' in params.models: 53 | # Download the Caffe GoogeLeNet Places205 model 54 | print("Downloading the Places205 GoogeLeNet model") 55 | fileurl = "https://github.com/ProGamerGov/pytorch-places/raw/master/googlenet_places205.pth" 56 | name = "googlenet_places205.pth" 57 | download_file(fileurl, name, params.download_path) 58 | 59 | if 'caffe-googlenet-places365' in params.models: 60 | # Download the Caffe GoogeLeNet Places365 model 61 | print("Downloading the Places365 GoogeLeNet model") 62 | fileurl = "https://github.com/ProGamerGov/pytorch-places/raw/master/googlenet_places365.pth" 63 | name = "googlenet_places365.pth" 64 | download_file(fileurl, name, params.download_path) 65 | 66 | if 'caffe-googlenet-bvlc' in params.models: 67 | # Download the Caffe BVLC GoogeLeNet model 68 | print("Downloading the BVLC GoogeLeNet model") 69 | fileurl = "https://github.com/ProGamerGov/pytorch-old-caffemodels/raw/master/bvlc_googlenet.pth" 70 | name = "bvlc_googlenet.pth" 71 | download_file(fileurl, name, params.download_path) 72 | 73 | if 'caffe-googlenet-cars' in params.models: 74 | # Download the Caffe GoogeLeNet Cars model 75 | print("Downloading the Cars GoogeLeNet model") 76 | fileurl = "https://github.com/ProGamerGov/pytorch-old-caffemodels/raw/master/googlenet_finetune_web_cars.pth" 77 | name = "googlenet_finetune_web_cars.pth" 78 | download_file(fileurl, name, params.download_path) 79 | 80 | if 'caffe-googlenet-sos' in params.models: 81 | # Download the Caffe GoogeLeNet SOS model 82 | print("Downloading the SOS GoogeLeNet model") 83 | fileurl = "https://github.com/ProGamerGov/pytorch-old-caffemodels/raw/master/GoogleNet_SOS.pth" 84 | name = "GoogleNet_SOS.pth" 85 | download_file(fileurl, name, params.download_path) 86 | 87 | if 'pytorch-vgg19' in params.models: 88 | # Download the PyTorch VGG19 model 89 | print("Downloading the PyTorch VGG 19 model") 90 | fileurl = "https://download.pytorch.org/models/vgg19-dcbb9e9d.pth" 91 | name = "vgg19-dcbb9e9d.pth" 92 | download_file(fileurl, name, params.download_path) 93 | 94 | if 'pytorch-vgg16' in params.models: 95 | # Download the PyTorch VGG16 model 96 | print("Downloading the PyTorch VGG 16 model") 97 | fileurl = "https://download.pytorch.org/models/vgg16-397923af.pth" 98 | name = "vgg16-397923af.pth" 99 | download_file(fileurl, name, params.download_path) 100 | 101 | if 'pytorch-googlenet' in params.models: 102 | # Download the PyTorch GoogLeNet model 103 | print("Downloading the PyTorch GoogLeNet model") 104 | fileurl = "https://download.pytorch.org/models/googlenet-1378be20.pth" 105 | name = "googlenet-1378be20.pth" 106 | download_file(fileurl, name, params.download_path) 107 | 108 | if 'pytorch-inception' in params.models: 109 | # Download the PyTorch Inception V3 model 110 | print("Downloading the PyTorch Inception V3 model") 111 | fileurl = "https://download.pytorch.org/models/inception_v3_google-1a9a5a14.pth" 112 | name = "inception_v3_google-1a9a5a14.pth" 113 | download_file(fileurl, name, params.download_path) 114 | 115 | if 'tensorflow-inception5h' in params.models: 116 | # Download the Inception5h model 117 | print("Downloading the TensorFlow Inception5h model") 118 | fileurl = "https://github.com/ProGamerGov/pytorch-old-tensorflow-models/raw/master/inception5h.pth" 119 | name = "inception5h.pth" 120 | download_file(fileurl, name, params.download_path) 121 | 122 | if 'keras-inceptionv3' in params.models: 123 | # Download the Keras Inception V3 model 124 | print("Downloading the Keras Inception V3 model") 125 | fileurl = "https://github.com/ProGamerGov/pytorch-old-tensorflow-models/raw/master/inceptionv3_keras.pth" 126 | name = "inceptionv3_keras.pth" 127 | download_file(fileurl, name, params.download_path) 128 | 129 | if 'caffe-resnet-opennsfw' in params.models: 130 | # Download the ResNet Yahoo Open NSFW model 131 | print("Downloading the ResNet Yahoo Open NSFW model") 132 | fileurl = "https://github.com/ProGamerGov/pytorch-old-caffemodels/raw/master/ResNet_50_1by2_nsfw.pth" 133 | name = "ResNet_50_1by2_nsfw.pth" 134 | download_file(fileurl, name, params.download_path) 135 | 136 | print("All selected models have been successfully downloaded") 137 | 138 | 139 | def params_list(): 140 | parser = argparse.ArgumentParser() 141 | parser.add_argument("-models", help="Models to download", default='caffe-googlenet-bvlc,caffe-nin', action=MultipleChoice) 142 | parser.add_argument("-download_path", help="Download location for models", default='models') 143 | params = parser.parse_args() 144 | return params 145 | 146 | 147 | def download_file(fileurl, name, download_path): 148 | if version_info[0] < 3: 149 | urllib.URLopener().retrieve(fileurl, path.join(download_path, name)) 150 | else: 151 | urllib.request.urlretrieve(fileurl, path.join(download_path, name)) 152 | 153 | 154 | class MultipleChoice(argparse.Action): 155 | def __call__(self, parser, namespace, values, option_string=None): 156 | self.options = options_list 157 | e = [o.lower() for o in values.split(',') if o.lower() not in self.options] 158 | if len(e) > 0: 159 | raise argparse.ArgumentError(self, 'invalid choices: ' + ','.join([str(v) for v in e]) + 160 | ' (choose from ' + ','.join([ "'"+str(v)+"'" for v in self.options])+')') 161 | setattr(namespace, self.dest, values) 162 | 163 | 164 | 165 | if __name__ == "__main__": 166 | main() 167 | -------------------------------------------------------------------------------- /neural_dream/CaffeLoader.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torch.nn as nn 3 | import torchvision 4 | from torchvision import models 5 | from neural_dream.models import * 6 | 7 | 8 | class VGG(nn.Module): 9 | def __init__(self, features, num_classes=1000): 10 | super(VGG, self).__init__() 11 | self.features = features 12 | self.classifier = nn.Sequential( 13 | nn.Linear(512 * 7 * 7, 4096), 14 | nn.ReLU(True), 15 | nn.Dropout(), 16 | nn.Linear(4096, 4096), 17 | nn.ReLU(True), 18 | nn.Dropout(), 19 | nn.Linear(4096, num_classes), 20 | ) 21 | 22 | 23 | class VGG_SOD(nn.Module): 24 | def __init__(self, features, num_classes=100): 25 | super(VGG_SOD, self).__init__() 26 | self.features = features 27 | self.classifier = nn.Sequential( 28 | nn.Linear(512 * 7 * 7, 4096), 29 | nn.ReLU(True), 30 | nn.Dropout(), 31 | nn.Linear(4096, 4096), 32 | nn.ReLU(True), 33 | nn.Dropout(), 34 | nn.Linear(4096, 100), 35 | ) 36 | 37 | 38 | class VGG_FCN32S(nn.Module): 39 | def __init__(self, features, num_classes=1000): 40 | super(VGG_FCN32S, self).__init__() 41 | self.features = features 42 | self.classifier = nn.Sequential( 43 | nn.Conv2d(512,4096,(7, 7)), 44 | nn.ReLU(True), 45 | nn.Dropout(0.5), 46 | nn.Conv2d(4096,4096,(1, 1)), 47 | nn.ReLU(True), 48 | nn.Dropout(0.5), 49 | ) 50 | 51 | 52 | class VGG_PRUNED(nn.Module): 53 | def __init__(self, features, num_classes=1000): 54 | super(VGG_PRUNED, self).__init__() 55 | self.features = features 56 | self.classifier = nn.Sequential( 57 | nn.Linear(512 * 7 * 7, 4096), 58 | nn.ReLU(True), 59 | nn.Dropout(0.5), 60 | nn.Linear(4096, 4096), 61 | nn.ReLU(True), 62 | nn.Dropout(0.5), 63 | ) 64 | 65 | 66 | class NIN(nn.Module): 67 | def __init__(self, pooling): 68 | super(NIN, self).__init__() 69 | if pooling == 'max': 70 | pool2d = nn.MaxPool2d((3, 3),(2, 2),(0, 0),ceil_mode=True) 71 | elif pooling == 'avg': 72 | pool2d = nn.AvgPool2d((3, 3),(2, 2),(0, 0),ceil_mode=True) 73 | 74 | self.features = nn.Sequential( 75 | nn.Conv2d(3,96,(11, 11),(4, 4)), 76 | nn.ReLU(inplace=True), 77 | nn.Conv2d(96,96,(1, 1)), 78 | nn.ReLU(inplace=True), 79 | nn.Conv2d(96,96,(1, 1)), 80 | nn.ReLU(inplace=True), 81 | pool2d, 82 | nn.Conv2d(96,256,(5, 5),(1, 1),(2, 2)), 83 | nn.ReLU(inplace=True), 84 | nn.Conv2d(256,256,(1, 1)), 85 | nn.ReLU(inplace=True), 86 | nn.Conv2d(256,256,(1, 1)), 87 | nn.ReLU(inplace=True), 88 | pool2d, 89 | nn.Conv2d(256,384,(3, 3),(1, 1),(1, 1)), 90 | nn.ReLU(inplace=True), 91 | nn.Conv2d(384,384,(1, 1)), 92 | nn.ReLU(inplace=True), 93 | nn.Conv2d(384,384,(1, 1)), 94 | nn.ReLU(inplace=True), 95 | pool2d, 96 | nn.Dropout(0.5), 97 | nn.Conv2d(384,1024,(3, 3),(1, 1),(1, 1)), 98 | nn.ReLU(inplace=True), 99 | nn.Conv2d(1024,1024,(1, 1)), 100 | nn.ReLU(inplace=True), 101 | nn.Conv2d(1024,1000,(1, 1)), 102 | nn.ReLU(inplace=True), 103 | nn.AvgPool2d((6, 6),(1, 1),(0, 0),ceil_mode=True), 104 | nn.Softmax(), 105 | ) 106 | 107 | 108 | 109 | class ModelParallel(nn.Module): 110 | def __init__(self, net, device_ids, device_splits): 111 | super(ModelParallel, self).__init__() 112 | self.device_list = self.name_devices(device_ids.split(',')) 113 | self.chunks = self.chunks_to_devices(self.split_net(net, device_splits.split(','))) 114 | 115 | def name_devices(self, input_list): 116 | device_list = [] 117 | for i, device in enumerate(input_list): 118 | if str(device).lower() != 'c': 119 | device_list.append("cuda:" + str(device)) 120 | else: 121 | device_list.append("cpu") 122 | return device_list 123 | 124 | def split_net(self, net, device_splits): 125 | chunks, cur_chunk = [], nn.Sequential() 126 | for i, l in enumerate(net): 127 | cur_chunk.add_module(str(i), net[i]) 128 | if str(i) in device_splits and device_splits != '': 129 | del device_splits[0] 130 | chunks.append(cur_chunk) 131 | cur_chunk = nn.Sequential() 132 | chunks.append(cur_chunk) 133 | return chunks 134 | 135 | def chunks_to_devices(self, chunks): 136 | for i, chunk in enumerate(chunks): 137 | chunk.to(self.device_list[i]) 138 | return chunks 139 | 140 | def c(self, input, i): 141 | if input.type() == 'torch.FloatTensor' and 'cuda' in self.device_list[i]: 142 | input = input.type('torch.cuda.FloatTensor') 143 | elif input.type() == 'torch.cuda.FloatTensor' and 'cpu' in self.device_list[i]: 144 | input = input.type('torch.FloatTensor') 145 | return input 146 | 147 | def forward(self, input): 148 | for i, chunk in enumerate(self.chunks): 149 | if i < len(self.chunks) -1: 150 | input = self.c(chunk(self.c(input, i).to(self.device_list[i])), i+1).to(self.device_list[i+1]) 151 | else: 152 | input = chunk(input) 153 | return input 154 | 155 | 156 | 157 | def buildSequential(channel_list, pooling): 158 | layers = [] 159 | in_channels = 3 160 | if pooling == 'max': 161 | pool2d = nn.MaxPool2d(kernel_size=2, stride=2) 162 | elif pooling == 'avg': 163 | pool2d = nn.AvgPool2d(kernel_size=2, stride=2) 164 | else: 165 | raise ValueError("Unrecognized pooling parameter") 166 | for c in channel_list: 167 | if c == 'P': 168 | layers += [pool2d] 169 | else: 170 | conv2d = nn.Conv2d(in_channels, c, kernel_size=3, padding=1) 171 | layers += [conv2d, nn.ReLU(inplace=True)] 172 | in_channels = c 173 | return nn.Sequential(*layers) 174 | 175 | 176 | channel_list = { 177 | 'VGG-11': [64, 'P', 128, 'P', 256, 256, 'P', 512, 512, 'P', 512, 512, 'P'], 178 | 'VGG-13': [64, 64, 'P', 128, 128, 'P', 256, 256, 'P', 512, 512, 'P', 512, 512, 'P'], 179 | 'VGG-16p': [24, 22, 'P', 41, 51, 'P', 108, 89, 111, 'P', 184, 276, 228, 'P', 512, 512, 512, 'P'], 180 | 'VGG-16': [64, 64, 'P', 128, 128, 'P', 256, 256, 256, 'P', 512, 512, 512, 'P', 512, 512, 512, 'P'], 181 | 'VGG-19': [64, 64, 'P', 128, 128, 'P', 256, 256, 256, 256, 'P', 512, 512, 512, 512, 'P', 512, 512, 512, 512, 'P'], 182 | } 183 | 184 | nin_dict = { 185 | 'C': ['conv1', 'cccp1', 'cccp2', 'conv2', 'cccp3', 'cccp4', 'conv3', 'cccp5', 'cccp6', 'conv4-1024', 'cccp7-1024', 'cccp8-1024'], 186 | 'R': ['relu0', 'relu1', 'relu2', 'relu3', 'relu5', 'relu6', 'relu7', 'relu8', 'relu9', 'relu10', 'relu11', 'relu12'], 187 | 'P': ['pool1', 'pool2', 'pool3', 'pool4'], 188 | 'D': ['drop'], 189 | } 190 | vgg11_dict = { 191 | 'C': ['conv1_1', 'conv2_1', 'conv3_1', 'conv3_2', 'conv4_1', 'conv4_2', 'conv5_1', 'conv5_2'], 192 | 'R': ['relu1_1', 'relu2_1', 'relu3_1', 'relu3_2', 'relu4_1', 'relu4_2', 'relu5_1', 'relu5_2', 'relu6', 'relu7'], 193 | 'P': ['pool1', 'pool2', 'pool3', 'pool4', 'pool5'], 194 | 'L': ['fc6', 'fc7', 'fc8'], 195 | 'D': ['drop6', 'drop7'], 196 | } 197 | vgg13_dict = { 198 | 'C': ['conv1_1', 'conv1_2', 'conv2_1', 'conv2_2', 'conv3_1', 'conv3_2', 'conv4_1', 'conv4_2', 'conv5_1', 'conv5_2'], 199 | 'R': ['relu1_1', 'relu1_2', 'relu2_1', 'relu2_2', 'relu3_1', 'relu3_2', 'relu4_1', 'relu4_2', 'relu5_1', 'relu5_2', 'relu6', 'relu7'], 200 | 'P': ['pool1', 'pool2', 'pool3', 'pool4', 'pool5'], 201 | 'L': ['fc6', 'fc7', 'fc8'], 202 | 'D': ['drop6', 'drop7'], 203 | } 204 | vgg16_dict = { 205 | 'C': ['conv1_1', 'conv1_2', 'conv2_1', 'conv2_2', 'conv3_1', 'conv3_2', 'conv3_3', 'conv4_1', 'conv4_2', 'conv4_3', 'conv5_1', 'conv5_2', 'conv5_3'], 206 | 'R': ['relu1_1', 'relu1_2', 'relu2_1', 'relu2_2', 'relu3_1', 'relu3_2', 'relu3_3', 'relu4_1', 'relu4_2', 'relu4_3', 'relu5_1', 'relu5_2', 'relu5_3', 'relu6', 'relu7'], 207 | 'P': ['pool1', 'pool2', 'pool3', 'pool4', 'pool5'], 208 | 'L': ['fc6', 'fc7', 'fc8'], 209 | 'D': ['drop6', 'drop7'], 210 | } 211 | vgg19_dict = { 212 | 'C': ['conv1_1', 'conv1_2', 'conv2_1', 'conv2_2', 'conv3_1', 'conv3_2', 'conv3_3', 'conv3_4', 'conv4_1', 'conv4_2', 'conv4_3', 'conv4_4', 'conv5_1', 'conv5_2', 'conv5_3', 'conv5_4'], 213 | 'R': ['relu1_1', 'relu1_2', 'relu2_1', 'relu2_2', 'relu3_1', 'relu3_2', 'relu3_3', 'relu3_4', 'relu4_1', 'relu4_2', 'relu4_3', 'relu4_4', 'relu5_1', 'relu5_2', 'relu5_3', 'relu5_4', 'relu6', 'relu7'], 214 | 'P': ['pool1', 'pool2', 'pool3', 'pool4', 'pool5'], 215 | 'L': ['fc6', 'fc7', 'fc8'], 216 | 'D': ['drop6', 'drop7'], 217 | } 218 | name_dict = { 219 | 'vgg': ['vgg'], 220 | 'vgg11': ['vgg-11', 'vgg11', 'vgg_11'], 221 | 'vgg13': ['vgg-13', 'vgg13', 'vgg_13'], 222 | 'vgg16': ['vgg-16', 'vgg16', 'vgg_16', 'fcn32s', 'pruning', 'sod'], 223 | 'vgg19': ['vgg-19', 'vgg19', 'vgg_19',], 224 | } 225 | ic_dict = { 226 | 'inception': ['inception'], 227 | 'googlenet': ['googlenet'], 228 | 'inceptionv3': ['inception_v3', 'inceptionv3'], 229 | 'resnet': ['resnet'], 230 | } 231 | 232 | 233 | 234 | def build_googlenet_list(cnn): 235 | main_layers = ['conv1', 'maxpool1', 'conv2', 'conv3', 'maxpool2', 'inception3a', 'inception3b', 'maxpool3', \ 236 | 'inception4a', 'inception4b', 'inception4c', 'inception4d', 'inception4e', 'maxpool4', 'inception5a', \ 237 | 'inception5b', 'aux1', 'aux2', 'avgpool', 'dropout', 'fc'] 238 | branch_list = ['branch1', 'branch2', 'branch3', 'branch4'] 239 | ax = ['conv', 'fc1', 'fc2'] 240 | conv_block =['conv', 'bn'] 241 | 242 | layer_name_list = [] 243 | 244 | for i, layer in enumerate(list(cnn.children())): 245 | if 'BasicConv2d' in str(type(layer)): 246 | for bl, block in enumerate(list(layer.children())): 247 | name = main_layers[i] + '/' + conv_block[bl] 248 | layer_name_list.append(name) 249 | elif 'Inception' in str(type(layer)) and 'Aux' not in str(type(layer)): 250 | for br, branch in enumerate(list(layer.children())): 251 | for bl, block in enumerate(list(branch.children())): 252 | name = main_layers[i] + '/' + branch_list[br] + '/' + conv_block[bl] 253 | layer_name_list.append(name) 254 | elif 'Inception' in str(type(layer)) and 'Aux' in str(type(layer)): 255 | for bl, block in enumerate(list(layer.children())): 256 | name = main_layers[i] + '/' + ax[bl] 257 | layer_name_list.append(name) 258 | elif isinstance(layer, nn.MaxPool2d) or isinstance(layer, nn.AdaptiveAvgPool2d) \ 259 | or isinstance(layer, nn.Dropout) or isinstance(layer, nn.Linear): 260 | layer_name_list.append(main_layers[i]) 261 | return layer_name_list 262 | 263 | 264 | def build_inceptionv3_list(cnn): 265 | main_layers = ['Conv2d_1a_3x3', 'Conv2d_2a_3x3', 'Conv2d_2b_3x3', 'Conv2d_3b_1x1', 'Conv2d_4a_3x3', 'Mixed_5b', \ 266 | 'Mixed_5c', 'Mixed_5d', 'Mixed_6a', 'Mixed_6b', 'Mixed_6c', 'Mixed_6d', 'Mixed_6e', 'AuxLogits', 'Mixed_7a', \ 267 | 'Mixed_7b', 'Mixed_7c', 'fc'] 268 | ba = ['branch1x1', 'branch5x5_1', 'branch5x5_2', 'branch3x3dbl_1', 'branch3x3dbl_2', 'branch3x3dbl_3', 'branch_pool'] 269 | bb = ['branch3x3', 'branch3x3dbl_1', 'branch3x3dbl_2', 'branch3x3dbl_3'] 270 | bc = ['branch1x1', 'branch7x7_1', 'branch7x7_2', 'branch7x7_3', 'branch7x7dbl_1', 'branch7x7dbl_2', 'branch7x7dbl_3', \ 271 | 'branch7x7dbl_4', 'branch7x7dbl_5', 'branch_pool'] 272 | bd = ['branch3x3_1', 'branch3x3_2', 'branch7x7x3_1', 'branch7x7x3_2', 'branch7x7x3_3', 'branch7x7x3_4'] 273 | be = ['branch1x1', 'branch3x3_1', 'branch3x3_2a', 'branch3x3_2b', 'branch3x3dbl_1', 'branch3x3dbl_2', \ 274 | 'branch3x3dbl_3a', 'branch3x3dbl_3b', 'branch_pool'] 275 | ax = ['conv0', 'conv1', 'fc'] 276 | conv_block =['conv', 'bn'] 277 | 278 | layer_name_list = [] 279 | 280 | for i, layer in enumerate(list(cnn.children())): 281 | if 'BasicConv2d' in str(type(layer)): 282 | for bl, block in enumerate(list(layer.children())): 283 | name = main_layers[i] + '/' + conv_block[bl] 284 | layer_name_list.append(name) 285 | elif 'Inception' in str(type(layer)) and 'Aux' not in str(type(layer)): 286 | if 'InceptionA' in str(type(layer)): 287 | branch_list = ba 288 | elif 'InceptionB' in str(type(layer)): 289 | branch_list = bb 290 | elif 'InceptionC' in str(type(layer)): 291 | branch_list = bc 292 | elif 'InceptionD' in str(type(layer)): 293 | branch_list = bd 294 | elif 'InceptionE' in str(type(layer)): 295 | branch_list = be 296 | for br, branch in enumerate(list(layer.children())): 297 | for bl, block in enumerate(list(branch.children())): 298 | name = main_layers[i] + '/' + branch_list[br] + '/' + conv_block[bl] 299 | layer_name_list.append(name) 300 | elif 'Inception' in str(type(layer)) and 'Aux' in str(type(layer)): 301 | for bl, block in enumerate(list(layer.children())): 302 | name = main_layers[i] + '/' + ax[bl] 303 | layer_name_list.append(name) 304 | elif isinstance(layer, nn.Linear): 305 | layer_name_list.append(main_layers[i]) 306 | return layer_name_list 307 | 308 | 309 | 310 | def modelSelector(model_file, pooling): 311 | if any(name in model_file for name in name_dict): 312 | if any(name in model_file for name in name_dict['vgg16']): 313 | print("VGG-16 Architecture Detected") 314 | if "pruning" in model_file: 315 | print("Using The Channel Pruning Model") 316 | cnn, layerList = VGG_PRUNED(buildSequential(channel_list['VGG-16p'], pooling)), vgg16_dict 317 | elif "fcn32s" in model_file: 318 | print("Using the fcn32s-heavy-pascal Model") 319 | cnn, layerList = VGG_FCN32S(buildSequential(channel_list['VGG-16'], pooling)), vgg16_dict 320 | layerList['C'] = layerList['C'] + layerList['L'] 321 | elif "sod" in model_file: 322 | print("Using The SOD Fintune Model") 323 | cnn, layerList = VGG_SOD(buildSequential(channel_list['VGG-16'], pooling)), vgg16_dict 324 | elif "16" in model_file: 325 | cnn, layerList = VGG(buildSequential(channel_list['VGG-16'], pooling)), vgg16_dict 326 | elif any(name in model_file for name in name_dict['vgg19']): 327 | print("VGG-19 Architecture Detected") 328 | if "19" in model_file: 329 | cnn, layerList = VGG(buildSequential(channel_list['VGG-19'], pooling)), vgg19_dict 330 | elif any(name in model_file for name in name_dict['vgg13']): 331 | print("VGG-13 Architecture Detected") 332 | cnn, layerList = VGG(buildSequential(channel_list['VGG-13'], pooling)), vgg13_dict 333 | elif any(name in model_file for name in name_dict['vgg11']): 334 | print("VGG-11 Architecture Detected") 335 | cnn, layerList = VGG(buildSequential(channel_list['VGG-11'], pooling)), vgg11_dict 336 | else: 337 | raise ValueError("VGG architecture not recognized.") 338 | elif "googlenet" in model_file: 339 | print("GoogLeNet Architecture Detected") 340 | if '205' in model_file: 341 | cnn, layerList = GoogLeNetPlaces(205), googlenet_layer_names('places') 342 | elif '365' in model_file: 343 | cnn, layerList = GoogLeNetPlaces(365), googlenet_layer_names('places') 344 | elif 'bvlc' in model_file: 345 | cnn, layerList = BVLC_GOOGLENET(), googlenet_layer_names('bvlc') 346 | elif 'cars' in model_file: 347 | cnn, layerList = GOOGLENET_CARS(), googlenet_layer_names('cars') 348 | elif 'sos' in model_file: 349 | cnn, layerList = GoogleNet_SOS(), googlenet_layer_names('sos') 350 | else: 351 | cnn, layerList = models.googlenet(pretrained=False, transform_input=False), '' 352 | elif "inception" in model_file: 353 | print("Inception Architecture Detected") 354 | if 'inception5h' in model_file: 355 | cnn, layerList = Inception5h(), inception_layer_names('5h') 356 | elif 'keras' in model_file: 357 | cnn, layerList = InceptionV3Keras(), inception_layer_names('keras-wtop') 358 | else: 359 | cnn, layerList = models.inception_v3(pretrained=False, transform_input=False), '' 360 | elif "resnet" in model_file: 361 | print("ResNet Architecture Detected") 362 | if 'resnet_50_1by2_nsfw' in model_file: 363 | cnn, layerList = ResNet_50_1by2_nsfw(), resnet_layer_names 364 | else: 365 | raise ValueError("ResNet architecture not recognized.") 366 | elif "nin" in model_file: 367 | print("NIN Architecture Detected") 368 | cnn, layerList = NIN(pooling), nin_dict 369 | else: 370 | raise ValueError("Model architecture not recognized.") 371 | return cnn, layerList 372 | 373 | 374 | # Print like Torch7/loadcaffe 375 | def print_loadcaffe(cnn, layerList): 376 | c = 0 377 | for l in list(cnn): 378 | if "Conv2d" in str(l) and "Basic" not in str(l): 379 | in_c, out_c, ks = str(l.in_channels), str(l.out_channels), str(l.kernel_size) 380 | print(layerList['C'][c] +": " + (out_c + " " + in_c + " " + ks).replace(")",'').replace("(",'').replace(",",'') ) 381 | c+=1 382 | if c == len(layerList['C']): 383 | break 384 | 385 | 386 | class Flatten(nn.Module): 387 | def forward(self, input): 388 | return torch.flatten(input, 1) 389 | 390 | 391 | def add_classifier_layers(cnn, pooling='avg'): 392 | new_cnn, cnn_classifier = cnn.features, cnn.classifier 393 | if 'avg' in pooling: 394 | adaptive_pool2d = nn.AdaptiveAvgPool2d((7, 7)) 395 | elif 'max' in pooling: 396 | adaptive_pool2d = nn.AdaptiveMaxPool2d((7, 7)) 397 | new_cnn.add_module(str(len(new_cnn)), adaptive_pool2d) 398 | 399 | if not isinstance(cnn, VGG_FCN32S): 400 | flatten_layer = Flatten() 401 | new_cnn.add_module(str(len(new_cnn)), flatten_layer) 402 | 403 | for layer in cnn_classifier: 404 | new_cnn.add_module(str(len(new_cnn)), layer) 405 | return new_cnn 406 | 407 | 408 | # Load the model, and configure pooling layer type 409 | def loadCaffemodel(model_file, pooling, use_gpu, disable_check, add_classifier=False): 410 | cnn, layerList = modelSelector(str(model_file).lower(), pooling) 411 | 412 | cnn.load_state_dict(torch.load(model_file, map_location='cpu'), strict=(not disable_check)) 413 | print("Successfully loaded " + str(model_file)) 414 | 415 | # Maybe convert the model to cuda now, to avoid later issues 416 | if "c" not in str(use_gpu).lower() or "c" not in str(use_gpu[0]).lower(): 417 | cnn = cnn.cuda() 418 | 419 | if not isinstance(cnn, NIN) and not any(name in model_file.lower() for name in ic_dict) and add_classifier: 420 | cnn, has_inception = add_classifier_layers(cnn, pooling), False 421 | elif any(name in model_file.lower() for name in ic_dict['googlenet']): 422 | if '205' in model_file or '365' in model_file: 423 | has_inception = True 424 | elif 'cars' in model_file.lower() or 'sos' in model_file.lower() or 'bvlc' in model_file.lower(): 425 | has_inception = True 426 | else: 427 | layerList, has_inception = build_googlenet_list(cnn), True 428 | elif any(name in model_file.lower() for name in ic_dict['inceptionv3']) and 'keras' not in model_file.lower(): 429 | layerList, has_inception = build_inceptionv3_list(cnn), True 430 | elif 'inception5h' in model_file.lower() or 'resnet' in model_file.lower() or 'keras' in model_file.lower(): 431 | has_inception = True 432 | else: 433 | cnn, has_inception = cnn.features, False 434 | 435 | if has_inception: 436 | cnn.eval() 437 | cnn.has_inception = True 438 | else: 439 | cnn.has_inception = False 440 | if not any(name in model_file.lower() for name in ic_dict): 441 | print_loadcaffe(cnn, layerList) 442 | 443 | try: 444 | cnn.add_layers() 445 | except: 446 | pass 447 | return cnn, layerList 448 | -------------------------------------------------------------------------------- /neural_dream/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/neural_dream/__init__.py -------------------------------------------------------------------------------- /neural_dream/dream_auto.py: -------------------------------------------------------------------------------- 1 | def model_name_list(): 2 | pytorch_names = ['vgg11-bbd30ac9.pth', 'vgg13-c768596a.pth', 'vgg16-397923af.pth' , \ 3 | 'vgg19-dcbb9e9d.pth', 'googlenet-1378be20.pth', 'inception_v3_google-1a9a5a14.pth'] 4 | caffe_names = ['vgg16-00b39a1b.pth', 'vgg19-d01eb7cb.pth', 'nin_imagenet.pth', \ 5 | 'VGG16_SOD_finetune.pth', 'VGG16-Stylized-ImageNet.pth', 'vgg16_places365.pth', \ 6 | 'vgg16_hybrid1365.pth', 'fcn32s-heavy-pascal.pth', 'nyud-fcn32s-color-heavy.pth', \ 7 | 'pascalcontext-fcn32s-heavy.pth', 'siftflow-fcn32s-heavy.pth', 'channel_pruning.pth', \ 8 | 'googlenet_places205.pth', 'googlenet_places365.pth', 'resnet_50_1by2_nsfw.pth', \ 9 | 'bvlc_googlenet.pth', 'googlenet_finetune_web_cars.pth', 'googlenet_sos.pth', 'inception5h'] 10 | keras_names = ['inceptionv3_keras.pth'] 11 | return pytorch_names, caffe_names, keras_names 12 | 13 | # Automatically determine model type 14 | def auto_model_mode(model_name): 15 | pytorch_names, caffe_names, keras_names = model_name_list() 16 | if any(name.lower() in model_name.lower() for name in pytorch_names): 17 | input_mode = 'pytorch' 18 | elif any(name.lower() in model_name.lower() for name in caffe_names): 19 | input_mode = 'caffe' 20 | elif any(name.lower() in model_name.lower() for name in keras_names): 21 | input_mode = 'keras' 22 | else: 23 | raise ValueError("Model not recognized, please manually specify the model type.") 24 | return input_mode 25 | 26 | 27 | # Automatically determine preprocessing to use for model 28 | def auto_mean(model_name, model_type): 29 | pytorch_names, caffe_names, keras_names = model_name_list() 30 | if any(name.lower() in model_name.lower() for name in pytorch_names) or model_type == 'pytorch': 31 | input_mean = [0.485, 0.456, 0.406] # PyTorch Imagenet 32 | elif any(name.lower() in model_name.lower() for name in caffe_names) or model_type == 'caffe': 33 | input_mean = [103.939, 116.779, 123.68] # Caffe Imagenet 34 | if 'googlenet_places205.pth' in model_name.lower(): 35 | input_mean = [105.417, 113.753, 116.047] # Caffe Places205 36 | elif 'googlenet_places365.pth' in model_name.lower(): 37 | input_mean = [104.051, 112.514, 116.676] # Caffe Places365 38 | elif 'resnet_50_1by2_nsfw.pth' in model_name.lower(): 39 | input_mean = [104, 117, 123] # Caffe Open NSFW 40 | elif any(name.lower() in model_name.lower() for name in keras_names) or model_type == 'keras': 41 | input_mean = [0, 0, 0] 42 | else: 43 | raise ValueError("Model not recognized, please manually specify the model type or model mean.") 44 | return input_mean -------------------------------------------------------------------------------- /neural_dream/dream_experimental.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torch.nn as nn 3 | 4 | 5 | # Define a function to partially zero inputs based on channel strength 6 | def percentile_zero(input, p=99.98, mode='norm'): 7 | if 'norm' in mode: 8 | px = input.norm(1) 9 | elif 'sum' in mode: 10 | px = input.sum(1) 11 | elif 'mean' in mode: 12 | px = input.sum(1) 13 | if 'abs' in mode: 14 | px, tp = dream_utils.tensor_percentile(abs(px), p), abs(px) 15 | else: 16 | tp = dream_utils.tensor_percentile(px, p) 17 | 18 | th = (0.01*tp) 19 | 20 | if 'abs' in mode: 21 | input[abs(input) < abs(th)] = 0 22 | else: 23 | input[input < th] = 0 24 | return input 25 | 26 | 27 | # Define an nn Module to mask inputs based on channel strength 28 | class ChannelMask(torch.nn.Module): 29 | 30 | def __init__(self, mode, channels=-1, rank_mode='norm', channel_percent=-1): 31 | super(ChannelMask, self).__init__() 32 | self.mode = mode 33 | self.channels = channels 34 | self.rank_mode = rank_mode 35 | self.channel_percent = channel_percent 36 | 37 | def list_channels(self, input): 38 | if input.is_cuda: 39 | channel_list = torch.zeros(input.size(1), device=input.get_device()) 40 | else: 41 | channel_list = torch.zeros(input.size(1)) 42 | for i in range(input.size(1)): 43 | y = input.clone().narrow(1,i,1) 44 | if self.rank_mode == 'norm': 45 | y = torch.norm(y) 46 | elif self.rank_mode == 'sum': 47 | y = torch.sum(y) 48 | elif self.rank_mode == 'mean': 49 | y = torch.mean(y) 50 | elif self.rank_mode == 'norm-abs': 51 | y = torch.norm(torch.abs(y)) 52 | elif self.rank_mode == 'sum-abs': 53 | y = torch.sum(torch.abs(y)) 54 | elif self.rank_mode == 'mean-abs': 55 | y = torch.mean(torch.abs(y)) 56 | channel_list[i] = y 57 | return channel_list 58 | 59 | def channel_strengths(self, input, num_channels): 60 | channel_list = self.list_channels(input) 61 | channels, idx = torch.sort(channel_list, 0, True) 62 | selected_channels = [] 63 | for i in range(num_channels): 64 | if i < input.size(1): 65 | selected_channels.append(idx[i]) 66 | return selected_channels 67 | 68 | def mask_threshold(self, input, channels, ft=1, fm=0.2, fw=5): 69 | t = torch.ones_like(input.squeeze(0)) * ft 70 | m = torch.ones_like(input.squeeze(0)) * fm 71 | for c in channels: 72 | m[c] = fw 73 | return (t * m).unsqueeze(0) 74 | 75 | def average_mask(self, input, channels): 76 | mask = torch.ones_like(input.squeeze(0)) 77 | avg = torch.sum(channels)/input.size(1) 78 | for i in range(channels.size(1)): 79 | w = avg/channels[i] 80 | mask[i] = w 81 | return mask.unsqueeze(0) 82 | 83 | def average_tensor(self, input): 84 | channel_list = self.list_channels(input) 85 | mask = self.average_mask(input, channel_list) 86 | self.mask = mask 87 | 88 | def weak_channels(self, input): 89 | channel_list = self.channel_strengths(input, self.channels) 90 | mask = self.mask_threshold(input, channel_list, ft=1, fm=2, fw=0.2) 91 | self.mask = mask 92 | 93 | def strong_channels(self, input): 94 | channel_list = self.channel_strengths(input, self.channels) 95 | mask = self.mask_threshold(input, channel_list, ft=1, fm=0.2, fw=5) 96 | self.mask = mask 97 | 98 | def zero_weak(self, input): 99 | channel_list = self.channel_strengths(input, self.channels) 100 | mask = self.mask_threshold(input, channel_list, ft=1, fm=0, fw=1) 101 | self.mask = mask 102 | 103 | def mask_input(self, input): 104 | if self.channel_percent > 0: 105 | channels = int((float(self.channel_percent)/100) * float(input.size(1))) 106 | if channels < input.size(1) and channels > 0: 107 | self.channels = channels 108 | else: 109 | self.channels = input.size(1) 110 | if self.mode == 'weak': 111 | input = self.weak_channels(input) 112 | elif self.mode == 'strong': 113 | input = self.strong_channels(input) 114 | elif self.mode == 'average': 115 | input = self.average_tensor(input) 116 | elif self.mode == 'zero_weak': 117 | input = self.zero_weak(input) 118 | 119 | def capture(self, input): 120 | self.mask_input(input) 121 | 122 | def forward(self, input): 123 | return self.mask * input 124 | 125 | 126 | # Define a function to partially zero inputs based on channel strength 127 | class ChannelMod(torch.nn.Module): 128 | 129 | def __init__(self, p_mode='fast', channels=0, norm_p=0, abs_p=0, mean_p=0): 130 | super(ChannelMod, self).__init__() 131 | self.p_mode = p_mode 132 | self.channels = channels 133 | self.norm_p = norm_p 134 | self.abs_p = abs_p 135 | self.mean_p = mean_p 136 | self.enabled = False 137 | if self.norm_p > 0 and self.p_mode == 'slow': 138 | self.zero_weak_norm = ChannelMask('zero_weak', self.channels, 'norm', channel_percent=self.norm_p) 139 | if self.abs_p > 0 and self.p_mode == 'slow': 140 | self.zero_weak_abs = ChannelMask('zero_weak', self.channels, 'sum', channel_percent=self.abs_p) 141 | if self.mean_p > 0 and self.p_mode == 'slow': 142 | self.zero_weak_mean = ChannelMask('zero_weak', self.channels, 'mean', channel_percent=self.mean_p) 143 | 144 | if self.norm_p > 0 or self.abs_p > 0 or self.mean_p > 0: 145 | self.enabled = True 146 | 147 | def forward(self, input): 148 | if self.norm_p > 0 and self.p_mode == 'fast': 149 | input = percentile_zero(input, p=self.norm_p, mode='abs-norm') 150 | if self.abs_p > 0 and self.p_mode == 'fast': 151 | input = percentile_zero(input, p=self.abs_p, mode='abs-sum') 152 | if self.mean_p > 0 and self.p_mode == 'fast': 153 | input = percentile_zero(input, p=self.mean_p, mode='abs-mean') 154 | 155 | if self.norm_p > 0 and self.p_mode == 'slow': 156 | self.zero_weak_norm.capture(input.clone()) 157 | input = self.zero_weak_norm(input) 158 | if self.abs_p > 0 and self.p_mode == 'slow': 159 | self.zero_weak_abs.capture(input.clone()) 160 | input = self.zero_weak_abs(input) 161 | if self.mean_p > 0 and self.p_mode == 'slow': 162 | self.zero_weak_mean.capture(input.clone()) 163 | input = self.zero_weak_mean(input) 164 | return input -------------------------------------------------------------------------------- /neural_dream/dream_image.py: -------------------------------------------------------------------------------- 1 | import os 2 | import torch 3 | import torch.nn as nn 4 | import neural_dream.dream_utils as dream_utils 5 | from PIL import Image 6 | 7 | 8 | # Adjust tensor contrast 9 | def adjust_contrast(t, r, p=99.98): 10 | return t * (r / dream_utils.tensor_percentile(t)) 11 | 12 | 13 | # Resize tensor 14 | def resize_tensor(tensor, size, mode='bilinear'): 15 | return torch.nn.functional.interpolate(tensor.clone(), size=size, mode=mode, align_corners=True) 16 | 17 | 18 | # Center crop a tensor 19 | def center_crop(input, crop_val, mode='percent'): 20 | h, w = input.size(2), input.size(3) 21 | if mode == 'percent': 22 | h_crop = int((crop_val / 100) * input.size(2)) 23 | w_crop = int((crop_val / 100) * input.size(3)) 24 | elif mode == 'pixel': 25 | h_crop = input.size(2) - crop_val 26 | w_crop = input.size(3) - crop_val 27 | sw, sh = w // 2 - (w_crop // 2), h // 2 - (h_crop // 2) 28 | return input[:, :, sh:sh + h_crop, sw:sw + w_crop] 29 | 30 | 31 | # Center crop and resize a tensor 32 | def zoom(input, crop_val, mode='percent'): 33 | h, w = input.size(2), input.size(3) 34 | input = center_crop(input.clone(), crop_val, mode=mode) 35 | input = resize_tensor(input, (h,w)) 36 | return input 37 | 38 | 39 | # Get most common Pillow Image size from list 40 | def common_size(l, v): 41 | l = [list(im.size)[v] for im in l] 42 | return max(set(l), key = l.count) 43 | 44 | 45 | # Create GIF from images 46 | def create_gif(base_name, duration=100): 47 | frames_dir, base_name = os.path.split(base_name) 48 | if frames_dir == '': 49 | frames_dir = '.' 50 | base_name = base_name.rsplit('.', 1)[0] 51 | 52 | ext = [".jpg", ".jpeg", ".png", ".tiff"] 53 | image_list = [file for file in os.listdir(frames_dir) if os.path.splitext(file)[1].lower() in ext] 54 | image_list = [im for im in image_list if base_name in im] 55 | if "_" in image_list[0]: 56 | fsorted = sorted(image_list,key=lambda x: int(os.path.splitext(x)[0].rsplit('_', 1)[1])) 57 | else: 58 | fsorted = sorted(image_list[1:],key=lambda x: int(os.path.splitext(x)[0].rsplit('_', 1)[1])) 59 | fsorted.append(image_list[0]) 60 | 61 | frames = [Image.open(os.path.join(frames_dir, im)).convert('RGBA') for im in fsorted] 62 | w, h = common_size(frames, 0), common_size(frames, 1) 63 | frames = [im for im in frames if list(im.size)[0] == w and list(im.size)[1] == h] 64 | frames[0].save(os.path.join(frames_dir, base_name+'.gif'), format='GIF', append_images=frames[1:], save_all=True, duration=duration, loop=0) 65 | -------------------------------------------------------------------------------- /neural_dream/dream_model.py: -------------------------------------------------------------------------------- 1 | import copy 2 | import torch 3 | import torch.nn as nn 4 | import neural_dream.dream_utils as dream_utils 5 | import neural_dream.loss_layers as dream_loss_layers 6 | from neural_dream.loss_layers import DreamLoss, DreamLossHook, DreamLossPreHook 7 | from neural_dream.CaffeLoader import Flatten 8 | 9 | 10 | # Register DreamLoss layers as hooks on model 11 | def add_to_incept(net, layer_name, layer_num, loss_module, capture='after'): 12 | if len(layer_name) == 1: 13 | if capture == 'after': 14 | getattr(net, layer_name[0]).register_forward_hook(loss_module) 15 | elif capture == 'before': 16 | getattr(net, layer_name[0]).register_forward_pre_hook(loss_module) 17 | elif len(layer_name) == 2: 18 | if isinstance(getattr(getattr(net, layer_name[0]), layer_name[1]), nn.Sequential): 19 | if capture == 'after': 20 | getattr(getattr(getattr(net, layer_name[0]), layer_name[1]), str(layer_num)).register_forward_hook(loss_module) 21 | elif capture == 'before': 22 | getattr(getattr(getattr(net, layer_name[0]), layer_name[1]), str(layer_num)).register_forward_pre_hook(loss_module) 23 | layer_num = layer_num+1 if layer_num < 1 else 0 24 | else: 25 | if capture == 'after': 26 | getattr(getattr(net, layer_name[0]), layer_name[1]).register_forward_hook(loss_module) 27 | elif capture == 'before': 28 | getattr(getattr(net, layer_name[0]), layer_name[1]).register_forward_pre_hook(loss_module) 29 | elif len(layer_name) == 3: 30 | if isinstance(getattr(getattr(net, layer_name[0]), layer_name[1]), nn.Sequential): 31 | if capture == 'after': 32 | getattr(getattr(getattr(getattr(net, layer_name[0]), layer_name[1]), str(layer_num)), layer_name[2]).register_forward_hook(loss_module) 33 | elif capture == 'before': 34 | getattr(getattr(getattr(getattr(net, layer_name[0]), layer_name[1]), str(layer_num)), layer_name[2]).register_forward_pre_hook(loss_module) 35 | layer_num = layer_num+1 if layer_num < 1 else 0 36 | else: 37 | if capture == 'after': 38 | getattr(getattr(getattr(net, layer_name[0]), layer_name[1]), layer_name[2]).register_forward_hook(loss_module) 39 | elif capture == 'before': 40 | getattr(getattr(getattr(net, layer_name[0]), layer_name[1]), layer_name[2]).register_forward_pre_hook(loss_module) 41 | return loss_module, layer_num 42 | 43 | 44 | # Create DeepDream model 45 | def build_net(cnn, dream_layers, has_inception, layerList, use_classify, start_params, primary_params, secondary_params): 46 | cnn = copy.deepcopy(cnn) 47 | dream_losses, tv_losses, l2_losses = [], [], [] 48 | 49 | lm_layer_names, loss_module_list = [], [] 50 | dtype = start_params[0] 51 | if not has_inception: 52 | next_dream_idx = 1 53 | net_base = nn.Sequential() 54 | c, r, p, l, d = 0, 0, 0, 0, 0 55 | net_base, tv_losses, l2_losses = start_network(*start_params) 56 | 57 | for i, layer in enumerate(list(cnn), 1): 58 | if next_dream_idx <= len(dream_layers): 59 | if isinstance(layer, nn.Conv2d): 60 | net_base.add_module(str(len(net_base)), layer) 61 | 62 | if layerList['C'][c] in dream_layers: 63 | print("Setting up dream layer " + str(i) + ": " + str(layerList['C'][c])) 64 | loss_module = DreamLoss(*primary_params, **secondary_params) 65 | net_base.add_module(str(len(net_base)), loss_module) 66 | dream_losses.append(loss_module) 67 | c+=1 68 | 69 | if isinstance(layer, nn.ReLU): 70 | net_base.add_module(str(len(net_base)), layer) 71 | 72 | if layerList['R'][r] in dream_layers: 73 | print("Setting up dream layer " + str(i) + ": " + str(layerList['R'][r])) 74 | loss_module = DreamLoss(*primary_params, **secondary_params) 75 | net_base.add_module(str(len(net_base)), loss_module) 76 | dream_losses.append(loss_module) 77 | next_dream_idx += 1 78 | r+=1 79 | 80 | if isinstance(layer, nn.MaxPool2d) or isinstance(layer, nn.AvgPool2d): 81 | net_base.add_module(str(len(net_base)), layer) 82 | 83 | if layerList['P'][p] in dream_layers: 84 | print("Setting up dream layer " + str(i) + ": " + str(layerList['P'][p])) 85 | loss_module = DreamLoss(*primary_params, **secondary_params) 86 | net_base.add_module(str(len(net_base)), loss_module) 87 | dream_losses.append(loss_module) 88 | next_dream_idx += 1 89 | p+=1 90 | 91 | if isinstance(layer, nn.AdaptiveAvgPool2d) or isinstance(layer, nn.AdaptiveMaxPool2d): 92 | net_base.add_module(str(len(net_base)), layer) 93 | 94 | if isinstance(layer, Flatten): 95 | flatten_mod = Flatten().type(dtype) 96 | net_base.add_module(str(len(net_base)), flatten_mod) 97 | 98 | if isinstance(layer, nn.Linear): 99 | net_base.add_module(str(len(net_base)), layer) 100 | 101 | if layerList['L'][l] in dream_layers: 102 | print("Setting up dream layer " + str(i) + ": " + str(layerList['L'][l])) 103 | loss_module = DreamLoss(*primary_params, **secondary_params) 104 | net_base.add_module(str(len(net_base)), loss_module) 105 | dream_losses.append(loss_module) 106 | next_dream_idx += 1 107 | l+=1 108 | 109 | if isinstance(layer, nn.Dropout): 110 | net_base.add_module(str(len(net_base)), layer) 111 | 112 | if layerList['D'][d] in dream_layers: 113 | print("Setting up dream layer " + str(i) + ": " + str(layerList['D'][d])) 114 | loss_module = DreamLoss(*primary_params, **secondary_params) 115 | net_base.add_module(str(len(net_base)), loss_module) 116 | dream_losses.append(loss_module) 117 | next_dream_idx += 1 118 | d+=1 119 | 120 | if use_classify > 0 and l == len(layerList['L']): 121 | next_dream_idx += 1 122 | 123 | elif has_inception: 124 | start_net, tv_losses, l2_losses = start_network(start_params) 125 | lm_layer_names, loss_module_list = [], [] 126 | net_base = copy.deepcopy(cnn) 127 | sn=0 128 | for i, n in enumerate(dream_layers): 129 | print("Setting up dream layer " + str(i+1) + ": " + n) 130 | if 'before_' in n: 131 | n = n.split('before_')[1] 132 | loss_module = DreamLossPreHook(*primary_params, **secondary_params) 133 | module_loc = 'before' 134 | else: 135 | loss_module = DreamLossHook(*primary_params, **secondary_params) 136 | module_loc = 'after' 137 | n = n.split('/') 138 | lm_layer_names.append(n) 139 | loss_module, sn = add_to_incept(net_base, n, sn, loss_module, module_loc) 140 | loss_module_list.append(loss_module) 141 | dream_losses.append(loss_module) 142 | if len(start_net) > 0: 143 | net_base = dream_utils.ModelPlus(start_net, net_base) 144 | return net_base, dream_losses, tv_losses, l2_losses, lm_layer_names, loss_module_list 145 | 146 | 147 | # Create preprocessing model / start of VGG model 148 | def start_network(dtype, random_transforms='none', jitter_val=32, tv_weight=0, l2_weight=0, layer_sigma=0): 149 | tv_losses, l2_losses = [], [] 150 | start_net = nn.Sequential() 151 | if random_transforms != 'none': 152 | rt_mod = dream_utils.RandomTransform(random_transforms).type(dtype) 153 | start_net.add_module(str(len(start_net)), rt_mod) 154 | if jitter_val > 0: 155 | jitter_mod = dream_utils.Jitter(jitter_val).type(dtype) 156 | start_net.add_module(str(len(start_net)), jitter_mod) 157 | if tv_weight > 0: 158 | tv_mod = dream_loss_layers.TVLoss(tv_weight).type(dtype) 159 | start_net.add_module(str(len(start_net)), tv_mod) 160 | tv_losses.append(tv_mod) 161 | if l2_weight > 0: 162 | l2_mod = dream_loss_layers.L2Regularizer(l2_weight).type(dtype) 163 | start_net.add_module(str(len(start_net)), l2_mod) 164 | l2_losses.append(l2_mod) 165 | if layer_sigma > 0: 166 | gauss_mod = dream_utils.GaussianBlurLayer(5, layer_sigma).type(dtype) 167 | start_net.add_module(str(len(start_net)), gauss_mod) 168 | return start_net, tv_losses, l2_losses 169 | 170 | 171 | # Reapply DreamLoss layer hooks 172 | def renew_net(start_params, net, loss_module_list, dream_layers): 173 | start_net, tv_losses, l2_losses = start_network(*start_params) 174 | if isinstance(net, dream_utils.ModelPlus): 175 | net = net.net 176 | new_dream_losses = [] 177 | sn=0 178 | for i, layer in enumerate(dream_layers): 179 | n = layer 180 | loss_module = loss_module_list[i] 181 | if str(loss_module).split('(')[0] == 'DreamLossPreHook': 182 | module_loc = 'before' 183 | else: 184 | module_loc = 'after' 185 | loss_module, sn = add_to_incept(net, n, sn, loss_module, module_loc) 186 | new_dream_losses.append(loss_module_list[i]) 187 | if len(start_net) > 0: 188 | net = dream_utils.ModelPlus(start_net, net) 189 | return net, new_dream_losses, tv_losses, l2_losses -------------------------------------------------------------------------------- /neural_dream/dream_tile.py: -------------------------------------------------------------------------------- 1 | import torch 2 | 3 | 4 | # Apply blend masks to tiles 5 | def mask_tile(tile, overlap, side='bottom'): 6 | h, w = tile.size(2), tile.size(3) 7 | top_overlap, bottom_overlap, right_overlap, left_overlap = overlap[0], overlap[1], overlap[2], overlap[3] 8 | if tile.is_cuda: 9 | if 'left' in side and 'left-special' not in side: 10 | lin_mask_left = torch.linspace(0,1,left_overlap, device=tile.get_device()).repeat(h,1).repeat(3,1,1).unsqueeze(0) 11 | if 'right' in side and 'right-special' not in side: 12 | lin_mask_right = torch.linspace(1,0,right_overlap, device=tile.get_device()).repeat(h,1).repeat(3,1,1).unsqueeze(0) 13 | if 'top' in side and 'top-special' not in side: 14 | lin_mask_top = torch.linspace(0,1,top_overlap, device=tile.get_device()).repeat(w,1).rot90(3).repeat(3,1,1).unsqueeze(0) 15 | if 'bottom' in side and 'bottom-special' not in side: 16 | lin_mask_bottom = torch.linspace(1,0,bottom_overlap, device=tile.get_device()).repeat(w,1).rot90(3).repeat(3,1,1).unsqueeze(0) 17 | 18 | if 'left-special' in side: 19 | lin_mask_left = torch.linspace(0,1,left_overlap, device=tile.get_device()) 20 | zeros_mask = torch.zeros(w-(left_overlap*2), device=tile.get_device()) 21 | ones_mask = torch.ones(left_overlap, device=tile.get_device()) 22 | lin_mask_left = torch.cat([zeros_mask, lin_mask_left, ones_mask], 0).repeat(h,1).repeat(3,1,1).unsqueeze(0) 23 | if 'right-special' in side: 24 | lin_mask_right = torch.linspace(1,0,right_overlap, device=tile.get_device()) 25 | ones_mask = torch.ones(w-right_overlap, device=tile.get_device()) 26 | lin_mask_right = torch.cat([ones_mask, lin_mask_right], 0).repeat(h,1).repeat(3,1,1).unsqueeze(0) 27 | if 'top-special' in side: 28 | lin_mask_top = torch.linspace(0,1,top_overlap, device=tile.get_device()) 29 | zeros_mask = torch.zeros(h-(top_overlap*2), device=tile.get_device()) 30 | ones_mask = torch.ones(top_overlap, device=tile.get_device()) 31 | lin_mask_top = torch.cat([zeros_mask, lin_mask_top, ones_mask], 0).repeat(w,1).rot90(3).repeat(3,1,1).unsqueeze(0) 32 | if 'bottom-special' in side: 33 | lin_mask_bottom = torch.linspace(1,0,bottom_overlap, device=tile.get_device()) 34 | ones_mask = torch.ones(h-bottom_overlap, device=tile.get_device()) 35 | lin_mask_bottom = torch.cat([ones_mask, lin_mask_bottom], 0).repeat(w,1).rot90(3).repeat(3,1,1).unsqueeze(0) 36 | else: 37 | if 'left' in side and 'left-special' not in side: 38 | lin_mask_left = torch.linspace(0,1,left_overlap).repeat(h,1).repeat(3,1,1).unsqueeze(0) 39 | if 'right' in side and 'right-special' not in side: 40 | lin_mask_right = torch.linspace(1,0,right_overlap).repeat(h,1).repeat(3,1,1).unsqueeze(0) 41 | if 'top' in side and 'top-special' not in side: 42 | lin_mask_top = torch.linspace(0,1,top_overlap).repeat(w,1).rot90(3).repeat(3,1,1).unsqueeze(0) 43 | if 'bottom' in side and 'bottom-special' not in side: 44 | lin_mask_bottom = torch.linspace(1,0,bottom_overlap).repeat(w,1).rot90(3).repeat(3,1,1).unsqueeze(0) 45 | 46 | if 'left-special' in side: 47 | lin_mask_left = torch.linspace(0,1,left_overlap) 48 | zeros_mask = torch.zeros(w-(left_overlap*2)) 49 | ones_mask = torch.ones(left_overlap) 50 | lin_mask_left = torch.cat([zeros_mask, lin_mask_left, ones_mask], 0).repeat(h,1).repeat(3,1,1).unsqueeze(0) 51 | if 'right-special' in side: 52 | lin_mask_right = torch.linspace(1,0,right_overlap) 53 | ones_mask = torch.ones(w-right_overlap) 54 | lin_mask_right = torch.cat([ones_mask, lin_mask_right], 0).repeat(h,1).repeat(3,1,1).unsqueeze(0) 55 | if 'top-special' in side: 56 | lin_mask_top = torch.linspace(0,1,top_overlap) 57 | zeros_mask = torch.zeros(h-(top_overlap*2)) 58 | ones_mask = torch.ones(top_overlap) 59 | lin_mask_top = torch.cat([zeros_mask, lin_mask_top, ones_mask], 0).repeat(w,1).rot90(3).repeat(3,1,1).unsqueeze(0) 60 | if 'bottom-special' in side: 61 | lin_mask_bottom = torch.linspace(1,0,bottom_overlap) 62 | ones_mask = torch.ones(h-bottom_overlap) 63 | lin_mask_bottom = torch.cat([ones_mask, lin_mask_bottom], 0).repeat(w,1).rot90(3).repeat(3,1,1).unsqueeze(0) 64 | 65 | base_mask = torch.ones_like(tile) 66 | 67 | if 'right' in side and 'right-special' not in side: 68 | base_mask[:,:,:,w-right_overlap:] = base_mask[:,:,:,w-right_overlap:] * lin_mask_right 69 | if 'left' in side and 'left-special' not in side: 70 | base_mask[:,:,:,:left_overlap] = base_mask[:,:,:,:left_overlap] * lin_mask_left 71 | if 'bottom' in side and 'bottom-special' not in side: 72 | base_mask[:,:,h-bottom_overlap:,:] = base_mask[:,:,h-bottom_overlap:,:] * lin_mask_bottom 73 | if 'top' in side and 'top-special' not in side: 74 | base_mask[:,:,:top_overlap,:] = base_mask[:,:,:top_overlap,:] * lin_mask_top 75 | 76 | if 'right-special' in side: 77 | base_mask = base_mask * lin_mask_right 78 | if 'left-special' in side: 79 | base_mask = base_mask * lin_mask_left 80 | if 'bottom-special' in side: 81 | base_mask = base_mask * lin_mask_bottom 82 | if 'top-special' in side: 83 | base_mask = base_mask * lin_mask_top 84 | return tile * base_mask 85 | 86 | 87 | def get_tile_coords(d, tile_dim, overlap=0): 88 | move = int(tile_dim * (1-overlap)) 89 | c, tile_start, coords = 1, 0, [0] 90 | while tile_start + tile_dim < d: 91 | tile_start = move * c 92 | if tile_start + tile_dim >= d: 93 | coords.append(d - tile_dim) 94 | else: 95 | coords.append(tile_start) 96 | c += 1 97 | return coords 98 | 99 | 100 | def get_tiles(img, tile_coords, tile_size, info_only=False): 101 | tile_list = [] 102 | for y in tile_coords[0]: 103 | for x in tile_coords[1]: 104 | tile = img[:, :, y:y+tile_size[0], x:x+tile_size[1]] 105 | tile_list.append(tile) 106 | if not info_only: 107 | return tile_list 108 | else: 109 | return tile_list[0].size(2), tile_list[0].size(3) 110 | 111 | 112 | def final_overlap(tile_coords): 113 | r, c = len(tile_coords[0]), len(tile_coords[1]) 114 | return (tile_coords[0][r-1] - tile_coords[0][r-2], tile_coords[1][c-1] - tile_coords[1][c-2]) 115 | 116 | 117 | def add_tiles(tiles, base_img, tile_coords, tile_size, overlap): 118 | f_ovlp = final_overlap(tile_coords) 119 | h, w = tiles[0].size(2), tiles[0].size(3) 120 | t=0 121 | column, row, = 0, 0 122 | for y in tile_coords[0]: 123 | for x in tile_coords[1]: 124 | mask_sides='' 125 | c_overlap = overlap.copy() 126 | if row == 0: 127 | if row == len(tile_coords[0]) - 2: 128 | mask_sides += 'bottom-special' 129 | c_overlap[1] = f_ovlp[0] # Change bottom overlap 130 | else: 131 | mask_sides += 'bottom' 132 | elif row > 0 and row < len(tile_coords[0]) -2: 133 | mask_sides += 'bottom,top' 134 | elif row == len(tile_coords[0]) - 2: 135 | if f_ovlp[0] > 0: 136 | mask_sides += 'bottom-special,top' 137 | c_overlap[1] = f_ovlp[0] # Change bottom overlap 138 | elif f_ovlp[0] <= 0: 139 | mask_sides += 'bottom,top' 140 | elif row == len(tile_coords[0]) -1: 141 | if f_ovlp[0] > 0: 142 | mask_sides += 'top-special' 143 | c_overlap[0] = f_ovlp[0] # Change top overlap 144 | elif f_ovlp[0] <= 0: 145 | mask_sides += 'top' 146 | 147 | if column == 0: 148 | if column == len(tile_coords[1]) -2: 149 | mask_sides += ',right-special' 150 | c_overlap[2] = f_ovlp[1] # Change right overlap 151 | else: 152 | mask_sides += ',right' 153 | elif column > 0 and column < len(tile_coords[1]) -2: 154 | mask_sides += ',right,left' 155 | elif column == len(tile_coords[1]) -2: 156 | if f_ovlp[1] > 0: 157 | mask_sides += ',right-special,left' 158 | c_overlap[2] = f_ovlp[1] # Change right overlap 159 | elif f_ovlp[1] <= 0: 160 | mask_sides += ',right,left' 161 | elif column == len(tile_coords[1]) -1: 162 | if f_ovlp[1] > 0: 163 | mask_sides += ',left-special' 164 | c_overlap[3] = f_ovlp[1] # Change left overlap 165 | elif f_ovlp[1] <= 0: 166 | mask_sides += ',left' 167 | 168 | tile = mask_tile(tiles[t], c_overlap, side=mask_sides) 169 | base_img[:, :, y:y+tile_size[0], x:x+tile_size[1]] = base_img[:, :, y:y+tile_size[0], x:x+tile_size[1]] + tile 170 | t+=1 171 | column+=1 172 | row+=1 173 | column=0 174 | return base_img 175 | 176 | 177 | def tile_setup(tile_size, overlap_percent, base_size): 178 | if type(tile_size) is not tuple and type(tile_size) is not list: 179 | tile_size = (tile_size, tile_size) 180 | if type(overlap_percent) is not tuple and type(overlap_percent) is not list: 181 | overlap_percent = (overlap_percent, overlap_percent) 182 | x_coords = get_tile_coords(base_size[1], tile_size[1], overlap_percent[1]) 183 | y_coords = get_tile_coords(base_size[0], tile_size[0], overlap_percent[0]) 184 | y_ovlp, x_ovlp = int(tile_size[0] * overlap_percent[0]), int(tile_size[1] * overlap_percent[1]) 185 | return (y_coords, x_coords), tile_size, [y_ovlp, y_ovlp, x_ovlp, x_ovlp] 186 | 187 | 188 | def tile_image(img, tile_size, overlap_percent, info_only=False): 189 | tile_coords, tile_size, _ = tile_setup(tile_size, overlap_percent, (img.size(2), img.size(3))) 190 | if not info_only: 191 | return get_tiles(img, tile_coords, tile_size) 192 | else: 193 | tile_size = get_tiles(img, tile_coords, tile_size, info_only) 194 | return tile_size[0], tile_size[1], (len(tile_coords[0]), len(tile_coords[1])), (len(tile_coords[0]) * len(tile_coords[1])) 195 | 196 | 197 | def rebuild_image(tiles, base_img, tile_size, overlap_percent): 198 | base_img = torch.zeros_like(base_img) 199 | tile_coords, tile_size, overlap = tile_setup(tile_size, overlap_percent, (base_img.size(2), base_img.size(3))) 200 | return add_tiles(tiles, base_img, tile_coords, tile_size, overlap) 201 | -------------------------------------------------------------------------------- /neural_dream/dream_utils.py: -------------------------------------------------------------------------------- 1 | import math 2 | import random 3 | import torch 4 | import torch.nn as nn 5 | 6 | 7 | # Get tensor percentile 8 | def tensor_percentile(t, p=99.98): 9 | return t.view(-1).kthvalue(1 + round(0.01 * float(p) * (t.numel() - 1))).values.item() 10 | 11 | 12 | # Shift tensor, possibly randomly 13 | def roll_tensor(tensor, h_shift=None, w_shift=None): 14 | if h_shift == None: 15 | h_shift = torch.LongTensor(10).random_(-tensor.size(1), tensor.size(1))[0].item() 16 | if w_shift == None: 17 | w_shift = torch.LongTensor(10).random_(-tensor.size(2), tensor.size(2))[0].item() 18 | tensor = torch.roll(torch.roll(tensor, shifts=h_shift, dims=2), shifts=w_shift, dims=3) 19 | return tensor, h_shift, w_shift 20 | 21 | 22 | # Define an nn Module to perform gaussian blurring 23 | class GaussianBlur(nn.Module): 24 | def __init__(self, k_size, sigma): 25 | super(GaussianBlur, self).__init__() 26 | self.k_size = k_size 27 | self.sigma = sigma 28 | 29 | def capture(self, input): 30 | if input.dim() == 4: 31 | d_val = 2 32 | self.groups = input.size(1) 33 | elif input.dim() == 2: 34 | d_val = 1 35 | self.groups = input.size(0) 36 | 37 | self.k_size, self.sigma = [self.k_size] * d_val, [self.sigma] * d_val 38 | kernel = 1 39 | 40 | meshgrid_tensor = torch.meshgrid([torch.arange(size, dtype=torch.float32, \ 41 | device=input.get_device()) for size in self.k_size]) 42 | 43 | for size, std, mgrid in zip(self.k_size, self.sigma, meshgrid_tensor): 44 | kernel *= 1 / (std * math.sqrt(2 * math.pi)) * \ 45 | torch.exp(-((mgrid - ((size - 1) / 2)) / std) ** 2 / 2) 46 | 47 | kernel = (kernel / torch.sum(kernel)).view(1, 1, * kernel.size()) 48 | kernel = kernel.repeat(self.groups, * [1] * (kernel.dim() - 1)) 49 | self.register_buffer('weight', kernel) 50 | 51 | if d_val == 2: 52 | self.conv = torch.nn.functional.conv2d 53 | elif d_val == 1: 54 | self.conv = torch.nn.functional.conv1d 55 | 56 | def forward(self, input, pad_mode='reflect'): 57 | d_val = input.dim() 58 | if input.dim() > 2: 59 | input = torch.nn.functional.pad(input, (2, 2, 2, 2), mode=pad_mode) 60 | else: 61 | input = input.view(1, 1, input.size(1)) 62 | input = self.conv(input, weight=self.weight, groups=self.groups) 63 | if d_val == 2: 64 | p1d = nn.ConstantPad1d(2, 0) 65 | input = p1d(input) 66 | input = input.view(1, input.size(2)) 67 | return input 68 | 69 | 70 | # Define a Module to create gaussian blur for laplacian pyramids 71 | class GaussianBlurLP(GaussianBlur): 72 | 73 | def __init__(self, input, k_size=5, sigma=0): 74 | super(GaussianBlur, self).__init__() 75 | self.gauss_blur = GaussianBlur(k_size, sigma) 76 | self.gauss_blur.capture(input) 77 | 78 | def forward(self, input): 79 | return self.gauss_blur(input) 80 | 81 | 82 | # Define an nn Module to apply gaussian blur as a layer 83 | class GaussianBlurLayer(nn.Module): 84 | 85 | def __init__(self, k_size=5, sigma=0): 86 | super(GaussianBlurLayer, self).__init__() 87 | self.blur = GaussianBlur(k_size, sigma) 88 | self.mode = 'None' 89 | 90 | def forward(self, input): 91 | if self.mode == 'loss': 92 | input = self.blur(input) 93 | if self.mode == 'None': 94 | self.mode = 'capture' 95 | if self.mode == 'capture': 96 | self.blur.capture(input.clone()) 97 | self.mode = 'loss' 98 | return input 99 | 100 | 101 | # Define an nn Module to create a laplacian pyramid 102 | class LaplacianPyramid(nn.Module): 103 | 104 | def __init__(self, input, scale=4, sigma=1): 105 | super(LaplacianPyramid, self).__init__() 106 | if len(sigma) == 1: 107 | sigma = (float(sigma[0]), float(sigma[0]) * scale) 108 | else: 109 | sigma = [float(s) for s in sigma] 110 | self.gauss_blur = GaussianBlurLP(input, 5, sigma[0]) 111 | self.gauss_blur_hi = GaussianBlurLP(input, 5, sigma[1]) 112 | self.scale = scale 113 | 114 | def split_lap(self, input): 115 | g = self.gauss_blur(input) 116 | gt = self.gauss_blur_hi(input) 117 | return g, input - gt 118 | 119 | def pyramid_list(self, input): 120 | pyramid_levels = [] 121 | for i in range(self.scale): 122 | input, hi = self.split_lap(input) 123 | pyramid_levels.append(hi) 124 | pyramid_levels.append(input) 125 | return pyramid_levels[::-1] 126 | 127 | def lap_merge(self, pyramid_levels): 128 | b = torch.zeros_like(pyramid_levels[0]) 129 | for p in pyramid_levels: 130 | b = b + p 131 | return b 132 | 133 | def forward(self, input): 134 | return self.lap_merge(self.pyramid_list(input)) 135 | 136 | 137 | # Define an nn Module to rank channels based on activation strength 138 | class RankChannels(torch.nn.Module): 139 | 140 | def __init__(self, channels=1, channel_mode='strong'): 141 | super(RankChannels, self).__init__() 142 | self.channels = channels 143 | self.channel_mode = channel_mode 144 | 145 | def sort_channels(self, input): 146 | channel_list = [] 147 | for i in range(input.size(1)): 148 | channel_list.append(torch.mean(input.clone().squeeze(0).narrow(0,i,1)).item()) 149 | return sorted((c,v) for v,c in enumerate(channel_list)) 150 | 151 | def get_middle(self, sequence): 152 | num = self.channels[0] 153 | m = (len(sequence) - 1)//2 - num//2 154 | return sequence[m:m+num] 155 | 156 | def remove_channels(self, cl): 157 | return [c for c in cl if c[1] not in self.channels] 158 | 159 | def rank_channel_list(self, input): 160 | top_channels = self.channels[0] 161 | channel_list = self.sort_channels(input) 162 | 163 | if 'strong' in self.channel_mode: 164 | channel_list.reverse() 165 | elif 'avg' in self.channel_mode: 166 | channel_list = self.get_middle(channel_list) 167 | elif 'ignore' in self.channel_mode: 168 | channel_list = self.remove_channels(channel_list) 169 | top_channels = len(channel_list) 170 | 171 | channels = [] 172 | for i in range(top_channels): 173 | channels.append(channel_list[i][1]) 174 | return channels 175 | 176 | def forward(self, input): 177 | return self.rank_channel_list(input) 178 | 179 | 180 | class FFTTensor(nn.Module): 181 | 182 | def __init__(self, r=1, bl=25): 183 | super(FFTTensor, self).__init__() 184 | self.r = r 185 | self.bl = bl 186 | 187 | def block_low(self, input): 188 | if input.dim() == 5: 189 | hh, hw = int(input.size(2)/2), int(input.size(3)/2) 190 | input[:, :, hh-self.bl:hh+self.bl+1, hw-self.bl:hw+self.bl+1, :] = self.r 191 | elif input.dim() == 3: 192 | m = (input.size(1) - 1)//2 - self.bl//2 193 | input[:, m:m+self.bl, :] = self.r 194 | return input 195 | 196 | def fft_image(self, input, s=0): 197 | s_dim = 3 if input.dim() == 4 else 1 198 | s_dim = s_dim if s == 0 else s 199 | input = torch.rfft(input, signal_ndim=s_dim, onesided=False) 200 | real, imaginary = torch.unbind(input, -1) 201 | for r_dim in range(1, len(real.size())): 202 | n_shift = real.size(r_dim)//2 203 | if real.size(r_dim) % 2 != 0: 204 | n_shift += 1 205 | real = torch.roll(real, n_shift, dims=r_dim) 206 | imaginary = torch.roll(imaginary, n_shift, dims=r_dim) 207 | return torch.stack((real, imaginary), -1) 208 | 209 | def ifft_image(self, input, s=0): 210 | s_dim = 3 if input.dim() == 5 else 1 211 | s_dim = s_dim if s == 0 else s 212 | real, imaginary = torch.unbind(input, -1) 213 | for r_dim in range(len(real.size()) - 1, 0, -1): 214 | real = torch.roll(real, real.size(r_dim)//2, dims=r_dim) 215 | imaginary = torch.roll(imaginary, imaginary.size(r_dim)//2, dims=r_dim) 216 | return torch.irfft(torch.stack((real, imaginary), -1), signal_ndim=s_dim, onesided=False) 217 | 218 | def forward(self, input): 219 | input = self.block_low(self.fft_image(input)) 220 | return torch.abs(self.ifft_image(input)) 221 | 222 | 223 | # Define an nn Module to apply jitter 224 | class Jitter(torch.nn.Module): 225 | 226 | def __init__(self, jitter_val): 227 | super(Jitter, self).__init__() 228 | self.jitter_val = jitter_val 229 | 230 | def roll_tensor(self, input): 231 | h_shift = random.randint(-self.jitter_val, self.jitter_val) 232 | w_shift = random.randint(-self.jitter_val, self.jitter_val) 233 | return torch.roll(torch.roll(input, shifts=h_shift, dims=2), shifts=w_shift, dims=3) 234 | 235 | def forward(self, input): 236 | return self.roll_tensor(input) 237 | 238 | 239 | # Define an nn Module to apply random transforms 240 | class RandomTransform(torch.nn.Module): 241 | 242 | def __init__(self, t_val): 243 | super(RandomTransform, self).__init__() 244 | self.rotate, self.flip = False, False 245 | if t_val == 'all' or t_val == 'rotate': 246 | self.rotate = True 247 | if t_val == 'all' or t_val == 'flip': 248 | self.flip = True 249 | 250 | def rotate_tensor(self, input): 251 | if self.rotate: 252 | k_val = random.randint(0,3) 253 | input = torch.rot90(input, k_val, [2,3]) 254 | return input 255 | 256 | def flip_tensor(self, input): 257 | if self.flip: 258 | flip_tensor = bool(random.randint(0,1)) 259 | if flip_tensor: 260 | input = input.flip([2,3]) 261 | return input 262 | 263 | def forward(self, input): 264 | return self.flip_tensor(self.rotate_tensor(input)) 265 | 266 | 267 | # Define an nn Module to label predicted channels 268 | class Classify(nn.Module): 269 | 270 | def __init__(self, labels, k=1): 271 | super(Classify, self).__init__() 272 | self.labels = [str(n) for n in labels] 273 | self.k = k 274 | 275 | def forward(self, input): 276 | channel_ids = torch.topk(input, self.k).indices 277 | channel_ids = [n.item() for n in channel_ids[0]] 278 | label_names = '' 279 | for i in channel_ids: 280 | if label_names != '': 281 | label_names += ', ' + self.labels[i] 282 | else: 283 | label_names += self.labels[i] 284 | print(' Predicted labels: ' + label_names) 285 | 286 | 287 | # Run inception modules with preprocessing layers 288 | class ModelPlus(nn.Module): 289 | 290 | def __init__(self, input_net, net): 291 | super(ModelPlus, self).__init__() 292 | self.input_net = input_net 293 | self.net = net 294 | 295 | def forward(self, input): 296 | return self.net(self.input_net(input)) 297 | -------------------------------------------------------------------------------- /neural_dream/helper_layers.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torch.nn as nn 3 | import torch.nn.functional as F 4 | 5 | 6 | class AdditionLayer(nn.Module): 7 | def forward(self, input, input2): 8 | return input + input2 9 | 10 | 11 | class MaxPool2dLayer(nn.Module): 12 | def forward(self, input, kernel_size=(3, 3), stride=(1, 1), padding=0, ceil_mode=False): 13 | return F.max_pool2d(input, kernel_size, stride=stride, padding=padding, ceil_mode=ceil_mode) 14 | 15 | 16 | class PadLayer(nn.Module): 17 | def forward(self, input, padding=(1, 1, 1, 1), value=None): 18 | if value == None: 19 | return F.pad(input, padding) 20 | else: 21 | return F.pad(input, padding, value=value) 22 | 23 | 24 | class ReluLayer(nn.Module): 25 | def forward(self, input): 26 | return F.relu(input) 27 | 28 | 29 | class SoftMaxLayer(nn.Module): 30 | def forward(self, input, dim=1): 31 | return F.softmax(input, dim=dim) 32 | 33 | 34 | class DropoutLayer(nn.Module): 35 | def forward(self, input, p=0.4000000059604645, training=False, inplace=True): 36 | return F.dropout(input = input, p = p, training = training, inplace = inplace) 37 | 38 | 39 | class CatLayer(nn.Module): 40 | def forward(self, input_list, dim=1): 41 | return torch.cat(input_list, dim) 42 | 43 | 44 | class LocalResponseNormLayer(nn.Module): 45 | def forward(self, input, size=5, alpha=9.999999747378752e-05, beta=0.75, k=1.0): 46 | return F.local_response_norm(input, size=size, alpha=alpha, beta=beta, k=k) 47 | 48 | 49 | class AVGPoolLayer(nn.Module): 50 | def forward(self, input, kernel_size=(7, 7), stride=(1, 1), padding=(0,), ceil_mode=False, count_include_pad=False): 51 | return F.avg_pool2d(input, kernel_size=kernel_size, stride=stride, padding=padding, ceil_mode=ceil_mode, count_include_pad=count_include_pad) -------------------------------------------------------------------------------- /neural_dream/loss_layers.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torch.nn as nn 3 | import neural_dream.dream_utils as dream_utils 4 | from neural_dream.dream_experimental import ChannelMod 5 | 6 | 7 | 8 | # Define an nn Module to compute DeepDream loss in different ways 9 | class DreamLossMode(torch.nn.Module): 10 | 11 | def __init__(self, loss_mode, use_fft, r): 12 | super(DreamLossMode, self).__init__() 13 | self.get_mode(loss_mode) 14 | self.use_fft = use_fft[0] 15 | self.fft_tensor = dream_utils.FFTTensor(r, use_fft[1]) 16 | 17 | def get_mode(self, loss_mode): 18 | self.loss_mode_string = loss_mode 19 | if loss_mode.lower() == 'norm': 20 | self.get_loss = self.norm_loss 21 | elif loss_mode.lower() == 'mean': 22 | self.get_loss = self.mean_loss 23 | elif loss_mode.lower() == 'l2': 24 | self.get_loss = self.l2_loss 25 | elif loss_mode.lower() == 'mse': 26 | self.crit = torch.nn.MSELoss() 27 | self.get_loss = self.crit_loss 28 | elif loss_mode.lower() == 'bce': 29 | self.crit = torch.nn.BCEWithLogitsLoss() 30 | self.get_loss = self.crit_loss 31 | elif loss_mode.lower() == 'abs_mean': 32 | self.get_loss = self.abs_mean 33 | elif loss_mode.lower() == 'abs_l2': 34 | self.get_loss = self.abs_l2_loss 35 | 36 | def norm_loss(self, input): 37 | return input.norm() 38 | 39 | def mean_loss(self, input): 40 | return input.mean() 41 | 42 | def l2_loss(self, input): 43 | return input.pow(2).sum().sqrt() 44 | 45 | def abs_mean(self, input): 46 | return input.abs().mean() 47 | 48 | def abs_l2_loss(self, input): 49 | return input.abs().pow(2).sum().sqrt() 50 | 51 | def crit_loss(self, input, target): 52 | return self.crit(input, target) 53 | 54 | def forward(self, input): 55 | if self.use_fft: 56 | input = self.fft_tensor(input) 57 | if self.loss_mode_string != 'bce' and self.loss_mode_string != 'mse': 58 | loss = self.get_loss(input) 59 | else: 60 | target = torch.zeros_like(input.detach()) 61 | loss = self.crit_loss(input, target) 62 | return loss 63 | 64 | 65 | # Define an nn Module for DeepDream 66 | class DeepDream(torch.nn.Module): 67 | 68 | def __init__(self, loss_mode, channels='-1', channel_mode='strong', \ 69 | channel_capture='once', scale=4, sigma=1, use_fft=(True, 25), r=1, p_mode='fast', norm_p=0, abs_p=0, mean_p=0): 70 | super(DeepDream, self).__init__() 71 | self.get_loss = DreamLossMode(loss_mode, use_fft, r) 72 | self.channels = [int(c) for c in channels.split(',')] 73 | self.channel_mode = channel_mode 74 | self.get_channels = dream_utils.RankChannels(self.channels, self.channel_mode) 75 | self.lap_scale = scale 76 | self.sigma = sigma.split(',') 77 | self.channel_capture = channel_capture 78 | 79 | self.zero_weak = ChannelMod(p_mode, self.channels[0], norm_p, abs_p, mean_p) 80 | 81 | 82 | def capture(self, input, lp=True): 83 | if -1 not in self.channels and 'all' not in self.channel_mode: 84 | self.channels = self.get_channels(input) 85 | elif self.channel_mode == 'all' and -1 not in self.channels: 86 | self.channels = self.channels 87 | if self.lap_scale > 0 and lp == True: 88 | self.lap_pyramid = dream_utils.LaplacianPyramid(input.clone(), self.lap_scale, self.sigma) 89 | 90 | def get_channel_loss(self, input): 91 | loss = 0 92 | if 'once' not in self.channel_capture: 93 | self.capture(input, False) 94 | for c in self.channels: 95 | if input.dim() > 0: 96 | if int(c) < input.size(1): 97 | loss += self.get_loss(input[:, int(c)]) 98 | else: 99 | loss += self.get_loss(input) 100 | return loss 101 | 102 | def forward(self, input): 103 | if self.lap_scale > 0: 104 | input = self.lap_pyramid(input) 105 | if self.zero_weak.enabled: 106 | input = self.zero_weak(input) 107 | if -1 in self.channels: 108 | loss = self.get_loss(input) 109 | else: 110 | loss = self.get_channel_loss(input) 111 | return loss 112 | 113 | 114 | 115 | # Define an nn Module to collect DeepDream loss 116 | class DreamLoss(torch.nn.Module): 117 | 118 | def __init__(self, loss_mode, strength, channels, channel_mode='all', **kwargs): 119 | super(DreamLoss, self).__init__() 120 | self.dream = DeepDream(loss_mode, channels, channel_mode, **kwargs) 121 | self.strength = strength 122 | self.mode = 'None' 123 | 124 | def forward(self, input): 125 | if self.mode == 'loss': 126 | self.loss = self.dream(input.clone()) * self.strength 127 | elif self.mode == 'capture': 128 | self.target_size = input.size() 129 | self.dream.capture(input.clone()) 130 | return input 131 | 132 | 133 | # Define a forward hook to collect DeepDream loss 134 | class DreamLossHook(DreamLoss): 135 | 136 | def forward(self, module, input, output): 137 | if self.mode == 'loss': 138 | self.loss = self.dream(output.clone()) * self.strength 139 | elif self.mode == 'capture': 140 | self.target_size = output.size() 141 | self.dream.capture(output.clone()) 142 | 143 | 144 | # Define a pre forward hook to collect DeepDream loss 145 | class DreamLossPreHook(DreamLoss): 146 | 147 | def forward(self, module, output): 148 | if self.mode == 'loss': 149 | self.loss = self.dream(output[0].clone()) * self.strength 150 | elif self.mode == 'capture': 151 | self.target_size = output[0].size() 152 | self.dream.capture(output[0].clone()) 153 | 154 | 155 | # Define an nn Module to compute l2 loss 156 | class L2Regularizer(nn.Module): 157 | 158 | def __init__(self, strength): 159 | super(L2Regularizer, self).__init__() 160 | self.strength = strength 161 | 162 | def forward(self, input): 163 | self.loss = self.strength * (input.clone().norm(3)/2) 164 | return input 165 | 166 | 167 | # Define an nn Module to compute tv loss 168 | class TVLoss(nn.Module): 169 | 170 | def __init__(self, strength): 171 | super(TVLoss, self).__init__() 172 | self.strength = strength 173 | 174 | def forward(self, input): 175 | self.x_diff = input[:,:,1:,:] - input[:,:,:-1,:] 176 | self.y_diff = input[:,:,:,1:] - input[:,:,:,:-1] 177 | self.loss = self.strength * (torch.sum(torch.abs(self.x_diff)) + torch.sum(torch.abs(self.y_diff))) 178 | return input -------------------------------------------------------------------------------- /neural_dream/models/__init__.py: -------------------------------------------------------------------------------- 1 | from neural_dream.models.googlenet.bvlc_googlenet import BVLC_GOOGLENET 2 | from neural_dream.models.googlenet.googlenet_cars import GOOGLENET_CARS 3 | from neural_dream.models.googlenet.googlenet_sos import GoogleNet_SOS 4 | from neural_dream.models.googlenet.googlenetplaces import GoogLeNetPlaces 5 | from neural_dream.models.googlenet.googlenet_layer_names import googlenet_layer_names 6 | 7 | from neural_dream.models.inception.inception5h import Inception5h 8 | from neural_dream.models.inception.inceptionv3_keras import InceptionV3Keras 9 | from neural_dream.models.inception.inception_layer_names import inception_layer_names 10 | 11 | from neural_dream.models.resnet.resnet_50_1by2_nsfw import ResNet_50_1by2_nsfw 12 | from neural_dream.models.resnet.resnet_layer_names import resnet_layer_names -------------------------------------------------------------------------------- /neural_dream/models/googlenet/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/neural_dream/models/googlenet/__init__.py -------------------------------------------------------------------------------- /neural_dream/models/googlenet/googlenet_layer_names.py: -------------------------------------------------------------------------------- 1 | def googlenet_layer_names(model_name='places'): 2 | if 'places' in model_name: 3 | hookable_layers = ['conv1_7x7_s2', 'conv1_relu_7x7', 'pool1_3x3_s2', 'conv2_3x3_reduce', 'conv2_relu_3x3_reduce', 'conv2_3x3', 'conv2_relu_3x3', 'pool2_3x3_s2', \ 4 | 'inception_3a_pool', 'inception_3a_1x1', 'inception_3a_5x5_reduce', 'inception_3a_3x3_reduce', 'inception_3a_pool_proj', 'inception_3a_relu_1x1', \ 5 | 'inception_3a_relu_5x5_reduce', 'inception_3a_relu_3x3_reduce', 'inception_3a_relu_pool_proj', 'inception_3a_5x5', 'inception_3a_3x3', \ 6 | 'inception_3a_relu_5x5', 'inception_3a_relu_3x3', 'inception_3a_output', 'inception_3b_3x3_reduce', 'inception_3b_pool', 'inception_3b_1x1', \ 7 | 'inception_3b_5x5_reduce', 'inception_3b_relu_3x3_reduce', 'inception_3b_pool_proj', 'inception_3b_relu_1x1', 'inception_3b_relu_5x5_reduce', \ 8 | 'inception_3b_3x3', 'inception_3b_relu_pool_proj', 'inception_3b_5x5', 'inception_3b_relu_3x3', 'inception_3b_relu_5x5', 'inception_3b_output', \ 9 | 'pool3_3x3_s2', 'inception_4a_1x1', 'inception_4a_3x3_reduce', 'inception_4a_5x5_reduce', 'inception_4a_pool', 'inception_4a_relu_1x1', \ 10 | 'inception_4a_relu_3x3_reduce', 'inception_4a_relu_5x5_reduce', 'inception_4a_pool_proj', 'inception_4a_3x3', 'inception_4a_5x5', \ 11 | 'inception_4a_relu_pool_proj', 'inception_4a_relu_3x3', 'inception_4a_relu_5x5', 'inception_4a_output', 'inception_4b_pool', 'inception_4b_5x5_reduce', \ 12 | 'inception_4b_1x1', 'inception_4b_3x3_reduce', 'inception_4b_pool_proj', 'inception_4b_relu_5x5_reduce', 'inception_4b_relu_1x1', \ 13 | 'inception_4b_relu_3x3_reduce', 'inception_4b_relu_pool_proj', 'inception_4b_5x5', 'inception_4b_3x3', 'inception_4b_relu_5x5', 'inception_4b_relu_3x3', \ 14 | 'inception_4b_output', 'inception_4c_5x5_reduce', 'inception_4c_pool', 'inception_4c_1x1', 'inception_4c_3x3_reduce', 'inception_4c_relu_5x5_reduce', \ 15 | 'inception_4c_pool_proj', 'inception_4c_relu_1x1', 'inception_4c_relu_3x3_reduce', 'inception_4c_5x5', 'inception_4c_relu_pool_proj', 'inception_4c_3x3', \ 16 | 'inception_4c_relu_5x5', 'inception_4c_relu_3x3', 'inception_4c_output', 'inception_4d_pool', 'inception_4d_3x3_reduce', 'inception_4d_1x1', \ 17 | 'inception_4d_5x5_reduce', 'inception_4d_pool_proj', 'inception_4d_relu_3x3_reduce', 'inception_4d_relu_1x1', 'inception_4d_relu_5x5_reduce', \ 18 | 'inception_4d_relu_pool_proj', 'inception_4d_3x3', 'inception_4d_5x5', 'inception_4d_relu_3x3', 'inception_4d_relu_5x5', 'inception_4d_output', \ 19 | 'inception_4e_1x1', 'inception_4e_5x5_reduce', 'inception_4e_3x3_reduce', 'inception_4e_pool', 'inception_4e_relu_1x1', 'inception_4e_relu_5x5_reduce', \ 20 | 'inception_4e_relu_3x3_reduce', 'inception_4e_pool_proj', 'inception_4e_5x5', 'inception_4e_3x3', 'inception_4e_relu_pool_proj', 'inception_4e_relu_5x5', \ 21 | 'inception_4e_relu_3x3', 'inception_4e_output', 'pool4_3x3_s2', 'inception_5a_1x1', 'inception_5a_5x5_reduce', 'inception_5a_pool', 'inception_5a_3x3_reduce', \ 22 | 'inception_5a_relu_1x1', 'inception_5a_relu_5x5_reduce', 'inception_5a_pool_proj', 'inception_5a_relu_3x3_reduce', 'inception_5a_5x5', 'inception_5a_relu_pool_proj', \ 23 | 'inception_5a_3x3', 'inception_5a_relu_5x5', 'inception_5a_relu_3x3', 'inception_5a_output', 'inception_5b_3x3_reduce', 'inception_5b_pool', 'inception_5b_5x5_reduce', \ 24 | 'inception_5b_1x1', 'inception_5b_relu_3x3_reduce', 'inception_5b_pool_proj', 'inception_5b_relu_5x5_reduce', 'inception_5b_relu_1x1', 'inception_5b_3x3', \ 25 | 'inception_5b_relu_pool_proj', 'inception_5b_5x5', 'inception_5b_relu_3x3', 'inception_5b_relu_5x5', 'inception_5b_output', 'pool5_drop_7x7_s1'] 26 | elif 'bvlc' in model_name: 27 | hookable_layers = ['conv1_7x7_s2', 'conv1_relu_7x7', 'pool1_3x3_s2', 'conv2_3x3_reduce', 'conv2_relu_3x3_reduce', 'conv2_3x3', 'conv2_relu_3x3', 'pool2_3x3_s2', \ 28 | 'inception_3a_pool', 'inception_3a_1x1', 'inception_3a_5x5_reduce', 'inception_3a_3x3_reduce', 'inception_3a_pool_proj', 'inception_3a_relu_1x1', \ 29 | 'inception_3a_relu_5x5_reduce', 'inception_3a_relu_3x3_reduce', 'inception_3a_relu_pool_proj', 'inception_3a_5x5', 'inception_3a_3x3', \ 30 | 'inception_3a_relu_5x5', 'inception_3a_relu_3x3', 'inception_3a_output', 'inception_3b_3x3_reduce', 'inception_3b_pool', 'inception_3b_1x1', \ 31 | 'inception_3b_5x5_reduce', 'inception_3b_relu_3x3_reduce', 'inception_3b_pool_proj', 'inception_3b_relu_1x1', 'inception_3b_relu_5x5_reduce', \ 32 | 'inception_3b_3x3', 'inception_3b_relu_pool_proj', 'inception_3b_5x5', 'inception_3b_relu_3x3', 'inception_3b_relu_5x5', 'inception_3b_output', \ 33 | 'pool3_3x3_s2', 'inception_4a_1x1', 'inception_4a_3x3_reduce', 'inception_4a_5x5_reduce', 'inception_4a_pool', 'inception_4a_relu_1x1', \ 34 | 'inception_4a_relu_3x3_reduce', 'inception_4a_relu_5x5_reduce', 'inception_4a_pool_proj', 'inception_4a_3x3', 'inception_4a_5x5', \ 35 | 'inception_4a_relu_pool_proj', 'inception_4a_relu_3x3', 'inception_4a_relu_5x5', 'inception_4a_output', 'inception_4b_pool', 'inception_4b_5x5_reduce', \ 36 | 'inception_4b_1x1', 'inception_4b_3x3_reduce', 'inception_4b_pool_proj', 'inception_4b_relu_5x5_reduce', 'inception_4b_relu_1x1', \ 37 | 'inception_4b_relu_3x3_reduce', 'inception_4b_relu_pool_proj', 'inception_4b_5x5', 'inception_4b_3x3', 'inception_4b_relu_5x5', 'inception_4b_relu_3x3', \ 38 | 'inception_4b_output', 'inception_4c_5x5_reduce', 'inception_4c_pool', 'inception_4c_1x1', 'inception_4c_3x3_reduce', 'inception_4c_relu_5x5_reduce', \ 39 | 'inception_4c_pool_proj', 'inception_4c_relu_1x1', 'inception_4c_relu_3x3_reduce', 'inception_4c_5x5', 'inception_4c_relu_pool_proj', 'inception_4c_3x3', \ 40 | 'inception_4c_relu_5x5', 'inception_4c_relu_3x3', 'inception_4c_output', 'inception_4d_pool', 'inception_4d_3x3_reduce', 'inception_4d_1x1', \ 41 | 'inception_4d_5x5_reduce', 'inception_4d_pool_proj', 'inception_4d_relu_3x3_reduce', 'inception_4d_relu_1x1', 'inception_4d_relu_5x5_reduce', \ 42 | 'inception_4d_relu_pool_proj', 'inception_4d_3x3', 'inception_4d_5x5', 'inception_4d_relu_3x3', 'inception_4d_relu_5x5', 'inception_4d_output', \ 43 | 'inception_4e_1x1', 'inception_4e_5x5_reduce', 'inception_4e_3x3_reduce', 'inception_4e_pool', 'inception_4e_relu_1x1', 'inception_4e_relu_5x5_reduce', \ 44 | 'inception_4e_relu_3x3_reduce', 'inception_4e_pool_proj', 'inception_4e_5x5', 'inception_4e_3x3', 'inception_4e_relu_pool_proj', 'inception_4e_relu_5x5', \ 45 | 'inception_4e_relu_3x3', 'inception_4e_output', 'pool4_3x3_s2 ', 'inception_5a_1x1', 'inception_5a_5x5_reduce', 'inception_5a_pool', \ 46 | 'inception_5a_3x3_reduce', 'inception_5a_relu_1x1', 'inception_5a_relu_5x5_reduce', 'inception_5a_pool_proj', 'inception_5a_relu_3x3_reduce', \ 47 | 'inception_5a_5x5', 'inception_5a_relu_pool_proj', 'inception_5a_3x3', 'inception_5a_relu_5x5', 'inception_5a_relu_3x3', 'inception_5a_output', \ 48 | 'inception_5b_3x3_reduce', 'inception_5b_pool', 'inception_5b_5x5_reduce', 'inception_5b_1x1', 'inception_5b_relu_3x3_reduce', 'inception_5b_pool_proj', \ 49 | 'inception_5b_relu_5x5_reduce', 'inception_5b_relu_1x1', 'inception_5b_3x3', 'inception_5b_relu_pool_proj', 'inception_5b_5x5', 'inception_5b_relu_3x3', \ 50 | 'inception_5b_relu_5x5', 'inception_5b_output', 'pool5_drop_7x7_s1'] 51 | elif 'sos' in model_name: 52 | hookable_layers = ['conv1_7x7_s2', 'conv1_relu_7x7', 'pool1_3x3_s2', 'conv2_3x3_reduce', 'conv2_relu_3x3_reduce', 'conv2_3x3', 'conv2_relu_3x3', 'pool2_3x3_s2', \ 53 | 'inception_3a_pool', 'inception_3a_1x1', 'inception_3a_5x5_reduce', 'inception_3a_3x3_reduce', 'inception_3a_pool_proj', 'inception_3a_relu_1x1', \ 54 | 'inception_3a_relu_5x5_reduce', 'inception_3a_relu_3x3_reduce', 'inception_3a_relu_pool_proj', 'inception_3a_5x5', 'inception_3a_3x3', \ 55 | 'inception_3a_relu_5x5', 'inception_3a_relu_3x3', 'inception_3a_output', 'inception_3b_3x3_reduce', 'inception_3b_pool', 'inception_3b_1x1', \ 56 | 'inception_3b_5x5_reduce', 'inception_3b_relu_3x3_reduce', 'inception_3b_pool_proj', 'inception_3b_relu_1x1', 'inception_3b_relu_5x5_reduce', \ 57 | 'inception_3b_3x3', 'inception_3b_relu_pool_proj', 'inception_3b_5x5', 'inception_3b_relu_3x3', 'inception_3b_relu_5x5', 'inception_3b_output', \ 58 | 'pool3_3x3_s2', 'inception_4a_1x1', 'inception_4a_3x3_reduce', 'inception_4a_5x5_reduce', 'inception_4a_pool', 'inception_4a_relu_1x1', \ 59 | 'inception_4a_relu_3x3_reduce', 'inception_4a_relu_5x5_reduce', 'inception_4a_pool_proj', 'inception_4a_3x3', 'inception_4a_5x5', 'inception_4a_relu_pool_proj', \ 60 | 'inception_4a_relu_3x3', 'inception_4a_relu_5x5', 'inception_4a_output', 'inception_4b_pool', 'inception_4b_5x5_reduce', 'inception_4b_1x1', \ 61 | 'inception_4b_3x3_reduce', 'inception_4b_pool_proj', 'inception_4b_relu_5x5_reduce', 'inception_4b_relu_1x1', 'inception_4b_relu_3x3_reduce', \ 62 | 'inception_4b_relu_pool_proj', 'inception_4b_5x5', 'inception_4b_3x3', 'inception_4b_relu_5x5', 'inception_4b_relu_3x3', 'inception_4b_output', \ 63 | 'inception_4c_5x5_reduce', 'inception_4c_pool', 'inception_4c_1x1', 'inception_4c_3x3_reduce', 'inception_4c_relu_5x5_reduce', 'inception_4c_pool_proj', \ 64 | 'inception_4c_relu_1x1', 'inception_4c_relu_3x3_reduce', 'inception_4c_5x5', 'inception_4c_relu_pool_proj', 'inception_4c_3x3', 'inception_4c_relu_5x5', \ 65 | 'inception_4c_relu_3x3', 'inception_4c_output', 'inception_4d_pool', 'inception_4d_3x3_reduce', 'inception_4d_1x1', 'inception_4d_5x5_reduce', \ 66 | 'inception_4d_pool_proj', 'inception_4d_relu_3x3_reduce', 'inception_4d_relu_1x1', 'inception_4d_relu_5x5_reduce', 'inception_4d_relu_pool_proj', \ 67 | 'inception_4d_3x3', 'inception_4d_5x5', 'inception_4d_relu_3x3', 'inception_4d_relu_5x5', 'inception_4d_output', 'inception_4e_5x5_reduce', 'inception_4e_1x1', \ 68 | 'inception_4e_3x3_reduce', 'inception_4e_pool', 'inception_4e_relu_5x5_reduce', 'inception_4e_relu_1x1', 'inception_4e_relu_3x3_reduce', 'inception_4e_pool_proj', \ 69 | 'inception_4e_5x5', 'inception_4e_3x3', 'inception_4e_relu_pool_proj', 'inception_4e_relu_5x5', 'inception_4e_relu_3x3', 'inception_4e_output', 'pool4_3x3_s2', \ 70 | 'inception_5a_1x1', 'inception_5a_5x5_reduce', 'inception_5a_pool', 'inception_5a_3x3_reduce', 'inception_5a_relu_1x1', 'inception_5a_relu_5x5_reduce', \ 71 | 'inception_5a_pool_proj', 'inception_5a_relu_3x3_reduce', 'inception_5a_5x5', 'inception_5a_relu_pool_proj', 'inception_5a_3x3', 'inception_5a_relu_5x5', \ 72 | 'inception_5a_relu_3x3', 'inception_5a_output', 'inception_5b_pool', 'inception_5b_3x3_reduce', 'inception_5b_5x5_reduce', 'inception_5b_1x1', \ 73 | 'inception_5b_pool_proj', 'inception_5b_relu_3x3_reduce', 'inception_5b_relu_5x5_reduce', 'inception_5b_relu_1x1', 'inception_5b_relu_pool_proj', \ 74 | 'inception_5b_3x3', 'inception_5b_5x5', 'inception_5b_relu_3x3', 'inception_5b_relu_5x5', 'inception_5b_output', 'pool5_drop_7x7_s1'] 75 | elif 'cars' in model_name: 76 | hookable_layers = ['conv1', 'relu1', 'pool1', 'conv2_1x1', 'relu_conv2_1x1', 'conv2_3x3', 'relu2_3x3', 'pool2', 'inception_3a_5x5_reduce', 'inception_3a_3x3_reduce', \ 77 | 'inception_3a_1x1', 'inception_3a_pool', 'relu_inception_3a_5x5_reduce', 'reulu_inception_3a_3x3_reduce', 'relu_inception_3a_1x1', 'inception_3a_pool_proj', \ 78 | 'inception_3a_5x5', 'inception_3a_3x3', 'relu_inception_3a_pool_proj', 'relu_inception_3a_5x5', 'relu_inception_3a_3x3', 'inception_3a_output', \ 79 | 'inception_3b_1x1', 'inception_3b_3x3_reduce', 'inception_3b_5x5_reduce', 'inception_3b_pool', 'relu_inception_3b_1x1', 'relu_inception_3b_3x3_reduce', \ 80 | 'relu_inception_3b_5x5_reduce', 'inception_3b_pool_proj', 'inception_3b_3x3', 'inception_3b_5x5', 'relu_inception_3b_pool_proj', 'relu_inception_3b_3x3', \ 81 | 'relu_inception_3b_5x5', 'inception_3b_output', 'pool3', 'inception_4a_pool', 'inception_4a_3x3_reduce', 'inception_4a_5x5_reduce', 'inception_4a_1x1', \ 82 | 'inception_4a_pool_proj', 'relu_inception_4a_3x3_reduce', 'relu_inception_4a_5x5_reduce', 'relu_inception_4a_1x1', 'relu_inception_4a_pool_proj', \ 83 | 'inception_4a_3x3', 'inception_4a_5x5', 'relu_inception_4a_3x3', 'relu_inception_4a_5x5', 'inception_4a_output', 'inception_4b_3x3_reduce', 'inception_4b_1x1', \ 84 | 'inception_4b_pool', 'inception_4b_5x5_reduce', 'inception_4b_relu_3x3_reduce', 'inception_4b_relu_1x1', 'inception_4b_pool_proj', 'inception_4b_relu_5x5_reduce', \ 85 | 'inception_4b_3x3', 'inception_4b_relu_pool_proj', 'inception_4b_5x5', 'inception_4b_relu_3x3', 'inception_4b_relu_5x5', 'inception_4b_output', 'inception_4c_pool', \ 86 | 'inception_4c_1x1', 'inception_4c_5x5_reduce', 'inception_4c_3x3_reduce', 'inception_4c_pool_proj', 'inception_4c_relu_1x1', 'inception_4c_relu_5x5_reduce', \ 87 | 'inception_4c_relu_3x3_reduce', 'inception_4c_relu_pool_proj', 'inception_4c_5x5', 'inception_4c_3x3', 'inception_4c_relu_5x5', 'inception_4c_relu_3x3', \ 88 | 'inception_4c_output', 'inception_4d_1x1', 'inception_4d_3x3_reduce', 'inception_4d_5x5_reduce', 'inception_4d_pool', 'inception_4d_relu_1x1', \ 89 | 'inception_4d_relu_3x3_reduce', 'inception_4d_relu_5x5_reduce', 'inception_4d_pool_proj', 'inception_4d_3x3', 'inception_4d_5x5', 'inception_4d_relu_pool_proj', \ 90 | 'inception_4d_relu_3x3', 'inception_4d_relu_5x5', 'inception_4d_output', 'inception_4e_pool', 'inception_4e_1x1', 'inception_4e_3x3_reduce', \ 91 | 'inception_4e_5x5_reduce', 'inception_4e_pool_proj', 'inception_4e_relu_1x1', 'inception_4e_relu_3x3_reduce', 'inception_4e_relu_5x5_reduce', \ 92 | 'inception_4e_relu_pool_proj', 'inception_4e_3x3', 'inception_4e_5x5', 'inception_4e_relu_3x3', 'inception_4e_relu_5x5', 'inception_4e_output', 'pool4', \ 93 | 'inception_5a_pool', 'inception_5a_5x5_reduce', 'inception_5a_3x3_reduce', 'inception_5a_1x1', 'inception_5a_pool_proj', 'inception_5a_relu_5x5_reduce', \ 94 | 'inception_5a_relu_3x3_reduce', 'inception_5a_relu_1x1', 'inception_5a_relu_pool_proj', 'inception_5a_5x5', 'inception_5a_3x3', 'inception_5a_relu_5x5', \ 95 | 'inception_5a_relu_3x3', 'inception_5a_output', 'inception_5b_3x3_reduce', 'inception_5b_1x1', 'inception_5b_5x5_reduce', 'inception_5b_pool', \ 96 | 'inception_5b_relu_3x3_reduce', 'inception_5b_relu_1x1', 'inception_5b_relu_5x5_reduce', 'inception_5b_pool_proj', 'inception_5b_3x3', 'inception_5b_5x5', \ 97 | 'inception_5b_relu_pool_proj', 'inception_5b_relu_3x3', 'inception_5b_relu_5x5', 'inception_5b_output', 'pool5_drop'] 98 | return hookable_layers -------------------------------------------------------------------------------- /neural_dream/models/inception/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/neural_dream/models/inception/__init__.py -------------------------------------------------------------------------------- /neural_dream/models/inception/inception5h.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torch.nn as nn 3 | import torch.nn.functional as F 4 | import neural_dream.helper_layers as helper_layers 5 | 6 | 7 | class Inception5h(nn.Module): 8 | 9 | def __init__(self): 10 | super(Inception5h, self).__init__() 11 | self.conv2d0_pre_relu_conv = nn.Conv2d(in_channels=3, out_channels=64, kernel_size=(7, 7), stride=(2, 2), groups=1, bias=True) 12 | self.conv2d1_pre_relu_conv = nn.Conv2d(in_channels=64, out_channels=64, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 13 | self.conv2d2_pre_relu_conv = nn.Conv2d(in_channels=64, out_channels=192, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True) 14 | self.mixed3a_1x1_pre_relu_conv = nn.Conv2d(in_channels=192, out_channels=64, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 15 | self.mixed3a_3x3_bottleneck_pre_relu_conv = nn.Conv2d(in_channels=192, out_channels=96, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 16 | self.mixed3a_5x5_bottleneck_pre_relu_conv = nn.Conv2d(in_channels=192, out_channels=16, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 17 | self.mixed3a_pool_reduce_pre_relu_conv = nn.Conv2d(in_channels=192, out_channels=32, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 18 | self.mixed3a_3x3_pre_relu_conv = nn.Conv2d(in_channels=96, out_channels=128, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True) 19 | self.mixed3a_5x5_pre_relu_conv = nn.Conv2d(in_channels=16, out_channels=32, kernel_size=(5, 5), stride=(1, 1), groups=1, bias=True) 20 | self.mixed3b_1x1_pre_relu_conv = nn.Conv2d(in_channels=256, out_channels=128, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 21 | self.mixed3b_3x3_bottleneck_pre_relu_conv = nn.Conv2d(in_channels=256, out_channels=128, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 22 | self.mixed3b_5x5_bottleneck_pre_relu_conv = nn.Conv2d(in_channels=256, out_channels=32, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 23 | self.mixed3b_pool_reduce_pre_relu_conv = nn.Conv2d(in_channels=256, out_channels=64, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 24 | self.mixed3b_3x3_pre_relu_conv = nn.Conv2d(in_channels=128, out_channels=192, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True) 25 | self.mixed3b_5x5_pre_relu_conv = nn.Conv2d(in_channels=32, out_channels=96, kernel_size=(5, 5), stride=(1, 1), groups=1, bias=True) 26 | self.mixed4a_1x1_pre_relu_conv = nn.Conv2d(in_channels=480, out_channels=192, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 27 | self.mixed4a_3x3_bottleneck_pre_relu_conv = nn.Conv2d(in_channels=480, out_channels=96, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 28 | self.mixed4a_5x5_bottleneck_pre_relu_conv = nn.Conv2d(in_channels=480, out_channels=16, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 29 | self.mixed4a_pool_reduce_pre_relu_conv = nn.Conv2d(in_channels=480, out_channels=64, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 30 | self.mixed4a_3x3_pre_relu_conv = nn.Conv2d(in_channels=96, out_channels=204, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True) 31 | self.mixed4a_5x5_pre_relu_conv = nn.Conv2d(in_channels=16, out_channels=48, kernel_size=(5, 5), stride=(1, 1), groups=1, bias=True) 32 | self.mixed4b_1x1_pre_relu_conv = nn.Conv2d(in_channels=508, out_channels=160, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 33 | self.mixed4b_3x3_bottleneck_pre_relu_conv = nn.Conv2d(in_channels=508, out_channels=112, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 34 | self.mixed4b_5x5_bottleneck_pre_relu_conv = nn.Conv2d(in_channels=508, out_channels=24, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 35 | self.mixed4b_pool_reduce_pre_relu_conv = nn.Conv2d(in_channels=508, out_channels=64, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 36 | self.mixed4b_3x3_pre_relu_conv = nn.Conv2d(in_channels=112, out_channels=224, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True) 37 | self.mixed4b_5x5_pre_relu_conv = nn.Conv2d(in_channels=24, out_channels=64, kernel_size=(5, 5), stride=(1, 1), groups=1, bias=True) 38 | self.mixed4c_1x1_pre_relu_conv = nn.Conv2d(in_channels=512, out_channels=128, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 39 | self.mixed4c_3x3_bottleneck_pre_relu_conv = nn.Conv2d(in_channels=512, out_channels=128, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 40 | self.mixed4c_5x5_bottleneck_pre_relu_conv = nn.Conv2d(in_channels=512, out_channels=24, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 41 | self.mixed4c_pool_reduce_pre_relu_conv = nn.Conv2d(in_channels=512, out_channels=64, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 42 | self.mixed4c_3x3_pre_relu_conv = nn.Conv2d(in_channels=128, out_channels=256, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True) 43 | self.mixed4c_5x5_pre_relu_conv = nn.Conv2d(in_channels=24, out_channels=64, kernel_size=(5, 5), stride=(1, 1), groups=1, bias=True) 44 | self.mixed4d_1x1_pre_relu_conv = nn.Conv2d(in_channels=512, out_channels=112, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 45 | self.mixed4d_3x3_bottleneck_pre_relu_conv = nn.Conv2d(in_channels=512, out_channels=144, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 46 | self.mixed4d_5x5_bottleneck_pre_relu_conv = nn.Conv2d(in_channels=512, out_channels=32, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 47 | self.mixed4d_pool_reduce_pre_relu_conv = nn.Conv2d(in_channels=512, out_channels=64, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 48 | self.mixed4d_3x3_pre_relu_conv = nn.Conv2d(in_channels=144, out_channels=288, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True) 49 | self.mixed4d_5x5_pre_relu_conv = nn.Conv2d(in_channels=32, out_channels=64, kernel_size=(5, 5), stride=(1, 1), groups=1, bias=True) 50 | self.mixed4e_1x1_pre_relu_conv = nn.Conv2d(in_channels=528, out_channels=256, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 51 | self.mixed4e_3x3_bottleneck_pre_relu_conv = nn.Conv2d(in_channels=528, out_channels=160, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 52 | self.mixed4e_5x5_bottleneck_pre_relu_conv = nn.Conv2d(in_channels=528, out_channels=32, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 53 | self.mixed4e_pool_reduce_pre_relu_conv = nn.Conv2d(in_channels=528, out_channels=128, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 54 | self.mixed4e_3x3_pre_relu_conv = nn.Conv2d(in_channels=160, out_channels=320, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True) 55 | self.mixed4e_5x5_pre_relu_conv = nn.Conv2d(in_channels=32, out_channels=128, kernel_size=(5, 5), stride=(1, 1), groups=1, bias=True) 56 | self.mixed5a_1x1_pre_relu_conv = nn.Conv2d(in_channels=832, out_channels=256, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 57 | self.mixed5a_3x3_bottleneck_pre_relu_conv = nn.Conv2d(in_channels=832, out_channels=160, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 58 | self.mixed5a_5x5_bottleneck_pre_relu_conv = nn.Conv2d(in_channels=832, out_channels=48, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 59 | self.mixed5a_pool_reduce_pre_relu_conv = nn.Conv2d(in_channels=832, out_channels=128, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 60 | self.mixed5a_3x3_pre_relu_conv = nn.Conv2d(in_channels=160, out_channels=320, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True) 61 | self.mixed5a_5x5_pre_relu_conv = nn.Conv2d(in_channels=48, out_channels=128, kernel_size=(5, 5), stride=(1, 1), groups=1, bias=True) 62 | self.mixed5b_1x1_pre_relu_conv = nn.Conv2d(in_channels=832, out_channels=384, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 63 | self.mixed5b_3x3_bottleneck_pre_relu_conv = nn.Conv2d(in_channels=832, out_channels=192, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 64 | self.mixed5b_5x5_bottleneck_pre_relu_conv = nn.Conv2d(in_channels=832, out_channels=48, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 65 | self.mixed5b_pool_reduce_pre_relu_conv = nn.Conv2d(in_channels=832, out_channels=128, kernel_size=(1, 1), stride=(1, 1), groups=1, bias=True) 66 | self.mixed5b_3x3_pre_relu_conv = nn.Conv2d(in_channels=192, out_channels=384, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True) 67 | self.mixed5b_5x5_pre_relu_conv = nn.Conv2d(in_channels=48, out_channels=128, kernel_size=(5, 5), stride=(1, 1), groups=1, bias=True) 68 | self.softmax2_pre_activation_matmul = nn.Linear(in_features = 1024, out_features = 1008, bias = True) 69 | 70 | 71 | def add_layers(self): 72 | self.conv2d0 = helper_layers.ReluLayer() 73 | self.maxpool0 = helper_layers.MaxPool2dLayer() 74 | self.conv2d1 = helper_layers.ReluLayer() 75 | self.conv2d2 = helper_layers.ReluLayer() 76 | self.maxpool1 = helper_layers.MaxPool2dLayer() 77 | self.mixed3a_pool = helper_layers.MaxPool2dLayer() 78 | self.mixed3a_1x1 = helper_layers.ReluLayer() 79 | self.mixed3a_3x3_bottleneck = helper_layers.ReluLayer() 80 | self.mixed3a_5x5_bottleneck = helper_layers.ReluLayer() 81 | self.mixed3a_pool_reduce = helper_layers.ReluLayer() 82 | self.mixed3a_3x3 = helper_layers.ReluLayer() 83 | self.mixed3a_5x5 = helper_layers.ReluLayer() 84 | self.mixed3a = helper_layers.CatLayer() 85 | self.mixed3b_pool = helper_layers.MaxPool2dLayer() 86 | self.mixed3b_1x1 = helper_layers.ReluLayer() 87 | self.mixed3b_3x3_bottleneck = helper_layers.ReluLayer() 88 | self.mixed3b_5x5_bottleneck = helper_layers.ReluLayer() 89 | self.mixed3b_pool_reduce = helper_layers.ReluLayer() 90 | self.mixed3b_3x3 = helper_layers.ReluLayer() 91 | self.mixed3b_5x5 = helper_layers.ReluLayer() 92 | self.mixed3b = helper_layers.CatLayer() 93 | self.maxpool4 = helper_layers.MaxPool2dLayer() 94 | self.mixed4a_pool = helper_layers.MaxPool2dLayer() 95 | self.mixed4a_1x1 = helper_layers.ReluLayer() 96 | self.mixed4a_3x3_bottleneck = helper_layers.ReluLayer() 97 | self.mixed4a_5x5_bottleneck = helper_layers.ReluLayer() 98 | self.mixed4a_pool_reduce = helper_layers.ReluLayer() 99 | self.mixed4a_3x3 = helper_layers.ReluLayer() 100 | self.mixed4a_5x5 = helper_layers.ReluLayer() 101 | self.mixed4a = helper_layers.CatLayer() 102 | self.mixed4b_pool = helper_layers.MaxPool2dLayer() 103 | self.mixed4b_1x1 = helper_layers.ReluLayer() 104 | self.mixed4b_3x3_bottleneck = helper_layers.ReluLayer() 105 | self.mixed4b_5x5_bottleneck = helper_layers.ReluLayer() 106 | self.mixed4b_pool_reduce = helper_layers.ReluLayer() 107 | self.mixed4b_3x3 = helper_layers.ReluLayer() 108 | self.mixed4b_5x5 = helper_layers.ReluLayer() 109 | self.mixed4b = helper_layers.CatLayer() 110 | self.mixed4c_pool = helper_layers.MaxPool2dLayer() 111 | self.mixed4c_1x1 = helper_layers.ReluLayer() 112 | self.mixed4c_3x3_bottleneck = helper_layers.ReluLayer() 113 | self.mixed4c_5x5_bottleneck = helper_layers.ReluLayer() 114 | self.mixed4c_pool_reduce = helper_layers.ReluLayer() 115 | self.mixed4c_3x3 = helper_layers.ReluLayer() 116 | self.mixed4c_5x5 = helper_layers.ReluLayer() 117 | self.mixed4c = helper_layers.CatLayer() 118 | self.mixed4d_pool = helper_layers.MaxPool2dLayer() 119 | self.mixed4d_1x1 = helper_layers.ReluLayer() 120 | self.mixed4d_3x3_bottleneck = helper_layers.ReluLayer() 121 | self.mixed4d_5x5_bottleneck = helper_layers.ReluLayer() 122 | self.mixed4d_pool_reduce = helper_layers.ReluLayer() 123 | self.mixed4d_3x3 = helper_layers.ReluLayer() 124 | self.mixed4d_5x5 = helper_layers.ReluLayer() 125 | self.mixed4d = helper_layers.CatLayer() 126 | self.mixed4e_pool = helper_layers.MaxPool2dLayer() 127 | self.mixed4e_1x1 = helper_layers.ReluLayer() 128 | self.mixed4e_3x3_bottleneck = helper_layers.ReluLayer() 129 | self.mixed4e_5x5_bottleneck = helper_layers.ReluLayer() 130 | self.mixed4e_pool_reduce = helper_layers.ReluLayer() 131 | self.mixed4e_3x3 = helper_layers.ReluLayer() 132 | self.mixed4e_5x5 = helper_layers.ReluLayer() 133 | self.mixed4e = helper_layers.CatLayer() 134 | self.maxpool10 = helper_layers.MaxPool2dLayer() 135 | self.mixed5a_pool = helper_layers.MaxPool2dLayer() 136 | self.mixed5a_1x1 = helper_layers.ReluLayer() 137 | self.mixed5a_3x3_bottleneck = helper_layers.ReluLayer() 138 | self.mixed5a_5x5_bottleneck = helper_layers.ReluLayer() 139 | self.mixed5a_pool_reduce = helper_layers.ReluLayer() 140 | self.mixed5a_3x3 = helper_layers.ReluLayer() 141 | self.mixed5a_5x5 = helper_layers.ReluLayer() 142 | self.mixed5a = helper_layers.CatLayer() 143 | self.mixed5b_pool = helper_layers.MaxPool2dLayer() 144 | self.mixed5b_1x1 = helper_layers.ReluLayer() 145 | self.mixed5b_3x3_bottleneck = helper_layers.ReluLayer() 146 | self.mixed5b_5x5_bottleneck = helper_layers.ReluLayer() 147 | self.mixed5b_pool_reduce = helper_layers.ReluLayer() 148 | self.mixed5b_3x3 = helper_layers.ReluLayer() 149 | self.mixed5b_5x5 = helper_layers.ReluLayer() 150 | self.mixed5b = helper_layers.CatLayer() 151 | self.softmax2 = helper_layers.SoftMaxLayer() 152 | 153 | 154 | def forward(self, x): 155 | conv2d0_pre_relu_conv_pad = F.pad(x, (2, 3, 2, 3)) 156 | conv2d0_pre_relu_conv = self.conv2d0_pre_relu_conv(conv2d0_pre_relu_conv_pad) 157 | conv2d0 = self.conv2d0(conv2d0_pre_relu_conv) 158 | maxpool0_pad = F.pad(conv2d0, (0, 1, 0, 1), value=float('-inf')) 159 | maxpool0 = self.maxpool0(maxpool0_pad, kernel_size=(3, 3), stride=(2, 2), padding=0, ceil_mode=False) 160 | localresponsenorm0 = F.local_response_norm(maxpool0, size=9, alpha=9.99999974738e-05, beta=0.5, k=1) 161 | conv2d1_pre_relu_conv = self.conv2d1_pre_relu_conv(localresponsenorm0) 162 | conv2d1 = self.conv2d1(conv2d1_pre_relu_conv) 163 | conv2d2_pre_relu_conv_pad = F.pad(conv2d1, (1, 1, 1, 1)) 164 | conv2d2_pre_relu_conv = self.conv2d2_pre_relu_conv(conv2d2_pre_relu_conv_pad) 165 | conv2d2 = self.conv2d2(conv2d2_pre_relu_conv) 166 | localresponsenorm1 = F.local_response_norm(conv2d2, size=9, alpha=9.99999974738e-05, beta=0.5, k=1) 167 | maxpool1_pad = F.pad(localresponsenorm1, (0, 1, 0, 1), value=float('-inf')) 168 | maxpool1 = self.maxpool1(maxpool1_pad, kernel_size=(3, 3), stride=(2, 2), padding=0, ceil_mode=False) 169 | mixed3a_1x1_pre_relu_conv = self.mixed3a_1x1_pre_relu_conv(maxpool1) 170 | mixed3a_3x3_bottleneck_pre_relu_conv = self.mixed3a_3x3_bottleneck_pre_relu_conv(maxpool1) 171 | mixed3a_5x5_bottleneck_pre_relu_conv = self.mixed3a_5x5_bottleneck_pre_relu_conv(maxpool1) 172 | mixed3a_pool_pad = F.pad(maxpool1, (1, 1, 1, 1), value=float('-inf')) 173 | mixed3a_pool = self.mixed3a_pool(mixed3a_pool_pad, kernel_size=(3, 3), stride=(1, 1), padding=0, ceil_mode=False) 174 | mixed3a_1x1 = self.mixed3a_1x1(mixed3a_1x1_pre_relu_conv) 175 | mixed3a_3x3_bottleneck = self.mixed3a_3x3_bottleneck(mixed3a_3x3_bottleneck_pre_relu_conv) 176 | mixed3a_5x5_bottleneck = self.mixed3a_5x5_bottleneck(mixed3a_5x5_bottleneck_pre_relu_conv) 177 | mixed3a_pool_reduce_pre_relu_conv = self.mixed3a_pool_reduce_pre_relu_conv(mixed3a_pool) 178 | mixed3a_3x3_pre_relu_conv_pad = F.pad(mixed3a_3x3_bottleneck, (1, 1, 1, 1)) 179 | mixed3a_3x3_pre_relu_conv = self.mixed3a_3x3_pre_relu_conv(mixed3a_3x3_pre_relu_conv_pad) 180 | mixed3a_5x5_pre_relu_conv_pad = F.pad(mixed3a_5x5_bottleneck, (2, 2, 2, 2)) 181 | mixed3a_5x5_pre_relu_conv = self.mixed3a_5x5_pre_relu_conv(mixed3a_5x5_pre_relu_conv_pad) 182 | mixed3a_pool_reduce = self.mixed3a_pool_reduce(mixed3a_pool_reduce_pre_relu_conv) 183 | mixed3a_3x3 = self.mixed3a_3x3(mixed3a_3x3_pre_relu_conv) 184 | mixed3a_5x5 = self.mixed3a_5x5(mixed3a_5x5_pre_relu_conv) 185 | mixed3a = self.mixed3a((mixed3a_1x1, mixed3a_3x3, mixed3a_5x5, mixed3a_pool_reduce), 1) 186 | mixed3b_1x1_pre_relu_conv = self.mixed3b_1x1_pre_relu_conv(mixed3a) 187 | mixed3b_3x3_bottleneck_pre_relu_conv = self.mixed3b_3x3_bottleneck_pre_relu_conv(mixed3a) 188 | mixed3b_5x5_bottleneck_pre_relu_conv = self.mixed3b_5x5_bottleneck_pre_relu_conv(mixed3a) 189 | mixed3b_pool_pad = F.pad(mixed3a, (1, 1, 1, 1), value=float('-inf')) 190 | mixed3b_pool = self.mixed3b_pool(mixed3b_pool_pad, kernel_size=(3, 3), stride=(1, 1), padding=0, ceil_mode=False) 191 | mixed3b_1x1 = self.mixed3b_1x1(mixed3b_1x1_pre_relu_conv) 192 | mixed3b_3x3_bottleneck = self.mixed3b_3x3_bottleneck(mixed3b_3x3_bottleneck_pre_relu_conv) 193 | mixed3b_5x5_bottleneck = self.mixed3b_5x5_bottleneck(mixed3b_5x5_bottleneck_pre_relu_conv) 194 | mixed3b_pool_reduce_pre_relu_conv = self.mixed3b_pool_reduce_pre_relu_conv(mixed3b_pool) 195 | mixed3b_3x3_pre_relu_conv_pad = F.pad(mixed3b_3x3_bottleneck, (1, 1, 1, 1)) 196 | mixed3b_3x3_pre_relu_conv = self.mixed3b_3x3_pre_relu_conv(mixed3b_3x3_pre_relu_conv_pad) 197 | mixed3b_5x5_pre_relu_conv_pad = F.pad(mixed3b_5x5_bottleneck, (2, 2, 2, 2)) 198 | mixed3b_5x5_pre_relu_conv = self.mixed3b_5x5_pre_relu_conv(mixed3b_5x5_pre_relu_conv_pad) 199 | mixed3b_pool_reduce = self.mixed3b_pool_reduce(mixed3b_pool_reduce_pre_relu_conv) 200 | mixed3b_3x3 = self.mixed3b_3x3(mixed3b_3x3_pre_relu_conv) 201 | mixed3b_5x5 = self.mixed3b_5x5(mixed3b_5x5_pre_relu_conv) 202 | mixed3b = self.mixed3b((mixed3b_1x1, mixed3b_3x3, mixed3b_5x5, mixed3b_pool_reduce), 1) 203 | maxpool4_pad = F.pad(mixed3b, (0, 1, 0, 1), value=float('-inf')) 204 | maxpool4 = self.maxpool4(maxpool4_pad, kernel_size=(3, 3), stride=(2, 2), padding=0, ceil_mode=False) 205 | mixed4a_1x1_pre_relu_conv = self.mixed4a_1x1_pre_relu_conv(maxpool4) 206 | mixed4a_3x3_bottleneck_pre_relu_conv = self.mixed4a_3x3_bottleneck_pre_relu_conv(maxpool4) 207 | mixed4a_5x5_bottleneck_pre_relu_conv = self.mixed4a_5x5_bottleneck_pre_relu_conv(maxpool4) 208 | mixed4a_pool_pad = F.pad(maxpool4, (1, 1, 1, 1), value=float('-inf')) 209 | mixed4a_pool = self.mixed4a_pool(mixed4a_pool_pad, kernel_size=(3, 3), stride=(1, 1), padding=0, ceil_mode=False) 210 | mixed4a_1x1 = self.mixed4a_1x1(mixed4a_1x1_pre_relu_conv) 211 | mixed4a_3x3_bottleneck = self.mixed4a_3x3_bottleneck(mixed4a_3x3_bottleneck_pre_relu_conv) 212 | mixed4a_5x5_bottleneck = self.mixed4a_5x5_bottleneck(mixed4a_5x5_bottleneck_pre_relu_conv) 213 | mixed4a_pool_reduce_pre_relu_conv = self.mixed4a_pool_reduce_pre_relu_conv(mixed4a_pool) 214 | mixed4a_3x3_pre_relu_conv_pad = F.pad(mixed4a_3x3_bottleneck, (1, 1, 1, 1)) 215 | mixed4a_3x3_pre_relu_conv = self.mixed4a_3x3_pre_relu_conv(mixed4a_3x3_pre_relu_conv_pad) 216 | mixed4a_5x5_pre_relu_conv_pad = F.pad(mixed4a_5x5_bottleneck, (2, 2, 2, 2)) 217 | mixed4a_5x5_pre_relu_conv = self.mixed4a_5x5_pre_relu_conv(mixed4a_5x5_pre_relu_conv_pad) 218 | mixed4a_pool_reduce = self.mixed4a_pool_reduce(mixed4a_pool_reduce_pre_relu_conv) 219 | mixed4a_3x3 = self.mixed4a_3x3(mixed4a_3x3_pre_relu_conv) 220 | mixed4a_5x5 = self.mixed4a_5x5(mixed4a_5x5_pre_relu_conv) 221 | mixed4a = self.mixed4a((mixed4a_1x1, mixed4a_3x3, mixed4a_5x5, mixed4a_pool_reduce), 1) 222 | mixed4b_1x1_pre_relu_conv = self.mixed4b_1x1_pre_relu_conv(mixed4a) 223 | mixed4b_3x3_bottleneck_pre_relu_conv = self.mixed4b_3x3_bottleneck_pre_relu_conv(mixed4a) 224 | mixed4b_5x5_bottleneck_pre_relu_conv = self.mixed4b_5x5_bottleneck_pre_relu_conv(mixed4a) 225 | mixed4b_pool_pad = F.pad(mixed4a, (1, 1, 1, 1), value=float('-inf')) 226 | mixed4b_pool = self.mixed4b_pool(mixed4b_pool_pad, kernel_size=(3, 3), stride=(1, 1), padding=0, ceil_mode=False) 227 | mixed4b_1x1 = self.mixed4b_1x1(mixed4b_1x1_pre_relu_conv) 228 | mixed4b_3x3_bottleneck = self.mixed4b_3x3_bottleneck(mixed4b_3x3_bottleneck_pre_relu_conv) 229 | mixed4b_5x5_bottleneck = self.mixed4b_5x5_bottleneck(mixed4b_5x5_bottleneck_pre_relu_conv) 230 | mixed4b_pool_reduce_pre_relu_conv = self.mixed4b_pool_reduce_pre_relu_conv(mixed4b_pool) 231 | mixed4b_3x3_pre_relu_conv_pad = F.pad(mixed4b_3x3_bottleneck, (1, 1, 1, 1)) 232 | mixed4b_3x3_pre_relu_conv = self.mixed4b_3x3_pre_relu_conv(mixed4b_3x3_pre_relu_conv_pad) 233 | mixed4b_5x5_pre_relu_conv_pad = F.pad(mixed4b_5x5_bottleneck, (2, 2, 2, 2)) 234 | mixed4b_5x5_pre_relu_conv = self.mixed4b_5x5_pre_relu_conv(mixed4b_5x5_pre_relu_conv_pad) 235 | mixed4b_pool_reduce = self.mixed4b_pool_reduce(mixed4b_pool_reduce_pre_relu_conv) 236 | mixed4b_3x3 = self.mixed4b_3x3(mixed4b_3x3_pre_relu_conv) 237 | mixed4b_5x5 = self.mixed4b_5x5(mixed4b_5x5_pre_relu_conv) 238 | mixed4b = self.mixed4b((mixed4b_1x1, mixed4b_3x3, mixed4b_5x5, mixed4b_pool_reduce), 1) 239 | mixed4c_1x1_pre_relu_conv = self.mixed4c_1x1_pre_relu_conv(mixed4b) 240 | mixed4c_3x3_bottleneck_pre_relu_conv = self.mixed4c_3x3_bottleneck_pre_relu_conv(mixed4b) 241 | mixed4c_5x5_bottleneck_pre_relu_conv = self.mixed4c_5x5_bottleneck_pre_relu_conv(mixed4b) 242 | mixed4c_pool_pad = F.pad(mixed4b, (1, 1, 1, 1), value=float('-inf')) 243 | mixed4c_pool = self.mixed4c_pool(mixed4c_pool_pad, kernel_size=(3, 3), stride=(1, 1), padding=0, ceil_mode=False) 244 | mixed4c_1x1 = self.mixed4c_1x1(mixed4c_1x1_pre_relu_conv) 245 | mixed4c_3x3_bottleneck = self.mixed4c_3x3_bottleneck(mixed4c_3x3_bottleneck_pre_relu_conv) 246 | mixed4c_5x5_bottleneck = self.mixed4c_5x5_bottleneck(mixed4c_5x5_bottleneck_pre_relu_conv) 247 | mixed4c_pool_reduce_pre_relu_conv = self.mixed4c_pool_reduce_pre_relu_conv(mixed4c_pool) 248 | mixed4c_3x3_pre_relu_conv_pad = F.pad(mixed4c_3x3_bottleneck, (1, 1, 1, 1)) 249 | mixed4c_3x3_pre_relu_conv = self.mixed4c_3x3_pre_relu_conv(mixed4c_3x3_pre_relu_conv_pad) 250 | mixed4c_5x5_pre_relu_conv_pad = F.pad(mixed4c_5x5_bottleneck, (2, 2, 2, 2)) 251 | mixed4c_5x5_pre_relu_conv = self.mixed4c_5x5_pre_relu_conv(mixed4c_5x5_pre_relu_conv_pad) 252 | mixed4c_pool_reduce = self.mixed4c_pool_reduce(mixed4c_pool_reduce_pre_relu_conv) 253 | mixed4c_3x3 = self.mixed4c_3x3(mixed4c_3x3_pre_relu_conv) 254 | mixed4c_5x5 = self.mixed4c_5x5(mixed4c_5x5_pre_relu_conv) 255 | mixed4c = self.mixed4c((mixed4c_1x1, mixed4c_3x3, mixed4c_5x5, mixed4c_pool_reduce), 1) 256 | mixed4d_1x1_pre_relu_conv = self.mixed4d_1x1_pre_relu_conv(mixed4c) 257 | mixed4d_3x3_bottleneck_pre_relu_conv = self.mixed4d_3x3_bottleneck_pre_relu_conv(mixed4c) 258 | mixed4d_5x5_bottleneck_pre_relu_conv = self.mixed4d_5x5_bottleneck_pre_relu_conv(mixed4c) 259 | mixed4d_pool_pad = F.pad(mixed4c, (1, 1, 1, 1), value=float('-inf')) 260 | mixed4d_pool = self.mixed4d_pool(mixed4d_pool_pad, kernel_size=(3, 3), stride=(1, 1), padding=0, ceil_mode=False) 261 | mixed4d_1x1 = self.mixed4d_1x1(mixed4d_1x1_pre_relu_conv) 262 | mixed4d_3x3_bottleneck = self.mixed4d_3x3_bottleneck(mixed4d_3x3_bottleneck_pre_relu_conv) 263 | mixed4d_5x5_bottleneck = self.mixed4d_5x5_bottleneck(mixed4d_5x5_bottleneck_pre_relu_conv) 264 | mixed4d_pool_reduce_pre_relu_conv = self.mixed4d_pool_reduce_pre_relu_conv(mixed4d_pool) 265 | mixed4d_3x3_pre_relu_conv_pad = F.pad(mixed4d_3x3_bottleneck, (1, 1, 1, 1)) 266 | mixed4d_3x3_pre_relu_conv = self.mixed4d_3x3_pre_relu_conv(mixed4d_3x3_pre_relu_conv_pad) 267 | mixed4d_5x5_pre_relu_conv_pad = F.pad(mixed4d_5x5_bottleneck, (2, 2, 2, 2)) 268 | mixed4d_5x5_pre_relu_conv = self.mixed4d_5x5_pre_relu_conv(mixed4d_5x5_pre_relu_conv_pad) 269 | mixed4d_pool_reduce = self.mixed4d_pool_reduce(mixed4d_pool_reduce_pre_relu_conv) 270 | mixed4d_3x3 = self.mixed4d_3x3(mixed4d_3x3_pre_relu_conv) 271 | mixed4d_5x5 = self.mixed4d_5x5(mixed4d_5x5_pre_relu_conv) 272 | mixed4d = self.mixed4d((mixed4d_1x1, mixed4d_3x3, mixed4d_5x5, mixed4d_pool_reduce), 1) 273 | mixed4e_1x1_pre_relu_conv = self.mixed4e_1x1_pre_relu_conv(mixed4d) 274 | mixed4e_3x3_bottleneck_pre_relu_conv = self.mixed4e_3x3_bottleneck_pre_relu_conv(mixed4d) 275 | mixed4e_5x5_bottleneck_pre_relu_conv = self.mixed4e_5x5_bottleneck_pre_relu_conv(mixed4d) 276 | mixed4e_pool_pad = F.pad(mixed4d, (1, 1, 1, 1), value=float('-inf')) 277 | mixed4e_pool = self.mixed4e_pool(mixed4e_pool_pad, kernel_size=(3, 3), stride=(1, 1), padding=0, ceil_mode=False) 278 | mixed4e_1x1 = self.mixed4e_1x1(mixed4e_1x1_pre_relu_conv) 279 | mixed4e_3x3_bottleneck = self.mixed4e_3x3_bottleneck(mixed4e_3x3_bottleneck_pre_relu_conv) 280 | mixed4e_5x5_bottleneck = self.mixed4e_5x5_bottleneck(mixed4e_5x5_bottleneck_pre_relu_conv) 281 | mixed4e_pool_reduce_pre_relu_conv = self.mixed4e_pool_reduce_pre_relu_conv(mixed4e_pool) 282 | mixed4e_3x3_pre_relu_conv_pad = F.pad(mixed4e_3x3_bottleneck, (1, 1, 1, 1)) 283 | mixed4e_3x3_pre_relu_conv = self.mixed4e_3x3_pre_relu_conv(mixed4e_3x3_pre_relu_conv_pad) 284 | mixed4e_5x5_pre_relu_conv_pad = F.pad(mixed4e_5x5_bottleneck, (2, 2, 2, 2)) 285 | mixed4e_5x5_pre_relu_conv = self.mixed4e_5x5_pre_relu_conv(mixed4e_5x5_pre_relu_conv_pad) 286 | mixed4e_pool_reduce = self.mixed4e_pool_reduce(mixed4e_pool_reduce_pre_relu_conv) 287 | mixed4e_3x3 = self.mixed4e_3x3(mixed4e_3x3_pre_relu_conv) 288 | mixed4e_5x5 = self.mixed4e_5x5(mixed4e_5x5_pre_relu_conv) 289 | mixed4e = self.mixed4e((mixed4e_1x1, mixed4e_3x3, mixed4e_5x5, mixed4e_pool_reduce), 1) 290 | maxpool10_pad = F.pad(mixed4e, (0, 1, 0, 1), value=float('-inf')) 291 | maxpool10 = self.maxpool10(maxpool10_pad, kernel_size=(3, 3), stride=(2, 2), padding=0, ceil_mode=False) 292 | mixed5a_1x1_pre_relu_conv = self.mixed5a_1x1_pre_relu_conv(maxpool10) 293 | mixed5a_3x3_bottleneck_pre_relu_conv = self.mixed5a_3x3_bottleneck_pre_relu_conv(maxpool10) 294 | mixed5a_5x5_bottleneck_pre_relu_conv = self.mixed5a_5x5_bottleneck_pre_relu_conv(maxpool10) 295 | mixed5a_pool_pad = F.pad(maxpool10, (1, 1, 1, 1), value=float('-inf')) 296 | mixed5a_pool = self.mixed5a_pool(mixed5a_pool_pad, kernel_size=(3, 3), stride=(1, 1), padding=0, ceil_mode=False) 297 | mixed5a_1x1 = self.mixed5a_1x1(mixed5a_1x1_pre_relu_conv) 298 | mixed5a_3x3_bottleneck = self.mixed5a_3x3_bottleneck(mixed5a_3x3_bottleneck_pre_relu_conv) 299 | mixed5a_5x5_bottleneck = self.mixed5a_5x5_bottleneck(mixed5a_5x5_bottleneck_pre_relu_conv) 300 | mixed5a_pool_reduce_pre_relu_conv = self.mixed5a_pool_reduce_pre_relu_conv(mixed5a_pool) 301 | mixed5a_3x3_pre_relu_conv_pad = F.pad(mixed5a_3x3_bottleneck, (1, 1, 1, 1)) 302 | mixed5a_3x3_pre_relu_conv = self.mixed5a_3x3_pre_relu_conv(mixed5a_3x3_pre_relu_conv_pad) 303 | mixed5a_5x5_pre_relu_conv_pad = F.pad(mixed5a_5x5_bottleneck, (2, 2, 2, 2)) 304 | mixed5a_5x5_pre_relu_conv = self.mixed5a_5x5_pre_relu_conv(mixed5a_5x5_pre_relu_conv_pad) 305 | mixed5a_pool_reduce = self.mixed5a_pool_reduce(mixed5a_pool_reduce_pre_relu_conv) 306 | mixed5a_3x3 = self.mixed5a_3x3(mixed5a_3x3_pre_relu_conv) 307 | mixed5a_5x5 = self.mixed5a_5x5(mixed5a_5x5_pre_relu_conv) 308 | mixed5a = self.mixed5a((mixed5a_1x1, mixed5a_3x3, mixed5a_5x5, mixed5a_pool_reduce), 1) 309 | mixed5b_1x1_pre_relu_conv = self.mixed5b_1x1_pre_relu_conv(mixed5a) 310 | mixed5b_3x3_bottleneck_pre_relu_conv = self.mixed5b_3x3_bottleneck_pre_relu_conv(mixed5a) 311 | mixed5b_5x5_bottleneck_pre_relu_conv = self.mixed5b_5x5_bottleneck_pre_relu_conv(mixed5a) 312 | mixed5b_pool_pad = F.pad(mixed5a, (1, 1, 1, 1), value=float('-inf')) 313 | mixed5b_pool = self.mixed5b_pool(mixed5b_pool_pad, kernel_size=(3, 3), stride=(1, 1), padding=0, ceil_mode=False) 314 | mixed5b_1x1 = self.mixed5b_1x1(mixed5b_1x1_pre_relu_conv) 315 | mixed5b_3x3_bottleneck = self.mixed5b_3x3_bottleneck(mixed5b_3x3_bottleneck_pre_relu_conv) 316 | mixed5b_5x5_bottleneck = self.mixed5b_5x5_bottleneck(mixed5b_5x5_bottleneck_pre_relu_conv) 317 | mixed5b_pool_reduce_pre_relu_conv = self.mixed5b_pool_reduce_pre_relu_conv(mixed5b_pool) 318 | mixed5b_3x3_pre_relu_conv_pad = F.pad(mixed5b_3x3_bottleneck, (1, 1, 1, 1)) 319 | mixed5b_3x3_pre_relu_conv = self.mixed5b_3x3_pre_relu_conv(mixed5b_3x3_pre_relu_conv_pad) 320 | mixed5b_5x5_pre_relu_conv_pad = F.pad(mixed5b_5x5_bottleneck, (2, 2, 2, 2)) 321 | mixed5b_5x5_pre_relu_conv = self.mixed5b_5x5_pre_relu_conv(mixed5b_5x5_pre_relu_conv_pad) 322 | mixed5b_pool_reduce = self.mixed5b_pool_reduce(mixed5b_pool_reduce_pre_relu_conv) 323 | mixed5b_3x3 = self.mixed5b_3x3(mixed5b_3x3_pre_relu_conv) 324 | mixed5b_5x5 = self.mixed5b_5x5(mixed5b_5x5_pre_relu_conv) 325 | mixed5b = self.mixed5b((mixed5b_1x1, mixed5b_3x3, mixed5b_5x5, mixed5b_pool_reduce), 1) 326 | avgpool0 = F.avg_pool2d(mixed5b, kernel_size=(7, 7), stride=(1, 1), padding=(0,), ceil_mode=False, count_include_pad=False) 327 | avgpool0_reshape = torch.reshape(input = avgpool0, shape = (-1,1024)) 328 | softmax2_pre_activation_matmul = self.softmax2_pre_activation_matmul(avgpool0_reshape) 329 | softmax2 = self.softmax2(softmax2_pre_activation_matmul) 330 | return softmax2 -------------------------------------------------------------------------------- /neural_dream/models/inception/inception_layer_names.py: -------------------------------------------------------------------------------- 1 | def inception_layer_names(model_name='5h'): 2 | if model_name == '5h': 3 | hookable_layers = ['conv2d0_pre_relu_conv', 'conv2d0', 'maxpool0', 'conv2d1_pre_relu_conv', 'conv2d1', 'conv2d2_pre_relu_conv', 'conv2d2', 'maxpool1', \ 4 | 'mixed3a_1x1_pre_relu_conv', 'mixed3a_3x3_bottleneck_pre_relu_conv', 'mixed3a_5x5_bottleneck_pre_relu_conv', 'mixed3a_pool', 'mixed3a_1x1', \ 5 | 'mixed3a_3x3_bottleneck', 'mixed3a_5x5_bottleneck', 'mixed3a_pool_reduce_pre_relu_conv', 'mixed3a_3x3_pre_relu_conv', 'mixed3a_5x5_pre_relu_conv', \ 6 | 'mixed3a_pool_reduce', 'mixed3a_3x3', 'mixed3a_5x5', 'mixed3a', 'mixed3b_1x1_pre_relu_conv', 'mixed3b_3x3_bottleneck_pre_relu_conv', \ 7 | 'mixed3b_5x5_bottleneck_pre_relu_conv', 'mixed3b_pool', 'mixed3b_1x1', 'mixed3b_3x3_bottleneck', 'mixed3b_5x5_bottleneck', \ 8 | 'mixed3b_pool_reduce_pre_relu_conv', 'mixed3b_3x3_pre_relu_conv', 'mixed3b_5x5_pre_relu_conv', 'mixed3b_pool_reduce', 'mixed3b_3x3', 'mixed3b_5x5', \ 9 | 'mixed3b', 'maxpool4', 'mixed4a_1x1_pre_relu_conv', 'mixed4a_3x3_bottleneck_pre_relu_conv', 'mixed4a_5x5_bottleneck_pre_relu_conv', 'mixed4a_pool', \ 10 | 'mixed4a_1x1', 'mixed4a_3x3_bottleneck', 'mixed4a_5x5_bottleneck', 'mixed4a_pool_reduce_pre_relu_conv', 'mixed4a_3x3_pre_relu_conv', \ 11 | 'mixed4a_5x5_pre_relu_conv', 'mixed4a_pool_reduce', 'mixed4a_3x3', 'mixed4a_5x5', 'mixed4a', 'mixed4b_1x1_pre_relu_conv', \ 12 | 'mixed4b_3x3_bottleneck_pre_relu_conv', 'mixed4b_5x5_bottleneck_pre_relu_conv', 'mixed4b_pool', 'mixed4b_1x1', 'mixed4b_3x3_bottleneck', \ 13 | 'mixed4b_5x5_bottleneck', 'mixed4b_pool_reduce_pre_relu_conv', 'mixed4b_3x3_pre_relu_conv', 'mixed4b_5x5_pre_relu_conv', 'mixed4b_pool_reduce', \ 14 | 'mixed4b_3x3', 'mixed4b_5x5', 'mixed4b', 'mixed4c_1x1_pre_relu_conv', 'mixed4c_3x3_bottleneck_pre_relu_conv', 'mixed4c_5x5_bottleneck_pre_relu_conv', \ 15 | 'mixed4c_pool', 'mixed4c_1x1', 'mixed4c_3x3_bottleneck', 'mixed4c_5x5_bottleneck', 'mixed4c_pool_reduce_pre_relu_conv', 'mixed4c_3x3_pre_relu_conv', \ 16 | 'mixed4c_5x5_pre_relu_conv', 'mixed4c_pool_reduce', 'mixed4c_3x3', 'mixed4c_5x5', 'mixed4c', 'mixed4d_1x1_pre_relu_conv', \ 17 | 'mixed4d_3x3_bottleneck_pre_relu_conv', 'mixed4d_5x5_bottleneck_pre_relu_conv', 'mixed4d_pool', 'mixed4d_1x1', 'mixed4d_3x3_bottleneck', \ 18 | 'mixed4d_5x5_bottleneck', 'mixed4d_pool_reduce_pre_relu_conv', 'mixed4d_3x3_pre_relu_conv', 'mixed4d_5x5_pre_relu_conv', 'mixed4d_pool_reduce', \ 19 | 'mixed4d_3x3', 'mixed4d_5x5', 'mixed4d', 'mixed4e_1x1_pre_relu_conv', 'mixed4e_3x3_bottleneck_pre_relu_conv', 'mixed4e_5x5_bottleneck_pre_relu_conv', \ 20 | 'mixed4e_pool', 'mixed4e_1x1', 'mixed4e_3x3_bottleneck', 'mixed4e_5x5_bottleneck', 'mixed4e_pool_reduce_pre_relu_conv', 'mixed4e_3x3_pre_relu_conv', \ 21 | 'mixed4e_5x5_pre_relu_conv', 'mixed4e_pool_reduce', 'mixed4e_3x3', 'mixed4e_5x5', 'mixed4e', 'maxpool10', 'mixed5a_1x1_pre_relu_conv', \ 22 | 'mixed5a_3x3_bottleneck_pre_relu_conv', 'mixed5a_5x5_bottleneck_pre_relu_conv', 'mixed5a_pool', 'mixed5a_1x1', 'mixed5a_3x3_bottleneck', \ 23 | 'mixed5a_5x5_bottleneck', 'mixed5a_pool_reduce_pre_relu_conv', 'mixed5a_3x3_pre_relu_conv', 'mixed5a_5x5_pre_relu_conv', 'mixed5a_pool_reduce', \ 24 | 'mixed5a_3x3', 'mixed5a_5x5', 'mixed5a', 'mixed5b_1x1_pre_relu_conv', 'mixed5b_3x3_bottleneck_pre_relu_conv', 'mixed5b_5x5_bottleneck_pre_relu_conv', \ 25 | 'mixed5b_pool', 'mixed5b_1x1', 'mixed5b_3x3_bottleneck', 'mixed5b_5x5_bottleneck', 'mixed5b_pool_reduce_pre_relu_conv', 'mixed5b_3x3_pre_relu_conv', \ 26 | 'mixed5b_5x5_pre_relu_conv', 'mixed5b_pool_reduce', 'mixed5b_3x3', 'mixed5b_5x5', 'mixed5b', 'softmax2_pre_activation_matmul', 'softmax2'] 27 | elif model_name == 'keras-wtop': 28 | hookable_layers = ['conv2d_1', 'batch_normalization_1', 'activation_1', 'conv2d_2', 'batch_normalization_2', 'activation_2', 'conv2d_3', 'batch_normalization_3', \ 29 | 'activation_3', 'max_pooling2d_1', 'conv2d_4', 'batch_normalization_4', 'activation_4', 'conv2d_5', 'batch_normalization_5', 'activation_5', \ 30 | 'max_pooling2d_2', 'conv2d_9', 'conv2d_7', 'average_pooling2d_1', 'conv2d_6', 'batch_normalization_9', 'batch_normalization_7', 'conv2d_12', \ 31 | 'batch_normalization_6', 'activation_9', 'activation_7', 'batch_normalization_12', 'activation_6', 'conv2d_10', 'conv2d_8', 'activation_12', \ 32 | 'batch_normalization_10', 'batch_normalization_8', 'activation_10', 'activation_8', 'conv2d_11', 'batch_normalization_11', 'activation_11', \ 33 | 'mixed0', 'conv2d_16', 'conv2d_14', 'average_pooling2d_2', 'conv2d_13', 'batch_normalization_16', 'batch_normalization_14', 'conv2d_19', \ 34 | 'batch_normalization_13', 'activation_16', 'activation_14', 'batch_normalization_19', 'activation_13', 'conv2d_17', 'conv2d_15', 'activation_19', \ 35 | 'batch_normalization_17', 'batch_normalization_15', 'activation_17', 'activation_15', 'conv2d_18', 'batch_normalization_18', 'activation_18', \ 36 | 'mixed1', 'conv2d_23', 'conv2d_21', 'average_pooling2d_3', 'conv2d_20', 'batch_normalization_23', 'batch_normalization_21', 'conv2d_26', \ 37 | 'batch_normalization_20', 'activation_23', 'activation_21', 'batch_normalization_26', 'activation_20', 'conv2d_24', 'conv2d_22', 'activation_26', \ 38 | 'batch_normalization_24', 'batch_normalization_22', 'activation_24', 'activation_22', 'conv2d_25', 'batch_normalization_25', 'activation_25', \ 39 | 'mixed2', 'conv2d_28', 'conv2d_27', 'max_pooling2d_3', 'batch_normalization_28', 'batch_normalization_27', 'activation_28', 'activation_27', \ 40 | 'conv2d_29', 'batch_normalization_29', 'activation_29', 'conv2d_30', 'batch_normalization_30', 'activation_30', 'mixed3', 'conv2d_35', 'conv2d_32', \ 41 | 'average_pooling2d_4', 'conv2d_31', 'batch_normalization_35', 'batch_normalization_32', 'conv2d_40', 'batch_normalization_31', 'activation_35', \ 42 | 'activation_32', 'batch_normalization_40', 'activation_31', 'conv2d_36', 'conv2d_33', 'activation_40', 'batch_normalization_36', \ 43 | 'batch_normalization_33', 'activation_36', 'activation_33', 'conv2d_37', 'conv2d_34', 'batch_normalization_37', 'batch_normalization_34', \ 44 | 'activation_37', 'activation_34', 'conv2d_38', 'batch_normalization_38', 'activation_38', 'conv2d_39', 'batch_normalization_39', 'activation_39', \ 45 | 'mixed4', 'conv2d_45', 'conv2d_42', 'average_pooling2d_5', 'conv2d_41', 'batch_normalization_45', 'batch_normalization_42', 'conv2d_50', \ 46 | 'batch_normalization_41', 'activation_45', 'activation_42', 'batch_normalization_50', 'activation_41', 'conv2d_46', 'conv2d_43', 'activation_50', \ 47 | 'batch_normalization_46', 'batch_normalization_43', 'activation_46', 'activation_43', 'conv2d_47', 'conv2d_44', 'batch_normalization_47', \ 48 | 'batch_normalization_44', 'activation_47', 'activation_44', 'conv2d_48', 'batch_normalization_48', 'activation_48', 'conv2d_49', \ 49 | 'batch_normalization_49', 'activation_49', 'mixed5', 'conv2d_55', 'conv2d_52', 'average_pooling2d_6', 'conv2d_51', 'batch_normalization_55', \ 50 | 'batch_normalization_52', 'conv2d_60', 'batch_normalization_51', 'activation_55', 'activation_52', 'batch_normalization_60', 'activation_51', \ 51 | 'conv2d_56', 'conv2d_53', 'activation_60', 'batch_normalization_56', 'batch_normalization_53', 'activation_56', 'activation_53', 'conv2d_57', \ 52 | 'conv2d_54', 'batch_normalization_57', 'batch_normalization_54', 'activation_57', 'activation_54', 'conv2d_58', 'batch_normalization_58', \ 53 | 'activation_58', 'conv2d_59', 'batch_normalization_59', 'activation_59', 'mixed6', 'conv2d_65', 'conv2d_62', 'average_pooling2d_7', 'conv2d_61', \ 54 | 'batch_normalization_65', 'batch_normalization_62', 'conv2d_70', 'batch_normalization_61', 'activation_65', 'activation_62', \ 55 | 'batch_normalization_70', 'activation_61', 'conv2d_66', 'conv2d_63', 'activation_70', 'batch_normalization_66', 'batch_normalization_63', \ 56 | 'activation_66', 'activation_63', 'conv2d_67', 'conv2d_64', 'batch_normalization_67', 'batch_normalization_64', 'activation_67', 'activation_64', \ 57 | 'conv2d_68', 'batch_normalization_68', 'activation_68', 'conv2d_69', 'batch_normalization_69', 'activation_69', 'mixed7', 'conv2d_73', 'conv2d_71', \ 58 | 'max_pooling2d_4', 'batch_normalization_73', 'batch_normalization_71', 'activation_73', 'activation_71', 'conv2d_74', 'conv2d_72', \ 59 | 'batch_normalization_74', 'batch_normalization_72', 'activation_74', 'activation_72', 'conv2d_75', 'batch_normalization_75', 'activation_75', \ 60 | 'conv2d_76', 'batch_normalization_76', 'activation_76', 'mixed8', 'conv2d_81', 'conv2d_78', 'average_pooling2d_8', 'conv2d_77', \ 61 | 'batch_normalization_81', 'batch_normalization_78', 'conv2d_85', 'batch_normalization_77', 'activation_81', 'activation_78', \ 62 | 'batch_normalization_85', 'activation_77', 'conv2d_82', 'conv2d_79', 'conv2d_80', 'activation_85', 'batch_normalization_82', \ 63 | 'batch_normalization_79', 'batch_normalization_80', 'activation_82', 'activation_79', 'activation_80', 'conv2d_83', 'conv2d_84', 'mixed9_0', \ 64 | 'batch_normalization_83', 'batch_normalization_84', 'activation_83', 'activation_84', 'concatenate_1', 'mixed9', 'conv2d_90', 'conv2d_87', \ 65 | 'average_pooling2d_9', 'conv2d_86', 'batch_normalization_90', 'batch_normalization_87', 'conv2d_94', 'batch_normalization_86', 'activation_90', \ 66 | 'activation_87', 'batch_normalization_94', 'activation_86', 'conv2d_91', 'conv2d_88', 'conv2d_89', 'activation_94', 'batch_normalization_91', \ 67 | 'batch_normalization_88', 'batch_normalization_89', 'activation_91', 'activation_88', 'activation_89', 'conv2d_92', 'conv2d_93', 'mixed9_1', \ 68 | 'batch_normalization_92', 'batch_normalization_93', 'activation_92', 'activation_93', 'concatenate_2', 'mixed10', 'avg_pool', 'predictions', \ 69 | 'predictions_activation'] 70 | return hookable_layers -------------------------------------------------------------------------------- /neural_dream/models/resnet/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ProGamerGov/neural-dream/bd66447fefa5a05c7ee9861e16813602f9521892/neural_dream/models/resnet/__init__.py -------------------------------------------------------------------------------- /neural_dream/models/resnet/resnet_layer_names.py: -------------------------------------------------------------------------------- 1 | def resnet_layer_names(mode): 2 | if mode == '50_1by2_nsfw': 3 | hookable_layers = ['conv_1', 'bn_1', 'conv_stage0_block0_branch2a', 'conv_stage0_block0_proj_shortcut', 'bn_stage0_block0_branch2a', 'bn_stage0_block0_proj_shortcut', \ 4 | 'conv_stage0_block0_branch2b', 'bn_stage0_block0_branch2b', 'conv_stage0_block0_branch2c', 'bn_stage0_block0_branch2c', 'conv_stage0_block1_branch2a', \ 5 | 'bn_stage0_block1_branch2a', 'conv_stage0_block1_branch2b', 'bn_stage0_block1_branch2b', 'conv_stage0_block1_branch2c', 'bn_stage0_block1_branch2c', \ 6 | 'conv_stage0_block2_branch2a', 'bn_stage0_block2_branch2a', 'conv_stage0_block2_branch2b', 'bn_stage0_block2_branch2b', 'conv_stage0_block2_branch2c', \ 7 | 'bn_stage0_block2_branch2c', 'conv_stage1_block0_proj_shortcut', 'conv_stage1_block0_branch2a', 'bn_stage1_block0_proj_shortcut', 'bn_stage1_block0_branch2a', \ 8 | 'conv_stage1_block0_branch2b', 'bn_stage1_block0_branch2b', 'conv_stage1_block0_branch2c', 'bn_stage1_block0_branch2c', 'conv_stage1_block1_branch2a', \ 9 | 'bn_stage1_block1_branch2a', 'conv_stage1_block1_branch2b', 'bn_stage1_block1_branch2b', 'conv_stage1_block1_branch2c', 'bn_stage1_block1_branch2c', \ 10 | 'conv_stage1_block2_branch2a', 'bn_stage1_block2_branch2a', 'conv_stage1_block2_branch2b', 'bn_stage1_block2_branch2b', 'conv_stage1_block2_branch2c', \ 11 | 'bn_stage1_block2_branch2c', 'conv_stage1_block3_branch2a', 'bn_stage1_block3_branch2a', 'conv_stage1_block3_branch2b', 'bn_stage1_block3_branch2b', \ 12 | 'conv_stage1_block3_branch2c', 'bn_stage1_block3_branch2c', 'conv_stage2_block0_proj_shortcut', 'conv_stage2_block0_branch2a', 'bn_stage2_block0_proj_shortcut', \ 13 | 'bn_stage2_block0_branch2a', 'conv_stage2_block0_branch2b', 'bn_stage2_block0_branch2b', 'conv_stage2_block0_branch2c', 'bn_stage2_block0_branch2c', \ 14 | 'conv_stage2_block1_branch2a', 'bn_stage2_block1_branch2a', 'conv_stage2_block1_branch2b', 'bn_stage2_block1_branch2b', 'conv_stage2_block1_branch2c', \ 15 | 'bn_stage2_block1_branch2c', 'conv_stage2_block2_branch2a', 'bn_stage2_block2_branch2a', 'conv_stage2_block2_branch2b', 'bn_stage2_block2_branch2b', \ 16 | 'conv_stage2_block2_branch2c', 'bn_stage2_block2_branch2c', 'conv_stage2_block3_branch2a', 'bn_stage2_block3_branch2a', 'conv_stage2_block3_branch2b', \ 17 | 'bn_stage2_block3_branch2b', 'conv_stage2_block3_branch2c', 'bn_stage2_block3_branch2c', 'conv_stage2_block4_branch2a', 'bn_stage2_block4_branch2a', \ 18 | 'conv_stage2_block4_branch2b', 'bn_stage2_block4_branch2b', 'conv_stage2_block4_branch2c', 'bn_stage2_block4_branch2c', 'conv_stage2_block5_branch2a', \ 19 | 'bn_stage2_block5_branch2a', 'conv_stage2_block5_branch2b', 'bn_stage2_block5_branch2b', 'conv_stage2_block5_branch2c', 'bn_stage2_block5_branch2c', \ 20 | 'conv_stage3_block0_proj_shortcut', 'conv_stage3_block0_branch2a', 'bn_stage3_block0_proj_shortcut', 'bn_stage3_block0_branch2a', 'conv_stage3_block0_branch2b', \ 21 | 'bn_stage3_block0_branch2b', 'conv_stage3_block0_branch2c', 'bn_stage3_block0_branch2c', 'conv_stage3_block1_branch2a', 'bn_stage3_block1_branch2a', \ 22 | 'conv_stage3_block1_branch2b', 'bn_stage3_block1_branch2b', 'conv_stage3_block1_branch2c', 'bn_stage3_block1_branch2c', 'conv_stage3_block2_branch2a', \ 23 | 'bn_stage3_block2_branch2a', 'conv_stage3_block2_branch2b', 'bn_stage3_block2_branch2b', 'conv_stage3_block2_branch2c', 'bn_stage3_block2_branch2c', 'fc_nsfw_1', \ 24 | 'eltwise_stage0_block0', 'eltwise_stage0_block1', 'eltwise_stage0_block2', 'eltwise_stage1_block0', 'eltwise_stage1_block1', 'eltwise_stage1_block2', \ 25 | 'eltwise_stage1_block3', 'eltwise_stage2_block0', 'eltwise_stage2_block1', 'eltwise_stage2_block2', 'eltwise_stage2_block3', 'eltwise_stage2_block4', \ 26 | 'eltwise_stage2_block5', 'eltwise_stage3_block0', 'eltwise_stage3_block1', 'eltwise_stage3_block2'] 27 | return hookable_layers --------------------------------------------------------------------------------