├── models ├── .gitignore └── download_models.sh ├── images ├── 1-content.png ├── 1-style.jpg ├── 2-content.jpg ├── 2-style.jpg ├── 3-content.jpg ├── 3-style.jpg ├── 4-content.jpg ├── 4-style.png ├── 5-content.jpg ├── 5-style.jpg ├── 6-content.jpg ├── out │ ├── andrew_oil.png │ ├── catriona.png │ ├── dubnation.png │ ├── shh_janelle.png │ ├── andrew_transfer.png │ ├── gloomy_bridge.png │ ├── golden_starry.png │ ├── catriona_transfer.png │ ├── janelle_transfer.png │ └── shh_janelle_preserve.png ├── janelle_transfer.png └── janelle_transfer_preserve.png └── README.md /models/.gitignore: -------------------------------------------------------------------------------- 1 | *.pth 2 | -------------------------------------------------------------------------------- /images/1-content.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/1-content.png -------------------------------------------------------------------------------- /images/1-style.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/1-style.jpg -------------------------------------------------------------------------------- /images/2-content.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/2-content.jpg -------------------------------------------------------------------------------- /images/2-style.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/2-style.jpg -------------------------------------------------------------------------------- /images/3-content.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/3-content.jpg -------------------------------------------------------------------------------- /images/3-style.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/3-style.jpg -------------------------------------------------------------------------------- /images/4-content.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/4-content.jpg -------------------------------------------------------------------------------- /images/4-style.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/4-style.png -------------------------------------------------------------------------------- /images/5-content.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/5-content.jpg -------------------------------------------------------------------------------- /images/5-style.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/5-style.jpg -------------------------------------------------------------------------------- /images/6-content.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/6-content.jpg -------------------------------------------------------------------------------- /images/out/andrew_oil.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/out/andrew_oil.png -------------------------------------------------------------------------------- /images/out/catriona.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/out/catriona.png -------------------------------------------------------------------------------- /images/out/dubnation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/out/dubnation.png -------------------------------------------------------------------------------- /images/janelle_transfer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/janelle_transfer.png -------------------------------------------------------------------------------- /images/out/shh_janelle.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/out/shh_janelle.png -------------------------------------------------------------------------------- /images/out/andrew_transfer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/out/andrew_transfer.png -------------------------------------------------------------------------------- /images/out/gloomy_bridge.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/out/gloomy_bridge.png -------------------------------------------------------------------------------- /images/out/golden_starry.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/out/golden_starry.png -------------------------------------------------------------------------------- /images/out/catriona_transfer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/out/catriona_transfer.png -------------------------------------------------------------------------------- /images/out/janelle_transfer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/out/janelle_transfer.png -------------------------------------------------------------------------------- /images/out/shh_janelle_preserve.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/out/shh_janelle_preserve.png -------------------------------------------------------------------------------- /models/download_models.sh: -------------------------------------------------------------------------------- 1 | cd models 2 | wget -c https://s3-us-west-2.amazonaws.com/jcjohns-models/vgg19-d01eb7cb.pth 3 | cd .. 4 | -------------------------------------------------------------------------------- /images/janelle_transfer_preserve.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rrmina/neural-style-pytorch/HEAD/images/janelle_transfer_preserve.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # neural-style: Neural Style in Pytorch! :art: 2 | 3 | An implementation of the neural style in PyTorch! This notebook implements [Image Style Transfer Using Convolutional Neural Networks](https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Gatys_Image_Style_Transfer_CVPR_2016_paper.pdf) by Leon Gatys, Alexander Ecker, and Matthias Bethge. Color preservation/Color transfer is based on the 2nd approach of discussed in [Preserving Color in Neural Artistic Style Transfer](https://arxiv.org/pdf/1606.05897.pdf) by Leon Gatys, Matthias Betge, Aaron Hertzmann, and Eli Schetman. 4 | 5 | This implementation is inspired by the implementations of: 6 | * Anish Athalye: [Neural Style in Tensorflow](https://github.com/anishathalye/neural-style), 7 | * Justin Johnson: [Neural Style in Torch](https://github.com/jcjohnson/neural-style), and 8 | * ProGamerGov: [Neural Style in PyTorch](https://github.com/ProGamerGov/neural-style-pt) 9 | 10 | The [original caffe pretrained weights of VGG19](https://github.com/jcjohnson/pytorch-vgg) were used for this implementation, instead of the pretrained VGG19's in PyTorch's model zoo. 11 | 12 | ## Examples: Style Transfer 13 | ### Catriona Gray and Woman I by Willem de Kooning 14 | ![Catriona](https://i.imgur.com/Cx7WEZo.jpg) 15 | 16 | ### Janelle Monae and Starry Night by Vincent van Gogh 17 | ![Janelle Monae](https://i.imgur.com/WWq6I1U.jpg) 18 | 19 | ### Andrew Y. Ng and Oil Painting of a Girl in Rain 20 | ![AndrewYNg](https://i.imgur.com/cO9YdZI.jpg) 21 | 22 | ### Style Transfers of Golden Bridge 23 | ![dubnation](https://i.imgur.com/K2eCqjA.jpg) 24 | 25 | ### [Some Old Man](https://www.google.com/search?q=philippine+idiot&source=lnms&tbm=isch&sa=X&ved=0ahUKEwi0p_PDqK3fAhVIabwKHRWeCPQQ_AUIDigB&biw=2560&bih=1311) + Increasing Style Weights of Starry Night 26 | ![Philippine Idiot](https://i.imgur.com/bK8bnCN.jpg) 27 | 28 | ## Examples: Style Transfer while Preserving the original color 29 | ### Janelle Monae and Starry Night by Vincent van Gogh + Preserve Original Color 30 | ![Janelle Monae Preserve](https://i.imgur.com/asrUS0A.jpg) 31 | 32 | ## Requirements 33 | `NOTE`: For `Google-Colab users` - All data files and dependencies can be installed by running the uppermost cell of the notebook! See `Usage`! 34 | 35 | ### Data Files 36 | * [Pre-trained VGG19 network weights](https://web.eecs.umich.edu/~justincj/models/vgg19-d01eb7cb.pth) - put it in `models/` directory 37 | * [torchvision](https://pytorch.org/) - `torchvision.models` contains the VGG19 model skeleton 38 | 39 | ### Dependecies 40 | * [PyTorch](https://pytorch.org/) 41 | * [NumPy](https://www.scipy.org/install.html) 42 | * [Jupyter](http://jupyter.org/install) 43 | * [opencv2](https://matplotlib.org/users/installing.html) 44 | * [Copy](https://docs.python.org/3/library/copy.html) 45 | 46 | ## Usage 47 | If you don't have a GPU, you may want to run the notebook in [Google Colab](https://colab.research.google.com/github/rrmina/neural-style-pytorch/blob/master/neural_style_preserve_color.ipynb)! Colab is a cloud-GPU service with an interface similar to Jupyter notebook. A separate instruction is included to get started with Colab. 48 | 49 | ### Local GPU 50 | After installing the dependencies, run `models/download_model.sh` script to download the pretrained VGG19 weights. 51 | ``` 52 | sh models/download_models.sh 53 | ``` 54 | 55 | Codes are implemented inside the `neural_style.ipynb` notebook. Jupyter notebook environment is needed to run notebook. 56 | ``` 57 | jupyter notebook 58 | ``` 59 | 60 | ### Google Colab 61 | The included notebook file is a `Google-Colab-ready` notebook! Uncomment and run the first cell to download the demo pictures, and VGG19 weights. It will also install the dependencies (i.e. PyTorch and torchvision). 62 | ``` 63 | # Download VGG19 Model 64 | !wget -c https://web.eecs.umich.edu/~justincj/models/vgg19-d01eb7cb.pth 65 | !mkdir models 66 | !cp vgg19-d01eb7cb.pth models/ 67 | 68 | # Download Images 69 | !wget -c https://github.com/iamRusty/neural-style-pytorch/archive/master.zip 70 | !unzip -q master.zip 71 | !mkdir images 72 | !cp neural-style-pytorch-master/images/1-content.png images 73 | !cp neural-style-pytorch-master/images/1-style.jpg images 74 | ``` 75 | ## Options 76 | ### Image 77 | * `MAX_IMAGE_SIZE`: sets the max dimension of height or weight. Bigger GPU memory is needed to run larger images. Default is `512`px. 78 | * `INIT_IMAGE`: sets the initial image file to either `'random'` or `'content'`. Default is `random` which initializes a noise image. Content copies a resized content image, giving free optimization of content loss! 79 | * `CONTENT_PATH`: path of the content image 80 | * `STYLE_PATH`: path of the style image 81 | * `PRESERVE_COLOR`: determines whether to preserve the color of the content image. `True` preserves the color of the content image. Default value is `False` 82 | * `PIXEL_CLIP`: determines whether to clip the resulting image. `True` clips the pixel values to [0, 255]. Default value is `True` 83 | 84 | ### Optimizer 85 | * `OPTIMIZER`: sets the optimizer to either 'adam' or 'lbfgs'. Default optimizer is `Adam` with learning rate of 10. L-BFGS was used in the original (matlab) implementation of the reference paper. 86 | * `ADAM_LR`: learning rate of the adam optimizer. Default is `1e1` 87 | * `CONTENT_WEIGHT`: Multiplier weight of the loss between content representations and the generated image. Default is `5e0` 88 | * `STYLE_WEIGHT`: Multiplier weight of the loss between style representations and the generated image. Default is `1e2` 89 | * `TV_WEIGHT`: Multiplier weight of the [Total Variation Denoising](https://github.com/jcjohnson/neural-style/issues/302). Default is `1e-3` 90 | * `NUM_ITER`: Iterations of the style transfer. Default is `500` 91 | * `SHOW_ITER`: Number of iterations before showing and saving the generated image. Default is `100` 92 | 93 | ### Model 94 | * `VGG19_PATH` = path of VGG19 Pretrained weights. Default is `'models/vgg19-d01eb7cb.pth'` 95 | * `POOL`: Defines which pooling layer to use. The reference paper suggests using average pooling! Default is `'max'` 96 | 97 | ## Todo! 98 | * Multiple Style blending 99 | * High-res Style Transfer 100 | * Color-preserving Style Transfer 101 | --------------------------------------------------------------------------------