├── deepdanbooru_model └── placeholder.txt ├── stylegan2_animeface_model └── placeholder.txt ├── input.png ├── shaw.png ├── example_data ├── latents.npy └── gradcam_tagmaps.png ├── tagged_dlatents ├── tag_dirs.pkl ├── tags_use.pkl ├── tag_distrib.pkl └── tag_dirs_cont.pkl ├── .gitmodules ├── LICENSE └── README.md /deepdanbooru_model/placeholder.txt: -------------------------------------------------------------------------------- 1 | put deepdanbooru model here -------------------------------------------------------------------------------- /stylegan2_animeface_model/placeholder.txt: -------------------------------------------------------------------------------- 1 | put stylegan2 anime face model here -------------------------------------------------------------------------------- /input.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ashishpatel26/AnimeFaceNotebooks/master/input.png -------------------------------------------------------------------------------- /shaw.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ashishpatel26/AnimeFaceNotebooks/master/shaw.png -------------------------------------------------------------------------------- /example_data/latents.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ashishpatel26/AnimeFaceNotebooks/master/example_data/latents.npy -------------------------------------------------------------------------------- /tagged_dlatents/tag_dirs.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ashishpatel26/AnimeFaceNotebooks/master/tagged_dlatents/tag_dirs.pkl -------------------------------------------------------------------------------- /tagged_dlatents/tags_use.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ashishpatel26/AnimeFaceNotebooks/master/tagged_dlatents/tags_use.pkl -------------------------------------------------------------------------------- /tagged_dlatents/tag_distrib.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ashishpatel26/AnimeFaceNotebooks/master/tagged_dlatents/tag_distrib.pkl -------------------------------------------------------------------------------- /example_data/gradcam_tagmaps.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ashishpatel26/AnimeFaceNotebooks/master/example_data/gradcam_tagmaps.png -------------------------------------------------------------------------------- /tagged_dlatents/tag_dirs_cont.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ashishpatel26/AnimeFaceNotebooks/master/tagged_dlatents/tag_dirs_cont.pkl -------------------------------------------------------------------------------- /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "DeepDanbooru"] 2 | path = DeepDanbooru 3 | url = https://github.com/KichangKim/DeepDanbooru.git 4 | [submodule "stylegan2"] 5 | path = stylegan2 6 | url = https://github.com/NVlabs/stylegan2 7 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Lorenz Diener 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | --- 24 | 25 | This repository links to two submodules: Stylegan 2 and DeepDanbooru. 26 | The Stylegan2 code is (c) NVidia and is distributed under the Nvidia Source 27 | Code License-NC, while the DeepDanbooru code is (c) Kichang Kim and 28 | distributed under the MIT license. Please see the appropriate repositories 29 | for details. 30 | 31 | --- 32 | 33 | This repository contains code that downloads pre-trained machine learning 34 | model weights. The weights for the Stylegan 2 model are (c) Skyli0n and Gwern 35 | and are provided under the Creative Commons Zero license. The weights for 36 | the DeepDanbooru model are (c) Kichang Kim and are provided under the MIT 37 | license. 38 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # AnimeFaceNotebooks 2 | Notebooks and some data for playing with animeface stylegan2 and deepdanbooru. Versions for colab included (colab/ directory, or links below) 3 | 4 | ## Stylegan2 Anime Face model 5 | This notebook has a few simple functions to use stylegan2s projection code to project input images (quality of results varies, but honestly surprisingly good), and sliders for interactively modifying a few tags. The directions in dlatent space were obtained by generating a lot of faces, tagging them with deepdanbooru, and then finding a regression line in disentangled latent space fit to the tag confidence values from deepdanbooru. This version uses Lasso regression to try and exploit sparsity in the disentangled latent space. There's some development notes in the notebook, and code that you can use to calculate your own directions. latents/dlatents + tags for 100000 generated images are included. 6 | 7 | Open In Colab 8 | 9 | Example Video: http://www.youtube.com/watch?v=GRG6czAZql0 10 | 11 | [![Stylegan2 Anime Face Interactive Modification](https://img.youtube.com/vi/GRG6czAZql0/0.jpg)](http://www.youtube.com/watch?v=GRG6czAZql0) 12 | 13 | ## Deepdanbooru + GradCam 14 | This notebook as functions for using DeepDanbooru for tagging, for calculating gradcam maps and post-processing them into activation maps for specific tags, and for then visualizing these maps. 15 | 16 | Open In Colab 17 | 18 | Screenshot: 19 | 20 | ![Shaw (arknights) with tail highlited using gradcam](https://raw.githubusercontent.com/halcy/AnimeFaceNotebooks/master/example_data/gradcam_tagmaps.png) 21 | --------------------------------------------------------------------------------