├── README.md
├── data
├── models
│ ├── haarcascade_frontalface_default.xml
│ └── keepme
├── screen_2dfilter.png
├── screen_classif_2D.png
├── screen_dico_2D.png
├── screen_face_detection.png
├── screen_fft.png
├── screen_ot.png
├── screen_spectrum.png
├── screen_style_transfer.png
├── screen_tv.png
└── styles
│ ├── afremov.jpg
│ ├── antimonocromatismo.jpg
│ ├── asheville.jpg
│ ├── baiser.jpg
│ ├── brushstrokes.jpg
│ ├── bubbles.jpg
│ ├── contrast_of_forms.jpg
│ ├── en_campo_gris.jpg
│ ├── hosi.jpg
│ ├── in2.jpg
│ ├── joconde.jpg
│ ├── joconde.webp
│ ├── la_muse.jpg
│ ├── miro.jpg
│ ├── mondrian.jpg
│ ├── monet1.jpg
│ ├── monet2.jpg
│ ├── nikopol.jpg
│ ├── picasso_seated_nude_hr.jpg
│ ├── picasso_self_portrait.jpg
│ ├── scene_de_rue.jpg
│ ├── sketch.png
│ ├── trial.jpg
│ ├── woman_in_peasant_dress_cropped.jpg
│ ├── woman_with_hat_matisse.jpg
│ └── zombie.jpg
├── demo.py
├── demos.ini
└── demos
├── demo_2d_filter.py
├── demo_classif_2D.py
├── demo_color_transfer.py
├── demo_dicolearn.py
├── demo_face_detection.py
├── demo_fft.py
├── demo_optimal_transport.py
├── demo_spectrum.py
├── demo_style_transfer.py
├── demo_tv.py
└── model_style_transfer.py
/README.md:
--------------------------------------------------------------------------------
1 | PYTHON Demos
2 | ============
3 |
4 | This repository contains several python3 scripts demonstrations. They all have
5 | MIT license.
6 |
7 | # How to run
8 |
9 | All scripts from the demos are available in the `demos` folder.
10 |
11 | One can run easily all demos using the `demo.py` script in the root folder.
12 |
13 | The list of available demos is given by:
14 | ```bash
15 | python3 demo.py -l
16 | ```
17 |
18 | To run the demo you can execute:
19 | ```bash
20 | python3 demo.py [demo_name]
21 | ```
22 |
23 | To print the list of dependencies for a given demo you can run:
24 | ```bash
25 | python3 demo.py -d [demo_name]
26 | ```
27 | you can store it in a text file and install it with pip.
28 |
29 | # Demos
30 |
31 | ## Total Variation on webcam
32 |
33 | In this demonstration we perform online Total Variation regularization on the
34 | video captured on the webcam.
35 |
36 | Screenshot:
37 |
38 | 
39 |
40 | Shortcuts:
41 |
42 | * q : Quit demo
43 | * s : Save screen to png
44 | * s : Save screen to png
45 | * Space : Do full image TV
46 | * h : Do half image TV (screenshot)
47 | * + : Make TV regularization stronger
48 | * - : Make TV regularization weaker
49 | * n : Add salt and pepper noise
50 | * b : Make noise more aggressive
51 | * , : Make noise less aggressive
52 |
53 | Dependencies:
54 |
55 | This demo uses [opencv-python](https://github.com/skvark/opencv-python) for
56 | webcam access and visualization and [prox_tv](https://github.com/albarji/proxTV) for total variation proximal
57 | operator.
58 |
59 | ## Real time audio spectrum
60 |
61 | In this demonstration we plot in real time the audio spectrum (and the time
62 | frequency analysis) of the microphone.
63 |
64 | 
65 |
66 | Shortcuts:
67 |
68 | * q or Esc : Quit demo
69 | * Space : Pause/unpause demo
70 | * r : Reset time
71 | * + or N : Multiply nfft by 2 (zoom in to lower frequencies)
72 | * - or n : Divide nfft by 2 (zoom out)
73 | * W : Multiply time window size by 2
74 | * w : Divide time window size by 2
75 | * P/p : Change scale of power spectrum (up/down)
76 | * S/s : Change scale of power spectrum (up/down)
77 |
78 | Dependencies:
79 |
80 | This demo uses [pygame](https://www.pygame.org/) for visualization and
81 | [PyAudio](http://people.csail.mit.edu/hubert/pyaudio/) for microphone recording.
82 |
83 | ## Real time 2D FFT of webcam
84 |
85 | In this demonstration we perform online 2D fft of a webcam input. Both the
86 | absolute value and the log of the absolute values are displayed and the 0
87 | frequency is centered on the window (fftshift).
88 |
89 | Screenshot:
90 |
91 | 
92 |
93 | Shortcuts:
94 |
95 | * q : Quit demo
96 | * s : Save screen to png
97 | * Space : Pause FFT
98 | * w : Active or deactivate windowing
99 |
100 |
101 | Dependencies:
102 |
103 | This demo uses [opencv-python](https://github.com/skvark/opencv-python) for
104 | webcam access and visualization.
105 |
106 |
107 |
108 | ## Real time 2D filtering of webcam
109 |
110 | In this demonstration we perform online 2D filtering of a webcam input. Both the
111 | original, filtered and FFT of the filtered video with 0
112 | frequency centered on the window (fftshift) are displayed.
113 |
114 | Screenshot:
115 |
116 | 
117 |
118 | Shortcuts:
119 |
120 | * q : Quit demo
121 | * s : Save screen to png
122 | * Space : Apply/deactivate filter
123 | * f : Change filter (average, high pass, Prewitt, Sobel, median, ...)
124 | * + - : Change cutoff frequency of filter (or its size)
125 | * w : Active or deactivate windowing
126 |
127 | Dependencies:
128 |
129 | This demo uses [opencv-python](https://github.com/skvark/opencv-python) for
130 | webcam access and visualization.
131 |
132 | ## 2D classification demo
133 |
134 | In this demonstration we illustrate the decision function and update of 2D
135 | classifiers when adding training samples from negative or positive classes. We
136 | provide decision functions for Linear and Gaussian Kernel SVMs.
137 |
138 | 
139 |
140 | * q : Quit demo
141 | * left click : Add samples from red class
142 | * right click : Add samples from blue class
143 | * c : Clear training data
144 | * Space : Show/hide decision function
145 | * m : Change classifier (Linear/Gaussian SVM)
146 | * Up : Make Gaussian Gamma parameter larger
147 | * Down : Make Gaussian Gamma parameter smaller
148 | * s : save screenshot
149 |
150 | Dependencies:
151 |
152 | This demo uses [pygame](https://www.pygame.org/) for visualization and
153 | [Scikit-learn](https://scikit-learn.org/) for classification.
154 |
155 | ## 2D Dictionary learning demo
156 |
157 | In this demonstration we illustrate dictionary learning on 2D data. The user cla
158 | click on the window to add samples (left click) and to remove samples (right
159 | click). The dictionary is learned in real time and the atoms are displayed.
160 |
161 |
162 |
163 | 
164 |
165 | * q : Quit demo
166 | * left click : Add sample
167 | * right click : Remove sample
168 | * c : Clear training data
169 | * Space : Show/hide dictionary atoms and color samples wrt their representation
170 | * m : Change method (PCA, ICA KMeans, NMF, SparsePCA, DictionaryLearning)
171 | * p : Show projected and reconstructed samples
172 | * Up : Increment the number of atoms
173 | * Down : Decrement the number of atoms
174 | * s : save screenshot
175 |
176 | Dependencies:
177 |
178 | This demo uses [pygame](https://www.pygame.org/) for visualization and
179 | [Scikit-learn](https://scikit-learn.org/) for classification.
180 |
181 |
182 | ## Style Transfer Demo
183 |
184 | In this demonstration we illustrate style transfer on image taken from the
185 | webcam. You can choose among several styles.
186 |
187 | The implementation for style transfer has been shamelessly copied from
188 | [Pytorch_WCT](https://github.com/irasin/Pytorch_WCT) that is an implementation
189 | of the paper [Universal Style Transfer via Feature Transforms](https://arxiv.org/pdf/1705.08086.pdf).
190 |
191 | Computational time for transfer takes a few seconds on a laptop with no GPU and freeze the windows.
192 |
193 | 
194 |
195 |
196 | * q : Quit demo
197 | * p : Take picture/unfreeze webcam
198 | * s : Switch target style
199 | * Space : Apply selected style
200 | * w : Save images to disk (folder out/)
201 | * r : Run transfer for all styles and save to file (takes forever)
202 |
203 | Dependencies:
204 |
205 | This demo uses [opencv-python](https://github.com/skvark/opencv-python) for
206 | webcam access and visualization and [torch/torchvision](https://pytorch.org/) as deep learning
207 | framework.
208 |
209 | ## Discrete Optimal Transport demo
210 |
211 | In this demonstration we plot in real time the solution of optimal transport (OT)
212 | between discrete distributions. The samples from source can be added by a left
213 | clock and samples from target distribution can be added by a right click.
214 |
215 |
216 | 
217 |
218 | Shortcuts:
219 |
220 | * q or Esc : Quit demo
221 | * Space : Show/Hide the optimal transport matrix
222 | * c : Clear all samples
223 | * r : Activate/deactivate entropic regularization
224 | * p or m : Increase or decrease regularization parameter
225 | * t : Rotate the target distribution
226 | * e : remove some samples so that they have equal number
227 | * d : change the metric in OT loss
228 |
229 | Dependencies:
230 |
231 | This demo uses [pygame](https://www.pygame.org/) for visualization and
232 | [POT](https://github.com/PythonOT/POT) for computing the solution of
233 | the OT problem.
234 |
235 | ## Face detection
236 |
237 | In this demo we use a Haar Cascade Classifier to detect faces and show the areas where a face was detected. We use the default face classifier from OpenCV that detect only frontal faces.
238 |
239 | 
240 |
241 | * q : Quit demo
242 | * s : Save screenshot
243 |
244 |
245 | This demo uses [opencv-python](https://github.com/skvark/opencv-python) for
246 | webcam access and visualization.
247 |
248 |
249 | # Installing dependencies
250 |
251 | There is an important number of dependencies to run all the demos. Each script
252 | has its own dependencies.
253 | To print the list of dependencies for a given demo you can run:
254 | ```bash
255 | python3 demo.py -d [demo_name]
256 | ```
257 |
258 |
259 |
260 |
261 |
--------------------------------------------------------------------------------
/data/models/keepme:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/models/keepme
--------------------------------------------------------------------------------
/data/screen_2dfilter.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/screen_2dfilter.png
--------------------------------------------------------------------------------
/data/screen_classif_2D.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/screen_classif_2D.png
--------------------------------------------------------------------------------
/data/screen_dico_2D.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/screen_dico_2D.png
--------------------------------------------------------------------------------
/data/screen_face_detection.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/screen_face_detection.png
--------------------------------------------------------------------------------
/data/screen_fft.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/screen_fft.png
--------------------------------------------------------------------------------
/data/screen_ot.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/screen_ot.png
--------------------------------------------------------------------------------
/data/screen_spectrum.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/screen_spectrum.png
--------------------------------------------------------------------------------
/data/screen_style_transfer.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/screen_style_transfer.png
--------------------------------------------------------------------------------
/data/screen_tv.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/screen_tv.png
--------------------------------------------------------------------------------
/data/styles/afremov.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/afremov.jpg
--------------------------------------------------------------------------------
/data/styles/antimonocromatismo.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/antimonocromatismo.jpg
--------------------------------------------------------------------------------
/data/styles/asheville.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/asheville.jpg
--------------------------------------------------------------------------------
/data/styles/baiser.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/baiser.jpg
--------------------------------------------------------------------------------
/data/styles/brushstrokes.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/brushstrokes.jpg
--------------------------------------------------------------------------------
/data/styles/bubbles.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/bubbles.jpg
--------------------------------------------------------------------------------
/data/styles/contrast_of_forms.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/contrast_of_forms.jpg
--------------------------------------------------------------------------------
/data/styles/en_campo_gris.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/en_campo_gris.jpg
--------------------------------------------------------------------------------
/data/styles/hosi.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/hosi.jpg
--------------------------------------------------------------------------------
/data/styles/in2.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/in2.jpg
--------------------------------------------------------------------------------
/data/styles/joconde.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/joconde.jpg
--------------------------------------------------------------------------------
/data/styles/joconde.webp:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/joconde.webp
--------------------------------------------------------------------------------
/data/styles/la_muse.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/la_muse.jpg
--------------------------------------------------------------------------------
/data/styles/miro.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/miro.jpg
--------------------------------------------------------------------------------
/data/styles/mondrian.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/mondrian.jpg
--------------------------------------------------------------------------------
/data/styles/monet1.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/monet1.jpg
--------------------------------------------------------------------------------
/data/styles/monet2.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/monet2.jpg
--------------------------------------------------------------------------------
/data/styles/nikopol.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/nikopol.jpg
--------------------------------------------------------------------------------
/data/styles/picasso_seated_nude_hr.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/picasso_seated_nude_hr.jpg
--------------------------------------------------------------------------------
/data/styles/picasso_self_portrait.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/picasso_self_portrait.jpg
--------------------------------------------------------------------------------
/data/styles/scene_de_rue.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/scene_de_rue.jpg
--------------------------------------------------------------------------------
/data/styles/sketch.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/sketch.png
--------------------------------------------------------------------------------
/data/styles/trial.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/trial.jpg
--------------------------------------------------------------------------------
/data/styles/woman_in_peasant_dress_cropped.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/woman_in_peasant_dress_cropped.jpg
--------------------------------------------------------------------------------
/data/styles/woman_with_hat_matisse.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/woman_with_hat_matisse.jpg
--------------------------------------------------------------------------------
/data/styles/zombie.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rflamary/demos/78c99c95992939b8f7041a404cb08fad608563dc/data/styles/zombie.jpg
--------------------------------------------------------------------------------
/demo.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/python3
2 | # -*- coding: utf-8 -*-
3 | """
4 | MIT License
5 |
6 | Copyright (c) 2020 Rémi Flamary
7 |
8 | Permission is hereby granted, free of charge, to any person obtaining a copy
9 | of this software and associated documentation files (the "Software"), to deal
10 | in the Software without restriction, including without limitation the rights
11 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
12 | copies of the Software, and to permit persons to whom the Software is
13 | furnished to do so, subject to the following conditions:
14 |
15 | The above copyright notice and this permission notice shall be included in all
16 | copies or substantial portions of the Software.
17 |
18 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
19 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
20 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
21 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
22 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
23 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
24 | SOFTWARE.
25 |
26 | """
27 | import argparse
28 | import sys
29 | import subprocess
30 | import os
31 | import configparser
32 |
33 | path = os.path.dirname(os.path.realpath(__file__))
34 |
35 | def load_config(path):
36 | cfg=configparser.ConfigParser()
37 |
38 | cfg.read(path)
39 | dic={}
40 |
41 | for key in cfg:
42 | if not key=='DEFAULT':
43 | dic[key]=cfg[key]
44 |
45 | return dic
46 |
47 | config=load_config(path+'/demos.ini')
48 |
49 | parser = argparse.ArgumentParser(description='Demos executions')
50 | parser.add_argument('-l','--list',action='store_true', help='List all demos')
51 | parser.add_argument('-d','--dependencies',action='store_true', help='List dependencies for current demo')
52 | parser.add_argument('run',default='',nargs='?', help='List all demos')
53 |
54 | args = parser.parse_args()
55 |
56 | if args.list:
57 | for key in config:
58 | print(key)
59 | print('\t{}'.format(config[key]['text']))
60 | elif args.dependencies:
61 | if not args.run in config:
62 | print('Error: "{}" is not a valid demo name. \nExecute demo.py with -l parameter to get a list of demos '.format(args.run))
63 | else:
64 | # execute the script
65 | print(config[args.run]['depends'].replace('\\n','\n'))
66 |
67 | elif args.run:
68 | if not args.run in config:
69 | print('Error: "{}" is not a valid demo name. \nExecute demo.py with -l parameter to get a list of demos '.format(args.run))
70 | else:
71 | # execute the script
72 | subprocess.call(['python3',config[args.run]['file']],cwd=path)
73 | else:
74 | print('No demo name given, choose between the following:')
75 | for key in config:
76 | print(key)
77 | print('\t{}'.format(config[key]['text']))
--------------------------------------------------------------------------------
/demos.ini:
--------------------------------------------------------------------------------
1 |
2 | [spectrum]
3 | text=Online plot of spectrum and spectrogram from microphone
4 | depends=pygame>1.9.1\nPyAudio\nnumpy\nscipy
5 | file=demos/demo_spectrum.py
6 |
7 | [tv]
8 | text=Real time Total Variation demo on webcam
9 | depends=opencv-python\nprox_tv
10 | file=demos/demo_tv.py
11 |
12 | [fft]
13 | text=Real time FFT of webcam
14 | depends=opencv-python
15 | file=demos/demo_fft.py
16 |
17 | [2dfilter]
18 | text=Real time 2D filtering of webcam
19 | depends=opencv-python
20 | file=demos/demo_2d_filter.py
21 |
22 | [ot]
23 | text=Demo of Optimal Transport between discrete dstributions
24 | depends=pygame>1.9.1\nPOT\nnumpy\nscipy
25 | file=demos/demo_optimal_transport.py
26 |
27 | [classif_2D]
28 | text=Demo of 2D classification (LDA, SVM)
29 | depends=numpy\nscipy\nscikit-learn
30 | file=demos/demo_classif_2D.py
31 |
32 | [dicolearn_2D]
33 | text=Demo of 2D Dictionary learning (PCA, ICA, NMF, Sparse DL, KMeans)
34 | depends=numpy\nscipy\nscikit-learn
35 | file=demos/demo_dicolearn_2D.py
36 |
37 | [style_transfer]
38 | text=Demo style transfer between images
39 | depends=numpy\ntorch\ntorchvision\nopencv-python
40 | file=demos/demo_style_transfer.py
41 |
42 | [face_detection]
43 | text=Face detector demo
44 | depends=numpy\nopencv-python
45 | file=demos/demo_face_detection.py
--------------------------------------------------------------------------------
/demos/demo_2d_filter.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 | # -*- coding: utf-8 -*-
3 | """
4 | MIT License
5 |
6 | Copyright (c) 2020 Rémi Flamary
7 |
8 | Permission is hereby granted, free of charge, to any person obtaining a copy
9 | of this software and associated documentation files (the "Software"), to deal
10 | in the Software without restriction, including without limitation the rights
11 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
12 | copies of the Software, and to permit persons to whom the Software is
13 | furnished to do so, subject to the following conditions:
14 |
15 | The above copyright notice and this permission notice shall be included in all
16 | copies or substantial portions of the Software.
17 |
18 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
19 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
20 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
21 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
22 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
23 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
24 | SOFTWARE.
25 |
26 | """
27 |
28 | import numpy as np
29 | import cv2
30 | import prox_tv as ptv
31 | import os
32 | import scipy as sp
33 | import scipy.signal
34 |
35 |
36 | cam=os.getenv("CAMERA")
37 | if cam is None:
38 | cap = cv2.VideoCapture(0)
39 | else:
40 | cap = cv2.VideoCapture(int(cam))
41 |
42 | reg=2e-1
43 | alpha=0.2
44 | pred=0
45 | idscreen=0
46 | loop=True
47 | do_fft=True
48 | half=False
49 | filt=False
50 | idfilt=0
51 |
52 | s=1
53 | cut=0.5
54 |
55 |
56 | font = cv2.FONT_HERSHEY_PLAIN
57 | bottomLeftCornerOfText = (10,30)
58 | fontScale = 2
59 | fontColor = (255,0,0)
60 | lineType = 2
61 |
62 |
63 | ret, frame0 = cap.read()
64 |
65 | n1=frame0.shape[0]
66 | n2=frame0.shape[1]
67 |
68 | win=False
69 |
70 | w1=sp.signal.get_window('parzen',n1)**.2
71 | w2=sp.signal.get_window('parzen',n2)**.2
72 | W=w1[:,None]*w2[None,:]
73 |
74 | F1,F2=np.meshgrid(np.fft.fftfreq(n2),np.fft.fftfreq(n1))
75 | R=np.sqrt(F1**2+F2**2)
76 |
77 |
78 | def filter_img(x,idfilt):
79 |
80 | idfilt= idfilt%13
81 |
82 | if idfilt==0:
83 | apply=lambda x: x
84 | txt= 'No filter'
85 | if idfilt==1:
86 | def apply(x):
87 | h=np.ones((2*s+1,2*s+1))/(2*s+1)**2
88 | return sp.signal.convolve(x,h,'same')
89 | txt= 'Average filter size={}'.format(s*2+1)
90 | if idfilt==2:
91 | def apply(x):
92 | h=np.array([[0,-1,0],[-1,4,-1],[0,-1,0]])/4
93 | return np.maximum(np.minimum(sp.signal.convolve(x,h,'same'),1),0)
94 | txt= 'High pass'
95 | if idfilt==3:
96 | def apply(x):
97 | h=np.array([[0,-1,0],[-1,4,-1],[0,-1,0]])/4
98 | return abs(sp.signal.convolve(x,h,'same'))**.5
99 | txt= 'High pass Abs.'
100 | if idfilt==4:
101 | def apply(x):
102 | h=np.array([[-1,0,1],[-1,0,1],[-1,0,1]])/3
103 | return np.maximum(np.minimum(sp.signal.convolve(x,h,'same'),1),0)
104 | txt= 'Prewitt Horiz. Abs.'
105 | if idfilt==5:
106 | def apply(x):
107 | h=h=np.array([[-1,0,1],[-1,0,1],[-1,0,1]]).T/3
108 | return np.maximum(np.minimum(sp.signal.convolve(x,h,'same'),1),0)
109 | txt= 'Prewitt Vert. Abs.'
110 | if idfilt==6:
111 | def apply(x):
112 | h=h=np.array([[-1,0,1],[-1,0,1],[-1,0,1]]).T/3
113 | return abs(sp.signal.convolve(x,h,'same'))**.5+abs(sp.signal.convolve(x,h.T,'same'))**.5
114 | txt= 'Prewitt Both Abs.'
115 | if idfilt==7:
116 | def apply(x):
117 | h=np.array([[-1,0,1],[-2,0,2],[-1,0,1]])/4
118 | return np.maximum(np.minimum(sp.signal.convolve(x,h,'same'),1),0)
119 | txt= 'Sobel Horiz. Abs.'
120 | if idfilt==8:
121 | def apply(x):
122 | h=h=np.array([[-1,0,1],[-2,0,2],[-1,0,1]]).T/4
123 | return np.maximum(np.minimum(sp.signal.convolve(x,h,'same'),1),0)
124 | txt= 'Sobel Vert. Abs.'
125 | if idfilt==9:
126 | def apply(x):
127 | h=h=np.array([[-1,0,1],[-2,0,2],[-1,0,1]]).T/4
128 | return abs(sp.signal.convolve(x,h,'same'))**.5+abs(sp.signal.convolve(x,h.T,'same'))**.5
129 | txt= 'Sobel Both Abs.'
130 | if idfilt==10:
131 | def apply(x):
132 | return sp.signal.medfilt2d(x,2*s+1)
133 | txt= 'Median filter size={}'.format(s*2+1)
134 | if idfilt==11:
135 | def apply(x):
136 | return np.fft.ifft2(np.fft.fft2(x)*(Rcut)),minv,maxv)
143 | txt= 'Ideal high-pass cutoff={}'.format(cut)
144 | xf=np.zeros_like(x)
145 | for i in range(3):
146 | xf[:,:,i]=apply(x[:,:,i])
147 |
148 | dt=5
149 | xf+=xf[dt:-dt,dt:-dt,:].min()
150 | xf/=xf[dt:-dt,dt:-dt,:].max()
151 |
152 | return xf, txt
153 |
154 |
155 | while(True):
156 | # Capture frame-by-frame
157 | ret, frame = cap.read()
158 |
159 |
160 | frame2=np.array(frame)*1.0/255
161 | #frame2=frame2[:,::-1,:]
162 | #frame2=frame2[::2,::2,:]
163 |
164 | if win:
165 | frame2=frame2*W[:,:,None]
166 |
167 | if do_fft:
168 | frame2,txt=filter_img(frame2,idfilt)
169 | else:
170 | txt='No filter'
171 |
172 |
173 |
174 | F=np.zeros_like(frame2)
175 | for i in range(3):
176 | F[:,:,i]=np.abs(np.fft.fftshift(np.fft.fft2(frame2[:,:,i]),axes=(0,1)))
177 | F=np.log(F+1e-2)
178 | F/=F.max()
179 |
180 |
181 |
182 | #frame2=(frame2+frame0)/2
183 |
184 | frame0=(frame2).astype(np.int8)
185 |
186 |
187 | cv2.putText(frame,txt,
188 | bottomLeftCornerOfText,
189 | font,
190 | fontScale,
191 | fontColor,
192 | lineType)
193 |
194 | #frame2-=frame2.min()
195 | #frame2/=frame2.max()
196 |
197 | # Display the resulting frame
198 | cv2.imshow('Webcam',frame)
199 |
200 | cv2.imshow('Filtered Image',frame2)
201 |
202 |
203 | cv2.imshow('FFT Filtered image',F)
204 |
205 | key=cv2.waitKey(1)
206 | if (key & 0xFF) in [ ord('q')]:
207 | break
208 | if (key & 0xFF) in [ ord('s')]:
209 | cv2.imwrite("screen_{}.png".format(idscreen),frame2*255)
210 | idscreen+=1
211 | if (key & 0xFF) in [ ord(' ')]:
212 | do_fft=not do_fft
213 | if (key & 0xFF) in [ ord('+')]:
214 | s+=1
215 | cut*=1.2
216 | if (key & 0xFF) in [ ord('-')]:
217 | s=max(1,s-1)
218 | cut/=1.2
219 | if (key & 0xFF) in [ ord('w')]:
220 | win=not win
221 | if (key & 0xFF) in [ ord('f')]:
222 | idfilt+=1
223 | s=1
224 | cut=0.5
225 | if (key & 0xFF) in [ ord('i')]:
226 | idfilt=11
227 | s=1
228 | cut=0.5
229 | if (key & 0xFF) in [ ord('F')]:
230 | idfilt-=1
231 | s=1
232 | cut=0.5
233 |
234 | # When everything done, release the capture
235 | cap.release()
236 | cv2.destroyAllWindows()
237 |
--------------------------------------------------------------------------------
/demos/demo_classif_2D.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 | # -*- coding: utf-8 -*-
3 | """
4 | MIT License
5 |
6 | Copyright (c) 2020 Rémi Flamary
7 |
8 | Permission is hereby granted, free of charge, to any person obtaining a copy
9 | of this software and associated documentation files (the "Software"), to deal
10 | in the Software without restriction, including without limitation the rights
11 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
12 | copies of the Software, and to permit persons to whom the Software is
13 | furnished to do so, subject to the following conditions:
14 |
15 | The above copyright notice and this permission notice shall be included in all
16 | copies or substantial portions of the Software.
17 |
18 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
19 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
20 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
21 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
22 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
23 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
24 | SOFTWARE.
25 |
26 | """
27 |
28 | import numpy as np
29 | import sys, pygame
30 | import scipy.misc
31 | import datetime
32 | import scipy as sp
33 |
34 | from sklearn.svm import SVC
35 | from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
36 | from sklearn.discriminant_analysis import QuadraticDiscriminantAnalysis
37 |
38 | classifiers = [
39 | LinearDiscriminantAnalysis,
40 | QuadraticDiscriminantAnalysis,
41 | SVC,
42 | SVC,
43 | ]
44 |
45 | classifier_params=[
46 | {},
47 | {},
48 | {"kernel":"linear", "C":0.025},
49 | {"gamma":5e-5, "C":10},
50 | ]
51 |
52 | classifier_names=['LDA', 'QDA', 'Linear SVM', 'Gaussian SVM']
53 |
54 | idclass=0
55 |
56 |
57 |
58 | # function to detect if key is caps or not (pygame doe not provide it...)
59 | def is_caps():
60 | return pygame.key.get_mods() & pygame.KMOD_SHIFT or pygame.key.get_mods() & pygame.KMOD_CAPS
61 |
62 | def get_pos(txt,y,x=0):
63 | pos=txt.get_rect()
64 | pos.left=x
65 | pos.centery=y
66 | return pos
67 |
68 | # init pygame
69 | pygame.init()
70 | # screen size
71 | size = width, height = 1024, 768
72 | screen = pygame.display.set_mode(size)
73 | pygame.display.set_caption('2D classifier demonstration')
74 |
75 |
76 |
77 | color_c1=pygame.Color(255,200,200)
78 | color_c2=pygame.Color(200,200,255)
79 |
80 | c2=np.array([255,200,200]).reshape((1,1,-1))
81 | c1=np.array([200,200,255]).reshape((1,1,-1))
82 |
83 | x1=np.arange(width)
84 | x2=np.arange(height)
85 |
86 | X1,X2=np.meshgrid(x1,x2)
87 |
88 | xgrid=np.concatenate((X1.ravel()[:,None],X2.ravel()[:,None]),1)
89 | ygrid=np.zeros_like(X1)
90 |
91 | # color palette
92 | pal = [(max((x-128)*2,0),x,min(x*2,255)) for x in range(256)]
93 |
94 | # background image
95 | world=pygame.Surface((width,height),depth=8) # MAIN SURFACE
96 | #world.set_palette(pal)
97 |
98 | data=np.array(np.zeros((height,width,3)),dtype=int)
99 |
100 |
101 | color_title=(50,50,100)
102 | color_text=(100,100,120)
103 |
104 | color_src_center=(250, 0, 0)
105 | color_src_out=(150, 0, 0)
106 |
107 | color_tgt_center=(0, 0, 250)
108 | color_tgt_out=(0, 0, 150)
109 |
110 | color_G=(255, 255, 255)
111 | # prepare texts
112 | font = pygame.font.Font(None, 25) # title
113 | font2 = pygame.font.Font(None, 20) # text
114 | font4 = pygame.font.Font(None, 20)
115 |
116 | radius=7
117 | width2=2
118 |
119 | width_m=2
120 |
121 | scale_G=1.0
122 |
123 | use_reg=False
124 | reg=1
125 |
126 |
127 | lst_src=[]
128 | lst_tgt=[]
129 |
130 | G=np.zeros((0,0))
131 |
132 | def draw_cov(C,mu,n,color):
133 |
134 | M=sp.linalg.sqrtm(C)
135 |
136 | angles=2*np.pi*np.linspace(0,1,n)
137 | v0=np.zeros(2)
138 | v0[0]=1
139 |
140 | for i in range(n-1):
141 | pos1=M.dot(np.array([np.cos(angles[i]),np.sin(angles[i])]))+mu
142 | pos2=M.dot(np.array([np.cos(angles[i+1]),np.sin(angles[i+1])]))+mu
143 | pygame.draw.line(world,color,pos1,pos2,2)
144 |
145 |
146 | def update_cov(src,tgt):
147 | xs=np.array(src)
148 | xt=np.array(tgt)
149 |
150 | mus=np.mean(xs,0)
151 | mut=np.mean(xt,0)
152 |
153 | Cs=np.cov(xs.T)
154 | Ct=np.cov(xt.T)
155 | C=(Cs*xs.shape[0]+Ct*xt.shape[0])/(xs.shape[0]+xt.shape[0])
156 |
157 | if idclass==0:
158 | draw_cov(C,mut,100,color_tgt_center)
159 | draw_cov(C,mus,100,color_src_center)
160 | elif idclass==1:
161 | draw_cov(Ct,mut,100,color_tgt_center)
162 | draw_cov(Cs,mus,100,color_src_center)
163 |
164 |
165 | def draw_source(world,pos):
166 | pygame.draw.circle(world,color_src_out, pos, radius+width2)
167 | pygame.draw.circle(world,color_src_center, pos, radius)
168 |
169 | def draw_target(world,pos):
170 | pygame.draw.circle(world,color_tgt_out, pos, radius+width2)
171 | pygame.draw.circle(world,color_tgt_center, pos, radius)
172 |
173 | def find_overlap(pos,lst):
174 | res=-1
175 | for i,pos2 in enumerate(lst):
176 | if np.sqrt(np.sum(np.square(np.array(pos)-np.array(pos2))))2 and len(lst_tgt)>2:
302 | update_cov(lst_src,lst_tgt)
303 |
304 |
305 | screen.blit(world, (0,0))
306 |
307 | for t,p in zip(lstf, lstp):
308 | screen.blit(t, p)
309 |
310 |
311 | pygame.display.flip() #RENDER WINDOW
312 |
313 |
314 |
--------------------------------------------------------------------------------
/demos/demo_color_transfer.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 | # -*- coding: utf-8 -*-
3 | """
4 | MIT License
5 |
6 | Copyright (c) 2020 Rémi Flamary
7 |
8 | Permission is hereby granted, free of charge, to any person obtaining a copy
9 | of this software and associated documentation files (the "Software"), to deal
10 | in the Software without restriction, including without limitation the rights
11 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
12 | copies of the Software, and to permit persons to whom the Software is
13 | furnished to do so, subject to the following conditions:
14 |
15 | The above copyright notice and this permission notice shall be included in all
16 | copies or substantial portions of the Software.
17 |
18 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
19 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
20 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
21 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
22 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
23 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
24 | SOFTWARE.
25 |
26 | """
27 |
28 | import numpy as np
29 | import cv2
30 | import scipy.ndimage
31 | from PIL import Image
32 | import torch
33 | from datetime import datetime
34 | import os
35 | import urllib
36 | import ot
37 |
38 | import os
39 | dir_path = os.path.dirname(os.path.realpath(__file__))
40 |
41 | folder = datetime.now().strftime("%Y_%m_%d-%H_%M_%S")
42 | style_path = dir_path+'/../data/styles/'
43 |
44 | if not os.path.exists('out'):
45 | os.mkdir('out')
46 |
47 | idimg = 0
48 |
49 | fname = "out/{}/{}_{}.jpg"
50 |
51 | lst_map = ['Linear OT','Exact OT', 'Sinkhorn OT']
52 | id_map = 0
53 |
54 | n_keep = 200
55 |
56 | def transfer(c, s, alpha=1):
57 | nxs = c.shape[0]*c.shape[1]
58 | nxt = s.shape[0]*s.shape[1]
59 | xs = c.copy().reshape((nxs,3))*1.0
60 | xt = s.copy().reshape((nxt,3))*1.0
61 | if id_map==0:
62 | adapt = ot.da.LinearTransport()
63 | adapt.fit(xs, Xt=xt)
64 | elif id_map==1:
65 | adapt = ot.da.EMDTransport()
66 | xs2 = xs[np.random.permutation(xs.shape[0])[:n_keep],:]
67 | xt2 = xt[np.random.permutation(xt.shape[0])[:n_keep],:]
68 | adapt.fit(xs2, Xt=xt2)
69 | elif id_map==2:
70 | adapt = ot.da.SinkhornTransport(reg_e=100)
71 | xs2 = xs[np.random.permutation(xs.shape[0])[:n_keep],:]
72 | xt2 = xt[np.random.permutation(xt.shape[0])[:n_keep],:]
73 | adapt.fit(xs2, Xt=xt2)
74 | return adapt.transform(xs).reshape(c.shape)
75 |
76 |
77 | cam = os.getenv("CAMERA")
78 | if cam is None:
79 | cap = cv2.VideoCapture(0)
80 | else:
81 | cap = cv2.VideoCapture(int(cam))
82 |
83 | lst_style = ['antimonocromatismo.jpg',
84 | 'sketch.png',
85 | 'hosi.jpg',
86 | 'asheville.jpg',
87 | 'brushstrokes.jpg',
88 | 'contrast_of_forms.jpg',
89 | 'en_campo_gris.jpg',
90 | 'la_muse.jpg',
91 | 'mondrian.jpg',
92 | 'picasso_seated_nude_hr.jpg',
93 | 'picasso_self_portrait.jpg',
94 | 'scene_de_rue.jpg',
95 | 'trial.jpg',
96 | 'woman_in_peasant_dress_cropped.jpg',
97 | 'woman_with_hat_matisse.jpg',
98 | 'afremov.jpg',
99 | 'bubbles.jpg',
100 | 'in2.jpg',
101 | 'baiser.jpg',
102 | 'zombie.jpg',
103 | 'nikopol.jpg',
104 | 'miro.jpg',
105 | 'monet1.jpg',
106 | 'monet2.jpg',
107 | 'nikopol.jpg', ]
108 |
109 | lst_style0 = lst_style
110 |
111 | lst_name = [f.split('.')[0] for f in lst_style]
112 |
113 | lst_style = [np.array(Image.open(style_path+f).convert('RGB'))
114 | for f in lst_style]
115 |
116 | col = [255, 1, 1]
117 | max_iters = 1
118 | alpha = 0.8
119 | id_style = 0
120 |
121 | from_RGB = [2, 1, 0]
122 |
123 |
124 | pause = False
125 |
126 | ret, frame0 = cap.read()
127 | frame0 = np.asfortranarray(np.array(frame0)/255)
128 | frame_webcam = frame0
129 | frame_style = frame0*0
130 |
131 |
132 | while (True):
133 | # Capture frame-by-frame
134 | ret, frame = cap.read()
135 |
136 | frame2 = np.array(frame)
137 | frame2 = frame2[:, ::-1, :]
138 |
139 | if not pause:
140 | frame_webcam = frame2
141 |
142 | # Display the images
143 | cv2.imshow('Webcam', frame_webcam)
144 | cv2.imshow('Transferred image ({})'.format(lst_map[id_map]), frame_style[:, :, from_RGB])
145 |
146 | cv2.namedWindow('Target Color', cv2.WINDOW_NORMAL)
147 | cv2.imshow('Target Color', lst_style[id_style][:, :, from_RGB])
148 |
149 | # handle inputs
150 | key = cv2.waitKey(1)
151 | if (key & 0xFF) in [ord('q')]:
152 | break
153 | if (key & 0xFF) in [ord('s')]:
154 | id_style = (id_style+1) % len(lst_name)
155 | if (key & 0xFF) in [ord('w')]:
156 | if not os.path.exists('out/{}'.format(folder)):
157 | os.mkdir('out/{}'.format(folder))
158 | cv2.imwrite(fname.format(folder, idimg, '0'), frame_webcam)
159 | cv2.imwrite(fname.format(folder, idimg,
160 | lst_name[id_style]), frame_style[:, :, from_RGB]*255)
161 | print("Images saved")
162 | if (key & 0xFF) in [ord('q')]:
163 | break
164 | if (key & 0xFF) in [ord('r')]:
165 | if not os.path.exists('out/{}'.format(folder)):
166 | os.mkdir('out/{}'.format(folder))
167 | cv2.imwrite(fname.format(folder, idimg, '0'), frame_webcam)
168 | for i in range(len(lst_style)):
169 | temp = np.array(transfer(frame_webcam, lst_style[i], alpha=alpha))
170 | # print(temp)
171 | for k in range(3):
172 | frame_style[:, :, k] = temp[k, :, :]/255
173 | cv2.imwrite(fname.format(folder, idimg,
174 | lst_name[i]), frame_style[:, :, from_RGB]*255)
175 | print('Applied style from file {}'.format(lst_style0[i]))
176 | cv2.imshow('Transferred image ({})'.format(
177 | lst_style0[i]), frame_style[:, :, from_RGB])
178 |
179 | if (key & 0xFF) in [ord('p')]:
180 | pause = not pause
181 | if pause:
182 | idimg += 1
183 | if (key & 0xFF) in [ord('A')]:
184 | alpha = min(1, alpha+0.1)
185 | print('alpha={}'.format(alpha))
186 | if (key & 0xFF) in [ord('a')]:
187 | alpha = max(0, alpha-0.1)
188 | print('alpha={}'.format(alpha))
189 | if (key & 0xFF) in [ord('m')]:
190 | id_map = (id_map+1) % len(lst_map)
191 | temp = np.array(
192 | transfer(frame_webcam, lst_style[id_style], alpha=alpha))
193 | # print(temp)
194 | for i in range(3):
195 | frame_style[:, :, i] = temp[ :, :, i]/255
196 | print('Applied style from file {}'.format(lst_style0[id_style]))
197 | if (key & 0xFF) in [ord(' ')]:
198 | pause = True
199 | temp = np.array(
200 | transfer(frame_webcam, lst_style[id_style], alpha=alpha))
201 | # print(temp)
202 | for i in range(3):
203 | frame_style[:, :, i] = temp[ :, :, i]/255
204 | print('Applied style from file {}'.format(lst_style0[id_style]))
205 |
206 |
207 | # When everything done, release the capture
208 | cap.release()
209 | cv2.destroyAllWindows()
210 |
--------------------------------------------------------------------------------
/demos/demo_dicolearn.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 | # -*- coding: utf-8 -*-
3 | """
4 | MIT License
5 |
6 | Copyright (c) 2020 Rémi Flamary
7 |
8 | Permission is hereby granted, free of charge, to any person obtaining a copy
9 | of this software and associated documentation files (the "Software"), to deal
10 | in the Software without restriction, including without limitation the rights
11 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
12 | copies of the Software, and to permit persons to whom the Software is
13 | furnished to do so, subject to the following conditions:
14 |
15 | The above copyright notice and this permission notice shall be included in all
16 | copies or substantial portions of the Software.
17 |
18 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
19 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
20 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
21 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
22 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
23 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
24 | SOFTWARE.
25 |
26 | """
27 |
28 | import numpy as np
29 | import sys, pygame
30 | import scipy.misc
31 | import datetime
32 | import scipy as sp
33 |
34 | from sklearn.decomposition import PCA, FastICA, DictionaryLearning, NMF, KernelPCA
35 | from sklearn.cluster import KMeans
36 |
37 | models = [
38 | PCA,
39 | FastICA,
40 | KMeans,
41 | NMF,
42 | DictionaryLearning,
43 | KernelPCA,
44 | ]
45 |
46 | models_params=[
47 | {},
48 | {},
49 | {"random_state":42},
50 | {"random_state":42, "alpha_W":0.1, "alpha_H":0.0},
51 | {"alpha":1, "max_iter":100,"random_state":42},
52 | {"kernel":"rbf", "gamma":0.5, "fit_inverse_transform":True},
53 | ]
54 |
55 | # list of models with max number of components to two
56 | limit_2 = [0,1 ]
57 |
58 |
59 | n_components=2
60 |
61 | model_names=['PCA', 'ICA', 'KMeans', 'NMF', 'Sparse DL', 'Kernel PCA' ]
62 |
63 | idclass=0
64 |
65 |
66 |
67 | # function to detect if key is caps or not (pygame doe not provide it...)
68 | def is_caps():
69 | return pygame.key.get_mods() & pygame.KMOD_SHIFT or pygame.key.get_mods() & pygame.KMOD_CAPS
70 |
71 | def get_pos(txt,y,x=0):
72 | pos=txt.get_rect()
73 | pos.left=x
74 | pos.centery=y
75 | return pos
76 |
77 | # init pygame
78 | pygame.init()
79 | # screen size
80 | size = width, height = 1024, 768
81 | screen = pygame.display.set_mode(size)
82 | pygame.display.set_caption('2D Dictionary Learning demonstration')
83 |
84 |
85 |
86 | color_c1=pygame.Color(255,200,200)
87 | color_c2=pygame.Color(200,200,255)
88 |
89 | c2=np.array([255,200,200]).reshape((1,1,-1))
90 | c1=np.array([200,200,255]).reshape((1,1,-1))
91 |
92 | x1=np.arange(width)
93 | x2=np.arange(height)
94 |
95 | X1,X2=np.meshgrid(x1,x2)
96 |
97 | xgrid=np.concatenate((X1.ravel()[:,None],X2.ravel()[:,None]),1)
98 | ygrid=np.zeros_like(X1)
99 |
100 | # color palette
101 | pal = [(max((x-128)*2,0),x,min(x*2,255)) for x in range(256)]
102 |
103 | # background image
104 | world=pygame.Surface((width,height),depth=8) # MAIN SURFACE
105 | #world.set_palette(pal)
106 |
107 | data=np.array(np.zeros((height,width,3)),dtype=int)
108 |
109 |
110 | color_title=(50,50,100)
111 | color_text=(100,100,120)
112 |
113 | color_src_center=(250, 0, 0)
114 | color_src_out=(150, 0, 0)
115 |
116 | color_tgt_center=(0, 0, 250)
117 | color_tgt_out=(0, 0, 150)
118 |
119 | color_data=(10, 10, 10)
120 |
121 | color_traj=(150, 150, 150)
122 |
123 | color_blue = (0,0,255)
124 | color_red = (255,0,0)
125 | color_green = (0,255,0)
126 | color_orange = (255,165,0)
127 | color_purple = (128,0,128)
128 | color_yellow = (255,255,0)
129 | color_pink = (255,192,203)
130 |
131 | colors = [color_blue, color_red, color_green, color_orange, color_purple, color_yellow, color_pink]
132 | colors_np = np.array(colors, dtype=float)
133 |
134 |
135 |
136 |
137 | # prepare texts
138 | font = pygame.font.Font(None, 25) # title
139 | font2 = pygame.font.Font(None, 20) # text
140 | font4 = pygame.font.Font(None, 20)
141 |
142 | radius=7
143 | width2=2
144 |
145 | width_m=2
146 |
147 | scale_G=1.0
148 | scale_dic=100
149 |
150 | use_reg=False
151 | reg=1
152 |
153 |
154 | lst_src=[]
155 | lst_tgt=[]
156 | xp=[]
157 |
158 | G=np.zeros((0,0))
159 |
160 |
161 | def draw_arrow(
162 | surface: pygame.Surface,
163 | start: pygame.Vector2,
164 | end: pygame.Vector2,
165 | color: pygame.Color,
166 | body_width: int = 1,
167 | head_width: int = 4,
168 | head_height: int = 2,
169 | ):
170 | """Draw an arrow between start and end with the arrow head at the end.
171 |
172 | Args:
173 | surface (pygame.Surface): The surface to draw on
174 | start (pygame.Vector2): Start position
175 | end (pygame.Vector2): End position
176 | color (pygame.Color): Color of the arrow
177 | body_width (int, optional): Defaults to 2.
178 | head_width (int, optional): Defaults to 4.
179 | head_height (float, optional): Defaults to 2.
180 | """
181 | start = pygame.Vector2(*start)
182 | end = pygame.Vector2(*end)
183 | arrow = start - end
184 | angle = arrow.angle_to(pygame.Vector2(0, -1))
185 | body_length = arrow.length() - head_height
186 |
187 | # Create the triangle head around the origin
188 | head_verts = [
189 | pygame.Vector2(0, head_height / 2), # Center
190 | pygame.Vector2(head_width / 2, -head_height / 2), # Bottomright
191 | pygame.Vector2(-head_width / 2, -head_height / 2), # Bottomleft
192 | ]
193 | # Rotate and translate the head into place
194 | translation = pygame.Vector2(0, arrow.length() - (head_height / 2)).rotate(-angle)
195 | for i in range(len(head_verts)):
196 | head_verts[i].rotate_ip(-angle)
197 | head_verts[i] += translation
198 | head_verts[i] += start
199 |
200 | pygame.draw.polygon(surface, color, head_verts)
201 | pygame.draw.aaline(surface, color, start, end, body_width)
202 |
203 | # Stop weird shapes when the arrow is shorter than arrow head
204 | if arrow.length() >= head_height:
205 | # Calculate the body rect, rotate and translate into place
206 | body_verts = [
207 | pygame.Vector2(-body_width / 2, body_length / 2), # Topleft
208 | pygame.Vector2(body_width / 2, body_length / 2), # Topright
209 | pygame.Vector2(body_width / 2, -body_length / 2), # Bottomright
210 | pygame.Vector2(-body_width / 2, -body_length / 2), # Bottomleft
211 | ]
212 | translation = pygame.Vector2(0, body_length / 2).rotate(-angle)
213 | for i in range(len(body_verts)):
214 | body_verts[i].rotate_ip(-angle)
215 | body_verts[i] += translation
216 | body_verts[i] += start
217 |
218 | pygame.draw.polygon(surface, color, body_verts)
219 | pygame.draw.aaline(surface, color, start, end, body_width)
220 |
221 |
222 | def draw_source(world,pos, color=color_data):
223 | #pygame.draw.circle(world,color_src_out, pos, radius+width2)
224 | #pygame.draw.circle(world,color_src_center, pos, radius)
225 |
226 | # draw a cross
227 | pygame.draw.line(world,color,(pos[0]-radius,pos[1]),(pos[0]+radius,pos[1]),width_m)
228 | pygame.draw.line(world,color,(pos[0],pos[1]-radius),(pos[0],pos[1]+radius),width_m)
229 |
230 | def draw_center(world,clf, color=color_data):
231 |
232 | pos=clf.mean_[:2]
233 | pos = pos.astype(int)
234 | #pygame.draw.circle(world,color_src_out, pos, radius+width2)
235 | #pygame.draw.circle(world,color_src_center, pos, radius)
236 |
237 | # draw a cross
238 | pygame.draw.line(world,color,(pos[0]-radius,pos[1]),(pos[0]+radius,pos[1]),width_m+2)
239 | pygame.draw.line(world,color,(pos[0],pos[1]-radius),(pos[0],pos[1]+radius),width_m+2)
240 |
241 | def draw_dictionary(world,clf):
242 | n = clf.components_.shape[0]
243 | D = clf.components_
244 | center = clf.mean_[:2].astype(int)
245 |
246 |
247 | for i in range(n):
248 | vec = D[i,:2]
249 | if model_names[idclass] in ['KMeans']: # NMF of kmenas no scaling
250 | pass
251 | else:
252 | vec = vec/np.linalg.norm(vec)*scale_dic
253 | vec = vec.astype(int)
254 | #pygame.draw.circle(world,color_src_out, pos, radius+width2)
255 | #pygame.draw.circle(world,color_src_center, pos, radius)
256 |
257 | # draw a cross
258 | draw_arrow(world, center, (center[0]+vec[0],center[1]+vec[1] ), darken_color(colors[i],0.7), body_width=2, head_width=10, head_height=10)
259 |
260 | def find_overlap(pos,lst):
261 | res=-1
262 | for i,pos2 in enumerate(lst):
263 | if np.sqrt(np.sum(np.square(np.array(pos)-np.array(pos2))))=2:
363 | clf, xp, xpr =update_method(lst_src)
364 | if event.key in [pygame.K_DOWN] :
365 | n_components=max(1,n_components-1)
366 | update_txt(lstf,reg,use_reg)
367 | if len(lst_src)>=2:
368 | clf, xp, xpr = update_method(lst_src)
369 | if event.key in [pygame.K_UP] :
370 | n_components+=1
371 | n_components=min(7,n_components)
372 | if idclass in limit_2:
373 | n_components=min(2,n_components)
374 | update_txt(lstf,reg,use_reg)
375 | if len(lst_src)>=2:
376 | clf, xp, xpr = update_method(lst_src)
377 | if event.type == pygame.MOUSEBUTTONDOWN:
378 | pos = pygame.mouse.get_pos()
379 | #print(event)
380 | if event.button==1:
381 | lst_src.append(pos)
382 | #print(pos)
383 | elif event.button==3:
384 | i=find_overlap(pos,lst_src)
385 | if not i<0:
386 | del lst_src[i]
387 |
388 | if len(lst_src)>=2:
389 | clf, xp, xpr = update_method(lst_src)
390 |
391 |
392 | # if plot_class:
393 | # data[:]=(ygrid==1)[:,:,None]*c1+(ygrid==2)[:,:,None]*c2+ (ygrid==0)[:,:,None]*np.ones((1,1,3),dtype=int)*255
394 | # else:
395 | # data[:]=255
396 |
397 | world=pygame.pixelcopy.make_surface(np.swapaxes(data,0,1))
398 |
399 | if plot_proj:
400 | for i,pos in enumerate(lst_src):
401 | pos2 = (xpr[i,:2]).astype(int)
402 | pygame.draw.aaline(world,get_color(xp[i],0.3),pos,pos2,2)
403 |
404 | for i,pos in enumerate(lst_src):
405 | if plot_dico and plot_proj:
406 | draw_source(world,pos, color=get_color(xp[i],0.5))
407 | draw_source(world,xpr[i], color=get_color(xp[i],0.7))
408 | elif plot_dico:
409 | #print(get_color(xp[i]))
410 | draw_source(world,pos, color=get_color(xp[i]))
411 | elif plot_proj:
412 | draw_source(world,xpr[i], color=get_color(xp[i],0.7))
413 | else:
414 | draw_source(world,pos)
415 | # for pos in lst_tgt:
416 | # draw_target(world,pos)
417 |
418 | # if len(lst_src)>2 and len(lst_tgt)>2:
419 | # update_cov(lst_src,lst_tgt)
420 |
421 | if plot_dico and len(lst_src)>2:
422 | if not model_names[idclass] in ['Kernel PCA']:
423 | draw_center(world,clf)
424 | draw_dictionary(world,clf)
425 |
426 |
427 | screen.blit(world, (0,0))
428 |
429 | for t,p in zip(lstf, lstp):
430 | screen.blit(t, p)
431 |
432 |
433 | pygame.display.flip() #RENDER WINDOW
434 |
435 |
436 |
--------------------------------------------------------------------------------
/demos/demo_face_detection.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 | # -*- coding: utf-8 -*-
3 | """
4 | MIT License
5 |
6 | Copyright (c) 2020 Rémi Flamary
7 |
8 | Permission is hereby granted, free of charge, to any person obtaining a copy
9 | of this software and associated documentation files (the "Software"), to deal
10 | in the Software without restriction, including without limitation the rights
11 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
12 | copies of the Software, and to permit persons to whom the Software is
13 | furnished to do so, subject to the following conditions:
14 |
15 | The above copyright notice and this permission notice shall be included in all
16 | copies or substantial portions of the Software.
17 |
18 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
19 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
20 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
21 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
22 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
23 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
24 | SOFTWARE.
25 |
26 | """
27 |
28 | import numpy as np
29 | import numpy as np
30 | import cv2
31 | import os
32 |
33 | path_classif=os.path.dirname(__file__)+'/../data/models/haarcascade_frontalface_default.xml'
34 |
35 |
36 | cam=os.getenv("CAMERA")
37 | if cam is None:
38 | cap = cv2.VideoCapture(0)
39 | else:
40 | cap = cv2.VideoCapture(int(cam))
41 |
42 | face_cascade = cv2.CascadeClassifier(path_classif)
43 |
44 | #screenshot index index
45 | idscreen=0
46 |
47 |
48 | while(True):
49 |
50 | # Capture frame-by-frame
51 | ret, frame = cap.read()
52 | img=frame
53 |
54 | # apply detector
55 | gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
56 | faces = face_cascade.detectMultiScale(gray, 1.3, 5)
57 |
58 | # print rectangles for detected faces
59 | for (x,y,w,h) in faces:
60 | img = cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)
61 | roi_gray = gray[y:y+h, x:x+w]
62 | roi_color = img[y:y+h, x:x+w]
63 |
64 | # Display the resulting frame
65 | cv2.imshow('Face detection',img)
66 |
67 | key=cv2.waitKey(1)
68 | if (key & 0xFF) in [ ord('q')]:
69 | break
70 | if (key & 0xFF) in [ ord('s')]:
71 | cv2.imwrite("screen_{}.png".format(idscreen),img*255)
72 | idscreen+=1
73 |
74 |
75 | # When everything done, release the capture
76 | cap.release()
77 | cv2.destroyAllWindows()
78 |
--------------------------------------------------------------------------------
/demos/demo_fft.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 | # -*- coding: utf-8 -*-
3 | """
4 | MIT License
5 |
6 | Copyright (c) 2020 Rémi Flamary
7 |
8 | Permission is hereby granted, free of charge, to any person obtaining a copy
9 | of this software and associated documentation files (the "Software"), to deal
10 | in the Software without restriction, including without limitation the rights
11 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
12 | copies of the Software, and to permit persons to whom the Software is
13 | furnished to do so, subject to the following conditions:
14 |
15 | The above copyright notice and this permission notice shall be included in all
16 | copies or substantial portions of the Software.
17 |
18 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
19 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
20 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
21 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
22 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
23 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
24 | SOFTWARE.
25 |
26 | """
27 |
28 | import numpy as np
29 | import cv2
30 | import prox_tv as ptv
31 | import os
32 | import scipy as sp
33 | import scipy.signal
34 |
35 |
36 | cam=os.getenv("CAMERA")
37 | if cam is None:
38 | cap = cv2.VideoCapture(0)
39 | else:
40 | cap = cv2.VideoCapture(int(cam))
41 |
42 | reg=2e-1
43 | alpha=0.2
44 | pred=0
45 | idscreen=0
46 | loop=True
47 | do_fft=True
48 | half=False
49 |
50 |
51 | tau_noise=0.002
52 |
53 | ret, frame0 = cap.read()
54 |
55 | n1=frame0.shape[0]
56 | n2=frame0.shape[1]
57 |
58 | win=False
59 |
60 | w1=sp.signal.get_window('parzen',n1)**.2
61 | w2=sp.signal.get_window('parzen',n2)**.2
62 | W=w1[:,None]*w2[None,:]
63 |
64 | scale = 0
65 |
66 | scales= ["Linear",'Sqrt','Log']
67 |
68 |
69 | while(True):
70 | # Capture frame-by-frame
71 | ret, frame = cap.read()
72 |
73 |
74 | frame2=np.array(frame)/255
75 | frame2=frame2[:,::-1,:]
76 | #frame2=frame2[::2,::2,:]
77 |
78 | if win:
79 | frame2=frame2*W[:,:,None]
80 |
81 | F=np.zeros_like(frame2)
82 | for i in range(3):
83 | F[:,:,i]=np.abs(np.fft.fftshift(np.fft.fft2(frame2[:,:,i]),axes=(0,1)))
84 | Fl=np.log(F+1e-2)
85 | Fl/=Fl.max()
86 |
87 | F =F/F.max()*100
88 |
89 | Fs = np.sqrt(F)
90 | Fs = Fs/Fs.max()
91 |
92 |
93 |
94 | #frame2=(frame2+frame0)/2
95 |
96 | frame0=frame2
97 |
98 | #frame2-=frame2.min()
99 | #frame2/=frame2.max()
100 |
101 | # Display the resulting frame
102 | cv2.imshow('Webcam',frame2)
103 |
104 | if do_fft:
105 | cv2.imshow('FFT webcam Log scale',Fl)
106 | cv2.imshow('FFT webcam Linear scale',F)
107 | #cv2.imshow('FFT webcam Sqrt scale',F)
108 |
109 | key=cv2.waitKey(1)
110 | if (key & 0xFF) in [ ord('q')]:
111 | break
112 | if (key & 0xFF) in [ ord('s')]:
113 | #cv2.imwrite("screen_{}.png".format(idscreen),frame2*255)
114 | #idscreen+=1
115 |
116 | scale = 0
117 | if (key & 0xFF) in [ ord(' ')]:
118 | do_fft=not do_fft
119 | if (key & 0xFF) in [ ord('w')]:
120 | win=not win
121 | if (key & 0xFF) in [ ord('h')]:
122 | if not half:
123 | do_tv=True
124 | half=True
125 | else:
126 | half=False
127 |
128 |
129 | # When everything done, release the capture
130 | cap.release()
131 | cv2.destroyAllWindows()
132 |
--------------------------------------------------------------------------------
/demos/demo_optimal_transport.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 | # -*- coding: utf-8 -*-
3 | """
4 | MIT License
5 |
6 | Copyright (c) 2020 Rémi Flamary
7 |
8 | Permission is hereby granted, free of charge, to any person obtaining a copy
9 | of this software and associated documentation files (the "Software"), to deal
10 | in the Software without restriction, including without limitation the rights
11 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
12 | copies of the Software, and to permit persons to whom the Software is
13 | furnished to do so, subject to the following conditions:
14 |
15 | The above copyright notice and this permission notice shall be included in all
16 | copies or substantial portions of the Software.
17 |
18 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
19 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
20 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
21 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
22 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
23 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
24 | SOFTWARE.
25 |
26 | """
27 |
28 |
29 | import numpy as np
30 | import sys
31 | import pygame
32 | import ot
33 | import random
34 | import scipy.misc
35 | import datetime
36 |
37 | # function to detect if key is caps or not (pygame doe not provide it...)
38 |
39 |
40 | def is_caps():
41 | return pygame.key.get_mods() & pygame.KMOD_SHIFT or pygame.key.get_mods() & pygame.KMOD_CAPS
42 |
43 |
44 | def get_pos(txt, y, x=0):
45 | pos = txt.get_rect()
46 | pos.left = x
47 | pos.centery = y
48 | return pos
49 |
50 |
51 | # init pygame
52 | pygame.init()
53 | # screen size
54 | size = width, height = 1024, 768
55 | screen = pygame.display.set_mode(size)
56 | pygame.display.set_caption('Discrete OT demonstration')
57 |
58 |
59 | # color palette
60 | pal = [(max((x-128)*2, 0), x, min(x*2, 255)) for x in range(256)]
61 |
62 | # background image
63 | world = pygame.Surface((width, height)) # MAIN SURFACE
64 | # world.set_palette(pal)
65 |
66 | data = np.array(np.zeros((height, width)), dtype=int)
67 |
68 |
69 | color_title = (50, 50, 100)
70 | color_text = (100, 100, 120)
71 |
72 | color_src_center = (250, 0, 0)
73 | color_src_out = (150, 0, 0)
74 |
75 | color_tgt_center = (0, 0, 250)
76 | color_tgt_out = (0, 0, 150)
77 |
78 | color_bary_center = (0, 250, 0)
79 | color_bary_out = (0, 150, 0)
80 |
81 | color_wall = (115, 0, 0)
82 | wall_pos = pygame.Rect(508, 384, 8, 384)
83 |
84 | color_G = (0, 0, 0)
85 |
86 | # prepare texts
87 | font = pygame.font.Font(None, 25) # title
88 | font2 = pygame.font.Font(None, 20) # text
89 |
90 | radius = 7
91 | width = 2
92 |
93 | width_m = 2
94 |
95 | scale_G = 1.0
96 |
97 | use_reg = False
98 | reg = 1
99 |
100 | wall = False
101 | plot_bary = False
102 | alpha_bary=0.501
103 |
104 |
105 | #
106 | lst_metrics = ['euclidean', 'sqeuclidean', 'cityblock', 'wall (geodesic)', 'euclidean (+paywall)']
107 | id_metric = 0
108 |
109 |
110 | lst_src = []
111 | lst_tgt = []
112 |
113 | G = np.zeros((0, 0))
114 |
115 |
116 | def get_color(s, color=None):
117 | if color is None:
118 | color = color_G
119 | col = pygame.Color(*color)
120 | H, S, L, A = col.hsla
121 |
122 | temp = pygame.Color(255, 255, 255)
123 | temp.hsla = (H, S, ((1-s)*100+s*L), A)
124 | # return pygame.Color(*[int(min(255*(1-s)+s*c,255)) for c in color])
125 | # return pygame.Color(255,255,255).lerp(color,s)
126 | return temp
127 |
128 |
129 | def draw_source(world, pos):
130 | pygame.draw.circle(world, color_src_out, pos, radius+width)
131 | pygame.draw.circle(world, color_src_center, pos, radius)
132 |
133 |
134 | def draw_target(world, pos):
135 | pygame.draw.circle(world, color_tgt_out, pos, radius+width)
136 | pygame.draw.circle(world, color_tgt_center, pos, radius)
137 |
138 | def draw_bary(world, pos,scale=1.0):
139 | pygame.draw.circle(world, color_bary_out, pos, scale*radius+width)
140 | pygame.draw.circle(world, color_bary_center, pos, scale*radius)
141 |
142 |
143 |
144 | def find_overlap(pos, lst):
145 | res = -1
146 | for i, pos2 in enumerate(lst):
147 | if np.sqrt(np.sum(np.square(np.array(pos)-np.array(pos2)))) < radius+width:
148 | res = i
149 | return res
150 |
151 |
152 | def rotate_list(lst, theta):
153 | x = np.array(lst)
154 | mx = x.mean(0, keepdims=True)
155 | R = np.array([[np.cos(theta), np.sin(theta)],
156 | [-np.sin(theta), np.cos(theta)]])
157 | xr = (x-mx).dot(R)+mx
158 | xr = xr.astype(int)
159 | return xr.tolist()
160 |
161 |
162 | def intercept_wall(x1, x2):
163 |
164 | xm = 1024/2
165 | if (x1[0]=xm and x2[0]>=xm):
166 | return False
167 | delta = abs(xm-x1[0])+abs(x2[0]-xm) + 1e-300
168 | ywal = x1[1]*abs(x2[0]-xm)/delta+x2[1]*abs(xm-x1[0])/delta
169 | return ywal > 384
170 |
171 | def value_y_wall(x1, x2):
172 |
173 | xm = 1024/2
174 | if (x1[0]=xm and x2[0]>=xm):
175 | return False
176 | delta = abs(xm-x1[0])+abs(x2[0]-xm) + 1e-300
177 | ywal = x1[1]*abs(xm-x1[0])/delta+x2[1]*abs(x2[0]-xm)/delta
178 | return ywal
179 |
180 | def dist(x1,x2):
181 | return np.sqrt(np.sum(np.square(np.array(x1)-np.array(x2))))
182 |
183 | def update_G(src, tgt):
184 | xs = np.array(src)
185 | xt = np.array(tgt)
186 | metric = lst_metrics[id_metric]
187 | mid = (512, 384)
188 | if id_metric == 3: # wall geodesic
189 | M = ot.dist(xs, xt, 'euclidean')
190 | for i in range(M.shape[0]):
191 | for j in range(M.shape[1]):
192 | if intercept_wall(src[i], tgt[j]):
193 | M[i, j] = dist(src[i], mid)+ dist(tgt[j], mid)
194 | elif id_metric == 4:
195 | M = ot.dist(xs, xt, 'euclidean')
196 | for i in range(M.shape[0]):
197 | for j in range(M.shape[1]):
198 | if intercept_wall(src[i], tgt[j]):
199 | M[i, j] += 10000
200 | else:
201 | M = ot.dist(xs, xt, metric)
202 | if not use_reg:
203 | return ot.emd([], [], M*1.0)
204 | else:
205 | return ot.sinkhorn([], [], M*1.0, reg)
206 |
207 |
208 | def draw_G2(world, G, src, tgt):
209 | Gmax = G.max()
210 | for i in range(G.shape[0]):
211 | for j in range(G.shape[1]):
212 | if G[i, j] > 1e-8:
213 | scale = G[i, j]/Gmax
214 | #print([int(255*(scale)) for c in color_G])
215 | pygame.draw.line(world, get_color(scale),
216 | src[i], tgt[j], width_m)
217 |
218 |
219 | def draw_G(world, G, src, tgt, only_bary=False):
220 | Gmax = G.max()
221 | isort = np.argsort(G.ravel())
222 | for k in isort:
223 | i, j = np.unravel_index(k, G.shape)
224 | if G[i, j] > 1e-8:
225 | scale = G[i, j]/Gmax
226 | #print([int(255*(scale)) for c in color_G])
227 | if not only_bary:
228 | if id_metric == 2:
229 | mid = (tgt[j][0], src[i][1])
230 | pygame.draw.line(world, get_color(scale), src[i], mid, width_m)
231 | pygame.draw.line(world, get_color(scale), mid, tgt[j], width_m)
232 |
233 | elif id_metric == 3:
234 | if intercept_wall(src[i], tgt[j]):
235 | mid = (512, 384)
236 | pygame.draw.line(world, get_color(scale), src[i], mid, width_m)
237 | pygame.draw.line(world, get_color(scale), mid, tgt[j], width_m)
238 | else:
239 | pygame.draw.line(world, get_color(scale),
240 | src[i], tgt[j], width_m)
241 | else:
242 | pygame.draw.line(world, get_color(scale), src[i], tgt[j], width_m)
243 | if plot_bary:
244 | mid = (int((src[i][0]*(1-alpha_bary)+alpha_bary*tgt[j][0])), int((src[i][1]*(1-alpha_bary)+alpha_bary*tgt[j][1])))
245 | draw_bary(world, mid, scale)
246 |
247 |
248 |
249 | def get_text(lst, deltay=0):
250 | lstf = []
251 | lstp = []
252 | for i, txt in enumerate(lst):
253 | lstf.append(font2.render('{}'.format(txt), 1, color_text))
254 | lstp.append(get_pos(lstf[-1], 15+20*i+deltay, 5))
255 |
256 | return lstf, lstp
257 |
258 |
259 | lst_txt = ['Source: left click', 'Target : right click',
260 | 'Clear : c', 'Switch Reg OT : r', 'Reg value : {}'.format(0), 'Metric : {}'.format(lst_metrics[id_metric])]
261 |
262 | lstf, lstp = get_text(lst_txt)
263 |
264 |
265 | def update_txt(lstf, reg, use_reg):
266 |
267 | if use_reg:
268 | lstf[4] = font2.render(
269 | 'Reg value : {:1.2e}'.format(reg), 1, color_text)
270 | else:
271 | lstf[4] = font2.render('Reg value : 0'.format(reg), 1, color_text)
272 | lstf[5] = font2.render('Metric : {}'.format(
273 | lst_metrics[id_metric]), 1, color_text)
274 |
275 |
276 | plot_G = False
277 | one_dist = False
278 |
279 | while 1:
280 |
281 | # keyboard events
282 | for event in pygame.event.get(): # check if we need to exit
283 | if event.type == pygame.QUIT:
284 | pygame.quit()
285 | sys.exit()
286 | if event.type == pygame.KEYDOWN:
287 | # print event.key
288 | if event.key in [pygame.K_ESCAPE, pygame.K_q]: # quit
289 | pygame.quit()
290 | sys.exit()
291 | if event.key in [pygame.K_c]: # clear samples
292 | lst_tgt = []
293 | lst_src = []
294 | if event.key in [pygame.K_o]: # plot one distributio
295 | one_dist = not one_dist
296 | lst_tgt = []
297 | if one_dist:
298 | lst_src = lst_tgt
299 | else:
300 | lst_src = []
301 | if event.key in [pygame.K_t]: # roatte target
302 | if lst_tgt:
303 | lst_tgt = rotate_list(lst_tgt, .1)
304 | if lst_tgt and lst_src:
305 | G = update_G(lst_src, lst_tgt)
306 | if event.key in [pygame.K_s]: # save screen
307 | imgdata = pygame.surfarray.array3d(world)
308 | imgdata = imgdata.swapaxes(0, 1)
309 | scipy.misc.imsave('screen_{}.png'.format(
310 | datetime.datetime.now()), imgdata)
311 | if event.key in [pygame.K_r]: # switch reg ot
312 | use_reg = not use_reg
313 | update_txt(lstf, reg, use_reg)
314 | if lst_tgt and lst_src:
315 | G = update_G(lst_src, lst_tgt)
316 | if event.key in [pygame.K_2, pygame.K_KP2, pygame.K_p]: # increase reg
317 | reg *= 1.5
318 | print(reg)
319 | update_txt(lstf, reg, use_reg)
320 | if lst_tgt and lst_src:
321 | G = update_G(lst_src, lst_tgt)
322 | if event.key in [pygame.K_1, pygame.K_KP1, pygame.K_m]: # decrease reg
323 | reg /= 1.5
324 | print(reg)
325 | update_txt(lstf, reg, use_reg)
326 | if lst_tgt and lst_src:
327 | G = update_G(lst_src, lst_tgt)
328 | if event.key in [pygame.K_SPACE]: # show OT plan or not
329 | plot_G = not plot_G
330 | if event.key in [pygame.K_e]: # force equal number of nodes
331 | imax = min(len(lst_tgt), len(lst_src))
332 | random.shuffle(lst_tgt)
333 | random.shuffle(lst_src)
334 | lst_tgt = lst_tgt[:imax]
335 | lst_src = lst_src[:imax]
336 | if lst_tgt and lst_src:
337 | G = update_G(lst_src, lst_tgt)
338 | if event.key in [pygame.K_w]: # add wall
339 | wall = not wall
340 | if wall:
341 | id_metric = 3
342 | else:
343 | id_metric = 0
344 | update_txt(lstf, reg, use_reg)
345 | if lst_tgt and lst_src:
346 | G = update_G(lst_src, lst_tgt)
347 | if event.key in [pygame.K_b]: # add wall
348 | plot_bary = not plot_bary
349 | if event.key in [pygame.K_i]: # add wall
350 | alpha_bary = max((alpha_bary+0.1)%1,0.1)
351 | if event.key in [pygame.K_d]:
352 | id_metric = (id_metric+1) % len(lst_metrics)
353 | wall = id_metric == 3
354 | update_txt(lstf, reg, use_reg)
355 | if lst_tgt and lst_src:
356 | G = update_G(lst_src, lst_tgt)
357 |
358 | if event.type == pygame.MOUSEBUTTONDOWN:
359 | pos = pygame.mouse.get_pos()
360 | # print(event)
361 | if event.button == 1:
362 | i = find_overlap(pos, lst_src)
363 | if i < 0:
364 | lst_src.append(pos)
365 | else:
366 | del lst_src[i]
367 | elif event.button == 3:
368 | i = find_overlap(pos, lst_tgt)
369 | if i < 0:
370 | lst_tgt.append(pos)
371 | else:
372 | del lst_tgt[i]
373 |
374 | if lst_tgt and lst_src:
375 | G = update_G(lst_src, lst_tgt)
376 |
377 | world.fill((255, 255, 255))
378 |
379 | if (wall and id_metric == 3) or id_metric == 4:
380 | pygame.draw.rect(world, color_wall, wall_pos)
381 |
382 | # print all texts
383 | if plot_G:
384 | if lst_tgt and lst_src:
385 | draw_G(world, G, lst_src, lst_tgt)
386 | else:
387 | if lst_tgt and lst_src and plot_bary:
388 | draw_G(world, G, lst_src, lst_tgt,only_bary=True)
389 |
390 | for pos in lst_src:
391 | draw_source(world, pos)
392 | for pos in lst_tgt:
393 | draw_target(world, pos)
394 |
395 | screen.blit(world, (0, 0))
396 |
397 | for t, p in zip(lstf, lstp):
398 | screen.blit(t, p)
399 |
400 | pygame.display.flip() # RENDER WINDOW
401 |
--------------------------------------------------------------------------------
/demos/demo_spectrum.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 | # -*- coding: utf-8 -*-
3 | """
4 | MIT License
5 |
6 | Copyright (c) 2020 Rémi Flamary
7 |
8 | Permission is hereby granted, free of charge, to any person obtaining a copy
9 | of this software and associated documentation files (the "Software"), to deal
10 | in the Software without restriction, including without limitation the rights
11 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
12 | copies of the Software, and to permit persons to whom the Software is
13 | furnished to do so, subject to the following conditions:
14 |
15 | The above copyright notice and this permission notice shall be included in all
16 | copies or substantial portions of the Software.
17 |
18 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
19 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
20 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
21 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
22 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
23 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
24 | SOFTWARE.
25 |
26 | """
27 |
28 | import pyaudio
29 | import numpy as np
30 | import sys, pygame
31 | import scipy.fftpack
32 |
33 |
34 | def get_pos(txt,y,x=0):
35 | pos=txt.get_rect()
36 | pos.left=x
37 | pos.centery=y
38 | return pos
39 |
40 | # print methods texts (with a x for selected method)
41 | def get_txtmethods(method,methodsname):
42 | tmethods=[]
43 | pmethods=[]
44 | for i,name in enumerate(methodsname):
45 | if i==method:
46 | txt='[{}] {}'.format('x',name)
47 | else:
48 | txt='[{}] {}'.format(' ',name)
49 | temp=font4.render(txt, 1, color_text)
50 | tmethods.append(temp)
51 | ptemp=get_pos(temp,i*20+40)
52 | pmethods.append(ptemp)
53 | return tmethods,pmethods
54 |
55 | # function to detect if key is caps or not (pygame doe not provide it...)
56 | def is_caps():
57 | return pygame.key.get_mods() & pygame.KMOD_SHIFT or pygame.key.get_mods() & pygame.KMOD_CAPS
58 |
59 |
60 | # sampling parameters for pyaudio
61 | CHUNK = 4096
62 | WIDTH = 2
63 | CHANNELS = 1
64 | RATE = 44100
65 |
66 |
67 | # init pygame
68 | pygame.init()
69 |
70 | # screen size
71 | size = width, height = 1024, 768
72 | screen = pygame.display.set_mode(size)
73 | pygame.display.set_caption('Periodogram demonstration')
74 |
75 | # color palette
76 | pal = [(max((x-128)*2,0),x,min(x*2,255)) for x in range(256)]
77 |
78 | # background image
79 | world=pygame.Surface((width,height),depth=8) # MAIN SURFACE
80 | world.set_palette(pal)
81 |
82 | data=np.array(np.zeros((height,width)),dtype=int)
83 |
84 | # column start (width of left texts)
85 | cstart=150
86 |
87 | # current col
88 | col=cstart
89 |
90 | # various scalings
91 | sc_pow=20
92 | sc_spec=100
93 | sc_prop0=2.5
94 |
95 | maxpower=167
96 | nfft=CHUNK
97 | nfftvisu=400
98 | nspec=200
99 |
100 |
101 | pmax=0
102 | pw=0;
103 |
104 | fmax=nfftvisu*RATE/nfft
105 |
106 |
107 | color_title=(150,150,255)
108 | color_text=(200,200,255)
109 |
110 | # prepare texts
111 | font = pygame.font.Font(None, 25) # title
112 | font2 = pygame.font.Font(None, 20) # text
113 | font4 = pygame.font.Font(None, 20)
114 |
115 |
116 | # all screen texts
117 | txtpower=font.render('Power', 1, color_title)
118 | txtpowerpos = get_pos(txtpower,height-maxpower+20)
119 |
120 | # power scale
121 | tpscale=font2.render('Scale={} (P/p)'.format(sc_pow), 1, color_text)
122 | ppscale = get_pos(tpscale,height-maxpower+40)
123 |
124 | #spetrogram
125 | txtspg=font.render('Spectrogram', 1, color_title)
126 | txtspgpos = get_pos(txtspg,height-nfftvisu-maxpower+20)
127 |
128 | tfmax=font2.render('Fmax={:5.1f} Hz'.format(fmax), 1, color_text)
129 | pfmax = get_pos(tfmax,height-nfftvisu-maxpower+40)
130 |
131 |
132 | tnfft=font2.render('NFFT={} (N/n)'.format(nfft), 1, color_text)
133 | pnfft = get_pos(tnfft,height-nfftvisu-maxpower+60)
134 |
135 | tpscale2=font2.render('Scale={} (S/s)'.format(sc_spec), 1, color_text)
136 | ppscale2 = get_pos(tpscale2,height-nfftvisu-maxpower+80)
137 |
138 |
139 | # Spectrum
140 | tspec=font.render('Spectrum', 1, color_title)
141 | pspec = get_pos(tspec,height-nfftvisu-maxpower-nspec+20)
142 |
143 |
144 |
145 |
146 | rho=0.2 # scaling spectrum
147 | old_spec=[]
148 |
149 |
150 |
151 | step=3 # step for printing unmixing
152 | pause=0 # pause (or not)
153 |
154 |
155 | # init pyaudio
156 | p = pyaudio.PyAudio()
157 | stream = p.open(format=p.get_format_from_width(WIDTH),
158 | channels=CHANNELS,
159 | rate=RATE,
160 | input=True,
161 | output=False,
162 | frames_per_buffer=CHUNK)
163 |
164 |
165 | while 1:
166 |
167 | # keyboard events
168 | for event in pygame.event.get(): #check if we need to exit
169 | if event.type == pygame.QUIT:
170 | pygame.quit();
171 | stream.stop_stream()
172 | stream.close()
173 | p.terminate()
174 | sys.exit()
175 | if event.type == pygame.KEYDOWN:
176 | #print event.key
177 | if event.key in [pygame.K_ESCAPE,pygame.K_q] :
178 | pygame.quit
179 | stream.stop_stream()
180 | stream.close()
181 | p.terminate()
182 | sys.exit()
183 | if event.key in [pygame.K_PLUS,270] :
184 | nfft*=2
185 | print('nfft:', nfft)
186 | if event.key in [pygame.K_MINUS,269] :
187 | if nfft>nfftvisu*2:
188 | if nfft>2048:
189 | nfft=int(nfft/2)
190 | print( 'nfft:',nfft)
191 | if event.key in [pygame.K_n] :
192 | if is_caps():
193 | nfft*=2
194 | elif nfft>nfftvisu*2:
195 | if nfft>2048:
196 | nfft=int(nfft/2)
197 | CHUNK=nfft
198 | print( 'nfft:',nfft)
199 | if event.key in [pygame.K_w] :
200 | if is_caps():
201 | CHUNK*=2
202 | elif CHUNK>512:
203 | CHUNK=int(CHUNK/2)
204 | nfft=CHUNK
205 | print('frame:',CHUNK)
206 |
207 | if event.key in [pygame.K_p] :
208 | if is_caps():
209 | sc_pow*=1.5
210 | else:
211 | sc_pow/=1.5
212 | print('Pow scale:',sc_pow)
213 |
214 | if event.key in [pygame.K_b] :
215 | pmax=pw
216 | print('Pow bkgrnd:',pmax)
217 |
218 | if event.key in [pygame.K_s] :
219 | if is_caps():
220 | sc_spec*=1.2
221 | else:
222 | sc_spec/=1.2
223 | if event.key in [pygame.K_SPACE] :
224 | if pause:
225 | pause=False
226 | else:
227 | pause=True
228 | print('Pause')
229 |
230 |
231 | if event.key in [pygame.K_r] :
232 | col=cstart
233 | data[:,:]=0
234 | print('reset!')
235 |
236 | # update text
237 | fmax=nfftvisu*RATE/nfft
238 | tnfft=font2.render('NFFT={} (N/n)'.format(nfft), 1, color_text)
239 | tpscale=font2.render('Scale={:4.1f} (P/p)'.format(sc_pow), 1, color_text)
240 | tfmax=font2.render('Fmax={:5.1f} Hz'.format(fmax), 1, color_text)
241 | tpscale2=font2.render('Scale={:4.1f} (S/s)'.format(sc_spec), 1, color_text)
242 |
243 |
244 | dt = stream.read(CHUNK)
245 | sig = (np.fromstring(dt, dtype=np.int16))*1.0/32768
246 | sig-=sig.mean()
247 |
248 | if not pause:
249 |
250 | pw0=np.sum(sig**2)
251 |
252 | #pmax=max(pw0,pmax)
253 | pw=np.log10(pw0)+3
254 | if np.isnan(pw):
255 | pw=0
256 | if pmax==0:
257 | pmax=pw
258 | print('Pow bkgrnd:',pmax)
259 |
260 | data[:,col:col+step]=0
261 | if (pw*sc_pow)>0:
262 | data[-max(int(pw*sc_pow),0):,col:col+step]=255
263 |
264 |
265 | # curent cursor
266 | if col0:
289 | data[-int(spec2[i]*0.7*(nspec)/255)-maxpower-nfftvisu:-maxpower-nfftvisu,cstart+i]=255
290 |
291 |
292 |
293 | # increment current col by step
294 | col+=step
295 | if col>=width-step:
296 | col=cstart
297 |
298 |
299 | # add separators
300 | data[:,cstart]=128
301 | data[-maxpower,:]=128 # power horz line
302 | data[-maxpower-nfftvisu-1,:]=128 # spectrum horz line
303 | data[-maxpower-nfftvisu-nspec-1,:]=128 # spectrum horz line
304 |
305 | # appedn data to the window
306 | pygame.surfarray.blit_array(world,data.T) #place data in window
307 |
308 |
309 | # print all texts
310 | screen.blit(world, (0,0))
311 | screen.blit(txtpower, txtpowerpos)
312 | screen.blit(txtspg, txtspgpos)
313 | screen.blit(tnfft, pnfft)
314 | screen.blit(tpscale, ppscale)
315 | screen.blit(tpscale2, ppscale2)
316 | screen.blit(tfmax, pfmax)
317 | screen.blit(tspec, pspec)
318 | pygame.display.flip() #RENDER WINDOW
319 |
320 |
321 |
--------------------------------------------------------------------------------
/demos/demo_style_transfer.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 | # -*- coding: utf-8 -*-
3 | """
4 | MIT License
5 |
6 | Copyright (c) 2020 Rémi Flamary
7 |
8 | Permission is hereby granted, free of charge, to any person obtaining a copy
9 | of this software and associated documentation files (the "Software"), to deal
10 | in the Software without restriction, including without limitation the rights
11 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
12 | copies of the Software, and to permit persons to whom the Software is
13 | furnished to do so, subject to the following conditions:
14 |
15 | The above copyright notice and this permission notice shall be included in all
16 | copies or substantial portions of the Software.
17 |
18 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
19 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
20 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
21 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
22 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
23 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
24 | SOFTWARE.
25 |
26 | """
27 |
28 | import numpy as np
29 | import cv2
30 | import scipy.ndimage
31 | from PIL import Image
32 | import torch
33 | from torchvision import transforms
34 | from torchvision.utils import save_image
35 | from model_style_transfer import MultiLevelAE
36 | from datetime import datetime
37 | import os
38 | import urllib
39 |
40 | import os
41 | dir_path = os.path.dirname(os.path.realpath(__file__))
42 |
43 | style_path = dir_path+'/../data/styles/'
44 | models_path = dir_path+'/../data/models'
45 | url_models = 'https://remi.flamary.com/download/models/'
46 |
47 | folder = datetime.now().strftime("%Y_%m_%d-%H_%M_%S")
48 |
49 | trans = transforms.Compose([transforms.ToTensor()])
50 |
51 | if not os.path.exists('out'):
52 | os.mkdir('out')
53 |
54 |
55 | lst_model_files = ["decoder_relu1_1.pth",
56 | "decoder_relu2_1.pth",
57 | "decoder_relu3_1.pth",
58 | "decoder_relu4_1.pth",
59 | "decoder_relu5_1.pth",
60 | "vgg_normalised_conv5_1.pth"]
61 |
62 | # test if models already downloaded
63 | for m in lst_model_files:
64 | if not os.path.exists(models_path+'/'+m):
65 | print('Downloading model file : {}'.format(m))
66 | urllib.request.urlretrieve(url_models+m, models_path+'/'+m)
67 |
68 |
69 | idimg = 0
70 |
71 | fname = "out/{}/{}_{}.jpg"
72 |
73 | if torch.cuda.is_available():
74 | device = torch.device(f'cuda')
75 | print(f'# CUDA available: {torch.cuda.get_device_name(0)}')
76 | else:
77 | device = 'cpu'
78 |
79 |
80 | model = MultiLevelAE(models_path)
81 | model = model.to(device)
82 | print("Model loaded")
83 |
84 |
85 | def transfer(c, s, alpha=1):
86 | c = c[:, :, ::-1].copy()
87 | c_tensor = trans(c.astype(np.float32)).unsqueeze(0).to(device)
88 | s_tensor = trans(s.astype(np.float32)).unsqueeze(0).to(device)
89 | with torch.no_grad():
90 | out = model(c_tensor, s_tensor, alpha)
91 | return out.numpy()[0, :, :, :]
92 |
93 |
94 | cam = os.getenv("CAMERA")
95 | if cam is None:
96 | cap = cv2.VideoCapture(0)
97 | else:
98 | cap = cv2.VideoCapture(int(cam))
99 |
100 | lst_style = ['antimonocromatismo.jpg',
101 | 'sketch.png',
102 | 'hosi.jpg',
103 | 'asheville.jpg',
104 | 'brushstrokes.jpg',
105 | 'contrast_of_forms.jpg',
106 | 'en_campo_gris.jpg',
107 | 'la_muse.jpg',
108 | 'mondrian.jpg',
109 | 'picasso_seated_nude_hr.jpg',
110 | 'picasso_self_portrait.jpg',
111 | 'scene_de_rue.jpg',
112 | 'trial.jpg',
113 | 'woman_in_peasant_dress_cropped.jpg',
114 | 'woman_with_hat_matisse.jpg',
115 | 'afremov.jpg',
116 | 'bubbles.jpg',
117 | 'in2.jpg',
118 | 'baiser.jpg',
119 | 'zombie.jpg',
120 | 'nikopol.jpg',
121 | 'miro.jpg',
122 | 'monet1.jpg',
123 | 'monet2.jpg',
124 | 'nikopol.jpg', ]
125 |
126 | lst_style0 = lst_style
127 |
128 | lst_name = [f.split('.')[0] for f in lst_style]
129 |
130 | lst_style = [np.array(Image.open(style_path+f).convert('RGB'))
131 | for f in lst_style]
132 |
133 | col = [255, 1, 1]
134 | max_iters = 1
135 | alpha = 0.8
136 | id_style = 0
137 |
138 | from_RGB = [2, 1, 0]
139 |
140 |
141 | pause = False
142 |
143 | ret, frame0 = cap.read()
144 | frame0 = np.asfortranarray(np.array(frame0)/255)
145 | frame_webcam = frame0
146 | frame_style = frame0*0
147 |
148 |
149 | while (True):
150 | # Capture frame-by-frame
151 | ret, frame = cap.read()
152 |
153 | frame2 = np.array(frame)
154 | frame2 = frame2[:, ::-1, :]
155 |
156 | if not pause:
157 | frame_webcam = frame2
158 |
159 | # Display the images
160 | cv2.imshow('Webcam', frame_webcam)
161 | cv2.imshow('Transferred image', frame_style[:, :, from_RGB])
162 |
163 | cv2.namedWindow('Target Style', cv2.WINDOW_NORMAL)
164 | cv2.imshow('Target Style', lst_style[id_style][:, :, from_RGB])
165 |
166 | # handle inputs
167 | key = cv2.waitKey(1)
168 | if (key & 0xFF) in [ord('q')]:
169 | break
170 | if (key & 0xFF) in [ord('s')]:
171 | id_style = (id_style+1) % len(lst_name)
172 | if (key & 0xFF) in [ord('w')]:
173 | if not os.path.exists('out/{}'.format(folder)):
174 | os.mkdir('out/{}'.format(folder))
175 | cv2.imwrite(fname.format(folder, idimg, '0'), frame_webcam)
176 | cv2.imwrite(fname.format(folder, idimg,
177 | lst_name[id_style]), frame_style[:, :, from_RGB]*255)
178 | print("Images saved")
179 | if (key & 0xFF) in [ord('q')]:
180 | break
181 | if (key & 0xFF) in [ord('r')]:
182 | if not os.path.exists('out/{}'.format(folder)):
183 | os.mkdir('out/{}'.format(folder))
184 | cv2.imwrite(fname.format(folder, idimg, '0'), frame_webcam)
185 | for i in range(len(lst_style)):
186 | temp = np.array(transfer(frame_webcam, lst_style[i], alpha=alpha))
187 | # print(temp)
188 | for k in range(3):
189 | frame_style[:, :, k] = temp[k, :, :]/255
190 | cv2.imwrite(fname.format(folder, idimg,
191 | lst_name[i]), frame_style[:, :, from_RGB]*255)
192 | print('Applied style from file {}'.format(lst_style0[i]))
193 | cv2.imshow('Transferred image ({})'.format(
194 | lst_style0[i]), frame_style[:, :, from_RGB])
195 |
196 | if (key & 0xFF) in [ord('p')]:
197 | pause = not pause
198 | if pause:
199 | idimg += 1
200 | if (key & 0xFF) in [ord('A')]:
201 | alpha = min(1, alpha+0.1)
202 | print('alpha={}'.format(alpha))
203 | if (key & 0xFF) in [ord('a')]:
204 | alpha = max(0, alpha-0.1)
205 | print('alpha={}'.format(alpha))
206 | if (key & 0xFF) in [ord(' ')]:
207 | pause = True
208 | temp = np.array(
209 | transfer(frame_webcam, lst_style[id_style], alpha=alpha))
210 | # print(temp)
211 | for i in range(3):
212 | frame_style[:, :, i] = temp[i, :, :]/255
213 | print('Applied style from file {}'.format(lst_style0[id_style]))
214 |
215 |
216 | # When everything done, release the capture
217 | cap.release()
218 | cv2.destroyAllWindows()
219 |
--------------------------------------------------------------------------------
/demos/demo_tv.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 | # -*- coding: utf-8 -*-
3 | """
4 | MIT License
5 |
6 | Copyright (c) 2020 Rémi Flamary
7 |
8 | Permission is hereby granted, free of charge, to any person obtaining a copy
9 | of this software and associated documentation files (the "Software"), to deal
10 | in the Software without restriction, including without limitation the rights
11 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
12 | copies of the Software, and to permit persons to whom the Software is
13 | furnished to do so, subject to the following conditions:
14 |
15 | The above copyright notice and this permission notice shall be included in all
16 | copies or substantial portions of the Software.
17 |
18 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
19 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
20 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
21 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
22 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
23 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
24 | SOFTWARE.
25 |
26 | """
27 |
28 | import numpy as np
29 | import cv2
30 | import prox_tv as ptv
31 | import os
32 |
33 | cam=os.getenv("CAMERA")
34 | if cam is None:
35 | cap = cv2.VideoCapture(0)
36 | else:
37 | cap = cv2.VideoCapture(int(cam))
38 |
39 | reg=2e-1
40 | alpha=0.2
41 | pred=0
42 | idscreen=0
43 | loop=True
44 | do_tv=False
45 | half=False
46 | noise=False
47 | signoise=0.2
48 | col=[255,1,1]
49 | max_iters=1
50 |
51 | tau_noise=0.002
52 |
53 | ret, frame0 = cap.read()
54 | frame0=np.asfortranarray(np.array(frame0)/255)
55 |
56 |
57 | def solve_tv(new,old,reg):
58 | #return ptv.tv1_2d(new,reg,max_iters=max_iters,n_threads=8)
59 | temp=np.asfortranarray(old)
60 | # une itération
61 | ptv._dr_tv2d(np.asfortranarray(new),reg,temp,max_iters,np.zeros(3),n_threads=4)
62 | return temp
63 |
64 |
65 | while(True):
66 | # Capture frame-by-frame
67 | ret, frame = cap.read()
68 |
69 |
70 | frame2=np.array(frame)/255
71 | frame2=frame2[:,::-1,:]
72 | #frame2=frame2[::2,::2,:]
73 |
74 | if noise:
75 |
76 | # gaussian noise
77 | #frame2+=signoise*np.random.randn(frame2.shape[0],frame2.shape[1],frame2.shape[2])
78 |
79 | # salt and pepper noise
80 | nz=np.sign(np.random.rand(frame2.shape[0],frame2.shape[1])-.5)*(np.random.rand(frame2.shape[0],frame2.shape[1])