├── .gitignore ├── LICENSE ├── README.md ├── docs ├── flow_io.html ├── flowpy.html └── index.html ├── examples ├── flow_backward_warp.py ├── flow_forward_warp.py ├── flow_rgb.py └── flow_with_arrows_tooltip_and_calibration.py ├── flowpy ├── __init__.py ├── flow_io.py └── flowpy.py ├── scripts └── flowread ├── setup.py ├── static ├── Dimetrodon.flo ├── example_arrows.png ├── example_backward_warp.png ├── example_forward_warp.png ├── example_rgb.png ├── kitti_000010_10.png ├── kitti_000010_11.png ├── kitti_noc_000010_10.png └── kitti_occ_000010_10.png └── tests ├── __init__.py └── flow_tests.py /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | pip-wheel-metadata/ 24 | share/python-wheels/ 25 | *.egg-info/ 26 | .installed.cfg 27 | *.egg 28 | MANIFEST 29 | 30 | # PyInstaller 31 | # Usually these files are written by a python script from a template 32 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 33 | *.manifest 34 | *.spec 35 | 36 | # Installer logs 37 | pip-log.txt 38 | pip-delete-this-directory.txt 39 | 40 | # Unit test / coverage reports 41 | htmlcov/ 42 | .tox/ 43 | .nox/ 44 | .coverage 45 | .coverage.* 46 | .cache 47 | nosetests.xml 48 | coverage.xml 49 | *.cover 50 | .hypothesis/ 51 | .pytest_cache/ 52 | 53 | # Translations 54 | *.mo 55 | *.pot 56 | 57 | # Django stuff: 58 | *.log 59 | local_settings.py 60 | db.sqlite3 61 | 62 | # Flask stuff: 63 | instance/ 64 | .webassets-cache 65 | 66 | # Scrapy stuff: 67 | .scrapy 68 | 69 | # Sphinx documentation 70 | docs/_build/ 71 | 72 | # PyBuilder 73 | target/ 74 | 75 | # Jupyter Notebook 76 | .ipynb_checkpoints 77 | 78 | # IPython 79 | profile_default/ 80 | ipython_config.py 81 | 82 | # pyenv 83 | .python-version 84 | 85 | # celery beat schedule file 86 | celerybeat-schedule 87 | 88 | # SageMath parsed files 89 | *.sage.py 90 | 91 | # Environments 92 | .env 93 | .venv 94 | env/ 95 | venv/ 96 | ENV/ 97 | env.bak/ 98 | venv.bak/ 99 | 100 | # Spyder project settings 101 | .spyderproject 102 | .spyproject 103 | 104 | # Rope project settings 105 | .ropeproject 106 | 107 | # mkdocs documentation 108 | /site 109 | 110 | # mypy 111 | .mypy_cache/ 112 | .dmypy.json 113 | dmypy.json 114 | 115 | # Pyre type checker 116 | .pyre/ 117 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Univ. Paris-Saclay, CNRS, CentraleSupelec, 4 | Thales Research & Technology 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy 7 | of this software and associated documentation files (the "Software"), to deal 8 | in the Software without restriction, including without limitation the rights 9 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | copies of the Software, and to permit persons to whom the Software is 11 | furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in all 14 | copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 21 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 22 | SOFTWARE. 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # flowpy 💾 - A python package for working with optical flows 2 | 3 | Optical flow is the displacement map of pixels between two frames. It is a low-level analysis used in many computer vision programs. 4 | 5 | Working with optical flow may be cumbersome: 6 | - It is quite hard to represent it in a comprehensible manner. 7 | - Multiple formats exist for storing it. 8 | 9 | Flowpy provides tools to work with optical flow more easily in python. 10 | 11 | ## Installing 12 | 13 | We recommend using pip: 14 | ```bash 15 | pip install flowpy 16 | ``` 17 | 18 | ## Features 19 | 20 | The main features of flowpy are: 21 | - Reading and writing optical flows in two formats: 22 | - **.flo** (as defined [here](http://vision.middlebury.edu/flow/)) 23 | - **.png** (as defined [here](http://www.cvlibs.net/datasets/kitti/eval_scene_flow.php?benchmark=flow)) 24 | - Visualizing optical flows with matplotlib 25 | - Backward and forward warp 26 | 27 | ## Examples 28 | 29 | ### A simple RGB plot 30 | 31 | This is the simplest example of how to use flowpy, it: 32 | - Reads a file using *flowpy.flow_read*. 33 | - Transforms the flow as an rgb image with *flowpy.flow_to_rgb* and shows it with matplotlib 34 | 35 | #### Code: 36 | ```python 37 | import flowpy 38 | import matplotlib.pyplot as plt 39 | 40 | flow = flowpy.flow_read("tests/data/kitti_occ_000010_10.flo") 41 | 42 | fig, ax = plt.subplots() 43 | ax.imshow(flowpy.flow_to_rgb(flow)) 44 | plt.show() 45 | ``` 46 | 47 | #### Result: 48 | ![simple_example] 49 | 50 | *Sample image from the [KITTI](http://www.cvlibs.net/datasets/kitti/eval_scene_flow.php?benchmark=flow) dataset* 51 | 52 | ### Plotting arrows, showing flow values and a calibration pattern 53 | 54 | Flowpy comes with more than just RGB plots, the main features here are: 55 | - Arrows to quickly visualize the flow 56 | - The flow values below cursor showing in the tooltips 57 | - A calibration pattern side by side as a legend for your graph 58 | 59 | #### Code: 60 | ```python 61 | flow = flowpy.flow_read("tests/data/Dimetrodon.flo") 62 | height, width, _ = flow.shape 63 | 64 | image_ratio = height / width 65 | max_radius = flowpy.get_flow_max_radius(flow) 66 | 67 | fig, (ax_1, ax_2) = plt.subplots( 68 | 1, 2, gridspec_kw={"width_ratios": [1, image_ratio]} 69 | ) 70 | 71 | ax_1.imshow(flowpy.flow_to_rgb(flow)) 72 | flowpy.attach_arrows(ax_1, flow) 73 | flowpy.attach_coord(ax_1, flow) 74 | 75 | flowpy.attach_calibration_pattern(ax_2, flow_max_radius=max_radius) 76 | 77 | plt.show() 78 | ``` 79 | 80 | #### Result: 81 | ![complex_example] 82 | 83 | *Sample image from the [Middlebury](http://vision.middlebury.edu/flow/data/) dataset* 84 | 85 | ### Warping images (backward): 86 | If you know the flow (first_image -> second_image), you can backward warp the second_image back to first_image. 87 | 88 | ```python 89 | flow = flowpy.flow_read("static/kitti_occ_000010_10.png") 90 | first_image = np.asarray(Image.open("static/kitti_000010_10.png")) 91 | second_image = np.asarray(Image.open("static/kitti_000010_11.png")) 92 | 93 | flow[np.isnan(flow)] = 0 94 | warped_first_image = flowpy.backward_warp(second_image, flow) 95 | 96 | fig, axes = plt.subplots(3, 1) 97 | for ax, image, title in zip(axes, (first_image, second_image, warped_first_image), 98 | ("First Image", "Second Image", "Second image warped to first image")): 99 | ax.imshow(image) 100 | ax.set_title(title) 101 | ax.set_axis_off() 102 | 103 | plt.show() 104 | ``` 105 | 106 | #### Result: 107 | ![backward_warp_example] 108 | 109 | Note that the artifacts in the warp are normal, they are caused by unknown flows and occlusions. 110 | 111 | ### Warping images (forward): 112 | 113 | Forward warp is often less used as it is quite more complex. It relies on a k-nearest neighbor search instead of direct bi-linear interpolation. 114 | 115 | `forward_warp` is about 10x slower than `backward_warp` but you still may find it useful. 116 | 117 | ```python 118 | flow = flowpy.flow_read("static/kitti_occ_000010_10.png") 119 | first_image = np.asarray(Image.open("static/kitti_000010_10.png")) 120 | second_image = np.asarray(Image.open("static/kitti_000010_11.png")) 121 | 122 | flow[np.isnan(flow)] = 0 123 | warped_second_image = flowpy.forward_warp(first_image, flow) 124 | 125 | fig, ax = plt.subplots() 126 | 127 | ax.imshow(warped_second_image) 128 | ax.set_title( "First image warped to the second") 129 | ax.set_axis_off() 130 | 131 | plt.show() 132 | ``` 133 | 134 | #### Result: 135 | ![forward_warp_example] 136 | 137 | 138 | ### More 139 | 140 | You can find the above examples in the `examples` folder. You can also look in `tests`. 141 | If you encounter a bug or have an idea for a new feature, feel free to open an issue. 142 | 143 | Most of the visualization and io handling has been translated from matlab and c code from the [Middlebury flow code](http://vision.middlebury.edu/flow/code/flow-code/). 144 | Credits to thank Simon Baker, Daniel Scharste, J. P. Lewis, Stefan Roth, Michael J. Black and Richard Szeliski. 145 | 146 | [simple_example]: https://raw.githubusercontent.com/mickaelseznec/flowpy/master/static/example_rgb.png "Displaying an optical flow as an RGB image" 147 | [complex_example]: https://raw.githubusercontent.com/mickaelseznec/flowpy/master/static/example_arrows.png "Displaying an optical flow as an RGB image with arrows, tooltip and legend" 148 | [backward_warp_example]: https://raw.githubusercontent.com/mickaelseznec/flowpy/master/static/example_backward_warp.png "An example of backward warp" 149 | [forward_warp_example]: https://raw.githubusercontent.com/mickaelseznec/flowpy/master/static/example_forward_warp.png "An example of forward warp" 150 | -------------------------------------------------------------------------------- /docs/flow_io.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | flowpy.flow_io API documentation 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 |
18 |
19 |
20 |

Module flowpy.flow_io

21 |
22 |
23 |
24 | 25 | Expand source code 26 | 27 | 
import numpy as np
 28 | import png
 29 | import struct
 30 | 
 31 | from pathlib import Path
 32 | from warnings import warn
 33 | 
 34 | 
 35 | def flow_write(output_file, flow, format=None):
 36 |     """
 37 |     Writes optical flow to file.
 38 | 
 39 |     Parameters
 40 |     ----------
 41 |     output_file: {str, pathlib.Path, file}
 42 |         Path of the file to write or file object.
 43 |     flow: numpy.ndarray
 44 |         3D flow in the HWF (Height, Width, Flow) layout.
 45 |         flow[..., 0] should be the x-displacement
 46 |         flow[..., 1] should be the y-displacement
 47 |     format: str, optional
 48 |         Specify in what format the flow is written, accepted formats: "png" or "flo"
 49 |         If None, it is guessed on the file extension
 50 | 
 51 |     See Also
 52 |     --------
 53 |     flow_read
 54 | 
 55 |     """
 56 | 
 57 |     supported_formats = ("png", "flo")
 58 | 
 59 |     output_format = guess_extension(output_file, override=format)
 60 | 
 61 |     with FileManager(output_file, "wb") as f:
 62 |         if output_format == "png":
 63 |             flow_write_png(f, flow)
 64 |         else:
 65 |             flow_write_flo(f, flow)
 66 | 
 67 | 
 68 | def flow_read(input_file, format=None):
 69 |     """
 70 |     Reads optical flow from file
 71 | 
 72 |     Parameters
 73 |     ----------
 74 |     output_file: {str, pathlib.Path, file}
 75 |         Path of the file to read or file object.
 76 |     format: str, optional
 77 |         Specify in what format the flow is raed, accepted formats: "png" or "flo"
 78 |         If None, it is guess on the file extension
 79 | 
 80 |     Returns
 81 |     -------
 82 |     flow: numpy.ndarray
 83 |         3D flow in the HWF (Height, Width, Flow) layout.
 84 |         flow[..., 0] is the x-displacement
 85 |         flow[..., 1] is the y-displacement
 86 | 
 87 |     Notes
 88 |     -----
 89 | 
 90 |     The flo format is dedicated to optical flow and was first used in Middlebury optical flow database.
 91 |     The original defition can be found here: http://vision.middlebury.edu/flow/code/flow-code/flowIO.cpp
 92 | 
 93 |     The png format uses 16-bit RGB png to store optical flows.
 94 |     It was developped along with the KITTI Vision Benchmark Suite.
 95 |     More information can be found here: http://www.cvlibs.net/datasets/kitti/eval_scene_flow.php?benchmark=flow
 96 | 
 97 |     The both handle flow with invalid ``invalid'' values, to deal with occlusion for example.
 98 |     We convert such invalid values to NaN.
 99 | 
100 |     See Also
101 |     --------
102 |     flow_write
103 | 
104 |     """
105 | 
106 |     input_format = guess_extension(input_file, override=format)
107 | 
108 |     with FileManager(input_file, "rb") as f:
109 |         if input_format == "png":
110 |             output = flow_read_png(f)
111 |         else:
112 |             output = flow_read_flo(f)
113 | 
114 |     return output
115 | 
116 | 
117 | def flow_read_flo(f):
118 |     if (f.read(4) != b'PIEH'):
119 |         warn("{} does not have a .flo file signature".format(f.name))
120 | 
121 |     width, height = struct.unpack("II", f.read(8))
122 |     result = np.fromfile(f, dtype="float32").reshape((height, width, 2))
123 | 
124 |     # Set invalid flows to NaN
125 |     mask_u = np.greater(np.abs(result[..., 0]), 1e9, where=(~np.isnan(result[..., 0])))
126 |     mask_v = np.greater(np.abs(result[..., 1]), 1e9, where=(~np.isnan(result[..., 1])))
127 | 
128 |     result[mask_u | mask_v] = np.NaN
129 | 
130 |     return result
131 | 
132 | 
133 | def flow_write_flo(f, flow):
134 |     SENTINEL = 1666666800.0  # Only here to look like Middlebury original files
135 |     height, width, _ = flow.shape
136 | 
137 |     image = flow.copy()
138 |     image[np.isnan(image)] = SENTINEL
139 | 
140 |     f.write(b'PIEH')
141 |     f.write(struct.pack("II", width, height))
142 |     image.astype(np.float32).tofile(f)
143 | 
144 | 
145 | def flow_read_png(f):
146 |     width, height, stream, *_ = png.Reader(f).read()
147 | 
148 |     file_content = np.concatenate(list(stream)).reshape((height, width, 3))
149 |     flow, valid = file_content[..., 0:2], file_content[..., 2]
150 | 
151 |     flow = (flow.astype(np.float) - 2 ** 15) / 64.
152 | 
153 |     flow[~valid.astype(np.bool)] = np.NaN
154 | 
155 |     return flow
156 | 
157 | 
158 | def flow_write_png(f, flow):
159 |     SENTINEL = 0.  # Only here to look like original KITTI files
160 |     height, width, _ = flow.shape
161 |     flow_copy = flow.copy()
162 | 
163 |     valid = ~(np.isnan(flow[..., 0]) | np.isnan(flow[..., 1]))
164 |     flow_copy[~valid] = SENTINEL
165 | 
166 |     flow_copy = (flow_copy * 64. + 2 ** 15).astype(np.uint16)
167 |     image = np.dstack((flow_copy, valid))
168 | 
169 |     writer = png.Writer(width, height, bitdepth=16, greyscale=False)
170 |     writer.write(f, image.reshape((height, 3 * width)))
171 | 
172 | 
173 | class FileManager:
174 |     def __init__(self, abstract_file, mode):
175 |         self.abstract_file = abstract_file
176 |         self.opened_file = None
177 |         self.mode = mode
178 | 
179 |     def __enter__(self):
180 |         if isinstance(self.abstract_file, str):
181 |             self.opened_file = open(self.abstract_file, self.mode)
182 |         elif isinstance(self.abstract_file, Path):
183 |             self.opened_file = self.abstract_file.open(self.mode)
184 |         else:
185 |             return self.abstract_file
186 | 
187 |         return self.opened_file
188 | 
189 |     def __exit__(self, exc_type, exc_value, traceback):
190 |         if self.opened_file is not None:
191 |             self.opened_file.close()
192 | 
193 | 
194 | def guess_extension(abstract_file, override=None):
195 |     if override is not None:
196 |         return override
197 | 
198 |     if isinstance(abstract_file, str):
199 |         return Path(abstract_file).suffix[1:]
200 |     elif isinstance(abstract_file, Path):
201 |         return abstract_file.suffix[1:]
202 | 
203 |     return Path(abstract_file.name).suffix[1:]
204 |
205 |
206 |
207 |
208 |
209 |
210 |
211 |

Functions

212 |
213 |
214 | def flow_read(input_file, format=None) 215 |
216 |
217 |

Reads optical flow from file

218 |

Parameters

219 |
220 |
output_file : {str, pathlib.Path, file}
221 |
Path of the file to read or file object.
222 |
format : str, optional
223 |
Specify in what format the flow is raed, accepted formats: "png" or "flo" 224 | If None, it is guess on the file extension
225 |
226 |

Returns

227 |
228 |
flow : numpy.ndarray
229 |
3D flow in the HWF (Height, Width, Flow) layout. 230 | flow[…, 0] is the x-displacement 231 | flow[…, 1] is the y-displacement
232 |
233 |

Notes

234 |

The flo format is dedicated to optical flow and was first used in Middlebury optical flow database. 235 | The original defition can be found here: http://vision.middlebury.edu/flow/code/flow-code/flowIO.cpp

236 |

The png format uses 16-bit RGB png to store optical flows. 237 | It was developped along with the KITTI Vision Benchmark Suite. 238 | More information can be found here: http://www.cvlibs.net/datasets/kitti/eval_scene_flow.php?benchmark=flow

239 |

The both handle flow with invalid ``invalid'' values, to deal with occlusion for example. 240 | We convert such invalid values to NaN.

241 |

See Also

242 |

flow_write()

243 |
244 | 245 | Expand source code 246 | 247 | 
def flow_read(input_file, format=None):
248 |     """
249 |     Reads optical flow from file
250 | 
251 |     Parameters
252 |     ----------
253 |     output_file: {str, pathlib.Path, file}
254 |         Path of the file to read or file object.
255 |     format: str, optional
256 |         Specify in what format the flow is raed, accepted formats: "png" or "flo"
257 |         If None, it is guess on the file extension
258 | 
259 |     Returns
260 |     -------
261 |     flow: numpy.ndarray
262 |         3D flow in the HWF (Height, Width, Flow) layout.
263 |         flow[..., 0] is the x-displacement
264 |         flow[..., 1] is the y-displacement
265 | 
266 |     Notes
267 |     -----
268 | 
269 |     The flo format is dedicated to optical flow and was first used in Middlebury optical flow database.
270 |     The original defition can be found here: http://vision.middlebury.edu/flow/code/flow-code/flowIO.cpp
271 | 
272 |     The png format uses 16-bit RGB png to store optical flows.
273 |     It was developped along with the KITTI Vision Benchmark Suite.
274 |     More information can be found here: http://www.cvlibs.net/datasets/kitti/eval_scene_flow.php?benchmark=flow
275 | 
276 |     The both handle flow with invalid ``invalid'' values, to deal with occlusion for example.
277 |     We convert such invalid values to NaN.
278 | 
279 |     See Also
280 |     --------
281 |     flow_write
282 | 
283 |     """
284 | 
285 |     input_format = guess_extension(input_file, override=format)
286 | 
287 |     with FileManager(input_file, "rb") as f:
288 |         if input_format == "png":
289 |             output = flow_read_png(f)
290 |         else:
291 |             output = flow_read_flo(f)
292 | 
293 |     return output
294 |
295 |
296 |
297 | def flow_read_flo(f) 298 |
299 |
300 |
301 |
302 | 303 | Expand source code 304 | 305 | 
def flow_read_flo(f):
306 |     if (f.read(4) != b'PIEH'):
307 |         warn("{} does not have a .flo file signature".format(f.name))
308 | 
309 |     width, height = struct.unpack("II", f.read(8))
310 |     result = np.fromfile(f, dtype="float32").reshape((height, width, 2))
311 | 
312 |     # Set invalid flows to NaN
313 |     mask_u = np.greater(np.abs(result[..., 0]), 1e9, where=(~np.isnan(result[..., 0])))
314 |     mask_v = np.greater(np.abs(result[..., 1]), 1e9, where=(~np.isnan(result[..., 1])))
315 | 
316 |     result[mask_u | mask_v] = np.NaN
317 | 
318 |     return result
319 |
320 |
321 |
322 | def flow_read_png(f) 323 |
324 |
325 |
326 |
327 | 328 | Expand source code 329 | 330 | 
def flow_read_png(f):
331 |     width, height, stream, *_ = png.Reader(f).read()
332 | 
333 |     file_content = np.concatenate(list(stream)).reshape((height, width, 3))
334 |     flow, valid = file_content[..., 0:2], file_content[..., 2]
335 | 
336 |     flow = (flow.astype(np.float) - 2 ** 15) / 64.
337 | 
338 |     flow[~valid.astype(np.bool)] = np.NaN
339 | 
340 |     return flow
341 |
342 |
343 |
344 | def flow_write(output_file, flow, format=None) 345 |
346 |
347 |

Writes optical flow to file.

348 |

Parameters

349 |
350 |
output_file : {str, pathlib.Path, file}
351 |
Path of the file to write or file object.
352 |
flow : numpy.ndarray
353 |
3D flow in the HWF (Height, Width, Flow) layout. 354 | flow[…, 0] should be the x-displacement 355 | flow[…, 1] should be the y-displacement
356 |
format : str, optional
357 |
Specify in what format the flow is written, accepted formats: "png" or "flo" 358 | If None, it is guessed on the file extension
359 |
360 |

See Also

361 |

flow_read()

362 |
363 | 364 | Expand source code 365 | 366 | 
def flow_write(output_file, flow, format=None):
367 |     """
368 |     Writes optical flow to file.
369 | 
370 |     Parameters
371 |     ----------
372 |     output_file: {str, pathlib.Path, file}
373 |         Path of the file to write or file object.
374 |     flow: numpy.ndarray
375 |         3D flow in the HWF (Height, Width, Flow) layout.
376 |         flow[..., 0] should be the x-displacement
377 |         flow[..., 1] should be the y-displacement
378 |     format: str, optional
379 |         Specify in what format the flow is written, accepted formats: "png" or "flo"
380 |         If None, it is guessed on the file extension
381 | 
382 |     See Also
383 |     --------
384 |     flow_read
385 | 
386 |     """
387 | 
388 |     supported_formats = ("png", "flo")
389 | 
390 |     output_format = guess_extension(output_file, override=format)
391 | 
392 |     with FileManager(output_file, "wb") as f:
393 |         if output_format == "png":
394 |             flow_write_png(f, flow)
395 |         else:
396 |             flow_write_flo(f, flow)
397 |
398 |
399 |
400 | def flow_write_flo(f, flow) 401 |
402 |
403 |
404 |
405 | 406 | Expand source code 407 | 408 | 
def flow_write_flo(f, flow):
409 |     SENTINEL = 1666666800.0  # Only here to look like Middlebury original files
410 |     height, width, _ = flow.shape
411 | 
412 |     image = flow.copy()
413 |     image[np.isnan(image)] = SENTINEL
414 | 
415 |     f.write(b'PIEH')
416 |     f.write(struct.pack("II", width, height))
417 |     image.astype(np.float32).tofile(f)
418 |
419 |
420 |
421 | def flow_write_png(f, flow) 422 |
423 |
424 |
425 |
426 | 427 | Expand source code 428 | 429 | 
def flow_write_png(f, flow):
430 |     SENTINEL = 0.  # Only here to look like original KITTI files
431 |     height, width, _ = flow.shape
432 |     flow_copy = flow.copy()
433 | 
434 |     valid = ~(np.isnan(flow[..., 0]) | np.isnan(flow[..., 1]))
435 |     flow_copy[~valid] = SENTINEL
436 | 
437 |     flow_copy = (flow_copy * 64. + 2 ** 15).astype(np.uint16)
438 |     image = np.dstack((flow_copy, valid))
439 | 
440 |     writer = png.Writer(width, height, bitdepth=16, greyscale=False)
441 |     writer.write(f, image.reshape((height, 3 * width)))
442 |
443 |
444 |
445 | def guess_extension(abstract_file, override=None) 446 |
447 |
448 |
449 |
450 | 451 | Expand source code 452 | 453 | 
def guess_extension(abstract_file, override=None):
454 |     if override is not None:
455 |         return override
456 | 
457 |     if isinstance(abstract_file, str):
458 |         return Path(abstract_file).suffix[1:]
459 |     elif isinstance(abstract_file, Path):
460 |         return abstract_file.suffix[1:]
461 | 
462 |     return Path(abstract_file.name).suffix[1:]
463 |
464 |
465 |
466 |
467 |
468 |

Classes

469 |
470 |
471 | class FileManager 472 | (abstract_file, mode) 473 |
474 |
475 |
476 |
477 | 478 | Expand source code 479 | 480 | 
class FileManager:
481 |     def __init__(self, abstract_file, mode):
482 |         self.abstract_file = abstract_file
483 |         self.opened_file = None
484 |         self.mode = mode
485 | 
486 |     def __enter__(self):
487 |         if isinstance(self.abstract_file, str):
488 |             self.opened_file = open(self.abstract_file, self.mode)
489 |         elif isinstance(self.abstract_file, Path):
490 |             self.opened_file = self.abstract_file.open(self.mode)
491 |         else:
492 |             return self.abstract_file
493 | 
494 |         return self.opened_file
495 | 
496 |     def __exit__(self, exc_type, exc_value, traceback):
497 |         if self.opened_file is not None:
498 |             self.opened_file.close()
499 |
500 |
501 |
502 |
503 |
504 | 535 |
536 | 539 | 540 | 541 | 542 | -------------------------------------------------------------------------------- /docs/flowpy.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | flowpy.flowpy API documentation 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 |
18 |
19 |
20 |

Module flowpy.flowpy

21 |
22 |
23 |
24 | 25 | Expand source code 26 | 27 | 
import matplotlib.pyplot as plt
 28 | import numpy as np
 29 | 
 30 | from collections import namedtuple
 31 | from itertools import accumulate
 32 | from matplotlib.ticker import AutoMinorLocator
 33 | 
 34 | DEFAULT_TRANSITIONS = (15, 6, 4, 11, 13, 6)
 35 | 
 36 | 
 37 | def flow_to_rgb(flow, flow_max_radius=None, background="bright", custom_colorwheel=None):
 38 |     """
 39 |     Creates a RGB representation of an optical flow.
 40 | 
 41 |     Parameters
 42 |     ----------
 43 |     flow: numpy.ndarray
 44 |         3D flow in the HWF (Height, Width, Flow) layout.
 45 |         flow[..., 0] should be the x-displacement
 46 |         flow[..., 1] should be the y-displacement
 47 | 
 48 |     flow_max_radius: float, optionnal
 49 |         Set the radius that gives the maximum color intensity, useful for comparing different flows.
 50 |         Default: The normalization is based on the input flow maximum radius.
 51 | 
 52 |     background: str, optionnal
 53 |         States if zero-valued flow should look 'bright' or 'dark'
 54 |         Default: "bright"
 55 | 
 56 |     custom_colorwheel: numpy.ndarray
 57 |         Use a custom colorwheel for specific hue transition lengths.
 58 |         By default, the default transition lengths are used.
 59 | 
 60 |     Returns
 61 |     -------
 62 |     rgb_image: numpy.ndarray
 63 |         A 2D RGB image that represents the flow
 64 | 
 65 |     See Also
 66 |     --------
 67 |     make_colorwheel
 68 | 
 69 |     """
 70 | 
 71 |     valid_backgrounds = ("bright", "dark")
 72 |     if background not in valid_backgrounds:
 73 |         raise ValueError("background should be one the following: {}, not {}".format(
 74 |             valid_backgrounds, background))
 75 | 
 76 |     wheel = make_colorwheel() if custom_colorwheel is None else custom_colorwheel
 77 | 
 78 |     flow_height, flow_width, _ = flow.shape
 79 | 
 80 |     complex_flow = flow[..., 0] + 1j * flow[..., 1]
 81 |     complex_flow, nan_mask = replace_nans(complex_flow)
 82 | 
 83 |     radius, angle = np.abs(complex_flow), np.angle(complex_flow)
 84 | 
 85 |     if flow_max_radius is None:
 86 |         flow_max_radius = np.max(radius)
 87 | 
 88 |     if flow_max_radius > 0:
 89 |         radius /= flow_max_radius
 90 | 
 91 |     ncols = len(wheel)
 92 | 
 93 |     # Map the angles from (-pi, pi] to [0, ncols - 1)
 94 |     angle = (-angle + np.pi) * ((ncols - 1) / (2 * np.pi))
 95 | 
 96 |     # Interpolate the hues
 97 |     (angle_fractional, angle_floor), angle_ceil = np.modf(angle), np.ceil(angle)
 98 |     angle_fractional = angle_fractional.reshape((angle_fractional.shape) + (1,))
 99 |     float_hue = (wheel[angle_floor.astype(np.int)] * (1 - angle_fractional) +
100 |                  wheel[angle_ceil.astype(np.int)] * angle_fractional)
101 | 
102 |     ColorizationArgs = namedtuple("ColorizationArgs", [
103 |         'move_hue_valid_radius',
104 |         'move_hue_oversized_radius',
105 |         'invalid_color'])
106 | 
107 |     def move_hue_on_V_axis(hues, factors):
108 |         return hues * np.expand_dims(factors, -1)
109 | 
110 |     def move_hue_on_S_axis(hues, factors):
111 |         return 255. - np.expand_dims(factors, -1) * (255. - hues)
112 | 
113 |     if background == "dark":
114 |         parameters = ColorizationArgs(move_hue_on_V_axis, move_hue_on_S_axis,
115 |                                       np.array([255, 255, 255], dtype=np.float))
116 |     else:
117 |         parameters = ColorizationArgs(move_hue_on_S_axis, move_hue_on_V_axis,
118 |                                       np.array([0, 0, 0], dtype=np.float))
119 | 
120 |     colors = parameters.move_hue_valid_radius(float_hue, radius)
121 | 
122 |     oversized_radius_mask = radius > 1
123 |     colors[oversized_radius_mask] = parameters.move_hue_oversized_radius(
124 |         float_hue[oversized_radius_mask],
125 |         1 / radius[oversized_radius_mask]
126 |     )
127 |     colors[nan_mask] = parameters.invalid_color
128 | 
129 |     return colors.astype(np.uint8)
130 | 
131 | 
132 | def make_colorwheel(transitions=DEFAULT_TRANSITIONS):
133 |     """
134 |     Creates a color wheel.
135 | 
136 |     A color wheel defines the transitions between the six primary hues:
137 |     Red(255, 0, 0), Yellow(255, 255, 0), Green(0, 255, 0), Cyan(0, 255, 255), Blue(0, 0, 255) and Magenta(255, 0, 255).
138 | 
139 |     Parameters
140 |     ----------
141 |     transitions: sequence_like
142 |         Contains the length of the six transitions.
143 |         Defaults to (15, 6, 4, 11, 13, 6), based on humain perception.
144 | 
145 |     Returns
146 |     -------
147 |     colorwheel: numpy.ndarray
148 |         The RGB values of the transitions in the color space.
149 | 
150 |     Notes
151 |     -----
152 |     For more information, take a look at
153 |     https://web.archive.org/web/20051107102013/http://members.shaw.ca/quadibloc/other/colint.htm
154 | 
155 |     """
156 | 
157 |     colorwheel_length = sum(transitions)
158 | 
159 |     # The red hue is repeated to make the color wheel cyclic
160 |     base_hues = map(np.array,
161 |                     ([255, 0, 0], [255, 255, 0], [0, 255, 0],
162 |                      [0, 255, 255], [0, 0, 255], [255, 0, 255],
163 |                      [255, 0, 0]))
164 | 
165 |     colorwheel = np.zeros((colorwheel_length, 3), dtype="uint8")
166 |     hue_from = next(base_hues)
167 |     start_index = 0
168 |     for hue_to, end_index in zip(base_hues, accumulate(transitions)):
169 |         transition_length = end_index - start_index
170 | 
171 |         colorwheel[start_index:end_index] = np.linspace(
172 |             hue_from, hue_to, transition_length, endpoint=False)
173 |         hue_from = hue_to
174 |         start_index = end_index
175 | 
176 |     return colorwheel
177 | 
178 | 
179 | def calibration_pattern(pixel_size=151, flow_max_radius=1, **flow_to_rgb_args):
180 |     """
181 |     Generates a calibration pattern.
182 | 
183 |     Useful to add a legend to your optical flow plots.
184 | 
185 |     Parameters
186 |     ----------
187 |     pixel_size: int
188 |         Radius of the square test pattern.
189 |     flow_max_radius: float
190 |         The maximum radius value represented by the calibration pattern.
191 |     flow_to_rgb_args: kwargs
192 |         Arguments passed to the flow_to_rgb function
193 | 
194 |     Returns
195 |     -------
196 |     calibration_img: numpy.ndarray
197 |         The RGB image representation of the calibration pattern.
198 |     calibration_flow: numpy.ndarray
199 |         The flow represented in the calibration_pattern. In HWF layout
200 | 
201 |     """
202 |     half_width = pixel_size // 2
203 | 
204 |     y_grid, x_grid = np.mgrid[:pixel_size, :pixel_size]
205 | 
206 |     u = flow_max_radius * (x_grid / half_width - 1)
207 |     v = flow_max_radius * (y_grid / half_width - 1)
208 | 
209 |     flow = np.zeros((pixel_size, pixel_size, 2))
210 |     flow[..., 0] = u
211 |     flow[..., 1] = v
212 | 
213 |     flow_to_rgb_args["flow_max_radius"] = flow_max_radius
214 |     img = flow_to_rgb(flow, **flow_to_rgb_args)
215 | 
216 |     return img, flow
217 | 
218 | 
219 | def attach_arrows(ax, flow, xy_steps=(20, 20),
220 |                   units="xy", color="w", angles="xy", **quiver_kwargs):
221 |     """
222 |     Attach the flow arrows to a matplotlib axes using quiver.
223 | 
224 |     Parameters:
225 |     -----------
226 |     ax: matplotlib.axes
227 |         The axes the arrows should be plotted on.
228 |     flow: numpy.ndarray
229 |         3D flow in the HWF (Height, Width, Flow) layout.
230 |         flow[..., 0] should be the x-displacement
231 |         flow[..., 1] should be the y-displacement
232 |     xy_steps: sequence_like
233 |         The arrows are plotted every xy_steps[0] in the x-dimension and xy_steps[1] in the y-dimension
234 | 
235 |     Quiver Parameters:
236 |     ------------------
237 |     The following parameters are here to override matplotlib.quiver's defaults.
238 |     units: str
239 |         See matplotlib.quiver documentation.
240 |     color: str
241 |         See matplotlib.quiver documentation.
242 |     angles: str
243 |         See matplotlib.quiver documentation.
244 |     quiver_kwargs: kwargs
245 |         Other parameters passed to matplotlib.quiver
246 |         See matplotlib.quiver documentation.
247 | 
248 |     Returns
249 |     -------
250 |     quiver_artist: matplotlib.artist
251 |         See matplotlib.quiver documentation
252 |         Useful for removing the arrows from the figure
253 | 
254 |     """
255 |     height, width, _ = flow.shape
256 | 
257 |     y_grid, x_grid = np.mgrid[:height, :width]
258 | 
259 |     step_x, step_y = xy_steps
260 |     half_step_x, half_step_y = step_x // 2, step_y // 2
261 | 
262 |     return ax.quiver(
263 |         x_grid[half_step_x::step_x, half_step_y::step_y],
264 |         y_grid[half_step_x::step_x, half_step_y::step_y],
265 |         flow[half_step_x::step_x, half_step_y::step_y, 0],
266 |         flow[half_step_x::step_x, half_step_y::step_y, 1],
267 |         angles=angles,
268 |         units=units, color=color, **quiver_kwargs,
269 |     )
270 | 
271 | 
272 | def attach_coord(ax, flow, extent=None):
273 |     """
274 |     Attach the flow value to the coordinate tooltip.
275 | 
276 |     It allows you to see on the same figure, the RGB value of the pixel and the underlying value of the flow.
277 |     Shows cartesian and polar coordinates.
278 | 
279 |     Parameters:
280 |     -----------
281 |     ax: matplotlib.axes
282 |         The axes the arrows should be plotted on.
283 |     flow: numpy.ndarray
284 |         3D flow in the HWF (Height, Width, Flow) layout.
285 |         flow[..., 0] should be the x-displacement
286 |         flow[..., 1] should be the y-displacement
287 |     extent: sequence_like, optional
288 |         Use this parameters in combination with matplotlib.imshow to resize the RGB plot.
289 |         See matplotlib.imshow extent parameter.
290 |         See attach_calibration_pattern
291 | 
292 |     """
293 |     height, width, _ = flow.shape
294 |     base_format = ax.format_coord
295 |     if extent is not None:
296 |         left, right, bottom, top = extent
297 |         x_ratio = width / (right - left)
298 |         y_ratio = height / (top - bottom)
299 | 
300 |     def new_format_coord(x, y):
301 |         if extent is None:
302 |             int_x = int(x + 0.5)
303 |             int_y = int(y + 0.5)
304 |         else:
305 |             int_x = int((x - left) * x_ratio)
306 |             int_y = int((y - bottom) * y_ratio)
307 | 
308 |         if 0 <= int_x < width and 0 <= int_y < height:
309 |             format_string = "Coord: x={}, y={} / Flow: ".format(int_x, int_y)
310 | 
311 |             u, v = flow[int_y, int_x, :]
312 |             if np.isnan(u) or np.isnan(v):
313 |                 format_string += "invalid"
314 |             else:
315 |                 complex_flow = u - 1j * v
316 |                 r, h = np.abs(complex_flow), np.angle(complex_flow, deg=True)
317 |                 format_string += ("u={:.2f}, v={:.2f} (cartesian) ρ={:.2f}, θ={:.2f}° (polar)"
318 |                                   .format(u, v, r, h))
319 |             return format_string
320 |         else:
321 |             return base_format(x, y)
322 | 
323 |     ax.format_coord = new_format_coord
324 | 
325 | 
326 | def attach_calibration_pattern(ax, **calibration_pattern_kwargs):
327 |     """
328 |     Attach a calibration pattern to axes.
329 | 
330 |     This function uses calibration_pattern to generate a figure and shows it as nicely as possible.
331 | 
332 |     Parameters:
333 |     -----------
334 |     calibration_pattern_kwargs: kwargs, optional
335 |         Parameters to be given to the calibration_pattern function.
336 | 
337 |     See Also:
338 |     ---------
339 |     calibration_pattern
340 | 
341 |     Returns
342 |     -------
343 |     image_axes: matplotlib.AxesImage
344 |         See matplotlib.imshow documentation
345 |         Useful for changing the image dynamically
346 |     circle_artist: matplotlib.artist
347 |         See matplotlib.circle documentation
348 |         Useful for removing the circle from the figure
349 | 
350 |     """
351 |     pattern, flow = calibration_pattern(**calibration_pattern_kwargs)
352 |     flow_max_radius = calibration_pattern_kwargs.get("flow_max_radius", 1)
353 | 
354 |     extent = (-flow_max_radius, flow_max_radius) * 2
355 | 
356 |     image = ax.imshow(pattern, extent=extent)
357 |     ax.spines["top"].set_visible(False)
358 |     ax.spines["right"].set_visible(False)
359 | 
360 |     for spine in ("bottom", "left"):
361 |         ax.spines[spine].set_position("zero")
362 |         ax.spines[spine].set_linewidth(1)
363 | 
364 |     ax.xaxis.set_minor_locator(AutoMinorLocator())
365 |     ax.yaxis.set_minor_locator(AutoMinorLocator())
366 | 
367 |     attach_coord(ax, flow, extent=extent)
368 | 
369 |     circle = plt.Circle((0, 0), flow_max_radius, fill=False, lw=1)
370 |     ax.add_artist(circle)
371 | 
372 |     return image, circle
373 | 
374 | 
375 | def replace_nans(array, value=0):
376 |     nan_mask = np.isnan(array)
377 |     array[nan_mask] = value
378 | 
379 |     return array, nan_mask
380 | 
381 | 
382 | def get_flow_max_radius(flow):
383 |     return np.sqrt(np.nanmax(np.sum(flow ** 2, axis=2)))
384 |
385 |
386 |
387 |
388 |
389 |
390 |
391 |

Functions

392 |
393 |
394 | def attach_arrows(ax, flow, xy_steps=(20, 20), units='xy', color='w', angles='xy', **quiver_kwargs) 395 |
396 |
397 |

Attach the flow arrows to a matplotlib axes using quiver.

398 |

Parameters:

399 |

ax: matplotlib.axes 400 | The axes the arrows should be plotted on. 401 | flow: numpy.ndarray 402 | 3D flow in the HWF (Height, Width, Flow) layout. 403 | flow[…, 0] should be the x-displacement 404 | flow[…, 1] should be the y-displacement 405 | xy_steps: sequence_like 406 | The arrows are plotted every xy_steps[0] in the x-dimension and xy_steps[1] in the y-dimension

407 |

Quiver Parameters:

408 |

The following parameters are here to override matplotlib.quiver's defaults. 409 | units: str 410 | See matplotlib.quiver documentation. 411 | color: str 412 | See matplotlib.quiver documentation. 413 | angles: str 414 | See matplotlib.quiver documentation. 415 | quiver_kwargs: kwargs 416 | Other parameters passed to matplotlib.quiver 417 | See matplotlib.quiver documentation.

418 |

Returns

419 |
420 |
quiver_artist : matplotlib.artist
421 |
See matplotlib.quiver documentation 422 | Useful for removing the arrows from the figure
423 |
424 |
425 | 426 | Expand source code 427 | 428 | 
def attach_arrows(ax, flow, xy_steps=(20, 20),
429 |                   units="xy", color="w", angles="xy", **quiver_kwargs):
430 |     """
431 |     Attach the flow arrows to a matplotlib axes using quiver.
432 | 
433 |     Parameters:
434 |     -----------
435 |     ax: matplotlib.axes
436 |         The axes the arrows should be plotted on.
437 |     flow: numpy.ndarray
438 |         3D flow in the HWF (Height, Width, Flow) layout.
439 |         flow[..., 0] should be the x-displacement
440 |         flow[..., 1] should be the y-displacement
441 |     xy_steps: sequence_like
442 |         The arrows are plotted every xy_steps[0] in the x-dimension and xy_steps[1] in the y-dimension
443 | 
444 |     Quiver Parameters:
445 |     ------------------
446 |     The following parameters are here to override matplotlib.quiver's defaults.
447 |     units: str
448 |         See matplotlib.quiver documentation.
449 |     color: str
450 |         See matplotlib.quiver documentation.
451 |     angles: str
452 |         See matplotlib.quiver documentation.
453 |     quiver_kwargs: kwargs
454 |         Other parameters passed to matplotlib.quiver
455 |         See matplotlib.quiver documentation.
456 | 
457 |     Returns
458 |     -------
459 |     quiver_artist: matplotlib.artist
460 |         See matplotlib.quiver documentation
461 |         Useful for removing the arrows from the figure
462 | 
463 |     """
464 |     height, width, _ = flow.shape
465 | 
466 |     y_grid, x_grid = np.mgrid[:height, :width]
467 | 
468 |     step_x, step_y = xy_steps
469 |     half_step_x, half_step_y = step_x // 2, step_y // 2
470 | 
471 |     return ax.quiver(
472 |         x_grid[half_step_x::step_x, half_step_y::step_y],
473 |         y_grid[half_step_x::step_x, half_step_y::step_y],
474 |         flow[half_step_x::step_x, half_step_y::step_y, 0],
475 |         flow[half_step_x::step_x, half_step_y::step_y, 1],
476 |         angles=angles,
477 |         units=units, color=color, **quiver_kwargs,
478 |     )
479 |
480 |
481 |
482 | def attach_calibration_pattern(ax, **calibration_pattern_kwargs) 483 |
484 |
485 |

Attach a calibration pattern to axes.

486 |

This function uses calibration_pattern to generate a figure and shows it as nicely as possible.

487 |

Parameters:

488 |

calibration_pattern_kwargs: kwargs, optional 489 | Parameters to be given to the calibration_pattern function.

490 |

See Also:

491 |

calibration_pattern

492 |

Returns

493 |
494 |
image_axes : matplotlib.AxesImage
495 |
See matplotlib.imshow documentation 496 | Useful for changing the image dynamically
497 |
circle_artist : matplotlib.artist
498 |
See matplotlib.circle documentation 499 | Useful for removing the circle from the figure
500 |
501 |
502 | 503 | Expand source code 504 | 505 | 
def attach_calibration_pattern(ax, **calibration_pattern_kwargs):
506 |     """
507 |     Attach a calibration pattern to axes.
508 | 
509 |     This function uses calibration_pattern to generate a figure and shows it as nicely as possible.
510 | 
511 |     Parameters:
512 |     -----------
513 |     calibration_pattern_kwargs: kwargs, optional
514 |         Parameters to be given to the calibration_pattern function.
515 | 
516 |     See Also:
517 |     ---------
518 |     calibration_pattern
519 | 
520 |     Returns
521 |     -------
522 |     image_axes: matplotlib.AxesImage
523 |         See matplotlib.imshow documentation
524 |         Useful for changing the image dynamically
525 |     circle_artist: matplotlib.artist
526 |         See matplotlib.circle documentation
527 |         Useful for removing the circle from the figure
528 | 
529 |     """
530 |     pattern, flow = calibration_pattern(**calibration_pattern_kwargs)
531 |     flow_max_radius = calibration_pattern_kwargs.get("flow_max_radius", 1)
532 | 
533 |     extent = (-flow_max_radius, flow_max_radius) * 2
534 | 
535 |     image = ax.imshow(pattern, extent=extent)
536 |     ax.spines["top"].set_visible(False)
537 |     ax.spines["right"].set_visible(False)
538 | 
539 |     for spine in ("bottom", "left"):
540 |         ax.spines[spine].set_position("zero")
541 |         ax.spines[spine].set_linewidth(1)
542 | 
543 |     ax.xaxis.set_minor_locator(AutoMinorLocator())
544 |     ax.yaxis.set_minor_locator(AutoMinorLocator())
545 | 
546 |     attach_coord(ax, flow, extent=extent)
547 | 
548 |     circle = plt.Circle((0, 0), flow_max_radius, fill=False, lw=1)
549 |     ax.add_artist(circle)
550 | 
551 |     return image, circle
552 |
553 |
554 |
555 | def attach_coord(ax, flow, extent=None) 556 |
557 |
558 |

Attach the flow value to the coordinate tooltip.

559 |

It allows you to see on the same figure, the RGB value of the pixel and the underlying value of the flow. 560 | Shows cartesian and polar coordinates.

561 |

Parameters:

562 |

ax: matplotlib.axes 563 | The axes the arrows should be plotted on. 564 | flow: numpy.ndarray 565 | 3D flow in the HWF (Height, Width, Flow) layout. 566 | flow[…, 0] should be the x-displacement 567 | flow[…, 1] should be the y-displacement 568 | extent: sequence_like, optional 569 | Use this parameters in combination with matplotlib.imshow to resize the RGB plot. 570 | See matplotlib.imshow extent parameter. 571 | See attach_calibration_pattern

572 |
573 | 574 | Expand source code 575 | 576 | 
def attach_coord(ax, flow, extent=None):
577 |     """
578 |     Attach the flow value to the coordinate tooltip.
579 | 
580 |     It allows you to see on the same figure, the RGB value of the pixel and the underlying value of the flow.
581 |     Shows cartesian and polar coordinates.
582 | 
583 |     Parameters:
584 |     -----------
585 |     ax: matplotlib.axes
586 |         The axes the arrows should be plotted on.
587 |     flow: numpy.ndarray
588 |         3D flow in the HWF (Height, Width, Flow) layout.
589 |         flow[..., 0] should be the x-displacement
590 |         flow[..., 1] should be the y-displacement
591 |     extent: sequence_like, optional
592 |         Use this parameters in combination with matplotlib.imshow to resize the RGB plot.
593 |         See matplotlib.imshow extent parameter.
594 |         See attach_calibration_pattern
595 | 
596 |     """
597 |     height, width, _ = flow.shape
598 |     base_format = ax.format_coord
599 |     if extent is not None:
600 |         left, right, bottom, top = extent
601 |         x_ratio = width / (right - left)
602 |         y_ratio = height / (top - bottom)
603 | 
604 |     def new_format_coord(x, y):
605 |         if extent is None:
606 |             int_x = int(x + 0.5)
607 |             int_y = int(y + 0.5)
608 |         else:
609 |             int_x = int((x - left) * x_ratio)
610 |             int_y = int((y - bottom) * y_ratio)
611 | 
612 |         if 0 <= int_x < width and 0 <= int_y < height:
613 |             format_string = "Coord: x={}, y={} / Flow: ".format(int_x, int_y)
614 | 
615 |             u, v = flow[int_y, int_x, :]
616 |             if np.isnan(u) or np.isnan(v):
617 |                 format_string += "invalid"
618 |             else:
619 |                 complex_flow = u - 1j * v
620 |                 r, h = np.abs(complex_flow), np.angle(complex_flow, deg=True)
621 |                 format_string += ("u={:.2f}, v={:.2f} (cartesian) ρ={:.2f}, θ={:.2f}° (polar)"
622 |                                   .format(u, v, r, h))
623 |             return format_string
624 |         else:
625 |             return base_format(x, y)
626 | 
627 |     ax.format_coord = new_format_coord
628 |
629 |
630 |
631 | def calibration_pattern(pixel_size=151, flow_max_radius=1, **flow_to_rgb_args) 632 |
633 |
634 |

Generates a calibration pattern.

635 |

Useful to add a legend to your optical flow plots.

636 |

Parameters

637 |
638 |
pixel_size : int
639 |
Radius of the square test pattern.
640 |
flow_max_radius : float
641 |
The maximum radius value represented by the calibration pattern.
642 |
flow_to_rgb_args : kwargs
643 |
Arguments passed to the flow_to_rgb function
644 |
645 |

Returns

646 |
647 |
calibration_img : numpy.ndarray
648 |
The RGB image representation of the calibration pattern.
649 |
calibration_flow : numpy.ndarray
650 |
The flow represented in the calibration_pattern. In HWF layout
651 |
652 |
653 | 654 | Expand source code 655 | 656 | 
def calibration_pattern(pixel_size=151, flow_max_radius=1, **flow_to_rgb_args):
657 |     """
658 |     Generates a calibration pattern.
659 | 
660 |     Useful to add a legend to your optical flow plots.
661 | 
662 |     Parameters
663 |     ----------
664 |     pixel_size: int
665 |         Radius of the square test pattern.
666 |     flow_max_radius: float
667 |         The maximum radius value represented by the calibration pattern.
668 |     flow_to_rgb_args: kwargs
669 |         Arguments passed to the flow_to_rgb function
670 | 
671 |     Returns
672 |     -------
673 |     calibration_img: numpy.ndarray
674 |         The RGB image representation of the calibration pattern.
675 |     calibration_flow: numpy.ndarray
676 |         The flow represented in the calibration_pattern. In HWF layout
677 | 
678 |     """
679 |     half_width = pixel_size // 2
680 | 
681 |     y_grid, x_grid = np.mgrid[:pixel_size, :pixel_size]
682 | 
683 |     u = flow_max_radius * (x_grid / half_width - 1)
684 |     v = flow_max_radius * (y_grid / half_width - 1)
685 | 
686 |     flow = np.zeros((pixel_size, pixel_size, 2))
687 |     flow[..., 0] = u
688 |     flow[..., 1] = v
689 | 
690 |     flow_to_rgb_args["flow_max_radius"] = flow_max_radius
691 |     img = flow_to_rgb(flow, **flow_to_rgb_args)
692 | 
693 |     return img, flow
694 |
695 |
696 |
697 | def flow_to_rgb(flow, flow_max_radius=None, background='bright', custom_colorwheel=None) 698 |
699 |
700 |

Creates a RGB representation of an optical flow.

701 |

Parameters

702 |
703 |
flow : numpy.ndarray
704 |
3D flow in the HWF (Height, Width, Flow) layout. 705 | flow[…, 0] should be the x-displacement 706 | flow[…, 1] should be the y-displacement
707 |
flow_max_radius : float, optionnal
708 |
Set the radius that gives the maximum color intensity, useful for comparing different flows. 709 | Default: The normalization is based on the input flow maximum radius.
710 |
background : str, optionnal
711 |
States if zero-valued flow should look 'bright' or 'dark' 712 | Default: "bright"
713 |
custom_colorwheel : numpy.ndarray
714 |
Use a custom colorwheel for specific hue transition lengths. 715 | By default, the default transition lengths are used.
716 |
717 |

Returns

718 |
719 |
rgb_image : numpy.ndarray
720 |
A 2D RGB image that represents the flow
721 |
722 |

See Also

723 |

make_colorwheel()

724 |
725 | 726 | Expand source code 727 | 728 | 
def flow_to_rgb(flow, flow_max_radius=None, background="bright", custom_colorwheel=None):
729 |     """
730 |     Creates a RGB representation of an optical flow.
731 | 
732 |     Parameters
733 |     ----------
734 |     flow: numpy.ndarray
735 |         3D flow in the HWF (Height, Width, Flow) layout.
736 |         flow[..., 0] should be the x-displacement
737 |         flow[..., 1] should be the y-displacement
738 | 
739 |     flow_max_radius: float, optionnal
740 |         Set the radius that gives the maximum color intensity, useful for comparing different flows.
741 |         Default: The normalization is based on the input flow maximum radius.
742 | 
743 |     background: str, optionnal
744 |         States if zero-valued flow should look 'bright' or 'dark'
745 |         Default: "bright"
746 | 
747 |     custom_colorwheel: numpy.ndarray
748 |         Use a custom colorwheel for specific hue transition lengths.
749 |         By default, the default transition lengths are used.
750 | 
751 |     Returns
752 |     -------
753 |     rgb_image: numpy.ndarray
754 |         A 2D RGB image that represents the flow
755 | 
756 |     See Also
757 |     --------
758 |     make_colorwheel
759 | 
760 |     """
761 | 
762 |     valid_backgrounds = ("bright", "dark")
763 |     if background not in valid_backgrounds:
764 |         raise ValueError("background should be one the following: {}, not {}".format(
765 |             valid_backgrounds, background))
766 | 
767 |     wheel = make_colorwheel() if custom_colorwheel is None else custom_colorwheel
768 | 
769 |     flow_height, flow_width, _ = flow.shape
770 | 
771 |     complex_flow = flow[..., 0] + 1j * flow[..., 1]
772 |     complex_flow, nan_mask = replace_nans(complex_flow)
773 | 
774 |     radius, angle = np.abs(complex_flow), np.angle(complex_flow)
775 | 
776 |     if flow_max_radius is None:
777 |         flow_max_radius = np.max(radius)
778 | 
779 |     if flow_max_radius > 0:
780 |         radius /= flow_max_radius
781 | 
782 |     ncols = len(wheel)
783 | 
784 |     # Map the angles from (-pi, pi] to [0, ncols - 1)
785 |     angle = (-angle + np.pi) * ((ncols - 1) / (2 * np.pi))
786 | 
787 |     # Interpolate the hues
788 |     (angle_fractional, angle_floor), angle_ceil = np.modf(angle), np.ceil(angle)
789 |     angle_fractional = angle_fractional.reshape((angle_fractional.shape) + (1,))
790 |     float_hue = (wheel[angle_floor.astype(np.int)] * (1 - angle_fractional) +
791 |                  wheel[angle_ceil.astype(np.int)] * angle_fractional)
792 | 
793 |     ColorizationArgs = namedtuple("ColorizationArgs", [
794 |         'move_hue_valid_radius',
795 |         'move_hue_oversized_radius',
796 |         'invalid_color'])
797 | 
798 |     def move_hue_on_V_axis(hues, factors):
799 |         return hues * np.expand_dims(factors, -1)
800 | 
801 |     def move_hue_on_S_axis(hues, factors):
802 |         return 255. - np.expand_dims(factors, -1) * (255. - hues)
803 | 
804 |     if background == "dark":
805 |         parameters = ColorizationArgs(move_hue_on_V_axis, move_hue_on_S_axis,
806 |                                       np.array([255, 255, 255], dtype=np.float))
807 |     else:
808 |         parameters = ColorizationArgs(move_hue_on_S_axis, move_hue_on_V_axis,
809 |                                       np.array([0, 0, 0], dtype=np.float))
810 | 
811 |     colors = parameters.move_hue_valid_radius(float_hue, radius)
812 | 
813 |     oversized_radius_mask = radius > 1
814 |     colors[oversized_radius_mask] = parameters.move_hue_oversized_radius(
815 |         float_hue[oversized_radius_mask],
816 |         1 / radius[oversized_radius_mask]
817 |     )
818 |     colors[nan_mask] = parameters.invalid_color
819 | 
820 |     return colors.astype(np.uint8)
821 |
822 |
823 |
824 | def get_flow_max_radius(flow) 825 |
826 |
827 |
828 |
829 | 830 | Expand source code 831 | 832 | 
def get_flow_max_radius(flow):
833 |     return np.sqrt(np.nanmax(np.sum(flow ** 2, axis=2)))
834 |
835 |
836 |
837 | def make_colorwheel(transitions=(15, 6, 4, 11, 13, 6)) 838 |
839 |
840 |

Creates a color wheel.

841 |

A color wheel defines the transitions between the six primary hues: 842 | Red(255, 0, 0), Yellow(255, 255, 0), Green(0, 255, 0), Cyan(0, 255, 255), Blue(0, 0, 255) and Magenta(255, 0, 255).

843 |

Parameters

844 |
845 |
transitions : sequence_like
846 |
Contains the length of the six transitions. 847 | Defaults to (15, 6, 4, 11, 13, 6), based on humain perception.
848 |
849 |

Returns

850 |
851 |
colorwheel : numpy.ndarray
852 |
The RGB values of the transitions in the color space.
853 |
854 |

Notes

855 |

For more information, take a look at 856 | https://web.archive.org/web/20051107102013/http://members.shaw.ca/quadibloc/other/colint.htm

857 |
858 | 859 | Expand source code 860 | 861 | 
def make_colorwheel(transitions=DEFAULT_TRANSITIONS):
862 |     """
863 |     Creates a color wheel.
864 | 
865 |     A color wheel defines the transitions between the six primary hues:
866 |     Red(255, 0, 0), Yellow(255, 255, 0), Green(0, 255, 0), Cyan(0, 255, 255), Blue(0, 0, 255) and Magenta(255, 0, 255).
867 | 
868 |     Parameters
869 |     ----------
870 |     transitions: sequence_like
871 |         Contains the length of the six transitions.
872 |         Defaults to (15, 6, 4, 11, 13, 6), based on humain perception.
873 | 
874 |     Returns
875 |     -------
876 |     colorwheel: numpy.ndarray
877 |         The RGB values of the transitions in the color space.
878 | 
879 |     Notes
880 |     -----
881 |     For more information, take a look at
882 |     https://web.archive.org/web/20051107102013/http://members.shaw.ca/quadibloc/other/colint.htm
883 | 
884 |     """
885 | 
886 |     colorwheel_length = sum(transitions)
887 | 
888 |     # The red hue is repeated to make the color wheel cyclic
889 |     base_hues = map(np.array,
890 |                     ([255, 0, 0], [255, 255, 0], [0, 255, 0],
891 |                      [0, 255, 255], [0, 0, 255], [255, 0, 255],
892 |                      [255, 0, 0]))
893 | 
894 |     colorwheel = np.zeros((colorwheel_length, 3), dtype="uint8")
895 |     hue_from = next(base_hues)
896 |     start_index = 0
897 |     for hue_to, end_index in zip(base_hues, accumulate(transitions)):
898 |         transition_length = end_index - start_index
899 | 
900 |         colorwheel[start_index:end_index] = np.linspace(
901 |             hue_from, hue_to, transition_length, endpoint=False)
902 |         hue_from = hue_to
903 |         start_index = end_index
904 | 
905 |     return colorwheel
906 |
907 |
908 |
909 | def replace_nans(array, value=0) 910 |
911 |
912 |
913 |
914 | 915 | Expand source code 916 | 917 | 
def replace_nans(array, value=0):
918 |     nan_mask = np.isnan(array)
919 |     array[nan_mask] = value
920 | 
921 |     return array, nan_mask
922 |
923 |
924 |
925 |
926 |
927 |
928 |
929 | 954 |
955 | 958 | 959 | 960 | 961 | -------------------------------------------------------------------------------- /docs/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | flowpy API documentation 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 |
18 |
19 |
20 |

Package flowpy

21 |
22 |
23 |

flowpy

24 |

flowpy

25 |

Contains several utilities for working with optical flows: 26 | 1. flow_read and flow_write let you manipulate flows in .flo and .png format 27 | 2. flow_to_rgb generates a RGB representation of a flow 28 | 3. attach_arrows, attach_coord, attach_calibration_pattern provide helper functions to generate beautiful graphs with matplotlib.

29 |

The library handles flow in the HWF format, a numpy.ndarray with 3 dimensions of size [H, W, 2] that hold respectively the height, width and 2d displacement in the (x, y) order.

30 |

Undefined flow is attributed a NaN value.

31 |
32 | 33 | Expand source code 34 | 35 | 
""" flowpy
36 | 
37 |     flowpy
38 |     ======
39 | 
40 |     Contains several utilities for working with optical flows:
41 |         1. flow_read and flow_write let you manipulate flows in .flo and .png format
42 |         2. flow_to_rgb generates a RGB representation of a flow
43 |         3. attach_arrows, attach_coord, attach_calibration_pattern provide helper functions to generate beautiful graphs with matplotlib.
44 | 
45 |     The library handles flow in the HWF format, a numpy.ndarray with 3 dimensions of size [H, W, 2] that hold respectively the height, width and 2d displacement in the (x, y) order.
46 | 
47 |     Undefined flow is attributed a NaN value.
48 | """
49 | 
50 | from .flowpy import (flow_to_rgb, make_colorwheel, calibration_pattern,
51 |                      attach_arrows, attach_coord, attach_calibration_pattern,
52 |                      get_flow_max_radius)
53 | from .flow_io import flow_read, flow_write
54 |
55 |
56 |
57 |

Sub-modules

58 |
59 |
flowpy.flow_io
60 |
61 |
62 |
63 |
flowpy.flowpy
64 |
65 |
66 |
67 |
68 |
69 |
70 |
71 |
72 |
73 |
74 |
75 |
76 | 92 |
93 | 96 | 97 | 98 | 99 | -------------------------------------------------------------------------------- /examples/flow_backward_warp.py: -------------------------------------------------------------------------------- 1 | import matplotlib.pyplot as plt 2 | import numpy as np 3 | 4 | from PIL import Image 5 | 6 | import flowpy 7 | 8 | flow = flowpy.flow_read("static/kitti_occ_000010_10.png") 9 | first_image = np.asarray(Image.open("static/kitti_000010_10.png")) 10 | second_image = np.asarray(Image.open("static/kitti_000010_11.png")) 11 | 12 | flow[np.isnan(flow)] = 0 13 | warped_first_image = flowpy.backward_warp(second_image, flow) 14 | 15 | fig, axes = plt.subplots(3, 1) 16 | for ax, image, title in zip(axes, (first_image, second_image, warped_first_image), 17 | ("First Image", "Second Image", "Second image warped to first image")): 18 | ax.imshow(image) 19 | ax.set_title(title) 20 | ax.set_axis_off() 21 | 22 | plt.show() 23 | -------------------------------------------------------------------------------- /examples/flow_forward_warp.py: -------------------------------------------------------------------------------- 1 | import matplotlib.pyplot as plt 2 | import numpy as np 3 | 4 | from PIL import Image 5 | 6 | import flowpy 7 | 8 | flow = flowpy.flow_read("static/kitti_occ_000010_10.png") 9 | first_image = np.asarray(Image.open("static/kitti_000010_10.png")) 10 | second_image = np.asarray(Image.open("static/kitti_000010_11.png")) 11 | 12 | flow[np.isnan(flow)] = 0 13 | warped_second_image = flowpy.forward_warp(first_image, flow) 14 | 15 | fig, ax = plt.subplots() 16 | 17 | ax.imshow(warped_second_image) 18 | ax.set_title( "First image warped to the second") 19 | ax.set_axis_off() 20 | 21 | plt.show() 22 | 23 | -------------------------------------------------------------------------------- /examples/flow_rgb.py: -------------------------------------------------------------------------------- 1 | import flowpy 2 | import matplotlib.pyplot as plt 3 | 4 | flow = flowpy.flow_read("static/kitti_occ_000010_10.png") 5 | 6 | fig, ax = plt.subplots() 7 | ax.imshow(flowpy.flow_to_rgb(flow)) 8 | plt.show() 9 | -------------------------------------------------------------------------------- /examples/flow_with_arrows_tooltip_and_calibration.py: -------------------------------------------------------------------------------- 1 | import flowpy 2 | import matplotlib.pyplot as plt 3 | 4 | flow = flowpy.flow_read("static/Dimetrodon.flo") 5 | height, width, _ = flow.shape 6 | 7 | image_ratio = height / width 8 | max_radius = flowpy.get_flow_max_radius(flow) 9 | 10 | fig, (ax_1, ax_2) = plt.subplots( 11 | 1, 2, gridspec_kw={"width_ratios": [1, image_ratio]} 12 | ) 13 | 14 | ax_1.imshow(flowpy.flow_to_rgb(flow)) 15 | flowpy.attach_arrows(ax_1, flow) 16 | flowpy.attach_coord(ax_1, flow) 17 | 18 | flowpy.attach_calibration_pattern(ax_2, flow_max_radius=max_radius) 19 | 20 | plt.show() 21 | -------------------------------------------------------------------------------- /flowpy/__init__.py: -------------------------------------------------------------------------------- 1 | """ flowpy 2 | 3 | flowpy 4 | ====== 5 | 6 | Contains several utilities for working with optical flows: 7 | 1. flow_read and flow_write let you manipulate flows in .flo and .png format 8 | 2. flow_to_rgb generates a RGB representation of a flow 9 | 3. attach_arrows, attach_coord, attach_calibration_pattern provide helper functions to generate beautiful graphs with matplotlib. 10 | 4. Warp an image according to a flow, in the direct and reverse order. 11 | 12 | The library handles flow in the HWF format, a numpy.ndarray with 3 dimensions of size [H, W, 2] that hold respectively the height, width and 2d displacement in the (x, y) order. 13 | 14 | Undefined flow is attributed a NaN value. 15 | """ 16 | 17 | from .flowpy import (flow_to_rgb, make_colorwheel, calibration_pattern, 18 | attach_arrows, attach_coord, attach_calibration_pattern, 19 | get_flow_max_radius, backward_warp, forward_warp, flowshow) 20 | from .flow_io import flow_read, flow_write 21 | -------------------------------------------------------------------------------- /flowpy/flow_io.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import png 3 | import struct 4 | 5 | from pathlib import Path 6 | from warnings import warn 7 | 8 | 9 | def flow_write(output_file, flow, format=None): 10 | """ 11 | Writes optical flow to file. 12 | 13 | Parameters 14 | ---------- 15 | output_file: {str, pathlib.Path, file} 16 | Path of the file to write or file object. 17 | flow: numpy.ndarray 18 | 3D flow in the HWF (Height, Width, Flow) layout. 19 | flow[..., 0] should be the x-displacement 20 | flow[..., 1] should be the y-displacement 21 | format: str, optional 22 | Specify in what format the flow is written, accepted formats: "png" or "flo" 23 | If None, it is guessed on the file extension 24 | 25 | See Also 26 | -------- 27 | flow_read 28 | 29 | """ 30 | 31 | supported_formats = ("png", "flo") 32 | 33 | output_format = guess_extension(output_file, override=format) 34 | 35 | with FileManager(output_file, "wb") as f: 36 | if output_format == "png": 37 | flow_write_png(f, flow) 38 | else: 39 | flow_write_flo(f, flow) 40 | 41 | 42 | def flow_read(input_file, format=None): 43 | """ 44 | Reads optical flow from file 45 | 46 | Parameters 47 | ---------- 48 | output_file: {str, pathlib.Path, file} 49 | Path of the file to read or file object. 50 | format: str, optional 51 | Specify in what format the flow is raed, accepted formats: "png" or "flo" 52 | If None, it is guess on the file extension 53 | 54 | Returns 55 | ------- 56 | flow: numpy.ndarray 57 | 3D flow in the HWF (Height, Width, Flow) layout. 58 | flow[..., 0] is the x-displacement 59 | flow[..., 1] is the y-displacement 60 | 61 | Notes 62 | ----- 63 | 64 | The flo format is dedicated to optical flow and was first used in Middlebury optical flow database. 65 | The original defition can be found here: http://vision.middlebury.edu/flow/code/flow-code/flowIO.cpp 66 | 67 | The png format uses 16-bit RGB png to store optical flows. 68 | It was developped along with the KITTI Vision Benchmark Suite. 69 | More information can be found here: http://www.cvlibs.net/datasets/kitti/eval_scene_flow.php?benchmark=flow 70 | 71 | The both handle flow with invalid ``invalid'' values, to deal with occlusion for example. 72 | We convert such invalid values to NaN. 73 | 74 | See Also 75 | -------- 76 | flow_write 77 | 78 | """ 79 | 80 | input_format = guess_extension(input_file, override=format) 81 | 82 | with FileManager(input_file, "rb") as f: 83 | if input_format == "png": 84 | output = flow_read_png(f) 85 | else: 86 | output = flow_read_flo(f) 87 | 88 | return output 89 | 90 | 91 | def flow_read_flo(f): 92 | if (f.read(4) != b'PIEH'): 93 | warn("{} does not have a .flo file signature".format(f.name)) 94 | 95 | width, height = struct.unpack("II", f.read(8)) 96 | result = np.fromfile(f, dtype="float32").reshape((height, width, 2)) 97 | 98 | # Set invalid flows to NaN 99 | mask_u = np.greater(np.abs(result[..., 0]), 1e9, where=(~np.isnan(result[..., 0]))) 100 | mask_v = np.greater(np.abs(result[..., 1]), 1e9, where=(~np.isnan(result[..., 1]))) 101 | 102 | result[mask_u | mask_v] = np.NaN 103 | 104 | return result 105 | 106 | 107 | def flow_write_flo(f, flow): 108 | SENTINEL = 1666666800.0 # Only here to look like Middlebury original files 109 | height, width, _ = flow.shape 110 | 111 | image = flow.copy() 112 | image[np.isnan(image)] = SENTINEL 113 | 114 | f.write(b'PIEH') 115 | f.write(struct.pack("II", width, height)) 116 | image.astype(np.float32).tofile(f) 117 | 118 | 119 | def flow_read_png(f): 120 | width, height, stream, *_ = png.Reader(f).read() 121 | 122 | file_content = np.concatenate(list(stream)).reshape((height, width, 3)) 123 | flow, valid = file_content[..., 0:2], file_content[..., 2] 124 | 125 | flow = (flow.astype(np.float) - 2 ** 15) / 64. 126 | 127 | flow[~valid.astype(np.bool)] = np.NaN 128 | 129 | return flow 130 | 131 | 132 | def flow_write_png(f, flow): 133 | SENTINEL = 0. # Only here to look like original KITTI files 134 | height, width, _ = flow.shape 135 | flow_copy = flow.copy() 136 | 137 | valid = ~(np.isnan(flow[..., 0]) | np.isnan(flow[..., 1])) 138 | flow_copy[~valid] = SENTINEL 139 | 140 | flow_copy = (flow_copy * 64. + 2 ** 15).astype(np.uint16) 141 | image = np.dstack((flow_copy, valid)) 142 | 143 | writer = png.Writer(width, height, bitdepth=16, greyscale=False) 144 | writer.write(f, image.reshape((height, 3 * width))) 145 | 146 | 147 | class FileManager: 148 | def __init__(self, abstract_file, mode): 149 | self.abstract_file = abstract_file 150 | self.opened_file = None 151 | self.mode = mode 152 | 153 | def __enter__(self): 154 | if isinstance(self.abstract_file, str): 155 | self.opened_file = open(self.abstract_file, self.mode) 156 | elif isinstance(self.abstract_file, Path): 157 | self.opened_file = self.abstract_file.open(self.mode) 158 | else: 159 | return self.abstract_file 160 | 161 | return self.opened_file 162 | 163 | def __exit__(self, exc_type, exc_value, traceback): 164 | if self.opened_file is not None: 165 | self.opened_file.close() 166 | 167 | 168 | def guess_extension(abstract_file, override=None): 169 | if override is not None: 170 | return override 171 | 172 | if isinstance(abstract_file, str): 173 | return Path(abstract_file).suffix[1:] 174 | elif isinstance(abstract_file, Path): 175 | return abstract_file.suffix[1:] 176 | 177 | return Path(abstract_file.name).suffix[1:] 178 | -------------------------------------------------------------------------------- /flowpy/flowpy.py: -------------------------------------------------------------------------------- 1 | import functools 2 | import matplotlib.pyplot as plt 3 | import numpy as np 4 | 5 | from collections import namedtuple 6 | from itertools import accumulate 7 | from matplotlib.ticker import AutoMinorLocator 8 | from scipy.ndimage import map_coordinates 9 | from scipy.spatial import cKDTree 10 | 11 | DEFAULT_TRANSITIONS = (15, 6, 4, 11, 13, 6) 12 | 13 | 14 | def flow_to_rgb(flow, flow_max_radius=None, background="bright", custom_colorwheel=None): 15 | """ 16 | Creates a RGB representation of an optical flow. 17 | 18 | Parameters 19 | ---------- 20 | flow: numpy.ndarray 21 | 3D flow in the HWF (Height, Width, Flow) layout. 22 | flow[..., 0] should be the x-displacement 23 | flow[..., 1] should be the y-displacement 24 | 25 | flow_max_radius: float, optionnal 26 | Set the radius that gives the maximum color intensity, useful for comparing different flows. 27 | Default: The normalization is based on the input flow maximum radius. 28 | 29 | background: str, optionnal 30 | States if zero-valued flow should look 'bright' or 'dark' 31 | Default: "bright" 32 | 33 | custom_colorwheel: numpy.ndarray 34 | Use a custom colorwheel for specific hue transition lengths. 35 | By default, the default transition lengths are used. 36 | 37 | Returns 38 | ------- 39 | rgb_image: numpy.ndarray 40 | A 2D RGB image that represents the flow 41 | 42 | See Also 43 | -------- 44 | make_colorwheel 45 | 46 | """ 47 | 48 | valid_backgrounds = ("bright", "dark") 49 | if background not in valid_backgrounds: 50 | raise ValueError("background should be one the following: {}, not {}".format( 51 | valid_backgrounds, background)) 52 | 53 | wheel = make_colorwheel() if custom_colorwheel is None else custom_colorwheel 54 | 55 | flow_height, flow_width, _ = flow.shape 56 | 57 | complex_flow = flow[..., 0] + 1j * flow[..., 1] 58 | complex_flow, nan_mask = replace_nans(complex_flow) 59 | 60 | radius, angle = np.abs(complex_flow), np.angle(complex_flow) 61 | 62 | if flow_max_radius is None: 63 | flow_max_radius = np.max(radius) 64 | 65 | if flow_max_radius > 0: 66 | radius /= flow_max_radius 67 | 68 | ncols = len(wheel) 69 | 70 | # Map the angles from (-pi, pi] to [0, 2pi) to [0, ncols - 1) 71 | angle[angle < 0] += 2 * np.pi 72 | angle = angle * ((ncols - 1) / (2 * np.pi)) 73 | 74 | # Make the wheel cyclic for interpolation 75 | wheel = np.vstack((wheel, wheel[0])) 76 | 77 | # Interpolate the hues 78 | (angle_fractional, angle_floor), angle_ceil = np.modf(angle), np.ceil(angle) 79 | angle_fractional = angle_fractional.reshape((angle_fractional.shape) + (1,)) 80 | float_hue = (wheel[angle_floor.astype(np.int)] * (1 - angle_fractional) + 81 | wheel[angle_ceil.astype(np.int)] * angle_fractional) 82 | 83 | ColorizationArgs = namedtuple("ColorizationArgs", [ 84 | 'move_hue_valid_radius', 85 | 'move_hue_oversized_radius', 86 | 'invalid_color']) 87 | 88 | def move_hue_on_V_axis(hues, factors): 89 | return hues * np.expand_dims(factors, -1) 90 | 91 | def move_hue_on_S_axis(hues, factors): 92 | return 255. - np.expand_dims(factors, -1) * (255. - hues) 93 | 94 | if background == "dark": 95 | parameters = ColorizationArgs(move_hue_on_V_axis, move_hue_on_S_axis, 96 | np.array([255, 255, 255], dtype=np.float)) 97 | else: 98 | parameters = ColorizationArgs(move_hue_on_S_axis, move_hue_on_V_axis, 99 | np.array([0, 0, 0], dtype=np.float)) 100 | 101 | colors = parameters.move_hue_valid_radius(float_hue, radius) 102 | 103 | oversized_radius_mask = radius > 1 104 | colors[oversized_radius_mask] = parameters.move_hue_oversized_radius( 105 | float_hue[oversized_radius_mask], 106 | 1 / radius[oversized_radius_mask] 107 | ) 108 | colors[nan_mask] = parameters.invalid_color 109 | 110 | return colors.astype(np.uint8) 111 | 112 | 113 | @functools.lru_cache(maxsize=2) 114 | def make_colorwheel(transitions=DEFAULT_TRANSITIONS): 115 | """ 116 | Creates a color wheel. 117 | 118 | A color wheel defines the transitions between the six primary hues: 119 | Red(255, 0, 0), Yellow(255, 255, 0), Green(0, 255, 0), Cyan(0, 255, 255), Blue(0, 0, 255) and Magenta(255, 0, 255). 120 | 121 | Parameters 122 | ---------- 123 | transitions: sequence_like 124 | Contains the length of the six transitions. 125 | Defaults to (15, 6, 4, 11, 13, 6), based on humain perception. 126 | 127 | Returns 128 | ------- 129 | colorwheel: numpy.ndarray 130 | The RGB values of the transitions in the color space. 131 | 132 | Notes 133 | ----- 134 | For more information, take a look at 135 | https://web.archive.org/web/20051107102013/http://members.shaw.ca/quadibloc/other/colint.htm 136 | 137 | """ 138 | 139 | colorwheel_length = sum(transitions) 140 | 141 | # The red hue is repeated to make the color wheel cyclic 142 | base_hues = map(np.array, 143 | ([255, 0, 0], [255, 255, 0], [0, 255, 0], 144 | [0, 255, 255], [0, 0, 255], [255, 0, 255], 145 | [255, 0, 0])) 146 | 147 | colorwheel = np.zeros((colorwheel_length, 3), dtype="uint8") 148 | hue_from = next(base_hues) 149 | start_index = 0 150 | for hue_to, end_index in zip(base_hues, accumulate(transitions)): 151 | transition_length = end_index - start_index 152 | 153 | colorwheel[start_index:end_index] = np.linspace( 154 | hue_from, hue_to, transition_length, endpoint=False) 155 | hue_from = hue_to 156 | start_index = end_index 157 | 158 | return colorwheel 159 | 160 | 161 | def calibration_pattern(pixel_size=151, flow_max_radius=1, **flow_to_rgb_args): 162 | """ 163 | Generates a calibration pattern. 164 | 165 | Useful to add a legend to your optical flow plots. 166 | 167 | Parameters 168 | ---------- 169 | pixel_size: int 170 | Radius of the square test pattern. 171 | flow_max_radius: float 172 | The maximum radius value represented by the calibration pattern. 173 | flow_to_rgb_args: kwargs 174 | Arguments passed to the flow_to_rgb function 175 | 176 | Returns 177 | ------- 178 | calibration_img: numpy.ndarray 179 | The RGB image representation of the calibration pattern. 180 | calibration_flow: numpy.ndarray 181 | The flow represented in the calibration_pattern. In HWF layout 182 | 183 | """ 184 | half_width = pixel_size // 2 185 | 186 | y_grid, x_grid = np.mgrid[:pixel_size, :pixel_size] 187 | 188 | u = flow_max_radius * (x_grid / half_width - 1) 189 | v = flow_max_radius * (y_grid / half_width - 1) 190 | 191 | flow = np.zeros((pixel_size, pixel_size, 2)) 192 | flow[..., 0] = u 193 | flow[..., 1] = v 194 | 195 | flow_to_rgb_args["flow_max_radius"] = flow_max_radius 196 | img = flow_to_rgb(flow, **flow_to_rgb_args) 197 | 198 | return img, flow 199 | 200 | 201 | def attach_arrows(ax, flow, xy_steps=(20, 20), 202 | units="xy", color="w", angles="xy", **quiver_kwargs): 203 | """ 204 | Attach the flow arrows to a matplotlib axes using quiver. 205 | 206 | Parameters: 207 | ----------- 208 | ax: matplotlib.axes 209 | The axes the arrows should be plotted on. 210 | flow: numpy.ndarray 211 | 3D flow in the HWF (Height, Width, Flow) layout. 212 | flow[..., 0] should be the x-displacement 213 | flow[..., 1] should be the y-displacement 214 | xy_steps: sequence_like 215 | The arrows are plotted every xy_steps[0] in the x-dimension and xy_steps[1] in the y-dimension 216 | 217 | Quiver Parameters: 218 | ------------------ 219 | The following parameters are here to override matplotlib.quiver's defaults. 220 | units: str 221 | See matplotlib.quiver documentation. 222 | color: str 223 | See matplotlib.quiver documentation. 224 | angles: str 225 | See matplotlib.quiver documentation. 226 | quiver_kwargs: kwargs 227 | Other parameters passed to matplotlib.quiver 228 | See matplotlib.quiver documentation. 229 | 230 | Returns 231 | ------- 232 | quiver_artist: matplotlib.artist 233 | See matplotlib.quiver documentation 234 | Useful for removing the arrows from the figure 235 | 236 | """ 237 | height, width, _ = flow.shape 238 | 239 | y_grid, x_grid = np.mgrid[:height, :width] 240 | 241 | step_x, step_y = xy_steps 242 | half_step_x, half_step_y = step_x // 2, step_y // 2 243 | 244 | return ax.quiver( 245 | x_grid[half_step_x::step_x, half_step_y::step_y], 246 | y_grid[half_step_x::step_x, half_step_y::step_y], 247 | flow[half_step_x::step_x, half_step_y::step_y, 0], 248 | flow[half_step_x::step_x, half_step_y::step_y, 1], 249 | angles=angles, 250 | units=units, color=color, **quiver_kwargs, 251 | ) 252 | 253 | 254 | def attach_coord(ax, flow, extent=None): 255 | """ 256 | Attach the flow value to the coordinate tooltip. 257 | 258 | It allows you to see on the same figure, the RGB value of the pixel and the underlying value of the flow. 259 | Shows cartesian and polar coordinates. 260 | 261 | Parameters: 262 | ----------- 263 | ax: matplotlib.axes 264 | The axes the arrows should be plotted on. 265 | flow: numpy.ndarray 266 | 3D flow in the HWF (Height, Width, Flow) layout. 267 | flow[..., 0] should be the x-displacement 268 | flow[..., 1] should be the y-displacement 269 | extent: sequence_like, optional 270 | Use this parameters in combination with matplotlib.imshow to resize the RGB plot. 271 | See matplotlib.imshow extent parameter. 272 | See attach_calibration_pattern 273 | 274 | """ 275 | height, width, _ = flow.shape 276 | base_format = ax.format_coord 277 | if extent is not None: 278 | left, right, bottom, top = extent 279 | x_ratio = width / (right - left) 280 | y_ratio = height / (top - bottom) 281 | 282 | def new_format_coord(x, y): 283 | if extent is None: 284 | int_x = int(x + 0.5) 285 | int_y = int(y + 0.5) 286 | else: 287 | int_x = int((x - left) * x_ratio) 288 | int_y = int((y - bottom) * y_ratio) 289 | 290 | if 0 <= int_x < width and 0 <= int_y < height: 291 | format_string = "Coord: x={}, y={} / Flow: ".format(int_x, int_y) 292 | 293 | u, v = flow[int_y, int_x, :] 294 | if np.isnan(u) or np.isnan(v): 295 | format_string += "invalid" 296 | else: 297 | complex_flow = u - 1j * v 298 | r, h = np.abs(complex_flow), np.angle(complex_flow, deg=True) 299 | format_string += ("u={:.2f}, v={:.2f} (cartesian) ρ={:.2f}, θ={:.2f}° (polar)" 300 | .format(u, v, r, h)) 301 | return format_string 302 | else: 303 | return base_format(x, y) 304 | 305 | ax.format_coord = new_format_coord 306 | 307 | 308 | def attach_calibration_pattern(ax, **calibration_pattern_kwargs): 309 | """ 310 | Attach a calibration pattern to axes. 311 | 312 | This function uses calibration_pattern to generate a figure and shows it as nicely as possible. 313 | 314 | Parameters: 315 | ----------- 316 | calibration_pattern_kwargs: kwargs, optional 317 | Parameters to be given to the calibration_pattern function. 318 | 319 | See Also: 320 | --------- 321 | calibration_pattern 322 | 323 | Returns 324 | ------- 325 | image_axes: matplotlib.AxesImage 326 | See matplotlib.imshow documentation 327 | Useful for changing the image dynamically 328 | circle_artist: matplotlib.artist 329 | See matplotlib.circle documentation 330 | Useful for removing the circle from the figure 331 | 332 | """ 333 | pattern, flow = calibration_pattern(**calibration_pattern_kwargs) 334 | flow_max_radius = calibration_pattern_kwargs.get("flow_max_radius", 1) 335 | 336 | extent = (-flow_max_radius, flow_max_radius) * 2 337 | 338 | image = ax.imshow(pattern, extent=extent) 339 | ax.spines["top"].set_visible(False) 340 | ax.spines["right"].set_visible(False) 341 | 342 | for spine in ("bottom", "left"): 343 | ax.spines[spine].set_position("zero") 344 | ax.spines[spine].set_linewidth(1) 345 | 346 | ax.xaxis.set_minor_locator(AutoMinorLocator()) 347 | ax.yaxis.set_minor_locator(AutoMinorLocator()) 348 | 349 | attach_coord(ax, flow, extent=extent) 350 | 351 | circle = plt.Circle((0, 0), flow_max_radius, fill=False, lw=1) 352 | ax.add_artist(circle) 353 | 354 | return image, circle 355 | 356 | 357 | def backward_warp(second_image, flow, **map_coordinates_kwargs): 358 | """ 359 | Compute the backwarp warp of an image. 360 | 361 | Given second_image and the flow from first_image to second_image, it warps the second_image to something close to the first image if the flow is accurate. 362 | 363 | Parameters: 364 | ----------- 365 | second_image: numpy.ndarray 366 | Image of the form [H, W] or [H, W, C] for greyscale or RGB images 367 | flow: numpy.ndarray 368 | 3D flow in the HWF (Height, Width, Flow) layout, from first_image to second_image. 369 | flow[..., 0] should be the x-displacement 370 | flow[..., 1] should be the y-displacement 371 | map_coordinates_kwargs: kwargs 372 | Keyword arguments passed to scipy.ndimage.map_coordinates 373 | Most important ones are *mode* for out-of-bound handling (defaults to nearest), 374 | and "order" to set the quality of the interpolation. 375 | see scipy.ndimage.map_coordinates 376 | 377 | Returns 378 | ------- 379 | first_image: numpy.ndarray 380 | The warped image with same dimensions as second_image. 381 | """ 382 | height, width, *_ = second_image.shape 383 | coord = np.mgrid[:height, :width] 384 | 385 | gx = (coord[1] + flow[..., 0]) 386 | gy = (coord[0] + flow[..., 1]) 387 | 388 | if "mode" not in map_coordinates_kwargs: 389 | map_coordinates_kwargs["mode"] = "nearest" 390 | 391 | first_image = np.zeros_like(second_image) 392 | if second_image.ndim == 3: 393 | for dim in range(second_image.shape[2]): 394 | map_coordinates(second_image[..., dim], (gy, gx), first_image[..., dim], **map_coordinates_kwargs) 395 | else: 396 | map_coordinates(second_image, (gy, gx), first_image, **map_coordinates_kwargs) 397 | return first_image 398 | 399 | 400 | def forward_warp(first_image, flow, k=4): 401 | """ 402 | Compute the forward warp of an image. 403 | 404 | Given first_image and the flow from first_image to second_image, it warps the first_image to something close to the first image if the flow is accurate. 405 | 406 | It uses a k-nearest neighbors search to perform an interpolation. 407 | 408 | Parameters: 409 | ----------- 410 | first_image: numpy.ndarray 411 | Image of the form [H, W] or [H, W, C] for greyscale or RGB images 412 | flow: numpy.ndarray 413 | 3D flow in the HWF (Height, Width, Flow) layout, from first_image to second_image. 414 | flow[..., 0] should be the x-displacement 415 | flow[..., 1] should be the y-displacement 416 | k: int, optional 417 | How many neighbors should be taken into account to interpolate. 418 | 419 | Returns 420 | ------- 421 | second_image: numpy.ndarray 422 | The warped image with same dimensions as first_image. 423 | """ 424 | first_image_3d = first_image[..., np.newaxis] if first_image.ndim == 2 else first_image 425 | height, width, channels = first_image_3d.shape 426 | 427 | coord = np.mgrid[:height, :width] 428 | grid = coord.transpose(1, 2, 0).reshape((width * height, 2)) 429 | 430 | gx = (coord[1] + flow[..., 0]) 431 | gy = (coord[0] + flow[..., 1]) 432 | 433 | warped_points = np.asarray((gy.flatten(), gx.flatten())).T 434 | kdt = cKDTree(warped_points) 435 | 436 | distance, neighbor = kdt.query(grid, k=k) 437 | 438 | y, x = neighbor // width, neighbor % width 439 | 440 | neigbor_values = first_image_3d[(y, x)] 441 | 442 | if k == 1: 443 | second_image_flat = neigbor_values 444 | else: 445 | weights = np.exp(-distance[..., np.newaxis]) 446 | normalizer = np.sum(weights, axis=1) 447 | 448 | second_image_flat = np.sum(neigbor_values * weights, axis=1) 449 | second_image_flat = (second_image_flat / normalizer).astype(first_image.dtype) 450 | 451 | return second_image_flat.reshape(first_image.shape) 452 | 453 | 454 | def replace_nans(array, value=0): 455 | nan_mask = np.isnan(array) 456 | array[nan_mask] = value 457 | 458 | return array, nan_mask 459 | 460 | 461 | def get_flow_max_radius(flow): 462 | return np.sqrt(np.nanmax(np.sum(flow ** 2, axis=2))) 463 | 464 | 465 | def flowshow(flow, with_calibration_pattern=True, with_arrows=True, with_tooltip=True, max_radius=None): 466 | height, width, _ = flow.shape 467 | image_ratio = height / width 468 | 469 | if max_radius is None: 470 | max_radius = get_flow_max_radius(flow) 471 | 472 | if with_calibration_pattern: 473 | fig, axes = plt.subplots(1, 2, gridspec_kw={"width_ratios": [1, image_ratio]}) 474 | ax_1, ax_2 = axes 475 | else: 476 | fig, axes = plt.subplots() 477 | ax_1 = axes 478 | 479 | ax_1.imshow(flow_to_rgb(flow)) 480 | 481 | if with_arrows: 482 | attach_arrows(ax_1, flow) 483 | 484 | if with_tooltip: 485 | attach_coord(ax_1, flow) 486 | 487 | if with_calibration_pattern: 488 | attach_calibration_pattern(ax_2, flow_max_radius=max_radius) 489 | 490 | return fig, axes 491 | -------------------------------------------------------------------------------- /scripts/flowread: -------------------------------------------------------------------------------- 1 | #! /usr/bin/env python3 2 | 3 | import argparse 4 | import flowpy 5 | import matplotlib 6 | import tifffile 7 | import numpy as np 8 | import sys 9 | from tqdm import tqdm 10 | matplotlib.use('Qt5Agg') 11 | 12 | from matplotlib.backends.backend_qt5agg import ( 13 | FigureCanvasQTAgg as FigureCanvas, 14 | NavigationToolbar2QT as NavigationToolbar 15 | ) 16 | from matplotlib.figure import Figure 17 | from matplotlib.widgets import Slider 18 | from PyQt5 import QtCore, QtWidgets 19 | from pathlib import Path 20 | 21 | try: 22 | import ffmpeg 23 | except ImportError: 24 | ffmpeg = None 25 | 26 | 27 | def main(): 28 | parser = argparse.ArgumentParser() 29 | parser.add_argument("file_paths", nargs="*") 30 | args = parser.parse_args() 31 | 32 | qt_app = QtWidgets.QApplication([""]) 33 | 34 | main_window = DOFQTWindow() 35 | if args.file_paths: 36 | main_window.set_flow_source(args.file_paths) 37 | 38 | main_window.show() 39 | sys.exit(qt_app.exec_()) 40 | 41 | 42 | class DOFQTWindow(QtWidgets.QMainWindow): 43 | flowSourceChanged = QtCore.pyqtSignal(list, name="flowSourceChanged") 44 | exportRequested = QtCore.pyqtSignal(str, name="exportRequested") 45 | 46 | def __init__(self): 47 | QtWidgets.QMainWindow.__init__(self) 48 | 49 | self.setAttribute(QtCore.Qt.WA_DeleteOnClose) 50 | self.setWindowTitle("DOFReader") 51 | 52 | self.main_widget = DOFMainWidget(parent=self) 53 | self.main_widget.setFocus() 54 | self.setCentralWidget(self.main_widget) 55 | 56 | menu_bar = self.menuBar() 57 | file_menu = menu_bar.addMenu("&File") 58 | file_menu.addAction("&Export flow...", self.export_flow_dialog) 59 | 60 | def set_flow_source(self, file_path): 61 | self.flowSourceChanged.emit(file_path) 62 | 63 | def export_flow_dialog(self): 64 | filename, _ = QtWidgets.QFileDialog.getSaveFileName( 65 | self, 66 | "Choose export path", 67 | "", 68 | "" 69 | ) 70 | self.exportRequested.emit(filename) 71 | 72 | 73 | class DOFMainWidget(QtWidgets.QWidget): 74 | flowScaleChanged = QtCore.pyqtSignal(float, name="flowScaleChanged") 75 | frameChanged = QtCore.pyqtSignal(np.ndarray, name="frameChanged") 76 | 77 | def __init__(self, parent=None): 78 | super(QtWidgets.QWidget, self).__init__(parent) 79 | 80 | self.flow_sequence = None 81 | 82 | self.plot_options = PlotOptions(self.flowScaleChanged, parent=self) 83 | self.canvas = MatplotlibCanvas(self.plot_options.getValue(), parent=self) 84 | self.toolbar = NavigationToolbar(self.canvas, self) 85 | self.slider_box = FrameSliderBox(self) 86 | 87 | self.layout = QtWidgets.QVBoxLayout(self) 88 | self.layout.addWidget(self.toolbar) 89 | self.layout.addWidget(self.canvas) 90 | self.layout.addWidget(self.slider_box) 91 | self.layout.addWidget(self.plot_options) 92 | 93 | self.plot_options.valueChanged.connect(self.canvas.handle_plot_options_changed) 94 | self.slider_box.slider.valueChanged.connect(self.handle_cursor_changed) 95 | self.parent().flowSourceChanged.connect(self.handle_flow_source_changed) 96 | self.parent().exportRequested.connect(self.handle_export_requested) 97 | 98 | def handle_flow_source_changed(self, path): 99 | self.flow_sequence = FlowOpener.open(path) 100 | self.slider_box.slider.setValue(1) 101 | self.slider_box.slider.setMaximum(self.flow_sequence.shape[0]) 102 | self.emit_frame_changed(self.flow_sequence[0]) 103 | 104 | def handle_cursor_changed(self, cursor_value): 105 | self.slider_box.slider_label.setText(str(cursor_value)) 106 | self.emit_frame_changed(self.flow_sequence[cursor_value - 1]) 107 | 108 | def handle_export_requested(self, filename): 109 | if ffmpeg is None: 110 | print("Export aborted, ffmpeg-python not found") 111 | return 112 | 113 | print("Exporting frames to " + filename) 114 | 115 | length, height, width, _ = self.flow_sequence.shape 116 | 117 | ffmpeg_process = ( 118 | ffmpeg 119 | .input('pipe:', format='rawvideo', pix_fmt='rgb24', s='{}x{}'.format(width, height)) 120 | .output(filename, pix_fmt='yuv420p') 121 | .overwrite_output() 122 | .run_async(pipe_stdin=True) 123 | ) 124 | 125 | plot_parameters = self.plot_options.getValue() 126 | flowpy_options = MatplotlibCanvas.get_flowpy_options(plot_parameters) 127 | 128 | for frame in tqdm(self.flow_sequence): 129 | if plot_parameters["auto_scale"]: 130 | flowpy_options["flow_max_radius"] = flowpy.get_flow_max_radius(frame) 131 | rendered_flow = flowpy.flow_to_rgb(frame, **flowpy_options) 132 | ffmpeg_process.stdin.write(rendered_flow.astype(np.uint8).tobytes()) 133 | 134 | ffmpeg_process.stdin.close() 135 | ffmpeg_process.wait() 136 | 137 | print("Export finished") 138 | 139 | def emit_frame_changed(self, flow): 140 | auto_scale_radius = flowpy.get_flow_max_radius(flow) 141 | self.plot_options.setAutoScaleRadius(auto_scale_radius) 142 | self.frameChanged.emit(flow) 143 | 144 | 145 | class MatplotlibCanvas(FigureCanvas): 146 | def __init__(self, plot_options, parent=None): 147 | fig = Figure(figsize=(5, 4), dpi=100) 148 | 149 | FigureCanvas.__init__(self, fig) 150 | FigureCanvas.setSizePolicy(self, QtWidgets.QSizePolicy.Expanding, QtWidgets.QSizePolicy.Expanding) 151 | FigureCanvas.updateGeometry(self) 152 | self.fig = fig 153 | self.setParent(parent) 154 | 155 | self.ax_im, self.ax_cal = fig.subplots(1, 2) 156 | self.flow_image = None 157 | self.arrows = None 158 | self.circle = None 159 | 160 | self.parent().frameChanged.connect(self.handle_flow_changed) 161 | 162 | def clean_canvas(self): 163 | if self.flow_image: 164 | self.flow_image.remove() 165 | self.flow_image = None 166 | 167 | if self.arrows: 168 | self.arrows.remove() 169 | self.arrows = None 170 | 171 | if self.circle: 172 | self.circle.remove() 173 | self.circle = None 174 | 175 | @staticmethod 176 | def get_flowpy_options(plot_options): 177 | flowpy_options = {} 178 | 179 | flowpy_options["background"] = plot_options["background"] 180 | if plot_options["auto_scale"] and plot_options["auto_scale_radius"] > 0: 181 | flowpy_options["flow_max_radius"] = plot_options["auto_scale_radius"] 182 | else: 183 | flowpy_options["flow_max_radius"] = plot_options["max_radius"] 184 | 185 | return flowpy_options 186 | 187 | def update_rendered_flow(self): 188 | self.clean_canvas() 189 | 190 | plot_options = self.parent().plot_options.getValue() 191 | height, width, _ = self.flow.shape 192 | 193 | grid_spec = matplotlib.gridspec.GridSpec(1, 2, width_ratios=[1, height/width]) 194 | self.ax_im.set_position(grid_spec[0].get_position(self.fig)) 195 | self.ax_cal.set_position(grid_spec[1].get_position(self.fig)) 196 | 197 | flowpy_options = self.get_flowpy_options(plot_options) 198 | rendered_flow = flowpy.flow_to_rgb(self.flow, **flowpy_options) 199 | self.flow_image = self.ax_im.imshow(rendered_flow) 200 | 201 | if plot_options["show_arrows"]: 202 | self.arrows = flowpy.attach_arrows(self.ax_im, self.flow, scale_units="xy", scale=1.0) 203 | flowpy.attach_coord(self.ax_im, self.flow) 204 | _, self.circle = flowpy.attach_calibration_pattern(self.ax_cal, **flowpy_options) 205 | 206 | self.flow_image.axes.figure.canvas.draw() 207 | 208 | def handle_flow_changed(self, flow): 209 | self.flow = flow 210 | self.update_rendered_flow() 211 | 212 | def handle_plot_options_changed(self, _): 213 | self.update_rendered_flow() 214 | 215 | 216 | class FrameSliderBox(QtWidgets.QGroupBox): 217 | def __init__(self, parent=None): 218 | super(FrameSliderBox, self).__init__(parent) 219 | self.setTitle("Frame index") 220 | 221 | main_layout = QtWidgets.QHBoxLayout() 222 | 223 | self.slider_label = QtWidgets.QLabel(str(1)) 224 | self.slider = QtWidgets.QSlider(QtCore.Qt.Horizontal, parent) 225 | self.slider.setTickPosition(QtWidgets.QSlider.TicksAbove) 226 | self.slider.setMinimum(1) 227 | self.slider.setMaximum(1) 228 | 229 | main_layout.addWidget(self.slider_label) 230 | main_layout.addWidget(self.slider) 231 | self.setLayout(main_layout) 232 | 233 | 234 | class PlotOptions(QtWidgets.QGroupBox): 235 | valueChanged = QtCore.pyqtSignal(dict, name="valueChanged") 236 | default_parameters = { 237 | "background": "bright", 238 | "show_arrows": False, 239 | "auto_scale": True, 240 | "max_radius": 0.0, 241 | "auto_scale_radius": 0.0, 242 | } 243 | 244 | def __init__(self, flowScaleChanged, parent=None): 245 | super().__init__("Plot parameters", parent) 246 | self.parameters = self.default_parameters.copy() 247 | 248 | main_layout = QtWidgets.QHBoxLayout() 249 | 250 | self.background = QtWidgets.QCheckBox("Black background", self) 251 | self.background.setChecked(self.default_parameters["background"] == "dark") 252 | self.background.toggled.connect(self.handle_state_changed) 253 | 254 | self.arrows = QtWidgets.QCheckBox("arrows", self) 255 | self.arrows.setChecked(self.default_parameters["show_arrows"]) 256 | self.arrows.toggled.connect(self.handle_state_changed) 257 | 258 | self.auto_scale = QtWidgets.QCheckBox("auto scale", self) 259 | self.auto_scale.setChecked(not self.default_parameters["max_radius"]) 260 | self.auto_scale.toggled.connect(self.handle_state_changed) 261 | 262 | self.max_radius = QtWidgets.QDoubleSpinBox(self) 263 | self.max_radius.setEnabled(False) 264 | self.max_radius.setValue(0.1) 265 | self.max_radius.setRange(0.1, 1e3) 266 | self.max_radius.setSingleStep(0.1) 267 | self.max_radius.valueChanged.connect(self.handle_state_changed) 268 | flowScaleChanged.connect(self.handle_flow_scale_changed) 269 | 270 | main_layout.addWidget(self.background) 271 | main_layout.addWidget(self.arrows) 272 | main_layout.addWidget(self.auto_scale) 273 | main_layout.addWidget(self.max_radius) 274 | 275 | self.setLayout(main_layout) 276 | self.handle_state_changed() 277 | 278 | def synchronize_parameters(self): 279 | self.parameters["background"] = "dark" if self.background.isChecked() else "bright" 280 | self.parameters["show_arrows"] = self.arrows.isChecked() 281 | self.parameters["auto_scale"] = self.auto_scale.isChecked() 282 | self.parameters["max_radius"] = self.max_radius.value() 283 | 284 | def setAutoScaleRadius(self, value): 285 | self.parameters["auto_scale_radius"] = value 286 | if self.parameters["auto_scale"]: 287 | self.max_radius.valueChanged.disconnect() 288 | self.max_radius.setValue(value) 289 | self.max_radius.valueChanged.connect(self.handle_state_changed) 290 | 291 | def handle_state_changed(self): 292 | self.synchronize_parameters() 293 | self.max_radius.setEnabled(not self.parameters["auto_scale"]) 294 | 295 | self.valueChanged.emit(self.getValue()) 296 | 297 | def handle_flow_scale_changed(self, value): 298 | self.max_radius.setValue(value) 299 | 300 | def getValue(self): 301 | return self.parameters 302 | 303 | 304 | class FlowOpener(): 305 | @staticmethod 306 | def open(input_paths): 307 | if Path(input_paths[0]).suffix.lower() in [".tiff", ".tif"]: 308 | assert len(input_paths) == 2, "Must provide two tiff files" 309 | 310 | data = tifffile.imread(input_paths) 311 | if data.ndim == 3: 312 | data = np.asarray([data]) 313 | data = data.transpose((1, 2, 3, 0)) 314 | else: 315 | data = np.stack([flowpy.flow_read(path) for path in input_paths]) 316 | 317 | return data 318 | 319 | 320 | if __name__ == "__main__": 321 | main() 322 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup 2 | 3 | with open("README.md") as f: 4 | long_description = f.read() 5 | 6 | setup(name='flowpy', 7 | version='0.6.0', 8 | description='Tools for working with optical flow', 9 | long_description=long_description, 10 | long_description_content_type='text/markdown', 11 | url='https://gitlab-research.centralesupelec.fr/2018seznecm/flowpy', 12 | author='Mickaël Seznec', 13 | author_email='mickael.seznec@centralesupelec.fr', 14 | license='MIT', 15 | packages=['flowpy'], 16 | install_requires=[ 17 | 'matplotlib', 18 | 'numpy', 19 | 'pypng', 20 | 'scipy', 21 | ], 22 | test_requires=[ 23 | 'PIL', 24 | ], 25 | scripts=[ 26 | 'scripts/flowread' 27 | ], 28 | zip_safe=False) 29 | -------------------------------------------------------------------------------- /static/Dimetrodon.flo: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mickaelseznec/flowpy/29b9f356d265d7c86dec9bf9ecdfbe308a2d53d1/static/Dimetrodon.flo -------------------------------------------------------------------------------- /static/example_arrows.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mickaelseznec/flowpy/29b9f356d265d7c86dec9bf9ecdfbe308a2d53d1/static/example_arrows.png -------------------------------------------------------------------------------- /static/example_backward_warp.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mickaelseznec/flowpy/29b9f356d265d7c86dec9bf9ecdfbe308a2d53d1/static/example_backward_warp.png -------------------------------------------------------------------------------- /static/example_forward_warp.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mickaelseznec/flowpy/29b9f356d265d7c86dec9bf9ecdfbe308a2d53d1/static/example_forward_warp.png -------------------------------------------------------------------------------- /static/example_rgb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mickaelseznec/flowpy/29b9f356d265d7c86dec9bf9ecdfbe308a2d53d1/static/example_rgb.png -------------------------------------------------------------------------------- /static/kitti_000010_10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mickaelseznec/flowpy/29b9f356d265d7c86dec9bf9ecdfbe308a2d53d1/static/kitti_000010_10.png -------------------------------------------------------------------------------- /static/kitti_000010_11.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mickaelseznec/flowpy/29b9f356d265d7c86dec9bf9ecdfbe308a2d53d1/static/kitti_000010_11.png -------------------------------------------------------------------------------- /static/kitti_noc_000010_10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mickaelseznec/flowpy/29b9f356d265d7c86dec9bf9ecdfbe308a2d53d1/static/kitti_noc_000010_10.png -------------------------------------------------------------------------------- /static/kitti_occ_000010_10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mickaelseznec/flowpy/29b9f356d265d7c86dec9bf9ecdfbe308a2d53d1/static/kitti_occ_000010_10.png -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mickaelseznec/flowpy/29b9f356d265d7c86dec9bf9ecdfbe308a2d53d1/tests/__init__.py -------------------------------------------------------------------------------- /tests/flow_tests.py: -------------------------------------------------------------------------------- 1 | #! /usr/bin/env python3 2 | import filecmp 3 | import matplotlib.pyplot as plt 4 | import numpy as np 5 | import os 6 | import sys 7 | import tempfile 8 | import unittest 9 | 10 | from PIL import Image 11 | 12 | import flowpy 13 | 14 | 15 | class CalibrationPatternTestCase(unittest.TestCase): 16 | def test_calibration_bright(self): 17 | fig, ax = plt.subplots() 18 | _, flow = flowpy.calibration_pattern() 19 | 20 | flowpy.attach_calibration_pattern( 21 | ax, pixel_size=255, flow_max_radius=5, background="bright" 22 | ) 23 | 24 | ax.set_title("Calibration Pattern (Bright)") 25 | 26 | plt.show() 27 | 28 | def test_calibration_with_arrows(self): 29 | pattern, flow = flowpy.calibration_pattern() 30 | 31 | fig, ax = plt.subplots() 32 | ax.imshow(pattern) 33 | flowpy.attach_arrows(ax, flow) 34 | plt.show() 35 | 36 | def test_calibration_dark(self): 37 | width = 255 38 | mid_width = width // 2 39 | pattern, _ = flowpy.calibration_pattern(width, background="dark") 40 | 41 | fig, ax = plt.subplots() 42 | ax.imshow(pattern) 43 | ax.hlines(mid_width, -.5, width-.5) 44 | ax.vlines(mid_width, -.5, width-.5) 45 | circle = plt.Circle((mid_width, mid_width), mid_width, fill=False) 46 | ax.add_artist(circle) 47 | ax.set_title("Calibration Pattern (Dark)") 48 | plt.show() 49 | 50 | 51 | class FlowInputOutput(unittest.TestCase): 52 | def test_read_write_flo(self): 53 | input_filepath = "static/Dimetrodon.flo" 54 | flow = flowpy.flow_read(input_filepath) 55 | 56 | with tempfile.NamedTemporaryFile("wb", suffix=".flo") as f: 57 | flowpy.flow_write(f, flow) 58 | self.assertTrue(filecmp.cmp(f.name, input_filepath)) 59 | 60 | def test_read_write_png_occ(self): 61 | input_filepath = "static/kitti_occ_000010_10.png" 62 | 63 | flow = flowpy.flow_read(input_filepath) 64 | 65 | _, output_filename = tempfile.mkstemp(suffix=".png") 66 | 67 | try: 68 | flowpy.flow_write(output_filename, flow) 69 | new_flow = flowpy.flow_read(output_filename) 70 | np.testing.assert_equal(new_flow, flow) 71 | finally: 72 | os.remove(output_filename) 73 | 74 | def test_read_write_png_noc(self): 75 | input_filepath = "static/kitti_noc_000010_10.png" 76 | 77 | flow = flowpy.flow_read(input_filepath) 78 | 79 | _, output_filename = tempfile.mkstemp(suffix=".png") 80 | 81 | try: 82 | flowpy.flow_write(output_filename, flow) 83 | new_flow = flowpy.flow_read(output_filename) 84 | np.testing.assert_equal(new_flow, flow) 85 | finally: 86 | os.remove(output_filename) 87 | 88 | 89 | class FlowDisplay(unittest.TestCase): 90 | def test_flow_to_rgb(self): 91 | flow = flowpy.flow_read("static/Dimetrodon.flo") 92 | plt.imshow(flowpy.flow_to_rgb(flow)) 93 | plt.show() 94 | 95 | def test_flow_with_arrows(self): 96 | flow = flowpy.flow_read("static/kitti_occ_000010_10.png") 97 | 98 | fig, ax = plt.subplots() 99 | ax.imshow(flowpy.flow_to_rgb(flow)) 100 | flowpy.attach_arrows(ax, flow, xy_steps=(20, 20), scale=1) 101 | 102 | plt.show() 103 | 104 | def test_flow_arrows_and_coord(self): 105 | flow = flowpy.flow_read("static/Dimetrodon.flo") 106 | 107 | fig, ax = plt.subplots() 108 | ax.imshow(flowpy.flow_to_rgb(flow)) 109 | flowpy.attach_arrows(ax, flow) 110 | flowpy.attach_coord(ax, flow) 111 | 112 | plt.show() 113 | 114 | def test_flow_arrows_coord_and_calibration_pattern(self): 115 | flow = flowpy.flow_read("static/Dimetrodon.flo") 116 | height, width, _ = flow.shape 117 | image_ratio = height / width 118 | 119 | max_radius = flowpy.get_flow_max_radius(flow) 120 | 121 | fig, (ax_1, ax_2) = plt.subplots(1, 2, 122 | gridspec_kw={"width_ratios": [1, image_ratio]}) 123 | 124 | ax_1.imshow(flowpy.flow_to_rgb(flow)) 125 | flowpy.attach_arrows(ax_1, flow) 126 | flowpy.attach_coord(ax_1, flow) 127 | 128 | flowpy.attach_calibration_pattern(ax_2, flow_max_radius=max_radius) 129 | 130 | plt.show() 131 | 132 | 133 | class FlowWarp(unittest.TestCase): 134 | def test_backward_warp_greyscale(self): 135 | flow = flowpy.flow_read("static/kitti_occ_000010_10.png") 136 | first_image = np.asarray(Image.open("static/kitti_000010_10.png").convert("L")) 137 | second_image = np.asarray(Image.open("static/kitti_000010_11.png").convert("L")) 138 | 139 | flow[np.isnan(flow)] = 0 140 | warped_first_image = flowpy.backward_warp(second_image, flow) 141 | 142 | fig, (ax_1, ax_2, ax_3) = plt.subplots(3, 1) 143 | ax_1.imshow(first_image) 144 | ax_2.imshow(second_image) 145 | ax_3.imshow(warped_first_image) 146 | 147 | plt.show() 148 | 149 | def test_backward_warp_rgb(self): 150 | flow = flowpy.flow_read("static/kitti_occ_000010_10.png") 151 | first_image = np.asarray(Image.open("static/kitti_000010_10.png")) 152 | second_image = np.asarray(Image.open("static/kitti_000010_11.png")) 153 | 154 | flow[np.isnan(flow)] = 0 155 | warped_first_image = flowpy.backward_warp(second_image, flow) 156 | 157 | fig, (ax_1, ax_2, ax_3) = plt.subplots(3, 1) 158 | ax_1.imshow(first_image) 159 | ax_2.imshow(second_image) 160 | ax_3.imshow(warped_first_image) 161 | 162 | plt.show() 163 | 164 | def test_forward_warp_rgb(self): 165 | flow = flowpy.flow_read("static/kitti_occ_000010_10.png") 166 | first_image = np.asarray(Image.open("static/kitti_000010_10.png")) 167 | second_image = np.asarray(Image.open("static/kitti_000010_11.png")) 168 | 169 | flow[np.isnan(flow)] = 0 170 | warped_second_image = flowpy.forward_warp(first_image, flow, k=1) 171 | 172 | fig, (ax_1, ax_2, ax_3) = plt.subplots(3, 1) 173 | ax_1.imshow(first_image) 174 | ax_2.imshow(flowpy.flow_to_rgb(flow)) 175 | ax_3.imshow(warped_second_image) 176 | 177 | plt.show() 178 | 179 | def test_forward_warp_greyscale(self): 180 | flow = flowpy.flow_read("static/kitti_occ_000010_10.png") 181 | first_image = np.asarray(Image.open("static/kitti_000010_10.png").convert("L")) 182 | second_image = np.asarray(Image.open("static/kitti_000010_11.png").convert("L")) 183 | 184 | flow[np.isnan(flow)] = 0 185 | warped_second_image = flowpy.forward_warp(first_image, flow, k=4) 186 | 187 | fig, (ax_1, ax_2, ax_3) = plt.subplots(3, 1) 188 | ax_1.imshow(first_image) 189 | ax_2.imshow(flowpy.flow_to_rgb(flow)) 190 | ax_3.imshow(warped_second_image) 191 | 192 | plt.show() 193 | 194 | if __name__ == "__main__": 195 | unittest.main() 196 | --------------------------------------------------------------------------------