9 |
10 | ## Requirements
11 | The code has been tested with PyTorch 1.6 and Cuda 10.1.
12 | ```Shell
13 | conda create --name raft
14 | conda activate raft
15 | conda install pytorch=1.6.0 torchvision=0.7.0 cudatoolkit=10.1 matplotlib tensorboard scipy opencv -c pytorch
16 | ```
17 |
18 | ## Demos
19 | Pretrained models can be downloaded by running
20 | ```Shell
21 | ./download_models.sh
22 | ```
23 | or downloaded from [google drive](https://drive.google.com/drive/folders/1sWDsfuZ3Up38EUQt7-JDTT1HcGHuJgvT?usp=sharing)
24 |
25 | You can demo a trained model on a sequence of frames
26 | ```Shell
27 | python demo.py --model=models/raft-things.pth --path=demo-frames
28 | ```
29 |
30 | ## Required Data
31 | To evaluate/train RAFT, you will need to download the required datasets.
32 | * [FlyingChairs](https://lmb.informatik.uni-freiburg.de/resources/datasets/FlyingChairs.en.html#flyingchairs)
33 | * [FlyingThings3D](https://lmb.informatik.uni-freiburg.de/resources/datasets/SceneFlowDatasets.en.html)
34 | * [Sintel](http://sintel.is.tue.mpg.de/)
35 | * [KITTI](http://www.cvlibs.net/datasets/kitti/eval_scene_flow.php?benchmark=flow)
36 | * [HD1K](http://hci-benchmark.iwr.uni-heidelberg.de/) (optional)
37 |
38 |
39 | By default `datasets.py` will search for the datasets in these locations. You can create symbolic links to wherever the datasets were downloaded in the `datasets` folder
40 |
41 | ```Shell
42 | ├── datasets
43 | ├── Sintel
44 | ├── test
45 | ├── training
46 | ├── KITTI
47 | ├── testing
48 | ├── training
49 | ├── devkit
50 | ├── FlyingChairs_release
51 | ├── data
52 | ├── FlyingThings3D
53 | ├── frames_cleanpass
54 | ├── frames_finalpass
55 | ├── optical_flow
56 | ```
57 |
58 | ## Evaluation
59 | You can evaluate a trained model using `evaluate.py`
60 | ```Shell
61 | python evaluate.py --model=models/raft-things.pth --dataset=sintel --mixed_precision
62 | ```
63 |
64 | ## Training
65 | We used the following training schedule in our paper (2 GPUs). Training logs will be written to the `runs` which can be visualized using tensorboard
66 | ```Shell
67 | ./train_standard.sh
68 | ```
69 |
70 | If you have a RTX GPU, training can be accelerated using mixed precision. You can expect similiar results in this setting (1 GPU)
71 | ```Shell
72 | ./train_mixed.sh
73 | ```
74 |
75 | ## (Optional) Efficent Implementation
76 | You can optionally use our alternate (efficent) implementation by compiling the provided cuda extension
77 | ```Shell
78 | cd alt_cuda_corr && python setup.py install && cd ..
79 | ```
80 | and running `demo.py` and `evaluate.py` with the `--alternate_corr` flag Note, this implementation is somewhat slower than all-pairs, but uses significantly less GPU memory during the forward pass.
81 |
--------------------------------------------------------------------------------
/RAFT/alt_cuda_corr/correlation.cpp:
--------------------------------------------------------------------------------
1 | #include 
5 |
6 | For the original Caffe version of this work, please see: https://github.com/twhui/LiteFlowNet
7 | 8 | Other optical flow implementations from me: [pytorch-pwc](https://github.com/sniklaus/pytorch-pwc), [pytorch-unflow](https://github.com/sniklaus/pytorch-unflow), [pytorch-spynet](https://github.com/sniklaus/pytorch-spynet) 9 | 10 | ## setup 11 | The correlation layer is implemented in CUDA using CuPy, which is why CuPy is a required dependency. It can be installed using `pip install cupy` or alternatively using one of the provided [binary packages](https://docs.cupy.dev/en/stable/install.html#installing-cupy) as outlined in the CuPy repository. If you would like to use Docker, you can take a look at [this](https://github.com/sniklaus/pytorch-liteflownet/pull/43) pull request to get started. 12 | 13 | ## usage 14 | To run it on your own pair of images, use the following command. You can choose between three models, please make sure to see their paper / the code for more details. 15 | 16 | ``` 17 | python run.py --model default --one ./images/one.png --two ./images/two.png --out ./out.flo 18 | ``` 19 | 20 | I am afraid that I cannot guarantee that this reimplementation is correct. However, it produced results pretty much identical to the implementation of the original authors in the examples that I tried. There are some numerical deviations that stem from differences in the `DownsampleLayer` of Caffe and the `torch.nn.functional.interpolate` function of PyTorch. Please feel free to contribute to this repository by submitting issues and pull requests. 21 | 22 | ## comparison 23 |
62 |
73 |
74 | Manipulated Interpolation of Anything
63 |
64 |
65 |
67 |
68 |
71 |
72 |
75 |
80 |
81 | );
82 | }
83 |
84 | export default App;
--------------------------------------------------------------------------------
/webapp/webapp/src/assets/images/loader.gif:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/zzh-tech/InterpAny-Clearer/50ece06a1fca91bfeed2bd3f2e724d696833ff63/webapp/webapp/src/assets/images/loader.gif
--------------------------------------------------------------------------------
/webapp/webapp/src/assets/index.html:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
9 |
76 |
77 |
79 | Loading...
78 |
35 | 
42 |
44 | );
45 | }
46 |
47 | export default Mask;
--------------------------------------------------------------------------------
/webapp/webapp/src/components/MaskList.tsx:
--------------------------------------------------------------------------------
1 | import React from "react";
2 | import { MaskListProps } from "./helpers/Interfaces";
3 | import Mask from './Mask';
4 |
5 | const MaskList = ({ masks, handleDelete }: MaskListProps) => {
6 |
7 | return (
8 | <>
9 | {
10 | masks.map(mask => {
11 | const index = masks.indexOf(mask);
12 | return
36 | = 0} onResize={undefined} onResizeCapture={undefined} crossOrigin={undefined}/>
37 |
40 |
41 |
73 |
84 | );
85 | };
86 |
87 | export default Stage;
88 |
--------------------------------------------------------------------------------
/webapp/webapp/src/components/Tool.tsx:
--------------------------------------------------------------------------------
1 | // Copyright (c) Meta Platforms, Inc. and affiliates.
2 | // All rights reserved.
3 |
4 | // This source code is licensed under the license found in the
5 | // LICENSE file in the root directory of this source tree.
6 |
7 | import React, { useContext, useEffect, useState } from "react";
8 | import { AppContext } from "./hooks/createContext";
9 | import { ToolProps } from "./helpers/Interfaces";
10 | import * as _ from "underscore";
11 |
12 | const Tool = ({ handleMouseClick, handleMouseMove }: ToolProps) => {
13 | const {
14 | image: [image],
15 | maskImg: [maskImg, setMaskImg],
16 | } = useContext(AppContext)!;
17 |
18 | // Determine if we should shrink or grow the images to match the
19 | // width or the height of the page and setup a ResizeObserver to
20 | // monitor changes in the size of the page
21 | const [shouldFitToWidth, setShouldFitToWidth] = useState(true);
22 | const bodyEl = document.body;
23 | const fitToPage = () => {
24 | if (!image) return;
25 | const imageAspectRatio = image.width / image.height;
26 | const screenAspectRatio = window.innerWidth / window.innerHeight;
27 | setShouldFitToWidth(imageAspectRatio > screenAspectRatio);
28 | };
29 | const resizeObserver = new ResizeObserver((entries) => {
30 | for (const entry of entries) {
31 | if (entry.target === bodyEl) {
32 | fitToPage();
33 | }
34 | }
35 | });
36 | useEffect(() => {
37 | fitToPage();
38 | resizeObserver.observe(bodyEl);
39 | return () => {
40 | resizeObserver.unobserve(bodyEl);
41 | };
42 | }, [image]);
43 |
44 | const imageClasses = "";
45 | const maskImageClasses = `absolute opacity-40 pointer-events-none`;
46 |
47 | // Render the image and the predicted mask image on top
48 | return (
49 | <>
50 | {image && (
51 |
74 | {image && (
75 |
83 |
76 | {image && (
77 | 
78 | )}
79 |
81 | )}
82 |
78 | )}
79 |