├── images ├── banner.gif ├── demoA.png └── demoB.png ├── environment.yml ├── LICENSE ├── utils.py └── README.md /images/banner.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/katieluo88/StayPositive/HEAD/images/banner.gif -------------------------------------------------------------------------------- /images/demoA.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/katieluo88/StayPositive/HEAD/images/demoA.png -------------------------------------------------------------------------------- /images/demoB.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/katieluo88/StayPositive/HEAD/images/demoB.png -------------------------------------------------------------------------------- /environment.yml: -------------------------------------------------------------------------------- 1 | name: staypositive 2 | channels: 3 | - pytorch 4 | - defaults 5 | dependencies: 6 | - python>=3.8.5 7 | - pytorch>=1.7.0 8 | - torchvision>=0.8 9 | - numpy 10 | - matplotlib 11 | - jupyterlab 12 | - tqdm -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2021 Katie Luo 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /utils.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import numpy as np 3 | 4 | 5 | def change_range(img, max_val, min_val=0.): 6 | """ Rescales image value to [0, max_val]. 7 | """ 8 | rescaled = np.array(img) * (max_val - min_val) / 255. + min_val 9 | return np.rint(rescaled).astype(int) 10 | 11 | 12 | def basic_normalization(img): 13 | # img: (B, channel, H, W) 14 | minimum = np.amin(img,axis=(0, 1)) 15 | maximum = np.amax(img,axis=(0, 1)) 16 | out = (img - minimum) / (maximum - minimum + 1e-6) # between [0, 1] 17 | return out 18 | 19 | 20 | def change_range_tensor(img, max_val, min_val): 21 | return img * (max_val - min_val) + min_val 22 | 23 | 24 | def norm_tensor(x): 25 | h, w = x.size(0), x.size(1) 26 | x_min = x.view(-1, 3).min(dim=0, keepdim=True).values.view(1, 1, -1) 27 | x_max = x.view(-1, 3).max(dim=0, keepdim=True).values.view(1, 1, -1) 28 | return (x - x_min) / (x_max - x_min) 29 | 30 | 31 | def norm_numpy(x): 32 | img_max = 1 33 | if x.max() > img_max: 34 | img_max = 255 35 | x /= float(img_max) 36 | x_min = x.min(axis=(0, 1), keepdims=True) 37 | x_max = x.max(axis=(0, 1), keepdims=True) 38 | out = (x - x_min) / (x_max - x_min) 39 | x *= float(img_max) 40 | return x 41 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Stay Positive: Non-Negative Image Synthesis for Augmented Reality 2 | 3 | This repository contains a PyTorch implementation of the paper: 4 | 5 | [Stay Positive: Non-Negative Image Synthesis for Augmented Reality](https://openaccess.thecvf.com/content/CVPR2021/papers/Luo_Stay_Positive_Non-Negative_Image_Synthesis_for_Augmented_Reality_CVPR_2021_paper.pdf) \[[Project Page](https://katieluo88.github.io/StayPositive/)\] \[[Video](https://youtu.be/wYEbZWtQ-T4)\] 6 | 7 | [Katie Luo*](https://www.cs.cornell.edu/~katieluo/), 8 | [Guandao Yang*](http://www.guandaoyang.com), 9 | [Wenqi Xian](https://www.cs.cornell.edu/~wenqixian/), 10 | [Harald Haraldsson](http://haraldharaldsson.com/), 11 | [Bharath Hariharan](http://home.bharathh.info/), 12 | [Serge Belongie](http://blogs.cornell.edu/techfaculty/serge-belongie/) 13 | 14 | (* Equal contribution) 15 | CVPR 2021 (**Oral**) 16 | 17 |

18 | 19 |

20 | 21 | ## Abstract 22 | 23 | In applications such as optical see-through and projector augmented reality, producing images amounts to solving non-negative image generation, where one can only add light to an existing image. Most image generation methods, however, are ill-suited to this problem setting, as they make the assumption that one can assign arbitrary color to each pixel. In fact, naive application of existing methods fails even in simple domains such as MNIST digits, since one cannot create darker pixels by adding light. We know, however, that the human visual system can be fooled by optical illusions involving certain spatial configurations of brightness and contrast. Our key insight is that one can leverage this behavior to produce high quality images with negligible artifacts. For example, we can create the illusion of darker patches by brightening surrounding pixels. We propose a novel optimization procedure to produce images that satisfy both semantic and non-negativity constraints. Our approach can incorporate existing state-of-the-art methods, and exhibits strong performance in a variety of tasks including image-to-image translation and style transfer. 24 | 25 | ## Installation 26 | 27 | This demo has been tested on the following package versions: 28 | * Conda (4.8.5) 29 | * Pytorch (1.7.0) 30 | * Matplotlib (3.3.2) 31 | * Numpy (1.19.2) 32 | * tqdm (4.51.0) 33 | * Jupyter notebook (1.0.0) 34 | 35 | Following is a simple way to install the required dependencies. 36 | 37 | ```bash 38 | conda env create -f environment.yml 39 | conda activate staypositive 40 | ``` 41 | 42 | ## Demo 43 | 44 | This is a small demo for the CVPR submission _Stay Positive: Non-Negative Image Synthesis for Augmented Reality_. 45 | The demo requires a simple installation process (presented in the previous section). 46 | It takes an input image (e.g. `demoA.png`) and a proposed target image (e.g. `demoB.png`), and run the optimization process proposed in Section 3 of the paper to produce an output image that tries to be perceptually similar to the propose target while fulfilling the non-negative constraints. 47 | The demo will be in `Demo.ipynb`: 48 | 49 | ```bash 50 | conda activate staypositive 51 | jupyter notebook 52 | ``` 53 | 54 | ## Cite 55 | Please cite our work if you find it useful: 56 | ```latex 57 | @InProceedings{StayPositive_2021, 58 | author = {Luo, Katie and Yang, Guandao and Xian, Wenqi and Haraldsson, Harald and Hariharan, Bharath and Belongie, Serge}, 59 | title = {Stay Positive: Non-Negative Image Synthesis for Augmented Reality}, 60 | booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, 61 | month = {June}, 62 | year = {2021}, 63 | pages = {10050-10060} 64 | } 65 | ``` 66 | --------------------------------------------------------------------------------