├── README.md ├── SWF_demo.ipynb ├── SideWindowFilter.py ├── aiaceo.jpg └── images ├── compare_0.png ├── compare_1.png ├── compare_10.png ├── compare_2.png ├── compare_3.png ├── compare_4.png ├── compare_5.png ├── execution_times.png ├── mean_3x3_iter1.png ├── mean_3x3_iter10.png ├── mean_3x3_iter20.png ├── mean_3x3_iter50.png └── origin&noise.png /README.md: -------------------------------------------------------------------------------- 1 | # Side-Window-Filtering-Python 2 | * Python implementation of [CVPR 2019 Oral paper Side Window Filtering](https://arxiv.org/pdf/1905.07177.pdf) 3 | 4 | * [中文的簡介 medium](https://medium.com/ai-academy-taiwan/%E5%8F%AF%E4%BB%A5%E4%BF%9D%E7%95%99%E5%BD%B1%E5%83%8F%E9%82%8A%E7%B7%A3%E7%9A%84%E8%B6%85%E5%BC%B7%E5%B9%B3%E6%BB%91%E6%BF%BE%E6%B3%A2-2985d2e433ad) 5 | 6 | # Usage 7 | * Download python file SideWindowFilter.py 8 | 9 | ```python 10 | import cv2 11 | from SideWindowFilter import SideWindowFiltering_3d, SideWindowFiltering 12 | 13 | # for RGB image 14 | img = cv2.imread('aiaceo.jpg') 15 | swf_img = SideWindowFiltering_3d(img, kernel=3, mode='mean') 16 | 17 | # for Gray image 18 | img = cv2.imread('aiaceo.jpg', 0) 19 | swf_img = SideWindowFiltering(img, kernel=3, mode='mean') 20 | ``` 21 | 22 | * SWF_demo.ipynb demonstrate some examples 23 | 24 | ```python 25 | swf_img = SideWindowFiltering_3d( 26 | img, 27 | kernel=3, # 3, 5, 7, 9, 11, ... 28 | mode='mean', # 'mean', 'gaussian', 'median' 29 | use_big=False # False: 8 angles, True: 12 angles 30 | ) 31 | ``` 32 | 33 | # Requirment 34 | 35 | ```python 36 | import cv2 37 | import numpy 38 | import numba 39 | ``` 40 | 41 | * Mostly use basic function to implement, version of packages might not be a big deal. 42 | * Use numba just-in-time to complie python code, otherwise it takes extremely long time when execution. 43 | 44 | # DEMO 45 | 46 | * Original image & Add salt noise image 47 | 48 | ![alt](images/origin&noise.png) 49 | 50 | * 3x3 Mean Filtering by traditional method and SWF method 51 | 52 | ![alt](images/mean_3x3_iter1.png) 53 | ![alt](images/mean_3x3_iter10.png) 54 | ![alt](images/mean_3x3_iter20.png) 55 | ![alt](images/mean_3x3_iter50.png) 56 | 57 | # Compare mean & gaussian & median 58 | 59 | * Median filter takes much more time than mean filter 60 | (There must exists more efficient way to implement it.) 61 | 62 | * Image size is 555 x 792 63 | * Run on Linux with Intel(R) Xeon(R) CPU E5-2667 v4 @ 3.20GHz 64 | 65 | ![alt](images/execution_times.png) 66 | 67 | * Results by apply (5, 5) kernel size 68 | 69 | ![alt](images/compare_0.png) 70 | ![alt](images/compare_1.png) 71 | ![alt](images/compare_2.png) 72 | ![alt](images/compare_3.png) 73 | ![alt](images/compare_4.png) 74 | ![alt](images/compare_5.png) 75 | ![alt](images/compare_10.png) 76 | 77 | 78 | 79 | -------------------------------------------------------------------------------- /SideWindowFilter.py: -------------------------------------------------------------------------------- 1 | import cv2 2 | import numpy as np 3 | import numba as nb 4 | 5 | SWF_base = np.array([ 6 | [[1, 1, 0], 7 | [1, 1, 0], 8 | [0, 0, 0]], 9 | [[0, 1, 1], 10 | [0, 1, 1], 11 | [0, 0, 0]], 12 | [[0, 0, 0], 13 | [0, 1, 1], 14 | [0, 1, 1]], 15 | [[0, 0, 0], 16 | [1, 1, 0], 17 | [1, 1, 0]], 18 | [[1, 1, 1], 19 | [1, 1, 1], 20 | [0, 0, 0]], 21 | [[0, 0, 0], 22 | [1, 1, 1], 23 | [1, 1, 1]], 24 | [[1, 1, 0], 25 | [1, 1, 0], 26 | [1, 1, 0]], 27 | [[0, 1, 1], 28 | [0, 1, 1], 29 | [0, 1, 1]], 30 | ], dtype=np.float32) 31 | 32 | SWF_big = np.array([ 33 | [[1, 1, 0], 34 | [1, 1, 0], 35 | [0, 0, 0]], 36 | [[0, 1, 1], 37 | [0, 1, 1], 38 | [0, 0, 0]], 39 | [[0, 0, 0], 40 | [0, 1, 1], 41 | [0, 1, 1]], 42 | [[0, 0, 0], 43 | [1, 1, 0], 44 | [1, 1, 0]], 45 | [[1, 1, 1], 46 | [1, 1, 1], 47 | [0, 0, 0]], 48 | [[0, 0, 0], 49 | [1, 1, 1], 50 | [1, 1, 1]], 51 | [[1, 1, 0], 52 | [1, 1, 0], 53 | [1, 1, 0]], 54 | [[0, 1, 1], 55 | [0, 1, 1], 56 | [0, 1, 1]], 57 | [[1, 1, 1], 58 | [1, 1, 0], 59 | [1, 0, 0]], 60 | [[1, 1, 1], 61 | [0, 1, 1], 62 | [0, 0, 1]], 63 | [[0, 0, 1], 64 | [0, 1, 1], 65 | [1, 1, 1]], 66 | [[1, 0, 0], 67 | [1, 1, 0], 68 | [1, 1, 1]], 69 | ], dtype=np.float32) 70 | 71 | @nb.jit(nopython=True) 72 | def numba_computation(h, w, ori_flatten, each_flattens): 73 | min_dist = np.full(h * w, np.inf) 74 | min_idx = np.zeros(h * w, dtype=np.int32) 75 | for i in range(len(each_flattens)): 76 | dist = (each_flattens[i] - ori_flatten)**2 77 | for j in range(h * w): 78 | if dist[j] < min_dist[j]: 79 | min_dist[j] = dist[j] 80 | min_idx[j] = i 81 | 82 | dst = np.zeros_like(ori_flatten) 83 | for j in range(h * w): 84 | dst[j] = each_flattens[min_idx[j]][j] 85 | 86 | return dst 87 | 88 | def SideWindowFiltering(img, kernel=3, mode='mean', use_big=False): 89 | if use_big: 90 | filters = np.array([cv2.resize(f, (kernel, kernel)) for f in SWF_big]) 91 | filters[filters < 0.99] = 0. 92 | else: 93 | filters = np.array([cv2.resize(f, (kernel, kernel)) for f in SWF_base]) 94 | filters[filters < 0.99] = 0. 95 | 96 | if mode == 'mean': 97 | filters = [f / np.sum(f) for f in filters] 98 | each_flattens = np.array([cv2.filter2D(img ,-1, filters[i]).reshape(-1) for i in range(len(filters))]) 99 | elif mode == 'gaussian': 100 | k = kernel // 2 101 | x, y = np.mgrid[-k:k+1,-k:k+1] 102 | sigma = 0.3*((kernel-1)*0.5 - 1) + 0.8 103 | gaussian_kernel = np.exp(-((x**2+y**2)/(2*sigma**2))) 104 | filters = [np.multiply(f, gaussian_kernel) for f in filters] 105 | filters = [f / np.sum(f) for f in filters] 106 | each_flattens = np.array([cv2.filter2D(img ,-1, filters[i]).reshape(-1) for i in range(len(filters))]) 107 | elif mode == 'median': 108 | kernel = filters.shape[-1] 109 | each_flattens = np.array([median_filter(img, filters[i], kernel).reshape(-1) for i in range(len(filters))]) 110 | 111 | h, w = img.shape 112 | ori_flatten = img.reshape(-1) 113 | dst = numba_computation(h, w, ori_flatten, each_flattens) 114 | dst = dst.reshape(h, w) 115 | return dst 116 | 117 | def SideWindowFiltering_3d(img, kernel=3, mode='mean', use_big=False): 118 | """ 119 | Args: 120 | 121 | img: A rgb image 122 | 123 | kernel: one of integers 3, 5, 7, ... 124 | use kernel size (3, 3) or (5, 5) or (7, 7) ... 125 | 126 | mode: 'mean' or 'median' 127 | use mean filter or median filter 128 | 129 | use_big: default 'False' use 8 angle filters 130 | 'True' use 12 angle filters 131 | 132 | 133 | """ 134 | img = img.copy() 135 | 136 | dsts = [0, 0, 0] 137 | for i in range(3): 138 | dsts[i] = SideWindowFiltering(img[:,:,i], kernel, mode, use_big) 139 | 140 | return np.dstack((dsts[0], dsts[1], dsts[2])) 141 | 142 | @nb.jit(nopython=True) 143 | def numba_acceleration(img, mask, kernel, h, w, median_idx, dst): 144 | for i in range(0, h - kernel + 1): 145 | for j in range(0, w - kernel + 1): 146 | tmp_arr = [] 147 | for a in range(kernel): 148 | for b in range(kernel): 149 | if mask[a, b] == 1: 150 | tmp_arr.append(img[i+a, j+b]) 151 | tmp_arr.sort() 152 | dst[i, j] = tmp_arr[median_idx] 153 | return dst 154 | 155 | def median_filter(img, mask, kernel=3): 156 | mask = mask.astype(np.int32) 157 | median_idx = int(len(np.where(mask > 0)[0]) / 2) 158 | dst = np.zeros_like(img) 159 | img = np.pad(img, kernel//2, 'edge') 160 | h, w = img.shape 161 | dst = numba_acceleration(img, mask, kernel, h, w, median_idx, dst) 162 | return dst -------------------------------------------------------------------------------- /aiaceo.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIAFammy/Side-Window-Filtering-Python/92781ce8cc23ecaaac0405d0c7b5ea58c0b7d4fb/aiaceo.jpg -------------------------------------------------------------------------------- /images/compare_0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIAFammy/Side-Window-Filtering-Python/92781ce8cc23ecaaac0405d0c7b5ea58c0b7d4fb/images/compare_0.png -------------------------------------------------------------------------------- /images/compare_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIAFammy/Side-Window-Filtering-Python/92781ce8cc23ecaaac0405d0c7b5ea58c0b7d4fb/images/compare_1.png -------------------------------------------------------------------------------- /images/compare_10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIAFammy/Side-Window-Filtering-Python/92781ce8cc23ecaaac0405d0c7b5ea58c0b7d4fb/images/compare_10.png -------------------------------------------------------------------------------- /images/compare_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIAFammy/Side-Window-Filtering-Python/92781ce8cc23ecaaac0405d0c7b5ea58c0b7d4fb/images/compare_2.png -------------------------------------------------------------------------------- /images/compare_3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIAFammy/Side-Window-Filtering-Python/92781ce8cc23ecaaac0405d0c7b5ea58c0b7d4fb/images/compare_3.png -------------------------------------------------------------------------------- /images/compare_4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIAFammy/Side-Window-Filtering-Python/92781ce8cc23ecaaac0405d0c7b5ea58c0b7d4fb/images/compare_4.png -------------------------------------------------------------------------------- /images/compare_5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIAFammy/Side-Window-Filtering-Python/92781ce8cc23ecaaac0405d0c7b5ea58c0b7d4fb/images/compare_5.png -------------------------------------------------------------------------------- /images/execution_times.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIAFammy/Side-Window-Filtering-Python/92781ce8cc23ecaaac0405d0c7b5ea58c0b7d4fb/images/execution_times.png -------------------------------------------------------------------------------- /images/mean_3x3_iter1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIAFammy/Side-Window-Filtering-Python/92781ce8cc23ecaaac0405d0c7b5ea58c0b7d4fb/images/mean_3x3_iter1.png -------------------------------------------------------------------------------- /images/mean_3x3_iter10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIAFammy/Side-Window-Filtering-Python/92781ce8cc23ecaaac0405d0c7b5ea58c0b7d4fb/images/mean_3x3_iter10.png -------------------------------------------------------------------------------- /images/mean_3x3_iter20.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIAFammy/Side-Window-Filtering-Python/92781ce8cc23ecaaac0405d0c7b5ea58c0b7d4fb/images/mean_3x3_iter20.png -------------------------------------------------------------------------------- /images/mean_3x3_iter50.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIAFammy/Side-Window-Filtering-Python/92781ce8cc23ecaaac0405d0c7b5ea58c0b7d4fb/images/mean_3x3_iter50.png -------------------------------------------------------------------------------- /images/origin&noise.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIAFammy/Side-Window-Filtering-Python/92781ce8cc23ecaaac0405d0c7b5ea58c0b7d4fb/images/origin&noise.png --------------------------------------------------------------------------------