├── HW1-Filters ├── blurred2.exr ├── input1.jpg ├── input2.png ├── input3A.jpg ├── input3B.jpg └── main.py ├── HW2-Panoramas ├── example_output1.png ├── example_output2.png ├── input1.png ├── input2.png ├── input3.png ├── main.py ├── new1output_homography.png └── new2output_cylindrical.png ├── HW3-DetectionTracking ├── 02-1.avi ├── detection_tracking.py ├── haarcascade_frontalface_default.xml ├── output_camshift.txt ├── output_kalman.txt ├── output_of.txt ├── output_particle.txt └── plot_results.py ├── HW4-Segmentation ├── astronaut.png ├── astronaut_marking.png ├── drawing.png ├── main.py └── main_bonus.py ├── HW5-StructredLight ├── binary_codes_ids_codebook.pckl ├── correspondence.jpg ├── images │ ├── pattern000.jpg │ ├── pattern001.jpg │ ├── pattern002.jpg │ ├── pattern003.jpg │ ├── pattern004.jpg │ ├── pattern005.jpg │ ├── pattern006.jpg │ ├── pattern007.jpg │ ├── pattern008.jpg │ ├── pattern009.jpg │ ├── pattern010.jpg │ ├── pattern011.jpg │ ├── pattern012.jpg │ ├── pattern013.jpg │ ├── pattern014.jpg │ ├── pattern015.jpg │ └── pattern016.jpg ├── images_new │ ├── aligned000.jpg │ ├── aligned001.jpg │ ├── aligned002.jpg │ ├── aligned003.jpg │ ├── aligned004.jpg │ ├── aligned005.jpg │ ├── aligned006.jpg │ ├── aligned007.jpg │ ├── aligned008.jpg │ ├── aligned009.jpg │ ├── aligned010.jpg │ ├── aligned011.jpg │ ├── aligned012.jpg │ ├── aligned013.jpg │ ├── aligned014.jpg │ ├── aligned015.jpg │ └── aligned016.jpg ├── output.xyz ├── output_color.xyz ├── reconstruct.py └── stereo_calibration.pckl └── README.md /HW1-Filters/blurred2.exr: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW1-Filters/blurred2.exr -------------------------------------------------------------------------------- /HW1-Filters/input1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW1-Filters/input1.jpg -------------------------------------------------------------------------------- /HW1-Filters/input2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW1-Filters/input2.png -------------------------------------------------------------------------------- /HW1-Filters/input3A.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW1-Filters/input3A.jpg -------------------------------------------------------------------------------- /HW1-Filters/input3B.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW1-Filters/input3B.jpg -------------------------------------------------------------------------------- /HW1-Filters/main.py: -------------------------------------------------------------------------------- 1 | # Instructions: 2 | # For question 1, only modify function: histogram_equalization 3 | # For question 2, only modify functions: low_pass_filter, high_pass_filter, deconvolution 4 | # For question 3, only modify function: laplacian_pyramid_blending 5 | 6 | import os 7 | import sys 8 | import cv2 9 | import numpy as np 10 | import math 11 | 12 | # import copy 13 | # from matplotlib import pyplot as plt 14 | 15 | 16 | def ft(im, newsize=None): 17 | dft = np.fft.fft2(np.float32(im), newsize) 18 | return np.fft.fftshift(dft) 19 | 20 | 21 | def ift(shift): 22 | f_ishift = np.fft.ifftshift(shift) 23 | img_back = np.fft.ifft2(f_ishift) 24 | return np.abs(img_back) 25 | 26 | 27 | def help_message(): 28 | print("Usage: [Question_Number] [Input_Options] [Output_Options]") 29 | print("[Question Number]") 30 | print("1 Histogram equalization") 31 | print("2 Frequency domain filtering") 32 | print("3 Laplacian pyramid blending") 33 | print("[Input_Options]") 34 | print("Path to the input images") 35 | print("[Output_Options]") 36 | print("Output directory") 37 | print("Example usages:") 38 | print(sys.argv[0] + " 1 " + "[path to input image] " + 39 | "[output directory]") # Single input, single output 40 | print(sys.argv[0] + " 2 " + "[path to input image1] " + 41 | "[path to input image2] " + 42 | "[output directory]") # Two inputs, three outputs 43 | print(sys.argv[0] + " 3 " + "[path to input image1] " + 44 | "[path to input image2] " + 45 | "[output directory]") # Two inputs, single output 46 | 47 | 48 | # =================================================== 49 | # ======== Question 1: Histogram equalization ======= 50 | # =================================================== 51 | 52 | 53 | def histogram_equalization(img_in): 54 | 55 | # Write histogram equalization here 56 | img_out = img_in # Histogram equalization result 57 | try: 58 | b, g, r = cv2.split(img_in) 59 | 60 | # using range instead of single number i.e. 256 to indicate a bin width of unity from 0 through 256 61 | # This creates 255 different bins, list(range(257)) is equivalent of saying 256 62 | hist_b = np.histogram(b, list(range(257)))[0] 63 | hist_g = np.histogram(g, list(range(257)))[0] 64 | hist_r = np.histogram(r, list(range(257)))[0] 65 | 66 | cdf_b = np.cumsum(hist_b) 67 | cdf_g = np.cumsum(hist_g) 68 | cdf_r = np.cumsum(hist_r) 69 | 70 | cdf_b_normalized = cdf_b * 255 / (cdf_b.max()) 71 | cdf_g_normalized = cdf_g * 255 / (cdf_g.max()) 72 | cdf_r_normalized = cdf_r * 255 / (cdf_r.max()) 73 | 74 | img_out = cv2.merge((cdf_b_normalized[b], cdf_g_normalized[g], 75 | cdf_r_normalized[r])) 76 | 77 | except: 78 | print('grayscale') 79 | return True, img_out 80 | 81 | 82 | def Question1(): 83 | 84 | # Read in input images 85 | input_image = cv2.imread(sys.argv[2], cv2.IMREAD_COLOR) 86 | 87 | # Histogram equalization 88 | succeed, output_image = histogram_equalization(input_image) 89 | 90 | # Write out the result 91 | output_name = sys.argv[3] + "1.jpg" 92 | cv2.imwrite(output_name, output_image) 93 | 94 | return True 95 | 96 | 97 | # =================================================== 98 | # ===== Question 2: Frequency domain filtering ====== 99 | # =================================================== 100 | 101 | 102 | def high_pass_filter(img_in): 103 | 104 | # Write high pass filter here 105 | img_out = img_in # Low pass filter result 106 | filter_size = 20 107 | try: 108 | b, g, r = cv2.split(img_in) 109 | #convert image to fft 110 | fshift_b = ft(b) 111 | fshift_g = ft(g) 112 | fshift_r = ft(r) 113 | 114 | rows, cols, channels = img_in.shape 115 | crow, ccol = int(rows / 2), int(cols / 2) 116 | 117 | fshift_b[crow - filter_size:crow + filter_size, ccol - filter_size: 118 | ccol + filter_size] = 0 119 | fshift_g[crow - filter_size:crow + filter_size, ccol - filter_size: 120 | ccol + filter_size] = 0 121 | fshift_r[crow - filter_size:crow + filter_size, ccol - filter_size: 122 | ccol + filter_size] = 0 123 | 124 | #bring back the DC components to top left corner 125 | img_back = cv2.merge((np.abs(ift(fshift_b)),np.abs(ift(fshift_g)),\ 126 | np.abs(ift(fshift_r)))) 127 | 128 | img_out = img_back 129 | 130 | except: 131 | 132 | #would still fail, failing to read grayscale, so, nothing reaches to this point 133 | fshift_b = ft(img_in) 134 | 135 | rows, cols = img_in.shape 136 | crow, ccol = int(rows / 2), int(cols / 2) 137 | fshift_b[crow - filter_size:crow + filter_size, ccol - filter_size: 138 | ccol + filter_size] = 0 139 | 140 | img_back = (np.abs(ift(fshift_b))) 141 | img_out = img_back 142 | 143 | return True, img_out 144 | 145 | 146 | def low_pass_filter(img_in): 147 | # Write low pass filter here 148 | img_out = img_in # High pass filter result 149 | filter_size = 20 150 | try: 151 | b, g, r = cv2.split(img_in) 152 | # convert image to fft 153 | fshift_b = ft(b) 154 | fshift_g = ft(g) 155 | fshift_r = ft(r) 156 | 157 | rows, cols, channels = img_in.shape 158 | crow, ccol = int(rows / 2), int(cols / 2) 159 | 160 | fshift_mask = np.zeros(shape=(rows, cols), dtype=int) 161 | fshift_mask[crow - filter_size:crow + filter_size, ccol - filter_size: 162 | ccol + filter_size] = 1 163 | 164 | fshift_b = np.multiply(fshift_mask, fshift_b) 165 | fshift_g = np.multiply(fshift_mask, fshift_g) 166 | fshift_r = np.multiply(fshift_mask, fshift_r) 167 | 168 | img_back = cv2.merge((np.abs(ift(fshift_b)), np.abs(ift(fshift_g)), 169 | np.abs(ift(fshift_r)))) 170 | 171 | img_out = img_back 172 | 173 | except: 174 | # would still fail, failing to read grayscale, so, nothing reaches to this point 175 | fshift_b = ft(img_in) 176 | rows, cols = img_in.shape 177 | crow, ccol = int(rows / 2), int(cols / 2) 178 | 179 | fshift_mask = np.zeros(shape=(rows, cols), dtype=int) 180 | fshift_mask[crow - filter_size:crow + filter_size, ccol - filter_size: 181 | ccol + filter_size] = 1 182 | 183 | fshift_b = np.multiply(fshift_mask, fshift_b) 184 | img_back = (np.abs(ift(fshift_b))) 185 | 186 | img_out = img_back 187 | return True, img_out 188 | 189 | 190 | #to complete deconvolution from the convoluted image 191 | def deconvolution(img_in): 192 | 193 | # Write deconvolution codes here 194 | img_out = img_in 195 | gk = cv2.getGaussianKernel(21, 5) 196 | gk = gk * gk.T 197 | gkf = ft(gk, (img_in.shape[0], 198 | img_in.shape[1])) # so we can multiple easily 199 | 200 | try: 201 | b, g, r = cv2.split(img_in) 202 | 203 | imf_b = ft(b, (img_in.shape[0], 204 | img_in.shape[1])) # make sure sizes match 205 | imf_g = ft(g, (img_in.shape[0], 206 | img_in.shape[1])) # make sure sizes match 207 | imf_r = ft(r, (img_in.shape[0], 208 | img_in.shape[1])) # make sure sizes match 209 | 210 | imconvf_b = np.divide(imf_b, gkf) 211 | imconvf_g = np.divide(imf_g, gkf) 212 | imconvf_r = np.divide(imf_r, gkf) 213 | 214 | # now for example we can reconstruct the corrected image from its FT 215 | corrected_b = ift(imconvf_b) 216 | corrected_g = ift(imconvf_g) 217 | corrected_r = ift(imconvf_r) 218 | 219 | img_out = cv2.merge((np.abs(corrected_b), np.abs(corrected_g), 220 | np.abs(corrected_r))) 221 | 222 | except: 223 | 224 | imf_b = ft(img_in, (img_in.shape[0], 225 | img_in.shape[1])) # make sure sizes match 226 | 227 | imconvf_b = np.divide(imf_b, gkf) 228 | 229 | # now for example we can reconstruct the blurred image from its FT 230 | corrected_b = ift(imconvf_b) 231 | 232 | img_out = np.abs(corrected_b) 233 | 234 | return True, img_out 235 | 236 | 237 | def Question2(): 238 | 239 | # Read in input images 240 | input_image1 = cv2.imread(sys.argv[2], cv2.IMREAD_GRAYSCALE) 241 | input_image2 = cv2.imread(sys.argv[3], 242 | cv2.IMREAD_ANYCOLOR | cv2.IMREAD_ANYDEPTH) 243 | 244 | succeed1, output_image1 = low_pass_filter(input_image1) 245 | succeed2, output_image2 = high_pass_filter(input_image1) 246 | 247 | # Deconvolution 248 | succeed3, output_image3 = deconvolution(input_image2) 249 | 250 | # Write out the result 251 | output_name1 = sys.argv[4] + "2.jpg" 252 | output_name2 = sys.argv[4] + "3.jpg" 253 | output_name3 = sys.argv[4] + "4.png" 254 | 255 | output_image3 = output_image3[:] * 255 / ( 256 | output_image3.max() - output_image3.min()) 257 | 258 | cv2.imwrite(output_name1, output_image1) 259 | cv2.imwrite(output_name2, output_image2) 260 | cv2.imwrite(output_name3, output_image3) 261 | 262 | return True 263 | 264 | 265 | # =================================================== 266 | # ===== Question 3: Laplacian pyramid blending ====== 267 | # =================================================== 268 | 269 | 270 | def laplacian_pyramid_blending(img_in1, img_in2): 271 | 272 | # Write laplacian pyramid blending codes here 273 | img_out = img_in1 # Blending result 274 | 275 | # generate Gaussian pyramid for A & B 276 | GA = img_in1.copy() 277 | GB = img_in2.copy() 278 | gpA = [GA] 279 | gpB = [GB] 280 | 281 | for i in range(6): 282 | GA = cv2.pyrDown(GA) 283 | GB = cv2.pyrDown(GB) 284 | gpA.append(GA) 285 | gpB.append(GB) 286 | 287 | # generate Laplacian Pyramid for A 288 | 289 | lpA = [gpA[len(gpA) - 1]] 290 | lpB = [gpB[len(gpA) - 1]] 291 | 292 | for i in range(len(gpA) - 1, 0, -1): 293 | GE_A = cv2.pyrUp( 294 | gpA[i], dstsize=(gpA[i - 1].shape[0], gpA[i - 1].shape[1])) 295 | GE_B = cv2.pyrUp( 296 | gpB[i], dstsize=(gpB[i - 1].shape[0], gpB[i - 1].shape[1])) 297 | L_A = cv2.subtract(gpA[i - 1], GE_A) 298 | L_B = cv2.subtract(gpB[i - 1], GE_B) 299 | 300 | lpA.append(L_A) 301 | lpB.append(L_B) 302 | 303 | # Now add left and right halves of images in each level 304 | LS = [] 305 | for la, lb in zip(lpA, lpB): 306 | rows, cols, dpt = la.shape 307 | ls = np.hstack((la[:, 0:int(cols / 2)], lb[:, int(cols / 2):])) 308 | LS.append(ls) 309 | 310 | # now reconstruct 311 | ls_ = LS[0] 312 | for i in range(1, 6): 313 | ls_ = cv2.pyrUp(ls_, dstsize=(LS[i].shape[0], LS[i].shape[1])) 314 | ls_ = cv2.add(ls_, LS[i]) 315 | 316 | return True, ls_ 317 | 318 | 319 | def Question3(): 320 | 321 | # Read in input images 322 | input_image1 = cv2.imread(sys.argv[2], cv2.IMREAD_COLOR) 323 | input_image2 = cv2.imread(sys.argv[3], cv2.IMREAD_COLOR) 324 | input_image1 = input_image1[:, :input_image1.shape[0]] 325 | input_image2 = input_image2[:input_image1.shape[0], :input_image1.shape[0]] 326 | succeed, output_image = laplacian_pyramid_blending(input_image1, 327 | input_image2) 328 | 329 | # Write out the result 330 | output_name = sys.argv[4] + "5.jpg" 331 | cv2.imwrite(output_name, output_image) 332 | 333 | return True 334 | 335 | 336 | if __name__ == '__main__': 337 | question_number = -1 338 | 339 | # Validate the input arguments 340 | if (len(sys.argv) < 4): 341 | help_message() 342 | sys.exit() 343 | else: 344 | question_number = int(sys.argv[1]) 345 | 346 | if (question_number == 1 and not (len(sys.argv) == 4)): 347 | help_message() 348 | sys.exit() 349 | if (question_number == 2 and not (len(sys.argv) == 5)): 350 | help_message() 351 | sys.exit() 352 | if (question_number == 3 and not (len(sys.argv) == 5)): 353 | help_message() 354 | sys.exit() 355 | if (question_number > 3 or question_number < 1 or len(sys.argv) > 5): 356 | print("Input parameters out of bound ...") 357 | sys.exit() 358 | 359 | function_launch = { 360 | 1: Question1, 361 | 2: Question2, 362 | 3: Question3, 363 | } 364 | 365 | # Call the function 366 | function_launch[question_number]() 367 | -------------------------------------------------------------------------------- /HW2-Panoramas/example_output1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW2-Panoramas/example_output1.png -------------------------------------------------------------------------------- /HW2-Panoramas/example_output2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW2-Panoramas/example_output2.png -------------------------------------------------------------------------------- /HW2-Panoramas/input1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW2-Panoramas/input1.png -------------------------------------------------------------------------------- /HW2-Panoramas/input2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW2-Panoramas/input2.png -------------------------------------------------------------------------------- /HW2-Panoramas/input3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW2-Panoramas/input3.png -------------------------------------------------------------------------------- /HW2-Panoramas/main.py: -------------------------------------------------------------------------------- 1 | # Instructions: 2 | # Do not change the output file names, use the helper functions as you see fit 3 | 4 | import os 5 | import sys 6 | import cv2 7 | import numpy as np 8 | import matplotlib.pyplot as plt 9 | import math 10 | 11 | 12 | def help_message(): 13 | print("Usage: [Question_Number] [Input_Options] [Output_Options]") 14 | print("[Question Number]") 15 | print("1 Perspective warping") 16 | print("2 Cylindrical warping") 17 | print("3 Bonus perspective warping") 18 | print("4 Bonus cylindrical warping") 19 | print("[Input_Options]") 20 | print("Path to the input images") 21 | print("[Output_Options]") 22 | print("Output directory") 23 | print("Example usages:") 24 | print(sys.argv[0] + " 1 " + "[path to input image1] " + 25 | "[path to input image2] " + "[path to input image3] " + 26 | "[output directory]") 27 | 28 | 29 | ''' 30 | Detect, extract and match features between img1 and img2. 31 | Using SIFT as the detector/extractor, but this is inconsequential to the user. 32 | 33 | Returns: (pts1, pts2), where ptsN are points on image N. 34 | The lists are "aligned", i.e. point i in pts1 matches with point i in pts2. 35 | 36 | Usage example: 37 | im1 = cv2.imread("image1.jpg", 0) 38 | im2 = cv2.imread("image2.jpg", 0) 39 | (pts1, pts2) = feature_matching(im1, im2) 40 | 41 | plt.subplot(121) 42 | plt.imshow(im1) 43 | plt.scatter(pts1[:,:,0],pts1[:,:,1], 0.5, c='r', marker='x') 44 | plt.subplot(122) 45 | plt.imshow(im2) 46 | plt.scatter(pts1[:,:,0],pts1[:,:,1], 0.5, c='r', marker='x') 47 | ''' 48 | 49 | 50 | def feature_matching(img1, img2, savefig=False): 51 | # Initiate SIFT detector 52 | sift = cv2.xfeatures2d.SIFT_create() 53 | # find the keypoints and descriptors with SIFT 54 | kp1, des1 = sift.detectAndCompute(img1, None) 55 | kp2, des2 = sift.detectAndCompute(img2, None) 56 | # FLANN parameters 57 | FLANN_INDEX_KDTREE = 1 58 | index_params = dict(algorithm=FLANN_INDEX_KDTREE, trees=5) 59 | search_params = dict(checks=50) # or pass empty dictionary 60 | flann = cv2.FlannBasedMatcher(index_params, search_params) 61 | matches2to1 = flann.knnMatch(des2, des1, k=2) 62 | 63 | matchesMask_ratio = [[0, 0] for i in range(len(matches2to1))] 64 | match_dict = {} 65 | for i, (m, n) in enumerate(matches2to1): 66 | if m.distance < 0.7 * n.distance: 67 | matchesMask_ratio[i] = [1, 0] 68 | match_dict[m.trainIdx] = m.queryIdx 69 | 70 | good = [] 71 | recip_matches = flann.knnMatch(des1, des2, k=2) 72 | matchesMask_ratio_recip = [[0, 0] for i in range(len(recip_matches))] 73 | 74 | for i, (m, n) in enumerate(recip_matches): 75 | if m.distance < 0.7 * n.distance: # ratio 76 | if m.queryIdx in match_dict and match_dict[m. 77 | queryIdx] == m.trainIdx: #reciprocal 78 | good.append(m) 79 | matchesMask_ratio_recip[i] = [1, 0] 80 | 81 | if savefig: 82 | draw_params = dict( 83 | matchColor=(0, 255, 0), 84 | singlePointColor=(255, 0, 0), 85 | matchesMask=matchesMask_ratio_recip, 86 | flags=0) 87 | img3 = cv2.drawMatchesKnn(img1, kp1, img2, kp2, recip_matches, None, 88 | **draw_params) 89 | 90 | plt.figure(), plt.xticks([]), plt.yticks([]) 91 | plt.imshow(img3,) 92 | plt.savefig("feature_matching.png", bbox_inches='tight') 93 | 94 | return ([kp1[m.queryIdx].pt 95 | for m in good], [kp2[m.trainIdx].pt for m in good]) 96 | 97 | 98 | ''' 99 | Warp an image from cartesian coordinates (x, y) into cylindrical coordinates (theta, h) 100 | Returns: (image, mask) 101 | Mask is [0,255], and has 255s wherever the cylindrical images has a valid value. 102 | Masks are useful for stitching 103 | 104 | Usage example: 105 | 106 | im = cv2.imread("myimage.jpg",0) #grayscale 107 | h,w = im.shape 108 | f = 700 109 | K = np.array([[f, 0, w/2], [0, f, h/2], [0, 0, 1]]) # mock calibration matrix 110 | imcyl = cylindricalWarpImage(im, K) 111 | ''' 112 | 113 | 114 | def cylindricalWarpImage(img1, K, savefig=False): 115 | f = K[0, 0] 116 | 117 | im_h, im_w = img1.shape 118 | 119 | # go inverse from cylindrical coord to the image 120 | # (this way there are no gaps) 121 | cyl = np.zeros_like(img1) 122 | cyl_mask = np.zeros_like(img1) 123 | cyl_h, cyl_w = cyl.shape 124 | x_c = float(cyl_w) / 2.0 125 | y_c = float(cyl_h) / 2.0 126 | for x_cyl in np.arange(0, cyl_w): 127 | for y_cyl in np.arange(0, cyl_h): 128 | theta = (x_cyl - x_c) / f 129 | h = (y_cyl - y_c) / f 130 | 131 | X = np.array([math.sin(theta), h, math.cos(theta)]) 132 | X = np.dot(K, X) 133 | x_im = X[0] / X[2] 134 | if x_im < 0 or x_im >= im_w: 135 | continue 136 | 137 | y_im = X[1] / X[2] 138 | if y_im < 0 or y_im >= im_h: 139 | continue 140 | 141 | cyl[int(y_cyl), int(x_cyl)] = img1[int(y_im), int(x_im)] 142 | cyl_mask[int(y_cyl), int(x_cyl)] = 255 143 | 144 | if savefig: 145 | plt.imshow(cyl, cmap='gray') 146 | plt.savefig("cyl.png", bbox_inches='tight') 147 | 148 | return (cyl, cyl_mask) 149 | 150 | 151 | ''' 152 | Calculate the geometric transform (only affine or homography) between two images, 153 | based on feature matching and alignment with a robust estimator (RANSAC). 154 | 155 | Returns: (M, pts1, pts2, mask) 156 | Where: M is the 3x3 transform matrix 157 | pts1 are the matched feature points in image 1 158 | pts2 are the matched feature points in image 2 159 | mask is a binary mask over the lists of points that selects the transformation inliers 160 | 161 | Usage example: 162 | im1 = cv2.imread("image1.jpg", 0) 163 | im2 = cv2.imread("image2.jpg", 0) 164 | (M, pts1, pts2, mask) = getTransform(im1, im2) 165 | 166 | # for example: transform im1 to im2's plane 167 | # first, make some room around im2 168 | im2 = cv2.copyMakeBorder(im2,200,200,500,500, cv2.BORDER_CONSTANT) 169 | # then transform im1 with the 3x3 transformation matrix 170 | out = cv2.warpPerspective(im1, M, (im1.shape[1],im2.shape[0]), dst=im2.copy(), borderMode=cv2.BORDER_TRANSPARENT) 171 | 172 | plt.imshow(out, cmap='gray') 173 | plt.show() 174 | ''' 175 | 176 | 177 | def getTransform(src, dst, method='affine'): 178 | pts1, pts2 = feature_matching(src, dst) 179 | 180 | src_pts = np.float32(pts1).reshape(-1, 1, 2) 181 | dst_pts = np.float32(pts2).reshape(-1, 1, 2) 182 | 183 | if method == 'affine': 184 | M, mask = cv2.estimateAffine2D( 185 | src_pts, dst_pts, cv2.RANSAC, ransacReprojThreshold=5.0) 186 | M = np.append(M, [[0, 0, 1]], axis=0) 187 | 188 | if method == 'homography': 189 | M, mask = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC, 5.0) 190 | 191 | matchesMask = mask.ravel().tolist() 192 | 193 | return (M, pts1, pts2, mask) 194 | 195 | 196 | # =================================================== 197 | # ================ Perspective Warping ============== 198 | # =================================================== 199 | def Perspective_warping(img1, img2, img3): 200 | 201 | # Write your codes here 202 | output_image = img1 # This is dummy output, change it to your output 203 | 204 | img1 = cv2.copyMakeBorder(img1, 200, 200, 500, 500, cv2.BORDER_CONSTANT) 205 | (M, pts1, pts2, mask) = getTransform(img2, img1, 'homography') 206 | out = cv2.warpPerspective( 207 | img2, 208 | M, (img1.shape[1], img1.shape[0]), 209 | dst=img1.copy(), 210 | borderMode=cv2.BORDER_TRANSPARENT) 211 | 212 | (M, pts1, pts2, mask) = getTransform(img3, out, 'homography') 213 | out2 = cv2.warpPerspective( 214 | img3, 215 | M, (out.shape[1], out.shape[0]), 216 | dst=out.copy(), 217 | borderMode=cv2.BORDER_TRANSPARENT) 218 | 219 | # plt.figure() 220 | # plt.imshow(out2, cmap='gray') 221 | # plt.show() 222 | 223 | # Write out the result 224 | output_name = sys.argv[5] + "output_homography.png" 225 | output_image = out2 226 | cv2.imwrite(output_name, output_image) 227 | 228 | #check error 229 | master = cv2.imread("example_output1.png", 0) 230 | print(RMSD(1,out2, master)) 231 | 232 | return True 233 | 234 | 235 | def Bonus_perspective_warping(img1, img2, img3): 236 | 237 | # Write your codes here 238 | output_image = img1 # This is dummy output, change it to your output 239 | 240 | # Write out the result 241 | output_name = sys.argv[5] + "output_homography_lpb.png" 242 | cv2.imwrite(output_name, output_image) 243 | 244 | return True 245 | 246 | 247 | # =================================================== 248 | # =============== Cynlindrical Warping ============== 249 | # =================================================== 250 | def Cylindrical_warping(img1, img2, img3): 251 | 252 | #img1=cv2.imread("input1.png",0) 253 | #img2=cv2.imread("input2.png",0) 254 | #img3=cv2.imread("input3.png",0) 255 | 256 | # cv2.imshow("image1",img1) 257 | # cv2.imshow("image2",img2) 258 | # cv2.imshow("image3",img3) 259 | # cv2.waitKey(0) 260 | 261 | 262 | h, w = img1.shape 263 | f = 450 264 | K = np.array([[f, 0, w / 2], [0, f, h / 2], [0, 0, 1]]) 265 | (cyl1, cyl1_mask) = cylindricalWarpImage(img1, K) 266 | f = 700 267 | K = np.array([[f, 0, w / 2], [0, f, h / 2], [0, 0, 1]]) 268 | (cyl2, cyl2_mask) = cylindricalWarpImage(img2, K) 269 | (cyl3, cyl3_mask) = cylindricalWarpImage(img3, K) 270 | 271 | # cv2.imshow("image1",cyl1) 272 | # cv2.imshow("image2",cyl2) 273 | # cv2.imshow("image3",cyl3) 274 | # cv2.waitKey(0) 275 | 276 | # print(cyl_mask1) 277 | 278 | #Create border for image and mask 279 | cyl1 = cv2.copyMakeBorder(cyl1, 50, 50, 300, 300, cv2.BORDER_CONSTANT) 280 | cyl1_mask=cv2.copyMakeBorder(cyl1_mask, 50, 50, 300, 300, cv2.BORDER_CONSTANT,value=0) 281 | 282 | # plt.figure() 283 | # plt.imshow(cv2.add(cyl1*0.9,cyl1_mask*0.1), cmap='gray') 284 | # plt.show() 285 | 286 | #transform right image to centre image using SIFT, RANSAC and lowe's ratio test and then 287 | # using the points to determine the transformation matrix 288 | #warp affine using the transformation matrix 289 | (M, pts1, pts2, mask) = getTransform(cyl2, cyl1) 290 | out = cv2.warpAffine(cyl2,M[:2,:],(cyl1.shape[1],cyl1.shape[0])) 291 | 292 | #copy the image only if its empty else dont copy to the original 293 | rows, cols = np.where(cyl1_mask == 0) 294 | cyl1[rows, cols] = out[rows, cols] 295 | 296 | 297 | # plt.figure() 298 | # plt.imshow(cyl1, cmap='gray') 299 | # plt.show() 300 | 301 | #transform the left and warp affine 302 | (M, pts1, pts2, mask) = getTransform(cyl3, cyl1) 303 | out2 = cv2.warpAffine(cyl3, M[:2,:], (cyl1.shape[1],cyl1.shape[0])) 304 | ##test this add top and bottom borders to work 305 | # out2 = cv2.warpAffine(cyl3, M[:2,:], (cyl3.shape[1],cyl3.shape[0]+100)) 306 | 307 | del rows,cols 308 | #find the cells with empty pixels from the mask on the left half of the image mask 309 | rows, cols = np.where(cyl1_mask[:,:np.uint(cyl1_mask.shape[1]/2)] == 0) 310 | ##test this 311 | # rows, cols = np.where(cyl1_mask[:cyl3.shape[0],:cyl3.shape[1]] == 0) 312 | cyl1[rows, cols] = out2[rows, cols] 313 | 314 | # plt.figure() 315 | # plt.imshow(cyl1, cmap='gray') 316 | # plt.show() 317 | 318 | # Write out the result 319 | 320 | output_name = sys.argv[5] + "output_cylindrical.png" 321 | output_image = cyl1 322 | cv2.imwrite(output_name, output_image) 323 | 324 | master = cv2.imread("example_output2.png", 0) 325 | print(RMSD(2,cyl1, master)) 326 | return True 327 | 328 | 329 | def Bonus_cylindrical_warping(img1, img2, img3): 330 | 331 | # Write your codes here 332 | output_image = img1 # This is dummy output, change it to your output 333 | 334 | # Write out the result 335 | output_name = sys.argv[5] + "output_cylindrical_lpb.png" 336 | cv2.imwrite(output_name, output_image) 337 | 338 | return True 339 | 340 | 341 | ''' 342 | This exact function will be used to evaluate your results for HW2 343 | Compare your result with master image and get the difference, the grading 344 | criteria is posted on Piazza 345 | ''' 346 | 347 | 348 | def RMSD(questionID, target, master): 349 | # Get width, height, and number of channels of the master image 350 | master_height, master_width = master.shape[:2] 351 | master_channel = len(master.shape) 352 | 353 | # Get width, height, and number of channels of the target image 354 | target_height, target_width = target.shape[:2] 355 | target_channel = len(target.shape) 356 | 357 | # Validate the height, width and channels of the input image 358 | if (master_height != target_height or master_width != target_width or master_channel != target_channel): 359 | print("failes in size comparision") 360 | return -1 361 | else: 362 | nonZero_target = cv2.countNonZero(target) 363 | nonZero_master = cv2.countNonZero(master) 364 | 365 | if (questionID == 1): 366 | if (nonZero_target < 1200000): 367 | print("failed in non zero target in one") 368 | return -1 369 | elif(questionID == 2): 370 | if (nonZero_target < 700000): 371 | print("failed in non zero target in two") 372 | return -1 373 | else: 374 | print("incorrect q id") 375 | return -1 376 | 377 | total_diff = 0.0; 378 | master_channels = cv2.split(master); 379 | target_channels = cv2.split(target); 380 | 381 | for i in range(0, len(master_channels), 1): 382 | dst = cv2.absdiff(master_channels[i], target_channels[i]) 383 | dst = cv2.pow(dst, 2) 384 | mean = cv2.mean(dst) 385 | total_diff = total_diff + mean[0]**(1/2.0) 386 | 387 | return total_diff; 388 | 389 | if __name__ == '__main__': 390 | question_number = -1 391 | 392 | # Validate the input arguments 393 | if (len(sys.argv) != 6): 394 | help_message() 395 | sys.exit() 396 | else: 397 | question_number = int(sys.argv[1]) 398 | if (question_number > 4 or question_number < 1): 399 | print("Input parameters out of bound ...") 400 | sys.exit() 401 | 402 | input_image1 = cv2.imread(sys.argv[2], 0) 403 | input_image2 = cv2.imread(sys.argv[3], 0) 404 | input_image3 = cv2.imread(sys.argv[4], 0) 405 | 406 | function_launch = { 407 | '1': Perspective_warping, 408 | '2': Cylindrical_warping, 409 | '3': Bonus_perspective_warping, 410 | '4': Bonus_cylindrical_warping 411 | } 412 | 413 | # Call the function 414 | # print(function_launch.items()) 415 | function_launch[sys.argv[1]](input_image1, input_image2, input_image3) 416 | -------------------------------------------------------------------------------- /HW2-Panoramas/new1output_homography.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW2-Panoramas/new1output_homography.png -------------------------------------------------------------------------------- /HW2-Panoramas/new2output_cylindrical.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW2-Panoramas/new2output_cylindrical.png -------------------------------------------------------------------------------- /HW3-DetectionTracking/02-1.avi: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW3-DetectionTracking/02-1.avi -------------------------------------------------------------------------------- /HW3-DetectionTracking/detection_tracking.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | import cv2 4 | import matplotlib.pyplot as plt 5 | import numpy as np 6 | 7 | #changed 8 | face_cascade = cv2.CascadeClassifier('./haarcascade_frontalface_default.xml') 9 | # face_cascade = cv2.CascadeClassifier('/usr/local/share/OpenCV/haarcascades/haarcascade_frontalface_default.xml') 10 | 11 | term_crit = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 1000, 0.002) 12 | 13 | 14 | def help_message(): 15 | print("Usage: [Question_Number] [Input_Video] [Output_Directory]") 16 | print("[Question Number]") 17 | print("1 Camshift") 18 | print("2 Particle Filter") 19 | print("3 Kalman Filter") 20 | print("4 Optical Flow") 21 | print("[Input_Video]") 22 | print("Path to the input video") 23 | print("[Output_Directory]") 24 | print("Output directory") 25 | print("Example usages:") 26 | print(sys.argv[0] + " 1 " + "02-1.avi " + "./") 27 | 28 | 29 | def detect_one_face(im): 30 | gray=cv2.cvtColor(im, cv2.COLOR_BGR2GRAY) 31 | 32 | # print(gray) 33 | # cv2.imshow('grs',gray) 34 | # cv2.waitKey(0) 35 | 36 | faces = face_cascade.detectMultiScale(gray, 1.2, 3) 37 | if len(faces) == 0: 38 | return (0,0,0,0) 39 | return faces[0] 40 | 41 | def hsv_histogram_for_window(frame, window): 42 | # set up the ROI for tracking 43 | c,r,w,h = window 44 | roi = frame[r:r+h, c:c+w] 45 | hsv_roi = cv2.cvtColor(roi, cv2.COLOR_BGR2HSV) 46 | mask = cv2.inRange(hsv_roi, np.array((0., 60.,32.)), np.array((180.,255.,255.))) 47 | roi_hist = cv2.calcHist([hsv_roi],[0],mask,[180],[0,180]) 48 | cv2.normalize(roi_hist,roi_hist,0,255,cv2.NORM_MINMAX) 49 | return roi_hist 50 | 51 | 52 | def resample(weights): 53 | n = len(weights) 54 | indices = [] 55 | C = [0.] + [sum(weights[:i+1]) for i in range(n)] 56 | u0, j = np.random.random(), 0 57 | for u in [(u0+i)/n for i in range(n)]: 58 | while u > C[j]: 59 | j+=1 60 | indices.append(j-1) 61 | return indices 62 | 63 | def cmShift_tracker(v, file_name): 64 | # Open output file 65 | output_name = sys.argv[3] + file_name 66 | output = open(output_name,"w") 67 | 68 | frameCounter = 0 69 | # read first frame 70 | ret ,frame = v.read() 71 | 72 | #debug 73 | # cv2.imshow('frame',frame) 74 | # cv2.waitKey(0) 75 | # print(type(frame),frame) 76 | 77 | if ret == False: 78 | return 79 | 80 | # detect face in first frame 81 | c,r,w,h = detect_one_face(frame) 82 | # Write track point for first frame 83 | 84 | pt_x, pt_y=c + w/2.0,r + h/2.0 85 | #channged 86 | output.write("%d,%d,%d\n" % (frameCounter,pt_x,pt_y)) # Write as 0,pt_x,pt_y 87 | frameCounter = frameCounter + 1 88 | 89 | # set the initial tracking window 90 | track_window = (c,r,w,h) 91 | 92 | # calculate the HSV histogram in the window 93 | # NOTE: you do not need this in the Kalman, Particle or OF trackers 94 | roi_hist = hsv_histogram_for_window(frame, (c,r,w,h)) # this is provided for you 95 | 96 | # initialize the tracker 97 | # e.g. kf = cv2.KalmanFilter(4,2,0) 98 | # or: particles = np.ones((n_particles, 2), int) * initial_pos 99 | 100 | while(1): 101 | ret ,frame = v.read() # read another frame 102 | if ret == False: 103 | break 104 | 105 | # perform the tracking 106 | # e.g. cv2.meanShift, cv2.CamShift, or kalman.predict(), kalman.correct() 107 | 108 | # use the tracking result to get the tracking point (pt): 109 | # if you track a rect (e.g. face detector) take the mid point, 110 | # if you track particles - take the weighted average 111 | # the Kalman filter already has the tracking point in the state vector 112 | 113 | hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV) 114 | prob = cv2.calcBackProject([hsv], [0], roi_hist, [0, 180], 1) 115 | retval,track_window=cv2.CamShift(prob,track_window,term_crit) 116 | 117 | # Draw it on image 118 | # print(retval) 119 | pts = cv2.boxPoints(retval) 120 | pts = np.int0(pts) 121 | img2 = cv2.polylines(frame,[pts],True, 255,2) 122 | cv2.imshow('img2',img2) 123 | k = cv2.waitKey(60) & 0xff 124 | c,r,w,h=track_window 125 | 126 | # write the result to the output file 127 | output.write("%d,%d,%d\n" % (frameCounter,c + w/2.0,r + h/2.0)) # Write as frame_index,pt_x,pt_y 128 | frameCounter = frameCounter + 1 129 | 130 | output.close() 131 | 132 | # a function that, given a particle position, will return the particle's "fitness" 133 | def particleevaluator(back_proj, particle): 134 | return back_proj[particle[1],particle[0]] 135 | 136 | 137 | def particle_tracker(v, file_name): 138 | # Open output file 139 | 140 | output_name = sys.argv[3] + file_name 141 | output = open(output_name,"w") 142 | 143 | frameCounter = 0 144 | stepsize= 1 145 | # read first frame 146 | ret, frame = v.read() 147 | 148 | if ret == False: 149 | return 150 | 151 | # detect face in first frame 152 | c,r,w,h = detect_one_face(frame) 153 | 154 | # Write track point for first frame 155 | 156 | pt_x, pt_y=c + w/2.0,r + h/2.0 157 | #channged 158 | output.write("%d,%d,%d\n" % (frameCounter,pt_x,pt_y)) # Write as 0,pt_x,pt_y 159 | frameCounter = frameCounter + 1 160 | 161 | # set the initial tracking window 162 | track_window = (c,r,w,h) 163 | 164 | # initialize the tracker 165 | # e.g. kf = cv2.KalmanFilter(4,2,0) 166 | # or: particles = np.ones((n_particles, 2), int) * initial_pos 167 | 168 | n_particles = 450 169 | 170 | roi_hist = hsv_histogram_for_window(frame, (c,r,w,h)) # this is provided for you 171 | init_pos = np.array([c + w / 2.0, r + h / 2.0], int) # Initial position 172 | particles = np.ones((n_particles, 2), int) * init_pos # Init particles to init position 173 | 174 | hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV) 175 | prob = cv2.calcBackProject([hsv], [0], roi_hist, [0, 180], 1) 176 | 177 | # print(init_pos.shape) 178 | # print(particles.shape) 179 | f0 = particleevaluator(prob, particles.T) * np.ones(n_particles) # Evaluate appearance model 180 | weights = np.ones(n_particles) / n_particles # weights are uniform (at first) 181 | # print(np.average(particles,axis=0)) 182 | pos=init_pos 183 | 184 | while(1): 185 | stepsize=30 186 | ret ,frame = v.read() # read another frame 187 | if ret == False: 188 | break 189 | 190 | # if you track particles - take the weighted average 191 | # the Kalman filter already has the tracking point in the state vector 192 | 193 | # hist_bp: obtain using cv2.calcBackProject and the HSV histogram 194 | # c,r,w,h: obtain using detect_one_face() 195 | # Particle motion model: uniform step (TODO: find a better motion model) 196 | 197 | hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV) 198 | prob = cv2.calcBackProject([hsv], [0], roi_hist, [0, 180], 1) 199 | 200 | #moving in the direction of particle with maximum weight 201 | # x=pos[0]+np.random.uniform(-5, 5) 202 | # y=pos[1]+np.random.uniform(-5, 5) 203 | # init_pos=pos 204 | np.add(particles, np.random.uniform(-stepsize,stepsize, particles.shape), out=particles, casting="unsafe") 205 | 206 | # print(particles.shape) 207 | # print(np.argmax()) 208 | # Clip out-of-bounds particles,determine the width and height and clip to this rnge 209 | particles = particles.clip(np.zeros(2), np.array((frame.shape[1], frame.shape[0])) - 1).astype(int) 210 | 211 | f = particleevaluator(prob, particles.T) # Evaluate particles 212 | weights = np.float32(f.clip(1)) # Weight ~ histogram response 213 | weights /= np.sum(weights) # Normalize w 214 | pos = np.sum(particles.T * weights, axis=1).astype(int) # expected position: weighted average 215 | 216 | # print(f-f.clip(1)) 217 | 218 | if 1. / np.sum(weights ** 2) < n_particles / 2: # If particle cloud degenerate: 219 | # print('resampled') 220 | particles = particles[resample(weights), :] # Resample particles according to weights 221 | # resample() function is provided for you 222 | 223 | img2 = cv2.drawMarker(frame,(pos[0],pos[1]),(0,255,0),markerType=1,markerSize=10) 224 | cv2.imshow('img',img2) 225 | cv2.waitKey(60) 226 | 227 | # plt.imshow(cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)) 228 | # # plt.scatter(particles[1],particles[0], c='b', s=5) 229 | # plt.pause(0.05) 230 | 231 | # write the result to the output file 232 | output.write("%d,%d,%d\n" % (frameCounter,pos[0],pos[1])) # Write as frame_index,pt_x,pt_y 233 | frameCounter = frameCounter + 1 234 | 235 | output.close() 236 | 237 | 238 | def kf_tracker(v, file_name): 239 | # Open output file 240 | 241 | output_name = sys.argv[3] + file_name 242 | output = open(output_name,"w") 243 | 244 | frameCounter = 0 245 | stepsize= 1 246 | # read first frame 247 | ret, frame = v.read() 248 | 249 | if ret == False: 250 | return 251 | 252 | # detect face in first frame 253 | c,r,w,h = detect_one_face(frame) 254 | 255 | # Write track point for first frame 256 | 257 | pt_x, pt_y=c + w/2.0,r + h/2.0 258 | output.write("%d,%d,%d\n" % (frameCounter,pt_x,pt_y)) # Write as 0,pt_x,pt_y 259 | frameCounter = frameCounter + 1 260 | 261 | state = np.array([c + w / 2, r + h / 2, 0, 0], dtype=np.float32) # initial position 262 | kalman = cv2.KalmanFilter(4, 2) 263 | kalman.measurementMatrix = np.array([[1, 0, 0, 0], [0, 1, 0, 0]], np.float32) 264 | kalman.transitionMatrix = np.array([[1, 0, 1, 0], [0, 1, 0, 1], [0, 0, 1, 0], [0, 0, 0, 1]], np.float32) 265 | kalman.processNoiseCov = np.array([[1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], [0, 0, 0, 1]], np.float32) * 0.03 266 | kalman.measurementNoiseCov = np.array([[1, 0], [0, 1]], np.float32) * 0.00003 267 | kalman.errorCovPost = 1e-1 * np.eye(4, 4,dtype=np.float32) 268 | kalman.statePost = state 269 | # print(kalman.measurementMatrix) 270 | 271 | while(1): 272 | ret, frame = v.read() # read another frame 273 | if ret == False: 274 | break 275 | c, r, w, h = detect_one_face(frame) 276 | 277 | # if you track particles - take the weighted average 278 | # the Kalman filter already has the tracking point in the state vector 279 | 280 | # hist_bp: obtain using cv2.calcBackProject and the HSV histogram 281 | # c,r,w,h: obtain using detect_one_face() 282 | # Particle motion model: uniform step (TODO: find a better motion model) 283 | 284 | # hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV) 285 | # prob = cv2.calcBackProject([hsv], [0], roi_hist, [0, 180], 1) 286 | # retval,track_window=cv2.CamShift(prob,track_window,term_crit) 287 | # c, r, w, h = track_window 288 | 289 | prediction = kalman.predict() 290 | pos=prediction[:2] 291 | 292 | # if not((c==0)and(r==0)and(w==0)and(h==0)): # e.g. face found 293 | if (any([c,r,w,h])>0): # e.g. face found 294 | posterior = kalman.correct(np.array([[np.float32(c+w/2)],[np.float32(r+h/2)]])) 295 | # print(posterior[:2]) 296 | pos = posterior[:2] 297 | else: 298 | pass 299 | # print('entered kalman',frameCounter) 300 | 301 | # use prediction or posterior as your tracking result 302 | 303 | img2 = cv2.drawMarker(frame,(pos[0],pos[1]),(0,255,0),markerType=1,markerSize=10) 304 | cv2.imshow('img',img2) 305 | cv2.waitKey(60) 306 | 307 | # write the result to the output file 308 | output.write("%d,%d,%d\n" % (frameCounter,pos[0],pos[1])) # Write as frame_index,pt_x,pt_y 309 | frameCounter = frameCounter + 1 310 | 311 | output.close() 312 | 313 | def of_tracker(v, file_name): 314 | # Open output file 315 | 316 | output_name = sys.argv[3] + file_name 317 | output = open(output_name,"w") 318 | 319 | frameCounter = 0 320 | # read first frame 321 | ret, frame = v.read() 322 | 323 | if ret == False: 324 | return 325 | 326 | # detect face in first frame 327 | c,r,w,h = detect_one_face(frame) 328 | # print(frame.shape) 329 | mask = np.zeros_like(frame) 330 | 331 | facemask = np.zeros(shape=(frame.shape[0],frame.shape[1]),dtype=np.uint8) 332 | facemask[r+5:r + h-5, c+8:c + w-8] = 255 333 | 334 | # cv2.imshow('mask',facemask) 335 | # cv2.waitKey(0) 336 | lk_params = dict(winSize=(10, 10), 337 | maxLevel=6, 338 | criteria=(cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 50, 0.0003)) 339 | feature_params = dict(maxCorners=100, 340 | qualityLevel=0.2, 341 | minDistance=9, 342 | blockSize=15) 343 | 344 | # cv2.imshow('face',frame[r:r+h,c:c+w]) 345 | # cv2.waitKey(0) 346 | 347 | # Write track point for first frame 348 | old_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) 349 | p0 = cv2.goodFeaturesToTrack(old_gray,mask=facemask , **feature_params) 350 | 351 | pt_x, pt_y=c + w/2.0,r + h/2.0 352 | output.write("%d,%d,%d\n" % (frameCounter,pt_x,pt_y)) # Write as 0,pt_x,pt_y 353 | frameCounter = frameCounter + 1 354 | color = np.random.randint(0, 255, (100, 3)) 355 | 356 | while(1): 357 | ret, frame = v.read() # read another frame 358 | if ret == False: 359 | break 360 | c, r, w, h = detect_one_face(frame) 361 | 362 | frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) 363 | p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params) 364 | 365 | # use prediction or posterior as your tracking result 366 | # Select good points 367 | good_new = p1[st == 1] 368 | good_old = p0[st == 1] 369 | # draw the tracks 370 | for i, (new, old) in enumerate(zip(good_new, good_old)): 371 | a, b = new.ravel() 372 | c, d = old.ravel() 373 | mask = cv2.line(mask, (a, b), (c, d), color[i].tolist(), 2) 374 | frame = cv2.circle(frame, (a, b), 5, color[i].tolist(), -1) 375 | img = cv2.add(frame, mask) 376 | cv2.imshow('frame', img) 377 | k = cv2.waitKey(30) & 0xff 378 | if k == 27: 379 | break 380 | 381 | # print(good_new.shape) 382 | pos =(c+w/2,r+h/2) 383 | if (all([c,r,w,h])==0): # e.g. face not found 384 | pos = np.average(good_new,axis=0) 385 | # print(frameCounter,'using optical flow') 386 | # print(pos) 387 | else: 388 | pass 389 | # print('entered kalman',frameCounter) 390 | 391 | # write the result to the output file 392 | output.write("%d,%d,%d\n" % (frameCounter,pos[0],pos[1])) # Write as frame_index,pt_x,pt_y 393 | frameCounter = frameCounter + 1 394 | old_gray = frame_gray.copy() 395 | p0 = good_new.reshape(-1, 1, 2) 396 | 397 | output.close() 398 | 399 | 400 | if __name__ == '__main__': 401 | question_number = -1 402 | 403 | # Validate the input arguments 404 | if (len(sys.argv) != 4): 405 | help_message() 406 | sys.exit() 407 | else: 408 | question_number = int(sys.argv[1]) 409 | if (question_number > 4 or question_number < 1): 410 | print("Input parameters out of bound ...") 411 | sys.exit() 412 | 413 | # read video file 414 | video = cv2.VideoCapture(sys.argv[2]); 415 | 416 | if (question_number == 1): 417 | cmShift_tracker(video, "output_camshift.txt") 418 | elif (question_number == 2): 419 | particle_tracker(video, "output_particle.txt") 420 | elif (question_number == 3): 421 | kf_tracker(video, "output_kalman.txt") 422 | elif (question_number == 4): 423 | of_tracker(video, "output_of.txt") 424 | 425 | ''' 426 | For Kalman Filter: 427 | 428 | # --- init 429 | 430 | state = np.array([c+w/2,r+h/2,0,0], dtype='float64') # initial position 431 | kalman.transitionMatrix = np.array([[1., 0., .1, 0.], 432 | [0., 1., 0., .1], 433 | [0., 0., 1., 0.], 434 | [0., 0., 0., 1.]]) 435 | kalman.measurementMatrix = 1. * np.eye(2, 4) 436 | kalman.processNoiseCov = 1e-5 * np.eye(4, 4) 437 | kalman.measurementNoiseCov = 1e-3 * np.eye(2, 2) 438 | kalman.errorCovPost = 1e-1 * np.eye(4, 4) 439 | kalman.statePost = state 440 | 441 | 442 | # --- tracking 443 | 444 | prediction = kalman.predict() 445 | 446 | # ... 447 | # obtain measurement 448 | 449 | if measurement_valid: # e.g. face found 450 | # ... 451 | posterior = kalman.correct(measurement) 452 | 453 | # use prediction or posterior as your tracking result 454 | ''' 455 | 456 | ''' 457 | For Particle Filter: 458 | 459 | # --- init 460 | 461 | # a function that, given a particle position, will return the particle's "fitness" 462 | def particleevaluator(back_proj, particle): 463 | return back_proj[particle[1],particle[0]] 464 | 465 | # hist_bp: obtain using cv2.calcBackProject and the HSV histogram 466 | # c,r,w,h: obtain using detect_one_face() 467 | n_particles = 200 468 | 469 | init_pos = np.array([c + w/2.0,r + h/2.0], int) # Initial position 470 | particles = np.ones((n_particles, 2), int) * init_pos # Init particles to init position 471 | f0 = particleevaluator(hist_bp, pos) * np.ones(n_particles) # Evaluate appearance model 472 | weights = np.ones(n_particles) / n_particles # weights are uniform (at first) 473 | 474 | 475 | # --- tracking 476 | 477 | # Particle motion model: uniform step (TODO: find a better motion model) 478 | np.add(particles, np.random.uniform(-stepsize, stepsize, particles.shape), out=particles, casting="unsafe") 479 | 480 | # Clip out-of-bounds particles 481 | particles = particles.clip(np.zeros(2), np.array((im_w,im_h))-1).astype(int) 482 | 483 | f = particleevaluator(hist_bp, particles.T) # Evaluate particles 484 | weights = np.float32(f.clip(1)) # Weight ~ histogram response 485 | weights /= np.sum(weights) # Normalize w 486 | pos = np.sum(particles.T * weights, axis=1).astype(int) # expected position: weighted average 487 | 488 | if 1. / np.sum(weights**2) < n_particles / 2.: # If particle cloud degenerate: 489 | particles = particles[resample(weights),:] # Resample particles according to weights 490 | # resample() function is provided for you 491 | ''' 492 | -------------------------------------------------------------------------------- /HW3-DetectionTracking/output_camshift.txt: -------------------------------------------------------------------------------- 1 | 0,92,67 2 | 1,85,70 3 | 2,86,72 4 | 3,85,71 5 | 4,86,71 6 | 5,85,72 7 | 6,85,71 8 | 7,85,71 9 | 8,85,71 10 | 9,86,69 11 | 10,86,70 12 | 11,86,70 13 | 12,86,70 14 | 13,87,69 15 | 14,87,67 16 | 15,87,68 17 | 16,86,67 18 | 17,88,64 19 | 18,88,65 20 | 19,88,68 21 | 20,89,67 22 | 21,89,66 23 | 22,90,64 24 | 23,92,63 25 | 24,90,65 26 | 25,92,63 27 | 26,92,64 28 | 27,93,63 29 | 28,94,59 30 | 29,93,60 31 | 30,94,60 32 | 31,93,59 33 | 32,94,60 34 | 33,94,61 35 | 34,94,60 36 | 35,94,59 37 | 36,96,60 38 | 37,94,61 39 | 38,95,60 40 | 39,93,61 41 | 40,93,62 42 | 41,91,63 43 | 42,90,64 44 | 43,87,63 45 | 44,85,67 46 | 45,81,66 47 | 46,78,67 48 | 47,76,67 49 | 48,73,68 50 | 49,71,70 51 | 50,70,72 52 | 51,67,73 53 | 52,65,74 54 | 53,63,74 55 | 54,62,76 56 | 55,62,76 57 | 56,62,75 58 | 57,60,75 59 | 58,63,77 60 | 59,62,78 61 | 60,63,76 62 | 61,63,75 63 | 62,65,77 64 | 63,65,77 65 | 64,65,77 66 | 65,66,75 67 | 66,66,74 68 | 67,67,75 69 | 68,67,75 70 | 69,68,75 71 | 70,69,74 72 | 71,67,73 73 | 72,68,72 74 | 73,67,69 75 | 74,69,67 76 | 75,69,66 77 | 76,70,64 78 | 77,71,62 79 | 78,71,61 80 | 79,71,61 81 | 80,76,61 82 | 81,77,60 83 | 82,78,60 84 | 83,80,58 85 | 84,86,60 86 | 85,88,59 87 | 86,93,59 88 | 87,93,56 89 | 88,98,57 90 | 89,99,59 91 | 90,104,57 92 | 91,106,58 93 | 92,110,58 94 | 93,110,55 95 | 94,111,55 96 | 95,112,55 97 | 96,111,55 98 | 97,110,56 99 | 98,110,55 100 | 99,109,60 101 | 100,105,62 102 | 101,102,62 103 | 102,101,61 104 | 103,96,64 105 | 104,94,61 106 | 105,91,63 107 | 106,88,64 108 | 107,86,63 109 | 108,84,63 110 | 109,83,62 111 | 110,81,62 112 | 111,80,61 113 | 112,78,61 114 | 113,78,60 115 | 114,77,61 116 | 115,77,60 117 | 116,76,61 118 | 117,76,59 119 | 118,77,60 120 | 119,77,60 121 | 120,77,59 122 | 121,76,60 123 | 122,76,60 124 | 123,77,61 125 | 124,76,60 126 | 125,77,60 127 | 126,76,59 128 | 127,76,57 129 | 128,75,58 130 | 129,75,57 131 | 130,76,59 132 | 131,76,57 133 | 132,75,58 134 | 133,75,59 135 | 134,75,57 136 | 135,75,55 137 | 136,75,56 138 | 137,75,57 139 | 138,77,58 140 | 139,78,59 141 | 140,79,58 142 | 141,79,58 143 | 142,81,56 144 | 143,82,58 145 | 144,82,58 146 | 145,82,57 147 | 146,82,57 148 | 147,81,56 149 | 148,83,56 150 | 149,83,57 151 | 150,83,56 152 | 151,83,56 153 | 152,82,56 154 | 153,83,55 155 | 154,83,56 156 | 155,82,58 157 | 156,82,56 158 | 157,83,57 159 | 158,83,57 160 | 159,83,58 161 | 160,82,57 162 | 161,83,61 163 | 162,83,60 164 | 163,82,59 165 | 164,83,59 166 | 165,84,59 167 | 166,84,61 168 | 167,84,61 169 | 168,85,61 170 | 169,84,60 171 | 170,85,61 172 | 171,84,63 173 | 172,85,64 174 | 173,86,64 175 | 174,87,65 176 | 175,87,66 177 | 176,87,65 178 | 177,86,66 179 | 178,87,67 180 | 179,87,66 181 | 180,87,65 182 | 181,87,64 183 | 182,87,67 184 | 183,89,65 185 | 184,89,66 186 | 185,91,68 187 | 186,92,68 188 | 187,93,70 189 | 188,94,71 190 | 189,95,72 191 | 190,99,73 192 | 191,99,75 193 | 192,102,76 194 | 193,103,74 195 | 194,104,77 196 | 195,107,78 197 | 196,107,78 198 | 197,109,81 199 | 198,110,83 200 | 199,112,83 201 | 200,115,85 202 | 201,116,84 203 | 202,118,85 204 | 203,119,84 205 | 204,119,84 206 | 205,121,82 207 | 206,122,80 208 | 207,121,78 209 | 208,121,76 210 | 209,120,75 211 | 210,117,73 212 | 211,113,68 213 | 212,107,66 214 | 213,101,65 215 | 214,96,65 216 | 215,90,63 217 | 216,85,62 218 | 217,80,61 219 | 218,75,62 220 | 219,73,61 221 | 220,68,61 222 | 221,66,60 223 | 222,64,63 224 | 223,61,62 225 | 224,60,63 226 | 225,59,63 227 | 226,59,63 228 | 227,59,63 229 | 228,60,66 230 | 229,61,66 231 | 230,64,66 232 | 231,67,69 233 | 232,70,68 234 | 233,74,70 235 | 234,76,70 236 | 235,79,73 237 | 236,82,72 238 | 237,85,72 239 | 238,87,72 240 | 239,89,73 241 | 240,91,73 242 | 241,93,73 243 | 242,94,73 244 | 243,96,73 245 | 244,93,74 246 | 245,94,75 247 | 246,95,73 248 | 247,93,73 249 | 248,94,71 250 | 249,93,69 251 | 250,90,69 252 | 251,89,69 253 | 252,87,67 254 | 253,87,68 255 | 254,86,69 256 | 255,84,67 257 | 256,85,67 258 | -------------------------------------------------------------------------------- /HW3-DetectionTracking/output_kalman.txt: -------------------------------------------------------------------------------- 1 | 0,92,67 2 | 1,91,67 3 | 2,90,68 4 | 3,90,68 5 | 4,92,66 6 | 5,93,65 7 | 6,93,66 8 | 7,93,66 9 | 8,93,65 10 | 9,92,66 11 | 10,91,64 12 | 11,91,64 13 | 12,92,64 14 | 13,92,64 15 | 14,92,64 16 | 15,92,61 17 | 16,93,60 18 | 17,95,61 19 | 18,94,61 20 | 19,94,59 21 | 20,95,58 22 | 21,95,58 23 | 22,95,56 24 | 23,96,55 25 | 24,98,55 26 | 25,98,55 27 | 26,99,55 28 | 27,99,54 29 | 28,99,53 30 | 29,100,53 31 | 30,100,52 32 | 31,100,52 33 | 32,100,52 34 | 33,100,52 35 | 34,100,51 36 | 35,101,53 37 | 36,102,54 38 | 37,102,54 39 | 38,102,54 40 | 39,100,54 41 | 40,99,53 42 | 41,98,54 43 | 42,96,54 44 | 43,92,55 45 | 44,88,55 46 | 45,85,58 47 | 46,83,60 48 | 47,79,61 49 | 48,76,61 50 | 49,74,62 51 | 50,71,63 52 | 51,68,64 53 | 52,66,65 54 | 53,63,71 55 | 54,55,68 56 | 55,53,68 57 | 56,50,68 58 | 57,47,68 59 | 58,44,68 60 | 59,41,68 61 | 60,37,68 62 | 61,34,68 63 | 62,31,67 64 | 63,28,67 65 | 64,24,67 66 | 65,21,67 67 | 66,18,67 68 | 67,65,69 69 | 68,68,70 70 | 69,71,71 71 | 70,74,72 72 | 71,65,66 73 | 72,68,64 74 | 73,71,63 75 | 74,70,62 76 | 75,69,57 77 | 76,70,55 78 | 77,72,53 79 | 78,75,51 80 | 79,78,51 81 | 80,81,51 82 | 81,83,49 83 | 82,86,47 84 | 83,90,46 85 | 84,94,45 86 | 85,96,46 87 | 86,99,46 88 | 87,103,46 89 | 88,106,46 90 | 89,108,46 91 | 90,111,47 92 | 91,114,47 93 | 92,117,48 94 | 93,117,47 95 | 94,117,46 96 | 95,118,45 97 | 96,119,45 98 | 97,120,44 99 | 98,121,44 100 | 99,114,48 101 | 100,112,50 102 | 101,110,52 103 | 102,107,53 104 | 103,103,54 105 | 104,99,55 106 | 105,96,53 107 | 106,93,53 108 | 107,91,54 109 | 108,89,53 110 | 109,86,51 111 | 110,83,50 112 | 111,82,50 113 | 112,81,49 114 | 113,80,49 115 | 114,80,50 116 | 115,79,48 117 | 116,79,47 118 | 117,78,47 119 | 118,78,47 120 | 119,77,47 121 | 120,77,47 122 | 121,77,48 123 | 122,77,49 124 | 123,78,49 125 | 124,78,49 126 | 125,78,49 127 | 126,77,49 128 | 127,76,49 129 | 128,76,48 130 | 129,76,48 131 | 130,76,48 132 | 131,75,48 133 | 132,75,48 134 | 133,75,48 135 | 134,75,49 136 | 135,76,49 137 | 136,77,49 138 | 137,77,49 139 | 138,77,47 140 | 139,77,46 141 | 140,79,47 142 | 141,80,48 143 | 142,80,48 144 | 143,81,49 145 | 144,82,49 146 | 145,83,49 147 | 146,84,49 148 | 147,85,49 149 | 148,85,49 150 | 149,85,49 151 | 150,85,49 152 | 151,85,49 153 | 152,84,48 154 | 153,84,48 155 | 154,84,48 156 | 155,83,47 157 | 156,83,47 158 | 157,84,48 159 | 158,85,49 160 | 159,84,48 161 | 160,84,48 162 | 161,85,49 163 | 162,85,49 164 | 163,86,50 165 | 164,84,51 166 | 165,84,52 167 | 166,85,52 168 | 167,86,53 169 | 168,86,54 170 | 169,86,55 171 | 170,87,55 172 | 171,87,58 173 | 172,87,58 174 | 173,87,58 175 | 174,87,58 176 | 175,88,60 177 | 176,88,61 178 | 177,87,61 179 | 178,87,61 180 | 179,89,61 181 | 180,90,60 182 | 181,90,60 183 | 182,91,59 184 | 183,93,58 185 | 184,93,59 186 | 185,93,60 187 | 186,96,61 188 | 187,98,61 189 | 188,100,61 190 | 189,102,62 191 | 190,105,65 192 | 191,108,67 193 | 192,110,70 194 | 193,111,71 195 | 194,112,71 196 | 195,113,72 197 | 196,115,73 198 | 197,116,74 199 | 198,117,75 200 | 199,119,76 201 | 200,120,77 202 | 201,122,78 203 | 202,123,78 204 | 203,124,79 205 | 204,126,80 206 | 205,127,81 207 | 206,128,82 208 | 207,130,83 209 | 208,131,84 210 | 209,133,85 211 | 210,134,85 212 | 211,135,86 213 | 212,137,87 214 | 213,109,57 215 | 214,104,55 216 | 215,98,55 217 | 216,90,53 218 | 217,83,52 219 | 218,78,52 220 | 219,73,52 221 | 220,69,52 222 | 221,68,53 223 | 222,66,54 224 | 223,63,54 225 | 224,61,55 226 | 225,60,55 227 | 226,60,56 228 | 227,62,56 229 | 228,64,56 230 | 229,66,59 231 | 230,69,63 232 | 231,73,64 233 | 232,77,65 234 | 233,81,66 235 | 234,85,67 236 | 235,89,68 237 | 236,93,69 238 | 237,97,70 239 | 238,100,71 240 | 239,98,68 241 | 240,100,68 242 | 241,103,68 243 | 242,105,68 244 | 243,108,68 245 | 244,110,69 246 | 245,103,64 247 | 246,101,65 248 | 247,101,65 249 | 248,99,62 250 | 249,97,60 251 | 250,96,61 252 | 251,94,62 253 | 252,92,61 254 | 253,91,60 255 | 254,90,59 256 | 255,89,57 257 | 256,87,57 258 | -------------------------------------------------------------------------------- /HW3-DetectionTracking/output_of.txt: -------------------------------------------------------------------------------- 1 | 0,92,67 2 | 1,127,68 3 | 2,94,70 4 | 3,94,70 5 | 4,122,66 6 | 5,94,69 7 | 6,122,67 8 | 7,121,66 9 | 8,121,66 10 | 9,124,67 11 | 10,120,64 12 | 11,123,65 13 | 12,126,65 14 | 13,95,66 15 | 14,96,65 16 | 15,129,61 17 | 16,127,61 18 | 17,130,62 19 | 18,129,61 20 | 19,129,59 21 | 20,128,59 22 | 21,129,58 23 | 22,131,56 24 | 23,132,56 25 | 24,132,56 26 | 25,129,56 27 | 26,131,55 28 | 27,130,54 29 | 28,129,54 30 | 29,128,53 31 | 30,129,53 32 | 31,102,56 33 | 32,102,55 34 | 33,102,55 35 | 34,103,55 36 | 35,128,54 37 | 36,130,54 38 | 37,103,56 39 | 38,103,56 40 | 39,132,54 41 | 40,101,57 42 | 41,133,54 43 | 42,130,55 44 | 43,132,55 45 | 44,93,60 46 | 45,122,59 47 | 46,121,61 48 | 47,115,61 49 | 48,110,62 50 | 49,76,66 51 | 50,73,67 52 | 51,69,69 53 | 52,66,71 54 | 53,89,72 55 | 54,80,68 56 | 55,78,69 57 | 56,59,71 58 | 57,60,71 59 | 58,59,69 60 | 59,61,70 61 | 60,63,70 62 | 61,64,70 63 | 62,65,70 64 | 63,65,70 65 | 64,65,69 66 | 65,65,69 67 | 66,65,69 68 | 67,91,70 69 | 68,95,71 70 | 69,65,69 71 | 70,66,68 72 | 71,92,66 73 | 72,94,65 74 | 73,69,62 75 | 74,92,62 76 | 75,93,57 77 | 76,96,56 78 | 77,95,54 79 | 78,100,52 80 | 79,99,52 81 | 80,106,51 82 | 81,105,49 83 | 82,113,48 84 | 83,119,46 85 | 84,117,46 86 | 85,122,47 87 | 86,125,46 88 | 87,132,47 89 | 88,135,46 90 | 89,140,47 91 | 90,114,47 92 | 91,116,47 93 | 92,147,48 94 | 93,150,47 95 | 94,119,47 96 | 95,119,47 97 | 96,119,48 98 | 97,118,48 99 | 98,117,49 100 | 99,146,49 101 | 100,149,51 102 | 101,145,52 103 | 102,142,53 104 | 103,139,54 105 | 104,102,53 106 | 105,129,53 107 | 106,126,54 108 | 107,125,54 109 | 108,119,53 110 | 109,118,51 111 | 110,113,51 112 | 111,111,51 113 | 112,108,49 114 | 113,108,50 115 | 114,106,50 116 | 115,104,48 117 | 116,105,48 118 | 117,104,48 119 | 118,101,48 120 | 119,102,48 121 | 120,101,48 122 | 121,100,49 123 | 122,100,49 124 | 123,99,49 125 | 124,99,49 126 | 125,99,49 127 | 126,98,50 128 | 127,97,49 129 | 128,98,49 130 | 129,97,49 131 | 130,98,49 132 | 131,98,49 133 | 132,97,48 134 | 133,97,49 135 | 134,96,49 136 | 135,96,49 137 | 136,94,49 138 | 137,79,51 139 | 138,97,47 140 | 139,80,50 141 | 140,94,48 142 | 141,94,48 143 | 142,94,49 144 | 143,83,50 145 | 144,98,49 146 | 145,85,50 147 | 146,99,49 148 | 147,99,49 149 | 148,100,49 150 | 149,85,49 151 | 150,100,49 152 | 151,100,49 153 | 152,99,48 154 | 153,100,49 155 | 154,99,48 156 | 155,98,48 157 | 156,99,48 158 | 157,99,49 159 | 158,99,49 160 | 159,99,48 161 | 160,100,49 162 | 161,85,51 163 | 162,85,52 164 | 163,85,53 165 | 164,99,52 166 | 165,98,52 167 | 166,99,53 168 | 167,100,54 169 | 168,87,55 170 | 169,103,56 171 | 170,102,55 172 | 171,106,59 173 | 172,105,58 174 | 173,88,60 175 | 174,89,61 176 | 175,109,61 177 | 176,110,61 178 | 177,90,62 179 | 178,90,62 180 | 179,113,61 181 | 180,113,61 182 | 181,111,60 183 | 182,112,59 184 | 183,114,59 185 | 184,116,60 186 | 185,95,61 187 | 186,120,61 188 | 187,98,62 189 | 188,100,63 190 | 189,102,64 191 | 190,129,66 192 | 191,107,68 193 | 192,139,71 194 | 193,139,71 195 | 194,116,68 196 | 195,118,70 197 | 196,121,71 198 | 197,123,73 199 | 198,126,74 200 | 199,128,76 201 | 200,130,76 202 | 201,131,77 203 | 202,133,77 204 | 203,134,77 205 | 204,136,75 206 | 205,135,75 207 | 206,136,73 208 | 207,137,70 209 | 208,136,68 210 | 209,133,65 211 | 210,130,62 212 | 211,124,59 213 | 212,118,57 214 | 213,151,57 215 | 214,144,55 216 | 215,134,56 217 | 216,129,53 218 | 217,119,53 219 | 218,108,53 220 | 219,100,53 221 | 220,95,52 222 | 221,93,54 223 | 222,89,54 224 | 223,85,55 225 | 224,86,56 226 | 225,84,55 227 | 226,83,57 228 | 227,83,56 229 | 228,86,57 230 | 229,88,60 231 | 230,91,64 232 | 231,96,64 233 | 232,105,65 234 | 233,84,63 235 | 234,87,64 236 | 235,90,65 237 | 236,93,65 238 | 237,95,65 239 | 238,97,65 240 | 239,127,68 241 | 240,100,65 242 | 241,101,65 243 | 242,101,64 244 | 243,102,64 245 | 244,102,64 246 | 245,123,64 247 | 246,127,66 248 | 247,127,65 249 | 248,123,62 250 | 249,121,61 251 | 250,119,62 252 | 251,115,62 253 | 252,116,61 254 | 253,111,61 255 | 254,112,59 256 | 255,111,58 257 | 256,111,58 258 | -------------------------------------------------------------------------------- /HW3-DetectionTracking/output_particle.txt: -------------------------------------------------------------------------------- 1 | 0,92,67 2 | 1,87,66 3 | 2,88,70 4 | 3,87,69 5 | 4,86,69 6 | 5,84,69 7 | 6,83,68 8 | 7,84,69 9 | 8,84,69 10 | 9,86,68 11 | 10,85,67 12 | 11,84,68 13 | 12,85,69 14 | 13,87,69 15 | 14,85,67 16 | 15,86,65 17 | 16,86,64 18 | 17,87,62 19 | 18,88,62 20 | 19,89,67 21 | 20,88,66 22 | 21,89,67 23 | 22,90,65 24 | 23,92,62 25 | 24,88,63 26 | 25,90,62 27 | 26,92,65 28 | 27,92,62 29 | 28,94,60 30 | 29,93,58 31 | 30,92,58 32 | 31,94,59 33 | 32,95,60 34 | 33,94,62 35 | 34,94,61 36 | 35,93,62 37 | 36,96,62 38 | 37,95,62 39 | 38,96,60 40 | 39,94,58 41 | 40,94,58 42 | 41,93,60 43 | 42,90,63 44 | 43,88,62 45 | 44,87,66 46 | 45,81,67 47 | 46,77,65 48 | 47,75,65 49 | 48,75,69 50 | 49,72,69 51 | 50,69,71 52 | 51,67,72 53 | 52,67,73 54 | 53,65,72 55 | 54,62,72 56 | 55,61,74 57 | 56,63,75 58 | 57,60,75 59 | 58,62,78 60 | 59,63,77 61 | 60,63,73 62 | 61,63,75 63 | 62,63,74 64 | 63,63,75 65 | 64,64,75 66 | 65,66,76 67 | 66,66,74 68 | 67,67,74 69 | 68,67,74 70 | 69,66,75 71 | 70,69,76 72 | 71,67,76 73 | 72,68,72 74 | 73,65,66 75 | 74,67,68 76 | 75,69,66 77 | 76,69,65 78 | 77,69,59 79 | 78,70,59 80 | 79,70,59 81 | 80,71,59 82 | 81,75,59 83 | 82,77,60 84 | 83,77,61 85 | 84,84,61 86 | 85,87,58 87 | 86,91,59 88 | 87,92,56 89 | 88,97,58 90 | 89,100,57 91 | 90,104,58 92 | 91,106,56 93 | 92,111,57 94 | 93,111,56 95 | 94,113,56 96 | 95,114,52 97 | 96,111,55 98 | 97,110,56 99 | 98,109,55 100 | 99,109,59 101 | 100,107,61 102 | 101,105,63 103 | 102,104,61 104 | 103,99,64 105 | 104,96,62 106 | 105,91,60 107 | 106,90,61 108 | 107,86,63 109 | 108,84,62 110 | 109,82,62 111 | 110,81,62 112 | 111,79,58 113 | 112,79,59 114 | 113,79,60 115 | 114,77,59 116 | 115,78,60 117 | 116,77,61 118 | 117,75,60 119 | 118,75,60 120 | 119,77,58 121 | 120,76,58 122 | 121,76,58 123 | 122,76,58 124 | 123,75,60 125 | 124,77,60 126 | 125,76,61 127 | 126,76,61 128 | 127,77,57 129 | 128,74,57 130 | 129,75,57 131 | 130,75,58 132 | 131,75,55 133 | 132,74,54 134 | 133,74,56 135 | 134,76,55 136 | 135,76,55 137 | 136,76,55 138 | 137,75,57 139 | 138,75,58 140 | 139,78,57 141 | 140,78,56 142 | 141,79,58 143 | 142,79,57 144 | 143,83,59 145 | 144,81,57 146 | 145,81,58 147 | 146,82,58 148 | 147,80,56 149 | 148,81,53 150 | 149,82,55 151 | 150,83,55 152 | 151,82,55 153 | 152,80,57 154 | 153,82,55 155 | 154,82,54 156 | 155,81,56 157 | 156,82,54 158 | 157,82,57 159 | 158,83,57 160 | 159,82,58 161 | 160,79,54 162 | 161,82,59 163 | 162,82,61 164 | 163,83,60 165 | 164,82,58 166 | 165,84,59 167 | 166,84,59 168 | 167,83,62 169 | 168,84,60 170 | 169,83,58 171 | 170,84,62 172 | 171,84,65 173 | 172,85,62 174 | 173,85,62 175 | 174,86,65 176 | 175,87,66 177 | 176,85,65 178 | 177,87,64 179 | 178,86,64 180 | 179,86,66 181 | 180,86,63 182 | 181,86,63 183 | 182,86,66 184 | 183,88,66 185 | 184,89,65 186 | 185,91,66 187 | 186,92,69 188 | 187,92,68 189 | 188,94,69 190 | 189,95,71 191 | 190,96,70 192 | 191,100,74 193 | 192,102,75 194 | 193,103,73 195 | 194,105,77 196 | 195,106,78 197 | 196,107,78 198 | 197,108,80 199 | 198,110,82 200 | 199,109,86 201 | 200,111,88 202 | 201,113,90 203 | 202,116,91 204 | 203,116,89 205 | 204,117,88 206 | 205,118,88 207 | 206,121,89 208 | 207,121,85 209 | 208,120,80 210 | 209,119,78 211 | 210,116,77 212 | 211,113,72 213 | 212,111,67 214 | 213,104,64 215 | 214,99,63 216 | 215,92,60 217 | 216,86,60 218 | 217,82,62 219 | 218,76,62 220 | 219,73,57 221 | 220,69,59 222 | 221,66,59 223 | 222,63,59 224 | 223,61,59 225 | 224,59,61 226 | 225,59,60 227 | 226,59,60 228 | 227,58,59 229 | 228,60,62 230 | 229,61,63 231 | 230,62,63 232 | 231,64,67 233 | 232,67,66 234 | 233,73,68 235 | 234,74,69 236 | 235,77,70 237 | 236,82,72 238 | 237,83,72 239 | 238,86,72 240 | 239,88,73 241 | 240,90,71 242 | 241,93,71 243 | 242,92,73 244 | 243,95,73 245 | 244,93,73 246 | 245,94,73 247 | 246,94,73 248 | 247,95,71 249 | 248,94,73 250 | 249,93,71 251 | 250,91,70 252 | 251,88,68 253 | 252,88,67 254 | 253,86,66 255 | 254,85,67 256 | 255,85,67 257 | 256,84,64 258 | -------------------------------------------------------------------------------- /HW3-DetectionTracking/plot_results.py: -------------------------------------------------------------------------------- 1 | import pandas as pd 2 | from matplotlib import pyplot as plt 3 | 4 | CAM = pd.read_csv('output_camshift.txt') 5 | CAM.columns = ['index', 'x', 'y'] 6 | Kalman = pd.read_csv('output_kalman.txt') 7 | Kalman.columns = ['index', 'x', 'y'] 8 | particle = pd.read_csv('output_particle.txt') 9 | particle.columns = ['index', 'x', 'y'] 10 | optical = pd.read_csv('output_of.txt') 11 | optical.columns = ['index', 'x', 'y'] 12 | plt.scatter(x='x', y='y', data=CAM) 13 | plt.scatter(x='x', y='y', data=Kalman) 14 | plt.scatter(x='x', y='y', data=particle) 15 | plt.scatter(x='x', y='y', data=optical) 16 | plt.legend(['CAM', 'Kalman', 'Particle', 'Optical']) 17 | plt.xlabel('x') 18 | plt.ylabel('y') 19 | plt.grid(True) 20 | plt.show() -------------------------------------------------------------------------------- /HW4-Segmentation/astronaut.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW4-Segmentation/astronaut.png -------------------------------------------------------------------------------- /HW4-Segmentation/astronaut_marking.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW4-Segmentation/astronaut_marking.png -------------------------------------------------------------------------------- /HW4-Segmentation/drawing.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW4-Segmentation/drawing.png -------------------------------------------------------------------------------- /HW4-Segmentation/main.py: -------------------------------------------------------------------------------- 1 | # ================================================ 2 | # Skeleton codes for HW4 3 | # Read the skeleton codes carefully and put all your 4 | # codes into main function 5 | # ================================================ 6 | 7 | import cv2 8 | import numpy as np 9 | import matplotlib.pyplot as plt 10 | from skimage.segmentation import slic 11 | from skimage.segmentation import mark_boundaries 12 | from skimage.data import astronaut 13 | from skimage.util import img_as_float 14 | import maxflow 15 | from scipy.spatial import Delaunay 16 | import sys 17 | 18 | def help_message(): 19 | print("Usage: [Input_Image] [Input_Marking] [Output_Directory]") 20 | print("[Input_Image]") 21 | print("Path to the input image") 22 | print("[Input_Marking]") 23 | print("Path to the input marking") 24 | print("[Output_Directory]") 25 | print("Output directory") 26 | print("Example usages:") 27 | print(sys.argv[0] + " astronaut.png " + "astronaut_marking.png " + "./") 28 | 29 | # Calculate the SLIC superpixels, their histograms and neighbors 30 | def superpixels_histograms_neighbors(img): 31 | # SLIC 32 | # #TUNE THIS 33 | segments = slic(img, n_segments=500, compactness=18.4534534535) 34 | segments_ids = np.unique(segments) 35 | 36 | # centers 37 | centers = np.array([np.mean(np.nonzero(segments==i),axis=1) for i in segments_ids]) 38 | 39 | # H-S histograms for all superpixels 40 | hsv = cv2.cvtColor(img.astype('float32'), cv2.COLOR_BGR2HSV) 41 | bins = [20, 20] # H = S = 20 42 | ranges = [0, 360, 0, 1] # H: [0, 360], S: [0, 1] 43 | colors_hists = np.float32([cv2.calcHist([hsv],[0, 1], np.uint8(segments==i), bins, ranges).flatten() for i in segments_ids]) 44 | 45 | # neighbors via Delaunay tesselation 46 | tri = Delaunay(centers) 47 | 48 | return (centers,colors_hists,segments,tri.vertex_neighbor_vertices) 49 | 50 | # Get superpixels IDs for FG and BG from marking 51 | def find_superpixels_under_marking(marking, superpixels): 52 | fg_segments = np.unique(superpixels[marking[:,:,0]!=255]) 53 | bg_segments = np.unique(superpixels[marking[:,:,2]!=255]) 54 | return (fg_segments, bg_segments) 55 | 56 | # Sum up the histograms for a given selection of superpixel IDs, normalize 57 | def cumulative_histogram_for_superpixels(ids, histograms): 58 | h = np.sum(histograms[ids],axis=0) 59 | return h / h.sum() 60 | 61 | # Get a bool mask of the pixels for a given selection of superpixel IDs 62 | def pixels_for_segment_selection(superpixels_labels, selection): 63 | pixels_mask = np.where(np.isin(superpixels_labels, selection), True, False) 64 | return pixels_mask 65 | 66 | # Get a normalized version of the given histograms (divide by sum) 67 | def normalize_histograms(histograms): 68 | return np.float32([h / h.sum() for h in histograms]) 69 | 70 | # Perform graph cut using superpixels histograms 71 | def do_graph_cut(fgbg_hists, fgbg_superpixels, norm_hists, neighbors): 72 | num_nodes = norm_hists.shape[0] 73 | # Create a graph of N nodes, and estimate of 5 edges per node 74 | g = maxflow.Graph[float](num_nodes, num_nodes * 5) 75 | # Add N nodes 76 | nodes = g.add_nodes(num_nodes) 77 | 78 | hist_comp_alg = cv2.HISTCMP_KL_DIV 79 | 80 | # Smoothness term: cost between neighbors 81 | indptr,indices = neighbors 82 | for i in range(len(indptr)-1): 83 | N = indices[indptr[i]:indptr[i+1]] # list of neighbor superpixels 84 | hi = norm_hists[i] # histogram for center 85 | for n in N: 86 | if (n < 0) or (n > num_nodes): 87 | continue 88 | # Create two edges (forwards and backwards) with capacities based on 89 | # histogram matching 90 | hn = norm_hists[n] # histogram for neighbor 91 | g.add_edge(nodes[i], nodes[n], 20-cv2.compareHist(hi, hn, hist_comp_alg), 92 | 20-cv2.compareHist(hn, hi, hist_comp_alg)) 93 | 94 | # Match term: cost to FG/BG 95 | for i,h in enumerate(norm_hists): 96 | if i in fgbg_superpixels[0]: 97 | g.add_tedge(nodes[i], 0, 1000) # FG - set high cost to BG 98 | elif i in fgbg_superpixels[1]: 99 | g.add_tedge(nodes[i], 1000, 0) # BG - set high cost to FG 100 | else: 101 | g.add_tedge(nodes[i], cv2.compareHist(fgbg_hists[0], h, hist_comp_alg), 102 | cv2.compareHist(fgbg_hists[1], h, hist_comp_alg)) 103 | 104 | g.maxflow() 105 | return g.get_grid_segments(nodes) 106 | 107 | def RMSD(target, master): 108 | # Note: use grayscale images only 109 | 110 | # Get width, height, and number of channels of the master image 111 | master_height, master_width = master.shape[:2] 112 | master_channel = len(master.shape) 113 | 114 | # Get width, height, and number of channels of the target image 115 | target_height, target_width = target.shape[:2] 116 | target_channel = len(target.shape) 117 | 118 | # Validate the height, width and channels of the input image 119 | if (master_height != target_height or master_width != target_width or master_channel != target_channel): 120 | return -1 121 | else: 122 | 123 | total_diff = 0.0; 124 | dst = cv2.absdiff(master, target) 125 | dst = cv2.pow(dst, 2) 126 | mean = cv2.mean(dst) 127 | total_diff = mean[0]**(1/2.0) 128 | 129 | return total_diff; 130 | 131 | if __name__ == '__main__': 132 | 133 | # validate the input arguments 134 | if (len(sys.argv) != 4): 135 | help_message() 136 | sys.exit() 137 | 138 | img = cv2.imread(sys.argv[1], cv2.IMREAD_COLOR) 139 | img_marking = cv2.imread(sys.argv[2], cv2.IMREAD_COLOR) 140 | 141 | # ======================================== # 142 | # write all your codes here 143 | 144 | # mask = cv2.cvtcolor(img_marking, cv2.color_bgr2gray) # dummy assignment for mask, change it to your result 145 | 146 | centers, colors_hists, superpixels, neighbors = superpixels_histograms_neighbors(img) 147 | # cv2.imshow('mask', mark_boundaries(img,superpixels)) 148 | # cv2.waitKey(0) 149 | 150 | fg_segments, bg_segments = find_superpixels_under_marking(img_marking, superpixels) 151 | 152 | fgbg_hists = list() 153 | fgbg_hists.append(cumulative_histogram_for_superpixels(fg_segments,colors_hists)) 154 | fgbg_hists.append(cumulative_histogram_for_superpixels(bg_segments,colors_hists)) 155 | 156 | fgbg_superpixels = list() 157 | fgbg_superpixels.append(fg_segments) 158 | fgbg_superpixels.append(bg_segments) 159 | 160 | norm_hists = normalize_histograms(colors_hists) 161 | 162 | graph_cut = do_graph_cut(fgbg_hists, fgbg_superpixels, norm_hists, neighbors) 163 | 164 | # print(graph_cut) 165 | # print(graph_cut.shape) 166 | selection=np.where(graph_cut) 167 | # print(selection) 168 | bool_mask=pixels_for_segment_selection(superpixels,selection) 169 | # print(bool_mask.shape) 170 | 171 | mask = np.zeros((img.shape[0],img.shape[1]),dtype=np.uint8) 172 | mask[np.where(bool_mask)] = 255 173 | 174 | # cv2.imshow('maskOut',mask) 175 | # cv2.waitKey(0) 176 | 177 | master = cv2.imread('example_output.png',cv2.IMREAD_GRAYSCALE) 178 | # print(master.shape) 179 | # print(mask.shape) 180 | print(RMSD(mask,master)) 181 | 182 | # ======================================== # 183 | 184 | # read video file 185 | output_name = sys.argv[3] + "mask.png" 186 | cv2.imwrite(output_name, mask); -------------------------------------------------------------------------------- /HW4-Segmentation/main_bonus.py: -------------------------------------------------------------------------------- 1 | # ================================================ 2 | # Skeleton codes for HW4 3 | # Read the skeleton codes carefully and put all your 4 | # codes into main function 5 | # ================================================ 6 | 7 | import cv2 8 | import numpy as np 9 | import matplotlib.pyplot as plt 10 | from skimage.segmentation import slic 11 | from skimage.segmentation import mark_boundaries 12 | from skimage.data import astronaut 13 | from skimage.util import img_as_float 14 | import maxflow 15 | from scipy.spatial import Delaunay 16 | import sys 17 | 18 | drawing = False # true if mouse is pressed 19 | mode = True # if True, draw rectangle. Press 'm' to toggle to curve 20 | ix,iy = -1,-1 21 | mouseCall=True 22 | 23 | def help_message(): 24 | print("Usage: [Input_Image] [Input_Marking] [Output_Directory]") 25 | print("[Input_Image]") 26 | print("Path to the input image") 27 | print("[Input_Marking]") 28 | print("Path to the input marking") 29 | print("[Output_Directory]") 30 | print("Output directory") 31 | print("Example usages:") 32 | print(sys.argv[0] + " astronaut.png " + "astronaut_marking.png " + "./") 33 | # mouse callback function 34 | def draw_circle(event,x,y,flags,param): 35 | global ix,iy,drawing,mode,mouseCall 36 | # print(mouseCall) 37 | if mouseCall == True: 38 | if event == cv2.EVENT_LBUTTONDOWN: 39 | drawing = True 40 | ix,iy = x,y 41 | 42 | elif event == cv2.EVENT_MOUSEMOVE: 43 | if drawing == True: 44 | if mode == True: 45 | cv2.circle(imgdraw, (x, y), 5, (0, 0, 255), -1) 46 | cv2.circle(imgMask, (x, y), 5, (0, 0, 255), -1) 47 | else: 48 | cv2.circle(imgdraw,(x,y), 5,(255,0,0),-1) 49 | cv2.circle(imgMask,(x,y), 5,(255,0,0),-1) 50 | 51 | elif event == cv2.EVENT_LBUTTONUP: 52 | drawing = False 53 | if mode == True: 54 | cv2.circle(imgdraw, (x, y), 5, (0, 0, 255), -1) 55 | cv2.circle(imgMask, (x, y), 5, (0, 0, 255), -1) 56 | else: 57 | cv2.circle(imgdraw,(x,y), 7,(255,0,0),-1) 58 | cv2.circle(imgMask,(x,y), 7,(255,0,0),-1) 59 | 60 | # Calculate the SLIC superpixels, their histograms and neighbors 61 | def superpixels_histograms_neighbors(img): 62 | # SLIC 63 | # #TUNE THIS 64 | segments = slic(img, n_segments=500, compactness=18.4534534535) 65 | segments_ids = np.unique(segments) 66 | 67 | # centers 68 | centers = np.array([np.mean(np.nonzero(segments==i),axis=1) for i in segments_ids]) 69 | 70 | # H-S histograms for all superpixels 71 | hsv = cv2.cvtColor(img.astype('float32'), cv2.COLOR_BGR2HSV) 72 | bins = [20, 20] # H = S = 20 73 | ranges = [0, 360, 0, 1] # H: [0, 360], S: [0, 1] 74 | colors_hists = np.float32([cv2.calcHist([hsv],[0, 1], np.uint8(segments==i), bins, ranges).flatten() for i in segments_ids]) 75 | 76 | # neighbors via Delaunay tesselation 77 | tri = Delaunay(centers) 78 | 79 | return (centers,colors_hists,segments,tri.vertex_neighbor_vertices) 80 | 81 | # Get superpixels IDs for FG and BG from marking 82 | def find_superpixels_under_marking(marking, superpixels): 83 | fg_segments = np.unique(superpixels[marking[:,:,0]!=255]) 84 | bg_segments = np.unique(superpixels[marking[:,:,2]!=255]) 85 | return (fg_segments, bg_segments) 86 | 87 | # Sum up the histograms for a given selection of superpixel IDs, normalize 88 | def cumulative_histogram_for_superpixels(ids, histograms): 89 | h = np.sum(histograms[ids],axis=0) 90 | return h / h.sum() 91 | 92 | # Get a bool mask of the pixels for a given selection of superpixel IDs 93 | def pixels_for_segment_selection(superpixels_labels, selection): 94 | pixels_mask = np.where(np.isin(superpixels_labels, selection), True, False) 95 | return pixels_mask 96 | 97 | # Get a normalized version of the given histograms (divide by sum) 98 | def normalize_histograms(histograms): 99 | return np.float32([h / h.sum() for h in histograms]) 100 | 101 | # Perform graph cut using superpixels histograms 102 | def do_graph_cut(fgbg_hists, fgbg_superpixels, norm_hists, neighbors): 103 | num_nodes = norm_hists.shape[0] 104 | # Create a graph of N nodes, and estimate of 5 edges per node 105 | g = maxflow.Graph[float](num_nodes, num_nodes * 5) 106 | # Add N nodes 107 | nodes = g.add_nodes(num_nodes) 108 | 109 | hist_comp_alg = cv2.HISTCMP_KL_DIV 110 | 111 | # Smoothness term: cost between neighbors 112 | indptr,indices = neighbors 113 | for i in range(len(indptr)-1): 114 | N = indices[indptr[i]:indptr[i+1]] # list of neighbor superpixels 115 | hi = norm_hists[i] # histogram for center 116 | for n in N: 117 | if (n < 0) or (n > num_nodes): 118 | continue 119 | # Create two edges (forwards and backwards) with capacities based on 120 | # histogram matching 121 | hn = norm_hists[n] # histogram for neighbor 122 | g.add_edge(nodes[i], nodes[n], 20-cv2.compareHist(hi, hn, hist_comp_alg), 123 | 20-cv2.compareHist(hn, hi, hist_comp_alg)) 124 | 125 | # Match term: cost to FG/BG 126 | for i,h in enumerate(norm_hists): 127 | if i in fgbg_superpixels[0]: 128 | g.add_tedge(nodes[i], 0, 1000) # FG - set high cost to BG 129 | elif i in fgbg_superpixels[1]: 130 | g.add_tedge(nodes[i], 1000, 0) # BG - set high cost to FG 131 | else: 132 | g.add_tedge(nodes[i], cv2.compareHist(fgbg_hists[0], h, hist_comp_alg), 133 | cv2.compareHist(fgbg_hists[1], h, hist_comp_alg)) 134 | 135 | g.maxflow() 136 | return g.get_grid_segments(nodes) 137 | 138 | def RMSD(target, master): 139 | # Note: use grayscale images only 140 | 141 | # Get width, height, and number of channels of the master image 142 | master_height, master_width = master.shape[:2] 143 | master_channel = len(master.shape) 144 | 145 | # Get width, height, and number of channels of the target image 146 | target_height, target_width = target.shape[:2] 147 | target_channel = len(target.shape) 148 | 149 | # Validate the height, width and channels of the input image 150 | if (master_height != target_height or master_width != target_width or master_channel != target_channel): 151 | return -1 152 | else: 153 | 154 | total_diff = 0.0; 155 | dst = cv2.absdiff(master, target) 156 | dst = cv2.pow(dst, 2) 157 | mean = cv2.mean(dst) 158 | total_diff = mean[0]**(1/2.0) 159 | 160 | return total_diff; 161 | 162 | if __name__ == '__main__': 163 | 164 | # validate the input arguments 165 | if (len(sys.argv) != 2): 166 | help_message() 167 | sys.exit() 168 | 169 | img = cv2.imread(sys.argv[1], cv2.IMREAD_COLOR) 170 | imgdraw = img.copy() 171 | # img_marking = cv2.imread(sys.argv[2], cv2.IMREAD_COLOR) 172 | imgMask = np.ones(img.shape, np.uint8)*255 173 | 174 | #drawing window and callback 175 | cv2.namedWindow('image') 176 | cv2.setMouseCallback('image', draw_circle) 177 | 178 | ##use c to change color for background and foreground marking and then press escape for segmenting 179 | while 1: 180 | cv2.imshow('image', imgdraw) 181 | k = cv2.waitKey(1) & 0xFF 182 | if k == ord('c') or k == ord('C'): 183 | mode = not mode 184 | elif k == 27: 185 | # cv2.imwrite('drawing.png', imgMask) 186 | # cv2.setMouseCallback('image', None) 187 | mouseCall= False 188 | break 189 | 190 | img_marking = imgMask 191 | cv2.destroyAllWindows() 192 | 193 | 194 | centers, colors_hists, superpixels, neighbors = superpixels_histograms_neighbors(img) 195 | # cv2.imshow('mask', mark_boundaries(img,superpixels)) 196 | # cv2.waitKey(0) 197 | 198 | #exit if no fg and bg is present 199 | if ( np.where(img_marking[:,:,0]!=255)[0].size==0 or np.where(img_marking[:,:,2]!=255)[0].size==0): 200 | exit(1) 201 | 202 | fg_segments, bg_segments = find_superpixels_under_marking(img_marking, superpixels) 203 | 204 | fgbg_hists = list() 205 | fgbg_hists.append(cumulative_histogram_for_superpixels(fg_segments,colors_hists)) 206 | fgbg_hists.append(cumulative_histogram_for_superpixels(bg_segments,colors_hists)) 207 | 208 | fgbg_superpixels = list() 209 | fgbg_superpixels.append(fg_segments) 210 | fgbg_superpixels.append(bg_segments) 211 | 212 | norm_hists = normalize_histograms(colors_hists) 213 | 214 | graph_cut = do_graph_cut(fgbg_hists, fgbg_superpixels, norm_hists, neighbors) 215 | 216 | # print(graph_cut) 217 | # print(graph_cut.shape) 218 | selection=np.where(graph_cut) 219 | # print(selection) 220 | bool_mask=pixels_for_segment_selection(superpixels,selection) 221 | # print(bool_mask.shape) 222 | 223 | mask = np.zeros((img.shape[0],img.shape[1]),dtype=np.uint8) 224 | mask[np.where(bool_mask)] = 255 225 | 226 | cv2.imshow('maskOut',mask) 227 | cv2.waitKey(0) 228 | 229 | # master = cv2.imread('example_output.png',cv2.IMREAD_GRAYSCALE) 230 | # print(master.shape) 231 | # print(mask.shape) 232 | # print(RMSD(mask,master)) 233 | 234 | # ======================================== # 235 | 236 | # read video file 237 | # output_name = sys.argv[3] + "mask_bonus.png" 238 | # cv2.imwrite(output_name, mask); -------------------------------------------------------------------------------- /HW5-StructredLight/correspondence.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/correspondence.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images/pattern000.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images/pattern000.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images/pattern001.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images/pattern001.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images/pattern002.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images/pattern002.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images/pattern003.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images/pattern003.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images/pattern004.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images/pattern004.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images/pattern005.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images/pattern005.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images/pattern006.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images/pattern006.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images/pattern007.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images/pattern007.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images/pattern008.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images/pattern008.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images/pattern009.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images/pattern009.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images/pattern010.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images/pattern010.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images/pattern011.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images/pattern011.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images/pattern012.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images/pattern012.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images/pattern013.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images/pattern013.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images/pattern014.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images/pattern014.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images/pattern015.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images/pattern015.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images/pattern016.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images/pattern016.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images_new/aligned000.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images_new/aligned000.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images_new/aligned001.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images_new/aligned001.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images_new/aligned002.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images_new/aligned002.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images_new/aligned003.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images_new/aligned003.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images_new/aligned004.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images_new/aligned004.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images_new/aligned005.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images_new/aligned005.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images_new/aligned006.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images_new/aligned006.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images_new/aligned007.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images_new/aligned007.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images_new/aligned008.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images_new/aligned008.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images_new/aligned009.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images_new/aligned009.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images_new/aligned010.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images_new/aligned010.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images_new/aligned011.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images_new/aligned011.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images_new/aligned012.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images_new/aligned012.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images_new/aligned013.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images_new/aligned013.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images_new/aligned014.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images_new/aligned014.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images_new/aligned015.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images_new/aligned015.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/images_new/aligned016.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dibya-pati/ComputerVision/8d15adc94425ea0f864ba4335481fef2eaaa450f/HW5-StructredLight/images_new/aligned016.jpg -------------------------------------------------------------------------------- /HW5-StructredLight/reconstruct.py: -------------------------------------------------------------------------------- 1 | # ================================================ 2 | # Skeleton codes for HW5 3 | # Read the skeleton codes carefully and put all your 4 | # codes into function "reconstruct_from_binary_patterns" 5 | # ================================================ 6 | 7 | import cv2 8 | import numpy as np 9 | from math import log, ceil, floor 10 | import matplotlib.pyplot as plt 11 | import pickle 12 | import sys 13 | 14 | def help_message(): 15 | # Note: it is assumed that "binary_codes_ids_codebook.pckl", "stereo_calibration.pckl", 16 | # and images folder are in the same root folder as your "generate_data.py" source file. 17 | # Same folder structure will be used when we test your program 18 | 19 | print("Usage: [Output_Directory]") 20 | print("[Output_Directory]") 21 | print("Where to put your output.xyz") 22 | print("Example usages:") 23 | print(sys.argv[0] + " ./") 24 | 25 | def reconstruct_from_binary_patterns(): 26 | scale_factor = 1.0 27 | # ref_white = cv2.resize(cv2.imread("images_new/aligned000.jpg", cv2.IMREAD_GRAYSCALE) / 255.0, (0,0), fx=scale_factor,fy=scale_factor) 28 | # ref_black = cv2.resize(cv2.imread("images_new/aligned001.jpg", cv2.IMREAD_GRAYSCALE) / 255.0, (0,0), fx=scale_factor,fy=scale_factor) 29 | # ref_color = cv2.resize(cv2.imread("images_new/aligned001.jpg", cv2.IMREAD_COLOR), (0,0), fx=scale_factor,fy=scale_factor) 30 | 31 | ref_white = cv2.resize(cv2.imread("images/pattern000.jpg", cv2.IMREAD_GRAYSCALE) / 255.0, (0,0), fx=scale_factor,fy=scale_factor) 32 | ref_color = cv2.resize(cv2.imread("images/pattern001.jpg", cv2.IMREAD_COLOR), (0,0), fx=scale_factor,fy=scale_factor) 33 | ref_black = cv2.resize(cv2.imread("images/pattern001.jpg", cv2.IMREAD_GRAYSCALE) / 255.0, (0,0), fx=scale_factor,fy=scale_factor) 34 | ref_avg = (ref_white + ref_black) / 2.0 35 | ref_on = ref_avg + 0.05 # a threshold for ON pixels 36 | ref_off = ref_avg - 0.05 # add a small buffer region 37 | 38 | h,w = ref_white.shape 39 | 40 | # mask of pixels where there is projection 41 | proj_mask = (ref_white > (ref_black + 0.05)) 42 | 43 | scan_bits = np.zeros((h,w), dtype=np.uint16) 44 | 45 | # analyze the binary patterns from the camera 46 | for i in range(0, 15): 47 | # read the file 48 | patt_gray = cv2.resize(cv2.imread("images/pattern%03d.jpg"%(i+2),cv2.IMREAD_GRAYSCALE) / 255.0, (0,0), fx=scale_factor,fy=scale_factor) 49 | # patt_gray = cv2.resize(cv2.imread("images_new/aligned%03d.jpg"%(i+2), cv2.IMREAD_GRAYSCALE) / 255.0, (0,0), fx=scale_factor,fy=scale_factor) 50 | 51 | # mask where the pixels are ON 52 | on_mask = (patt_gray > ref_on) & proj_mask 53 | 54 | # this code corresponds with the binary pattern code 55 | bit_code = np.uint16(1 << i) 56 | # populate scan_bits by putting the bit_code according to on_mask 57 | scan_bits[np.where(on_mask == True)] = scan_bits[np.where(on_mask == True)]+bit_code 58 | # print(np.count_nonzero(np.unique(scan_bits))) 59 | 60 | print("load codebook") 61 | # the codebook translates from to (x,y) in projector screen space 62 | with open("binary_codes_ids_codebook.pckl","r") as f: 63 | binary_codes_ids_codebook = pickle.load(f) 64 | 65 | camera_points = [] 66 | projector_points = [] 67 | camera_pointsC = [] 68 | color_points=[] 69 | # print(ref_color.shape) 70 | 71 | # print(np.count_nonzero(np.unique(np.asarray(binary_codes_ids_codebook.values())))) 72 | for x in range(w): 73 | for y in range(h): 74 | if not proj_mask[y, x]: 75 | continue # no projection here 76 | if scan_bits[y, x] not in binary_codes_ids_codebook: 77 | continue # bad binary code 78 | 79 | # use binary_codes_ids_codebook[...] and scan_bits[y,x] to 80 | # find for the camera (x,y) the projector (p_x, p_y). 81 | # store your points in camera_points and projector_points 82 | 83 | # IMPORTANT!!! : due to differences in calibration and acquisition - divide the camera points by 2 84 | # print(binary_codes_ids_codebook.__len__()) 85 | # print(scan_bits.max()) 86 | 87 | x_p, y_p = binary_codes_ids_codebook[scan_bits[y, x]] 88 | if x_p >= 1279 or y_p >= 799: # filter 89 | continue 90 | 91 | projector_points.append([x_p, y_p]) 92 | camera_points.append([x/2, y/2]) 93 | camera_pointsC.append([x, y]) 94 | color_points.append(ref_color[y,x]) 95 | 96 | 97 | # print(color_points[0]) 98 | # now that we have 2D-2D correspondances, we can triangulate 3D points! 99 | cimage= np.zeros((h,w,3), dtype=np.uint16) 100 | ppointsArr=np.asarray(projector_points,dtype=np.float32).reshape((projector_points.__len__(),1,2)) 101 | cpointsArr=np.asarray(camera_points,dtype=np.float32).reshape((projector_points.__len__(),1,2)) 102 | color_pointsArr=np.asarray(color_points,dtype=np.float32).reshape((projector_points.__len__(),3)) 103 | 104 | # for i,elm in enumerate(camera_pointsC): 105 | # x, y = elm 106 | # valx, valy = projector_points[i] 107 | # cimage[y,x,:]= valy*255/(valx+valy),valx*255/(valx+valy),0 108 | # plt.imshow(cimage) 109 | # plt.show() 110 | 111 | # load the prepared stereo calibration between projector and camera 112 | with open("stereo_calibration.pckl","r") as f: 113 | d = pickle.load(f) 114 | camera_K = d['camera_K'] 115 | camera_d = d['camera_d'] 116 | projector_K = d['projector_K'] 117 | projector_d = d['projector_d'] 118 | projector_R = d['projector_R'] 119 | projector_t = d['projector_t'] 120 | 121 | # use cv2.undistortPoints to get normalized points for camera, use camera_K and camera_d 122 | # use cv2.undistortPoints to get normalized points for projector, use projector_K and projector_d 123 | 124 | camera_normalized = cv2.undistortPoints(cpointsArr, camera_K, camera_d,P=camera_K) 125 | projector_normalized = cv2.undistortPoints(ppointsArr, projector_K, projector_d,P=projector_K) 126 | 127 | # print(projector_R) 128 | # print(projector_t) 129 | # print(projector_K) 130 | # print(camera_K) 131 | # print(projector_d) 132 | # print(camera_normalized[:10]) 133 | # pProjMat = np.dot(projector_K,np.hstack((projector_R,projector_t))) 134 | pProjMat = np.dot(np.concatenate((projector_K.T,[[0,0,0]]),axis=0).T, 135 | np.concatenate((np.concatenate((projector_R,projector_t),axis=1),[[0,0,0,1]]),axis=0)) 136 | 137 | # print(pProjMat) 138 | # cProjMat = np.dot(camera_K, [[1,0,0,0],[0,1,0,0],[0,0,1,0]]) 139 | cProjMat = np.dot(np.concatenate((camera_K.T,[[0,0,0]]),axis=0).T, 140 | np.eye(4,dtype=np.float32)) 141 | # print(cProjMat) 142 | 143 | # use cv2.triangulatePoints to triangulate the normalized points 144 | triangulatedPoints=cv2.triangulatePoints(pProjMat,cProjMat,projector_normalized,camera_normalized) 145 | # use cv2.convertPointsFromHomogeneous to get real 3D points 146 | # print(triangulatedPoints.shape) 147 | # print(camera_normalized.shape) 148 | points_3d_u = cv2.convertPointsFromHomogeneous(triangulatedPoints.T) 149 | # name the resulted 3D points as "points_3d" 150 | mask = (points_3d_u[:, :, 2] > 200) & (points_3d_u[:, :, 2] < 1400) 151 | points_3d_wc = points_3d_u[np.where(mask[:, 0])] 152 | color_points=color_pointsArr[np.where(mask[:, 0])] 153 | # print(points_3d_wc.shape,color_points.shape) 154 | 155 | shape=points_3d_wc.shape 156 | points_3d = np.hstack((points_3d_wc.reshape(shape[0],-1),color_points[:,::-1])).reshape(shape[0],1,6) 157 | return points_3d 158 | 159 | def write_3d_points(points_3d): 160 | 161 | # ===== DO NOT CHANGE THIS FUNCTION ===== 162 | 163 | print("write output point cloud") 164 | print(points_3d.shape) 165 | output_name = sys.argv[1] + "output.xyz" 166 | with open(output_name, "w") as f: 167 | for p in points_3d: 168 | f.write("%d %d %d\n"%(p[0,0],p[0,1],p[0,2])) 169 | # return points_3d, camera_points, projector_points 170 | return points_3d 171 | 172 | def write_3d_points_withColor(points_3d): 173 | 174 | # ===== DO NOT CHANGE THIS FUNCTION ===== 175 | 176 | print("write output point cloud") 177 | print(points_3d.shape) 178 | output_name = sys.argv[1] + "output_color.xyz" 179 | with open(output_name, "w") as f: 180 | for p in points_3d: 181 | f.write("%d %d %d %d %d %d\n" % (p[0, 0], p[0, 1], p[0, 2],p[0, 3], p[0, 4], p[0, 5])) 182 | # return points_3d, camera_points, projector_points 183 | return points_3d 184 | 185 | if __name__ == '__main__': 186 | 187 | 188 | # ===== DO NOT CHANGE THIS FUNCTION ===== 189 | # validate the input arguments 190 | if (len(sys.argv) != 2): 191 | help_message() 192 | sys.exit() 193 | 194 | points_3d = reconstruct_from_binary_patterns() 195 | write_3d_points(points_3d) 196 | write_3d_points_withColor(points_3d) 197 | -------------------------------------------------------------------------------- /HW5-StructredLight/stereo_calibration.pckl: -------------------------------------------------------------------------------- 1 | (dp0 2 | S'camera_K' 3 | p1 4 | cnumpy.core.multiarray 5 | _reconstruct 6 | p2 7 | (cnumpy 8 | ndarray 9 | p3 10 | (I0 11 | tp4 12 | S'b' 13 | p5 14 | tp6 15 | Rp7 16 | (I1 17 | (I3 18 | I3 19 | tp8 20 | cnumpy 21 | dtype 22 | p9 23 | (S'f8' 24 | p10 25 | I0 26 | I1 27 | tp11 28 | Rp12 29 | (I3 30 | S'<' 31 | p13 32 | NNNI-1 33 | I-1 34 | I0 35 | tp14 36 | bI00 37 | S'|D!W\n\x88\x8a@\x00\x00\x00\x00\x00\x00\x00\x00\xb6\nG1\x19;v@\x00\x00\x00\x00\x00\x00\x00\x00s\xcc>\xa0\xfe\x00\x89@\xa1m)=ifb@\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xf0?' 38 | p15 39 | tp16 40 | bsS'projector_R' 41 | p17 42 | g2 43 | (g3 44 | (I0 45 | tp18 46 | g5 47 | tp19 48 | Rp20 49 | (I1 50 | (I3 51 | I3 52 | tp21 53 | g12 54 | I00 55 | S'\xe2j\x9cs\xcf\xef\xef\xbf#N\x95l\xc3\xa0z\xbf\x92\xbf\x06\x97\x11\x00\xb0\xbf\xfe\x9c|\x82\x93\xff`?\x1bj\x86x\x87\xeb\xef\xbf\xc6Ua\xcf)\x14\xb2?\xee\xd5l\xb2\xeb\x13\xb0\xbf<6\xd1y\x84\x02\xb2?\\A\x90\x91}\xdb\xef?' 56 | p22 57 | tp23 58 | bsS'camera_d' 59 | p24 60 | g2 61 | (g3 62 | (I0 63 | tp25 64 | g5 65 | tp26 66 | Rp27 67 | (I1 68 | (I5 69 | I1 70 | tp28 71 | g12 72 | I00 73 | S'\xc3\x8dt\x95\x1e8\xee?\x1f\xf7.\xfb)e\x19\xc0x.\xa5K\xc4\xf5\x99\xbfB\x90\xcb6M\x89\xb2?\x9fO8]\xea03@' 74 | p29 75 | tp30 76 | bsS'projector_t' 77 | p31 78 | g2 79 | (g3 80 | (I0 81 | tp32 82 | g5 83 | tp33 84 | Rp34 85 | (I1 86 | (I3 87 | I1 88 | tp35 89 | g12 90 | I00 91 | S'\x95y\xc1\xa6\xfc\xd90\xc0t\x7f\xc6\xae\x9d\x08R@d\x91\x14m\xe3\x8a\x15@' 92 | p36 93 | tp37 94 | bsS'projector_K' 95 | p38 96 | g2 97 | (g3 98 | (I0 99 | tp39 100 | g5 101 | tp40 102 | Rp41 103 | (I1 104 | (I3 105 | I3 106 | tp42 107 | g12 108 | I00 109 | S'\xd6,4\xac.\x81\xa7@\x00\x00\x00\x00\x00\x00\x00\x00\x8e\xb7&"\xe2k\x80@\x00\x00\x00\x00\x00\x00\x00\x00\x14\xb5\xd2\xebr\xd6\xa4@B4N\xfb\x07\xb5\x80@\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xf0?' 110 | p43 111 | tp44 112 | bsS'projector_d' 113 | p45 114 | g2 115 | (g3 116 | (I0 117 | tp46 118 | g5 119 | tp47 120 | Rp48 121 | (I1 122 | (I5 123 | I1 124 | tp49 125 | g12 126 | I00 127 | S'\xf53@\xb0\x91A\xe2?u\xcc\x19\rz\xbaA\xc0\x1cF\xaf\xda\x9f\x1b\xad\xbf\xb0\x16\xd9>\xab]\x96\xbf%P\x8f.>\x1a\x80@' 128 | p50 129 | tp51 130 | bs. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Computer Vision 2 | Segmentation,Object Detection and many more 3 | 4 | This repo comprises of Python Open CV implementation of : 5 | 1. HW1: 6 | 7 | 1. Histogram equalization/Contrast Enhancement:This is based on Prince's approach of linearizing the channels by multiplying 8 | it with color depth i.e. 255 for 8bit, and dividing it with the number of pixels, more precisely eq 13.3 and 13.4 from 13.1.2 Histogram equalization of the book Computer vision where I and J are number of pixels in x and y respectively and then maping each pixel to its percentile value by, where K is the maximum intensity(if 8bit its 255 and so on) 9 | 2. Filter: Implemetation of Low pass and High Pass filter using Fast Fourier Transform(FFT), Convolution and again Inverse FFT 10 | 3. Deblurring and dividing gaussian : this involves using deconvolution i.e. dividing of image with the original gaussian kernel used for blurring. 11 | 4. Laplacian Pyramid blending: Perform blending of two images by creating pyramids and adding the LP and HP components at each level and traversing up the pyramid. 12 | 2. HW2: 13 | This exercise involves image stitching after having done affine(after converting to cylindrical coordinates) and homography transformations.It uses Lunchroom image : PASSTA Dataset 14 | 3. HW3: 15 | This execrcise implements object tracking and optical flow in OpenCV. 16 | 1. Haar cascades:Voila-Jones detectors 17 | 2. Camshift detector(modified mean shift) 18 | 3. Haar features with Kalman corrections 19 | 4. particle filters(from Lucas and Kanade) 20 | 5. Optical Flow(using Shi and Tomasi ) 21 | 4. HW4: 22 | This execrcise primarily concentrates on semi-supervised image segmentation on astronaut image. 23 | 1. It first uses SLIC(Simple Linear Iterative Clustering) to form superpixels, the segmentation(n) and compactness parameter of SLIC can be adjusted to get the desired segment size and shape. 24 | 2. Read the image markings from the file where blue marks the foreround and red the background, these pixel markings are used to creat the source and sink nodes for the graph-cut to separate the image into two classes (the foreground and background). 25 | 3. Use Energy based image segmentation by comparing the image histograms with the intial set of background and foreground histograms(from markings) 26 | 4. Perform normalized graph cut to the pixels to background(bg) and foreground(fg) 27 | 5. In the extra section the markings are read from a gui, where user can interactively draws the bg and fg markings, use 'c' or 'C' to toggle between color of marking and once done press 'ESC' to compute the segmented mask 28 | 5. HW5: 29 | In this section we experimented with 3-d reconstruction using structured light. The test comprises of projecting a series of harizontal and vertical strips of light on the object & then reading the pattern using a standard camera. Here, we consider a series of images for 3-d recontruction 30 | 1. Read the images and populate the scan value for each pixel(read a set a horizonal then vertical strips that form LSB and MSB of the binary code) 31 | 2. Use the lookup table to find the (x,y) position from the projector 32 | 3. Compute the undistorted values of (x,y) using the given K-matrix and distortion matrix of the projector and camera respectively. For Camera Calliberation please follow the steps from [Prof Roy's Blog ](http://www.morethantechnical.com/2017/11/17/projector-camera-calibration-the-easy-way/) 33 | 4. We have two sets (x,y) for a given pixel One from the projector and one from the camera's peprspective 34 | 5. Use OpenCV triangulation to determine 3-d coordinate(homogenous space) for the a given (x,y) 35 | 6. Compute the actual 3-d point from the homogenous matrix using OpenCV convert from homogenous api 36 | 7. Result 3-d location for a 2-d point 37 | 38 | --------------------------------------------------------------------------------