├── src
├── vgg16
│ ├── index.h5
│ ├── index.csv
│ ├── __pycache__
│ │ └── vgg16.cpython-36.pyc
│ ├── vgg16.py
│ └── index.py
├── hog
│ ├── __pycache__
│ │ └── hog.cpython-36.pyc
│ ├── index.py
│ └── hog.py
├── __pycache__
│ └── searcher.cpython-36.pyc
├── color
│ ├── __pycache__
│ │ └── color.cpython-36.pyc
│ ├── index.py
│ └── color.py
├── gabor
│ ├── __pycache__
│ │ └── gabor.cpython-36.pyc
│ ├── index.py
│ └── gabor.py
├── searcher.py
└── search.py
├── query_images
├── egypt.jpg
├── Pyramids.jpg
├── baleswar.jpg
├── bangkok.jpg
├── tajmahal.jpg
└── vijayawada.jpg
├── results
├── q1_hog_result.png
├── q1_color_result.png
├── q1_gabor_result.png
└── q1_vgg16_result.png
├── database
└── README.md
└── README.md
/src/vgg16/index.h5:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/vamc-stash/image-retrieval/HEAD/src/vgg16/index.h5
--------------------------------------------------------------------------------
/src/vgg16/index.csv:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/vamc-stash/image-retrieval/HEAD/src/vgg16/index.csv
--------------------------------------------------------------------------------
/query_images/egypt.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/vamc-stash/image-retrieval/HEAD/query_images/egypt.jpg
--------------------------------------------------------------------------------
/query_images/Pyramids.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/vamc-stash/image-retrieval/HEAD/query_images/Pyramids.jpg
--------------------------------------------------------------------------------
/query_images/baleswar.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/vamc-stash/image-retrieval/HEAD/query_images/baleswar.jpg
--------------------------------------------------------------------------------
/query_images/bangkok.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/vamc-stash/image-retrieval/HEAD/query_images/bangkok.jpg
--------------------------------------------------------------------------------
/query_images/tajmahal.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/vamc-stash/image-retrieval/HEAD/query_images/tajmahal.jpg
--------------------------------------------------------------------------------
/results/q1_hog_result.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/vamc-stash/image-retrieval/HEAD/results/q1_hog_result.png
--------------------------------------------------------------------------------
/query_images/vijayawada.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/vamc-stash/image-retrieval/HEAD/query_images/vijayawada.jpg
--------------------------------------------------------------------------------
/results/q1_color_result.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/vamc-stash/image-retrieval/HEAD/results/q1_color_result.png
--------------------------------------------------------------------------------
/results/q1_gabor_result.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/vamc-stash/image-retrieval/HEAD/results/q1_gabor_result.png
--------------------------------------------------------------------------------
/results/q1_vgg16_result.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/vamc-stash/image-retrieval/HEAD/results/q1_vgg16_result.png
--------------------------------------------------------------------------------
/src/hog/__pycache__/hog.cpython-36.pyc:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/vamc-stash/image-retrieval/HEAD/src/hog/__pycache__/hog.cpython-36.pyc
--------------------------------------------------------------------------------
/src/__pycache__/searcher.cpython-36.pyc:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/vamc-stash/image-retrieval/HEAD/src/__pycache__/searcher.cpython-36.pyc
--------------------------------------------------------------------------------
/src/color/__pycache__/color.cpython-36.pyc:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/vamc-stash/image-retrieval/HEAD/src/color/__pycache__/color.cpython-36.pyc
--------------------------------------------------------------------------------
/src/gabor/__pycache__/gabor.cpython-36.pyc:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/vamc-stash/image-retrieval/HEAD/src/gabor/__pycache__/gabor.cpython-36.pyc
--------------------------------------------------------------------------------
/src/vgg16/__pycache__/vgg16.cpython-36.pyc:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/vamc-stash/image-retrieval/HEAD/src/vgg16/__pycache__/vgg16.cpython-36.pyc
--------------------------------------------------------------------------------
/database/README.md:
--------------------------------------------------------------------------------
1 | Download images from the link i provided here and delete this README.md file.
2 | ftp://ftp.inrialpes.fr/pub/lear/douze/data/jpg1.tar.gz
3 |
--------------------------------------------------------------------------------
/src/hog/index.py:
--------------------------------------------------------------------------------
1 | import numpy as np
2 | import argparse
3 | import glob
4 | import cv2
5 |
6 | from hog import HOGDescriptor
7 |
8 | ap = argparse.ArgumentParser()
9 | ap.add_argument("--index", required = True,help = "Name of index file")
10 | args = vars(ap.parse_args())
11 |
12 | hog = HOGDescriptor()
13 |
14 | output = open(args["index"],"w")
15 |
16 | for imagePath in glob.glob("../../"+"database"+"/*.jpg"):
17 | imageId = imagePath[imagePath.rfind("/")+1:]
18 | image = cv2.imread(imagePath)
19 |
20 | features = hog.describe(image)
21 | features = [str(f) for f in features]
22 |
23 | output.write("%s,%s\n" % (imageId,",".join(features)))
24 |
25 | output.close()
--------------------------------------------------------------------------------
/src/gabor/index.py:
--------------------------------------------------------------------------------
1 | import numpy as np
2 | import glob
3 | import cv2
4 | import argparse
5 | from gabor import GaborDescriptor
6 |
7 | ap = argparse.ArgumentParser()
8 | ap.add_argument("-i","--index",required=True,help="Path to where the computed index will be stored")
9 | args = vars(ap.parse_args())
10 |
11 | params = {"theta":4, "frequency":(0,1,0.5,0.8), "sigma":(1,3),"n_slice":2}
12 |
13 | gd = GaborDescriptor(params)
14 |
15 | gaborKernels = gd.kernels()
16 |
17 | output = open(args["index"],"w")
18 |
19 | c = 1
20 | for imagePath in glob.glob("../../"+"database"+"/*.jpg"):
21 | imageId = imagePath[imagePath.rfind("/")+1:]
22 | image = cv2.imread(imagePath)
23 |
24 | features = gd.gaborHistogram(image,gaborKernels)
25 | features = [str(f) for f in features]
26 | print(c)
27 | c += 1
28 | output.write("%s,%s\n" % (imageId,",".join(features)))
29 |
30 | output.close()
--------------------------------------------------------------------------------
/src/color/index.py:
--------------------------------------------------------------------------------
1 | import argparse
2 | import glob
3 | import cv2
4 |
5 | from color import ColorDescriptor
6 |
7 | ap = argparse.ArgumentParser()
8 | ap.add_argument("-i","--index",required=True,help="Path to where the computed index will be stored")
9 | args = vars(ap.parse_args())
10 |
11 | #in this project we use 8 bins for hue channel,12 for saturation, 3 for value channel
12 | bins = (8,12,3)
13 | #initialize object
14 | cd = ColorDescriptor(bins)
15 |
16 | output = open(args["index"],"w")
17 |
18 | for imagePath in glob.glob("../../"+"database"+"/*.jpg"):
19 | imageId = imagePath[imagePath.rfind("/")+1:]
20 | image = cv2.imread(imagePath)
21 |
22 | features = cd.describe(image)
23 |
24 | features = [str(f) for f in features]
25 | #feature vector length 5(segments)*8*12*3 = 1440
26 | output.write("%s,%s\n" % (imageId,",".join(features)))
27 |
28 | output.close()
29 |
30 | #run
31 | #python3 index.py --index index.csv
--------------------------------------------------------------------------------
/src/vgg16/vgg16.py:
--------------------------------------------------------------------------------
1 | import numpy as np
2 | from numpy import linalg as LA
3 |
4 | from keras.applications.vgg16 import VGG16
5 | from keras.preprocessing import image
6 | from keras.applications.vgg16 import preprocess_input
7 |
8 | class VGGNet:
9 | def __init__(self):
10 | self.input_shape = (224, 224, 3)
11 | self.weight = 'imagenet'
12 | self.pooling = 'max'
13 | self.model = VGG16(weights = self.weight, input_shape = (self.input_shape[0], self.input_shape[1], self.input_shape[2]), pooling = self.pooling, include_top = False)
14 | self.model.predict(np.zeros((1, 224, 224 , 3)))
15 |
16 | def extract_feat(self, img_path):
17 | img = image.load_img(img_path, target_size=(self.input_shape[0], self.input_shape[1]))
18 | img = image.img_to_array(img)
19 | img = np.expand_dims(img, axis=0)
20 | img = preprocess_input(img)
21 | feat = self.model.predict(img)
22 | norm_feat = feat[0]/LA.norm(feat[0])
23 | return norm_feat
--------------------------------------------------------------------------------
/src/vgg16/index.py:
--------------------------------------------------------------------------------
1 | import os
2 | import h5py
3 | import numpy as np
4 | import argparse
5 | import glob
6 |
7 | from vgg16 import VGGNet
8 |
9 | ap = argparse.ArgumentParser()
10 | ap.add_argument("--index", required = True,help = "Name of index file")
11 | args = vars(ap.parse_args())
12 |
13 | if __name__ == "__main__":
14 |
15 | feats = []
16 | names = []
17 |
18 | model = VGGNet()
19 | # i=0
20 | for imgPath in glob.glob("../../"+"database"+"/*.jpg"):
21 | imageId = imgPath[imgPath.rfind("/")+1:]
22 | norm_feat = model.extract_feat(imgPath)
23 | feats.append(norm_feat)
24 | names.append(imageId)
25 | # print("extracting feature from image No. %d " %((i+1)))
26 | # i+=1
27 |
28 | feats = np.array(feats)
29 |
30 | output = args["index"]
31 |
32 | h5f = h5py.File(output, 'w')
33 | h5f.create_dataset('dataset_1', data = feats)
34 | # h5f.create_dataset('dataset_2', data = names)
35 | h5f.create_dataset('dataset_2', data = np.string_(names))
36 | h5f.close()
--------------------------------------------------------------------------------
/src/searcher.py:
--------------------------------------------------------------------------------
1 | import numpy as np
2 | import csv
3 |
4 | class Searcher:
5 | def __init__(self,indexPath):
6 | self.indexPath = indexPath
7 |
8 | def search(self, queryFeatures, limit=10):
9 | results = {}
10 |
11 | with open(self.indexPath) as i:
12 | reader = csv.reader(i)
13 |
14 | for row in reader:
15 | features = [float(x) for x in row[1:]]
16 | d = self.chi_squared_distance(features,queryFeatures)
17 | results[row[0]] = d
18 |
19 | i.close()
20 |
21 | results = sorted([(v,k) for (k,v) in results.items()])
22 |
23 | return results[:limit]
24 |
25 | def _gsearch(self, queryFeatures, limit=10):
26 | results = {}
27 | with open(self.indexPath) as i:
28 | reader = csv.reader(i)
29 |
30 | for row in reader:
31 | features = [float(x) for x in row[1:]]
32 | error = np.sum((queryFeatures - features)**2)
33 | results[row[0]] = error
34 |
35 | i.close()
36 |
37 | results = sorted([(v,k) for (k,v) in results.items()])
38 | return results[:limit]
39 |
40 | def chi_squared_distance(self, histA, histB, eps=1e-10):
41 |
42 | d = 0.5*np.sum([((a-b)**2)/(a+b+eps) for (a,b) in zip(histA,histB)])
43 |
44 | return d
--------------------------------------------------------------------------------
/src/hog/hog.py:
--------------------------------------------------------------------------------
1 | from skimage.feature import hog
2 | import numpy as np
3 | import cv2
4 |
5 | class HOGDescriptor:
6 | def __init__(self):
7 | self.n_bins = 10
8 | self.n_slice = 6
9 | self.n_orient = 8
10 | self.pixels_per_cell = (2,2)
11 | self.cells_per_block = (1,1)
12 |
13 | def describe(self,image):
14 | height,width,channel = image.shape
15 |
16 | hist = np.zeros((self.n_slice,self.n_slice,self.n_bins))
17 | h_silce = np.around(np.linspace(0, height, self.n_slice+1, endpoint=True)).astype(int)
18 | w_slice = np.around(np.linspace(0, width, self.n_slice+1, endpoint=True)).astype(int)
19 | for hs in range(len(h_silce)-1):
20 | for ws in range(len(w_slice)-1):
21 | img_r = image[h_silce[hs]:h_silce[hs+1], w_slice[ws]:w_slice[ws+1]] # slice img to regions
22 | hist[hs][ws] = self._HOG(img_r,self.n_bins)
23 |
24 | hist /= np.sum(hist)
25 |
26 | return hist.flatten()
27 |
28 | def _HOG(self,image,n_bins):
29 | gray_img = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
30 | feats = hog(gray_img,orientations=self.n_orient,pixels_per_cell=self.pixels_per_cell,cells_per_block=self.cells_per_block)
31 | bins = np.linspace(0, np.max(feats), n_bins+1, endpoint=True)
32 | hist, b = np.histogram(feats,bins=bins)
33 |
34 | hist = np.array(hist)/np.sum(hist)
35 |
36 | return hist
37 |
38 |
--------------------------------------------------------------------------------
/src/color/color.py:
--------------------------------------------------------------------------------
1 | import numpy as np
2 | import cv2
3 | import imutils
4 |
5 | class ColorDescriptor:
6 | def __init__(self,bins):
7 | self.bins = bins
8 |
9 | def describe(self,image):
10 | image = cv2.cvtColor(image,cv2.COLOR_BGR2HSV)
11 | features = []
12 |
13 | (h,w) = image.shape[:2]
14 | #divide the image into 5 parts(top-left,top-right,bottom-right,bottom-left,center)
15 | (cx,cy) = (int(w*0.5), int(h*0.5))
16 | #4 corner segments
17 | segments = [(0,cx,0,cy),(cx,w,0,cy),(cx,w,cy,h),(0,cx,cy,h)]
18 | #center (ellipse shape)
19 | (ex,ey) = (int(w*0.75)//2, int(h*0.75)//2) #axes length
20 | #elliptical black mask
21 | ellipMask = np.zeros(image.shape[:2],dtype= "uint8")
22 | cv2.ellipse(ellipMask,(cx,cy),(ex,ey),0,0,360,255,-1) # -1 :- fills entire ellipse with 255(white) color
23 |
24 | for (startX, endX, startY, endY) in segments:
25 | # construct a mask for each corner of the image, subtracting the elliptical center from it
26 | cornerMask = np.zeros(image.shape[:2],dtype = "uint8")
27 | cv2.rectangle(cornerMask,(startX,startY),(endX,endY),255,-1)
28 | cornerMask = cv2.subtract(cornerMask,ellipMask)
29 |
30 | hist = self.histogram(image,cornerMask)
31 | features.extend(hist)
32 |
33 | hist = self.histogram(image,ellipMask)
34 | features.extend(hist)
35 |
36 | return features
37 |
38 |
39 | def histogram(self,image,mask):
40 | hist = cv2.calcHist([image],[0,1,2],mask,self.bins,[0,180,0,256,0,256])
41 |
42 | if imutils.is_cv2():
43 | hist = cv2.normalize(hist).flatten() #cv 2.4
44 | else:
45 | hist = cv2.normalize(hist,hist).flatten() #cv 3+
46 |
47 | return hist
--------------------------------------------------------------------------------
/src/gabor/gabor.py:
--------------------------------------------------------------------------------
1 | import numpy as np
2 | from scipy import ndimage as ndi
3 | from skimage import data
4 | from skimage.util import img_as_float
5 | from skimage.filters import gabor_kernel
6 | import cv2
7 |
8 |
9 | class GaborDescriptor:
10 | def __init__(self,params):
11 | self.theta = params['theta']
12 | self.frequency = params['frequency']
13 | self.sigma = params['sigma']
14 | self.n_slice = params['n_slice']
15 |
16 | def kernels(self):
17 | kernels = []
18 | for theta in range(self.theta):
19 | theta = theta/4. * np.pi
20 | for frequency in self.frequency:
21 | for sigma in self.sigma:
22 | kernel = gabor_kernel(frequency,theta=theta,sigma_x=sigma,sigma_y=sigma)
23 | kernels.append(kernel)
24 | return kernels
25 |
26 | def gaborHistogram(self,image,gabor_kernels):
27 | height,width,channel = image.shape
28 | #height & width of image will equally sliced into N slices
29 | hist = np.zeros((self.n_slice,self.n_slice,2*len(gabor_kernels))) #2*len coz to store mean and variance
30 | h_silce = np.around(np.linspace(0, height, self.n_slice+1, endpoint=True)).astype(int)
31 | w_slice = np.around(np.linspace(0, width, self.n_slice+1, endpoint=True)).astype(int)
32 | for hs in range(len(h_silce)-1):
33 | for ws in range(len(w_slice)-1):
34 | img_r = image[h_silce[hs]:h_silce[hs+1], w_slice[ws]:w_slice[ws+1]] # slice img to regions
35 | hist[hs][ws] = self._gabor(img_r,gabor_kernels)
36 |
37 | hist /= np.sum(hist)
38 | #print(hist.shape)
39 | return hist.flatten()
40 |
41 | def _power(self,image,kernel):
42 | image = (image - image.mean()) / image.std()
43 | f_img = np.sqrt(ndi.convolve(image, np.real(kernel), mode='wrap')**2 + ndi.convolve(image, np.imag(kernel), mode='wrap')**2)
44 | feats = np.zeros(2, dtype=np.double)
45 | feats[0] = f_img.mean()
46 | feats[1] = f_img.var()
47 | return feats
48 |
49 | def _gabor(self,image,gabor_kernels):
50 | gray_img = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
51 | results = []
52 | for kernel in gabor_kernels:
53 | results.append(self._power(gray_img,kernel))
54 |
55 | hist = np.array(results)
56 | hist = hist / np.sum(hist, axis=0)
57 | #print(hist.flatten())
58 | #print(hist.T.flatten())
59 |
60 | return hist.T.flatten() # .T -> transpose
61 |
--------------------------------------------------------------------------------
/src/search.py:
--------------------------------------------------------------------------------
1 | import argparse
2 | import cv2
3 | import numpy as np
4 | import h5py
5 |
6 | import matplotlib.pyplot as plt
7 | import matplotlib.image as mpimg
8 |
9 | from color.color import ColorDescriptor
10 | from gabor.gabor import GaborDescriptor
11 | from hog.hog import HOGDescriptor
12 | from vgg16.vgg16 import VGGNet
13 |
14 | from searcher import Searcher
15 |
16 | ap = argparse.ArgumentParser()
17 | ap.add_argument("-q", "--query", required = True,help = "Path to the query image")
18 | ap.add_argument("-c", "--class", required = True,help = "Feature class")
19 | args = vars(ap.parse_args())
20 |
21 | if args["class"] == "color" or args["class"] == "gabor" or args["class"] == "hog":
22 |
23 | if(args["class"] == "color"):
24 | bins = (8,12,3)
25 | cd = ColorDescriptor(bins)
26 |
27 | query = cv2.imread(args["query"])
28 | features = cd.describe(query)
29 |
30 | searcher = Searcher("color/index.csv")
31 | results = searcher.search(features)
32 |
33 | elif args["class"] == "gabor":
34 | params = {"theta":4, "frequency":(0,1,0.5,0.8), "sigma":(1,3),"n_slice":2}
35 | gd = GaborDescriptor(params)
36 | gaborKernels = gd.kernels()
37 |
38 | query = cv2.imread(args["query"])
39 | features = gd.gaborHistogram(query,gaborKernels)
40 |
41 | searcher = Searcher("gabor/index.csv")
42 | results = searcher._gsearch(features)
43 |
44 | elif args["class"] == "hog":
45 | hog = HOGDescriptor()
46 |
47 | query = cv2.imread(args["query"])
48 | features = hog.describe(query)
49 |
50 | searcher = Searcher("hog/index.csv")
51 | results = searcher.search(features)
52 |
53 |
54 | myWindow = cv2.resize(query,(960,960))
55 | cv2.imshow("query",myWindow)
56 |
57 | for (score,resultId) in results:
58 | print(score)
59 | result = cv2.imread("../" + "database" +"/"+resultId)
60 | print(resultId)
61 | myWindow = cv2.resize(result,(480,480))
62 | cv2.imshow("Result: "+str(score),myWindow)
63 | ch = cv2.waitKey(0)
64 | if ch == ord('q'):
65 | pass
66 |
67 | elif args["class"] == "vgg16":
68 |
69 | h5f = h5py.File("vgg16/index.h5",'r')
70 | feats = h5f['dataset_1'][:]
71 | #print(feats)
72 | imgNames = h5f['dataset_2'][:]
73 | #print(imgNames)
74 | h5f.close()
75 |
76 | query = cv2.imread(args["query"])
77 |
78 | model = VGGNet()
79 |
80 | queryVec = model.extract_feat(args["query"])
81 | # dot product between two vectors can be used as aggregate for similarity as the projection of vector u on vector v (u^T.v) is considered as similar
82 | # when the angle between them is 0 degrees. Therefore, more is the resultant of their product implies more is the similarity b/n them
83 | scores = np.dot(queryVec, feats.T)
84 | #print(scores)
85 | rank_ID = np.argsort(scores)[::-1]
86 | #print(rank_ID)
87 | rank_score = scores[rank_ID]
88 | print(rank_score)
89 |
90 | myWindow = cv2.resize(query,(960,960))
91 | cv2.imshow("query",myWindow)
92 |
93 | maxres = 10
94 | imlist = [imgNames[index] for i,index in enumerate(rank_ID[0:maxres])]
95 | print("top %d images in order are: " %maxres, imlist)
96 |
97 | # show top #maxres retrieved result one by one
98 | for i,im in enumerate(imlist):
99 | result = cv2.imread("../" + "database" +"/"+str(im, 'utf-8'))
100 | myWindow = cv2.resize(result,(480,480))
101 | cv2.imshow("Result: ",myWindow)
102 | ch = cv2.waitKey(0)
103 | if ch == ord('q'):
104 | pass
105 |
106 | #run
107 | # python3 search.py --query ../query_images/results_pyramids.jpg --class color
108 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | [ Assignment-2 for Elective CSPE31 ]
2 |
3 | # Image-Retrieval (Content-Based Image Retrieval System)
4 |
5 | ## Introduction
6 | This repository implements `Image Search Engine` through `CBIR` (content based image retrieval) approach.
7 | CBIR approach pays greater attention to global and local information, such as color, shape, texture, region of an image.
8 |
9 | ## Part 1. Feature Extraction
10 |
11 | Feature extraction is a means of extracting compact but semantically valuable information from images. This information is used as a signature for the image. Similar images should have similar signatures.
12 |
13 | In this retrieval system, i implemented several image features descriptors:
14 |
15 | * `color-based`
16 | - [HSV histogram](https://github.com/vamc-stash/image-retrieval/blob/master/src/color/color.py)
17 | To extract the color features from the content of an image, a proper color space(HSV in this case) and an effective color descriptor are determined through color histogram(Local color histogram).
18 | * `texture-based`
19 | - [Gabor filter](https://github.com/vamc-stash/image-retrieval/blob/master/src/gabor/gabor.py)
20 | Texture can be thought of as repeated patterns of pixels over a spatial domain. Texture properties are the visual patterns in an image that have properties of homogeneity that do not result from the presence of only a single color or intensity.
21 | Frequency and orientation representations of the `Gabor filter` are similar to those of the human visual system. The images are filtered using the real parts of various different Gabor filter kernels. The mean and variance of the filtered images are then used as features for classification, which is based on the least squared error.
22 | * `shape-based`
23 | - [Histogram of Oriented Gradient(HOG)](https://github.com/vamc-stash/image-retrieval/blob/master/src/hog/hog.py)
24 | In this descriptor, feature vector is extracted by segementing image into smaller cells and for each cell, we accumulate a local histogram of gradient in several orientations over all the pixels in the cell.
25 | * `deep methods`
26 | - [VGGNet(VGG16)](https://github.com/vamc-stash/image-retrieval/blob/master/src/vgg16/vgg16.py)
27 | VGG16 is a convolutional neural network model pre-trained on ImageNet dataset.
28 |
29 |
30 | ## Part 2. Indexing dataset
31 |
32 | Now apply image descriptor to each image in your dataset, extract features from these images, and write the features to storage (ex. CSV file, RDBMS, Redis, etc.) so that they can be later compared for similarity.
33 | **color-based** - `run python3 /color/index.py --index index.csv`
34 | **texture-based** - `run python3 /gabor/index.py --index index.csv`
35 | **shape-based** - `run python3 /hog/index.py --index index.csv`
36 | **vgg16** - `run python3 /vgg16/index.py --index index.h5`
37 |
38 |
39 | ## Part 3. Define Similarity metric
40 |
41 | Depending upon dataset and types of features extracted, define a method (ex. Euclidean distance, Cosine distance, and chi-squared distance) to compare features for similarity.
42 | I used chi-squared distance for color histogram and HOG methods, Least square error technique for gabor filter method and orthogonal projection of one feature vector onto another for VGG16 method to compare similarity between features.
43 |
44 | ## Part 4. Searching
45 |
46 | This part performs actual search of user query image by (1) extracting features from this query image and then (2) apply your similarity function to compare the query features to the features already indexed. From there, system returns the most relevant results according to your similarity function.
47 |
48 | ### Query 1 - pyramids
49 | 
50 | ### Results (Top 5 of each method)
51 | **color**
52 | `run python3 search.py --query ../query_images/tajmahal.jpg --c color`
53 | 
54 | **gabor**
55 | `run python3 search.py --query ../query_images/tajmahal.jpg --c gabor`
56 | 
57 | **hog**
58 | `run python3 search.py --query ../query_images/tajmahal.jpg --c hog`
59 | 
60 | **vgg16**
61 | `run python3 search.py --query ../query_images/tajmahal.jpg --c vgg16`
62 | 
63 |
64 |
65 |
66 |
67 |
68 |
69 |
70 |
--------------------------------------------------------------------------------