├── .gitignore ├── LICENSE ├── README.CN.md ├── README.md ├── presentation.pdf ├── references └── Visual Search at Pinterest_2015.pdf ├── search_result.jpg └── visual_search ├── alexnet_forward.py ├── caffe_classes.py ├── classification.py ├── myalexnet_feature.py ├── tools ├── __init__.py ├── gallery.py ├── image_crop.py ├── image_mirror.py ├── image_resize.py ├── image_rotate.py └── image_watermark_text.py └── visual_search.py /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | env/ 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | downloads/ 16 | eggs/ 17 | .eggs/ 18 | lib/ 19 | lib64/ 20 | parts/ 21 | sdist/ 22 | var/ 23 | wheels/ 24 | *.egg-info/ 25 | .installed.cfg 26 | *.egg 27 | 28 | # PyInstaller 29 | # Usually these files are written by a python script from a template 30 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 31 | *.manifest 32 | *.spec 33 | 34 | # Installer logs 35 | pip-log.txt 36 | pip-delete-this-directory.txt 37 | 38 | # Unit test / coverage reports 39 | htmlcov/ 40 | .tox/ 41 | .coverage 42 | .coverage.* 43 | .cache 44 | nosetests.xml 45 | coverage.xml 46 | *,cover 47 | .hypothesis/ 48 | 49 | # Translations 50 | *.mo 51 | *.pot 52 | 53 | # Django stuff: 54 | *.log 55 | local_settings.py 56 | 57 | # Flask stuff: 58 | instance/ 59 | .webassets-cache 60 | 61 | # Scrapy stuff: 62 | .scrapy 63 | 64 | # Sphinx documentation 65 | docs/_build/ 66 | 67 | # PyBuilder 68 | target/ 69 | 70 | # Jupyter Notebook 71 | .ipynb_checkpoints 72 | 73 | # pyenv 74 | .python-version 75 | 76 | # celery beat schedule file 77 | celerybeat-schedule 78 | 79 | # dotenv 80 | .env 81 | 82 | # virtualenv 83 | .venv 84 | venv/ 85 | ENV/ 86 | 87 | # Spyder project settings 88 | .spyderproject 89 | 90 | # Rope project settings 91 | .ropeproject 92 | .idea/ 93 | *.npy -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.CN.md: -------------------------------------------------------------------------------- 1 | # 炒一锅基于深度学习的图像检索工具 2 | 本文介绍如何快速搭建一个基于深度学习的图像检索工具 3 | ## 原料 4 | - 数据集: [Caltech256](http://www.vision.caltech.edu/Image_Datasets/Caltech256/) 包含从 Google 图像搜索和PicSearch.com上获得的 30, 607张物体的图像.这些图像通过人工判别被分配在257个类别中. 在这个实验里我们把Caltech256作为我们要检索的图片库. [下载](http://www.vision.caltech.edu/Image_Datasets/Caltech256/256_ObjectCategories.tar) 5 | - 代码: 多伦多大学的老师 Michael Guerzhoy 在[个人网站](http://www.cs.toronto.edu/~guerzhoy/tf_alexnet/)上提供了 AlexNet 的 TensorFlow 实现及权重(weights). 搭建一台能够训练深度学习模型的机器不容易,更不要说训练一个好的模型要花多少时间了, 而有了这个训练好的模型,大家就可以快速的体验深度学习的魅力. 6 | [下载权重(bvlc_alexnet.npy)](http://www.cs.toronto.edu/~guerzhoy/tf_alexnet/bvlc_alexnet.npy) 7 | 8 | ## 厨具 9 | - 安装Python及相关库(TensorFlow等), 建议安装[Anaconda](https://www.continuum.io/downloads). 10 | 11 | ## 菜谱 12 | 1. 切菜: 修改图像大小 13 | 训练的 AlexNet 模型的输入图片的大小是固定的[227, 227],而 Caltech256 中的图片宽高是不固定的. `image_resize.py`可以将某一目录的下的图片批量resize,并保存到另一个目录下. 在终端中敲下`python ./visual_search/tools/image_resize.py -h`查看使用说明. 14 | 15 | ``` 16 | usage: image_resize.py [-h] [--input_data_dir INPUT_DATA_DIR] 17 | [--output_data_dir OUTPUT_DATA_DIR] [--width WIDTH] 18 | [--height HEIGHT] 19 | 20 | optional arguments: 21 | -h, --help show this help message and exit 22 | --input_data_dir INPUT_DATA_DIR 23 | Directory to put the input data. 24 | --output_data_dir OUTPUT_DATA_DIR 25 | Directory to put the output data. 26 | --width WIDTH Target image width. 27 | --height HEIGHT Target image height. 28 | ``` 29 | 30 | 2. 开火煮: 提取图像特征 31 | 用`visual_search/myalexnet_feature.py`提取图片库中每张图片的特征. 这个脚本会输出两个文件:一个图片的特征,一个是所有图像的完整路径. 32 | 33 | ``` 34 | $ cd visual_search 35 | $ python myalexnet_feature.py -h 36 | usage: myalexnet_feature.py [-h] [--input_data_dir INPUT_DATA_DIR] 37 | [--output_feature_file OUTPUT_FEATURE_FILE] 38 | [--output_image_name_file OUTPUT_IMAGE_NAME_FILE] 39 | 40 | optional arguments: 41 | -h, --help show this help message and exit 42 | --input_data_dir INPUT_DATA_DIR 43 | Directory to put the input data. 44 | --output_feature_file OUTPUT_FEATURE_FILE 45 | Output features path. 46 | --output_image_name_file OUTPUT_IMAGE_NAME_FILE 47 | Output image names path. 48 | ``` 49 | 50 | 51 | ## 上菜 52 | 在`visual_search/visual_search.py`脚本里修改图片特征的路径和图像名称的路径可以进行图片检索了. 输入图像可以是本地图片也可以一个图片的链接地址. 53 | 54 | ``` 55 | usage: visual_search.py [-h] [--img_file_path IMG_FILE_PATH] 56 | [--img_url IMG_URL] 57 | 58 | optional arguments: 59 | -h, --help show this help message and exit 60 | --img_file_path IMG_FILE_PATH 61 | Image file path. 62 | --img_url IMG_URL Image Url. 63 | 64 | ``` 65 | 66 | ![Search Result Demo](search_result.jpg) 67 | 68 | 你会发现有好几行图片,其实每一行都是一个检索记录. 输入图像是每一行的第一张图片. 第2到5行的输入图像是分别对原始图像进行加水印,旋转,裁剪,镜像操作得到. 69 | 70 | 做这个实验项目其实花了不到一周的业余时间,根据我提供的信息,相信大家可以在很短的时间内做出一套自己的基于深度学习的图像搜索工具. 71 | 72 | 项目地址: [https://github.com/GYXie/visual-search](https://github.com/GYXie/visual-search) 73 | 74 | 博客: [炒一锅基于深度学习的图像检索工具](http://gyxie.github.io/2017/02/26/%E7%82%92%E4%B8%80%E9%94%85%E5%9F%BA%E4%BA%8E%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%9A%84%E5%9B%BE%E5%83%8F%E6%A3%80%E7%B4%A2%E5%B7%A5%E5%85%B7/) 75 | 76 | 由于时间问题,很多细节没有写,以后会补上.如果有问题,可以提Issue. 77 | 78 | ## Reference 79 | - Jing Y, Liu D, Kislyuk D, et al. Visual search at pinterest[C]//Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2015: 1889-1898. 80 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [中文](https://github.com/GYXie/visual-search/blob/master/README.CN.md) 2 | # A toy project for visual search, based on deep learning 3 | This article describes how to quickly build an image retrieval tool based on deep learning. 4 | ## Data 5 | - Dataset: [Caltech256](http://www.vision.caltech.edu/Image_Datasets/Caltech256/) Contains 30, 607 images from Google Image Search and PicSearch.com. These images were assigned to 257 categories by manual discrimination. In this experiment we use Caltech256 as the image library we want to retrieve. [Download](http://www.vision.caltech.edu/Image_Datasets/Caltech256/256_ObjectCategories.tar) 6 | - Code: Michael Guerzhoy, a researcher at the University of Toronto, provides AlexNet's TensorFlow implementation and weights on his personal website(http://www.cs.toronto.edu/~guerzhoy/tf_alexnet/). It's not easy to build a machine that can train a deep learning model, let alone how long it takes to train a good model. With this well-trained model, everyone can quickly experience the charm of deep learning. 7 | 8 | [Download model weights(bvlc_alexnet.npy)](http://www.cs.toronto.edu/~guerzhoy/tf_alexnet/bvlc_alexnet.npy) 9 | 10 | ## Tools 11 | - Install Python and related lib(TensorFlow etc.). [Anaconda](https://www.continuum.io/downloads) is recommended. 12 | 13 | ## Preporcessing 14 | 1. Resize 15 | The size of the input image of the trained AlexNet model is fixed [227, 227], while the width and height of the picture in Caltech256 are not fixed. `image_resize.py` can batch resize the images under a certain directory and save them to another directory. Tap `python ./visual_search/tools/image_resize.py -h` in the terminal to view the instructions. 16 | 17 | ``` 18 | usage: image_resize.py [-h] [--input_data_dir INPUT_DATA_DIR] 19 | [--output_data_dir OUTPUT_DATA_DIR] [--width WIDTH] 20 | [--height HEIGHT] 21 | 22 | optional arguments: 23 | -h, --help show this help message and exit 24 | --input_data_dir INPUT_DATA_DIR 25 | Directory to put the input data. 26 | --output_data_dir OUTPUT_DATA_DIR 27 | Directory to put the output data. 28 | --width WIDTH Target image width. 29 | --height HEIGHT Target image height. 30 | ``` 31 | 32 | 2. Extract image features 33 | Use `visual_search/myalexnet_feature.py` to extract the feature of each image in the library. This script will output two files: the feature of every image, and the full path of all images. 34 | 35 | ``` 36 | $ cd visual_search 37 | $ python myalexnet_feature.py -h 38 | usage: myalexnet_feature.py [-h] [--input_data_dir INPUT_DATA_DIR] 39 | [--output_feature_file OUTPUT_FEATURE_FILE] 40 | [--output_image_name_file OUTPUT_IMAGE_NAME_FILE] 41 | 42 | optional arguments: 43 | -h, --help show this help message and exit 44 | --input_data_dir INPUT_DATA_DIR 45 | Directory to put the input data. 46 | --output_feature_file OUTPUT_FEATURE_FILE 47 | Output features path. 48 | --output_image_name_file OUTPUT_IMAGE_NAME_FILE 49 | Output image names path. 50 | ``` 51 | 52 | 53 | ## Play 54 | In the `visual_search/visual_search.py` script, you can modify the path of the image feature and the path of the image name for retrieval. The input can be a local image or a picture url. 55 | 56 | ``` 57 | usage: visual_search.py [-h] [--img_file_path IMG_FILE_PATH] 58 | [--img_url IMG_URL] 59 | 60 | optional arguments: 61 | -h, --help show this help message and exit 62 | --img_file_path IMG_FILE_PATH 63 | Image file path. 64 | --img_url IMG_URL Image Url. 65 | 66 | ``` 67 | 68 | ![Search Result Demo](search_result.jpg) 69 | 70 | You will find several lines of images. Each line is a search record. The input image is the first one of each line. The input images of lines 2 to 5 are obtained by watermarking, rotating, cropping, and mirroring the original image. 71 | 72 | In fact, this experimental project took less than a week of my spare time. According to the information I provided, I believe you can make your own image search tool based on deep learning in a short period of time. 73 | 74 | Github: [https://github.com/GYXie/visual-search](https://github.com/GYXie/visual-search) 75 | 76 | 中文博客: [炒一锅基于深度学习的图像检索工具](http://gyxie.github.io/2017/02/26/%E7%82%92%E4%B8%80%E9%94%85%E5%9F%BA%E4%BA%8E%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%9A%84%E5%9B%BE%E5%83%8F%E6%A3%80%E7%B4%A2%E5%B7%A5%E5%85%B7/) 77 | 78 | ## Reference 79 | - Jing Y, Liu D, Kislyuk D, et al. Visual search at pinterest[C]//Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2015: 1889-1898. 80 | -------------------------------------------------------------------------------- /presentation.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GYXie/visual-search/fcc5ffd0004dcdbf7a336504fcc74032bb31c3f9/presentation.pdf -------------------------------------------------------------------------------- /references/Visual Search at Pinterest_2015.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GYXie/visual-search/fcc5ffd0004dcdbf7a336504fcc74032bb31c3f9/references/Visual Search at Pinterest_2015.pdf -------------------------------------------------------------------------------- /search_result.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GYXie/visual-search/fcc5ffd0004dcdbf7a336504fcc74032bb31c3f9/search_result.jpg -------------------------------------------------------------------------------- /visual_search/alexnet_forward.py: -------------------------------------------------------------------------------- 1 | ################################################################################ 2 | #Michael Guerzhoy and Davi Frossard, 2016 3 | #AlexNet implementation in TensorFlow, with weights 4 | #Details: 5 | #http://www.cs.toronto.edu/~guerzhoy/tf_alexnet/ 6 | # 7 | #With code from https://github.com/ethereon/caffe-tensorflow 8 | #Model from https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet 9 | #Weights from Caffe converted using https://github.com/ethereon/caffe-tensorflow 10 | # 11 | # 12 | ################################################################################ 13 | 14 | from numpy import * 15 | import os 16 | from pylab import * 17 | import numpy as np 18 | import matplotlib.pyplot as plt 19 | import matplotlib.cbook as cbook 20 | import time 21 | from scipy.misc import imread 22 | from scipy.misc import imresize 23 | import matplotlib.image as mpimg 24 | from scipy.ndimage import filters 25 | import urllib 26 | from numpy import random 27 | 28 | 29 | import tensorflow as tf 30 | 31 | from caffe_classes import class_names 32 | 33 | train_x = zeros((1, 227,227,3)).astype(float32) 34 | train_y = zeros((1, 1000)) 35 | xdim = train_x.shape[1:] 36 | ydim = train_y.shape[1] 37 | 38 | 39 | 40 | ################################################################################ 41 | #Read Image 42 | 43 | 44 | im1 = (imread("d.jpg")[:,:,:3]).astype(float32) 45 | im1 = im1 - mean(im1) 46 | 47 | im2 = (imread("laska.png")[:,:,:3]).astype(float32) 48 | im2 = im2 - mean(im2) 49 | 50 | ################################################################################ 51 | 52 | # (self.feed('data') 53 | # .conv(11, 11, 96, 4, 4, padding='VALID', name='conv1') 54 | # .lrn(2, 2e-05, 0.75, name='norm1') 55 | # .max_pool(3, 3, 2, 2, padding='VALID', name='pool1') 56 | # .conv(5, 5, 256, 1, 1, group=2, name='conv2') 57 | # .lrn(2, 2e-05, 0.75, name='norm2') 58 | # .max_pool(3, 3, 2, 2, padding='VALID', name='pool2') 59 | # .conv(3, 3, 384, 1, 1, name='conv3') 60 | # .conv(3, 3, 384, 1, 1, group=2, name='conv4') 61 | # .conv(3, 3, 256, 1, 1, group=2, name='conv5') 62 | # .fc(4096, name='fc6') 63 | # .fc(4096, name='fc7') 64 | # .fc(1000, relu=False, name='fc8') 65 | # .softmax(name='prob')) 66 | 67 | 68 | net_data = load("bvlc_alexnet.npy").item() 69 | 70 | def conv(input, kernel, biases, k_h, k_w, c_o, s_h, s_w, padding="VALID", group=1): 71 | '''From https://github.com/ethereon/caffe-tensorflow 72 | ''' 73 | c_i = input.get_shape()[-1] 74 | assert c_i%group==0 75 | assert c_o%group==0 76 | convolve = lambda i, k: tf.nn.conv2d(i, k, [1, s_h, s_w, 1], padding=padding) 77 | 78 | 79 | if group==1: 80 | conv = convolve(input, kernel) 81 | else: 82 | input_groups = tf.split(3, group, input) 83 | kernel_groups = tf.split(3, group, kernel) 84 | output_groups = [convolve(i, k) for i,k in zip(input_groups, kernel_groups)] 85 | conv = tf.concat(3, output_groups) 86 | return tf.reshape(tf.nn.bias_add(conv, biases), [-1]+conv.get_shape().as_list()[1:]) 87 | 88 | 89 | 90 | x = tf.placeholder(tf.float32, (None,) + xdim) 91 | 92 | 93 | #conv1 94 | #conv(11, 11, 96, 4, 4, padding='VALID', name='conv1') 95 | k_h = 11; k_w = 11; c_o = 96; s_h = 4; s_w = 4 96 | conv1W = tf.Variable(net_data["conv1"][0]) 97 | conv1b = tf.Variable(net_data["conv1"][1]) 98 | conv1_in = conv(x, conv1W, conv1b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=1) 99 | conv1 = tf.nn.relu(conv1_in) 100 | 101 | #lrn1 102 | #lrn(2, 2e-05, 0.75, name='norm1') 103 | radius = 2; alpha = 2e-05; beta = 0.75; bias = 1.0 104 | lrn1 = tf.nn.local_response_normalization(conv1, 105 | depth_radius=radius, 106 | alpha=alpha, 107 | beta=beta, 108 | bias=bias) 109 | 110 | #maxpool1 111 | #max_pool(3, 3, 2, 2, padding='VALID', name='pool1') 112 | k_h = 3; k_w = 3; s_h = 2; s_w = 2; padding = 'VALID' 113 | maxpool1 = tf.nn.max_pool(lrn1, ksize=[1, k_h, k_w, 1], strides=[1, s_h, s_w, 1], padding=padding) 114 | 115 | 116 | #conv2 117 | #conv(5, 5, 256, 1, 1, group=2, name='conv2') 118 | k_h = 5; k_w = 5; c_o = 256; s_h = 1; s_w = 1; group = 2 119 | conv2W = tf.Variable(net_data["conv2"][0]) 120 | conv2b = tf.Variable(net_data["conv2"][1]) 121 | conv2_in = conv(maxpool1, conv2W, conv2b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=group) 122 | conv2 = tf.nn.relu(conv2_in) 123 | 124 | 125 | #lrn2 126 | #lrn(2, 2e-05, 0.75, name='norm2') 127 | radius = 2; alpha = 2e-05; beta = 0.75; bias = 1.0 128 | lrn2 = tf.nn.local_response_normalization(conv2, 129 | depth_radius=radius, 130 | alpha=alpha, 131 | beta=beta, 132 | bias=bias) 133 | 134 | #maxpool2 135 | #max_pool(3, 3, 2, 2, padding='VALID', name='pool2') 136 | k_h = 3; k_w = 3; s_h = 2; s_w = 2; padding = 'VALID' 137 | maxpool2 = tf.nn.max_pool(lrn2, ksize=[1, k_h, k_w, 1], strides=[1, s_h, s_w, 1], padding=padding) 138 | 139 | #conv3 140 | #conv(3, 3, 384, 1, 1, name='conv3') 141 | k_h = 3; k_w = 3; c_o = 384; s_h = 1; s_w = 1; group = 1 142 | conv3W = tf.Variable(net_data["conv3"][0]) 143 | conv3b = tf.Variable(net_data["conv3"][1]) 144 | conv3_in = conv(maxpool2, conv3W, conv3b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=group) 145 | conv3 = tf.nn.relu(conv3_in) 146 | 147 | #conv4 148 | #conv(3, 3, 384, 1, 1, group=2, name='conv4') 149 | k_h = 3; k_w = 3; c_o = 384; s_h = 1; s_w = 1; group = 2 150 | conv4W = tf.Variable(net_data["conv4"][0]) 151 | conv4b = tf.Variable(net_data["conv4"][1]) 152 | conv4_in = conv(conv3, conv4W, conv4b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=group) 153 | conv4 = tf.nn.relu(conv4_in) 154 | 155 | 156 | #conv5 157 | #conv(3, 3, 256, 1, 1, group=2, name='conv5') 158 | k_h = 3; k_w = 3; c_o = 256; s_h = 1; s_w = 1; group = 2 159 | conv5W = tf.Variable(net_data["conv5"][0]) 160 | conv5b = tf.Variable(net_data["conv5"][1]) 161 | conv5_in = conv(conv4, conv5W, conv5b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=group) 162 | conv5 = tf.nn.relu(conv5_in) 163 | 164 | #maxpool5 165 | #max_pool(3, 3, 2, 2, padding='VALID', name='pool5') 166 | k_h = 3; k_w = 3; s_h = 2; s_w = 2; padding = 'VALID' 167 | maxpool5 = tf.nn.max_pool(conv5, ksize=[1, k_h, k_w, 1], strides=[1, s_h, s_w, 1], padding=padding) 168 | 169 | #fc6 170 | #fc(4096, name='fc6') 171 | fc6W = tf.Variable(net_data["fc6"][0]) 172 | fc6b = tf.Variable(net_data["fc6"][1]) 173 | fc6 = tf.nn.relu_layer(tf.reshape(maxpool5, [-1, int(prod(maxpool5.get_shape()[1:]))]), fc6W, fc6b) 174 | 175 | #fc7 176 | #fc(4096, name='fc7') 177 | fc7W = tf.Variable(net_data["fc7"][0]) 178 | fc7b = tf.Variable(net_data["fc7"][1]) 179 | fc7 = tf.nn.relu_layer(fc6, fc7W, fc7b) 180 | 181 | #fc8 182 | #fc(1000, relu=False, name='fc8') 183 | fc8W = tf.Variable(net_data["fc8"][0]) 184 | fc8b = tf.Variable(net_data["fc8"][1]) 185 | fc8 = tf.nn.xw_plus_b(fc7, fc8W, fc8b) 186 | 187 | 188 | #prob 189 | #softmax(name='prob')) 190 | prob = tf.nn.softmax(fc8) 191 | 192 | init = tf.initialize_all_variables() 193 | sess = tf.Session() 194 | sess.run(init) 195 | 196 | t = time.time() 197 | output = sess.run(prob, feed_dict = {x:[im1,im2]}) 198 | ################################################################################ 199 | 200 | #Output: 201 | 202 | 203 | for input_im_ind in range(output.shape[0]): 204 | inds = argsort(output)[input_im_ind,:] 205 | print "Image", input_im_ind 206 | for i in range(5): 207 | print class_names[inds[-1-i]], output[input_im_ind, inds[-1-i]] 208 | 209 | print time.time()-t -------------------------------------------------------------------------------- /visual_search/caffe_classes.py: -------------------------------------------------------------------------------- 1 | class_names = '''tench, Tinca tinca 2 | goldfish, Carassius auratus 3 | great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias 4 | tiger shark, Galeocerdo cuvieri 5 | hammerhead, hammerhead shark 6 | electric ray, crampfish, numbfish, torpedo 7 | stingray 8 | cock 9 | hen 10 | ostrich, Struthio camelus 11 | brambling, Fringilla montifringilla 12 | goldfinch, Carduelis carduelis 13 | house finch, linnet, Carpodacus mexicanus 14 | junco, snowbird 15 | indigo bunting, indigo finch, indigo bird, Passerina cyanea 16 | robin, American robin, Turdus migratorius 17 | bulbul 18 | jay 19 | magpie 20 | chickadee 21 | water ouzel, dipper 22 | kite 23 | bald eagle, American eagle, Haliaeetus leucocephalus 24 | vulture 25 | great grey owl, great gray owl, Strix nebulosa 26 | European fire salamander, Salamandra salamandra 27 | common newt, Triturus vulgaris 28 | eft 29 | spotted salamander, Ambystoma maculatum 30 | axolotl, mud puppy, Ambystoma mexicanum 31 | bullfrog, Rana catesbeiana 32 | tree frog, tree-frog 33 | tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui 34 | loggerhead, loggerhead turtle, Caretta caretta 35 | leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea 36 | mud turtle 37 | terrapin 38 | box turtle, box tortoise 39 | banded gecko 40 | common iguana, iguana, Iguana iguana 41 | American chameleon, anole, Anolis carolinensis 42 | whiptail, whiptail lizard 43 | agama 44 | frilled lizard, Chlamydosaurus kingi 45 | alligator lizard 46 | Gila monster, Heloderma suspectum 47 | green lizard, Lacerta viridis 48 | African chameleon, Chamaeleo chamaeleon 49 | Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis 50 | African crocodile, Nile crocodile, Crocodylus niloticus 51 | American alligator, Alligator mississipiensis 52 | triceratops 53 | thunder snake, worm snake, Carphophis amoenus 54 | ringneck snake, ring-necked snake, ring snake 55 | hognose snake, puff adder, sand viper 56 | green snake, grass snake 57 | king snake, kingsnake 58 | garter snake, grass snake 59 | water snake 60 | vine snake 61 | night snake, Hypsiglena torquata 62 | boa constrictor, Constrictor constrictor 63 | rock python, rock snake, Python sebae 64 | Indian cobra, Naja naja 65 | green mamba 66 | sea snake 67 | horned viper, cerastes, sand viper, horned asp, Cerastes cornutus 68 | diamondback, diamondback rattlesnake, Crotalus adamanteus 69 | sidewinder, horned rattlesnake, Crotalus cerastes 70 | trilobite 71 | harvestman, daddy longlegs, Phalangium opilio 72 | scorpion 73 | black and gold garden spider, Argiope aurantia 74 | barn spider, Araneus cavaticus 75 | garden spider, Aranea diademata 76 | black widow, Latrodectus mactans 77 | tarantula 78 | wolf spider, hunting spider 79 | tick 80 | centipede 81 | black grouse 82 | ptarmigan 83 | ruffed grouse, partridge, Bonasa umbellus 84 | prairie chicken, prairie grouse, prairie fowl 85 | peacock 86 | quail 87 | partridge 88 | African grey, African gray, Psittacus erithacus 89 | macaw 90 | sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita 91 | lorikeet 92 | coucal 93 | bee eater 94 | hornbill 95 | hummingbird 96 | jacamar 97 | toucan 98 | drake 99 | red-breasted merganser, Mergus serrator 100 | goose 101 | black swan, Cygnus atratus 102 | tusker 103 | echidna, spiny anteater, anteater 104 | platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus 105 | wallaby, brush kangaroo 106 | koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus 107 | wombat 108 | jellyfish 109 | sea anemone, anemone 110 | brain coral 111 | flatworm, platyhelminth 112 | nematode, nematode worm, roundworm 113 | conch 114 | snail 115 | slug 116 | sea slug, nudibranch 117 | chiton, coat-of-mail shell, sea cradle, polyplacophore 118 | chambered nautilus, pearly nautilus, nautilus 119 | Dungeness crab, Cancer magister 120 | rock crab, Cancer irroratus 121 | fiddler crab 122 | king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica 123 | American lobster, Northern lobster, Maine lobster, Homarus americanus 124 | spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish 125 | crayfish, crawfish, crawdad, crawdaddy 126 | hermit crab 127 | isopod 128 | white stork, Ciconia ciconia 129 | black stork, Ciconia nigra 130 | spoonbill 131 | flamingo 132 | little blue heron, Egretta caerulea 133 | American egret, great white heron, Egretta albus 134 | bittern 135 | crane 136 | limpkin, Aramus pictus 137 | European gallinule, Porphyrio porphyrio 138 | American coot, marsh hen, mud hen, water hen, Fulica americana 139 | bustard 140 | ruddy turnstone, Arenaria interpres 141 | red-backed sandpiper, dunlin, Erolia alpina 142 | redshank, Tringa totanus 143 | dowitcher 144 | oystercatcher, oyster catcher 145 | pelican 146 | king penguin, Aptenodytes patagonica 147 | albatross, mollymawk 148 | grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus 149 | killer whale, killer, orca, grampus, sea wolf, Orcinus orca 150 | dugong, Dugong dugon 151 | sea lion 152 | Chihuahua 153 | Japanese spaniel 154 | Maltese dog, Maltese terrier, Maltese 155 | Pekinese, Pekingese, Peke 156 | Shih-Tzu 157 | Blenheim spaniel 158 | papillon 159 | toy terrier 160 | Rhodesian ridgeback 161 | Afghan hound, Afghan 162 | basset, basset hound 163 | beagle 164 | bloodhound, sleuthhound 165 | bluetick 166 | black-and-tan coonhound 167 | Walker hound, Walker foxhound 168 | English foxhound 169 | redbone 170 | borzoi, Russian wolfhound 171 | Irish wolfhound 172 | Italian greyhound 173 | whippet 174 | Ibizan hound, Ibizan Podenco 175 | Norwegian elkhound, elkhound 176 | otterhound, otter hound 177 | Saluki, gazelle hound 178 | Scottish deerhound, deerhound 179 | Weimaraner 180 | Staffordshire bullterrier, Staffordshire bull terrier 181 | American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier 182 | Bedlington terrier 183 | Border terrier 184 | Kerry blue terrier 185 | Irish terrier 186 | Norfolk terrier 187 | Norwich terrier 188 | Yorkshire terrier 189 | wire-haired fox terrier 190 | Lakeland terrier 191 | Sealyham terrier, Sealyham 192 | Airedale, Airedale terrier 193 | cairn, cairn terrier 194 | Australian terrier 195 | Dandie Dinmont, Dandie Dinmont terrier 196 | Boston bull, Boston terrier 197 | miniature schnauzer 198 | giant schnauzer 199 | standard schnauzer 200 | Scotch terrier, Scottish terrier, Scottie 201 | Tibetan terrier, chrysanthemum dog 202 | silky terrier, Sydney silky 203 | soft-coated wheaten terrier 204 | West Highland white terrier 205 | Lhasa, Lhasa apso 206 | flat-coated retriever 207 | curly-coated retriever 208 | golden retriever 209 | Labrador retriever 210 | Chesapeake Bay retriever 211 | German short-haired pointer 212 | vizsla, Hungarian pointer 213 | English setter 214 | Irish setter, red setter 215 | Gordon setter 216 | Brittany spaniel 217 | clumber, clumber spaniel 218 | English springer, English springer spaniel 219 | Welsh springer spaniel 220 | cocker spaniel, English cocker spaniel, cocker 221 | Sussex spaniel 222 | Irish water spaniel 223 | kuvasz 224 | schipperke 225 | groenendael 226 | malinois 227 | briard 228 | kelpie 229 | komondor 230 | Old English sheepdog, bobtail 231 | Shetland sheepdog, Shetland sheep dog, Shetland 232 | collie 233 | Border collie 234 | Bouvier des Flandres, Bouviers des Flandres 235 | Rottweiler 236 | German shepherd, German shepherd dog, German police dog, alsatian 237 | Doberman, Doberman pinscher 238 | miniature pinscher 239 | Greater Swiss Mountain dog 240 | Bernese mountain dog 241 | Appenzeller 242 | EntleBucher 243 | boxer 244 | bull mastiff 245 | Tibetan mastiff 246 | French bulldog 247 | Great Dane 248 | Saint Bernard, St Bernard 249 | Eskimo dog, husky 250 | malamute, malemute, Alaskan malamute 251 | Siberian husky 252 | dalmatian, coach dog, carriage dog 253 | affenpinscher, monkey pinscher, monkey dog 254 | basenji 255 | pug, pug-dog 256 | Leonberg 257 | Newfoundland, Newfoundland dog 258 | Great Pyrenees 259 | Samoyed, Samoyede 260 | Pomeranian 261 | chow, chow chow 262 | keeshond 263 | Brabancon griffon 264 | Pembroke, Pembroke Welsh corgi 265 | Cardigan, Cardigan Welsh corgi 266 | toy poodle 267 | miniature poodle 268 | standard poodle 269 | Mexican hairless 270 | timber wolf, grey wolf, gray wolf, Canis lupus 271 | white wolf, Arctic wolf, Canis lupus tundrarum 272 | red wolf, maned wolf, Canis rufus, Canis niger 273 | coyote, prairie wolf, brush wolf, Canis latrans 274 | dingo, warrigal, warragal, Canis dingo 275 | dhole, Cuon alpinus 276 | African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus 277 | hyena, hyaena 278 | red fox, Vulpes vulpes 279 | kit fox, Vulpes macrotis 280 | Arctic fox, white fox, Alopex lagopus 281 | grey fox, gray fox, Urocyon cinereoargenteus 282 | tabby, tabby cat 283 | tiger cat 284 | Persian cat 285 | Siamese cat, Siamese 286 | Egyptian cat 287 | cougar, puma, catamount, mountain lion, painter, panther, Felis concolor 288 | lynx, catamount 289 | leopard, Panthera pardus 290 | snow leopard, ounce, Panthera uncia 291 | jaguar, panther, Panthera onca, Felis onca 292 | lion, king of beasts, Panthera leo 293 | tiger, Panthera tigris 294 | cheetah, chetah, Acinonyx jubatus 295 | brown bear, bruin, Ursus arctos 296 | American black bear, black bear, Ursus americanus, Euarctos americanus 297 | ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus 298 | sloth bear, Melursus ursinus, Ursus ursinus 299 | mongoose 300 | meerkat, mierkat 301 | tiger beetle 302 | ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle 303 | ground beetle, carabid beetle 304 | long-horned beetle, longicorn, longicorn beetle 305 | leaf beetle, chrysomelid 306 | dung beetle 307 | rhinoceros beetle 308 | weevil 309 | fly 310 | bee 311 | ant, emmet, pismire 312 | grasshopper, hopper 313 | cricket 314 | walking stick, walkingstick, stick insect 315 | cockroach, roach 316 | mantis, mantid 317 | cicada, cicala 318 | leafhopper 319 | lacewing, lacewing fly 320 | dragonfly, darning needle, devil's darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk 321 | damselfly 322 | admiral 323 | ringlet, ringlet butterfly 324 | monarch, monarch butterfly, milkweed butterfly, Danaus plexippus 325 | cabbage butterfly 326 | sulphur butterfly, sulfur butterfly 327 | lycaenid, lycaenid butterfly 328 | starfish, sea star 329 | sea urchin 330 | sea cucumber, holothurian 331 | wood rabbit, cottontail, cottontail rabbit 332 | hare 333 | Angora, Angora rabbit 334 | hamster 335 | porcupine, hedgehog 336 | fox squirrel, eastern fox squirrel, Sciurus niger 337 | marmot 338 | beaver 339 | guinea pig, Cavia cobaya 340 | sorrel 341 | zebra 342 | hog, pig, grunter, squealer, Sus scrofa 343 | wild boar, boar, Sus scrofa 344 | warthog 345 | hippopotamus, hippo, river horse, Hippopotamus amphibius 346 | ox 347 | water buffalo, water ox, Asiatic buffalo, Bubalus bubalis 348 | bison 349 | ram, tup 350 | bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis 351 | ibex, Capra ibex 352 | hartebeest 353 | impala, Aepyceros melampus 354 | gazelle 355 | Arabian camel, dromedary, Camelus dromedarius 356 | llama 357 | weasel 358 | mink 359 | polecat, fitch, foulmart, foumart, Mustela putorius 360 | black-footed ferret, ferret, Mustela nigripes 361 | otter 362 | skunk, polecat, wood pussy 363 | badger 364 | armadillo 365 | three-toed sloth, ai, Bradypus tridactylus 366 | orangutan, orang, orangutang, Pongo pygmaeus 367 | gorilla, Gorilla gorilla 368 | chimpanzee, chimp, Pan troglodytes 369 | gibbon, Hylobates lar 370 | siamang, Hylobates syndactylus, Symphalangus syndactylus 371 | guenon, guenon monkey 372 | patas, hussar monkey, Erythrocebus patas 373 | baboon 374 | macaque 375 | langur 376 | colobus, colobus monkey 377 | proboscis monkey, Nasalis larvatus 378 | marmoset 379 | capuchin, ringtail, Cebus capucinus 380 | howler monkey, howler 381 | titi, titi monkey 382 | spider monkey, Ateles geoffroyi 383 | squirrel monkey, Saimiri sciureus 384 | Madagascar cat, ring-tailed lemur, Lemur catta 385 | indri, indris, Indri indri, Indri brevicaudatus 386 | Indian elephant, Elephas maximus 387 | African elephant, Loxodonta africana 388 | lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens 389 | giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca 390 | barracouta, snoek 391 | eel 392 | coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch 393 | rock beauty, Holocanthus tricolor 394 | anemone fish 395 | sturgeon 396 | gar, garfish, garpike, billfish, Lepisosteus osseus 397 | lionfish 398 | puffer, pufferfish, blowfish, globefish 399 | abacus 400 | abaya 401 | academic gown, academic robe, judge's robe 402 | accordion, piano accordion, squeeze box 403 | acoustic guitar 404 | aircraft carrier, carrier, flattop, attack aircraft carrier 405 | airliner 406 | airship, dirigible 407 | altar 408 | ambulance 409 | amphibian, amphibious vehicle 410 | analog clock 411 | apiary, bee house 412 | apron 413 | ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin 414 | assault rifle, assault gun 415 | backpack, back pack, knapsack, packsack, rucksack, haversack 416 | bakery, bakeshop, bakehouse 417 | balance beam, beam 418 | balloon 419 | ballpoint, ballpoint pen, ballpen, Biro 420 | Band Aid 421 | banjo 422 | bannister, banister, balustrade, balusters, handrail 423 | barbell 424 | barber chair 425 | barbershop 426 | barn 427 | barometer 428 | barrel, cask 429 | barrow, garden cart, lawn cart, wheelbarrow 430 | baseball 431 | basketball 432 | bassinet 433 | bassoon 434 | bathing cap, swimming cap 435 | bath towel 436 | bathtub, bathing tub, bath, tub 437 | beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon 438 | beacon, lighthouse, beacon light, pharos 439 | beaker 440 | bearskin, busby, shako 441 | beer bottle 442 | beer glass 443 | bell cote, bell cot 444 | bib 445 | bicycle-built-for-two, tandem bicycle, tandem 446 | bikini, two-piece 447 | binder, ring-binder 448 | binoculars, field glasses, opera glasses 449 | birdhouse 450 | boathouse 451 | bobsled, bobsleigh, bob 452 | bolo tie, bolo, bola tie, bola 453 | bonnet, poke bonnet 454 | bookcase 455 | bookshop, bookstore, bookstall 456 | bottlecap 457 | bow 458 | bow tie, bow-tie, bowtie 459 | brass, memorial tablet, plaque 460 | brassiere, bra, bandeau 461 | breakwater, groin, groyne, mole, bulwark, seawall, jetty 462 | breastplate, aegis, egis 463 | broom 464 | bucket, pail 465 | buckle 466 | bulletproof vest 467 | bullet train, bullet 468 | butcher shop, meat market 469 | cab, hack, taxi, taxicab 470 | caldron, cauldron 471 | candle, taper, wax light 472 | cannon 473 | canoe 474 | can opener, tin opener 475 | cardigan 476 | car mirror 477 | carousel, carrousel, merry-go-round, roundabout, whirligig 478 | carpenter's kit, tools kit 479 | carton 480 | car wheel 481 | cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM 482 | cassette 483 | cassette player 484 | castle 485 | catamaran 486 | CD player 487 | cello, violoncello 488 | cellular telephone, cellular phone, cellphone, cell, mobile phone 489 | chain 490 | chainlink fence 491 | chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour 492 | chain saw, chainsaw 493 | chest 494 | chiffonier, commode 495 | chime, bell, gong 496 | china cabinet, china closet 497 | Christmas stocking 498 | church, church building 499 | cinema, movie theater, movie theatre, movie house, picture palace 500 | cleaver, meat cleaver, chopper 501 | cliff dwelling 502 | cloak 503 | clog, geta, patten, sabot 504 | cocktail shaker 505 | coffee mug 506 | coffeepot 507 | coil, spiral, volute, whorl, helix 508 | combination lock 509 | computer keyboard, keypad 510 | confectionery, confectionary, candy store 511 | container ship, containership, container vessel 512 | convertible 513 | corkscrew, bottle screw 514 | cornet, horn, trumpet, trump 515 | cowboy boot 516 | cowboy hat, ten-gallon hat 517 | cradle 518 | crane 519 | crash helmet 520 | crate 521 | crib, cot 522 | Crock Pot 523 | croquet ball 524 | crutch 525 | cuirass 526 | dam, dike, dyke 527 | desk 528 | desktop computer 529 | dial telephone, dial phone 530 | diaper, nappy, napkin 531 | digital clock 532 | digital watch 533 | dining table, board 534 | dishrag, dishcloth 535 | dishwasher, dish washer, dishwashing machine 536 | disk brake, disc brake 537 | dock, dockage, docking facility 538 | dogsled, dog sled, dog sleigh 539 | dome 540 | doormat, welcome mat 541 | drilling platform, offshore rig 542 | drum, membranophone, tympan 543 | drumstick 544 | dumbbell 545 | Dutch oven 546 | electric fan, blower 547 | electric guitar 548 | electric locomotive 549 | entertainment center 550 | envelope 551 | espresso maker 552 | face powder 553 | feather boa, boa 554 | file, file cabinet, filing cabinet 555 | fireboat 556 | fire engine, fire truck 557 | fire screen, fireguard 558 | flagpole, flagstaff 559 | flute, transverse flute 560 | folding chair 561 | football helmet 562 | forklift 563 | fountain 564 | fountain pen 565 | four-poster 566 | freight car 567 | French horn, horn 568 | frying pan, frypan, skillet 569 | fur coat 570 | garbage truck, dustcart 571 | gasmask, respirator, gas helmet 572 | gas pump, gasoline pump, petrol pump, island dispenser 573 | goblet 574 | go-kart 575 | golf ball 576 | golfcart, golf cart 577 | gondola 578 | gong, tam-tam 579 | gown 580 | grand piano, grand 581 | greenhouse, nursery, glasshouse 582 | grille, radiator grille 583 | grocery store, grocery, food market, market 584 | guillotine 585 | hair slide 586 | hair spray 587 | half track 588 | hammer 589 | hamper 590 | hand blower, blow dryer, blow drier, hair dryer, hair drier 591 | hand-held computer, hand-held microcomputer 592 | handkerchief, hankie, hanky, hankey 593 | hard disc, hard disk, fixed disk 594 | harmonica, mouth organ, harp, mouth harp 595 | harp 596 | harvester, reaper 597 | hatchet 598 | holster 599 | home theater, home theatre 600 | honeycomb 601 | hook, claw 602 | hoopskirt, crinoline 603 | horizontal bar, high bar 604 | horse cart, horse-cart 605 | hourglass 606 | iPod 607 | iron, smoothing iron 608 | jack-o'-lantern 609 | jean, blue jean, denim 610 | jeep, landrover 611 | jersey, T-shirt, tee shirt 612 | jigsaw puzzle 613 | jinrikisha, ricksha, rickshaw 614 | joystick 615 | kimono 616 | knee pad 617 | knot 618 | lab coat, laboratory coat 619 | ladle 620 | lampshade, lamp shade 621 | laptop, laptop computer 622 | lawn mower, mower 623 | lens cap, lens cover 624 | letter opener, paper knife, paperknife 625 | library 626 | lifeboat 627 | lighter, light, igniter, ignitor 628 | limousine, limo 629 | liner, ocean liner 630 | lipstick, lip rouge 631 | Loafer 632 | lotion 633 | loudspeaker, speaker, speaker unit, loudspeaker system, speaker system 634 | loupe, jeweler's loupe 635 | lumbermill, sawmill 636 | magnetic compass 637 | mailbag, postbag 638 | mailbox, letter box 639 | maillot 640 | maillot, tank suit 641 | manhole cover 642 | maraca 643 | marimba, xylophone 644 | mask 645 | matchstick 646 | maypole 647 | maze, labyrinth 648 | measuring cup 649 | medicine chest, medicine cabinet 650 | megalith, megalithic structure 651 | microphone, mike 652 | microwave, microwave oven 653 | military uniform 654 | milk can 655 | minibus 656 | miniskirt, mini 657 | minivan 658 | missile 659 | mitten 660 | mixing bowl 661 | mobile home, manufactured home 662 | Model T 663 | modem 664 | monastery 665 | monitor 666 | moped 667 | mortar 668 | mortarboard 669 | mosque 670 | mosquito net 671 | motor scooter, scooter 672 | mountain bike, all-terrain bike, off-roader 673 | mountain tent 674 | mouse, computer mouse 675 | mousetrap 676 | moving van 677 | muzzle 678 | nail 679 | neck brace 680 | necklace 681 | nipple 682 | notebook, notebook computer 683 | obelisk 684 | oboe, hautboy, hautbois 685 | ocarina, sweet potato 686 | odometer, hodometer, mileometer, milometer 687 | oil filter 688 | organ, pipe organ 689 | oscilloscope, scope, cathode-ray oscilloscope, CRO 690 | overskirt 691 | oxcart 692 | oxygen mask 693 | packet 694 | paddle, boat paddle 695 | paddlewheel, paddle wheel 696 | padlock 697 | paintbrush 698 | pajama, pyjama, pj's, jammies 699 | palace 700 | panpipe, pandean pipe, syrinx 701 | paper towel 702 | parachute, chute 703 | parallel bars, bars 704 | park bench 705 | parking meter 706 | passenger car, coach, carriage 707 | patio, terrace 708 | pay-phone, pay-station 709 | pedestal, plinth, footstall 710 | pencil box, pencil case 711 | pencil sharpener 712 | perfume, essence 713 | Petri dish 714 | photocopier 715 | pick, plectrum, plectron 716 | pickelhaube 717 | picket fence, paling 718 | pickup, pickup truck 719 | pier 720 | piggy bank, penny bank 721 | pill bottle 722 | pillow 723 | ping-pong ball 724 | pinwheel 725 | pirate, pirate ship 726 | pitcher, ewer 727 | plane, carpenter's plane, woodworking plane 728 | planetarium 729 | plastic bag 730 | plate rack 731 | plow, plough 732 | plunger, plumber's helper 733 | Polaroid camera, Polaroid Land camera 734 | pole 735 | police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria 736 | poncho 737 | pool table, billiard table, snooker table 738 | pop bottle, soda bottle 739 | pot, flowerpot 740 | potter's wheel 741 | power drill 742 | prayer rug, prayer mat 743 | printer 744 | prison, prison house 745 | projectile, missile 746 | projector 747 | puck, hockey puck 748 | punching bag, punch bag, punching ball, punchball 749 | purse 750 | quill, quill pen 751 | quilt, comforter, comfort, puff 752 | racer, race car, racing car 753 | racket, racquet 754 | radiator 755 | radio, wireless 756 | radio telescope, radio reflector 757 | rain barrel 758 | recreational vehicle, RV, R.V. 759 | reel 760 | reflex camera 761 | refrigerator, icebox 762 | remote control, remote 763 | restaurant, eating house, eating place, eatery 764 | revolver, six-gun, six-shooter 765 | rifle 766 | rocking chair, rocker 767 | rotisserie 768 | rubber eraser, rubber, pencil eraser 769 | rugby ball 770 | rule, ruler 771 | running shoe 772 | safe 773 | safety pin 774 | saltshaker, salt shaker 775 | sandal 776 | sarong 777 | sax, saxophone 778 | scabbard 779 | scale, weighing machine 780 | school bus 781 | schooner 782 | scoreboard 783 | screen, CRT screen 784 | screw 785 | screwdriver 786 | seat belt, seatbelt 787 | sewing machine 788 | shield, buckler 789 | shoe shop, shoe-shop, shoe store 790 | shoji 791 | shopping basket 792 | shopping cart 793 | shovel 794 | shower cap 795 | shower curtain 796 | ski 797 | ski mask 798 | sleeping bag 799 | slide rule, slipstick 800 | sliding door 801 | slot, one-armed bandit 802 | snorkel 803 | snowmobile 804 | snowplow, snowplough 805 | soap dispenser 806 | soccer ball 807 | sock 808 | solar dish, solar collector, solar furnace 809 | sombrero 810 | soup bowl 811 | space bar 812 | space heater 813 | space shuttle 814 | spatula 815 | speedboat 816 | spider web, spider's web 817 | spindle 818 | sports car, sport car 819 | spotlight, spot 820 | stage 821 | steam locomotive 822 | steel arch bridge 823 | steel drum 824 | stethoscope 825 | stole 826 | stone wall 827 | stopwatch, stop watch 828 | stove 829 | strainer 830 | streetcar, tram, tramcar, trolley, trolley car 831 | stretcher 832 | studio couch, day bed 833 | stupa, tope 834 | submarine, pigboat, sub, U-boat 835 | suit, suit of clothes 836 | sundial 837 | sunglass 838 | sunglasses, dark glasses, shades 839 | sunscreen, sunblock, sun blocker 840 | suspension bridge 841 | swab, swob, mop 842 | sweatshirt 843 | swimming trunks, bathing trunks 844 | swing 845 | switch, electric switch, electrical switch 846 | syringe 847 | table lamp 848 | tank, army tank, armored combat vehicle, armoured combat vehicle 849 | tape player 850 | teapot 851 | teddy, teddy bear 852 | television, television system 853 | tennis ball 854 | thatch, thatched roof 855 | theater curtain, theatre curtain 856 | thimble 857 | thresher, thrasher, threshing machine 858 | throne 859 | tile roof 860 | toaster 861 | tobacco shop, tobacconist shop, tobacconist 862 | toilet seat 863 | torch 864 | totem pole 865 | tow truck, tow car, wrecker 866 | toyshop 867 | tractor 868 | trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi 869 | tray 870 | trench coat 871 | tricycle, trike, velocipede 872 | trimaran 873 | tripod 874 | triumphal arch 875 | trolleybus, trolley coach, trackless trolley 876 | trombone 877 | tub, vat 878 | turnstile 879 | typewriter keyboard 880 | umbrella 881 | unicycle, monocycle 882 | upright, upright piano 883 | vacuum, vacuum cleaner 884 | vase 885 | vault 886 | velvet 887 | vending machine 888 | vestment 889 | viaduct 890 | violin, fiddle 891 | volleyball 892 | waffle iron 893 | wall clock 894 | wallet, billfold, notecase, pocketbook 895 | wardrobe, closet, press 896 | warplane, military plane 897 | washbasin, handbasin, washbowl, lavabo, wash-hand basin 898 | washer, automatic washer, washing machine 899 | water bottle 900 | water jug 901 | water tower 902 | whiskey jug 903 | whistle 904 | wig 905 | window screen 906 | window shade 907 | Windsor tie 908 | wine bottle 909 | wing 910 | wok 911 | wooden spoon 912 | wool, woolen, woollen 913 | worm fence, snake fence, snake-rail fence, Virginia fence 914 | wreck 915 | yawl 916 | yurt 917 | web site, website, internet site, site 918 | comic book 919 | crossword puzzle, crossword 920 | street sign 921 | traffic light, traffic signal, stoplight 922 | book jacket, dust cover, dust jacket, dust wrapper 923 | menu 924 | plate 925 | guacamole 926 | consomme 927 | hot pot, hotpot 928 | trifle 929 | ice cream, icecream 930 | ice lolly, lolly, lollipop, popsicle 931 | French loaf 932 | bagel, beigel 933 | pretzel 934 | cheeseburger 935 | hotdog, hot dog, red hot 936 | mashed potato 937 | head cabbage 938 | broccoli 939 | cauliflower 940 | zucchini, courgette 941 | spaghetti squash 942 | acorn squash 943 | butternut squash 944 | cucumber, cuke 945 | artichoke, globe artichoke 946 | bell pepper 947 | cardoon 948 | mushroom 949 | Granny Smith 950 | strawberry 951 | orange 952 | lemon 953 | fig 954 | pineapple, ananas 955 | banana 956 | jackfruit, jak, jack 957 | custard apple 958 | pomegranate 959 | hay 960 | carbonara 961 | chocolate sauce, chocolate syrup 962 | dough 963 | meat loaf, meatloaf 964 | pizza, pizza pie 965 | potpie 966 | burrito 967 | red wine 968 | espresso 969 | cup 970 | eggnog 971 | alp 972 | bubble 973 | cliff, drop, drop-off 974 | coral reef 975 | geyser 976 | lakeside, lakeshore 977 | promontory, headland, head, foreland 978 | sandbar, sand bar 979 | seashore, coast, seacoast, sea-coast 980 | valley, vale 981 | volcano 982 | ballplayer, baseball player 983 | groom, bridegroom 984 | scuba diver 985 | rapeseed 986 | daisy 987 | yellow lady's slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum 988 | corn 989 | acorn 990 | hip, rose hip, rosehip 991 | buckeye, horse chestnut, conker 992 | coral fungus 993 | agaric 994 | gyromitra 995 | stinkhorn, carrion fungus 996 | earthstar 997 | hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa 998 | bolete 999 | ear, spike, capitulum 1000 | toilet tissue, toilet paper, bathroom tissue'''.split("\n") -------------------------------------------------------------------------------- /visual_search/classification.py: -------------------------------------------------------------------------------- 1 | # -*- coding=utf-8 -*- 2 | from __future__ import absolute_import 3 | from __future__ import print_function 4 | import argparse 5 | from numpy import * 6 | import os 7 | from pylab import * 8 | import numpy as np 9 | import matplotlib.pyplot as plt 10 | import matplotlib.cbook as cbook 11 | import time 12 | from scipy.misc import imread 13 | from scipy.misc import imresize 14 | import matplotlib.image as mpimg 15 | from scipy.ndimage import filters 16 | import urllib 17 | from random import sample 18 | import tensorflow as tf 19 | from caffe_classes import class_names 20 | import matplotlib.pyplot as plt 21 | plt.rcdefaults() 22 | 23 | args = None 24 | train_x = zeros((1, 227, 227, 3)).astype(float32) 25 | train_y = zeros((1, 1000)) 26 | xdim = train_x.shape[1:] 27 | ydim = train_y.shape[1] 28 | 29 | ################################################################################ 30 | # Read Image 31 | 32 | 33 | # im1 = (imread("d.jpg")[:, :, :3]).astype(float32) 34 | # im1 = im1 - mean(im1) 35 | # 36 | # im2 = (imread("laska.png")[:, :, :3]).astype(float32) 37 | # im2 = im2 - mean(im2) 38 | 39 | ################################################################################ 40 | 41 | # (self.feed('data') 42 | # .conv(11, 11, 96, 4, 4, padding='VALID', name='conv1') 43 | # .lrn(2, 2e-05, 0.75, name='norm1') 44 | # .max_pool(3, 3, 2, 2, padding='VALID', name='pool1') 45 | # .conv(5, 5, 256, 1, 1, group=2, name='conv2') 46 | # .lrn(2, 2e-05, 0.75, name='norm2') 47 | # .max_pool(3, 3, 2, 2, padding='VALID', name='pool2') 48 | # .conv(3, 3, 384, 1, 1, name='conv3') 49 | # .conv(3, 3, 384, 1, 1, group=2, name='conv4') 50 | # .conv(3, 3, 256, 1, 1, group=2, name='conv5') 51 | # .fc(4096, name='fc6') 52 | # .fc(4096, name='fc7') 53 | # .fc(1000, relu=False, name='fc8') 54 | # .softmax(name='prob')) 55 | 56 | 57 | net_data = load("bvlc_alexnet.npy").item() 58 | 59 | 60 | def conv(input, kernel, biases, k_h, k_w, c_o, s_h, s_w, padding="VALID", group=1): 61 | '''From https://github.com/ethereon/caffe-tensorflow 62 | ''' 63 | c_i = input.get_shape()[-1] 64 | assert c_i % group == 0 65 | assert c_o % group == 0 66 | convolve = lambda i, k: tf.nn.conv2d(i, k, [1, s_h, s_w, 1], padding=padding) 67 | 68 | if group == 1: 69 | conv = convolve(input, kernel) 70 | else: 71 | input_groups = tf.split(3, group, input) 72 | kernel_groups = tf.split(3, group, kernel) 73 | output_groups = [convolve(i, k) for i, k in zip(input_groups, kernel_groups)] 74 | conv = tf.concat(3, output_groups) 75 | return tf.reshape(tf.nn.bias_add(conv, biases), [-1] + conv.get_shape().as_list()[1:]) 76 | 77 | 78 | def build_model(): 79 | x = tf.placeholder(tf.float32, (None,) + xdim) 80 | 81 | # conv1 82 | # conv(11, 11, 96, 4, 4, padding='VALID', name='conv1') 83 | k_h = 11; 84 | k_w = 11; 85 | c_o = 96; 86 | s_h = 4; 87 | s_w = 4 88 | conv1W = tf.Variable(net_data["conv1"][0]) 89 | conv1b = tf.Variable(net_data["conv1"][1]) 90 | conv1_in = conv(x, conv1W, conv1b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=1) 91 | conv1 = tf.nn.relu(conv1_in) 92 | 93 | # lrn1 94 | # lrn(2, 2e-05, 0.75, name='norm1') 95 | radius = 2; 96 | alpha = 2e-05; 97 | beta = 0.75; 98 | bias = 1.0 99 | lrn1 = tf.nn.local_response_normalization(conv1, 100 | depth_radius=radius, 101 | alpha=alpha, 102 | beta=beta, 103 | bias=bias) 104 | 105 | # maxpool1 106 | # max_pool(3, 3, 2, 2, padding='VALID', name='pool1') 107 | k_h = 3; 108 | k_w = 3; 109 | s_h = 2; 110 | s_w = 2; 111 | padding = 'VALID' 112 | maxpool1 = tf.nn.max_pool(lrn1, ksize=[1, k_h, k_w, 1], strides=[1, s_h, s_w, 1], padding=padding) 113 | 114 | # conv2 115 | # conv(5, 5, 256, 1, 1, group=2, name='conv2') 116 | k_h = 5; 117 | k_w = 5; 118 | c_o = 256; 119 | s_h = 1; 120 | s_w = 1; 121 | group = 2 122 | conv2W = tf.Variable(net_data["conv2"][0]) 123 | conv2b = tf.Variable(net_data["conv2"][1]) 124 | conv2_in = conv(maxpool1, conv2W, conv2b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=group) 125 | conv2 = tf.nn.relu(conv2_in) 126 | 127 | # lrn2 128 | # lrn(2, 2e-05, 0.75, name='norm2') 129 | radius = 2; 130 | alpha = 2e-05; 131 | beta = 0.75; 132 | bias = 1.0 133 | lrn2 = tf.nn.local_response_normalization(conv2, 134 | depth_radius=radius, 135 | alpha=alpha, 136 | beta=beta, 137 | bias=bias) 138 | 139 | # maxpool2 140 | # max_pool(3, 3, 2, 2, padding='VALID', name='pool2') 141 | k_h = 3; 142 | k_w = 3; 143 | s_h = 2; 144 | s_w = 2; 145 | padding = 'VALID' 146 | maxpool2 = tf.nn.max_pool(lrn2, ksize=[1, k_h, k_w, 1], strides=[1, s_h, s_w, 1], padding=padding) 147 | 148 | # conv3 149 | # conv(3, 3, 384, 1, 1, name='conv3') 150 | k_h = 3; 151 | k_w = 3; 152 | c_o = 384; 153 | s_h = 1; 154 | s_w = 1; 155 | group = 1 156 | conv3W = tf.Variable(net_data["conv3"][0]) 157 | conv3b = tf.Variable(net_data["conv3"][1]) 158 | conv3_in = conv(maxpool2, conv3W, conv3b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=group) 159 | conv3 = tf.nn.relu(conv3_in) 160 | 161 | # conv4 162 | # conv(3, 3, 384, 1, 1, group=2, name='conv4') 163 | k_h = 3; 164 | k_w = 3; 165 | c_o = 384; 166 | s_h = 1; 167 | s_w = 1; 168 | group = 2 169 | conv4W = tf.Variable(net_data["conv4"][0]) 170 | conv4b = tf.Variable(net_data["conv4"][1]) 171 | conv4_in = conv(conv3, conv4W, conv4b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=group) 172 | conv4 = tf.nn.relu(conv4_in) 173 | 174 | # conv5 175 | # conv(3, 3, 256, 1, 1, group=2, name='conv5') 176 | k_h = 3; 177 | k_w = 3; 178 | c_o = 256; 179 | s_h = 1; 180 | s_w = 1; 181 | group = 2 182 | conv5W = tf.Variable(net_data["conv5"][0]) 183 | conv5b = tf.Variable(net_data["conv5"][1]) 184 | conv5_in = conv(conv4, conv5W, conv5b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=group) 185 | conv5 = tf.nn.relu(conv5_in) 186 | 187 | # maxpool5 188 | # max_pool(3, 3, 2, 2, padding='VALID', name='pool5') 189 | k_h = 3; 190 | k_w = 3; 191 | s_h = 2; 192 | s_w = 2; 193 | padding = 'VALID' 194 | maxpool5 = tf.nn.max_pool(conv5, ksize=[1, k_h, k_w, 1], strides=[1, s_h, s_w, 1], padding=padding) 195 | 196 | # fc6 197 | # fc(4096, name='fc6') 198 | fc6W = tf.Variable(net_data["fc6"][0]) 199 | fc6b = tf.Variable(net_data["fc6"][1]) 200 | fc6 = tf.nn.relu_layer(tf.reshape(maxpool5, [-1, int(prod(maxpool5.get_shape()[1:]))]), fc6W, fc6b) 201 | 202 | # fc7 203 | # fc(4096, name='fc7') 204 | fc7W = tf.Variable(net_data["fc7"][0]) 205 | fc7b = tf.Variable(net_data["fc7"][1]) 206 | fc7 = tf.nn.relu_layer(fc6, fc7W, fc7b) 207 | 208 | # fc8 209 | # fc(1000, relu=False, name='fc8') 210 | fc8W = tf.Variable(net_data["fc8"][0]) 211 | fc8b = tf.Variable(net_data["fc8"][1]) 212 | fc8 = tf.nn.xw_plus_b(fc7, fc8W, fc8b) 213 | 214 | # prob 215 | # softmax(name='prob')) 216 | prob = tf.nn.softmax(fc8) 217 | return x, prob 218 | 219 | 220 | ################################################################################ 221 | # def load_image_names(dir): 222 | # img_names = [] 223 | # for dir_path, dir_names, file_names in os.walk(dir): 224 | # for file_name in file_names: 225 | # img_name = dir_path + '/' + file_name 226 | # img = imread(img_name) 227 | # if len(img.shape) > 2: 228 | # print(img_name) 229 | # img_names.append(img_name) 230 | # return img_names 231 | 232 | def load_image_names(): 233 | img_names = [] 234 | with open('img_names.txt') as f: 235 | for img_name in f.readlines(): 236 | img_name = img_name.strip('\n') 237 | img_names.append(img_name) 238 | return img_names 239 | 240 | def load_images(image_names): 241 | imgs = [] 242 | for img_name in image_names: 243 | img = imread(img_name) 244 | img = (imresize(img, (227, 227))[:, :, :3]).astype(float32) 245 | img = img - mean(img) 246 | imgs.append(img) 247 | return imgs 248 | 249 | 250 | def download_image(url): 251 | tmp_img_path = 'img/tmp.jpg' 252 | urllib.urlretrieve(url, tmp_img_path) 253 | return tmp_img_path 254 | 255 | 256 | # Output: 257 | 258 | def main(): 259 | x, prob = build_model() 260 | # init = tf.initialize_all_variables() 261 | init = tf.global_variables_initializer() 262 | sess = tf.Session() 263 | sess.run(init) 264 | if not args.url == '': 265 | sample_image_names = [download_image(args.url)] 266 | else: 267 | image_names = load_image_names() 268 | sample_size = 20 269 | sample_image_names = sample(image_names, sample_size) 270 | 271 | imgs = load_images(sample_image_names) 272 | t = time.time() 273 | output = sess.run(prob, feed_dict={x: imgs}) 274 | 275 | fig, axes = plt.subplots(4, 10, figsize=(30, 12)) 276 | fig.subplots_adjust(hspace=0.2, wspace=0.1, left=.05, right=.95, top=.95, bottom=.05) 277 | 278 | if not os.path.exists(args.output_data_dir): 279 | os.mkdir(args.output_data_dir) 280 | 281 | for input_im_ind in range(output.shape[0]): 282 | inds = argsort(output)[input_im_ind, :] 283 | print("Image", input_im_ind, sample_image_names[input_im_ind]) 284 | top_classes = [] 285 | probabilities = [] 286 | for i in range(5): 287 | top_classes.append(class_names[inds[-1 - i]].split(',')[0]) 288 | probabilities.append(output[input_im_ind, inds[-1 - i]]) 289 | print(class_names[inds[-1 - i]], output[input_im_ind, inds[-1 - i]]) 290 | 291 | row = input_im_ind / 10 292 | col = input_im_ind % 10 293 | 294 | y_pos = np.arange(len(top_classes)) 295 | barh_axe = axes[row * 2 + 1, col] 296 | barh_axe.set_xlim(0, 1) 297 | barh_axe.set_yticks(y_pos) 298 | barh_axe.set_yticklabels(top_classes) 299 | barh_axe.barh(y_pos, probabilities, align='center', alpha=0.4, height=0.5) 300 | 301 | img = imread(sample_image_names[input_im_ind]) 302 | img_axe = axes[row * 2, col] 303 | img_axe.set_xticks([]) 304 | img_axe.set_yticks([]) 305 | img_axe.imshow(img) 306 | 307 | save_path = args.output_data_dir + '/classification.pdf' 308 | plt.savefig(save_path) 309 | os.system('open -a Preview.app -F ' + save_path) 310 | # plt.show() 311 | print(time.time() - t) 312 | 313 | 314 | if __name__ == '__main__': 315 | parser = argparse.ArgumentParser() 316 | parser.add_argument( 317 | '--input_data_dir', 318 | type=str, 319 | default='/Volumes/Transcend/dataset/Caltech256/256_ObjectCategories_resize', 320 | help='Directory to put the input data.' 321 | ) 322 | parser.add_argument( 323 | '--output_data_dir', 324 | type=str, 325 | default='/Volumes/Transcend/dataset/Caltech256/256_ObjectCategories_classification', 326 | help='Directory to put the output file.' 327 | ) 328 | parser.add_argument( 329 | '--url', 330 | type=str, 331 | default='', 332 | help='Image URL' 333 | ) 334 | args = parser.parse_args() 335 | print(args) 336 | main() 337 | -------------------------------------------------------------------------------- /visual_search/myalexnet_feature.py: -------------------------------------------------------------------------------- 1 | # -*- coding=utf-8 -*- 2 | from __future__ import absolute_import 3 | from __future__ import print_function 4 | import argparse 5 | from numpy import * 6 | import os 7 | from pylab import * 8 | import numpy as np 9 | import matplotlib.pyplot as plt 10 | import matplotlib.cbook as cbook 11 | import time 12 | from scipy.misc import imread 13 | from scipy.misc import imresize 14 | import matplotlib.image as mpimg 15 | from scipy.ndimage import filters 16 | import urllib 17 | from numpy import random 18 | from scipy.misc import imresize 19 | from sklearn import preprocessing 20 | import tensorflow as tf 21 | 22 | from caffe_classes import class_names 23 | 24 | args = None 25 | 26 | train_x = zeros((1, 227, 227, 3)).astype(float32) 27 | train_y = zeros((1, 1000)) 28 | xdim = train_x.shape[1:] 29 | ydim = train_y.shape[1] 30 | 31 | ################################################################################ 32 | 33 | # (self.feed('data') 34 | # .conv(11, 11, 96, 4, 4, padding='VALID', name='conv1') 35 | # .lrn(2, 2e-05, 0.75, name='norm1') 36 | # .max_pool(3, 3, 2, 2, padding='VALID', name='pool1') 37 | # .conv(5, 5, 256, 1, 1, group=2, name='conv2') 38 | # .lrn(2, 2e-05, 0.75, name='norm2') 39 | # .max_pool(3, 3, 2, 2, padding='VALID', name='pool2') 40 | # .conv(3, 3, 384, 1, 1, name='conv3') 41 | # .conv(3, 3, 384, 1, 1, group=2, name='conv4') 42 | # .conv(3, 3, 256, 1, 1, group=2, name='conv5') 43 | # .fc(4096, name='fc6') 44 | # .fc(4096, name='fc7') 45 | # .fc(1000, relu=False, name='fc8') 46 | # .softmax(name='prob')) 47 | 48 | 49 | net_data = load("bvlc_alexnet.npy").item() 50 | 51 | 52 | def conv(input, kernel, biases, k_h, k_w, c_o, s_h, s_w, padding="VALID", group=1): 53 | '''From https://github.com/ethereon/caffe-tensorflow 54 | ''' 55 | c_i = input.get_shape()[-1] 56 | assert c_i % group == 0 57 | assert c_o % group == 0 58 | convolve = lambda i, k: tf.nn.conv2d(i, k, [1, s_h, s_w, 1], padding=padding) 59 | 60 | if group == 1: 61 | conv = convolve(input, kernel) 62 | else: 63 | input_groups = tf.split(3, group, input) 64 | kernel_groups = tf.split(3, group, kernel) 65 | output_groups = [convolve(i, k) for i, k in zip(input_groups, kernel_groups)] 66 | conv = tf.concat(3, output_groups) 67 | return tf.reshape(tf.nn.bias_add(conv, biases), [-1] + conv.get_shape().as_list()[1:]) 68 | 69 | 70 | def initModel(): 71 | x = tf.placeholder(tf.float32, (None,) + xdim) 72 | # conv1 73 | # conv(11, 11, 96, 4, 4, padding='VALID', name='conv1') 74 | k_h = 11; 75 | k_w = 11; 76 | c_o = 96; 77 | s_h = 4; 78 | s_w = 4 79 | conv1W = tf.Variable(net_data["conv1"][0]) 80 | conv1b = tf.Variable(net_data["conv1"][1]) 81 | conv1_in = conv(x, conv1W, conv1b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=1) 82 | conv1 = tf.nn.relu(conv1_in) 83 | 84 | # lrn1 85 | # lrn(2, 2e-05, 0.75, name='norm1') 86 | radius = 2; 87 | alpha = 2e-05; 88 | beta = 0.75; 89 | bias = 1.0 90 | lrn1 = tf.nn.local_response_normalization(conv1, 91 | depth_radius=radius, 92 | alpha=alpha, 93 | beta=beta, 94 | bias=bias) 95 | 96 | # maxpool1 97 | # max_pool(3, 3, 2, 2, padding='VALID', name='pool1') 98 | k_h = 3; 99 | k_w = 3; 100 | s_h = 2; 101 | s_w = 2; 102 | padding = 'VALID' 103 | maxpool1 = tf.nn.max_pool(lrn1, ksize=[1, k_h, k_w, 1], strides=[1, s_h, s_w, 1], padding=padding) 104 | 105 | # conv2 106 | # conv(5, 5, 256, 1, 1, group=2, name='conv2') 107 | k_h = 5; 108 | k_w = 5; 109 | c_o = 256; 110 | s_h = 1; 111 | s_w = 1; 112 | group = 2 113 | conv2W = tf.Variable(net_data["conv2"][0]) 114 | conv2b = tf.Variable(net_data["conv2"][1]) 115 | conv2_in = conv(maxpool1, conv2W, conv2b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=group) 116 | conv2 = tf.nn.relu(conv2_in) 117 | 118 | # lrn2 119 | # lrn(2, 2e-05, 0.75, name='norm2') 120 | radius = 2; 121 | alpha = 2e-05; 122 | beta = 0.75; 123 | bias = 1.0 124 | lrn2 = tf.nn.local_response_normalization(conv2, 125 | depth_radius=radius, 126 | alpha=alpha, 127 | beta=beta, 128 | bias=bias) 129 | 130 | # maxpool2 131 | # max_pool(3, 3, 2, 2, padding='VALID', name='pool2') 132 | k_h = 3; 133 | k_w = 3; 134 | s_h = 2; 135 | s_w = 2; 136 | padding = 'VALID' 137 | maxpool2 = tf.nn.max_pool(lrn2, ksize=[1, k_h, k_w, 1], strides=[1, s_h, s_w, 1], padding=padding) 138 | 139 | # conv3 140 | # conv(3, 3, 384, 1, 1, name='conv3') 141 | k_h = 3; 142 | k_w = 3; 143 | c_o = 384; 144 | s_h = 1; 145 | s_w = 1; 146 | group = 1 147 | conv3W = tf.Variable(net_data["conv3"][0]) 148 | conv3b = tf.Variable(net_data["conv3"][1]) 149 | conv3_in = conv(maxpool2, conv3W, conv3b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=group) 150 | conv3 = tf.nn.relu(conv3_in) 151 | 152 | # conv4 153 | # conv(3, 3, 384, 1, 1, group=2, name='conv4') 154 | k_h = 3; 155 | k_w = 3; 156 | c_o = 384; 157 | s_h = 1; 158 | s_w = 1; 159 | group = 2 160 | conv4W = tf.Variable(net_data["conv4"][0]) 161 | conv4b = tf.Variable(net_data["conv4"][1]) 162 | conv4_in = conv(conv3, conv4W, conv4b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=group) 163 | conv4 = tf.nn.relu(conv4_in) 164 | 165 | # conv5 166 | # conv(3, 3, 256, 1, 1, group=2, name='conv5') 167 | k_h = 3; 168 | k_w = 3; 169 | c_o = 256; 170 | s_h = 1; 171 | s_w = 1; 172 | group = 2 173 | conv5W = tf.Variable(net_data["conv5"][0]) 174 | conv5b = tf.Variable(net_data["conv5"][1]) 175 | conv5_in = conv(conv4, conv5W, conv5b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=group) 176 | conv5 = tf.nn.relu(conv5_in) 177 | 178 | # maxpool5 179 | # max_pool(3, 3, 2, 2, padding='VALID', name='pool5') 180 | k_h = 3; 181 | k_w = 3; 182 | s_h = 2; 183 | s_w = 2; 184 | padding = 'VALID' 185 | maxpool5 = tf.nn.max_pool(conv5, ksize=[1, k_h, k_w, 1], strides=[1, s_h, s_w, 1], padding=padding) 186 | 187 | # fc6 188 | # fc(4096, name='fc6') 189 | fc6W = tf.Variable(net_data["fc6"][0]) 190 | fc6b = tf.Variable(net_data["fc6"][1]) 191 | fc6 = tf.nn.relu_layer(tf.reshape(maxpool5, [-1, int(prod(maxpool5.get_shape()[1:]))]), fc6W, fc6b) 192 | 193 | # fc7 194 | # fc(4096, name='fc7') 195 | fc7W = tf.Variable(net_data["fc7"][0]) 196 | fc7b = tf.Variable(net_data["fc7"][1]) 197 | fc7 = tf.nn.relu_layer(fc6, fc7W, fc7b) 198 | 199 | # fc8 200 | # fc(1000, relu=False, name='fc8') 201 | fc8W = tf.Variable(net_data["fc8"][0]) 202 | fc8b = tf.Variable(net_data["fc8"][1]) 203 | fc8 = tf.nn.xw_plus_b(fc7, fc8W, fc8b) 204 | 205 | # prob 206 | # softmax(name='prob')) 207 | prob = tf.nn.softmax(fc8) 208 | 209 | return (x, fc6) 210 | 211 | 212 | def load_image_names(dir): 213 | img_names = [] 214 | for dir_path, dir_names, file_names in os.walk(dir): 215 | for file_name in file_names: 216 | img_name = dir_path + '/' + file_name 217 | img = imread(img_name) 218 | if len(img.shape) > 2: 219 | print(img_name) 220 | img_names.append(img_name) 221 | return img_names 222 | 223 | 224 | def load_images(img_names): 225 | imgs = [] 226 | for img_name in img_names: 227 | img = imread(img_name) 228 | new_img = (imresize(img, (227, 227))[:, :, :3]).astype(float32) 229 | new_img = new_img - mean(new_img) 230 | imgs.append(new_img) 231 | return imgs 232 | 233 | 234 | def main(): 235 | x, fc6 = initModel() 236 | init = tf.global_variables_initializer() 237 | sess = tf.Session() 238 | sess.run(init) 239 | img_names = load_image_names(args.input_data_dir) 240 | 241 | with open(args.output_image_name_file, 'w') as img_names_file: 242 | for img_name in img_names: 243 | img_names_file.write(img_name + '\n') 244 | 245 | t = time.time() 246 | # 图像太多了，必须分批次 247 | batch_size = 100 248 | features = [] 249 | 250 | with open(args.output_feature_file, 'w') as output_file: 251 | for i in range(0, int(math.ceil(len(img_names) / (batch_size * 1.0)))): 252 | print('batch: %d' % i) 253 | if (i + 1) * batch_size < len(img_names): 254 | img_names_batch = img_names[i * batch_size:(i + 1) * batch_size] 255 | else: 256 | img_names_batch = img_names[i * batch_size:len(img_names)] 257 | img_batch = load_images(img_names_batch) 258 | output = sess.run(fc6, feed_dict={x: img_batch}) 259 | features.append(output) 260 | features = np.vstack(features) 261 | # binarizer = preprocessing.Binarizer().fit(features) 262 | # features = binarizer.transform(features) 263 | np.save(output_file, features) 264 | 265 | # with open('fc6.npy', 'w') as output_file: 266 | # for i in range(0, int(math.ceil(len(imgs) / (batch_size * 1.0)))): 267 | # print('batch: %d' % i) 268 | # if (i + 1) * batch_size < len(imgs): 269 | # img_batch = imgs[i * batch_size:(i + 1) * batch_size] 270 | # else: 271 | # img_batch = imgs[i * batch_size: len(imgs)] 272 | # output = sess.run(fc6, feed_dict={x: img_batch}) 273 | # features.append(output) 274 | # features = np.vstack(features) 275 | # np.save(output_file, features) 276 | 277 | print(time.time() - t) 278 | 279 | 280 | if __name__ == '__main__': 281 | parser = argparse.ArgumentParser() 282 | parser.add_argument( 283 | '--input_data_dir', 284 | type=str, 285 | help='Directory to put the input data.' 286 | ) 287 | parser.add_argument( 288 | '--output_feature_file', 289 | type=str, 290 | help='Output features path.' 291 | ) 292 | parser.add_argument( 293 | '--output_image_name_file', 294 | type=str, 295 | help='Output image names path.' 296 | ) 297 | args = parser.parse_args() 298 | main() 299 | -------------------------------------------------------------------------------- /visual_search/tools/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GYXie/visual-search/fcc5ffd0004dcdbf7a336504fcc74032bb31c3f9/visual_search/tools/__init__.py -------------------------------------------------------------------------------- /visual_search/tools/gallery.py: -------------------------------------------------------------------------------- 1 | # -*- coding=utf-8 -*- 2 | from __future__ import absolute_import 3 | from __future__ import print_function 4 | import argparse 5 | import os.path 6 | import math 7 | import matplotlib.pyplot as plt 8 | import numpy as np 9 | from scipy.misc import imread 10 | 11 | args = None 12 | 13 | 14 | def main(): 15 | args.input_data_dir = os.path.abspath(args.input_data_dir) 16 | if not os.path.exists(args.output_data_dir): 17 | os.mkdir(args.output_data_dir) 18 | for dir_path, dir_names, file_names in os.walk(args.input_data_dir): 19 | if len(file_names) > 0: 20 | print(dir_path) 21 | rows = int(math.ceil(len(file_names) / 6.0)) 22 | print(rows) 23 | fig, axes = plt.subplots(4, 12, subplot_kw={'xticks': [], 'yticks': []}) 24 | fig.subplots_adjust(hspace=0.01, wspace=0.01) 25 | for ax, file_name in zip(axes.flat, file_names): 26 | print(file_name) 27 | img = imread(dir_path + '/' + file_name) 28 | ax.imshow(img) 29 | # ax.set_title(os.path.splitext(file_name)[0].replace('.227x227', '')) 30 | plt.savefig(args.output_data_dir + dir_path.replace(args.input_data_dir, '') + '.pdf') 31 | 32 | 33 | if __name__ == '__main__': 34 | parser = argparse.ArgumentParser() 35 | parser.add_argument( 36 | '--input_data_dir', 37 | type=str, 38 | default='/Volumes/Transcend/dataset/Caltech256/256_ObjectCategories_resize', 39 | help='Directory to put the input data.' 40 | ) 41 | parser.add_argument( 42 | '--output_data_dir', 43 | type=str, 44 | default='/Volumes/Transcend/dataset/Caltech256/256_ObjectCategories_plot', 45 | help='Directory to put the output file.' 46 | ) 47 | args = parser.parse_args() 48 | print(args) 49 | main() 50 | # methods = [None, 'none', 'nearest', 'bilinear', 'bicubic', 'spline16', 51 | # 'spline36', 'hanning', 'hamming', 'hermite', 'kaiser', 'quadric', 52 | # 'catrom', 'gaussian', 'bessel', 'mitchell', 'sinc', 'lanczos'] 53 | # 54 | # grid = np.random.rand(4, 4) 55 | # 56 | # fig, axes = plt.subplots(3, 6, figsize=(12, 6), 57 | # subplot_kw={'xticks': [], 'yticks': []}) 58 | # 59 | # fig.subplots_adjust(hspace=0.3, wspace=0.05) 60 | # 61 | # for ax, interp_method in zip(axes.flat, methods): 62 | # ax.imshow(grid, interpolation=interp_method) 63 | # ax.set_title(interp_method) 64 | # 65 | # plt.show() 66 | -------------------------------------------------------------------------------- /visual_search/tools/image_crop.py: -------------------------------------------------------------------------------- 1 | # -*- coding=utf-8 -*- 2 | from __future__ import absolute_import 3 | from __future__ import print_function 4 | import argparse 5 | import matplotlib.pyplot as plt 6 | import numpy as np 7 | from scipy import ndimage 8 | from scipy.misc import imread 9 | from scipy import misc 10 | from PIL import Image, ImageDraw, ImageFont 11 | 12 | args = None 13 | 14 | def main(): 15 | image = Image.open(args.input_path) 16 | width, height = image.size 17 | box = (width * 0.1, height * 0.1, width * 0.9, height * 0.9) 18 | crop = image.crop(box) 19 | print(crop.size) 20 | image = Image.new('RGBA', crop.size) 21 | box = (0, 0, crop.size[0], crop.size[1]) 22 | image.paste(crop, box) 23 | image.save(args.output_path) 24 | 25 | 26 | if __name__ == '__main__': 27 | parser = argparse.ArgumentParser() 28 | parser.add_argument( 29 | '--input_path', 30 | type=str, 31 | default='g.jpg', 32 | help='Image file path.' 33 | ) 34 | parser.add_argument( 35 | '--output_path', 36 | type=str, 37 | default='g_crop.jpg', 38 | help='Output file path.' 39 | ) 40 | args = parser.parse_args() 41 | print(args) 42 | main() 43 | -------------------------------------------------------------------------------- /visual_search/tools/image_mirror.py: -------------------------------------------------------------------------------- 1 | # -*- coding=utf-8 -*- 2 | from __future__ import absolute_import 3 | from __future__ import print_function 4 | import argparse 5 | import matplotlib.pyplot as plt 6 | import numpy as np 7 | from scipy import ndimage 8 | from scipy.misc import imread 9 | from scipy import misc 10 | from scipy.misc import imread 11 | import matplotlib.cbook as cbook 12 | from PIL import Image, ImageOps 13 | 14 | args = None 15 | 16 | def main(): 17 | image = Image.open(args.input_path) 18 | image = ImageOps.mirror(image) 19 | # data = np.asarray(image) 20 | # print(type(data)) 21 | # print(type(data[0,0,0])) 22 | # print(data.shape) 23 | image.save(args.output_path) 24 | 25 | 26 | if __name__ == '__main__': 27 | parser = argparse.ArgumentParser() 28 | parser.add_argument( 29 | '--input_path', 30 | type=str, 31 | default='./g.jpg', 32 | help='Image file path.' 33 | ) 34 | parser.add_argument( 35 | '--output_path', 36 | type=str, 37 | default='g_mirror.jpg', 38 | help='Output file path.' 39 | ) 40 | args = parser.parse_args() 41 | print(args) 42 | main() 43 | -------------------------------------------------------------------------------- /visual_search/tools/image_resize.py: -------------------------------------------------------------------------------- 1 | # -*- coding=utf-8 -*- 2 | from __future__ import absolute_import 3 | from __future__ import print_function 4 | import argparse 5 | import os.path 6 | from scipy import misc 7 | from scipy.misc import imread 8 | 9 | args = None 10 | 11 | 12 | def main(): 13 | print(args) 14 | for dir_path, dir_names, file_names in os.walk(args.input_data_dir): 15 | # dir_path is a string, the path to the directory 16 | # dir_names is a list of the names of the subdirectories in dir_path (excluding '.' and '..') 17 | # file_names is a list of the names of the non-directory files in dir_path 18 | dir_absolute_path = args.output_data_dir + dir_path.replace(args.input_data_dir, '') 19 | if not os.path.exists(dir_absolute_path): 20 | os.mkdir(dir_absolute_path) 21 | for file_name in file_names: 22 | # Split the pathname path into a pair (root, ext) such that root + ext == path, and ext is empty or begins 23 | # with a period and contains at most one period. 24 | (root, ext) = os.path.splitext(file_name) 25 | new_file_name = '%s/%s.%dx%d%s' % ( 26 | dir_absolute_path, root, args.width, args.height, ext) 27 | print(new_file_name) 28 | if not os.path.exists(new_file_name): 29 | img = imread(dir_path + '/' + file_name) 30 | # type(img) = ndarray, https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.html 31 | (width, height) = img.shape[0:2] 32 | if width > height: 33 | size = (args.width, height * args.width / width) 34 | else: 35 | size = (width * args.height / height, args.height) 36 | new_img = misc.imresize(img, size) 37 | misc.imsave(new_file_name, new_img) 38 | 39 | 40 | if __name__ == '__main__': 41 | """Resize the images in [input_dir] and save them in [output_dir] 42 | """ 43 | parser = argparse.ArgumentParser() 44 | parser.add_argument( 45 | '--input_data_dir', 46 | type=str, 47 | help='Directory to put the input data.' 48 | ) 49 | parser.add_argument( 50 | '--output_data_dir', 51 | type=str, 52 | help='Directory to put the output data.' 53 | ) 54 | 55 | parser.add_argument( 56 | '--width', 57 | type=int, 58 | default=227, 59 | help='Target image width.' 60 | ) 61 | 62 | parser.add_argument( 63 | '--height', 64 | type=int, 65 | default=227, 66 | help='Target image height.' 67 | ) 68 | args = parser.parse_args() 69 | main() 70 | -------------------------------------------------------------------------------- /visual_search/tools/image_rotate.py: -------------------------------------------------------------------------------- 1 | # -*- coding=utf-8 -*- 2 | from __future__ import absolute_import 3 | from __future__ import print_function 4 | import argparse 5 | import matplotlib.pyplot as plt 6 | import numpy as np 7 | from scipy import ndimage 8 | from scipy.misc import imread 9 | from scipy import misc 10 | 11 | args = None 12 | 13 | # https://github.com/tiagopereira/python_tips/wiki/Scipy:-image-rotation 14 | def main(): 15 | img = imread(args.input_path) 16 | img = ndimage.rotate(img, args.angle, mode=args.mode) 17 | misc.imsave(args.output_path, img) 18 | 19 | 20 | 21 | if __name__ == '__main__': 22 | parser = argparse.ArgumentParser() 23 | parser.add_argument( 24 | '--input_path', 25 | type=str, 26 | default='g.jpg', 27 | help='Image file path.' 28 | ) 29 | parser.add_argument( 30 | '--output_path', 31 | type=str, 32 | default='g_rotate.jpg', 33 | help='Output file path.' 34 | ) 35 | parser.add_argument( 36 | '--angle', 37 | type=float, 38 | default='15', 39 | help='Rotate angle.' 40 | ) 41 | parser.add_argument( 42 | '--mode', 43 | type=str, 44 | default='nearest', 45 | help='Rotate mode.' 46 | ) 47 | args = parser.parse_args() 48 | print(args) 49 | main() 50 | -------------------------------------------------------------------------------- /visual_search/tools/image_watermark_text.py: -------------------------------------------------------------------------------- 1 | # -*- coding=utf-8 -*- 2 | from __future__ import absolute_import 3 | from __future__ import print_function 4 | import argparse 5 | import matplotlib.pyplot as plt 6 | import numpy as np 7 | from scipy import ndimage 8 | from scipy.misc import imread 9 | from scipy import misc 10 | from scipy.misc import imread 11 | import matplotlib.cbook as cbook 12 | from PIL import Image, ImageDraw, ImageFont 13 | 14 | args = None 15 | 16 | 17 | # 参考: http://www.thecodingcouple.com/watermark-images-python-pillow-pil/ 18 | def main(): 19 | image = Image.open(args.input_path) 20 | width, height = image.size 21 | draw = ImageDraw.Draw(image) 22 | text = "Deep Learning" 23 | 24 | font = ImageFont.truetype('/Library/Fonts/Arial.ttf', 30) 25 | text_width, text_height = draw.textsize(text, font) 26 | 27 | margin = 10 28 | x = width - text_width - margin 29 | y = height - text_height - margin 30 | draw.text((x, y), text, font=font, fill=(255, 0, 0)) 31 | image.save(args.output_path) 32 | 33 | 34 | if __name__ == '__main__': 35 | parser = argparse.ArgumentParser() 36 | parser.add_argument( 37 | '--input_path', 38 | type=str, 39 | default='./g.jpg', 40 | help='Image file path.' 41 | ) 42 | parser.add_argument( 43 | '--output_path', 44 | type=str, 45 | default='g_watermark_text.jpg', 46 | help='Output file path.' 47 | ) 48 | args = parser.parse_args() 49 | print(args) 50 | main() 51 | -------------------------------------------------------------------------------- /visual_search/visual_search.py: -------------------------------------------------------------------------------- 1 | # coding=utf-8 2 | from __future__ import absolute_import 3 | from __future__ import print_function 4 | 5 | import argparse 6 | import time 7 | import urllib 8 | import os 9 | 10 | import matplotlib.pyplot as plt 11 | import numpy as np 12 | import tensorflow as tf 13 | from PIL import Image, ImageOps, ImageDraw, ImageFont 14 | from numpy import * 15 | from scipy import ndimage 16 | from scipy.misc import imread 17 | from scipy.misc import imresize 18 | from scipy.spatial import distance 19 | from sklearn import preprocessing 20 | 21 | args = None 22 | train_x = zeros((1, 227, 227, 3)).astype(float32) 23 | train_y = zeros((1, 1000)) 24 | xdim = train_x.shape[1:] 25 | ydim = train_y.shape[1] 26 | 27 | net_data = load("bvlc_alexnet.npy").item() 28 | 29 | 30 | def conv(input, kernel, biases, k_h, k_w, c_o, s_h, s_w, padding="VALID", group=1): 31 | '''From https://github.com/ethereon/caffe-tensorflow 32 | ''' 33 | c_i = input.get_shape()[-1] 34 | assert c_i % group == 0 35 | assert c_o % group == 0 36 | convolve = lambda i, k: tf.nn.conv2d(i, k, [1, s_h, s_w, 1], padding=padding) 37 | 38 | if group == 1: 39 | conv = convolve(input, kernel) 40 | else: 41 | input_groups = tf.split(3, group, input) 42 | kernel_groups = tf.split(3, group, kernel) 43 | output_groups = [convolve(i, k) for i, k in zip(input_groups, kernel_groups)] 44 | conv = tf.concat(3, output_groups) 45 | return tf.reshape(tf.nn.bias_add(conv, biases), [-1] + conv.get_shape().as_list()[1:]) 46 | 47 | 48 | def initModel(): 49 | x = tf.placeholder(tf.float32, (None,) + xdim) 50 | # conv1 51 | # conv(11, 11, 96, 4, 4, padding='VALID', name='conv1') 52 | k_h = 11; 53 | k_w = 11; 54 | c_o = 96; 55 | s_h = 4; 56 | s_w = 4 57 | conv1W = tf.Variable(net_data["conv1"][0]) 58 | conv1b = tf.Variable(net_data["conv1"][1]) 59 | conv1_in = conv(x, conv1W, conv1b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=1) 60 | conv1 = tf.nn.relu(conv1_in) 61 | 62 | # lrn1 63 | # lrn(2, 2e-05, 0.75, name='norm1') 64 | radius = 2; 65 | alpha = 2e-05; 66 | beta = 0.75; 67 | bias = 1.0 68 | lrn1 = tf.nn.local_response_normalization(conv1, 69 | depth_radius=radius, 70 | alpha=alpha, 71 | beta=beta, 72 | bias=bias) 73 | 74 | # maxpool1 75 | # max_pool(3, 3, 2, 2, padding='VALID', name='pool1') 76 | k_h = 3; 77 | k_w = 3; 78 | s_h = 2; 79 | s_w = 2; 80 | padding = 'VALID' 81 | maxpool1 = tf.nn.max_pool(lrn1, ksize=[1, k_h, k_w, 1], strides=[1, s_h, s_w, 1], padding=padding) 82 | 83 | # conv2 84 | # conv(5, 5, 256, 1, 1, group=2, name='conv2') 85 | k_h = 5; 86 | k_w = 5; 87 | c_o = 256; 88 | s_h = 1; 89 | s_w = 1; 90 | group = 2 91 | conv2W = tf.Variable(net_data["conv2"][0]) 92 | conv2b = tf.Variable(net_data["conv2"][1]) 93 | conv2_in = conv(maxpool1, conv2W, conv2b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=group) 94 | conv2 = tf.nn.relu(conv2_in) 95 | 96 | # lrn2 97 | # lrn(2, 2e-05, 0.75, name='norm2') 98 | radius = 2; 99 | alpha = 2e-05; 100 | beta = 0.75; 101 | bias = 1.0 102 | lrn2 = tf.nn.local_response_normalization(conv2, 103 | depth_radius=radius, 104 | alpha=alpha, 105 | beta=beta, 106 | bias=bias) 107 | 108 | # maxpool2 109 | # max_pool(3, 3, 2, 2, padding='VALID', name='pool2') 110 | k_h = 3; 111 | k_w = 3; 112 | s_h = 2; 113 | s_w = 2; 114 | padding = 'VALID' 115 | maxpool2 = tf.nn.max_pool(lrn2, ksize=[1, k_h, k_w, 1], strides=[1, s_h, s_w, 1], padding=padding) 116 | 117 | # conv3 118 | # conv(3, 3, 384, 1, 1, name='conv3') 119 | k_h = 3; 120 | k_w = 3; 121 | c_o = 384; 122 | s_h = 1; 123 | s_w = 1; 124 | group = 1 125 | conv3W = tf.Variable(net_data["conv3"][0]) 126 | conv3b = tf.Variable(net_data["conv3"][1]) 127 | conv3_in = conv(maxpool2, conv3W, conv3b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=group) 128 | conv3 = tf.nn.relu(conv3_in) 129 | 130 | # conv4 131 | # conv(3, 3, 384, 1, 1, group=2, name='conv4') 132 | k_h = 3; 133 | k_w = 3; 134 | c_o = 384; 135 | s_h = 1; 136 | s_w = 1; 137 | group = 2 138 | conv4W = tf.Variable(net_data["conv4"][0]) 139 | conv4b = tf.Variable(net_data["conv4"][1]) 140 | conv4_in = conv(conv3, conv4W, conv4b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=group) 141 | conv4 = tf.nn.relu(conv4_in) 142 | 143 | # conv5 144 | # conv(3, 3, 256, 1, 1, group=2, name='conv5') 145 | k_h = 3; 146 | k_w = 3; 147 | c_o = 256; 148 | s_h = 1; 149 | s_w = 1; 150 | group = 2 151 | conv5W = tf.Variable(net_data["conv5"][0]) 152 | conv5b = tf.Variable(net_data["conv5"][1]) 153 | conv5_in = conv(conv4, conv5W, conv5b, k_h, k_w, c_o, s_h, s_w, padding="SAME", group=group) 154 | conv5 = tf.nn.relu(conv5_in) 155 | 156 | # maxpool5 157 | # max_pool(3, 3, 2, 2, padding='VALID', name='pool5') 158 | k_h = 3; 159 | k_w = 3; 160 | s_h = 2; 161 | s_w = 2; 162 | padding = 'VALID' 163 | maxpool5 = tf.nn.max_pool(conv5, ksize=[1, k_h, k_w, 1], strides=[1, s_h, s_w, 1], padding=padding) 164 | 165 | # fc6 166 | # fc(4096, name='fc6') 167 | fc6W = tf.Variable(net_data["fc6"][0]) 168 | fc6b = tf.Variable(net_data["fc6"][1]) 169 | fc6 = tf.nn.relu_layer(tf.reshape(maxpool5, [-1, int(prod(maxpool5.get_shape()[1:]))]), fc6W, fc6b) 170 | 171 | # fc7 172 | # fc(4096, name='fc7') 173 | fc7W = tf.Variable(net_data["fc7"][0]) 174 | fc7b = tf.Variable(net_data["fc7"][1]) 175 | fc7 = tf.nn.relu_layer(fc6, fc7W, fc7b) 176 | 177 | # fc8 178 | # fc(1000, relu=False, name='fc8') 179 | fc8W = tf.Variable(net_data["fc8"][0]) 180 | fc8b = tf.Variable(net_data["fc8"][1]) 181 | fc8 = tf.nn.xw_plus_b(fc7, fc8W, fc8b) 182 | 183 | # prob 184 | # softmax(name='prob')) 185 | prob = tf.nn.softmax(fc8) 186 | 187 | return (x, fc6) 188 | 189 | 190 | def load_image(img_file_path): 191 | img = imread(img_file_path) 192 | img = (imresize(img, (227, 227))[:, :, :3]).astype(float32) 193 | img = img - mean(img) 194 | 195 | return img 196 | 197 | 198 | def extract_feature(imgs): 199 | x, fc6 = initModel() 200 | # init = tf.initialize_all_variables() 201 | init = tf.global_variables_initializer() 202 | sess = tf.Session() 203 | sess.run(init) 204 | return sess.run(fc6, feed_dict={x: imgs}) 205 | 206 | 207 | def load_image_names(): 208 | img_names = [] 209 | with open('img_names.txt') as f: 210 | for img_name in f.readlines(): 211 | img_name = img_name.strip('\n') 212 | img_names.append(img_name) 213 | return img_names 214 | 215 | 216 | def download_image(url): 217 | tmp_img_path = 'img/tmp.jpg' 218 | urllib.urlretrieve(url, tmp_img_path) 219 | return tmp_img_path 220 | 221 | 222 | def crop(np_img): 223 | image = Image.fromarray(np_img) 224 | width, height = image.size 225 | box = (width * 0.2, height * 0.2, width * 0.8, height * 0.8) 226 | crop = image.crop(box) 227 | image = Image.new('RGBA', crop.size) 228 | box = (0, 0, crop.size[0], crop.size[1]) 229 | image.paste(crop, box) 230 | return np.asarray(image) 231 | 232 | 233 | def rotate(np_img): 234 | img = ndimage.rotate(np_img, 10, mode='nearest') 235 | return img 236 | 237 | 238 | def watermark(np_img): 239 | image = Image.fromarray(np_img) 240 | width, height = image.size 241 | draw = ImageDraw.Draw(image) 242 | text = "Deep Learning" 243 | 244 | font = ImageFont.truetype('/Library/Fonts/Arial.ttf', 30) 245 | text_width, text_height = draw.textsize(text, font) 246 | 247 | margin = 10 248 | x = width - text_width - margin 249 | y = height / 2 - text_height - margin 250 | draw.text((x, y), text, font=font, fill=(255, 0, 0)) 251 | return np.asarray(image) 252 | 253 | 254 | def mirror(np_img): 255 | image = Image.fromarray(np_img) 256 | image = ImageOps.mirror(image) 257 | data = np.asarray(image) 258 | return data 259 | 260 | 261 | def image_normalize(imgs): 262 | new_images = [] 263 | for img in imgs: 264 | img = (imresize(img, (227, 227))[:, :, :3]).astype(float32) 265 | img -= mean(img) 266 | new_images.append(img) 267 | return new_images 268 | 269 | 270 | def main(): 271 | t = time.time() 272 | img = imread(args.img_file_path) 273 | imgs = [img, watermark(img), rotate(img), crop(img), mirror(img)] 274 | imgs_norm = image_normalize(imgs) 275 | dataset_features = np.load('fc6.npy') 276 | 277 | query_start = time.time() 278 | query_features = extract_feature(imgs_norm) 279 | binarizer = preprocessing.Binarizer().fit(query_features) 280 | query_features = binarizer.transform(query_features) 281 | print(dataset_features) 282 | # https://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.distance.cdist.html#scipy.spatial.distance.cdist 283 | cosine = distance.cdist(dataset_features, query_features, 'cosine') 284 | print(cosine.shape) 285 | dis = cosine 286 | inds_all = argsort(dis, axis=0) # 按列排序 https://docs.scipy.org/doc/numpy/reference/generated/numpy.argsort.html 287 | print('query cost: %f, dataset: %d, query: %d' % (time.time() - query_start, len(dataset_features), len(imgs))) 288 | img_names = load_image_names() 289 | fig, axes = plt.subplots(5, 11, figsize=(22, 10), subplot_kw={'xticks': [], 'yticks': []}) 290 | fig.subplots_adjust(hspace=0.15, wspace=0.01, left=.02, right=.98, top=.92, bottom=.08) 291 | titles = ['original', 'watermark', 'rotate', 'crop', 'mirror'] 292 | for i in range(len(imgs)): 293 | topK = [] 294 | inds = inds_all[:, i] 295 | # print(inds) 296 | for k in range(10): 297 | topK.append(img_names[inds[k]]) 298 | print(inds[k], dis[inds[k], i], img_names[inds[k]]) 299 | 300 | original = axes[i, 0] 301 | original.set_title(titles[i]) 302 | img = imgs[i] 303 | original.imshow(img) 304 | for j in range(10): 305 | ax = axes[i, j + 1] 306 | img = imread(topK[j]) 307 | ax.imshow(img) 308 | title = '%d : %f' % (j + 1, dis[inds[j], i]) 309 | ax.set_title(title) 310 | 311 | savePath = args.img_file_path + '_search_result.jpg' 312 | plt.savefig(savePath) 313 | print(time.time() - t) 314 | # os.system('open -a Preview.app -F ' + savePath) 315 | 316 | 317 | if __name__ == '__main__': 318 | parser = argparse.ArgumentParser() 319 | parser.add_argument( 320 | '--img_file_path', 321 | type=str, 322 | default='041_0010.jpg', 323 | help='Image file path.' 324 | ) 325 | parser.add_argument( 326 | '--img_url', 327 | type=str, 328 | default='', 329 | help='Image Url.' 330 | ) 331 | args = parser.parse_args() 332 | if not args.img_url == '': 333 | args.img_file_path = download_image(args.url) 334 | main() 335 | --------------------------------------------------------------------------------