├── .gitignore ├── images ├── llama.jpeg ├── zebra.jpeg └── sealion.jpeg ├── LICENSE ├── README.md ├── datagenerator.py ├── finetune.py ├── alexnet.py └── caffe_classes.py /.gitignore: -------------------------------------------------------------------------------- 1 | *.npy 2 | **/*.pyc 3 | -------------------------------------------------------------------------------- /images/llama.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kratzert/finetune_alexnet_with_tensorflow/HEAD/images/llama.jpeg -------------------------------------------------------------------------------- /images/zebra.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kratzert/finetune_alexnet_with_tensorflow/HEAD/images/zebra.jpeg -------------------------------------------------------------------------------- /images/sealion.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kratzert/finetune_alexnet_with_tensorflow/HEAD/images/sealion.jpeg -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | BSD 3-Clause License 2 | 3 | Copyright (c) 2017, Frederik Kratzert 4 | All rights reserved. 5 | 6 | Redistribution and use in source and binary forms, with or without 7 | modification, are permitted provided that the following conditions are met: 8 | 9 | * Redistributions of source code must retain the above copyright notice, this 10 | list of conditions and the following disclaimer. 11 | 12 | * Redistributions in binary form must reproduce the above copyright notice, 13 | this list of conditions and the following disclaimer in the documentation 14 | and/or other materials provided with the distribution. 15 | 16 | * Neither the name of the copyright holder nor the names of its 17 | contributors may be used to endorse or promote products derived from 18 | this software without specific prior written permission. 19 | 20 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 21 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 22 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 23 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 24 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 25 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 26 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 27 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 28 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 29 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ![](https://zenodo.org/badge/DOI/10.5281/zenodo.1037359.svg) 2 | 3 | # Finetune AlexNet with Tensorflow 4 | 5 | **Update 15.06.2016** 6 | 7 | I revised the entire code base to work with the new input pipeline coming with TensorFlow >= version 1.2rc0. You can find an explanation of the new input pipeline in a new [blog post](https://kratzert.github.io/2017/06/15/example-of-tensorflows-new-input-pipeline.html) You can use this code as before for finetuning AlexNet on your own dataset, only the dependency of OpenCV isn't necessary anymore. The old code can be found in [this past commit](https://github.com/kratzert/finetune_alexnet_with_tensorflow/tree/5d751d62eb4d7149f4e3fd465febf8f07d4cea9d). 8 | 9 | This repository contains all the code needed to finetune [AlexNet](http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf) on any arbitrary dataset. Beside the comments in the code itself, I also wrote an article which you can find [here](https://kratzert.github.io/2017/02/24/finetuning-alexnet-with-tensorflow.html) with further explanation. 10 | 11 | All you need are the pretrained weights, which you can find [here](http://www.cs.toronto.edu/~guerzhoy/tf_alexnet/) or convert yourself from the caffe library using [caffe-to-tensorflow](https://github.com/ethereon/caffe-tensorflow). 12 | If you convert them on your own, take a look on the structure of the `.npy` weights file (dict of dicts or dict of lists). 13 | 14 | **Note**: I won't write to much of an explanation here, as I already wrote a long article about the entire code on my blog. 15 | 16 | ## Requirements 17 | 18 | - Python 3 19 | - TensorFlow >= 1.2rc0 20 | - Numpy 21 | 22 | 23 | ## TensorBoard support 24 | 25 | The code has TensorFlows summaries implemented so that you can follow the training progress in TensorBoard. (--logdir in the config section of `finetune.py`) 26 | 27 | ## Content 28 | 29 | - `alexnet.py`: Class with the graph definition of the AlexNet. 30 | - `finetune.py`: Script to run the finetuning process. 31 | - `datagenerator.py`: Contains a wrapper class for the new input pipeline. 32 | - `caffe_classes.py`: List of the 1000 class names of ImageNet (copied from [here](http://www.cs.toronto.edu/~guerzhoy/tf_alexnet/)). 33 | - `validate_alexnet_on_imagenet.ipynb`: Notebook to test the correct implementation of AlexNet and the pretrained weights on some images from the ImageNet database. 34 | - `images/*`: contains three example images, needed for the notebook. 35 | 36 | ## Usage 37 | 38 | All you need to touch is the `finetune.py`, although I strongly recommend to take a look at the entire code of this repository. In the `finetune.py` script you will find a section of configuration settings you have to adapt on your problem. 39 | If you do not want to touch the code any further than necessary you have to provide two `.txt` files to the script (`train.txt` and `val.txt`). Each of them list the complete path to your train/val images together with the class number in the following structure. 40 | 41 | ``` 42 | Example train.txt: 43 | /path/to/train/image1.png 0 44 | /path/to/train/image2.png 1 45 | /path/to/train/image3.png 2 46 | /path/to/train/image4.png 0 47 | . 48 | . 49 | ``` 50 | were the first column is the path and the second the class label. 51 | 52 | The other option is that you bring your own method of loading images and providing batches of images and labels, but then you have to adapt the code on a few lines. 53 | -------------------------------------------------------------------------------- /datagenerator.py: -------------------------------------------------------------------------------- 1 | # Created on Wed May 31 14:48:46 2017 2 | # 3 | # @author: Frederik Kratzert 4 | 5 | """Containes a helper class for image input pipelines in tensorflow.""" 6 | 7 | import tensorflow as tf 8 | import numpy as np 9 | 10 | from tensorflow.contrib.data import Dataset 11 | from tensorflow.python.framework import dtypes 12 | from tensorflow.python.framework.ops import convert_to_tensor 13 | 14 | IMAGENET_MEAN = tf.constant([123.68, 116.779, 103.939], dtype=tf.float32) 15 | 16 | 17 | class ImageDataGenerator(object): 18 | """Wrapper class around the new Tensorflows dataset pipeline. 19 | 20 | Requires Tensorflow >= version 1.12rc0 21 | """ 22 | 23 | def __init__(self, txt_file, mode, batch_size, num_classes, shuffle=True, 24 | buffer_size=1000): 25 | """Create a new ImageDataGenerator. 26 | 27 | Recieves a path string to a text file, which consists of many lines, 28 | where each line has first a path string to an image and seperated by 29 | a space an integer, referring to the class number. Using this data, 30 | this class will create TensrFlow datasets, that can be used to train 31 | e.g. a convolutional neural network. 32 | 33 | Args: 34 | txt_file: Path to the text file. 35 | mode: Either 'training' or 'validation'. Depending on this value, 36 | different parsing functions will be used. 37 | batch_size: Number of images per batch. 38 | num_classes: Number of classes in the dataset. 39 | shuffle: Wether or not to shuffle the data in the dataset and the 40 | initial file list. 41 | buffer_size: Number of images used as buffer for TensorFlows 42 | shuffling of the dataset. 43 | 44 | Raises: 45 | ValueError: If an invalid mode is passed. 46 | 47 | """ 48 | self.txt_file = txt_file 49 | self.num_classes = num_classes 50 | 51 | # retrieve the data from the text file 52 | self._read_txt_file() 53 | 54 | # number of samples in the dataset 55 | self.data_size = len(self.labels) 56 | 57 | # initial shuffling of the file and label lists (together!) 58 | if shuffle: 59 | self._shuffle_lists() 60 | 61 | # convert lists to TF tensor 62 | self.img_paths = convert_to_tensor(self.img_paths, dtype=dtypes.string) 63 | self.labels = convert_to_tensor(self.labels, dtype=dtypes.int32) 64 | 65 | # create dataset 66 | data = Dataset.from_tensor_slices((self.img_paths, self.labels)) 67 | 68 | # distinguish between train/infer. when calling the parsing functions 69 | if mode == 'training': 70 | data = data.map(self._parse_function_train, num_threads=8, 71 | output_buffer_size=100*batch_size) 72 | 73 | elif mode == 'inference': 74 | data = data.map(self._parse_function_inference, num_threads=8, 75 | output_buffer_size=100*batch_size) 76 | 77 | else: 78 | raise ValueError("Invalid mode '%s'." % (mode)) 79 | 80 | # shuffle the first `buffer_size` elements of the dataset 81 | if shuffle: 82 | data = data.shuffle(buffer_size=buffer_size) 83 | 84 | # create a new dataset with batches of images 85 | data = data.batch(batch_size) 86 | 87 | self.data = data 88 | 89 | def _read_txt_file(self): 90 | """Read the content of the text file and store it into lists.""" 91 | self.img_paths = [] 92 | self.labels = [] 93 | with open(self.txt_file, 'r') as f: 94 | lines = f.readlines() 95 | for line in lines: 96 | items = line.split(' ') 97 | self.img_paths.append(items[0]) 98 | self.labels.append(int(items[1])) 99 | 100 | def _shuffle_lists(self): 101 | """Conjoined shuffling of the list of paths and labels.""" 102 | path = self.img_paths 103 | labels = self.labels 104 | permutation = np.random.permutation(self.data_size) 105 | self.img_paths = [] 106 | self.labels = [] 107 | for i in permutation: 108 | self.img_paths.append(path[i]) 109 | self.labels.append(labels[i]) 110 | 111 | def _parse_function_train(self, filename, label): 112 | """Input parser for samples of the training set.""" 113 | # convert label number into one-hot-encoding 114 | one_hot = tf.one_hot(label, self.num_classes) 115 | 116 | # load and preprocess the image 117 | img_string = tf.read_file(filename) 118 | img_decoded = tf.image.decode_png(img_string, channels=3) 119 | img_resized = tf.image.resize_images(img_decoded, [227, 227]) 120 | """ 121 | Dataaugmentation comes here. 122 | """ 123 | img_centered = tf.subtract(img_resized, IMAGENET_MEAN) 124 | 125 | # RGB -> BGR 126 | img_bgr = img_centered[:, :, ::-1] 127 | 128 | return img_bgr, one_hot 129 | 130 | def _parse_function_inference(self, filename, label): 131 | """Input parser for samples of the validation/test set.""" 132 | # convert label number into one-hot-encoding 133 | one_hot = tf.one_hot(label, self.num_classes) 134 | 135 | # load and preprocess the image 136 | img_string = tf.read_file(filename) 137 | img_decoded = tf.image.decode_png(img_string, channels=3) 138 | img_resized = tf.image.resize_images(img_decoded, [227, 227]) 139 | img_centered = tf.subtract(img_resized, IMAGENET_MEAN) 140 | 141 | # RGB -> BGR 142 | img_bgr = img_centered[:, :, ::-1] 143 | 144 | return img_bgr, one_hot 145 | -------------------------------------------------------------------------------- /finetune.py: -------------------------------------------------------------------------------- 1 | """Script to finetune AlexNet using Tensorflow. 2 | 3 | With this script you can finetune AlexNet as provided in the alexnet.py 4 | class on any given dataset. Specify the configuration settings at the 5 | beginning according to your problem. 6 | This script was written for TensorFlow >= version 1.2rc0 and comes with a blog 7 | post, which you can find here: 8 | 9 | https://kratzert.github.io/2017/02/24/finetuning-alexnet-with-tensorflow.html 10 | 11 | Author: Frederik Kratzert 12 | contact: f.kratzert(at)gmail.com 13 | """ 14 | 15 | import os 16 | 17 | import numpy as np 18 | import tensorflow as tf 19 | 20 | from alexnet import AlexNet 21 | from datagenerator import ImageDataGenerator 22 | from datetime import datetime 23 | from tensorflow.contrib.data import Iterator 24 | 25 | """ 26 | Configuration Part. 27 | """ 28 | 29 | # Path to the textfiles for the trainings and validation set 30 | train_file = '/path/to/train.txt' 31 | val_file = '/path/to/val.txt' 32 | 33 | # Learning params 34 | learning_rate = 0.01 35 | num_epochs = 10 36 | batch_size = 128 37 | 38 | # Network params 39 | dropout_rate = 0.5 40 | num_classes = 2 41 | train_layers = ['fc8', 'fc7', 'fc6'] 42 | 43 | # How often we want to write the tf.summary data to disk 44 | display_step = 20 45 | 46 | # Path for tf.summary.FileWriter and to store model checkpoints 47 | filewriter_path = "/tmp/finetune_alexnet/tensorboard" 48 | checkpoint_path = "/tmp/finetune_alexnet/checkpoints" 49 | 50 | """ 51 | Main Part of the finetuning Script. 52 | """ 53 | 54 | # Create parent path if it doesn't exist 55 | if not os.path.isdir(checkpoint_path): 56 | os.mkdir(checkpoint_path) 57 | 58 | # Place data loading and preprocessing on the cpu 59 | with tf.device('/cpu:0'): 60 | tr_data = ImageDataGenerator(train_file, 61 | mode='training', 62 | batch_size=batch_size, 63 | num_classes=num_classes, 64 | shuffle=True) 65 | val_data = ImageDataGenerator(val_file, 66 | mode='inference', 67 | batch_size=batch_size, 68 | num_classes=num_classes, 69 | shuffle=False) 70 | 71 | # create an reinitializable iterator given the dataset structure 72 | iterator = Iterator.from_structure(tr_data.data.output_types, 73 | tr_data.data.output_shapes) 74 | next_batch = iterator.get_next() 75 | 76 | # Ops for initializing the two different iterators 77 | training_init_op = iterator.make_initializer(tr_data.data) 78 | validation_init_op = iterator.make_initializer(val_data.data) 79 | 80 | # TF placeholder for graph input and output 81 | x = tf.placeholder(tf.float32, [batch_size, 227, 227, 3]) 82 | y = tf.placeholder(tf.float32, [batch_size, num_classes]) 83 | keep_prob = tf.placeholder(tf.float32) 84 | 85 | # Initialize model 86 | model = AlexNet(x, keep_prob, num_classes, train_layers) 87 | 88 | # Link variable to model output 89 | score = model.fc8 90 | 91 | # List of trainable variables of the layers we want to train 92 | var_list = [v for v in tf.trainable_variables() if v.name.split('/')[0] in train_layers] 93 | 94 | # Op for calculating the loss 95 | with tf.name_scope("cross_ent"): 96 | loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=score, 97 | labels=y)) 98 | 99 | # Train op 100 | with tf.name_scope("train"): 101 | # Get gradients of all trainable variables 102 | gradients = tf.gradients(loss, var_list) 103 | gradients = list(zip(gradients, var_list)) 104 | 105 | # Create optimizer and apply gradient descent to the trainable variables 106 | optimizer = tf.train.GradientDescentOptimizer(learning_rate) 107 | train_op = optimizer.apply_gradients(grads_and_vars=gradients) 108 | 109 | # Add gradients to summary 110 | for gradient, var in gradients: 111 | tf.summary.histogram(var.name + '/gradient', gradient) 112 | 113 | # Add the variables we train to the summary 114 | for var in var_list: 115 | tf.summary.histogram(var.name, var) 116 | 117 | # Add the loss to summary 118 | tf.summary.scalar('cross_entropy', loss) 119 | 120 | 121 | # Evaluation op: Accuracy of the model 122 | with tf.name_scope("accuracy"): 123 | correct_pred = tf.equal(tf.argmax(score, 1), tf.argmax(y, 1)) 124 | accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32)) 125 | 126 | # Add the accuracy to the summary 127 | tf.summary.scalar('accuracy', accuracy) 128 | 129 | # Merge all summaries together 130 | merged_summary = tf.summary.merge_all() 131 | 132 | # Initialize the FileWriter 133 | writer = tf.summary.FileWriter(filewriter_path) 134 | 135 | # Initialize an saver for store model checkpoints 136 | saver = tf.train.Saver() 137 | 138 | # Get the number of training/validation steps per epoch 139 | train_batches_per_epoch = int(np.floor(tr_data.data_size/batch_size)) 140 | val_batches_per_epoch = int(np.floor(val_data.data_size / batch_size)) 141 | 142 | # Start Tensorflow session 143 | with tf.Session() as sess: 144 | 145 | # Initialize all variables 146 | sess.run(tf.global_variables_initializer()) 147 | 148 | # Add the model graph to TensorBoard 149 | writer.add_graph(sess.graph) 150 | 151 | # Load the pretrained weights into the non-trainable layer 152 | model.load_initial_weights(sess) 153 | 154 | print("{} Start training...".format(datetime.now())) 155 | print("{} Open Tensorboard at --logdir {}".format(datetime.now(), 156 | filewriter_path)) 157 | 158 | # Loop over number of epochs 159 | for epoch in range(num_epochs): 160 | 161 | print("{} Epoch number: {}".format(datetime.now(), epoch+1)) 162 | 163 | # Initialize iterator with the training dataset 164 | sess.run(training_init_op) 165 | 166 | for step in range(train_batches_per_epoch): 167 | 168 | # get next batch of data 169 | img_batch, label_batch = sess.run(next_batch) 170 | 171 | # And run the training op 172 | sess.run(train_op, feed_dict={x: img_batch, 173 | y: label_batch, 174 | keep_prob: dropout_rate}) 175 | 176 | # Generate summary with the current batch of data and write to file 177 | if step % display_step == 0: 178 | s = sess.run(merged_summary, feed_dict={x: img_batch, 179 | y: label_batch, 180 | keep_prob: 1.}) 181 | 182 | writer.add_summary(s, epoch*train_batches_per_epoch + step) 183 | 184 | # Validate the model on the entire validation set 185 | print("{} Start validation".format(datetime.now())) 186 | sess.run(validation_init_op) 187 | test_acc = 0. 188 | test_count = 0 189 | for _ in range(val_batches_per_epoch): 190 | 191 | img_batch, label_batch = sess.run(next_batch) 192 | acc = sess.run(accuracy, feed_dict={x: img_batch, 193 | y: label_batch, 194 | keep_prob: 1.}) 195 | test_acc += acc 196 | test_count += 1 197 | test_acc /= test_count 198 | print("{} Validation Accuracy = {:.4f}".format(datetime.now(), 199 | test_acc)) 200 | print("{} Saving checkpoint of model...".format(datetime.now())) 201 | 202 | # save checkpoint of the model 203 | checkpoint_name = os.path.join(checkpoint_path, 204 | 'model_epoch'+str(epoch+1)+'.ckpt') 205 | save_path = saver.save(sess, checkpoint_name) 206 | 207 | print("{} Model checkpoint saved at {}".format(datetime.now(), 208 | checkpoint_name)) 209 | -------------------------------------------------------------------------------- /alexnet.py: -------------------------------------------------------------------------------- 1 | """This is an TensorFLow implementation of AlexNet by Alex Krizhevsky at all. 2 | 3 | Paper: 4 | (http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf) 5 | 6 | Explanation can be found in my blog post: 7 | https://kratzert.github.io/2017/02/24/finetuning-alexnet-with-tensorflow.html 8 | 9 | This script enables finetuning AlexNet on any given Dataset with any number of 10 | classes. The structure of this script is strongly inspired by the fast.ai 11 | Deep Learning class by Jeremy Howard and Rachel Thomas, especially their vgg16 12 | finetuning script: 13 | Link: 14 | - https://github.com/fastai/courses/blob/master/deeplearning1/nbs/vgg16.py 15 | 16 | 17 | The pretrained weights can be downloaded here and should be placed in the same 18 | folder as this file: 19 | - http://www.cs.toronto.edu/~guerzhoy/tf_alexnet/ 20 | 21 | @author: Frederik Kratzert (contact: f.kratzert(at)gmail.com) 22 | """ 23 | 24 | import tensorflow as tf 25 | import numpy as np 26 | 27 | 28 | class AlexNet(object): 29 | """Implementation of the AlexNet.""" 30 | 31 | def __init__(self, x, keep_prob, num_classes, skip_layer, 32 | weights_path='DEFAULT'): 33 | """Create the graph of the AlexNet model. 34 | 35 | Args: 36 | x: Placeholder for the input tensor. 37 | keep_prob: Dropout probability. 38 | num_classes: Number of classes in the dataset. 39 | skip_layer: List of names of the layer, that get trained from 40 | scratch 41 | weights_path: Complete path to the pretrained weight file, if it 42 | isn't in the same folder as this code 43 | """ 44 | # Parse input arguments into class variables 45 | self.X = x 46 | self.NUM_CLASSES = num_classes 47 | self.KEEP_PROB = keep_prob 48 | self.SKIP_LAYER = skip_layer 49 | 50 | if weights_path == 'DEFAULT': 51 | self.WEIGHTS_PATH = 'bvlc_alexnet.npy' 52 | else: 53 | self.WEIGHTS_PATH = weights_path 54 | 55 | # Call the create function to build the computational graph of AlexNet 56 | self.create() 57 | 58 | def create(self): 59 | """Create the network graph.""" 60 | # 1st Layer: Conv (w ReLu) -> Lrn -> Pool 61 | conv1 = conv(self.X, 11, 11, 96, 4, 4, padding='VALID', name='conv1') 62 | norm1 = lrn(conv1, 2, 2e-05, 0.75, name='norm1') 63 | pool1 = max_pool(norm1, 3, 3, 2, 2, padding='VALID', name='pool1') 64 | 65 | # 2nd Layer: Conv (w ReLu) -> Lrn -> Pool with 2 groups 66 | conv2 = conv(pool1, 5, 5, 256, 1, 1, groups=2, name='conv2') 67 | norm2 = lrn(conv2, 2, 2e-05, 0.75, name='norm2') 68 | pool2 = max_pool(norm2, 3, 3, 2, 2, padding='VALID', name='pool2') 69 | 70 | # 3rd Layer: Conv (w ReLu) 71 | conv3 = conv(pool2, 3, 3, 384, 1, 1, name='conv3') 72 | 73 | # 4th Layer: Conv (w ReLu) splitted into two groups 74 | conv4 = conv(conv3, 3, 3, 384, 1, 1, groups=2, name='conv4') 75 | 76 | # 5th Layer: Conv (w ReLu) -> Pool splitted into two groups 77 | conv5 = conv(conv4, 3, 3, 256, 1, 1, groups=2, name='conv5') 78 | pool5 = max_pool(conv5, 3, 3, 2, 2, padding='VALID', name='pool5') 79 | 80 | # 6th Layer: Flatten -> FC (w ReLu) -> Dropout 81 | flattened = tf.reshape(pool5, [-1, 6*6*256]) 82 | fc6 = fc(flattened, 6*6*256, 4096, name='fc6') 83 | dropout6 = dropout(fc6, self.KEEP_PROB) 84 | 85 | # 7th Layer: FC (w ReLu) -> Dropout 86 | fc7 = fc(dropout6, 4096, 4096, name='fc7') 87 | dropout7 = dropout(fc7, self.KEEP_PROB) 88 | 89 | # 8th Layer: FC and return unscaled activations 90 | self.fc8 = fc(dropout7, 4096, self.NUM_CLASSES, relu=False, name='fc8') 91 | 92 | def load_initial_weights(self, session): 93 | """Load weights from file into network. 94 | 95 | As the weights from http://www.cs.toronto.edu/~guerzhoy/tf_alexnet/ 96 | come as a dict of lists (e.g. weights['conv1'] is a list) and not as 97 | dict of dicts (e.g. weights['conv1'] is a dict with keys 'weights' & 98 | 'biases') we need a special load function 99 | """ 100 | # Load the weights into memory 101 | weights_dict = np.load(self.WEIGHTS_PATH, encoding='bytes').item() 102 | 103 | # Loop over all layer names stored in the weights dict 104 | for op_name in weights_dict: 105 | 106 | # Check if layer should be trained from scratch 107 | if op_name not in self.SKIP_LAYER: 108 | 109 | with tf.variable_scope(op_name, reuse=True): 110 | 111 | # Assign weights/biases to their corresponding tf variable 112 | for data in weights_dict[op_name]: 113 | 114 | # Biases 115 | if len(data.shape) == 1: 116 | var = tf.get_variable('biases', trainable=False) 117 | session.run(var.assign(data)) 118 | 119 | # Weights 120 | else: 121 | var = tf.get_variable('weights', trainable=False) 122 | session.run(var.assign(data)) 123 | 124 | 125 | def conv(x, filter_height, filter_width, num_filters, stride_y, stride_x, name, 126 | padding='SAME', groups=1): 127 | """Create a convolution layer. 128 | 129 | Adapted from: https://github.com/ethereon/caffe-tensorflow 130 | """ 131 | # Get number of input channels 132 | input_channels = int(x.get_shape()[-1]) 133 | 134 | # Create lambda function for the convolution 135 | convolve = lambda i, k: tf.nn.conv2d(i, k, 136 | strides=[1, stride_y, stride_x, 1], 137 | padding=padding) 138 | 139 | with tf.variable_scope(name) as scope: 140 | # Create tf variables for the weights and biases of the conv layer 141 | weights = tf.get_variable('weights', shape=[filter_height, 142 | filter_width, 143 | input_channels/groups, 144 | num_filters]) 145 | biases = tf.get_variable('biases', shape=[num_filters]) 146 | 147 | if groups == 1: 148 | conv = convolve(x, weights) 149 | 150 | # In the cases of multiple groups, split inputs & weights and 151 | else: 152 | # Split input and weights and convolve them separately 153 | input_groups = tf.split(axis=3, num_or_size_splits=groups, value=x) 154 | weight_groups = tf.split(axis=3, num_or_size_splits=groups, 155 | value=weights) 156 | output_groups = [convolve(i, k) for i, k in zip(input_groups, weight_groups)] 157 | 158 | # Concat the convolved output together again 159 | conv = tf.concat(axis=3, values=output_groups) 160 | 161 | # Add biases 162 | bias = tf.reshape(tf.nn.bias_add(conv, biases), tf.shape(conv)) 163 | 164 | # Apply relu function 165 | relu = tf.nn.relu(bias, name=scope.name) 166 | 167 | return relu 168 | 169 | 170 | def fc(x, num_in, num_out, name, relu=True): 171 | """Create a fully connected layer.""" 172 | with tf.variable_scope(name) as scope: 173 | 174 | # Create tf variables for the weights and biases 175 | weights = tf.get_variable('weights', shape=[num_in, num_out], 176 | trainable=True) 177 | biases = tf.get_variable('biases', [num_out], trainable=True) 178 | 179 | # Matrix multiply weights and inputs and add bias 180 | act = tf.nn.xw_plus_b(x, weights, biases, name=scope.name) 181 | 182 | if relu: 183 | # Apply ReLu non linearity 184 | relu = tf.nn.relu(act) 185 | return relu 186 | else: 187 | return act 188 | 189 | 190 | def max_pool(x, filter_height, filter_width, stride_y, stride_x, name, 191 | padding='SAME'): 192 | """Create a max pooling layer.""" 193 | return tf.nn.max_pool(x, ksize=[1, filter_height, filter_width, 1], 194 | strides=[1, stride_y, stride_x, 1], 195 | padding=padding, name=name) 196 | 197 | 198 | def lrn(x, radius, alpha, beta, name, bias=1.0): 199 | """Create a local response normalization layer.""" 200 | return tf.nn.local_response_normalization(x, depth_radius=radius, 201 | alpha=alpha, beta=beta, 202 | bias=bias, name=name) 203 | 204 | 205 | def dropout(x, keep_prob): 206 | """Create a dropout layer.""" 207 | return tf.nn.dropout(x, keep_prob) 208 | -------------------------------------------------------------------------------- /caffe_classes.py: -------------------------------------------------------------------------------- 1 | class_names = '''tench, Tinca tinca 2 | goldfish, Carassius auratus 3 | great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias 4 | tiger shark, Galeocerdo cuvieri 5 | hammerhead, hammerhead shark 6 | electric ray, crampfish, numbfish, torpedo 7 | stingray 8 | cock 9 | hen 10 | ostrich, Struthio camelus 11 | brambling, Fringilla montifringilla 12 | goldfinch, Carduelis carduelis 13 | house finch, linnet, Carpodacus mexicanus 14 | junco, snowbird 15 | indigo bunting, indigo finch, indigo bird, Passerina cyanea 16 | robin, American robin, Turdus migratorius 17 | bulbul 18 | jay 19 | magpie 20 | chickadee 21 | water ouzel, dipper 22 | kite 23 | bald eagle, American eagle, Haliaeetus leucocephalus 24 | vulture 25 | great grey owl, great gray owl, Strix nebulosa 26 | European fire salamander, Salamandra salamandra 27 | common newt, Triturus vulgaris 28 | eft 29 | spotted salamander, Ambystoma maculatum 30 | axolotl, mud puppy, Ambystoma mexicanum 31 | bullfrog, Rana catesbeiana 32 | tree frog, tree-frog 33 | tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui 34 | loggerhead, loggerhead turtle, Caretta caretta 35 | leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea 36 | mud turtle 37 | terrapin 38 | box turtle, box tortoise 39 | banded gecko 40 | common iguana, iguana, Iguana iguana 41 | American chameleon, anole, Anolis carolinensis 42 | whiptail, whiptail lizard 43 | agama 44 | frilled lizard, Chlamydosaurus kingi 45 | alligator lizard 46 | Gila monster, Heloderma suspectum 47 | green lizard, Lacerta viridis 48 | African chameleon, Chamaeleo chamaeleon 49 | Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis 50 | African crocodile, Nile crocodile, Crocodylus niloticus 51 | American alligator, Alligator mississipiensis 52 | triceratops 53 | thunder snake, worm snake, Carphophis amoenus 54 | ringneck snake, ring-necked snake, ring snake 55 | hognose snake, puff adder, sand viper 56 | green snake, grass snake 57 | king snake, kingsnake 58 | garter snake, grass snake 59 | water snake 60 | vine snake 61 | night snake, Hypsiglena torquata 62 | boa constrictor, Constrictor constrictor 63 | rock python, rock snake, Python sebae 64 | Indian cobra, Naja naja 65 | green mamba 66 | sea snake 67 | horned viper, cerastes, sand viper, horned asp, Cerastes cornutus 68 | diamondback, diamondback rattlesnake, Crotalus adamanteus 69 | sidewinder, horned rattlesnake, Crotalus cerastes 70 | trilobite 71 | harvestman, daddy longlegs, Phalangium opilio 72 | scorpion 73 | black and gold garden spider, Argiope aurantia 74 | barn spider, Araneus cavaticus 75 | garden spider, Aranea diademata 76 | black widow, Latrodectus mactans 77 | tarantula 78 | wolf spider, hunting spider 79 | tick 80 | centipede 81 | black grouse 82 | ptarmigan 83 | ruffed grouse, partridge, Bonasa umbellus 84 | prairie chicken, prairie grouse, prairie fowl 85 | peacock 86 | quail 87 | partridge 88 | African grey, African gray, Psittacus erithacus 89 | macaw 90 | sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita 91 | lorikeet 92 | coucal 93 | bee eater 94 | hornbill 95 | hummingbird 96 | jacamar 97 | toucan 98 | drake 99 | red-breasted merganser, Mergus serrator 100 | goose 101 | black swan, Cygnus atratus 102 | tusker 103 | echidna, spiny anteater, anteater 104 | platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus 105 | wallaby, brush kangaroo 106 | koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus 107 | wombat 108 | jellyfish 109 | sea anemone, anemone 110 | brain coral 111 | flatworm, platyhelminth 112 | nematode, nematode worm, roundworm 113 | conch 114 | snail 115 | slug 116 | sea slug, nudibranch 117 | chiton, coat-of-mail shell, sea cradle, polyplacophore 118 | chambered nautilus, pearly nautilus, nautilus 119 | Dungeness crab, Cancer magister 120 | rock crab, Cancer irroratus 121 | fiddler crab 122 | king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica 123 | American lobster, Northern lobster, Maine lobster, Homarus americanus 124 | spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish 125 | crayfish, crawfish, crawdad, crawdaddy 126 | hermit crab 127 | isopod 128 | white stork, Ciconia ciconia 129 | black stork, Ciconia nigra 130 | spoonbill 131 | flamingo 132 | little blue heron, Egretta caerulea 133 | American egret, great white heron, Egretta albus 134 | bittern 135 | crane 136 | limpkin, Aramus pictus 137 | European gallinule, Porphyrio porphyrio 138 | American coot, marsh hen, mud hen, water hen, Fulica americana 139 | bustard 140 | ruddy turnstone, Arenaria interpres 141 | red-backed sandpiper, dunlin, Erolia alpina 142 | redshank, Tringa totanus 143 | dowitcher 144 | oystercatcher, oyster catcher 145 | pelican 146 | king penguin, Aptenodytes patagonica 147 | albatross, mollymawk 148 | grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus 149 | killer whale, killer, orca, grampus, sea wolf, Orcinus orca 150 | dugong, Dugong dugon 151 | sea lion 152 | Chihuahua 153 | Japanese spaniel 154 | Maltese dog, Maltese terrier, Maltese 155 | Pekinese, Pekingese, Peke 156 | Shih-Tzu 157 | Blenheim spaniel 158 | papillon 159 | toy terrier 160 | Rhodesian ridgeback 161 | Afghan hound, Afghan 162 | basset, basset hound 163 | beagle 164 | bloodhound, sleuthhound 165 | bluetick 166 | black-and-tan coonhound 167 | Walker hound, Walker foxhound 168 | English foxhound 169 | redbone 170 | borzoi, Russian wolfhound 171 | Irish wolfhound 172 | Italian greyhound 173 | whippet 174 | Ibizan hound, Ibizan Podenco 175 | Norwegian elkhound, elkhound 176 | otterhound, otter hound 177 | Saluki, gazelle hound 178 | Scottish deerhound, deerhound 179 | Weimaraner 180 | Staffordshire bullterrier, Staffordshire bull terrier 181 | American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier 182 | Bedlington terrier 183 | Border terrier 184 | Kerry blue terrier 185 | Irish terrier 186 | Norfolk terrier 187 | Norwich terrier 188 | Yorkshire terrier 189 | wire-haired fox terrier 190 | Lakeland terrier 191 | Sealyham terrier, Sealyham 192 | Airedale, Airedale terrier 193 | cairn, cairn terrier 194 | Australian terrier 195 | Dandie Dinmont, Dandie Dinmont terrier 196 | Boston bull, Boston terrier 197 | miniature schnauzer 198 | giant schnauzer 199 | standard schnauzer 200 | Scotch terrier, Scottish terrier, Scottie 201 | Tibetan terrier, chrysanthemum dog 202 | silky terrier, Sydney silky 203 | soft-coated wheaten terrier 204 | West Highland white terrier 205 | Lhasa, Lhasa apso 206 | flat-coated retriever 207 | curly-coated retriever 208 | golden retriever 209 | Labrador retriever 210 | Chesapeake Bay retriever 211 | German short-haired pointer 212 | vizsla, Hungarian pointer 213 | English setter 214 | Irish setter, red setter 215 | Gordon setter 216 | Brittany spaniel 217 | clumber, clumber spaniel 218 | English springer, English springer spaniel 219 | Welsh springer spaniel 220 | cocker spaniel, English cocker spaniel, cocker 221 | Sussex spaniel 222 | Irish water spaniel 223 | kuvasz 224 | schipperke 225 | groenendael 226 | malinois 227 | briard 228 | kelpie 229 | komondor 230 | Old English sheepdog, bobtail 231 | Shetland sheepdog, Shetland sheep dog, Shetland 232 | collie 233 | Border collie 234 | Bouvier des Flandres, Bouviers des Flandres 235 | Rottweiler 236 | German shepherd, German shepherd dog, German police dog, alsatian 237 | Doberman, Doberman pinscher 238 | miniature pinscher 239 | Greater Swiss Mountain dog 240 | Bernese mountain dog 241 | Appenzeller 242 | EntleBucher 243 | boxer 244 | bull mastiff 245 | Tibetan mastiff 246 | French bulldog 247 | Great Dane 248 | Saint Bernard, St Bernard 249 | Eskimo dog, husky 250 | malamute, malemute, Alaskan malamute 251 | Siberian husky 252 | dalmatian, coach dog, carriage dog 253 | affenpinscher, monkey pinscher, monkey dog 254 | basenji 255 | pug, pug-dog 256 | Leonberg 257 | Newfoundland, Newfoundland dog 258 | Great Pyrenees 259 | Samoyed, Samoyede 260 | Pomeranian 261 | chow, chow chow 262 | keeshond 263 | Brabancon griffon 264 | Pembroke, Pembroke Welsh corgi 265 | Cardigan, Cardigan Welsh corgi 266 | toy poodle 267 | miniature poodle 268 | standard poodle 269 | Mexican hairless 270 | timber wolf, grey wolf, gray wolf, Canis lupus 271 | white wolf, Arctic wolf, Canis lupus tundrarum 272 | red wolf, maned wolf, Canis rufus, Canis niger 273 | coyote, prairie wolf, brush wolf, Canis latrans 274 | dingo, warrigal, warragal, Canis dingo 275 | dhole, Cuon alpinus 276 | African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus 277 | hyena, hyaena 278 | red fox, Vulpes vulpes 279 | kit fox, Vulpes macrotis 280 | Arctic fox, white fox, Alopex lagopus 281 | grey fox, gray fox, Urocyon cinereoargenteus 282 | tabby, tabby cat 283 | tiger cat 284 | Persian cat 285 | Siamese cat, Siamese 286 | Egyptian cat 287 | cougar, puma, catamount, mountain lion, painter, panther, Felis concolor 288 | lynx, catamount 289 | leopard, Panthera pardus 290 | snow leopard, ounce, Panthera uncia 291 | jaguar, panther, Panthera onca, Felis onca 292 | lion, king of beasts, Panthera leo 293 | tiger, Panthera tigris 294 | cheetah, chetah, Acinonyx jubatus 295 | brown bear, bruin, Ursus arctos 296 | American black bear, black bear, Ursus americanus, Euarctos americanus 297 | ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus 298 | sloth bear, Melursus ursinus, Ursus ursinus 299 | mongoose 300 | meerkat, mierkat 301 | tiger beetle 302 | ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle 303 | ground beetle, carabid beetle 304 | long-horned beetle, longicorn, longicorn beetle 305 | leaf beetle, chrysomelid 306 | dung beetle 307 | rhinoceros beetle 308 | weevil 309 | fly 310 | bee 311 | ant, emmet, pismire 312 | grasshopper, hopper 313 | cricket 314 | walking stick, walkingstick, stick insect 315 | cockroach, roach 316 | mantis, mantid 317 | cicada, cicala 318 | leafhopper 319 | lacewing, lacewing fly 320 | dragonfly, darning needle, devil's darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk 321 | damselfly 322 | admiral 323 | ringlet, ringlet butterfly 324 | monarch, monarch butterfly, milkweed butterfly, Danaus plexippus 325 | cabbage butterfly 326 | sulphur butterfly, sulfur butterfly 327 | lycaenid, lycaenid butterfly 328 | starfish, sea star 329 | sea urchin 330 | sea cucumber, holothurian 331 | wood rabbit, cottontail, cottontail rabbit 332 | hare 333 | Angora, Angora rabbit 334 | hamster 335 | porcupine, hedgehog 336 | fox squirrel, eastern fox squirrel, Sciurus niger 337 | marmot 338 | beaver 339 | guinea pig, Cavia cobaya 340 | sorrel 341 | zebra 342 | hog, pig, grunter, squealer, Sus scrofa 343 | wild boar, boar, Sus scrofa 344 | warthog 345 | hippopotamus, hippo, river horse, Hippopotamus amphibius 346 | ox 347 | water buffalo, water ox, Asiatic buffalo, Bubalus bubalis 348 | bison 349 | ram, tup 350 | bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis 351 | ibex, Capra ibex 352 | hartebeest 353 | impala, Aepyceros melampus 354 | gazelle 355 | Arabian camel, dromedary, Camelus dromedarius 356 | llama 357 | weasel 358 | mink 359 | polecat, fitch, foulmart, foumart, Mustela putorius 360 | black-footed ferret, ferret, Mustela nigripes 361 | otter 362 | skunk, polecat, wood pussy 363 | badger 364 | armadillo 365 | three-toed sloth, ai, Bradypus tridactylus 366 | orangutan, orang, orangutang, Pongo pygmaeus 367 | gorilla, Gorilla gorilla 368 | chimpanzee, chimp, Pan troglodytes 369 | gibbon, Hylobates lar 370 | siamang, Hylobates syndactylus, Symphalangus syndactylus 371 | guenon, guenon monkey 372 | patas, hussar monkey, Erythrocebus patas 373 | baboon 374 | macaque 375 | langur 376 | colobus, colobus monkey 377 | proboscis monkey, Nasalis larvatus 378 | marmoset 379 | capuchin, ringtail, Cebus capucinus 380 | howler monkey, howler 381 | titi, titi monkey 382 | spider monkey, Ateles geoffroyi 383 | squirrel monkey, Saimiri sciureus 384 | Madagascar cat, ring-tailed lemur, Lemur catta 385 | indri, indris, Indri indri, Indri brevicaudatus 386 | Indian elephant, Elephas maximus 387 | African elephant, Loxodonta africana 388 | lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens 389 | giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca 390 | barracouta, snoek 391 | eel 392 | coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch 393 | rock beauty, Holocanthus tricolor 394 | anemone fish 395 | sturgeon 396 | gar, garfish, garpike, billfish, Lepisosteus osseus 397 | lionfish 398 | puffer, pufferfish, blowfish, globefish 399 | abacus 400 | abaya 401 | academic gown, academic robe, judge's robe 402 | accordion, piano accordion, squeeze box 403 | acoustic guitar 404 | aircraft carrier, carrier, flattop, attack aircraft carrier 405 | airliner 406 | airship, dirigible 407 | altar 408 | ambulance 409 | amphibian, amphibious vehicle 410 | analog clock 411 | apiary, bee house 412 | apron 413 | ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin 414 | assault rifle, assault gun 415 | backpack, back pack, knapsack, packsack, rucksack, haversack 416 | bakery, bakeshop, bakehouse 417 | balance beam, beam 418 | balloon 419 | ballpoint, ballpoint pen, ballpen, Biro 420 | Band Aid 421 | banjo 422 | bannister, banister, balustrade, balusters, handrail 423 | barbell 424 | barber chair 425 | barbershop 426 | barn 427 | barometer 428 | barrel, cask 429 | barrow, garden cart, lawn cart, wheelbarrow 430 | baseball 431 | basketball 432 | bassinet 433 | bassoon 434 | bathing cap, swimming cap 435 | bath towel 436 | bathtub, bathing tub, bath, tub 437 | beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon 438 | beacon, lighthouse, beacon light, pharos 439 | beaker 440 | bearskin, busby, shako 441 | beer bottle 442 | beer glass 443 | bell cote, bell cot 444 | bib 445 | bicycle-built-for-two, tandem bicycle, tandem 446 | bikini, two-piece 447 | binder, ring-binder 448 | binoculars, field glasses, opera glasses 449 | birdhouse 450 | boathouse 451 | bobsled, bobsleigh, bob 452 | bolo tie, bolo, bola tie, bola 453 | bonnet, poke bonnet 454 | bookcase 455 | bookshop, bookstore, bookstall 456 | bottlecap 457 | bow 458 | bow tie, bow-tie, bowtie 459 | brass, memorial tablet, plaque 460 | brassiere, bra, bandeau 461 | breakwater, groin, groyne, mole, bulwark, seawall, jetty 462 | breastplate, aegis, egis 463 | broom 464 | bucket, pail 465 | buckle 466 | bulletproof vest 467 | bullet train, bullet 468 | butcher shop, meat market 469 | cab, hack, taxi, taxicab 470 | caldron, cauldron 471 | candle, taper, wax light 472 | cannon 473 | canoe 474 | can opener, tin opener 475 | cardigan 476 | car mirror 477 | carousel, carrousel, merry-go-round, roundabout, whirligig 478 | carpenter's kit, tool kit 479 | carton 480 | car wheel 481 | cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM 482 | cassette 483 | cassette player 484 | castle 485 | catamaran 486 | CD player 487 | cello, violoncello 488 | cellular telephone, cellular phone, cellphone, cell, mobile phone 489 | chain 490 | chainlink fence 491 | chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour 492 | chain saw, chainsaw 493 | chest 494 | chiffonier, commode 495 | chime, bell, gong 496 | china cabinet, china closet 497 | Christmas stocking 498 | church, church building 499 | cinema, movie theater, movie theatre, movie house, picture palace 500 | cleaver, meat cleaver, chopper 501 | cliff dwelling 502 | cloak 503 | clog, geta, patten, sabot 504 | cocktail shaker 505 | coffee mug 506 | coffeepot 507 | coil, spiral, volute, whorl, helix 508 | combination lock 509 | computer keyboard, keypad 510 | confectionery, confectionary, candy store 511 | container ship, containership, container vessel 512 | convertible 513 | corkscrew, bottle screw 514 | cornet, horn, trumpet, trump 515 | cowboy boot 516 | cowboy hat, ten-gallon hat 517 | cradle 518 | crane 519 | crash helmet 520 | crate 521 | crib, cot 522 | Crock Pot 523 | croquet ball 524 | crutch 525 | cuirass 526 | dam, dike, dyke 527 | desk 528 | desktop computer 529 | dial telephone, dial phone 530 | diaper, nappy, napkin 531 | digital clock 532 | digital watch 533 | dining table, board 534 | dishrag, dishcloth 535 | dishwasher, dish washer, dishwashing machine 536 | disk brake, disc brake 537 | dock, dockage, docking facility 538 | dogsled, dog sled, dog sleigh 539 | dome 540 | doormat, welcome mat 541 | drilling platform, offshore rig 542 | drum, membranophone, tympan 543 | drumstick 544 | dumbbell 545 | Dutch oven 546 | electric fan, blower 547 | electric guitar 548 | electric locomotive 549 | entertainment center 550 | envelope 551 | espresso maker 552 | face powder 553 | feather boa, boa 554 | file, file cabinet, filing cabinet 555 | fireboat 556 | fire engine, fire truck 557 | fire screen, fireguard 558 | flagpole, flagstaff 559 | flute, transverse flute 560 | folding chair 561 | football helmet 562 | forklift 563 | fountain 564 | fountain pen 565 | four-poster 566 | freight car 567 | French horn, horn 568 | frying pan, frypan, skillet 569 | fur coat 570 | garbage truck, dustcart 571 | gasmask, respirator, gas helmet 572 | gas pump, gasoline pump, petrol pump, island dispenser 573 | goblet 574 | go-kart 575 | golf ball 576 | golfcart, golf cart 577 | gondola 578 | gong, tam-tam 579 | gown 580 | grand piano, grand 581 | greenhouse, nursery, glasshouse 582 | grille, radiator grille 583 | grocery store, grocery, food market, market 584 | guillotine 585 | hair slide 586 | hair spray 587 | half track 588 | hammer 589 | hamper 590 | hand blower, blow dryer, blow drier, hair dryer, hair drier 591 | hand-held computer, hand-held microcomputer 592 | handkerchief, hankie, hanky, hankey 593 | hard disc, hard disk, fixed disk 594 | harmonica, mouth organ, harp, mouth harp 595 | harp 596 | harvester, reaper 597 | hatchet 598 | holster 599 | home theater, home theatre 600 | honeycomb 601 | hook, claw 602 | hoopskirt, crinoline 603 | horizontal bar, high bar 604 | horse cart, horse-cart 605 | hourglass 606 | iPod 607 | iron, smoothing iron 608 | jack-o'-lantern 609 | jean, blue jean, denim 610 | jeep, landrover 611 | jersey, T-shirt, tee shirt 612 | jigsaw puzzle 613 | jinrikisha, ricksha, rickshaw 614 | joystick 615 | kimono 616 | knee pad 617 | knot 618 | lab coat, laboratory coat 619 | ladle 620 | lampshade, lamp shade 621 | laptop, laptop computer 622 | lawn mower, mower 623 | lens cap, lens cover 624 | letter opener, paper knife, paperknife 625 | library 626 | lifeboat 627 | lighter, light, igniter, ignitor 628 | limousine, limo 629 | liner, ocean liner 630 | lipstick, lip rouge 631 | Loafer 632 | lotion 633 | loudspeaker, speaker, speaker unit, loudspeaker system, speaker system 634 | loupe, jeweler's loupe 635 | lumbermill, sawmill 636 | magnetic compass 637 | mailbag, postbag 638 | mailbox, letter box 639 | maillot 640 | maillot, tank suit 641 | manhole cover 642 | maraca 643 | marimba, xylophone 644 | mask 645 | matchstick 646 | maypole 647 | maze, labyrinth 648 | measuring cup 649 | medicine chest, medicine cabinet 650 | megalith, megalithic structure 651 | microphone, mike 652 | microwave, microwave oven 653 | military uniform 654 | milk can 655 | minibus 656 | miniskirt, mini 657 | minivan 658 | missile 659 | mitten 660 | mixing bowl 661 | mobile home, manufactured home 662 | Model T 663 | modem 664 | monastery 665 | monitor 666 | moped 667 | mortar 668 | mortarboard 669 | mosque 670 | mosquito net 671 | motor scooter, scooter 672 | mountain bike, all-terrain bike, off-roader 673 | mountain tent 674 | mouse, computer mouse 675 | mousetrap 676 | moving van 677 | muzzle 678 | nail 679 | neck brace 680 | necklace 681 | nipple 682 | notebook, notebook computer 683 | obelisk 684 | oboe, hautboy, hautbois 685 | ocarina, sweet potato 686 | odometer, hodometer, mileometer, milometer 687 | oil filter 688 | organ, pipe organ 689 | oscilloscope, scope, cathode-ray oscilloscope, CRO 690 | overskirt 691 | oxcart 692 | oxygen mask 693 | packet 694 | paddle, boat paddle 695 | paddlewheel, paddle wheel 696 | padlock 697 | paintbrush 698 | pajama, pyjama, pj's, jammies 699 | palace 700 | panpipe, pandean pipe, syrinx 701 | paper towel 702 | parachute, chute 703 | parallel bars, bars 704 | park bench 705 | parking meter 706 | passenger car, coach, carriage 707 | patio, terrace 708 | pay-phone, pay-station 709 | pedestal, plinth, footstall 710 | pencil box, pencil case 711 | pencil sharpener 712 | perfume, essence 713 | Petri dish 714 | photocopier 715 | pick, plectrum, plectron 716 | pickelhaube 717 | picket fence, paling 718 | pickup, pickup truck 719 | pier 720 | piggy bank, penny bank 721 | pill bottle 722 | pillow 723 | ping-pong ball 724 | pinwheel 725 | pirate, pirate ship 726 | pitcher, ewer 727 | plane, carpenter's plane, woodworking plane 728 | planetarium 729 | plastic bag 730 | plate rack 731 | plow, plough 732 | plunger, plumber's helper 733 | Polaroid camera, Polaroid Land camera 734 | pole 735 | police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria 736 | poncho 737 | pool table, billiard table, snooker table 738 | pop bottle, soda bottle 739 | pot, flowerpot 740 | potter's wheel 741 | power drill 742 | prayer rug, prayer mat 743 | printer 744 | prison, prison house 745 | projectile, missile 746 | projector 747 | puck, hockey puck 748 | punching bag, punch bag, punching ball, punchball 749 | purse 750 | quill, quill pen 751 | quilt, comforter, comfort, puff 752 | racer, race car, racing car 753 | racket, racquet 754 | radiator 755 | radio, wireless 756 | radio telescope, radio reflector 757 | rain barrel 758 | recreational vehicle, RV, R.V. 759 | reel 760 | reflex camera 761 | refrigerator, icebox 762 | remote control, remote 763 | restaurant, eating house, eating place, eatery 764 | revolver, six-gun, six-shooter 765 | rifle 766 | rocking chair, rocker 767 | rotisserie 768 | rubber eraser, rubber, pencil eraser 769 | rugby ball 770 | rule, ruler 771 | running shoe 772 | safe 773 | safety pin 774 | saltshaker, salt shaker 775 | sandal 776 | sarong 777 | sax, saxophone 778 | scabbard 779 | scale, weighing machine 780 | school bus 781 | schooner 782 | scoreboard 783 | screen, CRT screen 784 | screw 785 | screwdriver 786 | seat belt, seatbelt 787 | sewing machine 788 | shield, buckler 789 | shoe shop, shoe-shop, shoe store 790 | shoji 791 | shopping basket 792 | shopping cart 793 | shovel 794 | shower cap 795 | shower curtain 796 | ski 797 | ski mask 798 | sleeping bag 799 | slide rule, slipstick 800 | sliding door 801 | slot, one-armed bandit 802 | snorkel 803 | snowmobile 804 | snowplow, snowplough 805 | soap dispenser 806 | soccer ball 807 | sock 808 | solar dish, solar collector, solar furnace 809 | sombrero 810 | soup bowl 811 | space bar 812 | space heater 813 | space shuttle 814 | spatula 815 | speedboat 816 | spider web, spider's web 817 | spindle 818 | sports car, sport car 819 | spotlight, spot 820 | stage 821 | steam locomotive 822 | steel arch bridge 823 | steel drum 824 | stethoscope 825 | stole 826 | stone wall 827 | stopwatch, stop watch 828 | stove 829 | strainer 830 | streetcar, tram, tramcar, trolley, trolley car 831 | stretcher 832 | studio couch, day bed 833 | stupa, tope 834 | submarine, pigboat, sub, U-boat 835 | suit, suit of clothes 836 | sundial 837 | sunglass 838 | sunglasses, dark glasses, shades 839 | sunscreen, sunblock, sun blocker 840 | suspension bridge 841 | swab, swob, mop 842 | sweatshirt 843 | swimming trunks, bathing trunks 844 | swing 845 | switch, electric switch, electrical switch 846 | syringe 847 | table lamp 848 | tank, army tank, armored combat vehicle, armoured combat vehicle 849 | tape player 850 | teapot 851 | teddy, teddy bear 852 | television, television system 853 | tennis ball 854 | thatch, thatched roof 855 | theater curtain, theatre curtain 856 | thimble 857 | thresher, thrasher, threshing machine 858 | throne 859 | tile roof 860 | toaster 861 | tobacco shop, tobacconist shop, tobacconist 862 | toilet seat 863 | torch 864 | totem pole 865 | tow truck, tow car, wrecker 866 | toyshop 867 | tractor 868 | trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi 869 | tray 870 | trench coat 871 | tricycle, trike, velocipede 872 | trimaran 873 | tripod 874 | triumphal arch 875 | trolleybus, trolley coach, trackless trolley 876 | trombone 877 | tub, vat 878 | turnstile 879 | typewriter keyboard 880 | umbrella 881 | unicycle, monocycle 882 | upright, upright piano 883 | vacuum, vacuum cleaner 884 | vase 885 | vault 886 | velvet 887 | vending machine 888 | vestment 889 | viaduct 890 | violin, fiddle 891 | volleyball 892 | waffle iron 893 | wall clock 894 | wallet, billfold, notecase, pocketbook 895 | wardrobe, closet, press 896 | warplane, military plane 897 | washbasin, handbasin, washbowl, lavabo, wash-hand basin 898 | washer, automatic washer, washing machine 899 | water bottle 900 | water jug 901 | water tower 902 | whiskey jug 903 | whistle 904 | wig 905 | window screen 906 | window shade 907 | Windsor tie 908 | wine bottle 909 | wing 910 | wok 911 | wooden spoon 912 | wool, woolen, woollen 913 | worm fence, snake fence, snake-rail fence, Virginia fence 914 | wreck 915 | yawl 916 | yurt 917 | web site, website, internet site, site 918 | comic book 919 | crossword puzzle, crossword 920 | street sign 921 | traffic light, traffic signal, stoplight 922 | book jacket, dust cover, dust jacket, dust wrapper 923 | menu 924 | plate 925 | guacamole 926 | consomme 927 | hot pot, hotpot 928 | trifle 929 | ice cream, icecream 930 | ice lolly, lolly, lollipop, popsicle 931 | French loaf 932 | bagel, beigel 933 | pretzel 934 | cheeseburger 935 | hotdog, hot dog, red hot 936 | mashed potato 937 | head cabbage 938 | broccoli 939 | cauliflower 940 | zucchini, courgette 941 | spaghetti squash 942 | acorn squash 943 | butternut squash 944 | cucumber, cuke 945 | artichoke, globe artichoke 946 | bell pepper 947 | cardoon 948 | mushroom 949 | Granny Smith 950 | strawberry 951 | orange 952 | lemon 953 | fig 954 | pineapple, ananas 955 | banana 956 | jackfruit, jak, jack 957 | custard apple 958 | pomegranate 959 | hay 960 | carbonara 961 | chocolate sauce, chocolate syrup 962 | dough 963 | meat loaf, meatloaf 964 | pizza, pizza pie 965 | potpie 966 | burrito 967 | red wine 968 | espresso 969 | cup 970 | eggnog 971 | alp 972 | bubble 973 | cliff, drop, drop-off 974 | coral reef 975 | geyser 976 | lakeside, lakeshore 977 | promontory, headland, head, foreland 978 | sandbar, sand bar 979 | seashore, coast, seacoast, sea-coast 980 | valley, vale 981 | volcano 982 | ballplayer, baseball player 983 | groom, bridegroom 984 | scuba diver 985 | rapeseed 986 | daisy 987 | yellow lady's slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum 988 | corn 989 | acorn 990 | hip, rose hip, rosehip 991 | buckeye, horse chestnut, conker 992 | coral fungus 993 | agaric 994 | gyromitra 995 | stinkhorn, carrion fungus 996 | earthstar 997 | hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa 998 | bolete 999 | ear, spike, capitulum 1000 | toilet tissue, toilet paper, bathroom tissue'''.split("\n") --------------------------------------------------------------------------------