├── .gitignore ├── tankbuster ├── __init__.py ├── engine │ ├── __init__.py │ └── detect.py ├── cnn │ ├── __init__.py │ └── nets.py └── downloads │ ├── tankbuster_conv_weights.h5 │ └── tankbuster_res_weights.h5 ├── demo_images ├── backyard.jpg ├── with_lada.jpg ├── knocked_out.jpg └── from_screen_capture.png ├── .gitattributes ├── setup.cfg ├── setup.py ├── LICENSE.txt ├── bust.py └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | .idea 3 | .eggs 4 | -------------------------------------------------------------------------------- /tankbuster/__init__.py: -------------------------------------------------------------------------------- 1 | from engine import bust 2 | -------------------------------------------------------------------------------- /tankbuster/engine/__init__.py: -------------------------------------------------------------------------------- 1 | from detect import bust 2 | -------------------------------------------------------------------------------- /tankbuster/cnn/__init__.py: -------------------------------------------------------------------------------- 1 | from nets import CNNArchitecture 2 | -------------------------------------------------------------------------------- /demo_images/backyard.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thiippal/tankbuster/HEAD/demo_images/backyard.jpg -------------------------------------------------------------------------------- /demo_images/with_lada.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thiippal/tankbuster/HEAD/demo_images/with_lada.jpg -------------------------------------------------------------------------------- /demo_images/knocked_out.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thiippal/tankbuster/HEAD/demo_images/knocked_out.jpg -------------------------------------------------------------------------------- /demo_images/from_screen_capture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thiippal/tankbuster/HEAD/demo_images/from_screen_capture.png -------------------------------------------------------------------------------- /.gitattributes: -------------------------------------------------------------------------------- 1 | tankbuster/downloads/tankbuster_conv_weights.h5 filter=lfs diff=lfs merge=lfs -text 2 | tankbuster/downloads/tankbuster_res_weights.h5 filter=lfs diff=lfs merge=lfs -text 3 | -------------------------------------------------------------------------------- /tankbuster/downloads/tankbuster_conv_weights.h5: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:3888a72ade8ccbf099a86d4e4692b009096496a29938956437d9b3c1232c842f 3 | size 76048208 4 | -------------------------------------------------------------------------------- /tankbuster/downloads/tankbuster_res_weights.h5: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:7a8f01bd35cdc7544aebd2557cc664f6eb163e9b83f35cc15010ba8247dbe7e5 3 | size 4211004 4 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [metadata] 2 | name = tankbuster 3 | description-file = README.md 4 | description = A neural network trained to detect Soviet/Russian military vehicles in photographs 5 | url = http://github.com/thiippal/tankbuster 6 | author = Tuomo Hiippala 7 | author-email = tuomo.hiippala@iki.fi 8 | 9 | [bdist_wheel] 10 | universal = 1 11 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import setuptools 2 | from setuptools import setup 3 | from setuptools.command.install import install 4 | 5 | setup(name='tankbuster', 6 | version='0.3.2', 7 | description='A neural network trained to detect Soviet/Russian military vehicles in photographs', 8 | url='https://github.com/thiippal/tankbuster', 9 | author='Tuomo Hiippala', 10 | packages=['tankbuster'], 11 | package_dir={'tankbuster': 'tankbuster'}, 12 | package_data={'tankbuster': ['*.py', 'cnn/*.py', 'engine/*.py']}, 13 | author_email='tuomo.hiippala@iki.fi', 14 | license='MIT', 15 | keywords=['osint', 'computer vision', 'object recognition', 'deep learning', 'neural network'], 16 | download_url='https://github.com/thiippal/tankbuster/archive/0.3.2.tar.gz', 17 | install_requires=["Keras>=2.0.2", 18 | "tensorflow>=1.0.1", 19 | "h5py>=2.7.0", 20 | "Pillow>=4.1.0"], 21 | classifiers=['Programming Language :: Python :: 2.7']) 22 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | 2 | The MIT License (MIT) 3 | 4 | Copyright (c) 2016- Tuomo Hiippala 5 | tuomo.hiippala, at, iki.fi 6 | 7 | Permission is hereby granted, free of charge, to any person obtaining a copy 8 | of this software and associated documentation files (the "Software"), to deal 9 | in the Software without restriction, including without limitation the rights 10 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 11 | copies of the Software, and to permit persons to whom the Software is 12 | furnished to do so, subject to the following conditions: 13 | 14 | The above copyright notice and this permission notice shall be included in all 15 | copies or substantial portions of the Software. 16 | 17 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 18 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 19 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 20 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 21 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 22 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 23 | SOFTWARE. 24 | -------------------------------------------------------------------------------- /bust.py: -------------------------------------------------------------------------------- 1 | import os 2 | import argparse 3 | from tankbuster.engine import bust 4 | from keras.preprocessing.image import list_pictures 5 | 6 | # Set up the argument parser 7 | ap = argparse.ArgumentParser() 8 | 9 | # Add argument for input 10 | ap.add_argument("-i", "--input", required=True, 11 | help="The image or directory to be passed to the neural net.") 12 | ap.add_argument("-n", "--network", required=True, 13 | help="The neural network to be used: 'ConvNet' or 'ResNet'.") 14 | 15 | # Parse the arguments 16 | args = vars(ap.parse_args()) 17 | 18 | # Assign arguments to variables 19 | user_input = args['input'] 20 | network = args['network'] 21 | 22 | # Check if the input is a directory 23 | if os.path.isdir(user_input): 24 | 25 | # Feed the images to the network 26 | images = list_pictures(user_input) 27 | 28 | # Loop over the images 29 | for i in images: 30 | 31 | # Feed images to the classifier to retrieve a dictionary of predictions 32 | preds = bust(i, network=network) 33 | 34 | # Get the prediction with the highest probability 35 | pred = max(preds, key=preds.get) 36 | 37 | # Print the prediction 38 | print "*** {} - predicted {} ({:.2f}%) ...".format(i, 39 | pred, 40 | preds[pred] * 100) 41 | 42 | # Check if the input is a file 43 | if os.path.isfile(user_input): 44 | 45 | # Feed images to the classifier to retrieve a dictionary of predictions 46 | preds = bust(user_input, network=network) 47 | 48 | # Get the prediction with the highest probability 49 | pred = max(preds, key=preds.get) 50 | 51 | # Print the prediction 52 | print "*** {} - predicted {} ({:.2f}%) ...".format(user_input, 53 | pred, 54 | preds[pred] * 100) 55 | -------------------------------------------------------------------------------- /tankbuster/cnn/nets.py: -------------------------------------------------------------------------------- 1 | # Import the necessary packages 2 | from keras.layers import Dense, Dropout, Activation, Flatten 3 | from keras.layers import Conv2D, MaxPooling2D, Input 4 | from keras.models import Sequential 5 | from keras.regularizers import l2 6 | from keras.applications import ResNet50 7 | from keras.models import Model 8 | 9 | """ 10 | This file, nets.py, contains the architectures for the neural networks. These 11 | architectures are created using the CNNArchitecture class and its select method. 12 | """ 13 | 14 | 15 | class CNNArchitecture: 16 | # A class for network architectures 17 | def __init__(self): 18 | pass 19 | 20 | @staticmethod 21 | def select(architecture, *args, **kargs): 22 | 23 | # Map strings to functions 24 | nets = { 25 | "ConvNet": CNNArchitecture.ConvNet, 26 | "ResNet": CNNArchitecture.ResNet, 27 | "TopNet": CNNArchitecture.TopNet 28 | } 29 | 30 | # Initialize architecture 31 | net = nets.get(architecture, None) 32 | 33 | # Check if a net has been requested 34 | if net is None: 35 | return None 36 | 37 | # If the net is named correctly, return the network architecture 38 | return net(*args, **kargs) 39 | 40 | @staticmethod 41 | def ConvNet(imgrows, imgcols, numchannels, numclasses): 42 | # Initialize the model 43 | model = Sequential() 44 | 45 | # Define the first convolutional block 46 | model.add(Conv2D(32, (3, 3), input_shape=(imgrows, imgcols, numchannels), 47 | data_format='channels_last', 48 | kernel_regularizer=l2(l=0.001))) 49 | model.add(Activation("relu")) 50 | model.add(Conv2D(32, (3, 3), kernel_regularizer=l2(l=0.001))) 51 | model.add(Activation("relu")) 52 | model.add(MaxPooling2D(pool_size=(2, 2))) 53 | 54 | # Define the second convolutional block 55 | model.add(Conv2D(64, (3, 3), kernel_regularizer=l2(l=0.001))) 56 | model.add(Activation("relu")) 57 | model.add(Conv2D(64, (3, 3), kernel_regularizer=l2(l=0.001))) 58 | model.add(Activation("relu")) 59 | model.add(MaxPooling2D(pool_size=(2, 2))) 60 | 61 | # Flatten the feature maps 62 | model.add(Flatten()) 63 | 64 | # Add dropout 65 | model.add(Dropout(rate=0.5)) 66 | 67 | # Add fully-connected layer 68 | model.add(Dense(256, kernel_regularizer=l2(l=0.001))) 69 | model.add(Activation("relu")) 70 | 71 | # Add dropout 72 | model.add(Dropout(rate=0.5)) 73 | 74 | # Define the SoftMAX classifier 75 | model.add(Dense(numclasses)) 76 | model.add(Activation("softmax")) 77 | 78 | # Return the network architecture 79 | return model 80 | 81 | @staticmethod 82 | def ResNet(imgrows, imgcols, numchannels): 83 | # Initialize model without pre-trained weights 84 | resnet = ResNet50(include_top=False, 85 | input_tensor=Input(shape=(imgrows, 86 | imgcols, 87 | numchannels))) 88 | 89 | # Get output from the average pooling layer 90 | model = Model(inputs=resnet.input, 91 | outputs=resnet.get_layer('avg_pool').output) 92 | 93 | # Return the model 94 | return model 95 | 96 | @staticmethod 97 | def TopNet(input_tensor): 98 | model = Sequential() 99 | model.add(Flatten(input_shape=input_tensor)) 100 | model.add(Dropout(0.5)) 101 | model.add(Dense(512, activation='relu', kernel_regularizer=l2(0.001))) 102 | model.add(Dropout(0.5)) 103 | model.add(Dense(3, activation='softmax')) 104 | 105 | return model 106 | -------------------------------------------------------------------------------- /tankbuster/engine/detect.py: -------------------------------------------------------------------------------- 1 | # Import the necessary packages 2 | import numpy as np 3 | from .. import cnn 4 | from keras.applications.resnet50 import preprocess_input 5 | from keras.preprocessing.image import load_img, img_to_array 6 | from keras.utils.data_utils import get_file 7 | 8 | """ 9 | This file, detect.py, contains the function that loads, pre-processes and feeds 10 | the input image to the selected neural network. To make a prediction on a given 11 | image, call the bust function. 12 | """ 13 | 14 | # Define download URL for weights 15 | CONVNET_WEIGHTS = 'https://github.com/thiippal/tankbuster/raw/master/' \ 16 | 'tankbuster/downloads/tankbuster_conv_weights.h5' 17 | RESNET_WEIGHTS = 'https://github.com/thiippal/tankbuster/raw/master/' \ 18 | 'tankbuster/downloads/tankbuster_res_weights.h5' 19 | 20 | 21 | def bust(image, network): 22 | """ 23 | Predict the class of the input image ('BMP', 'Other' or 'T-72'). 24 | 25 | Arguments: 26 | image: A path to an image. 27 | 28 | Parameters: 29 | network: The network architecture used for making a prediction. 30 | 31 | Returns: 32 | A dictionary with labels and their associated softmax probabilities. 33 | """ 34 | 35 | # Define a list of valid networks 36 | networks = ['ConvNet', 'ResNet'] 37 | 38 | # Check if a valid network has been requested 39 | if network in networks: 40 | 41 | # If the request is valid, begin processing the image with the 42 | # following steps: 43 | 44 | # Load image 45 | if network == "ConvNet": 46 | original = load_img(image, target_size=(150, 150)) 47 | if network == "ResNet": 48 | original = load_img(image, target_size=(224, 224)) 49 | 50 | # Convert image into NumPy array and expand dimensions for input 51 | arr = img_to_array(original, data_format='channels_last') 52 | arr = np.expand_dims(arr, axis=0) 53 | 54 | # Normalize the image pixel values 55 | if network == "ConvNet": 56 | # For ConvNet, scale the pixel values into range [0..1] 57 | normalized = arr.astype("float") / 255.0 58 | # For ResNet, apply ImageNet preprocessing, i.e. subtract mean 59 | elif network == "ResNet": 60 | normalized = preprocess_input(arr, data_format='channels_last') 61 | 62 | # Select neural network architecture 63 | if network == "ConvNet": 64 | # Fetch architecture 65 | model = cnn.CNNArchitecture.select(network, 150, 150, 3, 3) 66 | # Fetch weights 67 | weights = get_file('tankbuster_conv_weights.h5', CONVNET_WEIGHTS, 68 | cache_subdir='models') 69 | 70 | # Load weights 71 | model.load_weights(weights) 72 | 73 | # Return the prediction 74 | predictions = model.predict(normalized, verbose=0)[0] 75 | 76 | # Create a dictionary of predictions 77 | label_probs = {'bmp': (predictions[0]), 78 | 't-72': (predictions[1]), 79 | 'other': (predictions[2])} 80 | 81 | return label_probs 82 | 83 | elif network == "ResNet": 84 | # Fetch base model architecture 85 | base_model = cnn.CNNArchitecture.select(network, 224, 224, 3) 86 | 87 | # Extract features using ResNet50 88 | features = base_model.predict(normalized, batch_size=1, verbose=0) 89 | 90 | # Construct the top model 91 | top_model = cnn.CNNArchitecture.select("TopNet", features.shape[1:]) 92 | 93 | # Fetch topnet weights 94 | weights = get_file('tankbuster_res_weights.h5', RESNET_WEIGHTS, 95 | cache_subdir='models') 96 | # Load weights 97 | top_model.load_weights(weights) 98 | 99 | # Return the prediction 100 | predictions = top_model.predict(features, verbose=0)[0] 101 | 102 | # Create a dictionary of predictions 103 | label_probs = {'bmp': (predictions[0]), 104 | 't-72': (predictions[1]), 105 | 'other': (predictions[2])} 106 | 107 | return label_probs 108 | 109 | # If the requested network is not valid, quit and print an error message 110 | else: 111 | quit("'{}' is not a valid network. Terminating ...".format(network)) 112 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Tankbuster 2 | 3 | Tankbuster is a neural network trained to recognize Soviet/Russian T-72 main battle tanks and BMP armored personnel carriers in photographs. 4 | 5 | Built using Keras, the network has been trained using a collection of images showing T-72s (1457 images) and BMPs (1088 images) from various angles and at various distances against a collection of photographs featuring street and natural scenes (1477 images). 6 | 7 | Tankbuster provides two different architectures and models for recognizing the aforementioned vehicles / classes. 8 | 9 | The first alternative is a small convolutional neural net (ConvNet), which has been trained using augmented data, generating additional training data from the source images by introducing random shifts, flips, zooms and shears. The convolutional neural network achieves a 10-fold validation accuracy of 79%. 10 | 11 | The second alternative is a model based on a 50-layer residual neural network pre-trained on ImageNet (ResNet), which ships with Keras. This network has been used as a feature extractor, recording the output from the final average pooling layer and training a fully-connected block on top. The model achieves a 10-fold validation accuracy of 95%! 12 | 13 | While the convolutional neural network is faster, especially when running tankbuster on a CPU, the residual neural network provides superior performance, and frankly, generalizes much better. 14 | 15 | ## Installation 16 | 17 | Tankbuster requires Keras (2.0.2 or later), Pillow (4.1.0 or later) and TensorFlow (1.0.1 or later) and their dependencies, which may be all installed effortlessly from the Python Package Index by entering the following commands on the command line prompt: 18 | 19 | pip install tankbuster 20 | 21 | These commands will install the required libraries and their dependencies. As usual, it is a very good idea to install Tankbuster and its dependencies in its own virtual environment. 22 | 23 | Naturally, you can also clone the repository. To do so, enter the following command: 24 | 25 | git clone https://github.com/thiippal/tankbuster.git 26 | 27 | This will clone the repository into the subdirectory tankbuster, which includes the driver script bust.py. 28 | 29 | ## Usage 30 | 31 | ### Running Tankbuster from the command line 32 | 33 | If you wish to run Tankbuster from the command line, use the driver script bust.py. 34 | 35 | To feed a single image to the neural network, use the command line prompt to enter the following command: 36 | 37 | python bust.py -i image.jpg -n ResNet 38 | 39 | Alternatively, to use the faster convolutional neural network, replace "ResNet" with "ConvNet". 40 | 41 | To examine all images in a directory, simply enter a directory name instead of the filename: 42 | 43 | python bust.py -i directory -n ResNet 44 | 45 | ### Integrating Tankbuster into your own program 46 | 47 | If you wish to integrate Tankbuster into your Python program, import the key function of the module using: 48 | 49 | from tankbuster import bust 50 | 51 | This allows you to call the bust function, which takes a path to an image as the input: 52 | 53 | bust('image.png', network="ResNet") 54 | 55 | Again, if you wish to use the convolutional neural network instead of the residual neural network, pass the "ResNet" to the parameter network instead of "ConvNet". 56 | 57 | The bust function returns a dictionary of predicted class labels with their associated probabilities. These labels and probabilities can be used as the basis for further actions, such as flagging the image for further inspection. 58 | 59 | ## In action 60 | 61 | Parked next to a Lada
62 | ResNet: predicted T-72, probability 98.89%.
63 | ConvNet: predicted T-72, probability 95.44%. 64 | 65 | 66 | 67 | 68 | Featured in a low-quality screen capture
69 | ResNet: predicted T-72, probability 97.42%
70 | ConvNet: predicted BMP, probability 75.57%. 71 | 72 | 73 | 74 | 75 | Parked in the backyard
76 | ResNet: predicted BMP, probability 98.89%.
77 | ConvNet: predicted BMP, probability 49.17%. 78 | 79 | 80 | 81 | 82 | Knocked out in group
83 | ResNet: predicted BMP, probability 99.14%.
84 | ConvNet: predicted T-72, probability 41.63%. 85 | 86 | 87 | 88 | --------------------------------------------------------------------------------