├── LICENSE ├── README.md └── MAC_Model.ipynb /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 Shayan K 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Deep Learning for Aircraft Recognition 2 | A CNN model trained to classify and identify various military aircraft through satellite imagery 3 | https://drive.google.com/drive/folders/1abQQCeZ92jGfF2C1GjnJQW8Mw_Jk9cw2 4 | 5 | ------------------------------------------------------------------------------------------------- 6 | 7 | ## Abstract 8 | 9 | Today, many military branches around the world are investing in Computer Vision for Intelligence, Surveillance and Reconnaissance (ISR). Some applications related to this field include, recognition of military vehicles in social media [1], recognition of camouflaged military targets [2], and battlefield object detection [3]. Military superpowers such as the United States are investing heavily in AI and Deep Learning, with image recognition as one of the main applications [4]. The main objective of this field is to enhance ISR capabilities. Nowadays, combat and reconnaissance drones are becoming increasingly significant and crucial in modern militaries around the world. One of the crucial elements in UAVs and UCAVs are the imaging capabilities from high altitudes. However, as it stands today, reconnaissance and remote intelligence gathering is still done through satellite imagery. This project focuses on the essentials of how Computer Vision can be used by militaries for ISR purposes. 10 | 11 | ## Objective 12 | 13 | In this project, we use a Convolutional Neural Network to classify a variety of military aircraft through satellite imagery. The project’s aim is to build a Convolutional Neural Network with the capability to distinguish and accurately classify aircraft through a snapshot of satellite imagery. Additionally, the model should be able to identify and detect all types of aircraft when presented with a large snapshot of an airbase, as well as classifying snapshots of a single aircraft type. Through the use of training data, the model will be accurate enough to identify individual aircraft in addition to being scalable to a large dataset. 14 | 15 | ## Design 16 | 17 | ### **Dataset Description** 18 | 19 | We developed a dataset composed of 9 different military aircraft, as well as a dataset for various backgrounds, such as the concrete tarmac of air bases or sandy grounds for certain boneyards and air bases throughout the world. Overall, the dataset is composed of 6,300 images, each being 120 x 120, with each aircraft type containing ~ 610 - 660 images. 20 | 21 | The aircraft types included in this dataset are: 22 | 23 | ![Capture](https://user-images.githubusercontent.com/19809069/115973276-c1de3080-a521-11eb-885a-4bdfdebb1288.JPG) 24 | 25 | ### **Dataset creation** 26 | 27 | Due to the lack of pre-existing publicly accessible datasets on military aircraft, we were forced to create our own through satellite imagery taken by Google Maps. We visited many different air force bases belonging to the United States throughout the world, taking individual snapshots of different types of military aircraft. The specific aircraft belong to the categories shown in the table below. 28 | 29 | ![Capture](https://user-images.githubusercontent.com/19809069/115973223-58f6b880-a521-11eb-88ca-bd76ed276f21.JPG) 30 | 31 | 32 | To account for abstraction, we decided to augment our dataset using the ImageDataGenerator function provided by Keras. This function allowed us to take 1 image and create (n*) number of images with different rotations, zoom level and horizontal flips of the same original image. For example, the B-2 image shown above can be flipped or rotated into figure 1, shown below. This method of data augmentation ensures that the dataset is balanced and decreases overfitting in the long run. It accounted for differently orientated images of aircraft taken at random positions and zoom levels. 33 | 34 | ![Capture](https://user-images.githubusercontent.com/19809069/115973235-6d3ab580-a521-11eb-9df6-2494bc57a17a.JPG) 35 | 36 | ### **Convolutional Neural Network model design** 37 | 38 | The model was designed based on two crucial aspects, one being our relatively small dataset and the second was to minimize cross validation loss as much as possible. To avoid overfitting, a relatively small model with less complexity is crucial given a small dataset size. Our convolutional neural network is composed of the following: 39 | 40 | ![Capture](https://user-images.githubusercontent.com/19809069/115973253-880d2a00-a521-11eb-8d4a-64df9310180d.JPG) 41 | 42 | ### **Model summary** 43 | 44 | ![Capture](https://user-images.githubusercontent.com/19809069/115973258-99563680-a521-11eb-96d1-4c93e491fb84.JPG) 45 | 46 | 47 | With the help of the hyperparameter tuning function provided by Keras, we were able to find a model which focused on achieving the highest cross validation accuracy and the lowest cross validation loss value. The statistics for the model can be seen in Figure 2. The highest cross validation accuracy the model was able to achieve is ~ 91%, with the lowest cross validation accuracy achieved being ~ 0.3322. The training accuracy is ~ 98% and the training loss is ~ 0.0610. The model was compiled with the Adam optimizer with a learning rate of 0.0001 using the sparse categorical cross entropy loss method. The model was fitted with 30 epochs, a batch size of 64 and a validation split of .30. 48 | 49 | ![Capture](https://user-images.githubusercontent.com/19809069/115973270-aa06ac80-a521-11eb-956f-ac18f40f731e.JPG) 50 | 51 | ## Results 52 | 53 | We tested the model’s practical capabilities by using the predict function on several wide body snapshots of air force bases around the world. This was done in order to test two main capabilities of the model; the model’s ability to successfully identify and detect any type of aircraft, and its ability to identify the category that the detected aircraft belongs to. The model’s ability to detect aircraft can be seen in Figures 4, 5, 6 , 7. Furthermore, the model’s ability to accurately classify an aircraft can be seen in Figures 8, 9, 10, 11. During our testing phase, we noticed that the model tended to misclassify certain types of aircrafts. Taking a deeper look, we realize that the model can often have a hard time distinguishing between aircraft types with a high degree of similarities. For example, The E-3 and C-135 are both extremely similar aircrafts with more or less the same measurements and features. This means that the model needs to be trained to further identify unique characteristics rather than the general shape of the aircraft, which can be common amongst other aircrafts. This can be achieved through a more complex model, given that we expand on our dataset and improve its quality in terms of pixelation, data augmentation and balancing. 54 | 55 | ![Capture](https://user-images.githubusercontent.com/19809069/115973342-521c7580-a522-11eb-84fc-b7b7aa8812d5.JPG) 56 | 57 | ![Capture](https://user-images.githubusercontent.com/19809069/115973333-3b761e80-a522-11eb-8fe2-e1f9fb8f1a58.JPG) 58 | 59 | ![Capture](https://user-images.githubusercontent.com/19809069/115973316-079af900-a522-11eb-8739-a2370ec4ccb5.JPG) 60 | 61 | ![Capture](https://user-images.githubusercontent.com/19809069/115973323-1d102300-a522-11eb-9a92-c13b67382829.JPG) 62 | 63 | ![Capture](https://user-images.githubusercontent.com/19809069/115973330-2b5e3f00-a522-11eb-9ff2-823d32ee25b2.JPG) 64 | 65 | 66 | ## Conclusion 67 | 68 | The biggest take away from this experiment is the quality of the dataset and its significant effect on the model’s practical capabilities. Despite reaching a high cross validation accuracy of ~ 91%, the model can be trained to perform much better in both training and a practical setting. As described earlier in the results analysis, possessing a high quality dataset is crucial and beneficial for the model. The biggest improvement that we can make is to expand our dataset to around 15,000 images, 1,500 images for each category. Each category must include a high quality image with varying degrees of rotation, zoom levels and as well cut off points, where only distinguishable features are exposed. These improvements can drastically improve the accuracy of the model by minimizing misclassifications and generalizing to account for noise encountered in practical use. 69 | 70 | 71 | ## References 72 | 73 | [1] T. Hiippala, Recognizing Military Vehicles in Social Media Images Using Deep Learning (2017), Proceedings of the 2017 IEEE International Conference on Intelligence and Security Informatics. 74 | 75 | [2] A. Khashman, Automatic Detection of Military Targets Utilising Neural Networks and Scale Space Analysis (2000), RTO IST Symposium on “New Information Processing Techniques for Military Systems”. 76 | 77 | [3] S. Liu & Z. Liu, Multi-Channel CNN-based Object Detection for Enhanced Situational Awareness (2017), NATO Science and Technology Organisation Proceedings. 78 | 79 | [4] RAND Corporation, Military Applications of Artificial Intelligence (2020), RAND Research Reports. 80 | 81 | -------------------------------------------------------------------------------- /MAC_Model.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "accelerator": "GPU", 6 | "colab": { 7 | "name": "MAC.ipynb", 8 | "provenance": [], 9 | "collapsed_sections": [] 10 | }, 11 | "kernelspec": { 12 | "display_name": "Python 3", 13 | "name": "python3" 14 | }, 15 | "language_info": { 16 | "name": "python" 17 | } 18 | }, 19 | "cells": [ 20 | { 21 | "cell_type": "code", 22 | "metadata": { 23 | "id": "cBBPZDxlzLi_" 24 | }, 25 | "source": [ 26 | "import tensorflow as tf\n", 27 | "import tensorflow.keras.losses as losses\n", 28 | "from tensorflow import keras\n", 29 | "from tensorflow.keras.models import Sequential\n", 30 | "from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten\n", 31 | "from tensorflow.keras.layers import Conv2D, MaxPooling2D\n", 32 | "import tensorflow.keras.optimizers as optimizers\n", 33 | "!pip install -q -U keras-tuner\n", 34 | "from kerastuner.tuners import RandomSearch\n", 35 | "from kerastuner.engine.hyperparameters import HyperParameters\n", 36 | "import time\n", 37 | "LOG_DIR = f\"{int(time.time())}\"\n", 38 | "from keras.layers import BatchNormalization\n", 39 | "from tempfile import TemporaryFile\n", 40 | "import numpy as np\n", 41 | "import matplotlib.pyplot as plt\n", 42 | "import os\n", 43 | "import tensorflow.keras.regularizers as regularizers\n", 44 | "import cv2\n", 45 | "import random\n", 46 | "from PIL import Image\n", 47 | "import matplotlib.patches as patches\n", 48 | "from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img\n", 49 | "from google.colab import drive\n", 50 | "import time\n", 51 | "drive.mount('/content/drive/')\n", 52 | "dataset_dir = \"/content/drive/My Drive/dataset_aircraft\"\n", 53 | " \n", 54 | "training_data = []\n", 55 | "test_data = []\n", 56 | "RESIZE_VALUE = 130\n", 57 | "root_path = \"/content/drive/My Drive/dataset_aircraft/\"" 58 | ], 59 | "execution_count": null, 60 | "outputs": [] 61 | }, 62 | { 63 | "cell_type": "code", 64 | "metadata": { 65 | "id": "UTMV6ekteaG4" 66 | }, 67 | "source": [ 68 | "drive.mount('/content/drive/')\n", 69 | "dataset_dir = \"/content/drive/My Drive/dataset_aircraft\"" 70 | ], 71 | "execution_count": null, 72 | "outputs": [] 73 | }, 74 | { 75 | "cell_type": "code", 76 | "metadata": { 77 | "id": "lJ3j-wDnnBmi" 78 | }, 79 | "source": [ 80 | "def image_process():\n", 81 | " #image processing (dataset image randomizer)\n", 82 | " plane = \"c/\"\n", 83 | " datagen = ImageDataGenerator(\n", 84 | " rotation_range=360,\n", 85 | " width_shift_range=0,\n", 86 | " height_shift_range=0,\n", 87 | " shear_range=0,\n", 88 | " zoom_range=0.0,\n", 89 | " horizontal_flip=True,\n", 90 | " fill_mode='nearest')\n", 91 | " \n", 92 | " selected_path = root_path+plane\n", 93 | " for img_ in os.listdir(selected_path):\n", 94 | " img = load_img(selected_path+img_) # this is a PIL image\n", 95 | " x = img_to_array(img) # this is a Numpy array with shape (3, 150, 150)\n", 96 | " x = x.reshape((1,) + x.shape) # this is a Numpy array with shape (1, 3, 150, 150)\n", 97 | "\n", 98 | " # the .flow() command below generates batches of randomly transformed images\n", 99 | " # and saves the results to the `preview/` directory\n", 100 | " i = 0\n", 101 | " for batch in datagen.flow(x, batch_size=1,\n", 102 | " save_to_dir=root_path+plane, save_prefix=img_, save_format='jpeg'):\n", 103 | " i += 1\n", 104 | " if i > 20: # the number of image generation (amendments) per selected picture\n", 105 | " break\n", 106 | "image_process()" 107 | ], 108 | "execution_count": 61, 109 | "outputs": [] 110 | }, 111 | { 112 | "cell_type": "code", 113 | "metadata": { 114 | "id": "fBYS_Q7wzaeO" 115 | }, 116 | "source": [ 117 | "# outputs the names of the files currently in the google drive file (dataset) \n", 118 | "types_path = \"/content/drive/My Drive/dataset_aircraft/TYPE-NAMES.txt\"\n", 119 | "aircraft_types = []\n", 120 | "f = open(types_path, \"r\")\n", 121 | "for types in f.read().split():\n", 122 | " aircraft_types.append(types.replace('\"',''))\n", 123 | "print(len(aircraft_types))\n", 124 | "f.close()\n", 125 | "print(\"Selected aircraft files:\",aircraft_types)\n", 126 | "print(\"Files in dataset (Google drive):\",os.listdir(root_path))" 127 | ], 128 | "execution_count": null, 129 | "outputs": [] 130 | }, 131 | { 132 | "cell_type": "code", 133 | "metadata": { 134 | "id": "BfvpfKSAcbxG" 135 | }, 136 | "source": [ 137 | "def create_training_data():\n", 138 | " # function iterates through each file in the google drive dataset and converts each image into an array\n", 139 | " for type in aircraft_types:\n", 140 | " selected_path = root_path + type\n", 141 | " class_num = aircraft_types.index(type)\n", 142 | " for img in os.listdir(selected_path):\n", 143 | " try:\n", 144 | " image = Image.open(os.path.join(selected_path,img))\n", 145 | " img_array = np.array(image)\n", 146 | " aircraft_img = cv2.resize(img_array, (RESIZE_VALUE, RESIZE_VALUE))\n", 147 | " training_data.append([aircraft_img, class_num])\n", 148 | " except Exception as e:\n", 149 | " print(\"error:\",selected_path,image)\n", 150 | " \n", 151 | "create_training_data()\n", 152 | "random.shuffle(training_data)\n", 153 | "print(len(training_data))\n", 154 | "print(training_data[0][1])\n", 155 | "plt.imshow(training_data[0][0])" 156 | ], 157 | "execution_count": null, 158 | "outputs": [] 159 | }, 160 | { 161 | "cell_type": "code", 162 | "metadata": { 163 | "id": "cYF0U-hgchzb" 164 | }, 165 | "source": [ 166 | "# saves all generated arrays of images to a file\n", 167 | "np.savez_compressed(root_path+'primary_training_dataset.npz', a=training_data)" 168 | ], 169 | "execution_count": null, 170 | "outputs": [] 171 | }, 172 | { 173 | "cell_type": "code", 174 | "metadata": { 175 | "colab": { 176 | "base_uri": "https://localhost:8080/" 177 | }, 178 | "id": "CxKXkcrZ5flI", 179 | "outputId": "025c6bd0-e7cb-44ea-a0eb-11ddeee9946b" 180 | }, 181 | "source": [ 182 | "def load_training_data():\n", 183 | " #LOAD TRAINING DATA INTO ARRAY FROM FILE\n", 184 | " loaded = np.load(root_path+\"primary_training_dataset.npz\", allow_pickle=True)\n", 185 | " training_data = loaded['a']\n", 186 | " for x in range(len(training_data)):\n", 187 | " if (training_data[x][0].shape[2] == 4):\n", 188 | " training_data[x][0] = cv2.cvtColor(training_data[x][0], cv2.COLOR_BGRA2BGR)\n", 189 | " return training_data\n", 190 | "training_data = load_training_data()\n", 191 | "random.shuffle(training_data)\n", 192 | "print(len(training_data))" 193 | ], 194 | "execution_count": 153, 195 | "outputs": [ 196 | { 197 | "output_type": "stream", 198 | "text": [ 199 | "6272\n" 200 | ], 201 | "name": "stdout" 202 | } 203 | ] 204 | }, 205 | { 206 | "cell_type": "code", 207 | "metadata": { 208 | "id": "eOhCn2Dfzlvj" 209 | }, 210 | "source": [ 211 | "def load_testing():\n", 212 | " #LOAD TESTING DATA INTO ARRAY FROM FILE\n", 213 | " loaded = np.load(root_path+\"primary_training_dataset.npz\", allow_pickle=True)\n", 214 | " training_data = loaded['a']\n", 215 | " for x in range(len(training_data)):\n", 216 | " if (training_data[x][0].shape[2] == 4):\n", 217 | " training_data[x][0] = cv2.cvtColor(training_data[x][0], cv2.COLOR_BGRA2BGR)\n", 218 | " return training_data\n", 219 | "testing_data = load_testing()\n", 220 | "random.shuffle(testing_data)\n", 221 | "print(len(testing_data))" 222 | ], 223 | "execution_count": null, 224 | "outputs": [] 225 | }, 226 | { 227 | "cell_type": "code", 228 | "metadata": { 229 | "id": "Q-qmz7-9zy--" 230 | }, 231 | "source": [ 232 | "x_train = []\n", 233 | "y_train = []\n", 234 | " \n", 235 | "#x_test = []\n", 236 | "#y_test = []\n", 237 | " \n", 238 | "#for data, label in testing_data:\n", 239 | "# x_test.append(data)\n", 240 | "# y_test.append(label)\n", 241 | " \n", 242 | "for data, label in training_data:\n", 243 | " x_train.append(data)\n", 244 | " y_train.append(label)\n", 245 | " \n", 246 | "x_train = np.asarray(x_train)\n", 247 | "x_train = np.array(x_train).reshape(-1, RESIZE_VALUE, RESIZE_VALUE, 3)\n", 248 | "y_train = np.asarray(y_train)\n", 249 | " \n", 250 | "#x_test = np.asarray(x_test)\n", 251 | "#x_test = np.array(x_test).reshape(-1, RESIZE_VALUE, RESIZE_VALUE, 3)\n", 252 | "#y_test = np.asarray(y_test)\n", 253 | "\n", 254 | "x_train = x_train / 255\n", 255 | "#x_test = x_test / 255\n", 256 | "print(len(x_train))\n", 257 | "#print(len(x_test))" 258 | ], 259 | "execution_count": null, 260 | "outputs": [] 261 | }, 262 | { 263 | "cell_type": "code", 264 | "metadata": { 265 | "id": "7PGRuzpvz2bq" 266 | }, 267 | "source": [ 268 | "\n", 269 | "model = keras.models.load_model(root_path+\"Model V6\")" 270 | ], 271 | "execution_count": 210, 272 | "outputs": [] 273 | }, 274 | { 275 | "cell_type": "code", 276 | "metadata": { 277 | "id": "ChltQUnNLDjK" 278 | }, 279 | "source": [ 280 | "model.summary()" 281 | ], 282 | "execution_count": null, 283 | "outputs": [] 284 | }, 285 | { 286 | "cell_type": "code", 287 | "metadata": { 288 | "id": "F1EA_eWP7vfR" 289 | }, 290 | "source": [ 291 | "def build_model_MAC():\n", 292 | " # main model\n", 293 | " model = Sequential()\n", 294 | " model.add(Conv2D(32, (3, 3), input_shape = (RESIZE_VALUE,RESIZE_VALUE,3)))\n", 295 | " model.add(Activation(\"relu\"))\n", 296 | " model.add(MaxPooling2D(pool_size=(2, 2)))\n", 297 | " \n", 298 | " model.add(Conv2D(64, (3, 3)))\n", 299 | " model.add(Activation(\"relu\"))\n", 300 | " model.add(MaxPooling2D(pool_size=(2, 2)))\n", 301 | " \n", 302 | " model.add(Conv2D(32, (3, 3)))\n", 303 | " model.add(Activation(\"relu\"))\n", 304 | " model.add(MaxPooling2D(pool_size=(2, 2)))\n", 305 | "\n", 306 | "\n", 307 | " \n", 308 | " model.add(Flatten())\n", 309 | " \n", 310 | " model.add(Dense(450))\n", 311 | " model.add(Activation('relu'))\n", 312 | " model.add(Dropout(.65))\n", 313 | " \n", 314 | " model.add(Dense(len(aircraft_types)))\n", 315 | " model.add(Activation(\"softmax\"))\n", 316 | "\n", 317 | " opt = keras.optimizers.Adam(learning_rate=0.0001)\n", 318 | " model.compile(loss='sparse_categorical_crossentropy',\n", 319 | " optimizer=opt,\n", 320 | " metrics=['acc'])\n", 321 | " \n", 322 | " return model\n", 323 | "\n", 324 | "model = build_model_MAC()\n", 325 | "epochs = 30\n", 326 | "x = np.arange(epochs) + 1\n", 327 | "history_ = model.fit(x_train, y_train, epochs=epochs, validation_split=.30, batch_size=64)\n", 328 | "plt.plot(x,history_.history['acc'])\n", 329 | "plt.plot(x,history_.history['val_acc'])\n", 330 | "plt.xlabel(\"Epochs\");\n", 331 | "plt.ylabel(\"Accuracy\")\n", 332 | "plt.legend(['Training Accuacy', 'Cross Validation Accuracy']);\n", 333 | "\n", 334 | "fig, ax = plt.subplots()\n", 335 | "ax.plot(x,history_.history['loss'])\n", 336 | "ax.plot(x,history_.history['val_loss'])\n", 337 | "plt.xlabel(\"Epochs\");\n", 338 | "plt.ylabel(\"Loss\")\n", 339 | "plt.legend(['Training Loss', 'Cross Validation Loss']);\n" 340 | ], 341 | "execution_count": null, 342 | "outputs": [] 343 | }, 344 | { 345 | "cell_type": "code", 346 | "metadata": { 347 | "id": "Dt_TtM2Dx1MN" 348 | }, 349 | "source": [ 350 | "model.save(root_path+\"Model V6\")" 351 | ], 352 | "execution_count": null, 353 | "outputs": [] 354 | }, 355 | { 356 | "cell_type": "code", 357 | "metadata": { 358 | "id": "GhG6eJdgHNBC" 359 | }, 360 | "source": [ 361 | "def testSingleAircraft():\n", 362 | " # function takes a single image of an aircraft and classifies the aircraft category\n", 363 | " img = \"C-17.JPG\"\n", 364 | " image = Image.open(os.path.join(root_path+\"/Testing Images/\",img))\n", 365 | " img_array = np.array(image)\n", 366 | " aircraft_img = cv2.resize(img_array, (RESIZE_VALUE, RESIZE_VALUE))\n", 367 | " patch = np.expand_dims(aircraft_img, axis=0)\n", 368 | " plt.imshow(aircraft_img)\n", 369 | " x = model.predict_classes(patch)\n", 370 | " print (\"True value:\", img)\n", 371 | " print(\"Identified as:\",aircraft_types[x[0]],\"\\n\")" 372 | ], 373 | "execution_count": null, 374 | "outputs": [] 375 | }, 376 | { 377 | "cell_type": "code", 378 | "metadata": { 379 | "id": "aUotZsyVHPNg" 380 | }, 381 | "source": [ 382 | "def testMultipleAircraft():\n", 383 | " #function takes a wide picture consisting of multiple aircrafts and marks aircrafts\n", 384 | " img = \"1.JPG\"\n", 385 | " image = Image.open(os.path.join(root_path+\"/Testing Images/\",img))\n", 386 | " img_array = np.array(image)\n", 387 | " img_shape = img_array.shape\n", 388 | " aircraft_img = cv2.resize(img_array, (img_shape[1], img_shape[0]))\n", 389 | " test_img = cv2.cvtColor(aircraft_img, cv2.COLOR_BGRA2BGR)\n", 390 | " plt.imshow(test_img)\n", 391 | " ax = plt.gca()\n", 392 | " pred_dist = np.arange(len(aircraft_types))\n", 393 | " classif = []\n", 394 | " detectCount = 0;\n", 395 | " counter = 0\n", 396 | " # set length and width of patch\n", 397 | " length = 80\n", 398 | " width = 80\n", 399 | " # do not show list\n", 400 | " dshow = [\"BareLand\"]\n", 401 | " # iterate through the entire image and predict each patch\n", 402 | " for i in range(0,test_img.shape[0], 30):\n", 403 | " for j in range (0,test_img.shape[1], 30):\n", 404 | " if (j+length < test_img.shape[1] and i+width < test_img.shape[0]):\n", 405 | " raw_patch = test_img[i:i+length, j:j+width]\n", 406 | " aircraft_img = cv2.resize(raw_patch, (RESIZE_VALUE, RESIZE_VALUE))\n", 407 | " patch = np.expand_dims(aircraft_img, axis=0)\n", 408 | " ident = model.predict_classes(patch)\n", 409 | " pred_dist[ident[0]] += 1\n", 410 | " u = aircraft_types[ident[0]]\n", 411 | " # do not mark patch if patch is classified as bareland (background)\n", 412 | " if (u not in dshow):\n", 413 | " # mark the identified patch with a red rectangle ROI, region of interest\n", 414 | " ax.add_patch(patches.Rectangle((j, i), (j+length)-j, (i+width)-i, facecolor='none',edgecolor='red', linewidth=.5))\n", 415 | "\n", 416 | " plt.show()\n", 417 | "\n", 418 | "testMultipleAircraft()" 419 | ], 420 | "execution_count": null, 421 | "outputs": [] 422 | }, 423 | { 424 | "cell_type": "code", 425 | "metadata": { 426 | "id": "Mwn22NtlSPXT" 427 | }, 428 | "source": [ 429 | "def build_model(hp):\n", 430 | " # this function uses hyperparameter method provided by keras to fine tune a model \n", 431 | " model = Sequential()\n", 432 | " \n", 433 | " model.add(Conv2D(hp.Int('conv1', 32,64,12), (3, 3), input_shape = (RESIZE_VALUE,RESIZE_VALUE,3)))\n", 434 | " model.add(Activation(\"relu\"))\n", 435 | " model.add(MaxPooling2D(pool_size=(2, 2)))\n", 436 | " \n", 437 | " model.add(Conv2D(hp.Int('conv2', 32,64,12), (3, 3)))\n", 438 | " model.add(Activation(\"relu\"))\n", 439 | " model.add(MaxPooling2D(pool_size=(2, 2)))\n", 440 | " \n", 441 | " model.add(Conv2D(hp.Int('conv3', 32,64,12), (3, 3)))\n", 442 | " model.add(Activation(\"relu\"))\n", 443 | " model.add(MaxPooling2D(pool_size=(2, 2)))\n", 444 | "\n", 445 | "\n", 446 | "\n", 447 | " \n", 448 | " model.add(Flatten())\n", 449 | " \n", 450 | " model.add(Dense(hp.Int('dense1', 400,1000,100)))\n", 451 | " model.add(Activation('relu'))\n", 452 | " model.add(\n", 453 | " Dropout(rate=hp.Float(\n", 454 | " 'dropout_1',\n", 455 | " min_value=0.40,\n", 456 | " max_value=0.7,\n", 457 | " default=0.50,\n", 458 | " step=0.10,\n", 459 | " )))\n", 460 | " \n", 461 | " model.add(Dense(len(aircraft_types)))\n", 462 | " model.add(Activation(\"softmax\"))\n", 463 | "\n", 464 | " opt = keras.optimizers.Adam(learning_rate=0.0001)\n", 465 | " model.compile(loss='sparse_categorical_crossentropy',\n", 466 | " optimizer=opt,\n", 467 | " metrics=['acc'])\n", 468 | " \n", 469 | " return model\n", 470 | "\n", 471 | "def tuner():\n", 472 | " tuner = RandomSearch(\n", 473 | " build_model,\n", 474 | " objective=\"val_acc\",\n", 475 | " max_trials=20,\n", 476 | " executions_per_trial = 1,\n", 477 | " directory = LOG_DIR\n", 478 | " )\n", 479 | "\n", 480 | " tuner.search(x=x_train,\n", 481 | " y=y_train,\n", 482 | " epochs=50,\n", 483 | " batch_size=80,\n", 484 | " validation_split=.35)\n", 485 | "\n", 486 | "#tuner()" 487 | ], 488 | "execution_count": null, 489 | "outputs": [] 490 | } 491 | ] 492 | } --------------------------------------------------------------------------------