├── .gitignore ├── LICENSE ├── README.md ├── gen_dataset_images.py ├── models.py ├── pictures ├── autoencoder.png ├── convolutional_autoencoder ├── convolutional_autoencoder.png ├── deep_autoencoder.png ├── fashion_mnist.png ├── mnist.png ├── result_ep1.png ├── result_ep10.png ├── result_ep15.png ├── result_ep20.png └── result_ep5.png └── train.py /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__/ 2 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 Otenim 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # AnomalyDetectionUsingAutoencoder 2 | 3 | ## Overview 4 | 5 | We tried comparing three models: (1) autoencoder, (2) deep\_autoencoder, 6 | and (3) convolutional\_autoencoder in terms of capability of anomaly detection. 7 | 8 | In anomaly detection using autoencoders, we train an autoencoder on only normal 9 | dataset. So, when an input data that have different features from normal dataset are fed to 10 | the model, the corresponding reconstruction error will increase. We call such input data "abnormal data" here. 11 | 12 | ## Model Architecture 13 | 14 | #### autoencoder 15 | ![autoencoder](https://i.imgur.com/Ccx6TAG.png) 16 | 17 | #### deep_autoencoder 18 | ![deep_autoencoder](https://i.imgur.com/ladN1EJ.png) 19 | 20 | #### convolutional_autoencoder 21 | ![conv_autoencoder](https://i.imgur.com/AGlKpwU.png) 22 | 23 | ## Datasets 24 | 25 | Details for these model architectures are written in `models.py`. 26 | 27 | **Normal dataset** 28 | 29 | ![mnist](https://i.imgur.com/ia2Cqxf.png) 30 | 31 | * 28\*28\*1 gray-scale images 32 | * training samples: 60,000 33 | * validation samples: 50 (randomly sampled from the original 10,000 samples) 34 | 35 | **Abnormal dataset** 36 | 37 | ![fashion](https://i.imgur.com/NhjuFnx.png) 38 | 39 | * 28\*28\*1 gray-scale images 40 | * validation samples: 50 (randomly sampled from the original 10,000 samples) 41 | 42 | ## Evaluation Procedure 43 | 44 | 1. Train an autoencoder on only normal dataset. 45 | 2. Compute losses for validation dataset that consists of normal validation dataset and 46 | abnormal dataset. 47 | 48 | The above procedure is to be executed for each above three models. 49 | 50 | ## Result 51 | 52 | Sample index (x-axis) 0\~49 correspond to losses computed on 50 normal 53 | validation samples, and 50\~99 correspond to losses computed on 50 abnormal samples. 54 | You can see losses during sample index 50\~99 increase as expected. 55 | However, contraty to my expectations, autoencoder and deep\_autoencoder are more accurate than convolutional\_autoencoder in this settings. 56 | Since generalization performance of convolutional layers are generally higher than fully-connected layers, it seems that convolutional\_autoencoder could reconstruct 57 | accurately even abnormal samples. 58 | It should be noted that 59 | good models and tricks in classification problems may not necessarily perform well 60 | in problems of anomaly detection using Autoencoder. 61 | 62 | ##### 1 training epochs 63 | 64 | ![result\_ep1](https://i.imgur.com/lrW93M0.png) 65 | 66 | ##### 5 training epochs 67 | 68 | ![result\_ep5](https://i.imgur.com/IY54UIU.png) 69 | 70 | ##### 10 training epochs 71 | 72 | ![result\_ep10](https://i.imgur.com/Gb69PQd.png) 73 | 74 | ## Script 75 | 76 | `$ python train.py [--result] [--epochs] [--batch_size] [--test_samples]` 77 | 78 | All the above arguments are optional. 79 | 80 | * `--result`: a path to result graph image. the default value is ./result.png 81 | * `--epochs`: training epochs. the default value is 10. 82 | * `--batch_size`: batch size during training. the default value is 64. 83 | * `--test_samples`: number of validation samples for each dataset (i.e., normal validation dataset and abnormal dataset). the default value is 50. 84 | 85 | ex) `$ python train.py --result ./result.png --epochs 10 --batch_size 64 --test_samples 500` 86 | 87 | Dependencies are as follows. 88 | 89 | * Keras==2.1.4 90 | * Tensorflow==1.4.0 91 | * Matplotlib==2.1.2 92 | * Numpy==1.14.0 93 | 94 | All the above dependencies can be installed by pip command. 95 | -------------------------------------------------------------------------------- /gen_dataset_images.py: -------------------------------------------------------------------------------- 1 | from keras.datasets import fashion_mnist, mnist 2 | from PIL import Image 3 | import numpy as np 4 | 5 | # create an image for Mnist 6 | (x_train, y_train), (_,_) = mnist.load_data() 7 | num_classes = 10 8 | imgs = [] 9 | for i in range(num_classes): 10 | ind = (y_train == i) 11 | imgs.append(x_train[ind][0]) 12 | img = np.concatenate(imgs, axis=-1) 13 | img = Image.fromarray(img) 14 | img.save('pictures/mnist.png') 15 | 16 | # create an image for FashionMnist 17 | (x_train, y_train), (_,_) = fashion_mnist.load_data() 18 | num_classes = 10 19 | imgs = [] 20 | for i in range(num_classes): 21 | ind = (y_train == i) 22 | imgs.append(x_train[ind][0]) 23 | img = np.concatenate(imgs, axis=-1) 24 | img = Image.fromarray(img) 25 | img.save('pictures/fashion_mnist.png') 26 | -------------------------------------------------------------------------------- /models.py: -------------------------------------------------------------------------------- 1 | from keras.models import Sequential 2 | from keras.layers import Dense, Conv2D, MaxPool2D, UpSampling2D 3 | 4 | def autoencoder(): 5 | input_shape=(784,) 6 | model = Sequential() 7 | model.add(Dense(64, activation='relu', input_shape=input_shape)) 8 | model.add(Dense(784, activation='sigmoid')) 9 | return model 10 | 11 | def deep_autoencoder(): 12 | input_shape=(784,) 13 | model = Sequential() 14 | model.add(Dense(128, activation='relu', input_shape=input_shape)) 15 | model.add(Dense(64, activation='relu')) 16 | model.add(Dense(128, activation='relu')) 17 | model.add(Dense(784, activation='sigmoid')) 18 | return model 19 | 20 | def convolutional_autoencoder(): 21 | 22 | input_shape=(28,28,1) 23 | n_channels = input_shape[-1] 24 | model = Sequential() 25 | model.add(Conv2D(32, (3,3), activation='relu', padding='same', input_shape=input_shape)) 26 | model.add(MaxPool2D(padding='same')) 27 | model.add(Conv2D(16, (3,3), activation='relu', padding='same')) 28 | model.add(MaxPool2D(padding='same')) 29 | model.add(Conv2D(8, (3,3), activation='relu', padding='same')) 30 | model.add(UpSampling2D()) 31 | model.add(Conv2D(16, (3,3), activation='relu', padding='same')) 32 | model.add(UpSampling2D()) 33 | model.add(Conv2D(32, (3,3), activation='relu', padding='same')) 34 | model.add(Conv2D(n_channels, (3,3), activation='sigmoid', padding='same')) 35 | return model 36 | 37 | def load_model(name): 38 | if name=='autoencoder': 39 | return autoencoder() 40 | elif name=='deep_autoencoder': 41 | return deep_autoencoder() 42 | elif name=='convolutional_autoencoder': 43 | return convolutional_autoencoder() 44 | else: 45 | raise ValueError('Unknown model name %s was given' % name) 46 | -------------------------------------------------------------------------------- /pictures/autoencoder.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/otenim/AnomalyDetectionUsingAutoencoder/974b01c9f27f97337da45efef71f002a4c695b70/pictures/autoencoder.png -------------------------------------------------------------------------------- /pictures/convolutional_autoencoder: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/otenim/AnomalyDetectionUsingAutoencoder/974b01c9f27f97337da45efef71f002a4c695b70/pictures/convolutional_autoencoder -------------------------------------------------------------------------------- /pictures/convolutional_autoencoder.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/otenim/AnomalyDetectionUsingAutoencoder/974b01c9f27f97337da45efef71f002a4c695b70/pictures/convolutional_autoencoder.png -------------------------------------------------------------------------------- /pictures/deep_autoencoder.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/otenim/AnomalyDetectionUsingAutoencoder/974b01c9f27f97337da45efef71f002a4c695b70/pictures/deep_autoencoder.png -------------------------------------------------------------------------------- /pictures/fashion_mnist.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/otenim/AnomalyDetectionUsingAutoencoder/974b01c9f27f97337da45efef71f002a4c695b70/pictures/fashion_mnist.png -------------------------------------------------------------------------------- /pictures/mnist.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/otenim/AnomalyDetectionUsingAutoencoder/974b01c9f27f97337da45efef71f002a4c695b70/pictures/mnist.png -------------------------------------------------------------------------------- /pictures/result_ep1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/otenim/AnomalyDetectionUsingAutoencoder/974b01c9f27f97337da45efef71f002a4c695b70/pictures/result_ep1.png -------------------------------------------------------------------------------- /pictures/result_ep10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/otenim/AnomalyDetectionUsingAutoencoder/974b01c9f27f97337da45efef71f002a4c695b70/pictures/result_ep10.png -------------------------------------------------------------------------------- /pictures/result_ep15.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/otenim/AnomalyDetectionUsingAutoencoder/974b01c9f27f97337da45efef71f002a4c695b70/pictures/result_ep15.png -------------------------------------------------------------------------------- /pictures/result_ep20.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/otenim/AnomalyDetectionUsingAutoencoder/974b01c9f27f97337da45efef71f002a4c695b70/pictures/result_ep20.png -------------------------------------------------------------------------------- /pictures/result_ep5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/otenim/AnomalyDetectionUsingAutoencoder/974b01c9f27f97337da45efef71f002a4c695b70/pictures/result_ep5.png -------------------------------------------------------------------------------- /train.py: -------------------------------------------------------------------------------- 1 | from keras.datasets import mnist, fashion_mnist 2 | from models import load_model 3 | import numpy as np 4 | import os 5 | import argparse 6 | import matplotlib 7 | matplotlib.use('Agg') 8 | import matplotlib.pyplot as plt 9 | import math 10 | 11 | curdir = os.path.dirname(os.path.abspath(__file__)) 12 | parser = argparse.ArgumentParser() 13 | parser.add_argument('--optimizer', choices=['adam','sgd','adagrad'], default='adam') 14 | parser.add_argument('--loss', choices=['mean_squared_error', 'binary_crossentropy'], default='mean_squared_error') 15 | parser.add_argument('--epochs', type=int, default=10) 16 | parser.add_argument('--batch_size', type=int, default=64) 17 | parser.add_argument('--test_samples', type=int, default=50) 18 | parser.add_argument('--result', default=os.path.join(curdir, 'result.png')) 19 | 20 | def main(args): 21 | 22 | # prepare normal dataset (Mnist) 23 | (x_train, _), (x_test, _) = mnist.load_data() 24 | x_train = x_train / 255. # normalize into [0,1] 25 | x_test = x_test / 255. 26 | 27 | # prapare abnormal dataset (Fashion Mnist) 28 | (_, _), (x_abnormal, _) = fashion_mnist.load_data() 29 | x_abnormal = x_abnormal / 255. 30 | 31 | # sample args.test_samples images from eaech of x_test and x_abnormal 32 | perm = np.random.permutation(args.test_samples) 33 | x_test = x_test[perm][:args.test_samples] 34 | x_abnormal = x_abnormal[perm][:args.test_samples] 35 | 36 | # train each model and test their capabilities of anomaly deteciton 37 | model_names = ['autoencoder', 'deep_autoencoder', 'convolutional_autoencoder'] 38 | for model_name in model_names: 39 | # instantiate model 40 | model = load_model(model_name) 41 | 42 | # reshape input data according to the model's input tensor 43 | if model_name == 'convolutional_autoencoder': 44 | x_train = x_train.reshape(-1,28,28,1) 45 | x_test = x_test.reshape(-1,28,28,1) 46 | x_abnormal = x_abnormal.reshape(-1,28,28,1) 47 | elif model_name == 'autoencoder' or model_name == 'deep_autoencoder': 48 | x_train = x_train.reshape(-1,28*28) 49 | x_test = x_test.reshape(-1,28*28) 50 | x_abnormal = x_abnormal.reshape(-1,28*28) 51 | else: 52 | raise ValueError('Unknown model_name %s was given' % model_name) 53 | 54 | # compile model 55 | model.compile(optimizer=args.optimizer, loss=args.loss) 56 | 57 | # train on only normal training data 58 | model.fit( 59 | x=x_train, 60 | y=x_train, 61 | epochs=args.epochs, 62 | batch_size=args.batch_size, 63 | ) 64 | 65 | # test 66 | x_concat = np.concatenate([x_test, x_abnormal], axis=0) 67 | losses = [] 68 | for x in x_concat: 69 | # compule loss for each test sample 70 | x = np.expand_dims(x, axis=0) 71 | loss = model.test_on_batch(x, x) 72 | losses.append(loss) 73 | 74 | # plot 75 | plt.plot(range(len(losses)), losses, linestyle='-', linewidth=1, label=model_name) 76 | 77 | # delete model for saving memory 78 | del model 79 | 80 | # create graph 81 | plt.legend(loc='best') 82 | plt.grid() 83 | plt.xlabel('sample index') 84 | plt.ylabel('loss') 85 | plt.savefig(args.result) 86 | plt.clf() 87 | 88 | 89 | 90 | 91 | if __name__ == '__main__': 92 | args = parser.parse_args() 93 | main(args) 94 | --------------------------------------------------------------------------------