├── README.md ├── mnist.py └── autoencoder.py /README.md: -------------------------------------------------------------------------------- 1 | # deep-autoencoder-using-keras 2 | 3 | ## Getting Started 4 | An autoencoder is a type of artificial neural network used to learn efficient data codings in an unsupervised manner. The aim of an autoencoder is to learn a representation (encoding) for a set of data, typically for dimensionality reduction, by training the network to ignore signal “noise.” Along with the reduction side, a reconstructing side is learnt, where the autoencoder tries to generate from the reduced encoding a representation as close as possible to its original input, hence its name. 5 | 6 | In this example I use an autoencoder to encode/decode images from the MNIST dataset. In the first step the images are encoded to a smaller dimension (type of dimensionality reduction). After that the task of the decoder is to decode that lower dim. image to its original form. 7 | 8 | ![images](https://user-images.githubusercontent.com/28685502/43400785-396a18f8-942c-11e8-9251-807f13d5dd1c.png) 9 | 10 | 11 | ## Dataset 12 | The code downloads the mnnist dataset automatically if it does not exist on your local machine. Alternatively, download the four zip fles and place them in a directory and name it according to the code. 13 | 14 | ## Results 15 | ![untitled](https://user-images.githubusercontent.com/28685502/43400628-d4d02928-942b-11e8-9895-1adcfe62d4d6.png) 16 | -------------------------------------------------------------------------------- /mnist.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import urllib.request 3 | import gzip 4 | import pickle 5 | 6 | filename = [ 7 | ["training_images","train-images-idx3-ubyte.gz"], 8 | ["test_images","t10k-images-idx3-ubyte.gz"], 9 | ["training_labels","train-labels-idx1-ubyte.gz"], 10 | ["test_labels","t10k-labels-idx1-ubyte.gz"] 11 | ] 12 | 13 | def download_mnist(): 14 | base_url = "http://yann.lecun.com/exdb/mnist/" 15 | for name in filename: 16 | print("Downloading "+name[1]+"...") 17 | urllib.request.urlretrieve(base_url+name[1], name[1]) 18 | print("Download complete.") 19 | 20 | def save_mnist(): 21 | mnist = {} 22 | for name in filename[:2]: 23 | with gzip.open(name[1], 'rb') as f: 24 | mnist[name[0]] = np.frombuffer(f.read(), np.uint8, offset=16).reshape(-1,28*28) 25 | for name in filename[-2:]: 26 | with gzip.open(name[1], 'rb') as f: 27 | mnist[name[0]] = np.frombuffer(f.read(), np.uint8, offset=8) 28 | with open("mnist.pkl", 'wb') as f: 29 | pickle.dump(mnist,f) 30 | print("Save complete.") 31 | 32 | def init(): 33 | download_mnist() 34 | save_mnist() 35 | 36 | def load(): 37 | with open("mnist.pkl",'rb') as f: 38 | mnist = pickle.load(f) 39 | return mnist["training_images"], mnist["training_labels"], mnist["test_images"], mnist["test_labels"] 40 | 41 | if __name__ == '__main__': 42 | init() 43 | -------------------------------------------------------------------------------- /autoencoder.py: -------------------------------------------------------------------------------- 1 | """ 2 | This programm tries with the help of an autoencoder to encode images from the 3 | MNIST dataset to a lower dimension and decode them as good as possible. 4 | It will be interesting to see what the lowest encoding dimesion can be without 5 | loosing a lot of accuracy. 6 | 7 | Training data: 8 | X: (60000, 784) 9 | y: (60000, 1) 10 | 11 | Test data: 12 | X: (10000, 784) 13 | y: (10000, 1) 14 | """ 15 | import numpy as np 16 | from matplotlib import pyplot as plt 17 | from sklearn.preprocessing import Normalizer 18 | from keras.models import Sequential 19 | from keras.layers import Dense 20 | 21 | import mnist 22 | 23 | X_train, y_train, X_test, y_test = mnist.load() 24 | 25 | # this works relatively well. 26 | EPOCHS = 15 27 | BATCH_SIZE = 64 28 | OPTIMIZER = 'adam' 29 | LOSS_FUNCTION = 'kullback_leibler_divergence' 30 | ACTIVATION_FUNCTION = 'relu' # the last activation needs to be softmax. 31 | 32 | # how many nodes for each layer. 33 | # decoder 34 | INPUT_LAYER = 784 35 | ENCODER_LAYER_1 = 256 36 | ENCODER_LAYER_2 = 128 37 | ENCODER_LAYER_3 = 64 38 | # encoder 39 | DECODER_LAYER_1 = 128 40 | DECODER_LAYER_2 = 256 41 | DECODER_LAYER_3 = 784 42 | 43 | def normalize(X): 44 | """Normalizes X. 45 | 46 | Args: 47 | X (np.array): The image as an array. Shape=(784,). 48 | 49 | Returns: 50 | X (np.array): The image as an array normalized. Shape=(784,). 51 | normalizer (sklearn normalizer object): Normalizer object. 52 | """ 53 | normalizer = Normalizer() 54 | X = normalizer.fit_transform(X) 55 | 56 | return X, normalizer 57 | 58 | def show_digit(X, digit): 59 | """Visualizes one array as an image from the MNIST dataset. 60 | 61 | Args: 62 | X (np.array): The image as an array. Shape=(784,). 63 | digit (int): The digit shown in the image. 64 | """ 65 | pixels = X.reshape((28, 28)) 66 | plt.title(str(digit)) 67 | plt.imshow(pixels, cmap='gray') 68 | plt.show() 69 | 70 | def show_digit_before_after(X, digit, autoencoder, normalizer): 71 | """Visualizes one digit before encoding and another one after decoding. 72 | 73 | Args: 74 | X (np.array): The image as an array. Shape=(784,). 75 | digit (int): The digit shown in the image. 76 | autoencoder (keras.models): The trained autoencoder. 77 | normalizer (sklearn.preprocessing.Normalizer): Object used for 78 | normalization. 79 | """ 80 | # image before. 81 | show_digit(X, digit) 82 | 83 | # image after. 84 | X = X.reshape(1,784) 85 | 86 | # normalize. 87 | X = normalizer.transform(X) 88 | image_after = autoencoder.predict(X) 89 | show_digit(image_after, digit) 90 | 91 | 92 | def train(X): 93 | """Trains the autoencoder. 94 | 95 | Args: 96 | X (np.array): The images. Shape(60000, 784) 97 | """ 98 | autoencoder = Sequential() 99 | autoencoder.add(Dense(units=ENCODER_LAYER_1, activation=ACTIVATION_FUNCTION, input_dim=INPUT_LAYER)) 100 | autoencoder.add(Dense(units=ENCODER_LAYER_2, activation=ACTIVATION_FUNCTION)) 101 | autoencoder.add(Dense(units=ENCODER_LAYER_3, activation=ACTIVATION_FUNCTION)) 102 | autoencoder.add(Dense(units=DECODER_LAYER_1, activation=ACTIVATION_FUNCTION)) 103 | autoencoder.add(Dense(units=DECODER_LAYER_2, activation=ACTIVATION_FUNCTION)) 104 | autoencoder.add(Dense(units=DECODER_LAYER_3, activation='softmax')) 105 | 106 | autoencoder.compile(loss=LOSS_FUNCTION, optimizer=OPTIMIZER, metrics=['accuracy']) 107 | autoencoder.fit(X, X, epochs=EPOCHS, batch_size=BATCH_SIZE) 108 | 109 | return autoencoder 110 | 111 | if __name__ == '__main__': 112 | # normalize. 113 | X_train, normalizer = normalize(X_train) 114 | autoencoder = train(X_train) 115 | 116 | digit_1 = X_train[1] 117 | label_1 = y_train[1] 118 | 119 | digit_2 = X_train[6] 120 | label_2 = y_train[6] 121 | 122 | digit_3 = X_train[16] 123 | label_3 = y_train[16] 124 | 125 | show_digit_before_after(digit_1, label_1, autoencoder, normalizer) 126 | show_digit_before_after(digit_2, label_2, autoencoder, normalizer) 127 | show_digit_before_after(digit_3, label_3, autoencoder, normalizer) 128 | --------------------------------------------------------------------------------