├── 1936 ├── readme.txt ├── weights.75-0.0654-0.9801-0.0278-0.9913.hdf5 └── model.txt ├── 2180 ├── readme.txt ├── weights.71-0.0725-0.9776-0.0280-0.9917.hdf5 └── model.txt ├── 2374 ├── readme.txt ├── weights.71-0.0711-0.9778-0.0331-0.9900.hdf5 └── model.txt ├── 2720 ├── readme.txt ├── weights.54-0.0883-0.9727-0.0335-0.9898.hdf5 ├── weights.60-0.0893-0.9723-0.0338-0.9895.hdf5 ├── weights.71-0.0871-0.9733-0.0319-0.9895.hdf5 └── model.txt ├── 3089 ├── readme.txt ├── weights.63-0.0901-0.9721-0.0339-0.9893.hdf5 └── model.txt ├── 3197 ├── readme.txt ├── weights.63-0.0797-0.9760-0.0270-0.9925.hdf5 └── model.txt ├── 3295 ├── readme.txt ├── weights.58-0.0765-0.9764-0.0297-0.9915.hdf5 └── model.txt ├── 3354 ├── readme.txt ├── weights.70-0.0736-0.9768-0.0327-0.9915.hdf5 └── model.txt ├── 3462 ├── readme.txt ├── weights.36-0.0800-0.9744-0.0377-0.9905.hdf5 ├── weights.68-0.0751-0.9767-0.0364-0.9907.hdf5 ├── weights.74-0.0740-0.9774-0.0354-0.9903.hdf5 └── model.txt ├── 3650 ├── readme.txt ├── weights.30-0.0779-0.9749-0.0329-0.9905.hdf5 ├── weights.36-0.0714-0.9775-0.0349-0.9912.hdf5 ├── weights.39-0.0730-0.9775-0.0307-0.9918.hdf5 └── model.txt ├── 4142 ├── weights.26-0.0779-0.9758-0.0309-0.9912.hdf5 ├── weights.47-0.0697-0.9787-0.0286-0.9915.hdf5 ├── weights.49-0.0672-0.9798-0.0281-0.9908.hdf5 ├── readme.txt └── model.txt ├── 4862 ├── weights.29-0.0751-0.9766-0.0296-0.9907.hdf5 ├── weights.44-0.0684-0.9784-0.0296-0.9913.hdf5 ├── weights.49-0.0690-0.9783-0.0279-0.9918.hdf5 └── model.txt ├── 5238 ├── readme.txt ├── weights.08-0.0909-0.9720-0.0372-0.9895.hdf5 ├── weights.21-0.0718-0.9776-0.0362-0.9903.hdf5 ├── weights.24-0.0732-0.9769-0.0324-0.9912.hdf5 ├── weights.29-0.0679-0.9786-0.0301-0.9913.hdf5 └── model.txt ├── 6742 ├── weights.06-0.0320-0.9898-0.0425-0.9885.hdf5 ├── weights.15-0.0161-0.9949-0.0436-0.9897.hdf5 ├── weights.27-0.0098-0.9966-0.0435-0.9903.hdf5 └── model.txt ├── keras_cnn_test.py └── README.md /3089/readme.txt: -------------------------------------------------------------------------------- 1 | Replaced second third block with SeparableConv2D 2 | -------------------------------------------------------------------------------- /1936/readme.txt: -------------------------------------------------------------------------------- 1 | Replaced BatchNormalization with InstanceNormalization 2 | -------------------------------------------------------------------------------- /2720/readme.txt: -------------------------------------------------------------------------------- 1 | Replaced second and third conv blocks with SeparableConv2D 2 | -------------------------------------------------------------------------------- /3197/readme.txt: -------------------------------------------------------------------------------- 1 | Reduced number of neurons in dense layer from 20 to 18. 2 | -------------------------------------------------------------------------------- /3462/readme.txt: -------------------------------------------------------------------------------- 1 | Reduced number of neurons in dense layer from 24 to 20. 2 | -------------------------------------------------------------------------------- /3650/readme.txt: -------------------------------------------------------------------------------- 1 | Reduced number of kernels in the second layer from 16 to 12. 2 | -------------------------------------------------------------------------------- /2374/readme.txt: -------------------------------------------------------------------------------- 1 | Replaced the third layer with three SeparableConv2D with shared weights 2 | -------------------------------------------------------------------------------- /2180/readme.txt: -------------------------------------------------------------------------------- 1 | Decreased number of filters in second and following repeating layers to 26 2 | -------------------------------------------------------------------------------- /3354/readme.txt: -------------------------------------------------------------------------------- 1 | Replaced in second convlayer one convolution 3x3 with 3x1 and 1x3 convolutions. 2 | -------------------------------------------------------------------------------- /5238/readme.txt: -------------------------------------------------------------------------------- 1 | Achieved 99% on validation and test sets by using dropout. At the train set ~97.8. Without dropout achived only 98.7 2 | -------------------------------------------------------------------------------- /3295/readme.txt: -------------------------------------------------------------------------------- 1 | Reduced number of kernels in second fourth and fifth convolutional layers from 16 to 15. Added additional bacth normalization layers. 2 | -------------------------------------------------------------------------------- /1936/weights.75-0.0654-0.9801-0.0278-0.9913.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/1936/weights.75-0.0654-0.9801-0.0278-0.9913.hdf5 -------------------------------------------------------------------------------- /2180/weights.71-0.0725-0.9776-0.0280-0.9917.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/2180/weights.71-0.0725-0.9776-0.0280-0.9917.hdf5 -------------------------------------------------------------------------------- /2374/weights.71-0.0711-0.9778-0.0331-0.9900.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/2374/weights.71-0.0711-0.9778-0.0331-0.9900.hdf5 -------------------------------------------------------------------------------- /2720/weights.54-0.0883-0.9727-0.0335-0.9898.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/2720/weights.54-0.0883-0.9727-0.0335-0.9898.hdf5 -------------------------------------------------------------------------------- /2720/weights.60-0.0893-0.9723-0.0338-0.9895.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/2720/weights.60-0.0893-0.9723-0.0338-0.9895.hdf5 -------------------------------------------------------------------------------- /2720/weights.71-0.0871-0.9733-0.0319-0.9895.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/2720/weights.71-0.0871-0.9733-0.0319-0.9895.hdf5 -------------------------------------------------------------------------------- /3089/weights.63-0.0901-0.9721-0.0339-0.9893.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/3089/weights.63-0.0901-0.9721-0.0339-0.9893.hdf5 -------------------------------------------------------------------------------- /3197/weights.63-0.0797-0.9760-0.0270-0.9925.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/3197/weights.63-0.0797-0.9760-0.0270-0.9925.hdf5 -------------------------------------------------------------------------------- /3295/weights.58-0.0765-0.9764-0.0297-0.9915.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/3295/weights.58-0.0765-0.9764-0.0297-0.9915.hdf5 -------------------------------------------------------------------------------- /3354/weights.70-0.0736-0.9768-0.0327-0.9915.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/3354/weights.70-0.0736-0.9768-0.0327-0.9915.hdf5 -------------------------------------------------------------------------------- /3462/weights.36-0.0800-0.9744-0.0377-0.9905.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/3462/weights.36-0.0800-0.9744-0.0377-0.9905.hdf5 -------------------------------------------------------------------------------- /3462/weights.68-0.0751-0.9767-0.0364-0.9907.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/3462/weights.68-0.0751-0.9767-0.0364-0.9907.hdf5 -------------------------------------------------------------------------------- /3462/weights.74-0.0740-0.9774-0.0354-0.9903.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/3462/weights.74-0.0740-0.9774-0.0354-0.9903.hdf5 -------------------------------------------------------------------------------- /3650/weights.30-0.0779-0.9749-0.0329-0.9905.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/3650/weights.30-0.0779-0.9749-0.0329-0.9905.hdf5 -------------------------------------------------------------------------------- /3650/weights.36-0.0714-0.9775-0.0349-0.9912.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/3650/weights.36-0.0714-0.9775-0.0349-0.9912.hdf5 -------------------------------------------------------------------------------- /3650/weights.39-0.0730-0.9775-0.0307-0.9918.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/3650/weights.39-0.0730-0.9775-0.0307-0.9918.hdf5 -------------------------------------------------------------------------------- /4142/weights.26-0.0779-0.9758-0.0309-0.9912.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/4142/weights.26-0.0779-0.9758-0.0309-0.9912.hdf5 -------------------------------------------------------------------------------- /4142/weights.47-0.0697-0.9787-0.0286-0.9915.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/4142/weights.47-0.0697-0.9787-0.0286-0.9915.hdf5 -------------------------------------------------------------------------------- /4142/weights.49-0.0672-0.9798-0.0281-0.9908.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/4142/weights.49-0.0672-0.9798-0.0281-0.9908.hdf5 -------------------------------------------------------------------------------- /4862/weights.29-0.0751-0.9766-0.0296-0.9907.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/4862/weights.29-0.0751-0.9766-0.0296-0.9907.hdf5 -------------------------------------------------------------------------------- /4862/weights.44-0.0684-0.9784-0.0296-0.9913.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/4862/weights.44-0.0684-0.9784-0.0296-0.9913.hdf5 -------------------------------------------------------------------------------- /4862/weights.49-0.0690-0.9783-0.0279-0.9918.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/4862/weights.49-0.0690-0.9783-0.0279-0.9918.hdf5 -------------------------------------------------------------------------------- /5238/weights.08-0.0909-0.9720-0.0372-0.9895.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/5238/weights.08-0.0909-0.9720-0.0372-0.9895.hdf5 -------------------------------------------------------------------------------- /5238/weights.21-0.0718-0.9776-0.0362-0.9903.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/5238/weights.21-0.0718-0.9776-0.0362-0.9903.hdf5 -------------------------------------------------------------------------------- /5238/weights.24-0.0732-0.9769-0.0324-0.9912.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/5238/weights.24-0.0732-0.9769-0.0324-0.9912.hdf5 -------------------------------------------------------------------------------- /5238/weights.29-0.0679-0.9786-0.0301-0.9913.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/5238/weights.29-0.0679-0.9786-0.0301-0.9913.hdf5 -------------------------------------------------------------------------------- /6742/weights.06-0.0320-0.9898-0.0425-0.9885.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/6742/weights.06-0.0320-0.9898-0.0425-0.9885.hdf5 -------------------------------------------------------------------------------- /6742/weights.15-0.0161-0.9949-0.0436-0.9897.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/6742/weights.15-0.0161-0.9949-0.0436-0.9897.hdf5 -------------------------------------------------------------------------------- /6742/weights.27-0.0098-0.9966-0.0435-0.9903.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruslangrimov/mnist-minimal-model/HEAD/6742/weights.27-0.0098-0.9966-0.0435-0.9903.hdf5 -------------------------------------------------------------------------------- /4142/readme.txt: -------------------------------------------------------------------------------- 1 | Reduced number of weights from 4862 to 4142 by replacing the third 3x3 convotion with two convolutions with 3x1 and 1x3 kernels. The accuracy is the same as in model with 4862 weights. 2 | -------------------------------------------------------------------------------- /keras_cnn_test.py: -------------------------------------------------------------------------------- 1 | import os 2 | # os.environ['KERAS_BACKEND'] = 'theano' 3 | 4 | from keras.datasets import mnist 5 | from keras.models import Model, Sequential 6 | from keras.layers import Input, Conv2D, MaxPooling2D, Dense, Dropout 7 | from keras.layers import Activation, Flatten, GlobalMaxPooling2D 8 | from keras.layers import GlobalAveragePooling2D, ActivityRegularization 9 | from keras.layers import BatchNormalization 10 | from keras.utils import np_utils 11 | from keras import backend as K 12 | from keras.regularizers import l2 13 | from keras.utils import to_categorical 14 | from keras.callbacks import Callback, EarlyStopping, ModelCheckpoint 15 | 16 | img_rows, img_cols = 28, 28 17 | num_classes = 10 18 | # the data, shuffled and split between train and test sets 19 | (x_train, y_train), (x_test, y_test) = mnist.load_data() 20 | 21 | if K.image_data_format() == 'channels_first': 22 | x_train = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols) 23 | x_test = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols) 24 | input_shape = (1, img_rows, img_cols) 25 | else: 26 | x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1) 27 | x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1) 28 | input_shape = (img_rows, img_cols, 1) 29 | 30 | x_train = x_train.astype('float32') 31 | x_test = x_test.astype('float32') 32 | x_train /= 255 33 | x_test /= 255 34 | print('x_train shape:', x_train.shape) 35 | print(x_train.shape[0], 'train samples') 36 | print(x_test.shape[0], 'test samples') 37 | 38 | # convert class vectors to binary class matrices 39 | y_train = to_categorical(y_train, num_classes) 40 | y_test = to_categorical(y_test, num_classes) 41 | 42 | model = Sequential() 43 | 44 | # ... paste text from model.txt here 45 | 46 | score = model.evaluate(x_test, y_test, verbose=1) 47 | print('\n') 48 | print('Test score: ', score[0]) 49 | print('Test accuracy: ', score[1]) 50 | -------------------------------------------------------------------------------- /6742/model.txt: -------------------------------------------------------------------------------- 1 | model.add(Conv2D(8, (3, 3), input_shape=input_shape, activation='relu')) 2 | model.add(MaxPooling2D((2, 2))) 3 | model.add(BatchNormalization()) 4 | model.add(Conv2D(16, (3, 3), activation='relu')) 5 | model.add(MaxPooling2D((2, 2))) 6 | model.add(BatchNormalization()) 7 | model.add(Conv2D(16, (3, 3), activation='relu')) 8 | model.add(BatchNormalization()) 9 | model.add(Conv2D(4, (1, 1), activation='relu')) 10 | model.add(BatchNormalization()) 11 | model.add(Flatten()) 12 | model.add(Dense(64, activation='relu')) 13 | model.add(Dense(10, activation='softmax')) 14 | 15 | batch_size = 32 16 | num_epochs = 30 17 | 18 | model.summary() 19 | 20 | model.compile(loss='categorical_crossentropy', 21 | optimizer='adam', 22 | metrics=['accuracy']) 23 | 24 | 25 | model.fit(x_train, y_train, 26 | batch_size=batch_size, epochs=num_epochs, 27 | verbose=1, validation_split=0.1, 28 | callbacks=[ModelCheckpoint('mini/weights.{epoch:02d}-{loss:.4f}-{acc:.4f}-{val_loss:.4f}-{val_acc:.4f}.hdf5', 29 | monitor='loss,acc,val_loss,val_acc')]) 30 | 31 | Layer (type) Output Shape Param # 32 | ================================================================= 33 | conv2d_20 (Conv2D) (None, 26, 26, 8) 80 34 | _________________________________________________________________ 35 | max_pooling2d_11 (MaxPooling (None, 13, 13, 8) 0 36 | _________________________________________________________________ 37 | batch_normalization_20 (Batc (None, 13, 13, 8) 32 38 | _________________________________________________________________ 39 | conv2d_21 (Conv2D) (None, 11, 11, 16) 1168 40 | _________________________________________________________________ 41 | max_pooling2d_12 (MaxPooling (None, 5, 5, 16) 0 42 | _________________________________________________________________ 43 | batch_normalization_21 (Batc (None, 5, 5, 16) 64 44 | _________________________________________________________________ 45 | conv2d_22 (Conv2D) (None, 3, 3, 16) 2320 46 | _________________________________________________________________ 47 | batch_normalization_22 (Batc (None, 3, 3, 16) 64 48 | _________________________________________________________________ 49 | conv2d_23 (Conv2D) (None, 3, 3, 4) 68 50 | _________________________________________________________________ 51 | batch_normalization_23 (Batc (None, 3, 3, 4) 16 52 | _________________________________________________________________ 53 | flatten_6 (Flatten) (None, 36) 0 54 | _________________________________________________________________ 55 | dense_11 (Dense) (None, 64) 2368 56 | _________________________________________________________________ 57 | dense_12 (Dense) (None, 10) 650 58 | ================================================================= 59 | Total params: 6,830 60 | Trainable params: 6,742 61 | Non-trainable params: 88 62 | -------------------------------------------------------------------------------- /5238/model.txt: -------------------------------------------------------------------------------- 1 | model.add(Conv2D(8, (3, 3), input_shape=input_shape, activation='relu')) 2 | model.add(MaxPooling2D((2, 2))) 3 | model.add(BatchNormalization()) 4 | model.add(Dropout(rate=0.1)) 5 | model.add(Conv2D(16, (3, 3), activation='relu')) 6 | model.add(MaxPooling2D((2, 2))) 7 | model.add(BatchNormalization()) 8 | model.add(Dropout(rate=0.1)) 9 | model.add(Conv2D(16, (3, 3), activation='relu')) 10 | model.add(BatchNormalization()) 11 | model.add(Dropout(rate=0.1)) 12 | model.add(Conv2D(4, (1, 1), activation='relu')) 13 | model.add(BatchNormalization()) 14 | model.add(Dropout(rate=0.1)) 15 | model.add(Flatten()) 16 | model.add(Dense(32, activation='relu')) 17 | model.add(Dropout(rate=0.1)) 18 | model.add(Dense(10, activation='softmax')) 19 | 20 | batch_size = 32 21 | num_epochs = 30 22 | 23 | model.summary() 24 | 25 | model.compile(loss='categorical_crossentropy', 26 | optimizer='adam', 27 | metrics=['accuracy']) 28 | 29 | 30 | model.fit(x_train, y_train, 31 | batch_size=batch_size, epochs=num_epochs, 32 | verbose=1, validation_split=0.1, 33 | callbacks=[ModelCheckpoint('mini/weights.{epoch:02d}-{loss:.4f}-{acc:.4f}-{val_loss:.4f}-{val_acc:.4f}.hdf5', 34 | monitor='loss,acc,val_loss,val_acc')]) 35 | 36 | 37 | _________________________________________________________________ 38 | Layer (type) Output Shape Param # 39 | ================================================================= 40 | conv2d_60 (Conv2D) (None, 26, 26, 8) 80 41 | _________________________________________________________________ 42 | max_pooling2d_31 (MaxPooling (None, 13, 13, 8) 0 43 | _________________________________________________________________ 44 | batch_normalization_60 (Batc (None, 13, 13, 8) 32 45 | _________________________________________________________________ 46 | dropout_6 (Dropout) (None, 13, 13, 8) 0 47 | _________________________________________________________________ 48 | conv2d_61 (Conv2D) (None, 11, 11, 16) 1168 49 | _________________________________________________________________ 50 | max_pooling2d_32 (MaxPooling (None, 5, 5, 16) 0 51 | _________________________________________________________________ 52 | batch_normalization_61 (Batc (None, 5, 5, 16) 64 53 | _________________________________________________________________ 54 | dropout_7 (Dropout) (None, 5, 5, 16) 0 55 | _________________________________________________________________ 56 | conv2d_62 (Conv2D) (None, 3, 3, 16) 2320 57 | _________________________________________________________________ 58 | batch_normalization_62 (Batc (None, 3, 3, 16) 64 59 | _________________________________________________________________ 60 | dropout_8 (Dropout) (None, 3, 3, 16) 0 61 | _________________________________________________________________ 62 | conv2d_63 (Conv2D) (None, 3, 3, 4) 68 63 | _________________________________________________________________ 64 | batch_normalization_63 (Batc (None, 3, 3, 4) 16 65 | _________________________________________________________________ 66 | dropout_9 (Dropout) (None, 3, 3, 4) 0 67 | _________________________________________________________________ 68 | flatten_16 (Flatten) (None, 36) 0 69 | _________________________________________________________________ 70 | dense_32 (Dense) (None, 32) 1184 71 | _________________________________________________________________ 72 | dropout_10 (Dropout) (None, 32) 0 73 | _________________________________________________________________ 74 | dense_33 (Dense) (None, 10) 330 75 | ================================================================= 76 | Total params: 5,326 77 | Trainable params: 5,238 78 | Non-trainable params: 88 79 | 80 | Test score: 0.0286468310945 81 | Test accuracy: 0.9907 82 | -------------------------------------------------------------------------------- /4862/model.txt: -------------------------------------------------------------------------------- 1 | model.add(Conv2D(8, (3, 3), input_shape=input_shape, activation='relu')) 2 | model.add(MaxPooling2D((2, 2))) 3 | model.add(BatchNormalization()) 4 | model.add(Dropout(rate=0.1)) 5 | model.add(Conv2D(16, (3, 3), activation='relu')) 6 | model.add(MaxPooling2D((2, 2))) 7 | model.add(BatchNormalization()) 8 | model.add(Dropout(rate=0.1)) 9 | model.add(Conv2D(16, (3, 3), activation='relu')) 10 | model.add(BatchNormalization()) 11 | model.add(Dropout(rate=0.1)) 12 | model.add(Conv2D(4, (1, 1), activation='relu')) 13 | model.add(BatchNormalization()) 14 | model.add(Dropout(rate=0.1)) 15 | model.add(Flatten()) 16 | #model.add(Dense(32, activation='relu')) 17 | #model.add(Dropout(rate=0.1)) 18 | model.add(Dense(24, activation='relu')) 19 | model.add(Dropout(rate=0.1)) 20 | model.add(Dense(10, activation='softmax')) 21 | 22 | batch_size = 32 23 | num_epochs = 50 24 | 25 | model.summary() 26 | 27 | model.compile(loss='categorical_crossentropy', 28 | optimizer='adam', 29 | metrics=['accuracy']) 30 | 31 | 32 | model.fit(x_train, y_train, 33 | batch_size=batch_size, epochs=num_epochs, 34 | verbose=1, validation_split=0.1, 35 | callbacks=[ModelCheckpoint('mini/weights.{epoch:02d}-{loss:.4f}-{acc:.4f}-{val_loss:.4f}-{val_acc:.4f}.hdf5', 36 | monitor='loss,acc,val_loss,val_acc')]) 37 | 38 | 39 | 40 | _________________________________________________________________ 41 | Layer (type) Output Shape Param # 42 | ================================================================= 43 | conv2d_80 (Conv2D) (None, 26, 26, 8) 80 44 | _________________________________________________________________ 45 | max_pooling2d_41 (MaxPooling (None, 13, 13, 8) 0 46 | _________________________________________________________________ 47 | batch_normalization_80 (Batc (None, 13, 13, 8) 32 48 | _________________________________________________________________ 49 | dropout_33 (Dropout) (None, 13, 13, 8) 0 50 | _________________________________________________________________ 51 | conv2d_81 (Conv2D) (None, 11, 11, 16) 1168 52 | _________________________________________________________________ 53 | max_pooling2d_42 (MaxPooling (None, 5, 5, 16) 0 54 | _________________________________________________________________ 55 | batch_normalization_81 (Batc (None, 5, 5, 16) 64 56 | _________________________________________________________________ 57 | dropout_34 (Dropout) (None, 5, 5, 16) 0 58 | _________________________________________________________________ 59 | conv2d_82 (Conv2D) (None, 3, 3, 16) 2320 60 | _________________________________________________________________ 61 | batch_normalization_82 (Batc (None, 3, 3, 16) 64 62 | _________________________________________________________________ 63 | dropout_35 (Dropout) (None, 3, 3, 16) 0 64 | _________________________________________________________________ 65 | conv2d_83 (Conv2D) (None, 3, 3, 4) 68 66 | _________________________________________________________________ 67 | batch_normalization_83 (Batc (None, 3, 3, 4) 16 68 | _________________________________________________________________ 69 | dropout_36 (Dropout) (None, 3, 3, 4) 0 70 | _________________________________________________________________ 71 | flatten_21 (Flatten) (None, 36) 0 72 | _________________________________________________________________ 73 | dense_44 (Dense) (None, 24) 888 74 | _________________________________________________________________ 75 | dropout_37 (Dropout) (None, 24) 0 76 | _________________________________________________________________ 77 | dense_45 (Dense) (None, 10) 250 78 | ================================================================= 79 | Total params: 4,950 80 | Trainable params: 4,862 81 | Non-trainable params: 88 82 | -------------------------------------------------------------------------------- /4142/model.txt: -------------------------------------------------------------------------------- 1 | model.add(Conv2D(8, (3, 3), input_shape=input_shape, activation='relu')) 2 | model.add(MaxPooling2D((2, 2))) 3 | model.add(BatchNormalization()) 4 | model.add(Dropout(rate=0.1)) 5 | model.add(Conv2D(16, (3, 3), activation='relu')) 6 | model.add(MaxPooling2D((2, 2))) 7 | model.add(BatchNormalization()) 8 | model.add(Dropout(rate=0.1)) 9 | model.add(Conv2D(16, (3, 1), activation='relu')) 10 | model.add(BatchNormalization()) 11 | model.add(Conv2D(16, (1, 3), activation='relu')) 12 | model.add(BatchNormalization()) 13 | model.add(Dropout(rate=0.1)) 14 | model.add(Conv2D(4, (1, 1), activation='relu')) 15 | model.add(BatchNormalization()) 16 | model.add(Dropout(rate=0.1)) 17 | model.add(Flatten()) 18 | model.add(Dense(24, activation='relu')) 19 | model.add(Dropout(rate=0.1)) 20 | model.add(Dense(10, activation='softmax')) 21 | 22 | batch_size = 32 23 | num_epochs = 50 24 | 25 | model.summary() 26 | 27 | model.compile(loss='categorical_crossentropy', 28 | optimizer='adam', 29 | metrics=['accuracy']) 30 | 31 | 32 | model.fit(x_train, y_train, 33 | batch_size=batch_size, epochs=num_epochs, 34 | verbose=1, validation_split=0.1, 35 | callbacks=[ModelCheckpoint('mini/weights.{epoch:02d}-{loss:.4f}-{acc:.4f}-{val_loss:.4f}-{val_acc:.4f}.hdf5', 36 | monitor='loss,acc,val_loss,val_acc')]) 37 | 38 | 39 | Layer (type) Output Shape Param # 40 | ================================================================= 41 | conv2d_116 (Conv2D) (None, 26, 26, 8) 80 42 | _________________________________________________________________ 43 | max_pooling2d_57 (MaxPooling (None, 13, 13, 8) 0 44 | _________________________________________________________________ 45 | batch_normalization_116 (Bat (None, 13, 13, 8) 32 46 | _________________________________________________________________ 47 | dropout_73 (Dropout) (None, 13, 13, 8) 0 48 | _________________________________________________________________ 49 | conv2d_117 (Conv2D) (None, 11, 11, 16) 1168 50 | _________________________________________________________________ 51 | max_pooling2d_58 (MaxPooling (None, 5, 5, 16) 0 52 | _________________________________________________________________ 53 | batch_normalization_117 (Bat (None, 5, 5, 16) 64 54 | _________________________________________________________________ 55 | dropout_74 (Dropout) (None, 5, 5, 16) 0 56 | _________________________________________________________________ 57 | conv2d_118 (Conv2D) (None, 3, 5, 16) 784 58 | _________________________________________________________________ 59 | batch_normalization_118 (Bat (None, 3, 5, 16) 64 60 | _________________________________________________________________ 61 | conv2d_119 (Conv2D) (None, 3, 3, 16) 784 62 | _________________________________________________________________ 63 | batch_normalization_119 (Bat (None, 3, 3, 16) 64 64 | _________________________________________________________________ 65 | dropout_75 (Dropout) (None, 3, 3, 16) 0 66 | _________________________________________________________________ 67 | conv2d_120 (Conv2D) (None, 3, 3, 4) 68 68 | _________________________________________________________________ 69 | batch_normalization_120 (Bat (None, 3, 3, 4) 16 70 | _________________________________________________________________ 71 | dropout_76 (Dropout) (None, 3, 3, 4) 0 72 | _________________________________________________________________ 73 | flatten_29 (Flatten) (None, 36) 0 74 | _________________________________________________________________ 75 | dense_60 (Dense) (None, 24) 888 76 | _________________________________________________________________ 77 | dropout_77 (Dropout) (None, 24) 0 78 | _________________________________________________________________ 79 | dense_61 (Dense) (None, 10) 250 80 | ================================================================= 81 | Total params: 4,262 82 | Trainable params: 4,142 83 | Non-trainable params: 120 84 | 85 | 86 | Test score: 0.0264436473006 87 | Test accuracy: 0.9917 88 | -------------------------------------------------------------------------------- /3462/model.txt: -------------------------------------------------------------------------------- 1 | model.add(Conv2D(8, (3, 3), input_shape=input_shape, activation='relu')) 2 | model.add(MaxPooling2D((2, 2))) 3 | model.add(BatchNormalization()) 4 | model.add(Dropout(rate=0.1)) 5 | model.add(Conv2D(12, (3, 3), activation='relu')) 6 | model.add(MaxPooling2D((2, 2))) 7 | model.add(BatchNormalization()) 8 | model.add(Dropout(rate=0.1)) 9 | model.add(Conv2D(16, (3, 1), activation='relu')) 10 | model.add(BatchNormalization()) 11 | model.add(Conv2D(16, (1, 3), activation='relu')) 12 | model.add(BatchNormalization()) 13 | model.add(Dropout(rate=0.1)) 14 | model.add(Conv2D(4, (1, 1), activation='relu')) 15 | model.add(BatchNormalization()) 16 | model.add(Dropout(rate=0.1)) 17 | model.add(Flatten()) 18 | model.add(Dense(20, activation='relu')) 19 | model.add(Dropout(rate=0.1)) 20 | model.add(Dense(10, activation='softmax')) 21 | 22 | batch_size = 32 23 | num_epochs = 75 24 | 25 | model.summary() 26 | 27 | model.compile(loss='categorical_crossentropy', 28 | optimizer='adam', 29 | metrics=['accuracy']) 30 | 31 | 32 | model.fit(x_train, y_train, 33 | batch_size=batch_size, epochs=num_epochs, 34 | verbose=1, validation_split=0.1, 35 | callbacks=[ModelCheckpoint('mini/weights.{epoch:02d}-{loss:.4f}-{acc:.4f}-{val_loss:.4f}-{val_acc:.4f}.hdf5', 36 | monitor='loss,acc,val_loss,val_acc')]) 37 | 38 | 39 | 40 | Layer (type) Output Shape Param # 41 | ================================================================= 42 | conv2d_298 (Conv2D) (None, 26, 26, 8) 80 43 | _________________________________________________________________ 44 | max_pooling2d_129 (MaxPoolin (None, 13, 13, 8) 0 45 | _________________________________________________________________ 46 | batch_normalization_296 (Bat (None, 13, 13, 8) 32 47 | _________________________________________________________________ 48 | dropout_253 (Dropout) (None, 13, 13, 8) 0 49 | _________________________________________________________________ 50 | conv2d_299 (Conv2D) (None, 11, 11, 12) 876 51 | _________________________________________________________________ 52 | max_pooling2d_130 (MaxPoolin (None, 5, 5, 12) 0 53 | _________________________________________________________________ 54 | batch_normalization_297 (Bat (None, 5, 5, 12) 48 55 | _________________________________________________________________ 56 | dropout_254 (Dropout) (None, 5, 5, 12) 0 57 | _________________________________________________________________ 58 | conv2d_300 (Conv2D) (None, 3, 5, 16) 592 59 | _________________________________________________________________ 60 | batch_normalization_298 (Bat (None, 3, 5, 16) 64 61 | _________________________________________________________________ 62 | conv2d_301 (Conv2D) (None, 3, 3, 16) 784 63 | _________________________________________________________________ 64 | batch_normalization_299 (Bat (None, 3, 3, 16) 64 65 | _________________________________________________________________ 66 | dropout_255 (Dropout) (None, 3, 3, 16) 0 67 | _________________________________________________________________ 68 | conv2d_302 (Conv2D) (None, 3, 3, 4) 68 69 | _________________________________________________________________ 70 | batch_normalization_300 (Bat (None, 3, 3, 4) 16 71 | _________________________________________________________________ 72 | dropout_256 (Dropout) (None, 3, 3, 4) 0 73 | _________________________________________________________________ 74 | flatten_65 (Flatten) (None, 36) 0 75 | _________________________________________________________________ 76 | dense_132 (Dense) (None, 20) 740 77 | _________________________________________________________________ 78 | dropout_257 (Dropout) (None, 20) 0 79 | _________________________________________________________________ 80 | dense_133 (Dense) (None, 10) 210 81 | ================================================================= 82 | Total params: 3,574 83 | Trainable params: 3,462 84 | Non-trainable params: 112 85 | 86 | 87 | Test score: 0.0307938806147 88 | Test accuracy: 0.9906 89 | -------------------------------------------------------------------------------- /3650/model.txt: -------------------------------------------------------------------------------- 1 | model.add(Conv2D(8, (3, 3), input_shape=input_shape, activation='relu')) 2 | model.add(MaxPooling2D((2, 2))) 3 | model.add(BatchNormalization()) 4 | model.add(Dropout(rate=0.1)) 5 | model.add(Conv2D(12, (3, 3), activation='relu')) 6 | model.add(MaxPooling2D((2, 2))) 7 | model.add(BatchNormalization()) 8 | model.add(Dropout(rate=0.1)) 9 | model.add(Conv2D(16, (3, 1), activation='relu')) 10 | model.add(BatchNormalization()) 11 | model.add(Conv2D(16, (1, 3), activation='relu')) 12 | model.add(BatchNormalization()) 13 | model.add(Dropout(rate=0.1)) 14 | model.add(Conv2D(4, (1, 1), activation='relu')) 15 | model.add(BatchNormalization()) 16 | model.add(Dropout(rate=0.1)) 17 | model.add(Flatten()) 18 | model.add(Dense(24, activation='relu')) 19 | model.add(Dropout(rate=0.1)) 20 | model.add(Dense(10, activation='softmax')) 21 | 22 | batch_size = 32 23 | num_epochs = 50 24 | 25 | model.summary() 26 | 27 | model.compile(loss='categorical_crossentropy', 28 | optimizer='adam', 29 | metrics=['accuracy']) 30 | 31 | 32 | model.fit(x_train, y_train, 33 | batch_size=batch_size, epochs=num_epochs, 34 | verbose=1, validation_split=0.1, 35 | callbacks=[ModelCheckpoint('mini/weights.{epoch:02d}-{loss:.4f}-{acc:.4f}-{val_loss:.4f}-{val_acc:.4f}.hdf5', 36 | monitor='loss,acc,val_loss,val_acc')]) 37 | 38 | 39 | 40 | Layer (type) Output Shape Param # 41 | ================================================================= 42 | conv2d_153 (Conv2D) (None, 26, 26, 8) 80 43 | _________________________________________________________________ 44 | max_pooling2d_71 (MaxPooling (None, 13, 13, 8) 0 45 | _________________________________________________________________ 46 | batch_normalization_151 (Bat (None, 13, 13, 8) 32 47 | _________________________________________________________________ 48 | dropout_108 (Dropout) (None, 13, 13, 8) 0 49 | _________________________________________________________________ 50 | conv2d_154 (Conv2D) (None, 11, 11, 12) 876 51 | _________________________________________________________________ 52 | max_pooling2d_72 (MaxPooling (None, 5, 5, 12) 0 53 | _________________________________________________________________ 54 | batch_normalization_152 (Bat (None, 5, 5, 12) 48 55 | _________________________________________________________________ 56 | dropout_109 (Dropout) (None, 5, 5, 12) 0 57 | _________________________________________________________________ 58 | conv2d_155 (Conv2D) (None, 3, 5, 16) 592 59 | _________________________________________________________________ 60 | batch_normalization_153 (Bat (None, 3, 5, 16) 64 61 | _________________________________________________________________ 62 | conv2d_156 (Conv2D) (None, 3, 3, 16) 784 63 | _________________________________________________________________ 64 | batch_normalization_154 (Bat (None, 3, 3, 16) 64 65 | _________________________________________________________________ 66 | dropout_110 (Dropout) (None, 3, 3, 16) 0 67 | _________________________________________________________________ 68 | conv2d_157 (Conv2D) (None, 3, 3, 4) 68 69 | _________________________________________________________________ 70 | batch_normalization_155 (Bat (None, 3, 3, 4) 16 71 | _________________________________________________________________ 72 | dropout_111 (Dropout) (None, 3, 3, 4) 0 73 | _________________________________________________________________ 74 | flatten_36 (Flatten) (None, 36) 0 75 | _________________________________________________________________ 76 | dense_74 (Dense) (None, 24) 888 77 | _________________________________________________________________ 78 | dropout_112 (Dropout) (None, 24) 0 79 | _________________________________________________________________ 80 | dense_75 (Dense) (None, 10) 250 81 | ================================================================= 82 | Total params: 3,762 83 | Trainable params: 3,650 84 | Non-trainable params: 112 85 | 86 | 87 | Test score: 0.0323348842089 88 | Test accuracy: 0.9905 89 | -------------------------------------------------------------------------------- /3354/model.txt: -------------------------------------------------------------------------------- 1 | model.add(Conv2D(8, (3, 3), input_shape=input_shape, activation='relu')) 2 | model.add(MaxPooling2D((2, 2))) 3 | model.add(BatchNormalization()) 4 | model.add(Dropout(rate=0.1)) 5 | model.add(Conv2D(12, (3, 1), activation='relu')) 6 | model.add(BatchNormalization()) 7 | model.add(Conv2D(12, (1, 3), activation='relu')) 8 | model.add(MaxPooling2D((2, 2))) 9 | model.add(BatchNormalization()) 10 | model.add(Dropout(rate=0.1)) 11 | model.add(Conv2D(16, (3, 1), activation='relu')) 12 | model.add(BatchNormalization()) 13 | model.add(Conv2D(16, (1, 3), activation='relu')) 14 | model.add(BatchNormalization()) 15 | model.add(Dropout(rate=0.1)) 16 | model.add(Conv2D(4, (1, 1), activation='relu')) 17 | model.add(BatchNormalization()) 18 | model.add(Dropout(rate=0.1)) 19 | model.add(Flatten()) 20 | model.add(Dense(20, activation='relu')) 21 | model.add(Dropout(rate=0.1)) 22 | model.add(Dense(10, activation='softmax')) 23 | 24 | batch_size = 32 25 | num_epochs = 75 26 | 27 | model.summary() 28 | 29 | model.compile(loss='categorical_crossentropy', 30 | optimizer='adam', 31 | metrics=['accuracy']) 32 | 33 | 34 | model.fit(x_train, y_train, 35 | batch_size=batch_size, epochs=num_epochs, 36 | verbose=1, validation_split=0.1, 37 | callbacks=[ModelCheckpoint('mini/weights.{epoch:02d}-{loss:.4f}-{acc:.4f}-{val_loss:.4f}-{val_acc:.4f}.hdf5', 38 | monitor='loss,acc,val_loss,val_acc')]) 39 | 40 | 41 | 42 | 43 | Layer (type) Output Shape Param # 44 | ================================================================= 45 | conv2d_55 (Conv2D) (None, 26, 26, 8) 80 46 | _________________________________________________________________ 47 | max_pooling2d_19 (MaxPooling (None, 13, 13, 8) 0 48 | _________________________________________________________________ 49 | batch_normalization_55 (Batc (None, 13, 13, 8) 32 50 | _________________________________________________________________ 51 | dropout_46 (Dropout) (None, 13, 13, 8) 0 52 | _________________________________________________________________ 53 | conv2d_56 (Conv2D) (None, 11, 13, 12) 300 54 | _________________________________________________________________ 55 | batch_normalization_56 (Batc (None, 11, 13, 12) 48 56 | _________________________________________________________________ 57 | conv2d_57 (Conv2D) (None, 11, 11, 12) 444 58 | _________________________________________________________________ 59 | max_pooling2d_20 (MaxPooling (None, 5, 5, 12) 0 60 | _________________________________________________________________ 61 | batch_normalization_57 (Batc (None, 5, 5, 12) 48 62 | _________________________________________________________________ 63 | dropout_47 (Dropout) (None, 5, 5, 12) 0 64 | _________________________________________________________________ 65 | conv2d_58 (Conv2D) (None, 3, 5, 16) 592 66 | _________________________________________________________________ 67 | batch_normalization_58 (Batc (None, 3, 5, 16) 64 68 | _________________________________________________________________ 69 | conv2d_59 (Conv2D) (None, 3, 3, 16) 784 70 | _________________________________________________________________ 71 | batch_normalization_59 (Batc (None, 3, 3, 16) 64 72 | _________________________________________________________________ 73 | dropout_48 (Dropout) (None, 3, 3, 16) 0 74 | _________________________________________________________________ 75 | conv2d_60 (Conv2D) (None, 3, 3, 4) 68 76 | _________________________________________________________________ 77 | batch_normalization_60 (Batc (None, 3, 3, 4) 16 78 | _________________________________________________________________ 79 | dropout_49 (Dropout) (None, 3, 3, 4) 0 80 | _________________________________________________________________ 81 | flatten_10 (Flatten) (None, 36) 0 82 | _________________________________________________________________ 83 | dense_19 (Dense) (None, 20) 740 84 | _________________________________________________________________ 85 | dropout_50 (Dropout) (None, 20) 0 86 | _________________________________________________________________ 87 | dense_20 (Dense) (None, 10) 210 88 | ================================================================= 89 | Total params: 3,490 90 | Trainable params: 3,354 91 | Non-trainable params: 136 92 | 93 | 94 | Test score: 0.0342673188853 95 | Test accuracy: 0.9907 96 | -------------------------------------------------------------------------------- /2720/model.txt: -------------------------------------------------------------------------------- 1 | model = Sequential(name='24SConv') 2 | 3 | model.add(Conv2D(8, (3, 3), input_shape=input_shape)) 4 | model.add(Activation(activation='relu')) 5 | model.add(BatchNormalization()) 6 | model.add(MaxPooling2D((2, 2))) 7 | model.add(BatchNormalization()) 8 | model.add(Dropout(rate=0.1)) 9 | 10 | model.add(SeparableConv2D(16, (3, 3), depth_multiplier=2)) 11 | model.add(Activation(activation='relu')) 12 | model.add(BatchNormalization()) 13 | model.add(MaxPooling2D((2, 2))) 14 | model.add(BatchNormalization()) 15 | model.add(Dropout(rate=0.1)) 16 | 17 | model.add(SeparableConv2D(24, (3, 3), depth_multiplier=2)) 18 | model.add(Activation(activation='relu')) 19 | model.add(BatchNormalization()) 20 | model.add(Dropout(rate=0.1)) 21 | 22 | model.add(Conv2D(4, (1, 1))) 23 | model.add(Activation(activation='relu')) 24 | model.add(BatchNormalization()) 25 | model.add(Dropout(rate=0.1)) 26 | 27 | model.add(Flatten()) 28 | 29 | model.add(Dense(18, activation='relu')) 30 | model.add(BatchNormalization()) 31 | model.add(Dropout(rate=0.1)) 32 | model.add(Dense(10, activation='softmax')) 33 | 34 | batch_size = 32 35 | num_epochs = 75 36 | 37 | model.summary() 38 | 39 | model.compile(loss='categorical_crossentropy', 40 | optimizer='adam', 41 | metrics=['accuracy']) 42 | 43 | w_path = os.path.join('mini', model.name) 44 | 45 | if not os.path.exists(w_path): 46 | os.makedirs(w_path) 47 | 48 | f_name = os.path.join(w_path, 'weights.{epoch:02d}-{loss:.4f}-{acc:.4f}-{val_loss:.4f}-{val_acc:.4f}.hdf5') 49 | 50 | model.fit(x_train, y_train, 51 | batch_size=batch_size, epochs=num_epochs, 52 | verbose=1, validation_split=0.1, 53 | callbacks=[ModelCheckpoint(f_name)]) 54 | 55 | 56 | 57 | Layer (type) Output Shape Param # 58 | ================================================================= 59 | conv2d_111 (Conv2D) (None, 26, 26, 8) 80 60 | _________________________________________________________________ 61 | activation_157 (Activation) (None, 26, 26, 8) 0 62 | _________________________________________________________________ 63 | batch_normalization_253 (Bat (None, 26, 26, 8) 32 64 | _________________________________________________________________ 65 | max_pooling2d_64 (MaxPooling (None, 13, 13, 8) 0 66 | _________________________________________________________________ 67 | batch_normalization_254 (Bat (None, 13, 13, 8) 32 68 | _________________________________________________________________ 69 | dropout_169 (Dropout) (None, 13, 13, 8) 0 70 | _________________________________________________________________ 71 | separable_conv2d_47 (Separab (None, 11, 11, 16) 416 72 | _________________________________________________________________ 73 | activation_158 (Activation) (None, 11, 11, 16) 0 74 | _________________________________________________________________ 75 | batch_normalization_255 (Bat (None, 11, 11, 16) 64 76 | _________________________________________________________________ 77 | max_pooling2d_65 (MaxPooling (None, 5, 5, 16) 0 78 | _________________________________________________________________ 79 | batch_normalization_256 (Bat (None, 5, 5, 16) 64 80 | _________________________________________________________________ 81 | dropout_170 (Dropout) (None, 5, 5, 16) 0 82 | _________________________________________________________________ 83 | separable_conv2d_48 (Separab (None, 3, 3, 24) 1080 84 | _________________________________________________________________ 85 | activation_159 (Activation) (None, 3, 3, 24) 0 86 | _________________________________________________________________ 87 | batch_normalization_257 (Bat (None, 3, 3, 24) 96 88 | _________________________________________________________________ 89 | dropout_171 (Dropout) (None, 3, 3, 24) 0 90 | _________________________________________________________________ 91 | conv2d_112 (Conv2D) (None, 3, 3, 4) 100 92 | _________________________________________________________________ 93 | activation_160 (Activation) (None, 3, 3, 4) 0 94 | _________________________________________________________________ 95 | batch_normalization_258 (Bat (None, 3, 3, 4) 16 96 | _________________________________________________________________ 97 | dropout_172 (Dropout) (None, 3, 3, 4) 0 98 | _________________________________________________________________ 99 | flatten_35 (Flatten) (None, 36) 0 100 | _________________________________________________________________ 101 | dense_69 (Dense) (None, 18) 666 102 | _________________________________________________________________ 103 | batch_normalization_259 (Bat (None, 18) 72 104 | _________________________________________________________________ 105 | dropout_173 (Dropout) (None, 18) 0 106 | _________________________________________________________________ 107 | dense_70 (Dense) (None, 10) 190 108 | ================================================================= 109 | Total params: 2,908 110 | Trainable params: 2,720 111 | Non-trainable params: 188 112 | 113 | 114 | Test score: 0.0319789474596 115 | Test accuracy: 0.9905 116 | -------------------------------------------------------------------------------- /3089/model.txt: -------------------------------------------------------------------------------- 1 | model.add(Conv2D(8, (3, 3), input_shape=input_shape)) 2 | model.add(Activation(activation='relu')) 3 | model.add(BatchNormalization()) 4 | model.add(MaxPooling2D((2, 2))) 5 | model.add(BatchNormalization()) 6 | model.add(Dropout(rate=0.1)) 7 | 8 | model.add(Conv2D(12, (3, 1))) 9 | model.add(Activation(activation='relu')) 10 | model.add(BatchNormalization()) 11 | model.add(Conv2D(12, (1, 3))) 12 | model.add(Activation(activation='relu')) 13 | model.add(BatchNormalization()) 14 | model.add(MaxPooling2D((2, 2))) 15 | model.add(BatchNormalization()) 16 | model.add(Dropout(rate=0.1)) 17 | 18 | model.add(SeparableConv2D(15, (3, 3), depth_multiplier=4)) 19 | model.add(Activation(activation='relu')) 20 | model.add(BatchNormalization()) 21 | model.add(Dropout(rate=0.1)) 22 | 23 | model.add(Conv2D(4, (1, 1))) 24 | model.add(Activation(activation='relu')) 25 | model.add(BatchNormalization()) 26 | model.add(Dropout(rate=0.1)) 27 | 28 | model.add(Flatten()) 29 | 30 | model.add(Dense(18, activation='relu')) 31 | model.add(BatchNormalization()) 32 | model.add(Dropout(rate=0.1)) 33 | model.add(Dense(10, activation='softmax')) 34 | 35 | batch_size = 32 36 | num_epochs = 75 37 | 38 | model.summary() 39 | 40 | model.compile(loss='categorical_crossentropy', 41 | optimizer='adam', 42 | metrics=['accuracy']) 43 | 44 | 45 | model.fit(x_train, y_train, 46 | batch_size=batch_size, epochs=num_epochs, 47 | verbose=1, validation_split=0.1, 48 | callbacks=[ModelCheckpoint('mini/weights.{epoch:02d}-{loss:.4f}-{acc:.4f}-{val_loss:.4f}-{val_acc:.4f}.hdf5', 49 | monitor='loss,acc,val_loss,val_acc')]) 50 | 51 | 52 | 53 | Layer (type) Output Shape Param # 54 | ================================================================= 55 | conv2d_29 (Conv2D) (None, 26, 26, 8) 80 56 | _________________________________________________________________ 57 | activation_36 (Activation) (None, 26, 26, 8) 0 58 | _________________________________________________________________ 59 | batch_normalization_57 (Batc (None, 26, 26, 8) 32 60 | _________________________________________________________________ 61 | max_pooling2d_15 (MaxPooling (None, 13, 13, 8) 0 62 | _________________________________________________________________ 63 | batch_normalization_58 (Batc (None, 13, 13, 8) 32 64 | _________________________________________________________________ 65 | dropout_36 (Dropout) (None, 13, 13, 8) 0 66 | _________________________________________________________________ 67 | conv2d_30 (Conv2D) (None, 11, 13, 12) 300 68 | _________________________________________________________________ 69 | activation_37 (Activation) (None, 11, 13, 12) 0 70 | _________________________________________________________________ 71 | batch_normalization_59 (Batc (None, 11, 13, 12) 48 72 | _________________________________________________________________ 73 | conv2d_31 (Conv2D) (None, 11, 11, 12) 444 74 | _________________________________________________________________ 75 | activation_38 (Activation) (None, 11, 11, 12) 0 76 | _________________________________________________________________ 77 | batch_normalization_60 (Batc (None, 11, 11, 12) 48 78 | _________________________________________________________________ 79 | max_pooling2d_16 (MaxPooling (None, 5, 5, 12) 0 80 | _________________________________________________________________ 81 | batch_normalization_61 (Batc (None, 5, 5, 12) 48 82 | _________________________________________________________________ 83 | dropout_37 (Dropout) (None, 5, 5, 12) 0 84 | _________________________________________________________________ 85 | separable_conv2d_8 (Separabl (None, 3, 3, 15) 1167 86 | _________________________________________________________________ 87 | activation_39 (Activation) (None, 3, 3, 15) 0 88 | _________________________________________________________________ 89 | batch_normalization_62 (Batc (None, 3, 3, 15) 60 90 | _________________________________________________________________ 91 | dropout_38 (Dropout) (None, 3, 3, 15) 0 92 | _________________________________________________________________ 93 | conv2d_32 (Conv2D) (None, 3, 3, 4) 64 94 | _________________________________________________________________ 95 | activation_40 (Activation) (None, 3, 3, 4) 0 96 | _________________________________________________________________ 97 | batch_normalization_63 (Batc (None, 3, 3, 4) 16 98 | _________________________________________________________________ 99 | dropout_39 (Dropout) (None, 3, 3, 4) 0 100 | _________________________________________________________________ 101 | flatten_8 (Flatten) (None, 36) 0 102 | _________________________________________________________________ 103 | dense_15 (Dense) (None, 18) 666 104 | _________________________________________________________________ 105 | batch_normalization_64 (Batc (None, 18) 72 106 | _________________________________________________________________ 107 | dropout_40 (Dropout) (None, 18) 0 108 | _________________________________________________________________ 109 | dense_16 (Dense) (None, 10) 190 110 | ================================================================= 111 | Total params: 3,267 112 | Trainable params: 3,089 113 | Non-trainable params: 178 114 | 115 | 116 | Test score: 0.0299518044221 117 | Test accuracy: 0.9904 118 | -------------------------------------------------------------------------------- /3197/model.txt: -------------------------------------------------------------------------------- 1 | model.add(Conv2D(8, (3, 3), input_shape=input_shape)) 2 | model.add(Activation(activation='relu')) 3 | model.add(BatchNormalization()) 4 | model.add(MaxPooling2D((2, 2))) 5 | model.add(BatchNormalization()) 6 | model.add(Dropout(rate=0.1)) 7 | 8 | model.add(Conv2D(12, (3, 1))) 9 | model.add(Activation(activation='relu')) 10 | model.add(BatchNormalization()) 11 | model.add(Conv2D(12, (1, 3))) 12 | model.add(Activation(activation='relu')) 13 | model.add(BatchNormalization()) 14 | model.add(MaxPooling2D((2, 2))) 15 | model.add(BatchNormalization()) 16 | model.add(Dropout(rate=0.1)) 17 | 18 | model.add(Conv2D(15, (3, 1))) 19 | model.add(Activation(activation='relu')) 20 | model.add(BatchNormalization()) 21 | model.add(Conv2D(15, (1, 3))) 22 | model.add(Activation(activation='relu')) 23 | model.add(BatchNormalization()) 24 | model.add(Dropout(rate=0.1)) 25 | 26 | model.add(Conv2D(4, (1, 1))) 27 | model.add(Activation(activation='relu')) 28 | model.add(BatchNormalization()) 29 | model.add(Dropout(rate=0.1)) 30 | 31 | model.add(Flatten()) 32 | 33 | model.add(Dense(18, activation='relu')) 34 | model.add(BatchNormalization()) 35 | model.add(Dropout(rate=0.1)) 36 | model.add(Dense(10, activation='softmax')) 37 | 38 | batch_size = 32 39 | num_epochs = 75 40 | 41 | model.summary() 42 | 43 | model.compile(loss='categorical_crossentropy', 44 | optimizer='adam', 45 | metrics=['accuracy']) 46 | 47 | 48 | model.fit(x_train, y_train, 49 | batch_size=batch_size, epochs=num_epochs, 50 | verbose=1, validation_split=0.1, 51 | callbacks=[ModelCheckpoint('mini/weights.{epoch:02d}-{loss:.4f}-{acc:.4f}-{val_loss:.4f}-{val_acc:.4f}.hdf5', 52 | monitor='loss,acc,val_loss,val_acc')]) 53 | 54 | 55 | 56 | Layer (type) Output Shape Param # 57 | ================================================================= 58 | conv2d_217 (Conv2D) (None, 26, 26, 8) 80 59 | _________________________________________________________________ 60 | activation_217 (Activation) (None, 26, 26, 8) 0 61 | _________________________________________________________________ 62 | batch_normalization_324 (Bat (None, 26, 26, 8) 32 63 | _________________________________________________________________ 64 | max_pooling2d_73 (MaxPooling (None, 13, 13, 8) 0 65 | _________________________________________________________________ 66 | batch_normalization_325 (Bat (None, 13, 13, 8) 32 67 | _________________________________________________________________ 68 | dropout_181 (Dropout) (None, 13, 13, 8) 0 69 | _________________________________________________________________ 70 | conv2d_218 (Conv2D) (None, 11, 13, 12) 300 71 | _________________________________________________________________ 72 | activation_218 (Activation) (None, 11, 13, 12) 0 73 | _________________________________________________________________ 74 | batch_normalization_326 (Bat (None, 11, 13, 12) 48 75 | _________________________________________________________________ 76 | conv2d_219 (Conv2D) (None, 11, 11, 12) 444 77 | _________________________________________________________________ 78 | activation_219 (Activation) (None, 11, 11, 12) 0 79 | _________________________________________________________________ 80 | batch_normalization_327 (Bat (None, 11, 11, 12) 48 81 | _________________________________________________________________ 82 | max_pooling2d_74 (MaxPooling (None, 5, 5, 12) 0 83 | _________________________________________________________________ 84 | batch_normalization_328 (Bat (None, 5, 5, 12) 48 85 | _________________________________________________________________ 86 | dropout_182 (Dropout) (None, 5, 5, 12) 0 87 | _________________________________________________________________ 88 | conv2d_220 (Conv2D) (None, 3, 5, 15) 555 89 | _________________________________________________________________ 90 | activation_220 (Activation) (None, 3, 5, 15) 0 91 | _________________________________________________________________ 92 | batch_normalization_329 (Bat (None, 3, 5, 15) 60 93 | _________________________________________________________________ 94 | conv2d_221 (Conv2D) (None, 3, 3, 15) 690 95 | _________________________________________________________________ 96 | activation_221 (Activation) (None, 3, 3, 15) 0 97 | _________________________________________________________________ 98 | batch_normalization_330 (Bat (None, 3, 3, 15) 60 99 | _________________________________________________________________ 100 | dropout_183 (Dropout) (None, 3, 3, 15) 0 101 | _________________________________________________________________ 102 | conv2d_222 (Conv2D) (None, 3, 3, 4) 64 103 | _________________________________________________________________ 104 | activation_222 (Activation) (None, 3, 3, 4) 0 105 | _________________________________________________________________ 106 | batch_normalization_331 (Bat (None, 3, 3, 4) 16 107 | _________________________________________________________________ 108 | dropout_184 (Dropout) (None, 3, 3, 4) 0 109 | _________________________________________________________________ 110 | flatten_37 (Flatten) (None, 36) 0 111 | _________________________________________________________________ 112 | dense_73 (Dense) (None, 18) 666 113 | _________________________________________________________________ 114 | batch_normalization_332 (Bat (None, 18) 72 115 | _________________________________________________________________ 116 | dropout_185 (Dropout) (None, 18) 0 117 | _________________________________________________________________ 118 | dense_74 (Dense) (None, 10) 190 119 | ================================================================= 120 | Total params: 3,405 121 | Trainable params: 3,197 122 | Non-trainable params: 208 123 | 124 | 125 | Test score: 0.0289398800313 126 | Test accuracy: 0.9909 127 | -------------------------------------------------------------------------------- /3295/model.txt: -------------------------------------------------------------------------------- 1 | model.add(Conv2D(8, (3, 3), input_shape=input_shape)) 2 | model.add(Activation(activation='relu')) 3 | model.add(BatchNormalization()) 4 | model.add(MaxPooling2D((2, 2))) 5 | model.add(BatchNormalization()) 6 | model.add(Dropout(rate=0.1)) 7 | 8 | model.add(Conv2D(12, (3, 1))) 9 | model.add(Activation(activation='relu')) 10 | model.add(BatchNormalization()) 11 | model.add(Conv2D(12, (1, 3))) 12 | model.add(Activation(activation='relu')) 13 | model.add(BatchNormalization()) 14 | model.add(MaxPooling2D((2, 2))) 15 | model.add(BatchNormalization()) 16 | model.add(Dropout(rate=0.1)) 17 | 18 | model.add(Conv2D(15, (3, 1))) 19 | model.add(Activation(activation='relu')) 20 | model.add(BatchNormalization()) 21 | model.add(Conv2D(15, (1, 3))) 22 | model.add(Activation(activation='relu')) 23 | model.add(BatchNormalization()) 24 | model.add(Dropout(rate=0.1)) 25 | 26 | model.add(Conv2D(4, (1, 1))) 27 | model.add(Activation(activation='relu')) 28 | model.add(BatchNormalization()) 29 | model.add(Dropout(rate=0.1)) 30 | 31 | model.add(Flatten()) 32 | 33 | model.add(Dense(20, activation='relu')) 34 | model.add(BatchNormalization()) 35 | model.add(Dropout(rate=0.1)) 36 | model.add(Dense(10, activation='softmax')) 37 | 38 | batch_size = 32 39 | num_epochs = 75 40 | 41 | model.summary() 42 | 43 | model.compile(loss='categorical_crossentropy', 44 | optimizer='adam', 45 | metrics=['accuracy']) 46 | 47 | 48 | model.fit(x_train, y_train, 49 | batch_size=batch_size, epochs=num_epochs, 50 | verbose=1, validation_split=0.1, 51 | callbacks=[ModelCheckpoint('mini/weights.{epoch:02d}-{loss:.4f}-{acc:.4f}-{val_loss:.4f}-{val_acc:.4f}.hdf5', 52 | monitor='loss,acc,val_loss,val_acc')]) 53 | 54 | 55 | 56 | Layer (type) Output Shape Param # 57 | ================================================================= 58 | conv2d_25 (Conv2D) (None, 26, 26, 8) 80 59 | _________________________________________________________________ 60 | activation_25 (Activation) (None, 26, 26, 8) 0 61 | _________________________________________________________________ 62 | batch_normalization_36 (Batc (None, 26, 26, 8) 32 63 | _________________________________________________________________ 64 | max_pooling2d_9 (MaxPooling2 (None, 13, 13, 8) 0 65 | _________________________________________________________________ 66 | batch_normalization_37 (Batc (None, 13, 13, 8) 32 67 | _________________________________________________________________ 68 | dropout_21 (Dropout) (None, 13, 13, 8) 0 69 | _________________________________________________________________ 70 | conv2d_26 (Conv2D) (None, 11, 13, 12) 300 71 | _________________________________________________________________ 72 | activation_26 (Activation) (None, 11, 13, 12) 0 73 | _________________________________________________________________ 74 | batch_normalization_38 (Batc (None, 11, 13, 12) 48 75 | _________________________________________________________________ 76 | conv2d_27 (Conv2D) (None, 11, 11, 12) 444 77 | _________________________________________________________________ 78 | activation_27 (Activation) (None, 11, 11, 12) 0 79 | _________________________________________________________________ 80 | batch_normalization_39 (Batc (None, 11, 11, 12) 48 81 | _________________________________________________________________ 82 | max_pooling2d_10 (MaxPooling (None, 5, 5, 12) 0 83 | _________________________________________________________________ 84 | batch_normalization_40 (Batc (None, 5, 5, 12) 48 85 | _________________________________________________________________ 86 | dropout_22 (Dropout) (None, 5, 5, 12) 0 87 | _________________________________________________________________ 88 | conv2d_28 (Conv2D) (None, 3, 5, 15) 555 89 | _________________________________________________________________ 90 | activation_28 (Activation) (None, 3, 5, 15) 0 91 | _________________________________________________________________ 92 | batch_normalization_41 (Batc (None, 3, 5, 15) 60 93 | _________________________________________________________________ 94 | conv2d_29 (Conv2D) (None, 3, 3, 15) 690 95 | _________________________________________________________________ 96 | activation_29 (Activation) (None, 3, 3, 15) 0 97 | _________________________________________________________________ 98 | batch_normalization_42 (Batc (None, 3, 3, 15) 60 99 | _________________________________________________________________ 100 | dropout_23 (Dropout) (None, 3, 3, 15) 0 101 | _________________________________________________________________ 102 | conv2d_30 (Conv2D) (None, 3, 3, 4) 64 103 | _________________________________________________________________ 104 | activation_30 (Activation) (None, 3, 3, 4) 0 105 | _________________________________________________________________ 106 | batch_normalization_43 (Batc (None, 3, 3, 4) 16 107 | _________________________________________________________________ 108 | dropout_24 (Dropout) (None, 3, 3, 4) 0 109 | _________________________________________________________________ 110 | flatten_5 (Flatten) (None, 36) 0 111 | _________________________________________________________________ 112 | dense_9 (Dense) (None, 20) 740 113 | _________________________________________________________________ 114 | batch_normalization_44 (Batc (None, 20) 80 115 | _________________________________________________________________ 116 | dropout_25 (Dropout) (None, 20) 0 117 | _________________________________________________________________ 118 | dense_10 (Dense) (None, 10) 210 119 | ================================================================= 120 | Total params: 3,507 121 | Trainable params: 3,295 122 | Non-trainable params: 212 123 | 124 | 125 | Test score: 0.0271674046346 126 | Test accuracy: 0.9912 127 | -------------------------------------------------------------------------------- /1936/model.txt: -------------------------------------------------------------------------------- 1 | def dw_block(sh_l, prev_l): 2 | l = sh_l(prev_l) 3 | l = Activation(activation='relu')(l) 4 | l = InstanceNormalization(axis=None)(l) 5 | l = Dropout(rate=0.1)(l) 6 | return l 7 | 8 | inputs = Input((28, 28, 1), dtype=np.float32) 9 | l = inputs 10 | 11 | l = Conv2D(8, (3, 3), input_shape=input_shape)(l) 12 | l = Activation(activation='relu')(l) 13 | l = InstanceNormalization(axis=None)(l) 14 | l = MaxPooling2D((2, 2))(l) 15 | l = Dropout(rate=0.1)(l) 16 | 17 | l = SeparableConv2D(26, (3, 3), depth_multiplier=1)(l) 18 | l = Activation(activation='relu')(l) 19 | l = InstanceNormalization(axis=None)(l) 20 | l = Dropout(rate=0.1)(l) 21 | 22 | sh_l = SeparableConv2D(26, (3, 3), depth_multiplier=1, padding='same') 23 | 24 | for n in range(3): 25 | l = dw_block(sh_l, l) 26 | 27 | l = GlobalAveragePooling2D()(l) 28 | 29 | l = Dense(16, activation='relu')(l) 30 | l = InstanceNormalization(axis=None)(l) 31 | l = Dropout(rate=0.1)(l) 32 | l = Dense(10, activation='softmax')(l) 33 | 34 | model = Model(inputs, l) 35 | 36 | model.summary() 37 | 38 | __________________________________________________________________________________________________ 39 | Layer (type) Output Shape Param # Connected to 40 | ================================================================================================== 41 | input_2 (InputLayer) (None, 28, 28, 1) 0 42 | __________________________________________________________________________________________________ 43 | conv2d_2 (Conv2D) (None, 26, 26, 8) 80 input_2[0][0] 44 | __________________________________________________________________________________________________ 45 | activation_7 (Activation) (None, 26, 26, 8) 0 conv2d_2[0][0] 46 | __________________________________________________________________________________________________ 47 | instance_normalization_1 (Insta (None, 26, 26, 8) 2 activation_7[0][0] 48 | __________________________________________________________________________________________________ 49 | max_pooling2d_2 (MaxPooling2D) (None, 13, 13, 8) 0 instance_normalization_1[0][0] 50 | __________________________________________________________________________________________________ 51 | dropout_8 (Dropout) (None, 13, 13, 8) 0 max_pooling2d_2[0][0] 52 | __________________________________________________________________________________________________ 53 | separable_conv2d_3 (SeparableCo (None, 11, 11, 26) 306 dropout_8[0][0] 54 | __________________________________________________________________________________________________ 55 | activation_8 (Activation) (None, 11, 11, 26) 0 separable_conv2d_3[0][0] 56 | __________________________________________________________________________________________________ 57 | instance_normalization_2 (Insta (None, 11, 11, 26) 2 activation_8[0][0] 58 | __________________________________________________________________________________________________ 59 | dropout_9 (Dropout) (None, 11, 11, 26) 0 instance_normalization_2[0][0] 60 | __________________________________________________________________________________________________ 61 | separable_conv2d_4 (SeparableCo (None, 11, 11, 26) 936 dropout_9[0][0] 62 | dropout_10[0][0] 63 | dropout_11[0][0] 64 | __________________________________________________________________________________________________ 65 | activation_9 (Activation) (None, 11, 11, 26) 0 separable_conv2d_4[0][0] 66 | __________________________________________________________________________________________________ 67 | instance_normalization_3 (Insta (None, 11, 11, 26) 2 activation_9[0][0] 68 | __________________________________________________________________________________________________ 69 | dropout_10 (Dropout) (None, 11, 11, 26) 0 instance_normalization_3[0][0] 70 | __________________________________________________________________________________________________ 71 | activation_10 (Activation) (None, 11, 11, 26) 0 separable_conv2d_4[1][0] 72 | __________________________________________________________________________________________________ 73 | instance_normalization_4 (Insta (None, 11, 11, 26) 2 activation_10[0][0] 74 | __________________________________________________________________________________________________ 75 | dropout_11 (Dropout) (None, 11, 11, 26) 0 instance_normalization_4[0][0] 76 | __________________________________________________________________________________________________ 77 | activation_11 (Activation) (None, 11, 11, 26) 0 separable_conv2d_4[2][0] 78 | __________________________________________________________________________________________________ 79 | instance_normalization_5 (Insta (None, 11, 11, 26) 2 activation_11[0][0] 80 | __________________________________________________________________________________________________ 81 | dropout_12 (Dropout) (None, 11, 11, 26) 0 instance_normalization_5[0][0] 82 | __________________________________________________________________________________________________ 83 | global_average_pooling2d_2 (Glo (None, 26) 0 dropout_12[0][0] 84 | __________________________________________________________________________________________________ 85 | dense_3 (Dense) (None, 16) 432 global_average_pooling2d_2[0][0] 86 | __________________________________________________________________________________________________ 87 | instance_normalization_6 (Insta (None, 16) 2 dense_3[0][0] 88 | __________________________________________________________________________________________________ 89 | dropout_13 (Dropout) (None, 16) 0 instance_normalization_6[0][0] 90 | __________________________________________________________________________________________________ 91 | dense_4 (Dense) (None, 10) 170 dropout_13[0][0] 92 | ================================================================================================== 93 | Total params: 1,936 94 | Trainable params: 1,936 95 | Non-trainable params: 0 96 | 97 | Test score: 0.02555989224165678 98 | Test accuracy: 0.991 99 | -------------------------------------------------------------------------------- /2180/model.txt: -------------------------------------------------------------------------------- 1 | def dw_block(sh_l, prev_l): 2 | l = sh_l(prev_l) 3 | l = Activation(activation='relu')(l) 4 | l = BatchNormalization()(l) 5 | l = Dropout(rate=0.1)(l) 6 | return l 7 | 8 | inputs = Input((28, 28, 1), dtype=np.float32) 9 | l = inputs 10 | 11 | l = Conv2D(8, (3, 3), input_shape=input_shape)(l) 12 | l = Activation(activation='relu')(l) 13 | l = BatchNormalization()(l) 14 | l = MaxPooling2D((2, 2))(l) 15 | l = Dropout(rate=0.1)(l) 16 | 17 | l = SeparableConv2D(26, (3, 3), depth_multiplier=1)(l) 18 | l = Activation(activation='relu')(l) 19 | l = BatchNormalization()(l) 20 | l = Dropout(rate=0.1)(l) 21 | 22 | sh_l = SeparableConv2D(26, (3, 3), depth_multiplier=1, padding='same') 23 | 24 | for n in range(3): 25 | l = dw_block(sh_l, l) 26 | 27 | l = GlobalAveragePooling2D()(l) 28 | 29 | l = Dense(16, activation='relu')(l) 30 | l = BatchNormalization()(l) 31 | l = Dropout(rate=0.1)(l) 32 | l = Dense(10, activation='softmax')(l) 33 | 34 | model = Model(inputs, l) 35 | 36 | model.summary() 37 | 38 | __________________________________________________________________________________________________ 39 | Layer (type) Output Shape Param # Connected to 40 | ================================================================================================== 41 | input_16 (InputLayer) (None, 28, 28, 1) 0 42 | __________________________________________________________________________________________________ 43 | conv2d_80 (Conv2D) (None, 26, 26, 8) 80 input_16[0][0] 44 | __________________________________________________________________________________________________ 45 | activation_386 (Activation) (None, 26, 26, 8) 0 conv2d_80[0][0] 46 | __________________________________________________________________________________________________ 47 | batch_normalization_419 (BatchN (None, 26, 26, 8) 32 activation_386[0][0] 48 | __________________________________________________________________________________________________ 49 | max_pooling2d_80 (MaxPooling2D) (None, 13, 13, 8) 0 batch_normalization_419[0][0] 50 | __________________________________________________________________________________________________ 51 | dropout_402 (Dropout) (None, 13, 13, 8) 0 max_pooling2d_80[0][0] 52 | __________________________________________________________________________________________________ 53 | separable_conv2d_79 (SeparableC (None, 11, 11, 26) 306 dropout_402[0][0] 54 | __________________________________________________________________________________________________ 55 | activation_387 (Activation) (None, 11, 11, 26) 0 separable_conv2d_79[0][0] 56 | __________________________________________________________________________________________________ 57 | batch_normalization_420 (BatchN (None, 11, 11, 26) 104 activation_387[0][0] 58 | __________________________________________________________________________________________________ 59 | dropout_403 (Dropout) (None, 11, 11, 26) 0 batch_normalization_420[0][0] 60 | __________________________________________________________________________________________________ 61 | separable_conv2d_80 (SeparableC (None, 11, 11, 26) 936 dropout_403[0][0] 62 | dropout_404[0][0] 63 | dropout_405[0][0] 64 | __________________________________________________________________________________________________ 65 | activation_388 (Activation) (None, 11, 11, 26) 0 separable_conv2d_80[0][0] 66 | __________________________________________________________________________________________________ 67 | batch_normalization_421 (BatchN (None, 11, 11, 26) 104 activation_388[0][0] 68 | __________________________________________________________________________________________________ 69 | dropout_404 (Dropout) (None, 11, 11, 26) 0 batch_normalization_421[0][0] 70 | __________________________________________________________________________________________________ 71 | activation_389 (Activation) (None, 11, 11, 26) 0 separable_conv2d_80[1][0] 72 | __________________________________________________________________________________________________ 73 | batch_normalization_422 (BatchN (None, 11, 11, 26) 104 activation_389[0][0] 74 | __________________________________________________________________________________________________ 75 | dropout_405 (Dropout) (None, 11, 11, 26) 0 batch_normalization_422[0][0] 76 | __________________________________________________________________________________________________ 77 | activation_390 (Activation) (None, 11, 11, 26) 0 separable_conv2d_80[2][0] 78 | __________________________________________________________________________________________________ 79 | batch_normalization_423 (BatchN (None, 11, 11, 26) 104 activation_390[0][0] 80 | __________________________________________________________________________________________________ 81 | dropout_406 (Dropout) (None, 11, 11, 26) 0 batch_normalization_423[0][0] 82 | __________________________________________________________________________________________________ 83 | global_average_pooling2d_37 (Gl (None, 26) 0 dropout_406[0][0] 84 | __________________________________________________________________________________________________ 85 | dense_105 (Dense) (None, 16) 432 global_average_pooling2d_37[0][0] 86 | __________________________________________________________________________________________________ 87 | batch_normalization_424 (BatchN (None, 16) 64 dense_105[0][0] 88 | __________________________________________________________________________________________________ 89 | dropout_407 (Dropout) (None, 16) 0 batch_normalization_424[0][0] 90 | __________________________________________________________________________________________________ 91 | dense_106 (Dense) (None, 10) 170 dropout_407[0][0] 92 | ================================================================================================== 93 | Total params: 2,436 94 | Trainable params: 2,180 95 | Non-trainable params: 256 96 | 97 | Test score: 0.027863642352301394 98 | Test accuracy: 0.9915 99 | 100 | -------------------------------------------------------------------------------- /2374/model.txt: -------------------------------------------------------------------------------- 1 | def dw_block(sh_l, prev_l): 2 | l = sh_l(prev_l) 3 | l = Activation(activation='relu')(l) 4 | l = BatchNormalization()(l) 5 | l = Dropout(rate=0.1)(l) 6 | return l 7 | 8 | inputs = Input((28, 28, 1), dtype=np.float32) 9 | l = inputs 10 | 11 | l = Conv2D(8, (3, 3), input_shape=input_shape)(l) 12 | l = Activation(activation='relu')(l) 13 | l = BatchNormalization()(l) 14 | l = MaxPooling2D((2, 2))(l) 15 | l = Dropout(rate=0.1)(l) 16 | 17 | l = SeparableConv2D(28, (3, 3), depth_multiplier=1)(l) 18 | l = Activation(activation='relu')(l) 19 | l = BatchNormalization()(l) 20 | l = Dropout(rate=0.1)(l) 21 | 22 | sh_l = SeparableConv2D(28, (3, 3), depth_multiplier=1, padding='same') 23 | 24 | for n in range(3): 25 | l = dw_block(sh_l, l) 26 | 27 | l = GlobalAveragePooling2D()(l) 28 | 29 | l = Dense(16, activation='relu')(l) 30 | l = BatchNormalization()(l) 31 | l = Dropout(rate=0.1)(l) 32 | l = Dense(10, activation='softmax')(l) 33 | 34 | model = Model(inputs, l) 35 | 36 | model.summary() 37 | 38 | __________________________________________________________________________________________________ 39 | Layer (type) Output Shape Param # Connected to 40 | ================================================================================================== 41 | input_10 (InputLayer) (None, 28, 28, 1) 0 42 | __________________________________________________________________________________________________ 43 | conv2d_74 (Conv2D) (None, 26, 26, 8) 80 input_10[0][0] 44 | __________________________________________________________________________________________________ 45 | activation_346 (Activation) (None, 26, 26, 8) 0 conv2d_74[0][0] 46 | __________________________________________________________________________________________________ 47 | batch_normalization_381 (BatchN (None, 26, 26, 8) 32 activation_346[0][0] 48 | __________________________________________________________________________________________________ 49 | max_pooling2d_74 (MaxPooling2D) (None, 13, 13, 8) 0 batch_normalization_381[0][0] 50 | __________________________________________________________________________________________________ 51 | dropout_364 (Dropout) (None, 13, 13, 8) 0 max_pooling2d_74[0][0] 52 | __________________________________________________________________________________________________ 53 | separable_conv2d_67 (SeparableC (None, 11, 11, 28) 324 dropout_364[0][0] 54 | __________________________________________________________________________________________________ 55 | activation_347 (Activation) (None, 11, 11, 28) 0 separable_conv2d_67[0][0] 56 | __________________________________________________________________________________________________ 57 | batch_normalization_382 (BatchN (None, 11, 11, 28) 112 activation_347[0][0] 58 | __________________________________________________________________________________________________ 59 | dropout_365 (Dropout) (None, 11, 11, 28) 0 batch_normalization_382[0][0] 60 | __________________________________________________________________________________________________ 61 | separable_conv2d_68 (SeparableC (None, 11, 11, 28) 1064 dropout_365[0][0] 62 | dropout_366[0][0] 63 | dropout_367[0][0] 64 | __________________________________________________________________________________________________ 65 | activation_348 (Activation) (None, 11, 11, 28) 0 separable_conv2d_68[0][0] 66 | __________________________________________________________________________________________________ 67 | batch_normalization_383 (BatchN (None, 11, 11, 28) 112 activation_348[0][0] 68 | __________________________________________________________________________________________________ 69 | dropout_366 (Dropout) (None, 11, 11, 28) 0 batch_normalization_383[0][0] 70 | __________________________________________________________________________________________________ 71 | activation_349 (Activation) (None, 11, 11, 28) 0 separable_conv2d_68[1][0] 72 | __________________________________________________________________________________________________ 73 | batch_normalization_384 (BatchN (None, 11, 11, 28) 112 activation_349[0][0] 74 | __________________________________________________________________________________________________ 75 | dropout_367 (Dropout) (None, 11, 11, 28) 0 batch_normalization_384[0][0] 76 | __________________________________________________________________________________________________ 77 | activation_350 (Activation) (None, 11, 11, 28) 0 separable_conv2d_68[2][0] 78 | __________________________________________________________________________________________________ 79 | batch_normalization_385 (BatchN (None, 11, 11, 28) 112 activation_350[0][0] 80 | __________________________________________________________________________________________________ 81 | dropout_368 (Dropout) (None, 11, 11, 28) 0 batch_normalization_385[0][0] 82 | __________________________________________________________________________________________________ 83 | global_average_pooling2d_31 (Gl (None, 28) 0 dropout_368[0][0] 84 | __________________________________________________________________________________________________ 85 | dense_93 (Dense) (None, 16) 464 global_average_pooling2d_31[0][0] 86 | __________________________________________________________________________________________________ 87 | batch_normalization_386 (BatchN (None, 16) 64 dense_93[0][0] 88 | __________________________________________________________________________________________________ 89 | dropout_369 (Dropout) (None, 16) 0 batch_normalization_386[0][0] 90 | __________________________________________________________________________________________________ 91 | dense_94 (Dense) (None, 10) 170 dropout_369[0][0] 92 | ================================================================================================== 93 | Total params: 2,646 94 | Trainable params: 2,374 95 | Non-trainable params: 272 96 | 97 | Test score: 0.029238566553150304 98 | Test accuracy: 0.9912 99 | 100 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # The minimal neural network that achieves 99% on MNIST 2 | 3 | Here I am trying to find out what is the **minimal** model in terms of number of parameters that can achieve **99% accuracy** on **MNIST** dataset. 4 | 5 | All sub folders in this repository contains model.txt file with description of a model and hdf5 files with its weights. The subfolder name is the number of trainable parameters of the model. 6 | 7 | The record holder at the moment is the next network: 8 | 9 | ```python 10 | def dw_block(sh_l, prev_l): 11 | l = sh_l(prev_l) 12 | l = Activation(activation='relu')(l) 13 | l = InstanceNormalization(axis=None)(l) 14 | l = Dropout(rate=0.1)(l) 15 | return l 16 | 17 | inputs = Input((28, 28, 1), dtype=np.float32) 18 | l = inputs 19 | 20 | l = Conv2D(8, (3, 3), input_shape=input_shape)(l) 21 | l = Activation(activation='relu')(l) 22 | l = InstanceNormalization(axis=None)(l) 23 | l = MaxPooling2D((2, 2))(l) 24 | l = Dropout(rate=0.1)(l) 25 | 26 | l = SeparableConv2D(26, (3, 3), depth_multiplier=1)(l) 27 | l = Activation(activation='relu')(l) 28 | l = InstanceNormalization(axis=None)(l) 29 | l = Dropout(rate=0.1)(l) 30 | 31 | sh_l = SeparableConv2D(26, (3, 3), depth_multiplier=1, padding='same') 32 | 33 | for n in range(3): 34 | l = dw_block(sh_l, l) 35 | 36 | l = GlobalAveragePooling2D()(l) 37 | 38 | l = Dense(16, activation='relu')(l) 39 | l = InstanceNormalization(axis=None)(l) 40 | l = Dropout(rate=0.1)(l) 41 | l = Dense(10, activation='softmax')(l) 42 | 43 | model = Model(inputs, l) 44 | 45 | batch_size = 32 46 | num_epochs = 75 47 | 48 | model.summary() 49 | 50 | model.compile(loss='categorical_crossentropy', 51 | optimizer='adam', 52 | metrics=['accuracy']) 53 | 54 | w_path = os.path.join('mini', model.name) 55 | 56 | if not os.path.exists(w_path): 57 | os.makedirs(w_path) 58 | 59 | f_name = os.path.join(w_path, 'weights.{epoch:02d}-{loss:.4f}-{acc:.4f}-{val_loss:.4f}-{val_acc:.4f}.hdf5') 60 | 61 | model.fit(x_train, y_train, 62 | batch_size=batch_size, epochs=num_epochs, 63 | verbose=1, validation_split=0.1, 64 | callbacks=[ModelCheckpoint(f_name)]) 65 | 66 | ``` 67 | 68 | ``` 69 | __________________________________________________________________________________________________ 70 | Layer (type) Output Shape Param # Connected to 71 | ================================================================================================== 72 | input_2 (InputLayer) (None, 28, 28, 1) 0 73 | __________________________________________________________________________________________________ 74 | conv2d_2 (Conv2D) (None, 26, 26, 8) 80 input_2[0][0] 75 | __________________________________________________________________________________________________ 76 | activation_7 (Activation) (None, 26, 26, 8) 0 conv2d_2[0][0] 77 | __________________________________________________________________________________________________ 78 | instance_normalization_1 (Insta (None, 26, 26, 8) 2 activation_7[0][0] 79 | __________________________________________________________________________________________________ 80 | max_pooling2d_2 (MaxPooling2D) (None, 13, 13, 8) 0 instance_normalization_1[0][0] 81 | __________________________________________________________________________________________________ 82 | dropout_8 (Dropout) (None, 13, 13, 8) 0 max_pooling2d_2[0][0] 83 | __________________________________________________________________________________________________ 84 | separable_conv2d_3 (SeparableCo (None, 11, 11, 26) 306 dropout_8[0][0] 85 | __________________________________________________________________________________________________ 86 | activation_8 (Activation) (None, 11, 11, 26) 0 separable_conv2d_3[0][0] 87 | __________________________________________________________________________________________________ 88 | instance_normalization_2 (Insta (None, 11, 11, 26) 2 activation_8[0][0] 89 | __________________________________________________________________________________________________ 90 | dropout_9 (Dropout) (None, 11, 11, 26) 0 instance_normalization_2[0][0] 91 | __________________________________________________________________________________________________ 92 | separable_conv2d_4 (SeparableCo (None, 11, 11, 26) 936 dropout_9[0][0] 93 | dropout_10[0][0] 94 | dropout_11[0][0] 95 | __________________________________________________________________________________________________ 96 | activation_9 (Activation) (None, 11, 11, 26) 0 separable_conv2d_4[0][0] 97 | __________________________________________________________________________________________________ 98 | instance_normalization_3 (Insta (None, 11, 11, 26) 2 activation_9[0][0] 99 | __________________________________________________________________________________________________ 100 | dropout_10 (Dropout) (None, 11, 11, 26) 0 instance_normalization_3[0][0] 101 | __________________________________________________________________________________________________ 102 | activation_10 (Activation) (None, 11, 11, 26) 0 separable_conv2d_4[1][0] 103 | __________________________________________________________________________________________________ 104 | instance_normalization_4 (Insta (None, 11, 11, 26) 2 activation_10[0][0] 105 | __________________________________________________________________________________________________ 106 | dropout_11 (Dropout) (None, 11, 11, 26) 0 instance_normalization_4[0][0] 107 | __________________________________________________________________________________________________ 108 | activation_11 (Activation) (None, 11, 11, 26) 0 separable_conv2d_4[2][0] 109 | __________________________________________________________________________________________________ 110 | instance_normalization_5 (Insta (None, 11, 11, 26) 2 activation_11[0][0] 111 | __________________________________________________________________________________________________ 112 | dropout_12 (Dropout) (None, 11, 11, 26) 0 instance_normalization_5[0][0] 113 | __________________________________________________________________________________________________ 114 | global_average_pooling2d_2 (Glo (None, 26) 0 dropout_12[0][0] 115 | __________________________________________________________________________________________________ 116 | dense_3 (Dense) (None, 16) 432 global_average_pooling2d_2[0][0] 117 | __________________________________________________________________________________________________ 118 | instance_normalization_6 (Insta (None, 16) 2 dense_3[0][0] 119 | __________________________________________________________________________________________________ 120 | dropout_13 (Dropout) (None, 16) 0 instance_normalization_6[0][0] 121 | __________________________________________________________________________________________________ 122 | dense_4 (Dense) (None, 10) 170 dropout_13[0][0] 123 | ================================================================================================== 124 | Total params: 1,936 125 | Trainable params: 1,936 126 | Non-trainable params: 0 127 | 128 | ``` 129 | 130 | ``` 131 | Test score: 0.02555989224165678 132 | Test accuracy: 0.991 133 | ``` 134 | --------------------------------------------------------------------------------