├── README.md ├── check.py ├── data_utils.py ├── dataset_sample ├── 1377.png ├── 1380.png ├── 1418.png ├── 1489.png ├── 1511.png ├── 1512.png ├── 1513.png ├── 1514.png ├── 1515.png ├── 1516.png ├── 1517.png ├── 1518.png ├── 1519.png └── 1520.png └── main.py /README.md: -------------------------------------------------------------------------------- 1 | # Bone Age Estimation 2 | ### Contribured by Vishal Agarwal, Jayanth Reddy 3 | 4 | A Deep Learning aproach for estimating bone age from hand x-ray images 5 | 6 | This was done as a part of NVIDIA problem statement at an 24 hour Hack2Innovate hackathon, conducted at Indian Institute of Technology Guwahati. 7 | 8 | The dataset used for this problem statement was taken from a RSNA challenge for Pediatric Bone Age. Some of the images from the dataset are shown below 9 | 10 | [![1377.png](https://raw.githubusercontent.com/thevishalagarwal/BoneAgeEstimation/master/dataset_sample/1377.png)](https://postimg.org/image/6ccovzmdb/) [![1380.png](https://raw.githubusercontent.com/thevishalagarwal/BoneAgeEstimation/master/dataset_sample/1380.png)](https://postimg.org/image/jujl8alnz/) [![1489.png](https://raw.githubusercontent.com/thevishalagarwal/BoneAgeEstimation/master/dataset_sample/1489.png)](https://postimg.org/image/8uydwpaof/) 11 | 12 | We used the concept of end-to-end learning using InceptionV3 architechture with custom fully connected layers in the final stage for regression. Transfer learning from ImageNet pre-trained models was used to initialize the weights of InceptionV3. Experiments were done with other architechtures as well and InceptionV3 was found to be performing better among those. 13 | The evaluation metric used in our model is Mean Absolute Error(MAE). 14 | 15 | #### Evaluation Metrics 16 | 17 | Train : MAE = 8.726 18 | 19 | Valid : MAE = 10.958 20 | 21 | Test : MAE = 8.578 22 | 23 | #### Clinical application of bone age 24 | - Beckwith–Wiedemann syndrome 25 | - Constitutional Delay of Growth and Puberty 26 | - Idiopathic Short Stature 27 | - Sotos syndrome 28 | - Short Children Born Small for Gestational Age 29 | - GH Deficiency 30 | - Turner Syndrome 31 | - Chronic Renal Failure 32 | - Marshall–Smith syndrome 33 | 34 | #### Future Work 35 | - Advanced feature extraction techniques can be used to improve further accuracy. 36 | - The RSNA Pediatric Bone Age Dataset also contains gender labels. So the model trained separately for male and female classes will improve the accuracy. 37 | 38 | The weights for our model can be found [here](https://www.dropbox.com/s/rfivlkm0uqxi3f4/model.h5?dl=0) 39 | -------------------------------------------------------------------------------- /check.py: -------------------------------------------------------------------------------- 1 | # Check metrics using trained weight files 2 | 3 | from keras.applications.inception_v3 import InceptionV3 4 | from keras.preprocessing import image 5 | from keras.models import Model 6 | from keras.layers import Flatten, Dense, Input 7 | from keras import backend as K 8 | import cPickle 9 | import numpy as np 10 | import matplotlib.pyplot as plt 11 | 12 | batch_size = 32 13 | epochs = 10 14 | 15 | f = open('../data.pkl', 'rb') 16 | x = cPickle.load(f) 17 | f.close() 18 | f = open('../data_age.pkl', 'rb') 19 | y = cPickle.load(f) 20 | f.close() 21 | x = np.asarray(x, dtype=np.float32) 22 | y = np.asarray(y) 23 | x /= 255. 24 | x_final = [] 25 | y_final = [] 26 | random_no = np.random.choice(x.shape[0], size=x.shape[0], replace=False) 27 | for i in random_no: 28 | x_final.append(x[i,:,:,:]) 29 | y_final.append(y[i]) 30 | x_final = np.asarray(x_final) 31 | y_final = np.asarray(y_final) 32 | 33 | k = 1000 # Decides split count 34 | x_test = x_final[:k,:,:,:] 35 | y_test = y_final[:k] 36 | x_valid = x_final[k:2*k,:,:,:] 37 | y_valid = y_final[k:2*k] 38 | x_train = x_final[2*k:,:,:,:] 39 | y_train = y_final[2*k:] 40 | print 'x_train shape:'+ str(x_train.shape) 41 | print 'y_train shape:'+ str(y_train.shape) 42 | print 'x_valid shape:'+ str(x_valid.shape) 43 | print 'y_valid shape:'+ str(y_valid.shape) 44 | print 'x_test shape:'+ str(x_test.shape) 45 | print 'y_test shape:'+ str(y_test.shape) 46 | 47 | base_model = InceptionV3(weights='imagenet', include_top=False) 48 | input = Input(shape=(224,224,3)) 49 | output_vgg16 = base_model(input) 50 | x = Flatten()(output_vgg16) 51 | x = Dense(512, activation='relu')(x) 52 | predictions = Dense(1)(x) 53 | 54 | model = Model(inputs=input, outputs=predictions) 55 | 56 | model.compile(optimizer='adam', loss='mean_squared_error', metrics=['MAE']) 57 | model.load_weights('model.h5') 58 | 59 | score = model.evaluate(x_test, y_test, batch_size=batch_size) 60 | print('Test loss:', score[0]) 61 | print('Test MAE:', score[1]) 62 | 63 | 64 | -------------------------------------------------------------------------------- /data_utils.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import cv2 3 | import os 4 | import pandas as pd 5 | from six.moves import cPickle 6 | 7 | # For this problem the validation and test data provided by the concerned authority did not have labels, so the training data was split into train, test and validation sets 8 | train_dir = '/mnt/boneage-training-dataset/' 9 | 10 | X_train = [] 11 | y_age = [] 12 | y_gender = [] 13 | 14 | df = pd.read_csv('../train.csv') 15 | a = df.as_matrix() 16 | m = a.shape[0] 17 | 18 | path = train_dir 19 | k = 0 20 | print 'Loading data set...' 21 | for i in os.listdir(path): 22 | y_age.append(df.boneage[df.id == int(i[:-4])].tolist()[0]) 23 | a = df.male[df.id == int(i[:-4])].tolist()[0] 24 | if a: 25 | y_gender.append(1) 26 | else: 27 | y_gender.append(0) 28 | img_path = path + i 29 | img = cv2.imread(img_path) 30 | img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) 31 | img = cv2.resize(img,(224,224)) 32 | x = np.asarray(img, dtype=np.uint8) 33 | X_train.append(x) 34 | 35 | print '' 36 | print '100% completed loading data' 37 | 38 | # Save data 39 | train_pkl = open('data.pkl','wb') 40 | cPickle.dump(X_train, train_pkl, protocol=cPickle.HIGHEST_PROTOCOL) 41 | train_pkl.close() 42 | 43 | train_age_pkl = open('data_age.pkl','wb') 44 | cPickle.dump(y_age, train_age_pkl, protocol=cPickle.HIGHEST_PROTOCOL) 45 | train_age_pkl.close() 46 | 47 | train_gender_pkl = open('data_gender.pkl','wb') 48 | cPickle.dump(y_gender, train_gender_pkl, protocol=cPickle.HIGHEST_PROTOCOL) 49 | train_gender_pkl.close() 50 | 51 | -------------------------------------------------------------------------------- /dataset_sample/1377.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevishalagarwal/BoneAgeEstimation/592dc3d1d77d87d32204be6108bef0cad07bd9e2/dataset_sample/1377.png -------------------------------------------------------------------------------- /dataset_sample/1380.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevishalagarwal/BoneAgeEstimation/592dc3d1d77d87d32204be6108bef0cad07bd9e2/dataset_sample/1380.png -------------------------------------------------------------------------------- /dataset_sample/1418.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevishalagarwal/BoneAgeEstimation/592dc3d1d77d87d32204be6108bef0cad07bd9e2/dataset_sample/1418.png -------------------------------------------------------------------------------- /dataset_sample/1489.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevishalagarwal/BoneAgeEstimation/592dc3d1d77d87d32204be6108bef0cad07bd9e2/dataset_sample/1489.png -------------------------------------------------------------------------------- /dataset_sample/1511.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevishalagarwal/BoneAgeEstimation/592dc3d1d77d87d32204be6108bef0cad07bd9e2/dataset_sample/1511.png -------------------------------------------------------------------------------- /dataset_sample/1512.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevishalagarwal/BoneAgeEstimation/592dc3d1d77d87d32204be6108bef0cad07bd9e2/dataset_sample/1512.png -------------------------------------------------------------------------------- /dataset_sample/1513.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevishalagarwal/BoneAgeEstimation/592dc3d1d77d87d32204be6108bef0cad07bd9e2/dataset_sample/1513.png -------------------------------------------------------------------------------- /dataset_sample/1514.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevishalagarwal/BoneAgeEstimation/592dc3d1d77d87d32204be6108bef0cad07bd9e2/dataset_sample/1514.png -------------------------------------------------------------------------------- /dataset_sample/1515.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevishalagarwal/BoneAgeEstimation/592dc3d1d77d87d32204be6108bef0cad07bd9e2/dataset_sample/1515.png -------------------------------------------------------------------------------- /dataset_sample/1516.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevishalagarwal/BoneAgeEstimation/592dc3d1d77d87d32204be6108bef0cad07bd9e2/dataset_sample/1516.png -------------------------------------------------------------------------------- /dataset_sample/1517.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevishalagarwal/BoneAgeEstimation/592dc3d1d77d87d32204be6108bef0cad07bd9e2/dataset_sample/1517.png -------------------------------------------------------------------------------- /dataset_sample/1518.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevishalagarwal/BoneAgeEstimation/592dc3d1d77d87d32204be6108bef0cad07bd9e2/dataset_sample/1518.png -------------------------------------------------------------------------------- /dataset_sample/1519.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevishalagarwal/BoneAgeEstimation/592dc3d1d77d87d32204be6108bef0cad07bd9e2/dataset_sample/1519.png -------------------------------------------------------------------------------- /dataset_sample/1520.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevishalagarwal/BoneAgeEstimation/592dc3d1d77d87d32204be6108bef0cad07bd9e2/dataset_sample/1520.png -------------------------------------------------------------------------------- /main.py: -------------------------------------------------------------------------------- 1 | from keras.applications.inception_v3 import InceptionV3 2 | from keras.preprocessing import image 3 | from keras.models import Model 4 | from keras.layers import Flatten, Dense, Input 5 | from keras import backend as K 6 | import cPickle 7 | import numpy as np 8 | import matplotlib.pyplot as plt 9 | 10 | batch_size = 32 11 | epochs = 30 12 | 13 | # Load data 14 | print('...loading training data') 15 | f = open('../data.pkl', 'rb') 16 | x = cPickle.load(f) 17 | f.close() 18 | 19 | f = open('../data_age.pkl', 'rb') 20 | y = cPickle.load(f) 21 | f.close() 22 | 23 | x = np.asarray(x, dtype=np.float32) 24 | y = np.asarray(y) 25 | 26 | x /= 255. 27 | 28 | x_final = [] 29 | y_final = [] 30 | 31 | # Shuffle images and split into train, validation and test sets 32 | random_no = np.random.choice(x.shape[0], size=x.shape[0], replace=False) 33 | for i in random_no: 34 | x_final.append(x[i,:,:,:]) 35 | y_final.append(y[i]) 36 | 37 | x_final = np.asarray(x_final) 38 | y_final = np.asarray(y_final) 39 | 40 | k = 1000 # Decides split count 41 | x_test = x_final[:k,:,:,:] 42 | y_test = y_final[:k] 43 | x_valid = x_final[k:2*k,:,:,:] 44 | y_valid = y_final[k:2*k] 45 | x_train = x_final[2*k:,:,:,:] 46 | y_train = y_final[2*k:] 47 | 48 | print 'x_train shape:'+ str(x_train.shape) 49 | print 'y_train shape:'+ str(y_train.shape) 50 | print 'x_valid shape:'+ str(x_valid.shape) 51 | print 'y_valid shape:'+ str(y_valid.shape) 52 | print 'x_test shape:'+ str(x_test.shape) 53 | print 'y_test shape:'+ str(y_test.shape) 54 | 55 | # Using InceptionV3 with pretrained weights from Imagenet 56 | base_model = InceptionV3(weights='imagenet', include_top=False) 57 | input = Input(shape=(224,224,3)) 58 | output_vgg16 = base_model(input) 59 | x = Flatten()(output_vgg16) 60 | x = Dense(512, activation='relu')(x) 61 | predictions = Dense(1)(x) 62 | 63 | model = Model(inputs=input, outputs=predictions) 64 | model.compile(optimizer='adam', loss='mean_squared_error', metrics=['MAE']) 65 | 66 | # Save weights after every epoch 67 | checkpoint = keras.callbacks.ModelCheckpoint( 68 | filepath='weights/weights.{epoch:02d}-{val_loss:.2f}.hdf5', 69 | save_weights_only=True, 70 | period=1) 71 | history = model.fit(x_train, y_train,batch_size=batch_size,epochs=epochs,verbose=1,validation_data=(x_valid,y_valid), callbacks = [checkpoint]) 72 | 73 | model.save_weights("model.h5") 74 | 75 | score = model.evaluate(x_test, y_test, batch_size=batch_size) 76 | print('Test loss:', score[0]) 77 | print('Test MAE:', score[1]) 78 | 79 | with open('history.pkl', 'wb') as f: 80 | cPickle.dump(history.history, f) 81 | f.close() 82 | 83 | 84 | --------------------------------------------------------------------------------