├── requirements.txt ├── LICENSE ├── .gitignore ├── README.md ├── baseline_gender.py └── baseline_age.py /requirements.txt: -------------------------------------------------------------------------------- 1 | pandas==0.20.3 2 | Keras==2.2.4 3 | numpy==1.16.3 4 | tqdm==4.23.2 5 | Pillow==6.0.0 6 | scikit_learn==0.21.1 7 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 myouness 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | env/ 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | downloads/ 16 | eggs/ 17 | .eggs/ 18 | lib/ 19 | lib64/ 20 | parts/ 21 | sdist/ 22 | var/ 23 | wheels/ 24 | *.egg-info/ 25 | .installed.cfg 26 | *.egg 27 | 28 | # PyInstaller 29 | # Usually these files are written by a python script from a template 30 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 31 | *.manifest 32 | *.spec 33 | 34 | # Installer logs 35 | pip-log.txt 36 | pip-delete-this-directory.txt 37 | 38 | # Unit test / coverage reports 39 | htmlcov/ 40 | .tox/ 41 | .coverage 42 | .coverage.* 43 | .cache 44 | nosetests.xml 45 | coverage.xml 46 | *.cover 47 | .hypothesis/ 48 | 49 | # Translations 50 | *.mo 51 | *.pot 52 | 53 | # Django stuff: 54 | *.log 55 | local_settings.py 56 | 57 | # Flask stuff: 58 | instance/ 59 | .webassets-cache 60 | 61 | # Scrapy stuff: 62 | .scrapy 63 | 64 | # Sphinx documentation 65 | docs/_build/ 66 | 67 | # PyBuilder 68 | target/ 69 | 70 | # Jupyter Notebook 71 | .ipynb_checkpoints 72 | 73 | # pyenv 74 | .python-version 75 | 76 | # celery beat schedule file 77 | celerybeat-schedule 78 | 79 | # SageMath parsed files 80 | *.sage.py 81 | 82 | # dotenv 83 | .env 84 | 85 | # virtualenv 86 | .venv 87 | venv/ 88 | ENV/ 89 | 90 | # Spyder project settings 91 | .spyderproject 92 | .spyproject 93 | 94 | # Rope project settings 95 | .ropeproject 96 | 97 | # mkdocs documentation 98 | /site 99 | 100 | # mypy 101 | .mypy_cache/ 102 | 103 | .idea/* -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # face_age_gender 2 | 3 | Data : https://www.openu.ac.il/home/hassner/Adience/data.html 4 | 5 | Description of the approach : https://medium.com/@CVxTz/predicting-apparent-age-and-gender-from-face-picture-keras-tensorflow-a99413d8fd5e 6 | 7 | requirements : Keras, tensorflow, numpy, PIL, cv2 8 | 9 | 10 | # Predicting apparent Age and Gender from face picture : Keras + Tensorflow 11 | 12 | Source : 13 | [https://www.openu.ac.il/home/hassner/Adience/data.html](https://www.openu.ac.il/home/hassner/Adience/data.html) 14 | 15 | Predicting the apparent age and gender from a picture is a very interesting 16 | problem from a technical point of view but can also be very useful when applied 17 | to better understand consumer segments or a user base for example. It can be 18 | used to infer the age or gender of a user and use this information to make 19 | personalized products and experiences for each user. 20 | 21 | In this post we will train a model to predict those attributes given a face 22 | picture. 23 | 24 | ### Data : 25 | 26 | We use data from 27 | [https://www.openu.ac.il/home/hassner/Adience/data.html](https://www.openu.ac.il/home/hassner/Adience/data.html) 28 | which is a dataset of face photos in the wild that are labeled into 8 age groups 29 | (0–2, 4–6, 8–13, 15–20, 25–32, 38–43, 48–53, 60-) and into 2 gender classes.
30 | There are around 26,580 images (with missing labels in some cases) that are 31 | pre-split into 5 folds. 32 | 33 | ### Existing results : 34 | 35 | As this dataset is usually used as a benchmark for this type of tasks in many 36 | research papers, I was able to find many prior **accuracy** results for apparent 37 | age and gender prediction => 38 | 39 | [[1] 40 | https://www.openu.ac.il/home/hassner/Adience/EidingerEnbarHassner_tifs.pdf](https://www.openu.ac.il/home/hassner/Adience/EidingerEnbarHassner_tifs.pdf) 41 |
Gender : 76.1±0.9
Age : 45.1±2.6 42 | 43 | [2] 44 | [https://www.openu.ac.il/home/hassner/projects/cnn_agegender/CNN_AgeGenderEstimation.pdf](https://www.openu.ac.il/home/hassner/projects/cnn_agegender/CNN_AgeGenderEstimation.pdf)
45 | Gender : 86.8±1.4
Age : 50.7±5.1 46 | 47 | [3] 48 | [https://arxiv.org/pdf/1702.04280.pdf](https://arxiv.org/pdf/1702.04280.pdf)
49 | Gender : 91
Age : 61.3±3.7 50 | 51 | ### Preprocessing : 52 | 53 | Faces are cropped and aligned using this tool : 54 | [https://www.openu.ac.il/home/hassner/Adience/code.html#inplanealign](https://www.openu.ac.il/home/hassner/Adience/code.html#inplanealign) 55 | 56 | ### Data augmentation : 57 | 58 | We use Random shift, Zoom, Horizontal Flip as a form of data augmentation to 59 | create synthetic examples used during training to improve the generalization of 60 | the model. 61 | 62 | ### Model : 63 | 64 | We use a [Resnet](https://arxiv.org/pdf/1512.03385.pdf) architecture pre-trained 65 | on ImageNet : 66 | 67 | Resnet ( Deep Residual Networks ) are an architecture that reduces the 68 | under-fitting and optimization issues that occur in deep neural networks. 69 | 70 | Residual Block : 71 | [https://arxiv.org/pdf/1512.03385.pdf](https://arxiv.org/pdf/1512.03385.pdf) 72 | 73 | ### Results : 74 | 75 | We train the model 3 times for each fold and average the predictions and get the 76 | following results :
Gender : 89.4±1.4
Age : 57.1±5.3
Those results 77 | are better than [1] and [2] probably because of the bagging and the pretrained 78 | weights but worse than [3] probably because Sighthound Inc used a bigger and 79 | internal Faces dataset for pretraining. 80 | 81 | Code to reproduce the results can be found at : 82 | [https://github.com/CVxTz/face_age_gender](https://github.com/CVxTz/face_age_gender) 83 | -------------------------------------------------------------------------------- /baseline_gender.py: -------------------------------------------------------------------------------- 1 | import pandas as pd 2 | import os 3 | from PIL import Image 4 | import numpy as np 5 | from keras.callbacks import ModelCheckpoint, LearningRateScheduler, EarlyStopping, ReduceLROnPlateau, TensorBoard 6 | from keras import optimizers, losses, activations, models 7 | from keras.layers import Convolution2D, Dense, Input, Flatten, Dropout, MaxPooling2D, BatchNormalization, \ 8 | GlobalMaxPool2D, Concatenate, GlobalMaxPooling2D, GlobalAveragePooling2D, Lambda 9 | from keras.applications.resnet50 import ResNet50 10 | from keras.preprocessing.image import ImageDataGenerator 11 | from keras.models import Model 12 | from sklearn.model_selection import train_test_split 13 | from sklearn.metrics import accuracy_score 14 | from keras import backend as K 15 | 16 | from tqdm import tqdm 17 | 18 | 19 | def read_and_resize(filepath, input_shape=(256, 256)): 20 | im = Image.open((filepath)).convert('RGB') 21 | im = im.resize(input_shape) 22 | im_array = np.array(im, dtype="uint8")#[..., ::-1] 23 | return np.array(im_array / (np.max(im_array)+ 0.001), dtype="float32") 24 | 25 | datagen = ImageDataGenerator( 26 | rotation_range=6, 27 | width_shift_range=0.1, 28 | height_shift_range=0.1, 29 | horizontal_flip=True, 30 | zoom_range=0.1) 31 | 32 | def augment(im_array): 33 | im_array = datagen.random_transform(im_array) 34 | return im_array 35 | 36 | def gen(df, batch_size=32, aug=False): 37 | df = df.sample(frac=1) 38 | while True: 39 | for i, batch in enumerate([df[i:i+batch_size] for i in range(0,df.shape[0],batch_size)]): 40 | if aug: 41 | images = np.array([augment(read_and_resize(file_path)) for file_path in batch.path.values]) 42 | else: 43 | images = np.array([read_and_resize(file_path) for file_path in batch.path.values]) 44 | 45 | 46 | labels = np.array([int(g=="m") for g in batch.gender.values]) 47 | 48 | yield images, labels 49 | 50 | 51 | def get_model(n_classes=1): 52 | 53 | base_model = ResNet50(weights='imagenet', include_top=False) 54 | 55 | #for layer in base_model.layers: 56 | # layer.trainable = False 57 | 58 | x = base_model.output 59 | x = GlobalMaxPooling2D()(x) 60 | x = Dropout(0.5)(x) 61 | x = Dense(100, activation="relu")(x) 62 | x = Dropout(0.5)(x) 63 | if n_classes == 1: 64 | x = Dense(n_classes, activation="sigmoid")(x) 65 | else: 66 | x = Dense(n_classes, activation="softmax")(x) 67 | 68 | base_model = Model(base_model.input, x, name="base_model") 69 | if n_classes == 1: 70 | base_model.compile(loss="binary_crossentropy", metrics=['acc'], optimizer="adam") 71 | else: 72 | base_model.compile(loss="sparse_categorical_crossentropy", metrics=['acc'], optimizer="adam") 73 | 74 | return base_model 75 | 76 | def create_path(df, base_path): 77 | 78 | df['path'] = df.apply(lambda x: base_path+"aligned/"+x['user_id']+"/landmark_aligned_face.%s.%s" 79 | %(x['face_id'], x['original_image']), axis=1) 80 | 81 | return df 82 | def filter_df(df): 83 | 84 | df['f'] = df.gender.apply(lambda x: int(x in ['f', 'm'])) 85 | df = df[df.f == 1] 86 | return df 87 | 88 | if __name__ == "__main__": 89 | base_path = "/media/ml/data_ml/face_age_gender/" 90 | 91 | bag = 3 92 | 93 | all_indexes = list(range(5)) 94 | 95 | accuracies = [] 96 | 97 | for test_id in tqdm(all_indexes): 98 | 99 | train_id = [j for j in all_indexes if j!=test_id] 100 | print(train_id, test_id) 101 | 102 | train_df = pd.concat([pd.read_csv(base_path+"fold_%s_data.txt"%i, sep="\t") for i in train_id]) 103 | test_df = pd.read_csv(base_path+"fold_%s_data.txt"%test_id, sep="\t") 104 | 105 | train_df = filter_df(train_df) 106 | test_df = filter_df(test_df) 107 | 108 | print(train_df.shape, test_df.shape) 109 | 110 | train_df = create_path(train_df, base_path=base_path) 111 | test_df = create_path(test_df, base_path=base_path) 112 | 113 | cnt_ave = 0 114 | predictions = 0 115 | 116 | test_images = np.array([read_and_resize(file_path) for file_path in test_df.path.values]) 117 | test_labels = np.array([int(g == "m") for g in test_df.gender.values]) 118 | 119 | for k in tqdm(range(bag)): 120 | 121 | 122 | tr_tr, tr_val = train_test_split(train_df, test_size=0.1) 123 | 124 | file_path = "baseline_gender.h5" 125 | 126 | checkpoint = ModelCheckpoint(file_path, monitor='val_acc', verbose=1, save_best_only=True, mode='max') 127 | 128 | early = EarlyStopping(monitor="val_acc", mode="max", patience=10) 129 | 130 | reduce_on_plateau = ReduceLROnPlateau(monitor="val_acc", mode="max", factor=0.1, patience=3) 131 | 132 | callbacks_list = [checkpoint, early, reduce_on_plateau] # early 133 | 134 | model = get_model(n_classes=1) 135 | model.fit_generator(gen(tr_tr, aug=True), validation_data=gen(tr_val), epochs=200, verbose=2, workers=4, 136 | callbacks=callbacks_list, steps_per_epoch=50, validation_steps=30) 137 | 138 | model.load_weights(file_path) 139 | 140 | predictions += model.predict(test_images) 141 | cnt_ave += 1 142 | 143 | test_images = test_images[:, :, ::-1, :] 144 | 145 | predictions += model.predict(test_images) 146 | cnt_ave += 1 147 | 148 | K.clear_session() 149 | 150 | predictions = predictions/cnt_ave 151 | 152 | predictions = (predictions>0.5).astype(np.int8) 153 | 154 | acc = accuracy_score(test_labels, predictions) 155 | 156 | print("accuracy : %s " %acc) 157 | 158 | accuracies.append(acc) 159 | 160 | print("mean acc : %s (%s) " % (np.mean(accuracies), np.std(accuracies))) -------------------------------------------------------------------------------- /baseline_age.py: -------------------------------------------------------------------------------- 1 | import pandas as pd 2 | import os 3 | from PIL import Image 4 | import numpy as np 5 | from keras.callbacks import ModelCheckpoint, LearningRateScheduler, EarlyStopping, ReduceLROnPlateau, TensorBoard 6 | from keras import optimizers, losses, activations, models 7 | from keras.layers import Convolution2D, Dense, Input, Flatten, Dropout, MaxPooling2D, BatchNormalization, \ 8 | GlobalMaxPool2D, Concatenate, GlobalMaxPooling2D, GlobalAveragePooling2D, Lambda 9 | from keras.applications.resnet50 import ResNet50 10 | from keras.preprocessing.image import ImageDataGenerator 11 | from keras.models import Model 12 | from sklearn.model_selection import train_test_split 13 | from sklearn.metrics import accuracy_score 14 | from keras import backend as K 15 | 16 | from tqdm import tqdm 17 | from collections import Counter 18 | 19 | def read_and_resize(filepath, input_shape=(256, 256)): 20 | im = Image.open((filepath)).convert('RGB') 21 | im = im.resize(input_shape) 22 | im_array = np.array(im, dtype="uint8")#[..., ::-1] 23 | return np.array(im_array / (np.max(im_array)+ 0.001), dtype="float32") 24 | 25 | datagen = ImageDataGenerator( 26 | rotation_range=6, 27 | width_shift_range=0.1, 28 | height_shift_range=0.1, 29 | horizontal_flip=True, 30 | zoom_range=0.1) 31 | 32 | def augment(im_array): 33 | im_array = datagen.random_transform(im_array) 34 | return im_array 35 | 36 | def gen(df, batch_size=32, aug=False): 37 | df = df.sample(frac=1) 38 | 39 | dict_age = {'(0, 2)' : 0, 40 | '(4, 6)' : 1, 41 | '(8, 12)' : 2, 42 | '(15, 20)' : 3, 43 | '(25, 32)' : 4, 44 | '(38, 43)' : 5, 45 | '(48, 53)' : 6, 46 | '(60, 100)' : 7} 47 | 48 | while True: 49 | for i, batch in enumerate([df[i:i+batch_size] for i in range(0,df.shape[0],batch_size)]): 50 | if aug: 51 | images = np.array([augment(read_and_resize(file_path)) for file_path in batch.path.values]) 52 | else: 53 | images = np.array([read_and_resize(file_path) for file_path in batch.path.values]) 54 | 55 | 56 | labels = np.array([dict_age[g] for g in batch.age.values]) 57 | labels = labels[..., np.newaxis] 58 | 59 | yield images, labels 60 | 61 | 62 | def get_model(n_classes=1): 63 | 64 | base_model = ResNet50(weights='imagenet', include_top=False) 65 | 66 | #for layer in base_model.layers: 67 | # layer.trainable = False 68 | 69 | x = base_model.output 70 | x = GlobalMaxPooling2D()(x) 71 | x = Dropout(0.5)(x) 72 | x = Dense(100, activation="relu")(x) 73 | x = Dropout(0.5)(x) 74 | if n_classes == 1: 75 | x = Dense(n_classes, activation="sigmoid")(x) 76 | else: 77 | x = Dense(n_classes, activation="softmax")(x) 78 | 79 | base_model = Model(base_model.input, x, name="base_model") 80 | if n_classes == 1: 81 | base_model.compile(loss="binary_crossentropy", metrics=['acc'], optimizer="adam") 82 | else: 83 | base_model.compile(loss="sparse_categorical_crossentropy", metrics=['acc'], optimizer="adam") 84 | 85 | return base_model 86 | 87 | def create_path(df, base_path): 88 | 89 | df['path'] = df.apply(lambda x: base_path+"aligned/"+x['user_id']+"/landmark_aligned_face.%s.%s" 90 | %(x['face_id'], x['original_image']), axis=1) 91 | 92 | return df 93 | def filter_df(df): 94 | 95 | dict_age = {'(0, 2)' : 0, 96 | '(4, 6)' : 1, 97 | '(8, 12)' : 2, 98 | '(15, 20)' : 3, 99 | '(25, 32)' : 4, 100 | '(38, 43)' : 5, 101 | '(48, 53)' : 6, 102 | '(60, 100)' : 7} 103 | 104 | 105 | df['f'] = df.age.apply(lambda x: int(x in dict_age)) 106 | df = df[df.f == 1] 107 | return df 108 | 109 | if __name__ == "__main__": 110 | base_path = "/media/ml/data_ml/face_age_gender/" 111 | 112 | dict_age = {'(0, 2)' : 0, 113 | '(4, 6)' : 1, 114 | '(8, 12)' : 2, 115 | '(15, 20)' : 3, 116 | '(25, 32)' : 4, 117 | '(38, 43)' : 5, 118 | '(48, 53)' : 6, 119 | '(60, 100)' : 7} 120 | 121 | bag = 3 122 | 123 | all_indexes = list(range(5)) 124 | 125 | accuracies = [] 126 | 127 | for test_id in tqdm(all_indexes): 128 | 129 | train_id = [j for j in all_indexes if j!=test_id] 130 | print(train_id, test_id) 131 | 132 | train_df = pd.concat([pd.read_csv(base_path+"fold_%s_data.txt"%i, sep="\t") for i in train_id]) 133 | test_df = pd.read_csv(base_path+"fold_%s_data.txt"%test_id, sep="\t") 134 | 135 | train_df = filter_df(train_df) 136 | test_df = filter_df(test_df) 137 | 138 | print(train_df.shape, test_df.shape) 139 | 140 | train_df = create_path(train_df, base_path=base_path) 141 | test_df = create_path(test_df, base_path=base_path) 142 | 143 | cnt_ave = 0 144 | predictions = 0 145 | 146 | test_images = np.array([read_and_resize(file_path) for file_path in test_df.path.values]) 147 | test_labels = np.array([dict_age[a] for a in test_df.age.values]) 148 | 149 | for k in tqdm(range(bag)): 150 | 151 | 152 | tr_tr, tr_val = train_test_split(train_df, test_size=0.1) 153 | 154 | file_path = "baseline_age.h5" 155 | 156 | checkpoint = ModelCheckpoint(file_path, monitor='val_acc', verbose=1, save_best_only=True, mode='max') 157 | 158 | early = EarlyStopping(monitor="val_acc", mode="max", patience=10) 159 | 160 | reduce_on_plateau = ReduceLROnPlateau(monitor="val_acc", mode="max", factor=0.1, patience=3) 161 | 162 | callbacks_list = [checkpoint, early, reduce_on_plateau] # early 163 | 164 | model = get_model(n_classes=len(dict_age)) 165 | model.fit_generator(gen(tr_tr, aug=True), validation_data=gen(tr_val), epochs=200, verbose=2, workers=4, 166 | callbacks=callbacks_list, steps_per_epoch=50, validation_steps=30) 167 | 168 | model.load_weights(file_path) 169 | 170 | predictions += model.predict(test_images) 171 | cnt_ave += 1 172 | 173 | test_images = test_images[:, :, ::-1, :] 174 | 175 | predictions += model.predict(test_images) 176 | cnt_ave += 1 177 | 178 | K.clear_session() 179 | 180 | predictions = predictions/cnt_ave 181 | 182 | predictions = predictions.argmax(axis=-1) 183 | 184 | acc = accuracy_score(test_labels, predictions) 185 | 186 | print("accuracy : %s " %acc) 187 | 188 | accuracies.append(acc) 189 | 190 | print("mean acc : %s (%s) " % (np.mean(accuracies), np.std(accuracies))) --------------------------------------------------------------------------------