├── .gitignore ├── README.md ├── app.py ├── config.py ├── notebooks ├── DatagGeneration.ipynb ├── Real-time-prediction.ipynb └── SiameseNetwork-TripletLoss.ipynb ├── reqirement.txt ├── siameseNetwork.py ├── static ├── css │ └── style.css └── images │ └── favicon.ico ├── templates ├── index.html └── results.html └── utils.py /.gitignore: -------------------------------------------------------------------------------- 1 | logs/ 2 | venv/ 3 | data/ 4 | features/ 5 | __pycache__/ 6 | .ipynb_checkpoints/ 7 | *.zip -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Face Recognition Software 2 | 3 | I have explained the details about the model architecture, dataset, and other implementational details in the blog post [here](https://towardsdatascience.com/building-face-recognition-model-under-30-minutes-2d1b0ef72fda). Here, I will provide the steps for using this repository. 4 | 5 | # Installations 6 | 7 | The following steps are tested for ubuntu 20.04 with a python version of 3.8 8 | 9 | ``` 10 | sudo apt update 11 | sudo apt upgrade 12 | sudo snap install cmake 13 | ``` 14 | Clone the project to any directory and open the terminal in that directory. 15 | We will have to create some directories that will be essential for our storing images and features 16 | ``` 17 | mkdir data 18 | mkdir features 19 | ``` 20 | 21 | I generally like to create a virtual environment for each of my projects so the next step is optional. 22 | 23 | ``` 24 | virtualenv -p /usr/bin/python3.8 venv 25 | source venv/bin/activate 26 | ``` 27 | Now we will install all the pip packages required for running this applications. 28 | ``` 29 | pip install -r reqirement.txt 30 | ``` 31 | If you want to just try out the Browser-Based UI tool or run the notebook, then you can download the pre-trained model weights from [here](https://drive.google.com/file/d/1MegWliwXx2J-xHYX6iETl7hXUtLRk2sC/view?usp=sharing). 32 | ## Check your setup 33 | After extracting the files, your directory should look like this. 34 | ``` 35 | . 36 | ├── app.py 37 | ├── config.py 38 | ├── data 39 | ├── features 40 | ├── logs 41 | │   ├── func 42 | │   ├── model 43 | │   └── scalars 44 | ├── notebooks 45 | │   ├── DatagGeneration.ipynb 46 | │   ├── Real-time-prediction.ipynb 47 | │   └── SiameseNetwork-TripletLoss.ipynb 48 | ├── README.md 49 | ├── reqirement.txt 50 | ├── siameseNetwork.py 51 | ├── static 52 | │   ├── css 53 | │   └── images 54 | ├── templates 55 | │   ├── index.html 56 | │   └── results.html 57 | ├── utils.py 58 | ``` 59 | 60 | ## Running the browser-based tool 61 | 62 | If you have the same set of files and folders in the directory then you can run the following command 63 | ``` 64 | python app.py 65 | ``` 66 | The flask app will start and you will be able to collect training data for any new person and generate features for that person and check the real-time recognition. You can add as many people as you want to. The images collected from this tool will be added to the data folder and the corresponding features generated will be stored in the features folder. 67 | 68 | _**Note :** If you delete any person's images from the data folder, you need to delete the .pkl file inside the features folder as well. Also, the pickle file will be generated only when you hit submit images in the browser tool._ 69 | 70 | ## Running the Notebooks 71 | 72 | You can start the jupyter-lab or jupyter notebook server and check the notebooks folder. 73 | Notebook [Real-time-prediction.ipynb](https://github.com/dedhiaparth98/face-recognition/blob/master/notebooks/Real-time-prediction.ipynb) can be used for evaluating the model. It's the notebook version of the browser-based tool. However, the prediction in real-time webcam frame is much faster here as the browser sends API calls to the backend and each image frame of the video is send whereas here, it's not necessary. 74 | 75 | Other instructions for running this notebook are provided in the notebook itself. However, the data directory is shared between this notebook and the browser-based tool. 76 | 77 | ## Training from scratch 78 | 79 | If you wish to train your model and get your own weights, then you can use [SiameseNetwork-TripletLoss.ipynb](https://github.com/dedhiaparth98/face-recognition/blob/master/notebooks/SiameseNetwork-TripletLoss.ipynb). I had trained the same on colab and kept the lines of code for mounting the drive and other TensorFlow logging. Please refer to the blog post link above for learning more about the training details. 80 | 81 | ## For Web Developers - I have a question 82 | 83 | I have tried using socket connection as well as ajax calls for sending data to the backend while running prediction calls on the images. It was counter-intuitive to know that the socket connection was giving me a slower frame rate than the ajax calls. 84 | 85 | The current implementation is with Ajax call but the commented code for the socket is kept in both frontend and backend. **So if you know why is socket slow than ajax ?** then please drop me a message on [twitter](https://twitter.com/Parth_dedhia98) or [linkedin](https://www.linkedin.com/in/parth-dedhia). Thanks in advance !! 86 | 87 | ## References 88 | 89 | 1. O. M. Parkhi, A. Vedaldi, A. Zisserman, Deep Face Recognition, British Machine Vision Conference, 2015. 90 | 1. Q. Cao, L. Shen, W. Xie, O. M. Parkhi, A. Zisserman, VGGFace2: A dataset for recognising face across pose and age, International Conference on Automatic Face and Gesture Recognition, 2018. 91 | 3. F. Schroff, D. Kalenichenko, J. Philbin, FaceNet: A Unified Embedding for Face Recognition and Clustering, CVPR, 2015. 92 | 4. G. Koch, R. Zemel, R. Salakhutdinov, Siamese Neural Networks for One-shot Image Recognition, ICML deep learning workshop. Vol. 2. 2015. 93 | 5. [https://github.com/rcmalli/keras-vggface](https://github.com/rcmalli/keras-vggface) 94 | 6. [https://stanford.edu/~shervine/blog/keras-how-to-generate-data-on-the-fly](https://stanford.edu/~shervine/blog/keras-how-to-generate-data-on-the-fly) 95 | 7. [https://medium.com/datadriveninvestor/speed-up-your-image-training-on-google-colab-dc95ea1491cf](https://medium.com/datadriveninvestor/speed-up-your-image-training-on-google-colab-dc95ea1491cf) 96 | -------------------------------------------------------------------------------- /app.py: -------------------------------------------------------------------------------- 1 | import os 2 | import cv2 3 | import json 4 | import base64 5 | import config 6 | import numpy as np 7 | from utils import generate_dataset_festures, predict_people 8 | from flask import Flask, request, render_template, jsonify 9 | # from flask_socketio import SocketIO, emit 10 | 11 | app = Flask(__name__) 12 | # app.config['SECRET_KEY'] = 'secret!' 13 | # socketio = SocketIO(app) 14 | 15 | generate_dataset_festures() 16 | 17 | @app.route('/') 18 | def index(): 19 | return render_template('index.html') 20 | 21 | @app.route('/submit-name', methods=['POST']) 22 | def submit_name(): 23 | if request.method == 'POST': 24 | name = request.form['name'].lower() 25 | if name not in os.listdir(config.data_dir): 26 | return "SUCCESS" 27 | return "FAILURE" 28 | 29 | @app.route('/submit-photos', methods=['POST']) 30 | def submit_photos(): 31 | if request.method == 'POST': 32 | name = request.form['name'].lower() 33 | images = json.loads(request.form['images']) 34 | 35 | os.mkdir(os.path.join(config.data_dir, str(name))) 36 | 37 | person_directory = os.path.join(config.data_dir, name) 38 | for i, image in enumerate(images): 39 | image_numpy = np.fromstring(base64.b64decode(image.split(",")[1]), np.uint8) 40 | image = cv2.imdecode(image_numpy, cv2.IMREAD_COLOR) 41 | cv2.imwrite(os.path.join(person_directory, str(i) + '.png'), image) 42 | 43 | generate_dataset_festures() 44 | 45 | return "results" 46 | 47 | @app.route("/results") 48 | def results(): 49 | return render_template("results.html") 50 | 51 | @app.route("/predict-frame", methods=['POST']) 52 | def predict_frame(): 53 | if request.method == 'POST': 54 | image = request.form['image'] 55 | image_numpy = np.fromstring(base64.b64decode(image.split(",")[1]), np.uint8) 56 | image = cv2.imdecode(image_numpy, cv2.IMREAD_COLOR) 57 | 58 | image = predict_people(image) 59 | 60 | retval, buffer = cv2.imencode('.png', image) 61 | img_as_text = base64.b64encode(buffer) 62 | 63 | return img_as_text 64 | 65 | # I have tried out the prediction code with socket library as well as with AJAX calls, 66 | # and for my system locally the ajax calls work much faster than the socket connection. 67 | # Even I fell it's counter intuitive but still, its fast. 68 | 69 | # @socketio.on('connect', namespace='/socket-connection') 70 | # def socket_connection(): 71 | # emit('connection-response', {'data': 'Connected'}) 72 | 73 | # @socketio.on('prediction', namespace='/socket-connection') 74 | # def prediction_image(message): 75 | # image = message['data'] 76 | # image_numpy = np.fromstring(base64.b64decode(image.split(",")[1]), np.uint8) 77 | # image = cv2.imdecode(image_numpy, cv2.IMREAD_COLOR) 78 | 79 | # image = predict_people(image) 80 | 81 | # retval, buffer = cv2.imencode('.png', image) 82 | # img_as_text = base64.b64encode(buffer).decode('utf-8') 83 | 84 | # emit('prediction-response', {'img': img_as_text}) 85 | 86 | 87 | if __name__ == "__main__": 88 | # socketio.run(app, debug=False) 89 | app.run(debug=False) 90 | -------------------------------------------------------------------------------- /config.py: -------------------------------------------------------------------------------- 1 | data_dir = './data' 2 | feature_dir = './features' 3 | people = None 4 | features = None 5 | model = None 6 | face_detector = None -------------------------------------------------------------------------------- /notebooks/DatagGeneration.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import os\n", 10 | "import cv2\n", 11 | "import dlib\n", 12 | "import shutil\n", 13 | "import numpy as np\n", 14 | "from tqdm.notebook import tqdm\n", 15 | "from imutils import face_utils" 16 | ] 17 | }, 18 | { 19 | "cell_type": "code", 20 | "execution_count": null, 21 | "metadata": {}, 22 | "outputs": [], 23 | "source": [ 24 | "face_detector = dlib.get_frontal_face_detector()" 25 | ] 26 | }, 27 | { 28 | "cell_type": "code", 29 | "execution_count": null, 30 | "metadata": {}, 31 | "outputs": [], 32 | "source": [ 33 | "if os.path.isdir('./dataset'):\n", 34 | " shutil.rmtree('./dataset')\n", 35 | "os.mkdir('./dataset')\n", 36 | "os.mkdir('./dataset/images')\n", 37 | " \n", 38 | "dataset_path = './dataset/images'\n", 39 | "path = './vggface2_test/test'\n", 40 | "list_of_images = []\n", 41 | "for dirname in tqdm(os.listdir(path)):\n", 42 | " image_folder_path = os.path.join(path, dirname)\n", 43 | " os.mkdir(os.path.join(dataset_path, dirname))\n", 44 | " for image in tqdm(os.listdir(image_folder_path), leave=True, position=1):\n", 45 | " image_path = os.path.join(image_folder_path, image)\n", 46 | " img = cv2.imread(image_path)\n", 47 | " gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n", 48 | " faces = face_detector(gray, 0)\n", 49 | " if len(faces) == 1:\n", 50 | " for face in faces:\n", 51 | " face_bounding_box = face_utils.rect_to_bb(face)\n", 52 | " if all(i >= 0 for i in face_bounding_box):\n", 53 | " [x, y, w, h] = face_bounding_box\n", 54 | " frame = img[y:y + h, x:x + w]\n", 55 | " save_image = os.path.join(os.path.join(dataset_path, dirname), image)\n", 56 | " cv2.imwrite(save_image, frame)\n", 57 | " list_of_images.append(dirname + '/' + image)" 58 | ] 59 | }, 60 | { 61 | "cell_type": "code", 62 | "execution_count": null, 63 | "metadata": {}, 64 | "outputs": [], 65 | "source": [ 66 | "with open('./dataset/list.txt', 'w') as f:\n", 67 | " for item in list_of_images:\n", 68 | " f.write(\"%s\\n\" % item)" 69 | ] 70 | }, 71 | { 72 | "cell_type": "code", 73 | "execution_count": null, 74 | "metadata": {}, 75 | "outputs": [], 76 | "source": [] 77 | } 78 | ], 79 | "metadata": { 80 | "kernelspec": { 81 | "display_name": "Python 3", 82 | "language": "python", 83 | "name": "python3" 84 | }, 85 | "language_info": { 86 | "codemirror_mode": { 87 | "name": "ipython", 88 | "version": 3 89 | }, 90 | "file_extension": ".py", 91 | "mimetype": "text/x-python", 92 | "name": "python", 93 | "nbconvert_exporter": "python", 94 | "pygments_lexer": "ipython3", 95 | "version": "3.8.2" 96 | } 97 | }, 98 | "nbformat": 4, 99 | "nbformat_minor": 4 100 | } 101 | -------------------------------------------------------------------------------- /notebooks/Real-time-prediction.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import os\n", 10 | "import cv2\n", 11 | "import dlib\n", 12 | "import pickle\n", 13 | "import random\n", 14 | "import numpy as np\n", 15 | "from tqdm import tqdm\n", 16 | "import tensorflow as tf\n", 17 | "from datetime import datetime\n", 18 | "from imutils import face_utils" 19 | ] 20 | }, 21 | { 22 | "cell_type": "code", 23 | "execution_count": null, 24 | "metadata": {}, 25 | "outputs": [], 26 | "source": [ 27 | "base_dir = \".\"\n", 28 | "checkpoint_path = os.path.join(base_dir, '../logs/model/siamese-1')" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": null, 34 | "metadata": {}, 35 | "outputs": [], 36 | "source": [ 37 | "'''\n", 38 | "The following code cell is taken from the source code of keras_vggface.'\n", 39 | "I tried using the preprocess_input function provided by tf.keras but they provide different results.\n", 40 | "To my knowledge, it seems that the mean values which are subtracted in each image are different.\n", 41 | "'''\n", 42 | "K = tf.keras.backend\n", 43 | "\n", 44 | "def preprocess_input(x, data_format=None, version=1):\n", 45 | " x_temp = np.copy(x)\n", 46 | " if data_format is None:\n", 47 | " data_format = K.image_data_format()\n", 48 | " assert data_format in {'channels_last', 'channels_first'}\n", 49 | "\n", 50 | " if version == 1:\n", 51 | " if data_format == 'channels_first':\n", 52 | " x_temp = x_temp[:, ::-1, ...]\n", 53 | " x_temp[:, 0, :, :] -= 93.5940\n", 54 | " x_temp[:, 1, :, :] -= 104.7624\n", 55 | " x_temp[:, 2, :, :] -= 129.1863\n", 56 | " else:\n", 57 | " x_temp = x_temp[..., ::-1]\n", 58 | " x_temp[..., 0] -= 93.5940\n", 59 | " x_temp[..., 1] -= 104.7624\n", 60 | " x_temp[..., 2] -= 129.1863\n", 61 | "\n", 62 | " elif version == 2:\n", 63 | " if data_format == 'channels_first':\n", 64 | " x_temp = x_temp[:, ::-1, ...]\n", 65 | " x_temp[:, 0, :, :] -= 91.4953\n", 66 | " x_temp[:, 1, :, :] -= 103.8827\n", 67 | " x_temp[:, 2, :, :] -= 131.0912\n", 68 | " else:\n", 69 | " x_temp = x_temp[..., ::-1]\n", 70 | " x_temp[..., 0] -= 91.4953\n", 71 | " x_temp[..., 1] -= 103.8827\n", 72 | " x_temp[..., 2] -= 131.0912\n", 73 | " else:\n", 74 | " raise NotImplementedError\n", 75 | "\n", 76 | " return x_temp" 77 | ] 78 | }, 79 | { 80 | "cell_type": "code", 81 | "execution_count": null, 82 | "metadata": {}, 83 | "outputs": [], 84 | "source": [ 85 | "vggface = tf.keras.models.Sequential()\n", 86 | "vggface.add(tf.keras.layers.Convolution2D(64, (3, 3), activation='relu', padding=\"SAME\", input_shape=(224,224, 3)))\n", 87 | "vggface.add(tf.keras.layers.Convolution2D(64, (3, 3), activation='relu', padding=\"SAME\"))\n", 88 | "vggface.add(tf.keras.layers.MaxPooling2D((2,2), strides=(2,2)))\n", 89 | " \n", 90 | "vggface.add(tf.keras.layers.Convolution2D(128, (3, 3), activation='relu', padding=\"SAME\"))\n", 91 | "vggface.add(tf.keras.layers.Convolution2D(128, (3, 3), activation='relu', padding=\"SAME\"))\n", 92 | "vggface.add(tf.keras.layers.MaxPooling2D((2,2), strides=(2,2)))\n", 93 | " \n", 94 | "vggface.add(tf.keras.layers.Convolution2D(256, (3, 3), activation='relu', padding=\"SAME\"))\n", 95 | "vggface.add(tf.keras.layers.Convolution2D(256, (3, 3), activation='relu', padding=\"SAME\"))\n", 96 | "vggface.add(tf.keras.layers.Convolution2D(256, (3, 3), activation='relu', padding=\"SAME\"))\n", 97 | "vggface.add(tf.keras.layers.MaxPooling2D((2,2), strides=(2,2)))\n", 98 | " \n", 99 | "vggface.add(tf.keras.layers.Convolution2D(512, (3, 3), activation='relu', padding=\"SAME\"))\n", 100 | "vggface.add(tf.keras.layers.Convolution2D(512, (3, 3), activation='relu', padding=\"SAME\"))\n", 101 | "vggface.add(tf.keras.layers.Convolution2D(512, (3, 3), activation='relu', padding=\"SAME\"))\n", 102 | "vggface.add(tf.keras.layers.MaxPooling2D((2,2), strides=(2,2)))\n", 103 | " \n", 104 | "vggface.add(tf.keras.layers.Convolution2D(512, (3, 3), activation='relu', padding=\"SAME\"))\n", 105 | "vggface.add(tf.keras.layers.Convolution2D(512, (3, 3), activation='relu', padding=\"SAME\"))\n", 106 | "vggface.add(tf.keras.layers.Convolution2D(512, (3, 3), activation='relu', padding=\"SAME\"))\n", 107 | "vggface.add(tf.keras.layers.MaxPooling2D((2,2), strides=(2,2)))\n", 108 | "\n", 109 | "vggface.add(tf.keras.layers.Flatten())\n", 110 | "\n", 111 | "vggface.add(tf.keras.layers.Dense(4096, activation='relu'))\n", 112 | "vggface.add(tf.keras.layers.Dropout(0.5))\n", 113 | "vggface.add(tf.keras.layers.Dense(4096, activation='relu'))\n", 114 | "vggface.add(tf.keras.layers.Dropout(0.5))\n", 115 | "vggface.add(tf.keras.layers.Dense(2622, activation='softmax'))\n", 116 | "\n", 117 | "vggface.pop()\n", 118 | "vggface.add(tf.keras.layers.Dense(128, use_bias=False))\n", 119 | "\n", 120 | "for layer in vggface.layers[:-2]:\n", 121 | " layer.trainable = False" 122 | ] 123 | }, 124 | { 125 | "cell_type": "code", 126 | "execution_count": null, 127 | "metadata": {}, 128 | "outputs": [], 129 | "source": [ 130 | "class SiameseNetwork(tf.keras.Model):\n", 131 | " def __init__(self, vgg_face):\n", 132 | " super(SiameseNetwork, self).__init__()\n", 133 | " self.vgg_face = vgg_face\n", 134 | " \n", 135 | " @tf.function\n", 136 | " def call(self, inputs):\n", 137 | " image_1, image_2, image_3 = inputs\n", 138 | " with tf.name_scope(\"Anchor\") as scope:\n", 139 | " feature_1 = self.vgg_face(image_1)\n", 140 | " feature_1 = tf.math.l2_normalize(feature_1, axis=-1)\n", 141 | " with tf.name_scope(\"Positive\") as scope:\n", 142 | " feature_2 = self.vgg_face(image_2)\n", 143 | " feature_2 = tf.math.l2_normalize(feature_2, axis=-1)\n", 144 | " with tf.name_scope(\"Negative\") as scope:\n", 145 | " feature_3 = self.vgg_face(image_3)\n", 146 | " feature_3 = tf.math.l2_normalize(feature_3, axis=-1)\n", 147 | " return [feature_1, feature_2, feature_3]\n", 148 | " \n", 149 | " @tf.function\n", 150 | " def get_features(self, inputs):\n", 151 | " return tf.math.l2_normalize(self.vgg_face(inputs), axis=-1)" 152 | ] 153 | }, 154 | { 155 | "cell_type": "code", 156 | "execution_count": null, 157 | "metadata": {}, 158 | "outputs": [], 159 | "source": [ 160 | "model = SiameseNetwork(vggface)" 161 | ] 162 | }, 163 | { 164 | "cell_type": "code", 165 | "execution_count": null, 166 | "metadata": {}, 167 | "outputs": [], 168 | "source": [ 169 | "_ = model([tf.zeros((32,224,224,3)), tf.zeros((32,224,224,3)), tf.zeros((32,224,224,3))])\n", 170 | "_ = model.get_features(tf.zeros((32,224,224,3)))" 171 | ] 172 | }, 173 | { 174 | "cell_type": "code", 175 | "execution_count": null, 176 | "metadata": {}, 177 | "outputs": [], 178 | "source": [ 179 | "checkpoint = tf.train.Checkpoint(model=model)\n", 180 | "checkpoint.restore(checkpoint_path)" 181 | ] 182 | }, 183 | { 184 | "cell_type": "code", 185 | "execution_count": null, 186 | "metadata": {}, 187 | "outputs": [], 188 | "source": [ 189 | "data_dir = '../data/'" 190 | ] 191 | }, 192 | { 193 | "cell_type": "markdown", 194 | "metadata": {}, 195 | "source": [ 196 | "# Data Collection Details\n", 197 | "The cell below should be re-run for data collection for multiple people.\n", 198 | "For a single person, you could collect 8-10 images tops. Entering the name of the person would add a directory to the data folder which will be the name of that person. \n", 199 | "If the name of the person exists it will give an error. So make sure to keep different names." 200 | ] 201 | }, 202 | { 203 | "cell_type": "code", 204 | "execution_count": null, 205 | "metadata": {}, 206 | "outputs": [], 207 | "source": [ 208 | "name = input(\"Enter the name of the person : \")\n", 209 | "os.mkdir(os.path.join(data_dir, name))\n", 210 | "cap = cv2.VideoCapture(0)\n", 211 | "count = 0\n", 212 | "while True:\n", 213 | " ret, frame = cap.read()\n", 214 | " cv2.imshow('Image', frame)\n", 215 | " k = cv2.waitKey(1)\n", 216 | " if k == ord('s'):\n", 217 | " cv2.imwrite(os.path.join(data_dir, name + '/' + str(count) + '.png') , frame)\n", 218 | " count += 1\n", 219 | " if k ==ord('q'):\n", 220 | " break\n", 221 | "cap.release()\n", 222 | "cv2.destroyAllWindows()" 223 | ] 224 | }, 225 | { 226 | "cell_type": "code", 227 | "execution_count": null, 228 | "metadata": {}, 229 | "outputs": [], 230 | "source": [ 231 | "features = []" 232 | ] 233 | }, 234 | { 235 | "cell_type": "code", 236 | "execution_count": null, 237 | "metadata": {}, 238 | "outputs": [], 239 | "source": [ 240 | "people = sorted(os.listdir(data_dir))\n", 241 | "face_detector = dlib.get_frontal_face_detector()\n", 242 | "features = []\n", 243 | "dumpable_features = {}" 244 | ] 245 | }, 246 | { 247 | "cell_type": "code", 248 | "execution_count": null, 249 | "metadata": {}, 250 | "outputs": [], 251 | "source": [ 252 | "for person in people:\n", 253 | " person_path = os.path.join(data_dir, person)\n", 254 | " print(person_path)\n", 255 | " images = []\n", 256 | " for image in os.listdir(person_path):\n", 257 | " image_path = os.path.join(person_path, image)\n", 258 | " img = cv2.imread(image_path)\n", 259 | " gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n", 260 | " faces = face_detector(gray, 0)\n", 261 | " if len(faces) == 1:\n", 262 | " for face in faces:\n", 263 | " face_bounding_box = face_utils.rect_to_bb(face)\n", 264 | " if all(i >= 0 for i in face_bounding_box):\n", 265 | " [x, y, w, h] = face_bounding_box\n", 266 | " frame = img[y:y + h, x:x + w]\n", 267 | " frame = cv2.resize(frame, (224, 224))\n", 268 | " frame = np.asarray(frame, dtype=np.float64)\n", 269 | " images.append(frame)\n", 270 | " images = np.asarray(images)\n", 271 | " images = preprocess_input(images)\n", 272 | " images = tf.convert_to_tensor(images)\n", 273 | " feature = model.get_features(images)\n", 274 | " feature = tf.reduce_mean(feature, axis=0)\n", 275 | " features.append(feature.numpy())\n", 276 | " dumpable_features[person] = feature.numpy()" 277 | ] 278 | }, 279 | { 280 | "cell_type": "markdown", 281 | "metadata": {}, 282 | "source": [ 283 | "# Dumping Features Rather than Calculating Them every time\n", 284 | "\n", 285 | "I have added a variable dumpable_features which could be pickled. It will save a dictionary format of {'name':'features (numpy array)'} for each person. Simply pickle using the following commands\n", 286 | "```\n", 287 | "with open('weigths.pkl', 'wb') as f:\n", 288 | " pickle.dump(dumpable_features, f) \n", 289 | "```\n", 290 | "and reload them using\n", 291 | "```\n", 292 | "with open('weigths.pkl', 'rb') as f:\n", 293 | " dumpable_features_reloaded = pickle.load(f)\n", 294 | " \n", 295 | "people = []\n", 296 | "features = []\n", 297 | "for key, value in dumpable_features_reloaded.items():\n", 298 | " people.append(key)\n", 299 | " features.append(value)\n", 300 | "```\n", 301 | "During my testing I tested on 8-10 people so regenerating features wasn't that much slow. " 302 | ] 303 | }, 304 | { 305 | "cell_type": "code", 306 | "execution_count": null, 307 | "metadata": {}, 308 | "outputs": [], 309 | "source": [ 310 | "features = np.asarray(features)" 311 | ] 312 | }, 313 | { 314 | "cell_type": "code", 315 | "execution_count": null, 316 | "metadata": {}, 317 | "outputs": [], 318 | "source": [ 319 | "print(people)" 320 | ] 321 | }, 322 | { 323 | "cell_type": "code", 324 | "execution_count": null, 325 | "metadata": {}, 326 | "outputs": [], 327 | "source": [ 328 | "cap = cv2.VideoCapture(0)\n", 329 | "count = 0\n", 330 | "name = 'not identified'\n", 331 | "while True:\n", 332 | " ret, img = cap.read()\n", 333 | " gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n", 334 | " \n", 335 | " faces = face_detector(gray, 0)\n", 336 | " for face in faces:\n", 337 | " face_bounding_box = face_utils.rect_to_bb(face)\n", 338 | " if all(i >= 0 for i in face_bounding_box):\n", 339 | " [x, y, w, h] = face_bounding_box\n", 340 | " frame = img[y:y + h, x:x + w]\n", 341 | " cv2.rectangle(img, (x, y), (x + w, y + h), (0, 255, 0), 2)\n", 342 | " frame = cv2.resize(frame, (224, 224))\n", 343 | " frame = np.asarray(frame, dtype=np.float64)\n", 344 | " frame = np.expand_dims(frame, axis=0)\n", 345 | " frame = preprocess_input(frame)\n", 346 | " feature = model.get_features(frame)\n", 347 | " \n", 348 | " dist = tf.norm(features - feature, axis=1)\n", 349 | " name = 'not identified'\n", 350 | " loc = tf.argmin(dist)\n", 351 | " if dist[loc] < 0.8:\n", 352 | " name = people[loc]\n", 353 | " else:\n", 354 | "# print(dist.numpy())\n", 355 | " pass\n", 356 | " \n", 357 | " font_face = cv2.FONT_HERSHEY_SIMPLEX\n", 358 | " cv2.putText(img, name, (x, y-5), font_face, 0.8, (0,0,255), 3)\n", 359 | " cv2.imshow('Image', img)\n", 360 | " k = cv2.waitKey(1)\n", 361 | " if k ==ord('q'):\n", 362 | " break\n", 363 | "cap.release()\n", 364 | "cv2.destroyAllWindows()" 365 | ] 366 | }, 367 | { 368 | "cell_type": "code", 369 | "execution_count": null, 370 | "metadata": {}, 371 | "outputs": [], 372 | "source": [] 373 | } 374 | ], 375 | "metadata": { 376 | "kernelspec": { 377 | "display_name": "Python 3", 378 | "language": "python", 379 | "name": "python3" 380 | }, 381 | "language_info": { 382 | "codemirror_mode": { 383 | "name": "ipython", 384 | "version": 3 385 | }, 386 | "file_extension": ".py", 387 | "mimetype": "text/x-python", 388 | "name": "python", 389 | "nbconvert_exporter": "python", 390 | "pygments_lexer": "ipython3", 391 | "version": "3.8.2" 392 | } 393 | }, 394 | "nbformat": 4, 395 | "nbformat_minor": 4 396 | } 397 | -------------------------------------------------------------------------------- /reqirement.txt: -------------------------------------------------------------------------------- 1 | absl-py==0.10.0 2 | appdirs==1.4.3 3 | argon2-cffi==20.1.0 4 | astunparse==1.6.3 5 | async-generator==1.10 6 | attrs==20.2.0 7 | backcall==0.2.0 8 | bleach==3.2.1 9 | CacheControl==0.12.6 10 | cachetools==4.1.1 11 | certifi==2019.11.28 12 | cffi==1.14.3 13 | chardet==3.0.4 14 | click==7.1.2 15 | colorama==0.4.3 16 | contextlib2==0.6.0 17 | decorator==4.4.2 18 | defusedxml==0.6.0 19 | distlib==0.3.0 20 | distro==1.4.0 21 | dlib==19.21.0 22 | entrypoints==0.3 23 | Flask==1.1.2 24 | Flask-SocketIO==4.3.1 25 | gast==0.3.3 26 | gevent==20.9.0 27 | gevent-websocket==0.10.1 28 | google-auth==1.21.3 29 | google-auth-oauthlib==0.4.1 30 | google-pasta==0.2.0 31 | greenlet==0.4.17 32 | grpcio==1.32.0 33 | h5py==2.10.0 34 | html5lib==1.0.1 35 | idna==2.8 36 | imutils==0.5.3 37 | ipaddr==2.2.0 38 | ipykernel==5.3.4 39 | ipython==7.18.1 40 | ipython-genutils==0.2.0 41 | itsdangerous==1.1.0 42 | jedi==0.17.2 43 | Jinja2==2.11.2 44 | json5==0.9.5 45 | jsonschema==3.2.0 46 | jupyter-client==6.1.7 47 | jupyter-core==4.6.3 48 | jupyterlab==2.2.8 49 | jupyterlab-pygments==0.1.1 50 | jupyterlab-server==1.2.0 51 | Keras-Preprocessing==1.1.2 52 | lockfile==0.12.2 53 | Markdown==3.2.2 54 | MarkupSafe==1.1.1 55 | mistune==0.8.4 56 | msgpack==0.6.2 57 | nbclient==0.5.0 58 | nbconvert==6.0.6 59 | nbformat==5.0.7 60 | nest-asyncio==1.4.0 61 | notebook==6.1.4 62 | numpy==1.18.5 63 | oauthlib==3.1.0 64 | opencv-contrib-python==4.4.0.44 65 | opencv-python==4.4.0.44 66 | opt-einsum==3.3.0 67 | packaging==20.3 68 | pandocfilters==1.4.2 69 | parso==0.7.1 70 | pep517==0.8.2 71 | pexpect==4.8.0 72 | pickleshare==0.7.5 73 | progress==1.5 74 | prometheus-client==0.8.0 75 | prompt-toolkit==3.0.7 76 | protobuf==3.13.0 77 | ptyprocess==0.6.0 78 | pyasn1==0.4.8 79 | pyasn1-modules==0.2.8 80 | pycparser==2.20 81 | Pygments==2.7.1 82 | pyparsing==2.4.6 83 | pyrsistent==0.17.3 84 | python-dateutil==2.8.1 85 | python-engineio==3.13.2 86 | python-socketio==4.6.0 87 | pytoml==0.1.21 88 | pyzmq==19.0.2 89 | requests==2.22.0 90 | requests-oauthlib==1.3.0 91 | retrying==1.3.3 92 | rsa==4.6 93 | scipy==1.4.1 94 | Send2Trash==1.5.0 95 | six==1.14.0 96 | tensorboard==2.3.0 97 | tensorboard-plugin-wit==1.7.0 98 | tensorflow==2.3.0 99 | tensorflow-estimator==2.3.0 100 | termcolor==1.1.0 101 | terminado==0.9.1 102 | testpath==0.4.4 103 | tornado==6.0.4 104 | tqdm==4.49.0 105 | traitlets==5.0.4 106 | urllib3==1.25.8 107 | wcwidth==0.2.5 108 | webencodings==0.5.1 109 | Werkzeug==1.0.1 110 | wrapt==1.12.1 111 | zope.event==4.5.0 112 | zope.interface==5.1.0 113 | -------------------------------------------------------------------------------- /siameseNetwork.py: -------------------------------------------------------------------------------- 1 | import os 2 | import numpy as np 3 | import tensorflow as tf 4 | 5 | 6 | K = tf.keras.backend 7 | 8 | def preprocess_input(x, data_format=None, version=1): 9 | x_temp = np.copy(x) 10 | if data_format is None: 11 | data_format = K.image_data_format() 12 | assert data_format in {'channels_last', 'channels_first'} 13 | 14 | if version == 1: 15 | if data_format == 'channels_first': 16 | x_temp = x_temp[:, ::-1, ...] 17 | x_temp[:, 0, :, :] -= 93.5940 18 | x_temp[:, 1, :, :] -= 104.7624 19 | x_temp[:, 2, :, :] -= 129.1863 20 | else: 21 | x_temp = x_temp[..., ::-1] 22 | x_temp[..., 0] -= 93.5940 23 | x_temp[..., 1] -= 104.7624 24 | x_temp[..., 2] -= 129.1863 25 | 26 | elif version == 2: 27 | if data_format == 'channels_first': 28 | x_temp = x_temp[:, ::-1, ...] 29 | x_temp[:, 0, :, :] -= 91.4953 30 | x_temp[:, 1, :, :] -= 103.8827 31 | x_temp[:, 2, :, :] -= 131.0912 32 | else: 33 | x_temp = x_temp[..., ::-1] 34 | x_temp[..., 0] -= 91.4953 35 | x_temp[..., 1] -= 103.8827 36 | x_temp[..., 2] -= 131.0912 37 | else: 38 | raise NotImplementedError 39 | 40 | return x_temp 41 | 42 | class SiameseNetwork(tf.keras.Model): 43 | def __init__(self, vgg_face): 44 | super(SiameseNetwork, self).__init__() 45 | self.vgg_face = vgg_face 46 | 47 | @tf.function 48 | def call(self, inputs): 49 | image_1, image_2, image_3 = inputs 50 | with tf.name_scope("Anchor") as scope: 51 | feature_1 = self.vgg_face(image_1) 52 | feature_1 = tf.math.l2_normalize(feature_1, axis=-1) 53 | with tf.name_scope("Positive") as scope: 54 | feature_2 = self.vgg_face(image_2) 55 | feature_2 = tf.math.l2_normalize(feature_2, axis=-1) 56 | with tf.name_scope("Negative") as scope: 57 | feature_3 = self.vgg_face(image_3) 58 | feature_3 = tf.math.l2_normalize(feature_3, axis=-1) 59 | return [feature_1, feature_2, feature_3] 60 | 61 | @tf.function 62 | def get_features(self, inputs): 63 | return tf.math.l2_normalize(self.vgg_face(inputs), axis=-1) 64 | 65 | def get_siamese_model(): 66 | vggface = tf.keras.models.Sequential() 67 | vggface.add(tf.keras.layers.Convolution2D(64, (3, 3), activation='relu', padding="SAME", input_shape=(224,224, 3))) 68 | vggface.add(tf.keras.layers.Convolution2D(64, (3, 3), activation='relu', padding="SAME")) 69 | vggface.add(tf.keras.layers.MaxPooling2D((2,2), strides=(2,2))) 70 | 71 | vggface.add(tf.keras.layers.Convolution2D(128, (3, 3), activation='relu', padding="SAME")) 72 | vggface.add(tf.keras.layers.Convolution2D(128, (3, 3), activation='relu', padding="SAME")) 73 | vggface.add(tf.keras.layers.MaxPooling2D((2,2), strides=(2,2))) 74 | 75 | vggface.add(tf.keras.layers.Convolution2D(256, (3, 3), activation='relu', padding="SAME")) 76 | vggface.add(tf.keras.layers.Convolution2D(256, (3, 3), activation='relu', padding="SAME")) 77 | vggface.add(tf.keras.layers.Convolution2D(256, (3, 3), activation='relu', padding="SAME")) 78 | vggface.add(tf.keras.layers.MaxPooling2D((2,2), strides=(2,2))) 79 | 80 | vggface.add(tf.keras.layers.Convolution2D(512, (3, 3), activation='relu', padding="SAME")) 81 | vggface.add(tf.keras.layers.Convolution2D(512, (3, 3), activation='relu', padding="SAME")) 82 | vggface.add(tf.keras.layers.Convolution2D(512, (3, 3), activation='relu', padding="SAME")) 83 | vggface.add(tf.keras.layers.MaxPooling2D((2,2), strides=(2,2))) 84 | 85 | vggface.add(tf.keras.layers.Convolution2D(512, (3, 3), activation='relu', padding="SAME")) 86 | vggface.add(tf.keras.layers.Convolution2D(512, (3, 3), activation='relu', padding="SAME")) 87 | vggface.add(tf.keras.layers.Convolution2D(512, (3, 3), activation='relu', padding="SAME")) 88 | vggface.add(tf.keras.layers.MaxPooling2D((2,2), strides=(2,2))) 89 | 90 | vggface.add(tf.keras.layers.Flatten()) 91 | 92 | vggface.add(tf.keras.layers.Dense(4096, activation='relu')) 93 | vggface.add(tf.keras.layers.Dropout(0.5)) 94 | vggface.add(tf.keras.layers.Dense(4096, activation='relu')) 95 | vggface.add(tf.keras.layers.Dropout(0.5)) 96 | vggface.add(tf.keras.layers.Dense(2622, activation='softmax')) 97 | 98 | vggface.pop() 99 | vggface.add(tf.keras.layers.Dense(128, use_bias=False)) 100 | 101 | for layer in vggface.layers[:-2]: 102 | layer.trainable = False 103 | 104 | model = SiameseNetwork(vggface) 105 | 106 | base_dir = "." 107 | checkpoint_path = os.path.join(base_dir, 'logs/model/siamese-1') 108 | 109 | _ = model([tf.zeros((1,224,224,3)), tf.zeros((1,224,224,3)), tf.zeros((1,224,224,3))]) 110 | _ = model.get_features(tf.zeros((1,224,224,3))) 111 | 112 | checkpoint = tf.train.Checkpoint(model=model) 113 | checkpoint.restore(checkpoint_path) 114 | 115 | return model -------------------------------------------------------------------------------- /static/css/style.css: -------------------------------------------------------------------------------- 1 | * { 2 | box-sizing: border-box; 3 | } 4 | 5 | #header { 6 | text-align: center; 7 | margin: 0px; 8 | padding: 10px; 9 | background-color: rgb(88, 88, 88); 10 | color: white; 11 | } 12 | 13 | #header p { 14 | display: inline-block; 15 | width: 50%; 16 | margin: 10px auto; 17 | } 18 | 19 | #sticky-note { 20 | margin: 10px auto; 21 | text-align: center; 22 | width: 50%; 23 | /* background-color: rgb(193, 255, 193); 24 | border: 2px solid rgb(94, 167, 94); 25 | border-radius: 8px; */ 26 | } 27 | 28 | #sticky-note p{ 29 | margin: 10px; 30 | } 31 | 32 | #enter-name { 33 | margin: 20px auto; 34 | text-align: center; 35 | width: 60%; 36 | } 37 | 38 | #enter-name #input-name-div { 39 | display: inline-block; 40 | width: 40%; 41 | } 42 | 43 | #enter-name #button-name-div { 44 | display: inline-block; 45 | width: 20%; 46 | } 47 | 48 | #name-of-person { 49 | padding: 5px 10px; 50 | border-radius: 8px; 51 | } 52 | 53 | #name-of-person:read-only { 54 | background-color: rgb(180, 180, 248); 55 | } 56 | 57 | #submit-name-of-person { 58 | padding: 5px 10px; 59 | border-radius: 8px; 60 | background-color: rgb(159, 215, 252); 61 | border: 2px solid rgb(39, 169, 255); 62 | transition: 0.2s; 63 | } 64 | 65 | #submit-name-of-person:hover { 66 | background-color: rgb(200, 233, 255); 67 | box-shadow: 3px 3px 3px gray; 68 | } 69 | 70 | #submit-name-of-person:disabled { 71 | background-color: rgba(73, 99, 116, 0.1); 72 | border: 2px solid rgba(26, 35, 41, 0.1); 73 | } 74 | 75 | #change-name-of-person { 76 | display: none; 77 | } 78 | 79 | #instructions { 80 | display: none; 81 | justify-content: space-around; 82 | flex-direction: row; 83 | flex-flow: row; 84 | flex-wrap: wrap; 85 | align-content: center; 86 | } 87 | 88 | #capture-instructions { 89 | margin: 15px; 90 | text-align: justify; 91 | width: 30%; 92 | } 93 | 94 | #image-count-instruction { 95 | margin: 15px; 96 | text-align: center; 97 | width: 30%; 98 | align-self: center; 99 | } 100 | 101 | #submit-all-images-button { 102 | padding: 5px 10px; 103 | border-radius: 8px; 104 | background-color: rgb(159, 215, 252); 105 | border: 2px solid rgb(39, 169, 255); 106 | transition: 0.2s; 107 | } 108 | 109 | #submit-all-images-button:hover { 110 | background-color: rgb(200, 233, 255); 111 | box-shadow: 3px 3px 3px gray; 112 | } 113 | 114 | #video-photo { 115 | display: none; 116 | /* background-color: blue; */ 117 | flex-direction: row; 118 | flex-flow: row; 119 | flex-wrap: wrap; 120 | justify-content: space-around; 121 | } 122 | 123 | #video-div { 124 | height: 480px; 125 | width: 640px; 126 | position: relative; 127 | /* background-color: white; */ 128 | margin: 15px; 129 | } 130 | 131 | #video-div #take-photo-button { 132 | position: absolute; 133 | bottom: 2%; 134 | left: 50%; 135 | transform: translate(-50%, -50%); 136 | z-index: 1; 137 | background-color: rgb(159, 215, 252); 138 | border: 2px solid rgb(39, 169, 255); 139 | border-radius: 8px; 140 | padding: 2px 5px; 141 | } 142 | 143 | #webcam-video { 144 | height: 480px; 145 | width: 640px; 146 | border-radius: 5px; 147 | } 148 | 149 | #video-div #take-photo-button:hover { 150 | background-color: rgb(200, 233, 255); 151 | } 152 | 153 | #webcam-canvas { 154 | display: none; 155 | height: 480px; 156 | width: 640px; 157 | } 158 | 159 | #photo-div { 160 | height: 480px; 161 | min-width: 640px; 162 | /* background-color: white; */ 163 | position: relative; 164 | margin: 15px; 165 | } 166 | 167 | .captured-image { 168 | position: absolute; 169 | height: 480; 170 | width: 640; 171 | border: 2px solid black; 172 | border-radius: 5px; 173 | transition: 0.5s; 174 | } 175 | 176 | .captured-image:hover { 177 | z-index: 1000 !important; 178 | transform: translate(-10%, 0); 179 | } 180 | 181 | #image-modal-div { 182 | display: none; 183 | position: fixed; 184 | background-color: rgb(0, 0, 0, 0.9); 185 | top: 0; 186 | left: 0; 187 | height: 100%; 188 | width: 100%; 189 | z-index: 5000; 190 | } 191 | 192 | #close-image-modal { 193 | position: absolute; 194 | top: 15px; 195 | right: 40px; 196 | font-size: 40px; 197 | color: white; 198 | font-weight: bold; 199 | transition: 0.3s; 200 | } 201 | 202 | #close-image-modal:hover { 203 | color: rgb(182, 182, 182); 204 | } 205 | 206 | #delete-modal-image-button { 207 | display: block; 208 | margin: auto; 209 | margin-top: 15px; 210 | padding: 5px 10px; 211 | border-radius: 8px; 212 | background-color: rgb(159, 215, 252); 213 | border: 2px solid rgb(39, 169, 255); 214 | transition: 0.2s; 215 | } 216 | 217 | #delete-modal-image-button:hover { 218 | background-color: rgb(200, 233, 255); 219 | } 220 | 221 | #image-modal-img { 222 | margin: auto; 223 | display: block; 224 | height: 480px; 225 | width: 640px; 226 | border-radius: 5px; 227 | margin-top: 100px; 228 | } 229 | 230 | #spinner-div { 231 | display: none; 232 | position: fixed; 233 | background-color: rgb(0, 0, 0, 0.9); 234 | top: 0; 235 | left: 0; 236 | height: 100%; 237 | width: 100%; 238 | z-index: 5000; 239 | } 240 | 241 | #spinner { 242 | border: 10px solid rgb(78, 78, 78); 243 | margin: auto; 244 | margin-top: 300px; 245 | border-top-color: white; 246 | border-radius: 50%; 247 | height: 100px; 248 | width: 100px; 249 | animation: spin 2s linear infinite; 250 | } 251 | 252 | #spinner-text { 253 | margin: 20px auto; 254 | text-align: center; 255 | color: white; 256 | font-size: 200%; 257 | } 258 | 259 | @keyframes spin { 260 | 0% { transform: rotate(0deg); } 261 | 100% { transform: rotate(360deg); } 262 | } 263 | 264 | #prediction-video-stream { 265 | display: none; 266 | height: 480px; 267 | width: 640px; 268 | } 269 | 270 | #video-prediction{ 271 | /* background-color: green; */ 272 | margin: 30px auto; 273 | width: 640px; 274 | height: 480px; 275 | } 276 | 277 | #prediction-result-image { 278 | height: 480px; 279 | width: 640px; 280 | border-radius: 5px; 281 | } 282 | 283 | @media only screen and (max-width: 900px){ 284 | #enter-name #button-name-div, #enter-name #input-name-div{ 285 | display: block; 286 | width: 100%; 287 | margin-top: 10px; 288 | } 289 | 290 | #capture-instructions, #image-count-instruction { 291 | width: 100%; 292 | } 293 | 294 | } -------------------------------------------------------------------------------- /static/images/favicon.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dedhiaparth98/face-recognition/5d320e91119dfbdabeedcc824453149b9999c1b6/static/images/favicon.ico -------------------------------------------------------------------------------- /templates/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | Face Recognition 18 | 19 | 20 | 28 | 29 |
30 |

If you have already added the member. Click here to skip this step

31 |
32 | 33 |
34 |
35 | 36 | 37 |
38 |
39 | 40 |

Refresh for new submission

41 |
42 |
43 | 44 |
45 | 46 |
47 | Note: 8-10 images for a candidate are suffice. If you use spectacles, take images with and without them. 48 | Click on the captured images for viewing them (Delete option also provided). 49 |
50 | 51 |
52 | 53 |
54 | 55 |
56 | 57 |
58 | 59 |
60 | 61 | 62 |
63 | 64 |
65 | 66 |
67 | 68 | 69 | 70 |
71 | × 72 | 73 | 74 |
75 | 76 |
77 |
78 |
79 |
80 | Generating Features 81 |
82 |
83 | 84 | 240 | 241 | 242 | -------------------------------------------------------------------------------- /templates/results.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | Face Recognition 22 | 23 | 24 | 32 | 33 | 34 | 35 |
36 | 37 |
38 | 39 | 40 | 41 | 127 | 128 | 129 | -------------------------------------------------------------------------------- /utils.py: -------------------------------------------------------------------------------- 1 | import os 2 | import cv2 3 | import dlib 4 | import pickle 5 | import config 6 | import numpy as np 7 | import tensorflow as tf 8 | from imutils import face_utils 9 | from siameseNetwork import get_siamese_model, preprocess_input 10 | 11 | config.model = get_siamese_model() 12 | config.face_detector = dlib.get_frontal_face_detector() 13 | 14 | 15 | def predict_people(image): 16 | name = 'not known' 17 | gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) 18 | 19 | faces = config.face_detector(gray, 0) 20 | for face in faces: 21 | face_bounding_box = face_utils.rect_to_bb(face) 22 | if all(i >= 0 for i in face_bounding_box): 23 | [x, y, w, h] = face_bounding_box 24 | frame = image[y:y + h, x:x + w] 25 | cv2.rectangle(image, (x, y), (x + w, y + h), (0, 255, 0), 2) 26 | frame = cv2.resize(frame, (224, 224)) 27 | frame = np.asarray(frame, dtype=np.float64) 28 | frame = np.expand_dims(frame, axis=0) 29 | frame = preprocess_input(frame) 30 | feature = config.model.get_features(frame) 31 | 32 | dist = tf.norm(config.features - feature, axis=1) 33 | name = 'not known' 34 | loc = tf.argmin(dist) 35 | if dist[loc] < 0.8: 36 | name = config.people[loc] 37 | 38 | font_face = cv2.FONT_HERSHEY_SIMPLEX 39 | cv2.putText(image, name, (x, y-5), font_face, 0.8, (0,0,255), 3) 40 | return image 41 | 42 | def generate_dataset_festures(): 43 | people = [] 44 | features = [] 45 | dumpable_features = {} 46 | 47 | pickle_file = os.path.join(config.feature_dir, 'weights.pkl') 48 | if os.path.isfile(pickle_file): 49 | people, features, dumpable_features = load_pickle_file(pickle_file) 50 | 51 | image_dir_people = os.listdir(config.data_dir) 52 | for name in image_dir_people: 53 | if name not in people: 54 | nparr = generate_image_features(os.path.join(config.data_dir, name)) 55 | features.append(nparr) 56 | people.append(name) 57 | dumpable_features[name] = nparr 58 | 59 | config.features = features 60 | config.people = people 61 | 62 | dump_pickle_file(pickle_file, dumpable_features) 63 | print("Model Dumpped !!") 64 | 65 | 66 | def generate_image_features(directory): 67 | images = [] 68 | for image in os.listdir(directory): 69 | image_path = os.path.join(directory, image) 70 | img = cv2.imread(image_path) 71 | gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) 72 | faces = config.face_detector(gray, 0) 73 | if len(faces) == 0: 74 | continue 75 | for face in [faces[0]]: 76 | face_bounding_box = face_utils.rect_to_bb(face) 77 | if all(i >= 0 for i in face_bounding_box): 78 | [x, y, w, h] = face_bounding_box 79 | frame = img[y:y + h, x:x + w] 80 | frame = cv2.resize(frame, (224, 224)) 81 | frame = np.asarray(frame, dtype=np.float64) 82 | images.append(frame) 83 | images = np.asarray(images) 84 | images = preprocess_input(images) 85 | images = tf.convert_to_tensor(images) 86 | feature = config.model.get_features(images) 87 | feature = tf.reduce_mean(feature, axis=0) 88 | return feature 89 | 90 | 91 | def load_pickle_file(pickle_file): 92 | with open(pickle_file, 'rb') as f: 93 | dumpable_features = pickle.load(f) 94 | 95 | people = [] 96 | features = [] 97 | for key, value in dumpable_features.items(): 98 | people.append(key) 99 | features.append(value) 100 | return people, features, dumpable_features 101 | 102 | 103 | def dump_pickle_file(pickle_file, dumpable_features): 104 | if len(list(dumpable_features.keys())) > 0: 105 | with open(pickle_file, 'wb') as f: 106 | pickle.dump(dumpable_features, f) 107 | --------------------------------------------------------------------------------