├── README.md ├── data.csv ├── datagen.py ├── facemesh.jpg ├── model.pkl ├── requirements.txt ├── test.py └── training.py /README.md: -------------------------------------------------------------------------------- 1 | Facial Emotion Detection using Face Mesh and Machine Learning 2 | Project Overview 3 | This project involves creating a facial emotion detection system using face mesh points captured via a webcam or video, combined with a machine learning model. The project generates face data (x, y coordinates of facial landmarks) using OpenCV and cvzone, stores it in a CSV file, trains a machine learning model to classify emotions, and uses the trained model to predict emotions from live video feed. 4 | 5 | The project consists of three major components: 6 | 7 | Data Generation: Captures face mesh landmarks and stores them with emotion labels. 8 | Model Training: Trains a machine learning model (Logistic Regression) on the face mesh data. 9 | Emotion Prediction: Uses the trained model to classify the emotion in real-time based on face mesh data. 10 | 11 | Table of Contents 12 | Installation 13 | Project Structure 14 | How It Works 15 | Usage 16 | Customization 17 | Dependencies 18 | License 19 | Installation 20 | 21 | Clone the Repository 22 | Open your terminal and run: 23 | git clone 24 | cd 25 | Install Dependencies 26 | Make sure you have Python installed. Install the required packages using: 27 | pip install -r requirements.txt 28 | Project Structure 29 | 30 | ├── data.csv # CSV file for storing face mesh data with emotion labels 31 | ├── datagen.py # Script to generate and save face data 32 | ├── model.pkl # Pre-trained machine learning model (Logistic Regression) 33 | ├── requirements.txt # List of project dependencies 34 | ├── test.py # Script to predict emotions using the trained model 35 | └── training.py # Script for training the model on face data 36 | 37 | How It Works 38 | 1. Data Generation (datagen.py) 39 | This script uses a webcam or video input to capture face mesh points (468 points per frame) using cvzone.FaceMeshModule. 40 | It flattens these points and saves them in a CSV file (data.csv) along with the emotion label (e.g., 'happy', 'sad'). 41 | 42 | The key facial landmarks (x, y coordinates) are extracted for each frame. 43 | The emotion label is inserted manually into the script and saved with the data. 44 | 2. Model Training (training.py) 45 | This script loads the captured face mesh data from data.csv and trains a Logistic Regression model on the data to classify emotions. It uses a pipeline with standard scaling and logistic regression. The trained model is then saved as model.pkl. 46 | 47 | The dataset is split into features (landmark points) and labels (emotion classes). 48 | A pipeline is created with StandardScaler and LogisticRegression. 49 | The trained model is evaluated and saved for later use. 50 | 3. Emotion Prediction (test.py) 51 | This script uses the trained model (model.pkl) to predict emotions in real-time from the webcam feed. It captures face mesh points in each frame, feeds them into the model, and displays the predicted emotion on the video. 52 | 53 | Real-time prediction using the trained machine learning model. 54 | The live video feed is processed frame by frame to detect face mesh points. 55 | The predicted emotion is overlaid on the video. 56 | Usage 57 | Step 1: Data Generation 58 | Run the datagen.py script to capture and store face mesh data. The data is stored in data.csv with the class label for each frame. 59 | 60 | 61 | python datagen.py 62 | Step 2: Model Training 63 | Train the model using the captured data by running the training.py script. The trained model will be saved as model.pkl. 64 | 65 | 66 | python training.py 67 | Step 3: Emotion Prediction 68 | Run the test.py script to start real-time emotion prediction using your webcam. 69 | python test.py 70 | 71 | Customization 72 | Adding New Emotions: To train the model on additional emotions, modify the class_name in datagen.py to the desired emotion and capture new data. 73 | Model Parameters: You can replace the Logistic Regression model with other classifiers (e.g., SVM or Random Forest) by modifying the training.py script. 74 | Video Input: Change the cap = cv2.VideoCapture(1) in datagen.py and test.py to cap = cv2.VideoCapture('video.mp4') if you want to use a video file instead of live webcam input. 75 | 76 | Dependencies 77 | cvzone: For face mesh detection and easy text overlay. 78 | OpenCV: For video processing and capturing frames from the webcam. 79 | NumPy: For handling numerical operations. 80 | Scikit-learn: For machine learning algorithms and data preprocessing. 81 | To install the dependencies, simply run: 82 | pip install -r requirements.txt 83 | 84 | License 85 | This project is licensed under the MIT License. You are free to use, modify, and distribute this project. 86 | 87 | -------------------------------------------------------------------------------- /datagen.py: -------------------------------------------------------------------------------- 1 | from cvzone.FaceMeshModule import FaceMeshDetector 2 | import cv2 3 | import numpy as np 4 | import csv 5 | 6 | videopath = 'happy.mp4' 7 | cap = cv2.VideoCapture(1) 8 | 9 | FMD = FaceMeshDetector() 10 | class_name = 'sad' 11 | 12 | # Columns = ['Class'] 13 | # for val in range(1,468+1): 14 | # Columns += ['x{}'.format(val),'y{}'.format(val)] 15 | 16 | # print(Columns) 17 | 18 | 19 | # with open('data.csv','w',newline='') as f: 20 | # csv_writer = csv.writer(f,delimiter = ',') 21 | # csv_writer.writerow(Columns) 22 | 23 | 24 | while cap.isOpened(): 25 | rt,frame = cap.read() 26 | frame = cv2.resize(frame,(720,480)) 27 | img , faces = FMD.findFaceMesh(frame) 28 | 29 | if faces: 30 | face = faces[0] 31 | face_data = list(np.array(face).flatten()) 32 | face_data.insert(0,class_name) 33 | 34 | # print(face_data) 35 | # print(len(face_data)) 36 | # print(face_data) 37 | 38 | with open('data.csv','a',newline='') as f: 39 | csv_writer = csv.writer(f,delimiter = ',') 40 | csv_writer.writerow(face_data) 41 | 42 | cv2.imshow('frame',frame) 43 | cv2.waitKey(1) 44 | cap.release() 45 | cv2.destroyWindow() -------------------------------------------------------------------------------- /facemesh.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Tech-Watt/Emotion-Detection/628f3bd5203e48a35a67475396218447bec6af01/facemesh.jpg -------------------------------------------------------------------------------- /model.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Tech-Watt/Emotion-Detection/628f3bd5203e48a35a67475396218447bec6af01/model.pkl -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | opencv-python 2 | mediapipe 3 | pandas 4 | cvzone 5 | numpy 6 | -U scikit-learn -------------------------------------------------------------------------------- /test.py: -------------------------------------------------------------------------------- 1 | import cvzone 2 | import pickle 3 | from cvzone.FaceMeshModule import FaceMeshDetector 4 | import cv2 5 | import numpy as np 6 | 7 | 8 | cap = cv2.VideoCapture(1) 9 | 10 | FMD = FaceMeshDetector() 11 | 12 | 13 | 14 | with open('model.pkl','rb') as f: 15 | Behaviour_model = pickle.load(f) 16 | 17 | 18 | # taking video frame by frame 19 | while cap.isOpened(): 20 | rt,frame = cap.read() 21 | frame = cv2.resize(frame,(720,480)) 22 | 23 | real_frame = frame.copy() 24 | 25 | img , faces = FMD.findFaceMesh(frame) 26 | cvzone.putTextRect(frame, ('Mood'), (10, 80)) 27 | if faces: 28 | face = faces[0] 29 | face_data = list(np.array(face).flatten()) 30 | 31 | 32 | try: 33 | # feeding newpoints to model for prediction 34 | result = Behaviour_model.predict([face_data]) 35 | cvzone.putTextRect(frame, str(result[0]), (250, 80)) 36 | print(result) 37 | 38 | # resultproba = Behaviour_model.predict_proba([face_data]) 39 | # print(resultproba) 40 | 41 | 42 | except Exception as e: 43 | pass 44 | all_frames = cvzone.stackImages([real_frame,frame],2,0.70) 45 | cv2.imshow('frame',all_frames) 46 | cv2.waitKey(1) -------------------------------------------------------------------------------- /training.py: -------------------------------------------------------------------------------- 1 | import pandas as pd 2 | from sklearn.ensemble import RandomForestClassifier 3 | from sklearn.pipeline import Pipeline 4 | from sklearn.svm import SVC 5 | from sklearn.linear_model import LogisticRegression 6 | from sklearn.metrics import classification_report 7 | from sklearn.preprocessing import StandardScaler 8 | from sklearn.model_selection import train_test_split 9 | import pickle 10 | 11 | 12 | 13 | df = pd.read_csv('data.csv') 14 | 15 | print(df.head()) 16 | print(df.columns) 17 | print(df.info()) 18 | 19 | print(df.describe()) 20 | 21 | df = df.dropna() 22 | 23 | features = df.drop('Class',axis=1) 24 | labels = df['Class'] 25 | 26 | 27 | print(features) 28 | print(labels) 29 | 30 | # # Split the data into training and test sets 31 | X_train, X_test, y_train, y_test = train_test_split(features, labels, test_size=0.2, random_state=42) 32 | 33 | # Create a pipeline 34 | pipeline = Pipeline([ 35 | ('scaler', StandardScaler()), 36 | ('svm', LogisticRegression()) 37 | ]) 38 | 39 | # Fit the pipeline on the training data 40 | pipeline.fit(X_train, y_train) 41 | 42 | # Predict on the test data 43 | predictions = pipeline.predict(X_test) 44 | 45 | # Evaluate the model 46 | yhat = pipeline.predict(X_test) 47 | print(yhat) 48 | model_performance = classification_report(y_test,yhat) 49 | print(f"Model Report: {model_performance}") 50 | 51 | model_name = 'model.pkl' 52 | with open(model_name,'wb')as f: 53 | pickle.dump(pipeline,f) 54 | 55 | --------------------------------------------------------------------------------