├── ARDUINO_TO_PC_DATA.ino ├── README.md └── ANIMATRONIC_EYE.py /ARDUINO_TO_PC_DATA.ino: -------------------------------------------------------------------------------- 1 | /* 2 | Author: Aditya 3 | 4 | 5 | This script is designed to control a 16-channel PWM servo driver using the Adafruit_PWMServoDriver library. 6 | It uses the Wire library for I2C communication. The script initializes the servo driver, sets the PWM frequency, 7 | and then enters a loop where it drives each servo one at a time. 8 | 9 | The script is divided into several parts: 10 | 1. Importing the necessary libraries 11 | 2. Defining the minimum and maximum pulse lengths 12 | 3. Initializing the servo driver 13 | 4. Defining the setup function, which initializes the serial port and the servo driver 14 | 5. Defining a function to set the PWM signal for a specific servo 15 | 6. Defining the main loop, which drives each servo one at a time by varying the pulse length 16 | */ 17 | 18 | // Import the required libraries 19 | #include 20 | #include 21 | 22 | // Initialize the servo driver with the default address 0x40 23 | Adafruit_PWMServoDriver pwm = Adafruit_PWMServoDriver(); 24 | 25 | // Define the minimum and maximum pulse lengths 26 | #define SERVOMIN 150 // this is the 'minimum' pulse length count (out of 4096) 27 | #define SERVOMAX 600 // this is the 'maximum' pulse length count (out of 4096) 28 | 29 | // Initialize the servo number counter 30 | uint8_t servonum = 0; 31 | 32 | void setup() { 33 | // Initialize the serial port and print a message 34 | Serial.begin(250000); 35 | Serial.println("16 channel PWM test!"); 36 | 37 | // Initialize the servo driver 38 | pwm.begin(); 39 | 40 | // Set the PWM frequency for analog servos 41 | pwm.setPWMFreq(60); // Analog servos run at ~60 Hz updates 42 | 43 | delay(10); 44 | } 45 | 46 | // Function to set the PWM signal for a specific servo 47 | // This function can be used as a drop-in replacement for 'analogWrite' 48 | void setPWM(uint8_t num, uint16_t on, uint16_t off) { 49 | pwm.setPWM(num, on, off); 50 | } 51 | 52 | void loop() { 53 | // Print the current servo number 54 | Serial.println(servonum); 55 | 56 | // Drive the current servo by varying the pulse length from minimum to maximum 57 | for (uint16_t pulselen = SERVOMIN; pulselen < SERVOMAX; pulselen++) { 58 | pwm.setPWM(servonum, 0, pulselen); 59 | } 60 | 61 | delay(500); 62 | 63 | // Drive the current servo by varying the pulse length from maximum to minimum 64 | for (uint16_t pulselen = SERVOMAX; pulselen > SERVOMIN; pulselen--) { 65 | pwm.setPWM(servonum, 0, pulselen); 66 | } 67 | 68 | delay(500); 69 | 70 | // Increment the servo number counter 71 | servonum ++; 72 | if (servonum > 15) servonum = 0; 73 | } 74 | 75 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Real-Time Eye Blink and Movement Detection with Servo Control 2 | 3 | This project is designed to detect eye blinks and movements in real-time using a webcam, process the information using Python and OpenCV, and then control a servo motor based on the detected eye movements. The project uses a combination of software (Python, OpenCV) and hardware (webcam, Arduino, servo motor). 4 | 5 | ## Hardware Components 6 | 7 | 1. **Webcam**: Used to capture video frames in real-time. 8 | 2. **Arduino**: Used as an interface between the Python script and the servo motor. The Arduino receives commands from the Python script via serial communication and controls the servo motor accordingly. 9 | 3. **Servo Motor**: Moves based on the commands received from the Arduino. 10 | 11 | ## Software Components 12 | 13 | 1. **Python**: The main programming language used for this project. 14 | 2. **OpenCV**: A powerful computer vision library used for real-time image processing. 15 | 3. **Haar Cascade Classifiers**: Machine learning-based approach for object detection, used to detect faces and eyes in the video frames. 16 | 17 | ## Arduino Code Explanation 18 | 19 | The Arduino code is responsible for receiving commands from the Python script via serial communication and controlling the servo motor based on these commands. 20 | 21 | The Arduino code uses the Adafruit_PWMServoDriver library to control a 16-channel PWM servo driver. It sets the PWM frequency for the servo driver to 60 Hz, which is the standard update frequency for analog servos. 22 | 23 | The Arduino code receives two types of commands from the Python script: one for eye blinks and one for eye movements. For eye blinks, the Arduino receives a command to move the servo to a specific position. For eye movements, the Arduino receives a command to move the servo based on the calculated error between the center of the face and the center of the frame. 24 | 25 | ## How It Works 26 | 27 | The Python script captures video frames from the webcam in real-time. It then uses OpenCV and Haar cascade classifiers to detect faces and eyes in each frame. 28 | 29 | If eyes are detected, it means the eyes are open. If no eyes are detected for a certain period, it indicates a blink. 30 | 31 | The position of the detected face in the frame is used to calculate the direction of eye movement. This is done by calculating the error between the center of the face and the center of the frame. 32 | 33 | These eye blink and movement detections are then sent as commands to the Arduino via serial communication. The Arduino controls the servo motor based on these commands. 34 | 35 | ## Usage 36 | 37 | To use this project, you need to have the hardware components set up correctly, with the Arduino connected to the servo motor and the computer, and the webcam connected to the computer. 38 | 39 | You also need to have Python installed on your computer, along with the OpenCV library. The Python script should be run on the computer, and the Arduino should be programmed to receive serial commands and control the servo motor accordingly. 40 | 41 | ## References 42 | This project is inspired by and uses STL files from Will Cogley's original animatronic eye project. His work has been a great resource and inspiration in the development of this project. 43 | 44 | -------------------------------------------------------------------------------- /ANIMATRONIC_EYE.py: -------------------------------------------------------------------------------- 1 | """ 2 | Author: Aditya 3 | 4 | This script is designed to detect faces and eyes in real-time using a webcam. 5 | It uses OpenCV for image processing and pySerial for serial communication. 6 | The script captures video from the webcam, detects faces and eyes in each frame, 7 | and sends commands to a serial port based on the position of the detected face. 8 | 9 | The script is divided into several parts: 10 | 1. Importing the necessary libraries 11 | 2. Defining a function to set the resolution of the video capture 12 | 3. Initializing the serial port and video capture 13 | 4. Loading the cascade classifiers for face and eye detection 14 | 5. Starting the video capture loop, where each frame is processed and faces and eyes are detected 15 | 6. If a face is detected, the script calculates the error between the center of the face and the center of the frame, and sends this error to the serial port 16 | 7. If eyes are detected, the script sends a specific command to the serial port 17 | 8. The loop continues until the 'E' key is pressed 18 | 9. Finally, the serial port and video capture are closed, and all windows are destroyed 19 | """ 20 | 21 | # Import the required libraries 22 | import cv2 23 | import serial 24 | 25 | # Function to set the resolution of the video capture 26 | def set_res(cap, x, y): 27 | cap.set(cv2.CAP_PROP_FRAME_WIDTH, int(x)) 28 | cap.set(cv2.CAP_PROP_FRAME_HEIGHT, int(y)) 29 | 30 | # Initialize the serial port 31 | ser = serial.Serial('COM4', 250000) 32 | 33 | # Initialize the video capture 34 | cap = cv2.VideoCapture(0) 35 | 36 | # Set the frame width and height 37 | frame_w = 640 38 | frame_h = 480 39 | set_res(cap, frame_w, frame_h) 40 | 41 | # Load the cascade classifiers for face and eye detection 42 | face_cascade = cv2.CascadeClassifier('C:/Users/addua/Downloads/haarcascade_frontalface_default.xml') 43 | EYE_cascade = cv2.CascadeClassifier('C:/Users/addua/Downloads/archive/haarcascade_eye.xml') 44 | 45 | # Initialize error variables 46 | err_x = 0 47 | err_y = 0 48 | 49 | # Start the video capture loop 50 | while True: 51 | # Read a frame from the video capture 52 | ret, frame = cap.read() 53 | 54 | # If the frame is None, print an error message and continue 55 | if frame is None: 56 | print("Error: Frame is None") 57 | continue 58 | 59 | # Flip the frame and convert it to grayscale 60 | frame = cv2.flip(frame, 1) 61 | gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) 62 | 63 | # Detect faces in the frame 64 | faces = face_cascade.detectMultiScale(gray, scaleFactor=1.1, minNeighbors=10, minSize=(30, 30)) 65 | 66 | # For each detected face, draw a rectangle and detect eyes within the face 67 | for (x, y, w, h) in faces: 68 | cv2.rectangle(frame, (x, y), (x+w, y+h), (255, 0, 0), 2) 69 | cv2.putText(frame, "Aditya", (x, y-10), cv2.FONT_HERSHEY_SIMPLEX, 0.9, (255,0,0), 2) 70 | faceROI = gray[y:y+h, x:x+w] 71 | eyes = EYE_cascade.detectMultiScale(faceROI, scaleFactor=1.05, minNeighbors=300,minSize=(5,5)) 72 | 73 | # For each detected eye, draw a rectangle 74 | for (ex, ey, ew, eh) in eyes: 75 | cv2.rectangle(frame, (x+ex, y+ey), (x+ex+ew, y+ey+eh), (0, 0, 0), 2) 76 | 77 | # Display the frame 78 | cv2.imshow('frame', frame) 79 | 80 | # If 'E' is pressed, break the loop 81 | if cv2.waitKey(1) & 0xFF == ord('E'): 82 | break 83 | 84 | # If eyes are detected, print a message and write to the serial port 85 | if len(eyes)>0: 86 | print('the eyes are opened') 87 | ser.write((str(-30)+'z!').encode()) 88 | else: 89 | print('the eyes are closed') 90 | ser.write((str(70)+'z!').encode()) 91 | 92 | # If faces are detected, calculate the error and write to the serial port 93 | if len(faces) > 0: 94 | face_center_x = faces[0,0] + faces[0,2] / 2 95 | face_center_y = faces[0,1] + faces[0,3] / 2 96 | err_x = 70 * (face_center_x - frame_w / 2) / (frame_w / 2) 97 | err_y = 100* (face_center_y - frame_h / 2) / (frame_h / 2) 98 | ser.write((str(err_x) + "x!").encode()) 99 | ser.write((str(err_y) + "y!").encode()) 100 | print("X: ", err_x, " ", "Y: ", err_y) 101 | 102 | # Close the serial port and video capture, and destroy all windows 103 | ser.close() 104 | cap.release() 105 | cv2.destroyAllWindows() 106 | --------------------------------------------------------------------------------