├── LICENSE ├── README.md ├── media ├── elon-sample.gif ├── facial_landmarks_68markup-768x619.jpg ├── obama-sample.gif ├── obama.mp4 └── readme.md ├── models └── readme.md ├── python ├── faster-rpicamera.py ├── fastest-rpicamera.py ├── headpose.py ├── rpicam-pose.py ├── run-pose.sh ├── save-video-pose.py ├── video-headpose.py └── webcam-headpose.py ├── requirements.txt ├── run.sh └── setup.py /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 Ross Mauck 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## About headpose 2 | Process camera feed for head pose estimation is a Python application for computer vision live face parameterization 3 | 4 | Using dlib's face landmark predictor, I added my implementation of a real-time by building a graphics pipeline to support the 2D to 3D head pose estimation method built by Satya Mallick in the referenced code below. 5 | 6 | --- 7 | ### Referenced Code 8 | * https://www.learnopencv.com/head-pose-estimation-using-opencv-and-dlib 9 | * https://www.pyimagesearch.com/2017/04/03/facial-landmarks-dlib-opencv-python 10 | 11 | ### Dependencies 12 | You need to have Python 2.6+ as a minimum and: 13 | 14 | * [NumPy](http://numpy.scipy.org/) 15 | * [OpenCV 3](http://opencv.org/) Prefer OpenCV 3.4+ 16 | * [Dlib](http://dlib.net/) 17 | * [Imutils](https://github.com/jrosebr1/imutils) 18 | 19 |

20 | 21 |

22 | 23 | ### Structure 24 | 25 | *python/* the code. 26 | 27 | *models/* contains the models used in this example we use Facial Landmark detection 68 points. 28 | *one must download shape_detector_68_facial_landmarks.dat because it is too large a file to host here. 29 | 30 | *media/* contains images and video. 31 | 32 | ### Installation 33 | 34 | Open a terminal in the headpose directory and run (with sudo if needed on your system): 35 | 36 | pip install -r requirements.txt 37 | 38 | Now you should have installed the necessary packages 39 | 40 | You still need to download dlib model file: "shape_predictor_68_face_landmarks.dat" 41 | 42 | This is a location where that file is hosted: https://github.com/AKSHAYUBHAT/TensorFace/blob/master/openface/models/dlib/shape_predictor_68_face_landmarks.dat 43 | 44 | Put this model file "shape_predictor_68_face_landmarks.dat" located in YOUR_MODEL_DOWNLOAD_PATH into the model folder headpose/models 45 | 46 | cp YOUR_MODEL_DOWNLOAD_PATH/shape_predictor_68_face_landmarks.dat headpose/models/shape_predictor_68_face_landmarks.dat 47 | 48 | Give privilages to run the shell script to start application 49 | 50 | chmod +x run.sh 51 | 52 | Then run the shell script 53 | 54 | ./run.sh 55 | 56 | in your Python session or script. Try one of the sample code examples to check that the installation works. 57 | 58 | For good resources on these topics see: 59 | 60 | Detecting face landmarks: Adrian Rosebrock implementation of dlib's shape_predictor's for face landmark detection 61 | *[Face Landmark Detection](https://www.learnopencv.com/head-pose-estimation-using-opencv-and-dlib/) 62 | 63 | Pose estimation: Satya Mallick's implementation of OpenCV's PnP function 64 | *[Pose Estimation](https://www.pyimagesearch.com/2017/04/03/facial-landmarks-dlib-opencv-python/) 65 | 66 | Using this and this 67 | to get this information 68 | 69 | and this to take that information which would produce this output. 70 | 71 | I wanted to write a graphics pipeline that would produce this in th 72 | I did this 73 | 74 |

75 | 76 |

77 | 78 | ### License 79 | 80 | All code in this project is provided as open source under the MIT license 81 | 82 | 83 | --- 84 | -Ross Mauck 85 | 86 | 87 | -------------------------------------------------------------------------------- /media/elon-sample.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mauckc/headpose/0826031098fbb22866ff9f87e4607c90939acec5/media/elon-sample.gif -------------------------------------------------------------------------------- /media/facial_landmarks_68markup-768x619.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mauckc/headpose/0826031098fbb22866ff9f87e4607c90939acec5/media/facial_landmarks_68markup-768x619.jpg -------------------------------------------------------------------------------- /media/obama-sample.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mauckc/headpose/0826031098fbb22866ff9f87e4607c90939acec5/media/obama-sample.gif -------------------------------------------------------------------------------- /media/obama.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mauckc/headpose/0826031098fbb22866ff9f87e4607c90939acec5/media/obama.mp4 -------------------------------------------------------------------------------- /media/readme.md: -------------------------------------------------------------------------------- 1 | # Media files 2 | Media files stored here. 3 | -------------------------------------------------------------------------------- /models/readme.md: -------------------------------------------------------------------------------- 1 | # Models stored here 2 | I am using dlib supplied "shape_predictor_68_face_landmarks.dat" 3 | 4 | You still need to download dlib model file: "shape_predictor_68_face_landmarks.dat" 5 | 6 | This is a location where that file is hosted: https://github.com/AKSHAYUBHAT/TensorFace/blob/master/openface/models/dlib/shape_predictor_68_face_landmarks.dat 7 | 8 | -------------------------------------------------------------------------------- /python/faster-rpicamera.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # import the necessary packages 3 | from __future__ import print_function 4 | from imutils.video import VideoStream 5 | from imutils import face_utils 6 | import argparse 7 | import imutils 8 | import time 9 | import dlib 10 | import cv2 11 | import numpy as np 12 | from picamera.array import PiRGBArray 13 | from picamera import PiCamera 14 | from threading import Thread 15 | 16 | from imutils.video.pivideostream import PiVideoStream 17 | from imutils.video import FPS 18 | 19 | # construct the argument parser and parse the arguments 20 | ap = argparse.ArgumentParser() 21 | ap.add_argument("-p", "--shape-predictor", required=True, 22 | help="path to facial landmark predictor") 23 | args = vars(ap.parse_args()) 24 | 25 | # initialize dlib's face detector (HOG-based) and then create the 26 | # facial landmark predictor 27 | print("[INFO] loading facial landmark predictor...") 28 | detector = dlib.get_frontal_face_detector() 29 | predictor = dlib.shape_predictor(args["shape_predictor"]) 30 | 31 | # initialize the video stream and sleep for a bit, allowing the 32 | # camera sensor to warm up 33 | print("[INFO] camera sensor warming up...") 34 | 35 | #vs = VideoStream(src=0).start() 36 | # vs = VideoStream(usePiCamera=True).start() # Raspberry Pi 37 | time.sleep(2.0) 38 | 39 | # 400x225 to 1024x576 40 | frame_width = 400 41 | frame_height = 225 42 | 43 | # Define the codec and create VideoWriter object.The output is stored in 'outpy.avi' file. 44 | #out = cv2.VideoWriter('outpy.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 30, (frame_width,frame_height)) 45 | 46 | # loop over the frames from the video stream 47 | #2D image points. If you change the image, you need to change vector 48 | image_points = np.array([ 49 | (259, 291), # Nose tip 34 50 | (399, 261), # Chin 9 51 | (337, 297), # Left eye left corner 37 52 | (513, 301), # Right eye right corne 46 53 | (345, 265), # Left Mouth corner 49 54 | (453, 269) # Right mouth corner 55 55 | ], dtype="double") 56 | 57 | # 3D model points. 58 | model_points = np.array([ 59 | (0.0, 0.0, 0.0), # Nose tip 34 60 | (0.0, -330.0, -65.0), # Chin 9 61 | (-225.0, 170.0, -135.0), # Left eye left corner 37 62 | (225.0, 170.0, -135.0), # Right eye right corne 46 63 | (-150.0, -150.0, -125.0), # Left Mouth corner 49 64 | (150.0, -150.0, -125.0) # Right mouth corner 55 65 | 66 | ]) 67 | # created a *threaded *video stream, allow the camera sensor to warmup, 68 | # and start the FPS counter 69 | print("[INFO] sampling THREADED frames from `picamera` module...") 70 | vs = PiVideoStream().start() 71 | time.sleep(2.0) 72 | fps = FPS().start() 73 | 74 | # loop over some frames...this time using the threaded stream 75 | while fps._numFrames < args["num_frames"]: 76 | # grab the frame from the threaded video stream and resize it 77 | # to have a maximum width of 400 pixels 78 | frame = vs.read() 79 | frame = imutils.resize(frame, width=400) 80 | 81 | # check to see if the frame should be displayed to our screen 82 | if args["display"] > 0: 83 | cv2.imshow("Frame", frame) 84 | key = cv2.waitKey(1) & 0xFF 85 | 86 | # update the FPS counter 87 | fps.update() 88 | 89 | # stop the timer and display FPS information 90 | fps.stop() 91 | print("[INFO] elasped time: {:.2f}".format(fps.elapsed())) 92 | print("[INFO] approx. FPS: {:.2f}".format(fps.fps())) 93 | 94 | # do a bit of cleanup 95 | cv2.destroyAllWindows() 96 | vs.stop() 97 | while True: 98 | frame = vs.read() 99 | frame = imutils.resize(image, width=frame_width, height=frame_height) 100 | frame = cv2.flip(frame,1) 101 | gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) 102 | size = frame.shape 103 | 104 | # detect faces in the grayscale frame 105 | rects = detector(gray, 0) 106 | 107 | # check to see if a face was detected, and if so, draw the total 108 | # number of faces on the frame 109 | if len(rects) > 0: 110 | text = "{} face(s) found".format(len(rects)) 111 | cv2.putText(frame, text, (10, 20), cv2.FONT_HERSHEY_SIMPLEX,0.5, (0, 0, 255), 2) 112 | 113 | # loop over the face detections 114 | for rect in rects: 115 | # compute the bounding box of the face and draw it on the 116 | # frame 117 | (bX, bY, bW, bH) = face_utils.rect_to_bb(rect) 118 | cv2.rectangle(frame, (bX, bY), (bX + bW, bY + bH),(0, 255, 0), 1) 119 | # determine the facial landmarks for the face region, then 120 | # convert the facial landmark (x, y)-coordinates to a NumPy 121 | # array 122 | shape = predictor(gray, rect) 123 | shape = face_utils.shape_to_np(shape) 124 | # loop over the (x, y)-coordinates for the facial landmarks 125 | # and draw each of them 126 | for (i, (x, y)) in enumerate(shape): 127 | if i == 33: 128 | #something to our key landmarks 129 | # save to our new key point list 130 | # i.e. keypoints = [(i,(x,y))] 131 | image_points[0] = np.array([x,y],dtype='double') 132 | # write on frame in Green 133 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 134 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 135 | elif i == 8: 136 | #something to our key landmarks 137 | # save to our new key point list 138 | # i.e. keypoints = [(i,(x,y))] 139 | image_points[1] = np.array([x,y],dtype='double') 140 | # write on frame in Green 141 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 142 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 143 | elif i == 36: 144 | #something to our key landmarks 145 | # save to our new key point list 146 | # i.e. keypoints = [(i,(x,y))] 147 | image_points[2] = np.array([x,y],dtype='double') 148 | # write on frame in Green 149 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 150 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 151 | elif i == 45: 152 | #something to our key landmarks 153 | # save to our new key point list 154 | # i.e. keypoints = [(i,(x,y))] 155 | image_points[3] = np.array([x,y],dtype='double') 156 | # write on frame in Green 157 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 158 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 159 | elif i == 48: 160 | #something to our key landmarks 161 | # save to our new key point list 162 | # i.e. keypoints = [(i,(x,y))] 163 | image_points[4] = np.array([x,y],dtype='double') 164 | # write on frame in Green 165 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 166 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 167 | elif i == 54: 168 | #something to our key landmarks 169 | # save to our new key point list 170 | # i.e. keypoints = [(i,(x,y))] 171 | image_points[5] = np.array([x,y],dtype='double') 172 | # write on frame in Green 173 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 174 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 175 | else: 176 | #everything to all other landmarks 177 | # write on frame in Red 178 | cv2.circle(frame, (x, y), 1, (0, 0, 255), -1) 179 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 0, 255), 1) 180 | focal_length = size[1] 181 | center = (size[1]/2, size[0]/2) 182 | camera_matrix = np.array([[focal_length,0,center[0]],[0, focal_length, center[1]],[0,0,1]], dtype="double") 183 | 184 | #print "Camera Matrix :\n {0}".format(camera_matrix) 185 | 186 | dist_coeffs = np.zeros((4,1)) # Assuming no lens distortion 187 | (success, rotation_vector, translation_vector) = cv2.solvePnP(model_points, image_points, camera_matrix, dist_coeffs, flags=cv2.SOLVEPNP_ITERATIVE)#flags=cv2.CV_ITERATIVE) 188 | 189 | #print "Rotation Vector:\n {0}".format(rotation_vector) 190 | #print "Translation Vector:\n {0}".format(translation_vector) 191 | # Project a 3D point (0, 0 , 1000.0) onto the image plane 192 | # We use this to draw a line sticking out of the nose_end_point2D 193 | (nose_end_point2D, jacobian) = cv2.projectPoints(np.array([(0.0, 0.0, 1000.0)]),rotation_vector, translation_vector, camera_matrix, dist_coeffs) 194 | for p in image_points: 195 | cv2.circle(frame, (int(p[0]), int(p[1])), 3, (0,0,255), -1) 196 | 197 | p1 = ( int(image_points[0][0]), int(image_points[0][1])) 198 | p2 = ( int(nose_end_point2D[0][0][0]), int(nose_end_point2D[0][0][1])) 199 | 200 | cv2.line(frame, p1, p2, (255,0,0), 2) 201 | 202 | # show the frame 203 | (h,w) = frame.shape[:2] 204 | center = (w / 2, h / 2) 205 | M = cv2.getRotationMatrix2D(center, 180, 1.0) 206 | rotated = cv2.warpAffine(frame, M, (w,h)) 207 | 208 | cv2.imshow("rotated", rotated) 209 | key = cv2.waitKey(1) & 0xFF 210 | 211 | # clear the stream for the next frame 212 | rawCapture.truncate(0) 213 | # if the `q` key was pressed, break from the loop 214 | if key == ord("q"): 215 | break 216 | print(image_points) 217 | 218 | # do a bit of cleanup 219 | cv2.destroyAllWindows() 220 | vs.stop() 221 | 222 | 223 | -------------------------------------------------------------------------------- /python/fastest-rpicamera.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # import the necessary packages 3 | from __future__ import print_function 4 | from imutils.video import VideoStream 5 | from imutils import face_utils 6 | import argparse 7 | import imutils 8 | import time 9 | import dlib 10 | import cv2 11 | import numpy as np 12 | from picamera.array import PiRGBArray 13 | from picamera import PiCamera 14 | from threading import Thread 15 | 16 | from imutils.video.pivideostream import PiVideoStream 17 | from imutils.video import FPS 18 | 19 | # construct the argument parser and parse the arguments 20 | ap = argparse.ArgumentParser() 21 | ap.add_argument("-p", "--shape-predictor", required=True, 22 | help="path to facial landmark predictor") 23 | args = vars(ap.parse_args()) 24 | 25 | # initialize dlib's face detector (HOG-based) and then create the 26 | # facial landmark predictor 27 | print("[INFO] loading facial landmark predictor...") 28 | detector = dlib.get_frontal_face_detector() 29 | predictor = dlib.shape_predictor(args["shape_predictor"]) 30 | 31 | # initialize the video stream and sleep for a bit, allowing the 32 | # camera sensor to warm up 33 | print("[INFO] camera sensor warming up...") 34 | 35 | #vs = VideoStream(src=0).start() 36 | # vs = VideoStream(usePiCamera=True).start() # Raspberry Pi 37 | time.sleep(2.0) 38 | 39 | # 400x225 to 1024x576 40 | frame_width = 400 41 | frame_height = 225 42 | 43 | # Define the codec and create VideoWriter object.The output is stored in 'outpy.avi' file. 44 | #out = cv2.VideoWriter('outpy.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 30, (frame_width,frame_height)) 45 | 46 | # loop over the frames from the video stream 47 | #2D image points. If you change the image, you need to change vector 48 | image_points = np.array([ 49 | (259, 291), # Nose tip 34 50 | (399, 261), # Chin 9 51 | (337, 297), # Left eye left corner 37 52 | (513, 301), # Right eye right corne 46 53 | (345, 265), # Left Mouth corner 49 54 | (453, 269) # Right mouth corner 55 55 | ], dtype="double") 56 | 57 | # 3D model points. 58 | model_points = np.array([ 59 | (0.0, 0.0, 0.0), # Nose tip 34 60 | (0.0, -330.0, -65.0), # Chin 9 61 | (-225.0, 170.0, -135.0), # Left eye left corner 37 62 | (225.0, 170.0, -135.0), # Right eye right corne 46 63 | (-150.0, -150.0, -125.0), # Left Mouth corner 49 64 | (150.0, -150.0, -125.0) # Right mouth corner 55 65 | 66 | ]) 67 | # created a *threaded *video stream, allow the camera sensor to warmup, 68 | # and start the FPS counter 69 | print("[INFO] sampling THREADED frames from `picamera` module...") 70 | vs = PiVideoStream().start() 71 | time.sleep(2.0) 72 | fps = FPS().start() 73 | count = 0 74 | while True: 75 | if count % 4 == 0: 76 | frame = vs.read() 77 | frame = imutils.resize(frame, width=frame_width, height=frame_height) 78 | #frame = cv2.flip(frame,1) 79 | #(h,w) = frame.shape[:2] 80 | (h,w) = (frame_height, frame_width) 81 | center = (w/2,h/2) 82 | M = cv2.getRotationMatrix2D(center, 180, 1.0) 83 | 84 | frame = cv2.warpAffine(frame, M, (w,h)) 85 | gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) 86 | size = gray.shape 87 | 88 | # detect faces in the grayscale frame 89 | rects = detector(gray, 0) 90 | 91 | # check to see if a face was detected, and if so, draw the total 92 | # number of faces on the frame 93 | if len(rects) > 0: 94 | text = "{} face(s) found".format(len(rects)) 95 | cv2.putText(frame, text, (10, 20), cv2.FONT_HERSHEY_SIMPLEX,0.5, (0, 0, 255), 2) 96 | 97 | # loop over the face detections 98 | for rect in rects: 99 | # compute the bounding box of the face and draw it on the 100 | # frame 101 | (bX, bY, bW, bH) = face_utils.rect_to_bb(rect) 102 | cv2.rectangle(frame, (bX, bY), (bX + bW, bY + bH),(0, 255, 0), 1) 103 | # determine the facial landmarks for the face region, then 104 | # convert the facial landmark (x, y)-coordinates to a NumPy 105 | # array 106 | shape = predictor(gray, rect) 107 | shape = face_utils.shape_to_np(shape) 108 | # loop over the (x, y)-coordinates for the facial landmarks 109 | # and draw each of them 110 | for (i, (x, y)) in enumerate(shape): 111 | if i == 33: 112 | #something to our key landmarks 113 | # save to our new key point list 114 | # i.e. keypoints = [(i,(x,y))] 115 | image_points[0] = np.array([x,y],dtype='double') 116 | # write on frame in Green 117 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 118 | #cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 119 | elif i == 8: 120 | #something to our key landmarks 121 | # save to our new key point list 122 | # i.e. keypoints = [(i,(x,y))] 123 | image_points[1] = np.array([x,y],dtype='double') 124 | # write on frame in Green 125 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 126 | #cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 127 | elif i == 36: 128 | #something to our key landmarks 129 | # save to our new key point list 130 | # i.e. keypoints = [(i,(x,y))] 131 | image_points[2] = np.array([x,y],dtype='double') 132 | # write on frame in Green 133 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 134 | #cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 135 | elif i == 45: 136 | #something to our key landmarks 137 | # save to our new key point list 138 | # i.e. keypoints = [(i,(x,y))] 139 | image_points[3] = np.array([x,y],dtype='double') 140 | # write on frame in Green 141 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 142 | #cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 143 | elif i == 48: 144 | #something to our key landmarks 145 | # save to our new key point list 146 | # i.e. keypoints = [(i,(x,y))] 147 | image_points[4] = np.array([x,y],dtype='double') 148 | # write on frame in Green 149 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 150 | #cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 151 | elif i == 53: 152 | #something to our key landmarks 153 | # save to our new key point list 154 | # i.e. keypoints = [(i,(x,y))] 155 | image_points[5] = np.array([x,y],dtype='double') 156 | # write on frame in Green 157 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 158 | #cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 159 | else: 160 | #everything to all other landmarks 161 | # write on frame in Red 162 | pass 163 | #cv2.circle(frame, (x, y), 1, (0, 0, 255), -1) 164 | #cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 0, 255), 1) 165 | focal_length = size[1] 166 | center = (size[1]/2, size[0]/2) 167 | camera_matrix = np.array([[focal_length,0,center[0]],[0, focal_length, center[1]],[0,0,1]], dtype="double") 168 | 169 | #print "Camera Matrix :\n {0}".format(camera_matrix) 170 | 171 | dist_coeffs = np.zeros((4,1)) # Assuming no lens distortion 172 | (success, rotation_vector, translation_vector) = cv2.solvePnP(model_points, image_points, camera_matrix, dist_coeffs, flags=cv2.SOLVEPNP_ITERATIVE)#flags=cv2.CV_ITERATIVE) 173 | 174 | #print "Rotation Vector:\n {0}".format(rotation_vector) 175 | #print "Translation Vector:\n {0}".format(translation_vector) 176 | # Project a 3D point (0, 0 , 1000.0) onto the image plane 177 | # We use this to draw a line sticking out of the nose_end_point2D 178 | (nose_end_point2D, jacobian) = cv2.projectPoints(np.array([(0.0, 0.0, 1000.0)]),rotation_vector, translation_vector, camera_matrix, dist_coeffs) 179 | for p in image_points: 180 | cv2.circle(frame, (int(p[0]), int(p[1])), 3, (0,0,255), -1) 181 | 182 | p1 = ( int(image_points[0][0]), int(image_points[0][1])) 183 | p2 = ( int(nose_end_point2D[0][0][0]), int(nose_end_point2D[0][0][1])) 184 | 185 | cv2.line(frame, p1, p2, (255,0,0), 2) 186 | else: 187 | # show the frame 188 | cv2.imshow("frame", frame) 189 | key = cv2.waitKey(1) & 0xFF 190 | 191 | # clear the stream for the next frame 192 | #rawCapture.truncate(0) 193 | # if the `q` key was pressed, break from the loop 194 | if key == ord("q"): 195 | break 196 | 197 | 198 | count += 1 199 | # show the frame 200 | cv2.imshow("frame", frame) 201 | key = cv2.waitKey(1) & 0xFF 202 | 203 | # clear the stream for the next frame 204 | #rawCapture.truncate(0) 205 | # if the `q` key was pressed, break from the loop 206 | if key == ord("q"): 207 | break 208 | print(image_points) 209 | 210 | # do a bit of cleanup 211 | cv2.destroyAllWindows() 212 | vs.stop() 213 | 214 | 215 | -------------------------------------------------------------------------------- /python/headpose.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # 2018-05-20 I aim to have this run in a while loop getting the camera frame 3 | 4 | from imutils.video import VideoStream 5 | from imutils import face_utils 6 | import argparse 7 | import imutils 8 | import time 9 | import dlib 10 | import cv2 11 | import numpy as np 12 | 13 | # construct the argument parser and parse the arguments 14 | ap = argparse.ArgumentParser() 15 | ap.add_argument("-p", "--shape-predictor", required=True, 16 | help="path to facial landmark predictor") 17 | args = vars(ap.parse_args()) 18 | 19 | # initialize dlib's face detector (HOG-based) and then create the 20 | # facial landmark predictor 21 | print("[INFO] loading facial landmark predictor...") 22 | detector = dlib.get_frontal_face_detector() 23 | predictor = dlib.shape_predictor(args["shape_predictor"]) 24 | 25 | # initialize the video stream and sleep for a bit, allowing the 26 | # camera sensor to warm up 27 | print("[INFO] camera sensor warming up...") 28 | 29 | vs = VideoStream(src=0).start() 30 | # vs = VideoStream(usePiCamera=True).start() # Raspberry Pi 31 | time.sleep(2.0) 32 | 33 | # 400x225 to 1024x576 34 | frame_width = 1024 35 | frame_height = 576 36 | 37 | # Define the codec and create VideoWriter object.The output is stored in 'outpy.avi' file. 38 | #out = cv2.VideoWriter('outpy.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 30, (frame_width,frame_height)) 39 | 40 | # loop over the frames from the video stream 41 | #2D image points. If you change the image, you need to change vector 42 | image_points = np.array([ 43 | (359, 391), # Nose tip 34 44 | (399, 561), # Chin 9 45 | (337, 297), # Left eye left corner 37 46 | (513, 301), # Right eye right corne 46 47 | (345, 465), # Left Mouth corner 49 48 | (453, 469) # Right mouth corner 55 49 | ], dtype="double") 50 | 51 | # 3D model points. 52 | model_points = np.array([ 53 | (0.0, 0.0, 0.0), # Nose tip 34 54 | (0.0, -330.0, -65.0), # Chin 9 55 | (-225.0, 170.0, -135.0), # Left eye left corner 37 56 | (225.0, 170.0, -135.0), # Right eye right corne 46 57 | (-150.0, -150.0, -125.0), # Left Mouth corner 49 58 | (150.0, -150.0, -125.0) # Right mouth corner 55 59 | 60 | ]) 61 | 62 | while True: 63 | # grab the frame from the threaded video stream, resize it to 64 | # have a maximum width of 400 pixels, and convert it to 65 | # grayscale 66 | frame = vs.read() 67 | frame = imutils.resize(frame, width=1024, height=576) 68 | gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) 69 | size = gray.shape 70 | 71 | # detect faces in the grayscale frame 72 | rects = detector(gray, 0) 73 | 74 | # check to see if a face was detected, and if so, draw the total 75 | # number of faces on the frame 76 | if len(rects) > 0: 77 | text = "{} face(s) found".format(len(rects)) 78 | cv2.putText(frame, text, (10, 20), cv2.FONT_HERSHEY_SIMPLEX, 79 | 0.5, (0, 0, 255), 2) 80 | 81 | # loop over the face detections 82 | for rect in rects: 83 | # compute the bounding box of the face and draw it on the 84 | # frame 85 | (bX, bY, bW, bH) = face_utils.rect_to_bb(rect) 86 | cv2.rectangle(frame, (bX, bY), (bX + bW, bY + bH), 87 | (0, 255, 0), 1) 88 | 89 | # determine the facial landmarks for the face region, then 90 | # convert the facial landmark (x, y)-coordinates to a NumPy 91 | # array 92 | shape = predictor(gray, rect) 93 | shape = face_utils.shape_to_np(shape) 94 | # loop over the (x, y)-coordinates for the facial landmarks 95 | # and draw each of them 96 | for (i, (x, y)) in enumerate(shape): 97 | if i == 33: 98 | #something to our key landmarks 99 | # save to our new key point list 100 | # i.e. keypoints = [(i,(x,y))] 101 | image_points[0] = np.array([x,y],dtype='double') 102 | # write on frame in Green 103 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 104 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 105 | elif i == 8: 106 | #something to our key landmarks 107 | # save to our new key point list 108 | # i.e. keypoints = [(i,(x,y))] 109 | image_points[1] = np.array([x,y],dtype='double') 110 | # write on frame in Green 111 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 112 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 113 | elif i == 36: 114 | #something to our key landmarks 115 | # save to our new key point list 116 | # i.e. keypoints = [(i,(x,y))] 117 | image_points[2] = np.array([x,y],dtype='double') 118 | # write on frame in Green 119 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 120 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 121 | elif i == 45: 122 | #something to our key landmarks 123 | # save to our new key point list 124 | # i.e. keypoints = [(i,(x,y))] 125 | image_points[3] = np.array([x,y],dtype='double') 126 | # write on frame in Green 127 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 128 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 129 | elif i == 48: 130 | #something to our key landmarks 131 | # save to our new key point list 132 | # i.e. keypoints = [(i,(x,y))] 133 | image_points[0] = np.array([x,y],dtype='double') 134 | # write on frame in Green 135 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 136 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 137 | elif i == 54: 138 | #something to our key landmarks 139 | # save to our new key point list 140 | # i.e. keypoints = [(i,(x,y))] 141 | image_points[5] = np.array([x,y],dtype='double') 142 | # write on frame in Green 143 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 144 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 145 | else: 146 | #everything to all other landmarks 147 | # write on frame in Red 148 | cv2.circle(frame, (x, y), 1, (0, 0, 255), -1) 149 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 0, 255), 1) 150 | ### TO DO: Take Feature Points I need and save as smaller array 151 | # Write the frame into the file 'output.avi' 152 | #out.write(frame) 153 | ##### 154 | # Add pose estimation 155 | 156 | # camera internals 157 | focal_length = size[1] 158 | center = (size[1]/2, size[0]/2) 159 | camera_matrix = np.array([[focal_length,0,center[0]],[0, focal_length, center[1]],[0,0,1]], dtype="double") 160 | 161 | print "Camera Matrix :\n {0}".format(camera_matrix) 162 | 163 | dist_coeffs = np.zeros((4,1)) # Assuming no lens distortion 164 | (success, rotation_vector, translation_vector) = cv2.solvePnP(model_points, image_points, camera_matrix, dist_coeffs, flags=cv2.SOLVEPNP_ITERATIVE)#flags=cv2.CV_ITERATIVE) 165 | 166 | print "Rotation Vector:\n {0}".format(rotation_vector) 167 | print "Translation Vector:\n {0}".format(translation_vector) 168 | 169 | # Project a 3D point (0, 0 , 1000.0) onto the image plane 170 | # We use this to draw a line sticking out of the nose_end_point2D 171 | (nose_end_point2D, jacobian) = cv2.projectPoints(np.array([(0.0, 0.0, 1000.0)]),rotation_vector, translation_vector, camera_matrix, dist_coeffs) 172 | 173 | for p in image_points: 174 | cv2.circle(frame, (int(p[0]), int(p[1])), 3, (0,0,255), -1) 175 | 176 | p1 = ( int(image_points[0][0]), int(image_points[0][1])) 177 | p2 = ( int(nose_end_point2D[0][0][0]), int(nose_end_point2D[0][0][1])) 178 | 179 | cv2.line(frame, p1, p2, (255,0,0), 2) 180 | # show the frame 181 | cv2.imshow("Frame", frame) 182 | key = cv2.waitKey(1) & 0xFF 183 | 184 | # if the `q` key was pressed, break from the loop 185 | if key == ord("q"): 186 | break 187 | # 188 | print(image_points) 189 | 190 | # do a bit of cleanup 191 | cv2.destroyAllWindows() 192 | vs.stop() 193 | -------------------------------------------------------------------------------- /python/rpicam-pose.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # 2018-05-20 I aim to have this run in a while loop getting the camera frame 3 | 4 | from imutils.video import VideoStream 5 | from imutils import face_utils 6 | import argparse 7 | import imutils 8 | import time 9 | import dlib 10 | import cv2 11 | import numpy as np 12 | from picamera.array import PiRGBArray 13 | from picamera import PiCamera 14 | 15 | # initialize the camera and grab a refeerence to the raw camera capture 16 | camera = PiCamera() 17 | camera.resolution = (640, 480) 18 | camera.framerate = 32 19 | rawCapture = PiRGBArray(camera, size=(640, 480)) 20 | 21 | time.sleep(0.1) 22 | 23 | # construct the argument parser and parse the arguments 24 | ap = argparse.ArgumentParser() 25 | ap.add_argument("-p", "--shape-predictor", required=True, 26 | help="path to facial landmark predictor") 27 | args = vars(ap.parse_args()) 28 | 29 | # initialize dlib's face detector (HOG-based) and then create the 30 | # facial landmark predictor 31 | print("[INFO] loading facial landmark predictor...") 32 | detector = dlib.get_frontal_face_detector() 33 | predictor = dlib.shape_predictor(args["shape_predictor"]) 34 | 35 | # initialize the video stream and sleep for a bit, allowing the 36 | # camera sensor to warm up 37 | print("[INFO] camera sensor warming up...") 38 | 39 | #vs = VideoStream(src=0).start() 40 | # vs = VideoStream(usePiCamera=True).start() # Raspberry Pi 41 | time.sleep(2.0) 42 | 43 | # 400x225 to 1024x576 44 | frame_width = 400 45 | frame_height = 225 46 | 47 | # Define the codec and create VideoWriter object.The output is stored in 'outpy.avi' file. 48 | #out = cv2.VideoWriter('outpy.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 30, (frame_width,frame_height)) 49 | 50 | # loop over the frames from the video stream 51 | #2D image points. If you change the image, you need to change vector 52 | image_points = np.array([ 53 | (259, 291), # Nose tip 34 54 | (399, 261), # Chin 9 55 | (337, 297), # Left eye left corner 37 56 | (513, 301), # Right eye right corne 46 57 | (345, 265), # Left Mouth corner 49 58 | (453, 269) # Right mouth corner 55 59 | ], dtype="double") 60 | 61 | # 3D model points. 62 | model_points = np.array([ 63 | (0.0, 0.0, 0.0), # Nose tip 34 64 | (0.0, -330.0, -65.0), # Chin 9 65 | (-225.0, 170.0, -135.0), # Left eye left corner 37 66 | (225.0, 170.0, -135.0), # Right eye right corne 46 67 | (-150.0, -150.0, -125.0), # Left Mouth corner 49 68 | (150.0, -150.0, -125.0) # Right mouth corner 55 69 | 70 | ]) 71 | 72 | for frame in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True): 73 | image = frame.array 74 | frame = imutils.resize(image, width=frame_width, height=frame_height) 75 | frame = cv2.flip(frame,1) 76 | gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) 77 | size = frame.shape 78 | 79 | # detect faces in the grayscale frame 80 | rects = detector(gray, 0) 81 | 82 | # check to see if a face was detected, and if so, draw the total 83 | # number of faces on the frame 84 | if len(rects) > 0: 85 | text = "{} face(s) found".format(len(rects)) 86 | cv2.putText(frame, text, (10, 20), cv2.FONT_HERSHEY_SIMPLEX,0.5, (0, 0, 255), 2) 87 | 88 | # loop over the face detections 89 | for rect in rects: 90 | # compute the bounding box of the face and draw it on the 91 | # frame 92 | (bX, bY, bW, bH) = face_utils.rect_to_bb(rect) 93 | cv2.rectangle(frame, (bX, bY), (bX + bW, bY + bH),(0, 255, 0), 1) 94 | # determine the facial landmarks for the face region, then 95 | # convert the facial landmark (x, y)-coordinates to a NumPy 96 | # array 97 | shape = predictor(gray, rect) 98 | shape = face_utils.shape_to_np(shape) 99 | # loop over the (x, y)-coordinates for the facial landmarks 100 | # and draw each of them 101 | for (i, (x, y)) in enumerate(shape): 102 | if i == 33: 103 | #something to our key landmarks 104 | # save to our new key point list 105 | # i.e. keypoints = [(i,(x,y))] 106 | image_points[0] = np.array([x,y],dtype='double') 107 | # write on frame in Green 108 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 109 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 110 | elif i == 8: 111 | #something to our key landmarks 112 | # save to our new key point list 113 | # i.e. keypoints = [(i,(x,y))] 114 | image_points[1] = np.array([x,y],dtype='double') 115 | # write on frame in Green 116 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 117 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 118 | elif i == 36: 119 | #something to our key landmarks 120 | # save to our new key point list 121 | # i.e. keypoints = [(i,(x,y))] 122 | image_points[2] = np.array([x,y],dtype='double') 123 | # write on frame in Green 124 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 125 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 126 | elif i == 45: 127 | #something to our key landmarks 128 | # save to our new key point list 129 | # i.e. keypoints = [(i,(x,y))] 130 | image_points[3] = np.array([x,y],dtype='double') 131 | # write on frame in Green 132 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 133 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 134 | elif i == 48: 135 | #something to our key landmarks 136 | # save to our new key point list 137 | # i.e. keypoints = [(i,(x,y))] 138 | image_points[4] = np.array([x,y],dtype='double') 139 | # write on frame in Green 140 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 141 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 142 | elif i == 54: 143 | #something to our key landmarks 144 | # save to our new key point list 145 | # i.e. keypoints = [(i,(x,y))] 146 | image_points[5] = np.array([x,y],dtype='double') 147 | # write on frame in Green 148 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 149 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 150 | else: 151 | #everything to all other landmarks 152 | # write on frame in Red 153 | cv2.circle(frame, (x, y), 1, (0, 0, 255), -1) 154 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 0, 255), 1) 155 | focal_length = size[1] 156 | center = (size[1]/2, size[0]/2) 157 | camera_matrix = np.array([[focal_length,0,center[0]],[0, focal_length, center[1]],[0,0,1]], dtype="double") 158 | 159 | #print "Camera Matrix :\n {0}".format(camera_matrix) 160 | 161 | dist_coeffs = np.zeros((4,1)) # Assuming no lens distortion 162 | (success, rotation_vector, translation_vector) = cv2.solvePnP(model_points, image_points, camera_matrix, dist_coeffs, flags=cv2.SOLVEPNP_ITERATIVE)#flags=cv2.CV_ITERATIVE) 163 | 164 | #print "Rotation Vector:\n {0}".format(rotation_vector) 165 | #print "Translation Vector:\n {0}".format(translation_vector) 166 | # Project a 3D point (0, 0 , 1000.0) onto the image plane 167 | # We use this to draw a line sticking out of the nose_end_point2D 168 | (nose_end_point2D, jacobian) = cv2.projectPoints(np.array([(0.0, 0.0, 1000.0)]),rotation_vector, translation_vector, camera_matrix, dist_coeffs) 169 | for p in image_points: 170 | cv2.circle(frame, (int(p[0]), int(p[1])), 3, (0,0,255), -1) 171 | 172 | p1 = ( int(image_points[0][0]), int(image_points[0][1])) 173 | p2 = ( int(nose_end_point2D[0][0][0]), int(nose_end_point2D[0][0][1])) 174 | 175 | cv2.line(frame, p1, p2, (255,0,0), 2) 176 | 177 | # show the frame 178 | (h,w) = frame.shape[:2] 179 | center = (w / 2, h / 2) 180 | M = cv2.getRotationMatrix2D(center, 180, 1.0) 181 | rotated = cv2.warpAffine(frame, M, (w,h)) 182 | 183 | cv2.imshow("rotated", rotated) 184 | key = cv2.waitKey(1) & 0xFF 185 | 186 | # clear the stream for the next frame 187 | rawCapture.truncate(0) 188 | # if the `q` key was pressed, break from the loop 189 | if key == ord("q"): 190 | break 191 | 192 | print(image_points) 193 | 194 | # do a bit of cleanup 195 | cv2.destroyAllWindows() 196 | #vs.stop() 197 | -------------------------------------------------------------------------------- /python/run-pose.sh: -------------------------------------------------------------------------------- 1 | python3 webcam-headpose.py -p ../models/shape_predictor_68_face_landmarks.dat 2 | -------------------------------------------------------------------------------- /python/save-video-pose.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # 2018-05-20 I aim to have this run in a while loop getting the camera frame 3 | 4 | from imutils.video import VideoStream 5 | from imutils import face_utils 6 | import argparse 7 | import imutils 8 | import time 9 | import dlib 10 | import cv2 11 | import numpy as np 12 | # construct the argument parser and parse the arguments 13 | # construct the argument parser and parse the arguments 14 | ap = argparse.ArgumentParser() 15 | ap.add_argument("-p", "--shape-predictor", required=True, 16 | help="path to facial landmark predictor") 17 | ap.add_argument("-v", "--video", required=True, 18 | help="path to input video file") 19 | args = vars(ap.parse_args()) 20 | # initialize dlib's face detector (HOG-based) and then create the 21 | # facial landmark predictor 22 | print("[INFO] loading facial landmark predictor...") 23 | detector = dlib.get_frontal_face_detector() 24 | predictor = dlib.shape_predictor(args["shape_predictor"]) 25 | 26 | # initialize the video stream and sleep for a bit, allowing the 27 | # camera sensor to warm up 28 | print("[INFO] reading video file and warming up...") 29 | 30 | # Create a VideoCapture object and read from input file 31 | # If the input is the camera, pass 0 instead of the video file name 32 | 33 | cap = cv2.VideoCapture(str(args["video"])) # sample: '../media/obama.mp4' 34 | 35 | # Default resolutions of the frame are obtained.The default resolutions are system dependent. 36 | # We convert the resolutions from float to integer. 37 | frame_width = int(cap.get(3)) 38 | frame_height = int(cap.get(4)) 39 | fps = int(cap.get(5)) 40 | print("input frame width {}".format(frame_width)) 41 | print("input frame height {}".format(frame_height)) 42 | 43 | # 400x225 to 1024x576 44 | frame_width = 1024 45 | frame_height = 576 46 | 47 | print("output frame width {}".format(frame_width)) 48 | print("output frame height {}".format(frame_height)) 49 | # Define the codec and create VideoWriter object.The output is stored in 'outpy.avi' file. 50 | out = cv2.VideoWriter('outpy.avi',cv2.VideoWriter_fourcc('M','J','P','G'), fps, (frame_width,frame_height)) 51 | 52 | time.sleep(2.0) 53 | 54 | 55 | 56 | # Define the codec and create VideoWriter object.The output is stored in 'outpy.avi' file. 57 | #out = cv2.VideoWriter('outpy.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 30, (frame_width,frame_height)) 58 | 59 | # loop over the frames from the video stream 60 | #2D image points. If you change the image, you need to change vector 61 | image_points = np.array([ 62 | (355, 391), # Nose tip 34 63 | (389, 541), # Chin 9 64 | (327, 227), # Left eye left corner 37 65 | (533, 301), # Right eye right corne 46 66 | (345, 465), # Left Mouth corner 49 67 | (455, 415) # Right mouth corner 55 68 | ], dtype="double") 69 | 70 | # 3D model points. 71 | model_points = np.array([ 72 | (0.0, 0.0, 0.0), # Nose tip 34 73 | (0.0, -330.0, -65.0), # Chin 9 74 | (-225.0, 170.0, -135.0), # Left eye left corner 37 75 | (225.0, 170.0, -135.0), # Right eye right corne 46 76 | (-150.0, -150.0, -125.0), # Left Mouth corner 49 77 | (150.0, -150.0, -125.0) # Right mouth corner 55 78 | 79 | ]) 80 | 81 | # Iterating count 82 | i=0 83 | # Read until video is completed 84 | while(cap.isOpened()): 85 | # Capture frame-by-frame 86 | ret, image = cap.read() 87 | if ret == True: 88 | frame = imutils.resize(image, width=frame_width, height=frame_height) 89 | # Convert image to grayscale 90 | gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) 91 | # Process The Image 92 | size = gray.shape 93 | # Detect faces 94 | rects = detector(gray,0) 95 | # check to see if a face was detected, and if so, draw the total 96 | # number of faces on the frame 97 | if len(rects) > 0: 98 | text = "{} face(s) found".format(len(rects)) 99 | cv2.putText(frame, text, (10, 20), cv2.FONT_HERSHEY_SIMPLEX,0.5, (0, 0, 255), 2) 100 | # loop over the face detections 101 | for rect in rects: 102 | # compute the bounding box of the face and draw it on the 103 | # frame 104 | (bX, bY, bW, bH) = face_utils.rect_to_bb(rect) 105 | cv2.rectangle(frame, (bX, bY), (bX + bW, bY + bH),(0, 255, 0), 1) 106 | # determine the facial landmarks for the face region, then 107 | # convert the facial landmark (x, y)-coordinates to a NumPy 108 | # array 109 | shape = predictor(gray, rect) 110 | shape = face_utils.shape_to_np(shape) 111 | # loop over the (x, y)-coordinates for the facial landmarks 112 | # and draw each of them 113 | for (i, (x, y)) in enumerate(shape): 114 | if i == 33: 115 | #something to our key landmarks 116 | # save to our new key point list 117 | # i.e. keypoints = [(i,(x,y))] 118 | image_points[0] = np.array([x,y],dtype='double') 119 | # write on frame in Green 120 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 121 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 122 | elif i == 8: 123 | #something to our key landmarks 124 | # save to our new key point list 125 | # i.e. keypoints = [(i,(x,y))] 126 | image_points[1] = np.array([x,y],dtype='double') 127 | # write on frame in Green 128 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 129 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 130 | elif i == 36: 131 | #something to our key landmarks 132 | # save to our new key point list 133 | # i.e. keypoints = [(i,(x,y))] 134 | image_points[2] = np.array([x,y],dtype='double') 135 | # write on frame in Green 136 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 137 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 138 | elif i == 45: 139 | #something to our key landmarks 140 | # save to our new key point list 141 | # i.e. keypoints = [(i,(x,y))] 142 | image_points[3] = np.array([x,y],dtype='double') 143 | # write on frame in Green 144 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 145 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 146 | elif i == 48: 147 | #something to our key landmarks 148 | # save to our new key point list 149 | # i.e. keypoints = [(i,(x,y))] 150 | image_points[4] = np.array([x,y],dtype='double') 151 | # write on frame in Green 152 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 153 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 154 | elif i == 54: 155 | #something to our key landmarks 156 | # save to our new key point list 157 | # i.e. keypoints = [(i,(x,y))] 158 | image_points[5] = np.array([x,y],dtype='double') 159 | # write on frame in Green 160 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 161 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 162 | else: 163 | #everything to all other landmarks 164 | # write on frame in Red 165 | cv2.circle(frame, (x, y), 1, (0, 0, 255), -1) 166 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 0, 255), 1) 167 | ##### 168 | # Add pose estimation 169 | # camera internals 170 | focal_length = size[1] 171 | center = (size[1]/2, size[0]/2) 172 | camera_matrix = np.array([[focal_length,0,center[0]],[0, focal_length, center[1]],[0,0,1]], dtype="double") 173 | 174 | print "Camera Matrix :\n {0}".format(camera_matrix) 175 | 176 | dist_coeffs = np.zeros((4,1)) # Assuming no lens distortion 177 | (success, rotation_vector, translation_vector) = cv2.solvePnP(model_points, image_points, camera_matrix, dist_coeffs, flags=cv2.SOLVEPNP_ITERATIVE)#flags=cv2.CV_ITERATIVE) 178 | 179 | print "Rotation Vector:\n {0}".format(rotation_vector) 180 | print "Translation Vector:\n {0}".format(translation_vector) 181 | # Project a 3D point (0, 0 , 1000.0) onto the image plane 182 | # We use this to draw a line sticking out of the nose_end_point2D 183 | (nose_end_point2D, jacobian) = cv2.projectPoints(np.array([(0.0, 0.0, 1000.0)]),rotation_vector, translation_vector, camera_matrix, dist_coeffs) 184 | for p in image_points: 185 | cv2.circle(frame, (int(p[0]), int(p[1])), 3, (0,0,255), -1) 186 | 187 | p1 = ( int(image_points[0][0]), int(image_points[0][1])) 188 | p2 = ( int(nose_end_point2D[0][0][0]), int(nose_end_point2D[0][0][1])) 189 | cv2.line(frame, p1, p2, (255,0,0), 2) 190 | 191 | 192 | # Get image dimensions 193 | #height, width = image.shape[:2] 194 | 195 | # Output frame to file 196 | out.write(frame) 197 | # Display image to screen 198 | cv2.imshow('Output Image',frame) 199 | # Wait for key press 0 200 | cv2.waitKey(1) 201 | 202 | # Write the frame into the file 'output.avi' 203 | #out.write(res) 204 | i+=1 205 | 206 | # Press Q on keyboard to exit 207 | # If live camera replace waitKey(25) to waitKey(1) 208 | if cv2.waitKey(25) & 0xFF == ord('q'): 209 | break 210 | # Break the loop 211 | else: 212 | break 213 | 214 | # Remove all windows when finished 215 | out.release() 216 | cv2.destroyAllWindows() 217 | -------------------------------------------------------------------------------- /python/video-headpose.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # 2018-05-20 I aim to have this run in a while loop getting the camera frame 3 | 4 | 5 | ### Warning this file has the the vector being computed for every facial feature point on every frame! 6 | ### This is fixed in the other files 7 | 8 | from imutils.video import VideoStream 9 | from imutils import face_utils 10 | import argparse 11 | import imutils 12 | import time 13 | import dlib 14 | import cv2 15 | import numpy as np 16 | 17 | # construct the argument parser and parse the arguments 18 | ap = argparse.ArgumentParser() 19 | ap.add_argument("-p", "--shape-predictor", required=True, 20 | help="path to facial landmark predictor") 21 | ap.add_argument("-v", "--video", required=True, 22 | help="path to input video file") 23 | args = vars(ap.parse_args()) 24 | 25 | # initialize dlib's face detector (HOG-based) and then create the 26 | # facial landmark predictor 27 | print("[INFO] loading facial landmark predictor...") 28 | detector = dlib.get_frontal_face_detector() 29 | predictor = dlib.shape_predictor(args["shape_predictor"]) 30 | 31 | # initialize the video stream and sleep for a bit, allowing the 32 | # camera sensor to warm up 33 | print("[INFO] camera sensor warming up...") 34 | 35 | vs = VideoStream(src=str(args["video"]), framerate=30).start() 36 | # vs = VideoStream(usePiCamera=True).start() # Raspberry Pi 37 | time.sleep(2.0) 38 | 39 | # 400x225 to 1024x576 40 | frame_width = 1024 41 | frame_height = 576 42 | 43 | # Define the codec and create VideoWriter object.The output is stored in 'outpy.avi' file. 44 | out = cv2.VideoWriter('outpy.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 30, (frame_width,frame_height)) 45 | 46 | # loop over the frames from the video stream 47 | #2D image points. If you change the image, you need to change vector 48 | image_points = np.array([ 49 | (359, 391), # Nose tip 34 50 | (399, 561), # Chin 9 51 | (337, 297), # Left eye left corner 37 52 | (513, 301), # Right eye right corne 46 53 | (345, 465), # Left Mouth corner 49 54 | (453, 469) # Right mouth corner 55 55 | ], dtype="double") 56 | 57 | # 3D model points. 58 | model_points = np.array([ 59 | (0.0, 0.0, 0.0), # Nose tip 34 60 | (0.0, -330.0, -65.0), # Chin 9 61 | (-225.0, 170.0, -135.0), # Left eye left corner 37 62 | (225.0, 170.0, -135.0), # Right eye right corne 46 63 | (-150.0, -150.0, -125.0), # Left Mouth corner 49 64 | (150.0, -150.0, -125.0) # Right mouth corner 55 65 | 66 | ]) 67 | 68 | while True: 69 | # grab the frame from the threaded video stream, resize it to 70 | # have a maximum width of 400 pixels, and convert it to 71 | # grayscale 72 | time.sleep(0.1) 73 | frame = vs.read() 74 | frame = imutils.resize(frame, width=1024, height=576) 75 | gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) 76 | size = gray.shape 77 | 78 | # detect faces in the grayscale frame 79 | rects = detector(gray, 0) 80 | 81 | # check to see if a face was detected, and if so, draw the total 82 | # number of faces on the frame 83 | if len(rects) > 0: 84 | text = "{} face(s) found".format(len(rects)) 85 | cv2.putText(frame, text, (10, 20), cv2.FONT_HERSHEY_SIMPLEX, 86 | 0.5, (0, 0, 255), 2) 87 | 88 | # loop over the face detections 89 | for rect in rects: 90 | # compute the bounding box of the face and draw it on the 91 | # frame 92 | (bX, bY, bW, bH) = face_utils.rect_to_bb(rect) 93 | cv2.rectangle(frame, (bX, bY), (bX + bW, bY + bH), 94 | (0, 255, 0), 1) 95 | 96 | # determine the facial landmarks for the face region, then 97 | # convert the facial landmark (x, y)-coordinates to a NumPy 98 | # array 99 | shape = predictor(gray, rect) 100 | shape = face_utils.shape_to_np(shape) 101 | # loop over the (x, y)-coordinates for the facial landmarks 102 | # and draw each of them 103 | for (i, (x, y)) in enumerate(shape): 104 | if i == 33: 105 | #something to our key landmarks 106 | # save to our new key point list 107 | # i.e. keypoints = [(i,(x,y))] 108 | image_points[0] = np.array([x,y],dtype='double') 109 | # write on frame in Green 110 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 111 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 112 | elif i == 8: 113 | #something to our key landmarks 114 | # save to our new key point list 115 | # i.e. keypoints = [(i,(x,y))] 116 | image_points[1] = np.array([x,y],dtype='double') 117 | # write on frame in Green 118 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 119 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 120 | elif i == 36: 121 | #something to our key landmarks 122 | # save to our new key point list 123 | # i.e. keypoints = [(i,(x,y))] 124 | image_points[2] = np.array([x,y],dtype='double') 125 | # write on frame in Green 126 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 127 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 128 | elif i == 45: 129 | #something to our key landmarks 130 | # save to our new key point list 131 | # i.e. keypoints = [(i,(x,y))] 132 | image_points[3] = np.array([x,y],dtype='double') 133 | # write on frame in Green 134 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 135 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 136 | elif i == 48: 137 | #something to our key landmarks 138 | # save to our new key point list 139 | # i.e. keypoints = [(i,(x,y))] 140 | image_points[4] = np.array([x,y],dtype='double') 141 | # write on frame in Green 142 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 143 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 144 | elif i == 54: 145 | #something to our key landmarks 146 | # save to our new key point list 147 | # i.e. keypoints = [(i,(x,y))] 148 | image_points[5] = np.array([x,y],dtype='double') 149 | # write on frame in Green 150 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 151 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 152 | else: 153 | #everything to all other landmarks 154 | # write on frame in Red 155 | cv2.circle(frame, (x, y), 1, (0, 0, 255), -1) 156 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 0, 255), 1) 157 | 158 | ##### 159 | # Add pose estimation 160 | 161 | # camera internals 162 | focal_length = size[1] 163 | center = (size[1]/2, size[0]/2) 164 | camera_matrix = np.array([[focal_length,0,center[0]],[0, focal_length, center[1]],[0,0,1]], dtype="double") 165 | 166 | print("Camera Matrix :\n {0}".format(camera_matrix)) 167 | 168 | dist_coeffs = np.zeros((4,1)) # Assuming no lens distortion 169 | (success, rotation_vector, translation_vector) = cv2.solvePnP(model_points, image_points, camera_matrix, dist_coeffs, flags=cv2.SOLVEPNP_ITERATIVE)#flags=cv2.CV_ITERATIVE) 170 | 171 | print("Rotation Vector:\n {0}".format(rotation_vector)) 172 | print("Translation Vector:\n {0}".format(translation_vector)) 173 | 174 | # Project a 3D point (0, 0 , 1000.0) onto the image plane 175 | # We use this to draw a line sticking out of the nose_end_point2D 176 | (nose_end_point2D, jacobian) = cv2.projectPoints(np.array([(0.0, 0.0, 1000.0)]),rotation_vector, translation_vector, camera_matrix, dist_coeffs) 177 | 178 | for p in image_points: 179 | cv2.circle(frame, (int(p[0]), int(p[1])), 3, (0,0,255), -1) 180 | 181 | p1 = ( int(image_points[0][0]), int(image_points[0][1])) 182 | p2 = ( int(nose_end_point2D[0][0][0]), int(nose_end_point2D[0][0][1])) 183 | 184 | cv2.line(frame, p1, p2, (255,0,0), 2) 185 | # show the frame 186 | cv2.imshow("Frame", frame) 187 | cv2.waitKey(1) 188 | 189 | # Write the frame into the file 'output.avi' 190 | out.write(frame) 191 | # Press Q on keyboard to exit 192 | # If live camera replace waitKey(25) to waitKey(1) 193 | if cv2.waitKey(25) & 0xFF == ord('q'): 194 | break 195 | print(image_points) 196 | 197 | # do a bit of cleanup 198 | cv2.destroyAllWindows() 199 | vs.stop() 200 | -------------------------------------------------------------------------------- /python/webcam-headpose.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # 2018-05-20 I aim to have this run in a while loop getting the camera frame 3 | 4 | from imutils.video import VideoStream 5 | from imutils import face_utils 6 | import argparse 7 | import imutils 8 | import time 9 | import dlib 10 | import cv2 11 | import numpy as np 12 | 13 | # construct the argument parser and parse the arguments 14 | ap = argparse.ArgumentParser() 15 | ap.add_argument("-p", "--shape-predictor", required=True, 16 | help="path to facial landmark predictor") 17 | args = vars(ap.parse_args()) 18 | 19 | # initialize dlib's face detector (HOG-based) and then create the 20 | # facial landmark predictor 21 | print("[INFO] loading facial landmark predictor...") 22 | detector = dlib.get_frontal_face_detector() 23 | predictor = dlib.shape_predictor(args["shape_predictor"]) 24 | 25 | # initialize the video stream and sleep for a bit, allowing the 26 | # camera sensor to warm up 27 | print("[INFO] camera sensor warming up...") 28 | 29 | vs = VideoStream(src=0).start() 30 | # vs = VideoStream(usePiCamera=True).start() # Raspberry Pi 31 | time.sleep(2.0) 32 | 33 | # 400x225 to 1024x576 34 | frame_width = 1024 35 | frame_height = 576 36 | 37 | # Define the codec and create VideoWriter object.The output is stored in 'outpy.avi' file. 38 | #out = cv2.VideoWriter('outpy.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 30, (frame_width,frame_height)) 39 | 40 | # loop over the frames from the video stream 41 | #2D image points. If you change the image, you need to change vector 42 | image_points = np.array([ 43 | (359, 391), # Nose tip 34 44 | (399, 561), # Chin 9 45 | (337, 297), # Left eye left corner 37 46 | (513, 301), # Right eye right corne 46 47 | (345, 465), # Left Mouth corner 49 48 | (453, 469) # Right mouth corner 55 49 | ], dtype="double") 50 | 51 | # 3D model points. 52 | model_points = np.array([ 53 | (0.0, 0.0, 0.0), # Nose tip 34 54 | (0.0, -330.0, -65.0), # Chin 9 55 | (-225.0, 170.0, -135.0), # Left eye left corner 37 56 | (225.0, 170.0, -135.0), # Right eye right corne 46 57 | (-150.0, -150.0, -125.0), # Left Mouth corner 49 58 | (150.0, -150.0, -125.0) # Right mouth corner 55 59 | 60 | ]) 61 | 62 | while True: 63 | # grab the frame from the threaded video stream, resize it to 64 | # have a maximum width of 400 pixels, and convert it to 65 | # grayscale 66 | frame = vs.read() 67 | frame = imutils.resize(frame, width=1024, height=576) 68 | gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) 69 | size = gray.shape 70 | 71 | # detect faces in the grayscale frame 72 | rects = detector(gray, 0) 73 | 74 | # check to see if a face was detected, and if so, draw the total 75 | # number of faces on the frame 76 | if len(rects) > 0: 77 | text = "{} face(s) found".format(len(rects)) 78 | cv2.putText(frame, text, (10, 20), cv2.FONT_HERSHEY_SIMPLEX,0.5, (0, 0, 255), 2) 79 | 80 | # loop over the face detections 81 | for rect in rects: 82 | # compute the bounding box of the face and draw it on the 83 | # frame 84 | (bX, bY, bW, bH) = face_utils.rect_to_bb(rect) 85 | cv2.rectangle(frame, (bX, bY), (bX + bW, bY + bH),(0, 255, 0), 1) 86 | # determine the facial landmarks for the face region, then 87 | # convert the facial landmark (x, y)-coordinates to a NumPy 88 | # array 89 | shape = predictor(gray, rect) 90 | shape = face_utils.shape_to_np(shape) 91 | # loop over the (x, y)-coordinates for the facial landmarks 92 | # and draw each of them 93 | for (i, (x, y)) in enumerate(shape): 94 | if i == 33: 95 | #something to our key landmarks 96 | # save to our new key point list 97 | # i.e. keypoints = [(i,(x,y))] 98 | image_points[0] = np.array([x,y],dtype='double') 99 | # write on frame in Green 100 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 101 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 102 | elif i == 8: 103 | #something to our key landmarks 104 | # save to our new key point list 105 | # i.e. keypoints = [(i,(x,y))] 106 | image_points[1] = np.array([x,y],dtype='double') 107 | # write on frame in Green 108 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 109 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 110 | elif i == 36: 111 | #something to our key landmarks 112 | # save to our new key point list 113 | # i.e. keypoints = [(i,(x,y))] 114 | image_points[2] = np.array([x,y],dtype='double') 115 | # write on frame in Green 116 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 117 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 118 | elif i == 45: 119 | #something to our key landmarks 120 | # save to our new key point list 121 | # i.e. keypoints = [(i,(x,y))] 122 | image_points[3] = np.array([x,y],dtype='double') 123 | # write on frame in Green 124 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 125 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 126 | elif i == 48: 127 | #something to our key landmarks 128 | # save to our new key point list 129 | # i.e. keypoints = [(i,(x,y))] 130 | image_points[4] = np.array([x,y],dtype='double') 131 | # write on frame in Green 132 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 133 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 134 | elif i == 54: 135 | #something to our key landmarks 136 | # save to our new key point list 137 | # i.e. keypoints = [(i,(x,y))] 138 | image_points[5] = np.array([x,y],dtype='double') 139 | # write on frame in Green 140 | cv2.circle(frame, (x, y), 1, (0, 255, 0), -1) 141 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 255, 0), 1) 142 | else: 143 | #everything to all other landmarks 144 | # write on frame in Red 145 | cv2.circle(frame, (x, y), 1, (0, 0, 255), -1) 146 | cv2.putText(frame, str(i + 1), (x - 10, y - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 0, 255), 1) 147 | focal_length = size[1] 148 | center = (size[1]/2, size[0]/2) 149 | camera_matrix = np.array([[focal_length,0,center[0]],[0, focal_length, center[1]],[0,0,1]], dtype="double") 150 | 151 | #print "Camera Matrix :\n {0}".format(camera_matrix) 152 | 153 | dist_coeffs = np.zeros((4,1)) # Assuming no lens distortion 154 | (success, rotation_vector, translation_vector) = cv2.solvePnP(model_points, image_points, camera_matrix, dist_coeffs, flags=cv2.SOLVEPNP_ITERATIVE)#flags=cv2.CV_ITERATIVE) 155 | 156 | #print "Rotation Vector:\n {0}".format(rotation_vector) 157 | #print "Translation Vector:\n {0}".format(translation_vector) 158 | # Project a 3D point (0, 0 , 1000.0) onto the image plane 159 | # We use this to draw a line sticking out of the nose_end_point2D 160 | (nose_end_point2D, jacobian) = cv2.projectPoints(np.array([(0.0, 0.0, 1000.0)]),rotation_vector, translation_vector, camera_matrix, dist_coeffs) 161 | for p in image_points: 162 | cv2.circle(frame, (int(p[0]), int(p[1])), 3, (0,0,255), -1) 163 | 164 | p1 = ( int(image_points[0][0]), int(image_points[0][1])) 165 | p2 = ( int(nose_end_point2D[0][0][0]), int(nose_end_point2D[0][0][1])) 166 | 167 | cv2.line(frame, p1, p2, (255,0,0), 2) 168 | 169 | # show the frame 170 | cv2.imshow("Frame", frame) 171 | key = cv2.waitKey(1) & 0xFF 172 | 173 | # if the `q` key was pressed, break from the loop 174 | if key == ord("q"): 175 | break 176 | # 177 | print(image_points) 178 | 179 | # do a bit of cleanup 180 | cv2.destroyAllWindows() 181 | vs.stop() 182 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | dlib 2 | numpy 3 | opencv-python 4 | imutils 5 | -------------------------------------------------------------------------------- /run.sh: -------------------------------------------------------------------------------- 1 | python python/save-video-pose.py -p models/shape_predictor_68_face_landmarks.dat -v media/obama.mp4 2 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from subprocess import call 2 | 3 | call(['wget', 'https://github.com/AKSHAYUBHAT/TensorFace/blob/master/openface/models/dlib/shape_predictor_68_face_landmarks.dat']) 4 | call(['mkdir', '-vp', 'models']) 5 | call(['mv', '-v', 'shape_predictor_68_face_landmarks.dat', 'models']) 6 | --------------------------------------------------------------------------------