├── .gitignore
├── LICENSE
├── README.md
├── SnakeGame.py
├── game-control-using-object-tracking-multithreaded.py
├── game-control-using-object-tracking.py
├── images
├── Game-Play.png
└── SnakeGameWelcomeScreen.png
├── object-detection.py
├── object-tracking-direction-detection.py
├── requirements.txt
├── settingsSnakeFun.py
└── sounds
├── appleEatSound.wav
└── bgmusic.mid
/.gitignore:
--------------------------------------------------------------------------------
1 | #Byte-compiled / optimized / DLL files
2 | __pycache__/
3 | *.py[cod]
4 |
5 | #PyInstaller
6 | *.manifest
7 | *.spec
8 | build/
9 | dist/
10 |
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | MIT License
2 |
3 | Copyright (c) 2018 Mohit Singh
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | ## Gesture Controlled Snake Game
2 | This program can be used to play a Snake Game (or like it) by detecting hand gestures to control movement of the snake on the screen. It has been implemented in Python using OpenCV library. Although, the demo shown here refers to playing a Snake Game, it can be used to play any game or control any application using hand gestures only.
3 | This program shall be used to play a Snake Game by detecting hand gestures to control movement of the snake on the screen. It has been implemented in Python using OpenCV library. Although, the demo shown here refers to playing a Snake Game, it can be used to play any game or control any application using hand gestures only.
4 |
5 | 
6 |
7 | Watch on Youtube
8 |
9 | [Gesture Controlled Snake Game Using OpenCV and Python](http://www.youtube.com/watch?v=PE_rgc2K0sg)
10 |
11 | ## Getting Started
12 | ### Prerequisites
13 | The program depends on the following libraries-
14 |
15 | numpy==1.15.2
16 | imutils==0.5.1
17 | PyAutoGUI==0.9.38
18 | opencv_python==3.4.3.18
19 | pygame==1.9.4
20 |
21 | Install the libraries using `pip install -r requirements.txt`
22 |
23 | ### Installing
24 |
25 | 1. Clone the repository in your local computer.
26 | 2. Use `python ` to run specific files, which are described below.
27 |
28 | ## Built With
29 |
30 | - [OpenCV](https://opencv.org/) - The Open Computer Vision Library
31 | - [PyAutoGUI](https://pypi.org/project/PyAutoGUI/) - Cross platform GUI Automation Python Module
32 |
33 | ## Contributing
34 | Please feel free to contribute to the project and Pull Requests are open for everyone willing to improve upon the project. Feel free to provide any suggestions.
35 |
36 | ## License
37 | This project is licensed under the MIT License - see the [LICENSE.md](https://github.com/mohitwildbeast/Gesture-Controlled-Snake-Game/blob/master/LICENSE) file for details.
38 | ## Acknowledgements
39 |
40 | - The inspiration for the project came through PyImageSearch blog's article- [OpenCV Tracking Object in Images](https://www.pyimagesearch.com/2015/09/21/opencv-track-object-movement/) blog post.
41 | - PyAutoGUI helped a lot for keyboard automation tasks.
42 | ## File Contents
43 | The entire project has been made using a bottom up approach, and the project consists of the following files which are described below-
44 |
45 | ## [Object Detection](https://github.com/mohitwildbeast/Gesture-Controlled-Snake-Game/blob/master/object-detection.py)
46 |
47 | > This script detects a object of specified object colour from the webcam video feed. Using OpenCV library for vision tasks and HSV color space for detecting object of given specific color.
48 | >See the demo on Youtube - [ Object Detection and Motion Tracking in OpenCV](https://www.youtube.com/watch?v=mtGBuMlusXQ)
49 |
50 |
51 |
52 | ## [Object Tracking and Direction Detection](https://github.com/mohitwildbeast/Gesture-Controlled-Snake-Game/blob/master/object-tracking-direction-detection.py)
53 |
54 | > This script can detect objects specified by the HSV color and also sense
55 | direction of their movement.
56 |
57 | > See the demo on Youtube - [Object Tracking and Direction Detection using OpenCV](https://www.youtube.com/watch?v=zapq9QT9uwc)
58 |
59 | ## [Game Control Using Object Tracking (Multithreaded Implementation)](https://github.com/mohitwildbeast/Gesture-Controlled-Snake-Game/blob/master/game-control-using-object-tracking-multithreaded.py)
60 | > This script can detect objects specified by the HSV color and also sense the
61 | direction of their movement.Using this script a Snake Game which has been loaded in the repo, can be played. Implemented using OpenCV. Uses seperate thread for reading frames through OpenCV.
62 |
63 | >See the demo on Youtube - [Gesture Controlled Snake Game Playing with OpenCV and Computer Vision](https://www.youtube.com/watch?v=PE_rgc2K0sg)
64 |
65 | ## Snake Game
66 | >The Snake Game present in this video, has been taken from my previous repository [SnakeFun](https://github.com/mohitwildbeast/SnakeFun). The files from the game corrrespond to SnakeFun.py and settingsSnakeFun.py
67 |
68 | >Run the game using the code ```python SnakeFun.py```
69 |
70 |
71 |
--------------------------------------------------------------------------------
/SnakeGame.py:
--------------------------------------------------------------------------------
1 | import random
2 | import pygame
3 | import sys
4 | from pygame.locals import *
5 | from settingsSnakeFun import *
6 |
7 | def main():
8 | global CLOCK, SCREEN, FONT
9 |
10 | pygame.init()
11 | CLOCK = pygame.time.Clock()
12 | SCREEN = pygame.display.set_mode((WINDOWWIDTH, WINDOWHEIGHT))
13 | FONT = pygame.font.Font('freesansbold.ttf', 18)
14 | pygame.display.set_caption('Snake Game')
15 |
16 | showStartScreen()
17 | while True:
18 | pygame.mixer.music.play(-1,0.0)
19 | runGame()
20 | pygame.mixer.music.stop()
21 | showGameOverScreen()
22 |
23 | def runGame():
24 | #Set a random starting point
25 | startx = random.randint(5, CELLWIDTH - 6)
26 | starty = random.randint(5, CELLHEIGHT - 6)
27 | global wormCoords
28 | wormCoords = [{'x' : startx, 'y' : starty}, {'x': startx - 1, 'y':starty}, {'x':startx - 2, 'y':starty}]
29 | direction = RIGHT
30 |
31 | apple = getRandomLocation()
32 |
33 | while True:
34 | for event in pygame.event.get():
35 | if event.type == QUIT:
36 | terminate()
37 | elif event.type == KEYDOWN:
38 | if (event.key == K_LEFT or event.key == K_a) and direction != RIGHT:
39 | direction = LEFT
40 | elif (event.key == K_RIGHT or event.key == K_d) and direction != LEFT:
41 | direction = RIGHT
42 | elif (event.key == K_UP or event.key == K_w) and direction != DOWN:
43 | direction = UP
44 | elif (event.key == K_DOWN or event.key == K_s) and direction != UP:
45 | direction = DOWN
46 | elif event.key == K_ESCAPE:
47 | terminate()
48 | #Collision Detection
49 | #Check Collision with edges
50 | if wormCoords[HEAD]['x'] == -1 or wormCoords[HEAD]['x'] == CELLWIDTH or wormCoords[HEAD]['y'] == -1 or wormCoords[HEAD]['y'] == CELLHEIGHT:
51 | return
52 | #Check Collision with snake's body
53 | for wormBody in wormCoords[1:]:
54 | if wormBody['x'] == wormCoords[HEAD]['x'] and wormBody['y'] == wormCoords[HEAD]['y']:
55 | return
56 | #Check Collision with Apple
57 | if wormCoords[HEAD]['x'] == apple['x'] and wormCoords[HEAD]['y'] == apple['y']:
58 | APPLEEATSOUND.play()
59 | apple = getRandomLocation()
60 | else:
61 | del wormCoords[-1]
62 |
63 | #Moving the Snake
64 | if direction == UP:
65 | newHead = {'x': wormCoords[HEAD]['x'], 'y': wormCoords[HEAD]['y'] - 1}
66 | elif direction == DOWN:
67 | newHead = {'x': wormCoords[HEAD]['x'], 'y': wormCoords[HEAD]['y'] + 1}
68 | elif direction == RIGHT:
69 | newHead = {'x': wormCoords[HEAD]['x'] + 1, 'y': wormCoords[HEAD]['y']}
70 | elif direction == LEFT:
71 | newHead = {'x': wormCoords[HEAD]['x'] - 1, 'y': wormCoords[HEAD]['y']}
72 | wormCoords.insert(0, newHead)
73 |
74 | #Drawing the Screen
75 | SCREEN.fill(BGCOLOR)
76 | drawGrid()
77 | drawWorm(wormCoords)
78 | drawApple(apple)
79 | drawScore((len(wormCoords) - 3) * 10)
80 | pygame.display.update()
81 | CLOCK.tick(FPS)
82 |
83 | def getTotalScore():
84 | return ((len(wormCoords) - 3) * 10)
85 |
86 | def drawPressKeyMsg():
87 | pressKeyText = FONT.render('Press A Key To Play', True, YELLOW)
88 | pressKeyRect = pressKeyText.get_rect()
89 | pressKeyRect.center = (WINDOWWIDTH - 200, WINDOWHEIGHT - 100)
90 | SCREEN.blit(pressKeyText, pressKeyRect)
91 |
92 | def drawSettingsMsg():
93 | SCREEN.blit(SETTINGSBUTTON, (WINDOWWIDTH - SETTINGSBUTTON.get_width(), WINDOWHEIGHT - SETTINGSBUTTON.get_height()))
94 |
95 | def checkForKeyPress():
96 | if len(pygame.event.get(QUIT)) > 0:
97 | terminate()
98 |
99 | keyUpEvents = pygame.event.get(KEYUP)
100 | if len(keyUpEvents) == 0:
101 | return None
102 | if keyUpEvents[0].key == K_ESCAPE:
103 | terminate()
104 | return keyUpEvents[0].key
105 |
106 | def showStartScreen():
107 | titlefont = pygame.font.Font('freesansbold.ttf', 100)
108 | titleText = titlefont.render('SNAKE FUN', True, DARKGREEN)
109 | while True:
110 | SCREEN.fill(BGCOLOR)
111 | titleTextRect = titleText.get_rect()
112 | titleTextRect.center = (WINDOWWIDTH / 2, WINDOWHEIGHT / 2)
113 | SCREEN.blit(titleText, titleTextRect)
114 |
115 | drawPressKeyMsg()
116 | if checkForKeyPress():
117 | pygame.event.get()
118 | return
119 | pygame.display.update()
120 | CLOCK.tick(FPS)
121 |
122 | def terminate():
123 | pygame.quit()
124 | sys.exit()
125 |
126 | def getRandomLocation():
127 | return {'x': random.randint(0, CELLWIDTH - 1), 'y': random.randint(0, CELLHEIGHT - 1)}
128 |
129 | def showGameOverScreen():
130 | gameOverFont = pygame.font.Font('freesansbold.ttf', 100)
131 | gameOverText = gameOverFont.render('Game Over', True, WHITE)
132 | gameOverRect = gameOverText.get_rect()
133 | totalscoreFont = pygame.font.Font('freesansbold.ttf', 40)
134 | totalscoreText = totalscoreFont.render('Total Score: %s' % (getTotalScore()), True, WHITE)
135 | totalscoreRect = totalscoreText.get_rect()
136 | totalscoreRect.midtop = (WINDOWWIDTH/2, 150)
137 | gameOverRect.midtop = (WINDOWWIDTH/2, 30)
138 | SCREEN.fill(BGCOLOR)
139 | SCREEN.blit(gameOverText, gameOverRect)
140 | SCREEN.blit(totalscoreText, totalscoreRect)
141 | drawPressKeyMsg()
142 | pygame.display.update()
143 | pygame.time.wait(1000)
144 | checkForKeyPress()
145 |
146 | while True:
147 | if checkForKeyPress():
148 | pygame.event.get()
149 | return
150 |
151 | def drawScore(score):
152 | scoreText = FONT.render('Score: %s' % (score), True, WHITE)
153 | scoreRect = scoreText.get_rect()
154 | scoreRect.center = (WINDOWWIDTH - 100, 30)
155 | SCREEN.blit(scoreText, scoreRect)
156 |
157 | def drawWorm(wormCoords):
158 | x = wormCoords[HEAD]['x'] * CELLSIZE
159 | y = wormCoords[HEAD]['y'] * CELLSIZE
160 | wormHeadRect = pygame.Rect(x, y, CELLSIZE, CELLSIZE)
161 | pygame.draw.rect(SCREEN, YELLOW, wormHeadRect)
162 |
163 | for coord in wormCoords[1:]:
164 | x = coord['x'] * CELLSIZE
165 | y = coord['y'] * CELLSIZE
166 | wormSegmentRect = pygame.Rect(x, y, CELLSIZE, CELLSIZE)
167 | pygame.draw.rect(SCREEN, GREEN, wormSegmentRect)
168 |
169 | def drawApple(coord):
170 | x = coord['x'] * CELLSIZE
171 | y = coord['y'] * CELLSIZE
172 | appleRect = pygame.Rect(x, y, CELLSIZE, CELLSIZE)
173 | pygame.draw.rect(SCREEN, RED, appleRect)
174 |
175 | def drawGrid():
176 | for x in range(0, WINDOWWIDTH, CELLSIZE):
177 | pygame.draw.line(SCREEN, DARKGRAY, (x, 0), (x, WINDOWHEIGHT))
178 | for y in range(0, WINDOWHEIGHT, CELLSIZE):
179 | pygame.draw.line(SCREEN, DARKGRAY, (0, y), (WINDOWWIDTH, y))
180 |
181 | if __name__ == '__main__':
182 | main()
183 |
--------------------------------------------------------------------------------
/game-control-using-object-tracking-multithreaded.py:
--------------------------------------------------------------------------------
1 | '''This script can detect objects specified by the HSV color and also sense the
2 | direction of their movement.Using this script a Snake Game which has been loaed
3 | in the repo, can be played. Implemented using OpenCV.
4 | Uses seperate thread for reading frames through OpenCV.'''
5 |
6 | #Import necessary modules
7 | import cv2
8 | import imutils
9 | import numpy as np
10 | from collections import deque
11 | import time
12 | import pyautogui
13 | from threading import Thread
14 |
15 | #Class implemeting seperate threading for reading of frames.
16 | class WebcamVideoStream:
17 | def __init__(self):
18 | self.stream = cv2.VideoCapture(0)
19 | self.ret, self.frame = self.stream.read()
20 | self.stopped = False
21 | def start(self):
22 | Thread(target = self.update, args=()).start()
23 | return self
24 | def update(self):
25 | while True:
26 | if self.stopped:
27 | return
28 | self.ret, self.frame = self.stream.read()
29 | def read(self):
30 | return self.frame
31 | def stop(self):
32 | self.stopped = True
33 |
34 | """class VideoShow:
35 | def __init__(self, frame = None):
36 | self.frame = frame
37 | self.stopped = False
38 | def start(self):
39 | while not self.stopped:
40 | cv2.imshow('Game Control Window', self.frame)
41 | if(cv2.waitKey(1) == ord('q')):
42 | self.stopped = True
43 | def stop(self):
44 | self.stopped = True
45 | """
46 | #Define HSV colour range for green colour objects
47 | greenLower = (29, 86, 6)
48 | greenUpper = (64, 255, 255)
49 |
50 | #Used in deque structure to store no. of given buffer points
51 | buffer = 20
52 |
53 | #Used so that pyautogui doesn't click the center of the screen at every frame
54 | flag = 0
55 |
56 | #Points deque structure storing 'buffer' no. of object coordinates
57 | pts = deque(maxlen = buffer)
58 | #Counts the minimum no. of frames to be detected where direction change occurs
59 | counter = 0
60 | #Change in direction is stored in dX, dY
61 | (dX, dY) = (0, 0)
62 | #Variable to store direction string
63 | direction = ''
64 | #Last pressed variable to detect which key was pressed by pyautogui
65 | last_pressed = ''
66 |
67 | #Sleep for 2 seconds to let camera initialize properly.
68 | time.sleep(2)
69 |
70 | #Use pyautogui function to detect width and height of the screen
71 | width,height = pyautogui.size()
72 |
73 | #Start video capture in a seperate thread from main thread.
74 | vs = WebcamVideoStream().start()
75 | #video_shower = VideoShow(vs.read()).start()
76 |
77 | #Click on the centre of the screen, game window should be placed here.
78 | pyautogui.click(int(width/2), int(height/2))
79 |
80 | while True:
81 |
82 | '''game_window = pyautogui.locateOnScreen(r'images\SnakeGameWelcomeScreen.png')
83 | game_window_center = pyautogui.center(game_window)
84 | pyautogui.click(game_window_center)'''
85 |
86 | #Store the readed frame in frame
87 | frame = vs.read()
88 | #Flip the frame to avoid mirroring effect
89 | frame = cv2.flip(frame,1)
90 | #Resize the given frame to a 600*600 window
91 | frame = imutils.resize(frame, width = 600)
92 | #Blur the frame using Gaussian Filter of kernel size 11, to remove excessivve noise
93 | blurred_frame = cv2.GaussianBlur(frame, (5,5), 0)
94 | #Convert the frame to HSV, as HSV allow better segmentation.
95 | hsv_converted_frame = cv2.cvtColor(blurred_frame, cv2.COLOR_BGR2HSV)
96 |
97 | #Create a mask for the frame, showing green values
98 | mask = cv2.inRange(hsv_converted_frame, greenLower, greenUpper)
99 | #Erode the masked output to delete small white dots present in the masked image
100 | mask = cv2.erode(mask, None, iterations = 2)
101 | #Dilate the resultant image to restore our target
102 | mask = cv2.dilate(mask, None, iterations = 2)
103 |
104 | #Find all contours in the masked image
105 | _,cnts,_ = cv2.findContours(mask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
106 |
107 | #Define center of the ball to be detected as None
108 | center = None
109 |
110 | #If any object is detected, then only proceed
111 | if(len(cnts) > 0):
112 | #Find the contour with maximum area
113 | c = max(cnts, key = cv2.contourArea)
114 | #Find the center of the circle, and its radius of the largest detected contour.
115 | ((x, y), radius) = cv2.minEnclosingCircle(c)
116 | #Calculate the centroid of the ball, as we need to draw a circle around it.
117 | M = cv2.moments(c)
118 | center = (int(M['m10'] / M['m00']), int(M['m01'] / M['m00']))
119 |
120 | #Proceed only if a ball of considerable size is detected
121 | if radius > 10:
122 | #Draw circles around the object as well as its centre
123 | cv2.circle(frame, (int(x), int(y)), int(radius), (0,255,255), 2)
124 | cv2.circle(frame, center, 5, (0,255,255), -1)
125 | #Append the detected object in the frame to pts deque structure
126 | pts.appendleft(center)
127 |
128 | #Using numpy arange function for better performance. Loop till all detected points
129 | for i in np.arange(1, len(pts)):
130 | #If no points are detected, move on.
131 | if(pts[i-1] == None or pts[i] == None):
132 | continue
133 |
134 | #If atleast 10 frames have direction change, proceed
135 | if counter >= 10 and i == 1 and pts[-10] is not None:
136 | #Calculate the distance between the current frame and 10th frame before
137 | dX = pts[-10][0] - pts[i][0]
138 | dY = pts[-10][1] - pts[i][1]
139 | (dirX, dirY) = ('', '')
140 |
141 | #If distance is greater than 50 pixels, considerable direction change has occured.
142 | if np.abs(dX) > 50:
143 | dirX = 'West' if np.sign(dX) == 1 else 'East'
144 |
145 | if np.abs(dY) > 50:
146 | dirY = 'North' if np.sign(dY) == 1 else 'South'
147 |
148 | #Set direction variable to the detected direction
149 | direction = dirX if dirX != '' else dirY
150 | #Write the detected direction on the frame.
151 | cv2.putText(frame, direction, (20,40), cv2.FONT_HERSHEY_SIMPLEX, 1, (0,0,255), 3)
152 |
153 | #Draw a trailing red line to depict motion of the object.
154 | thickness = int(np.sqrt(buffer / float(i + 1)) * 2.5)
155 | cv2.line(frame, pts[i - 1], pts[i], (0, 0, 255), thickness)
156 |
157 | #If deteced direction is East, press right button
158 | if direction == 'East':
159 | if last_pressed != 'right':
160 | pyautogui.press('right')
161 | last_pressed = 'right'
162 | print("Right Pressed")
163 | #pyautogui.PAUSE = 2
164 | #If deteced direction is West, press Left button
165 | elif direction == 'West':
166 | if last_pressed != 'left':
167 | pyautogui.press('left')
168 | last_pressed = 'left'
169 | print("Left Pressed")
170 | #pyautogui.PAUSE = 2
171 | #if detected direction is North, press Up key
172 | elif direction == 'North':
173 | if last_pressed != 'up':
174 | last_pressed = 'up'
175 | pyautogui.press('up')
176 | print("Up Pressed")
177 | #pyautogui.PAUSE = 2
178 | #If detected direction is South, press down key
179 | elif direction == 'South':
180 | if last_pressed != 'down':
181 | pyautogui.press('down')
182 | last_pressed = 'down'
183 | print("Down Pressed")
184 | #pyautogui.PAUSE = 2
185 |
186 |
187 | #video_shower.frame = frame
188 | #Show the output frame.
189 | cv2.imshow('Game Control Window', frame)
190 | key = cv2.waitKey(1) & 0xFF
191 | #Update counter as the direction change has been detected.
192 | counter += 1
193 |
194 | #If pyautogui has not clicked on center, click it once to focus on game window.
195 | if (flag == 0):
196 | pyautogui.click(int(width/2), int(height/2))
197 | flag = 1
198 |
199 | #If q is pressed, close the window
200 | if(key == ord('q')):
201 | break
202 | #After all the processing, release webcam and destroy all windows
203 | vs.stop()
204 | cv2.destroyAllWindows()
205 |
--------------------------------------------------------------------------------
/game-control-using-object-tracking.py:
--------------------------------------------------------------------------------
1 | '''This script can detect objects specified by the HSV color and also sense the
2 | direction of their movement.Using this script a Snake Game which has been loaed
3 | in the repo, can be played. Implemented using OpenCV.'''
4 |
5 | #Import necessary modules
6 | import cv2
7 | import imutils
8 | import numpy as np
9 | from collections import deque
10 | import time
11 | import pyautogui
12 |
13 | #Define HSV colour range for green colour objects
14 | greenLower = (29, 86, 6)
15 | greenUpper = (64, 255, 255)
16 |
17 | #Used in deque structure to store no. of given buffer points
18 | buffer = 20
19 |
20 | #Used so that pyautogui doesn't click the center of the screen at every frame
21 | flag = 0
22 |
23 | #Points deque structure storing 'buffer' no. of object coordinates
24 | pts = deque(maxlen = buffer)
25 | #Counts the minimum no. of frames to be detected where direction change occurs
26 | counter = 0
27 | #Change in direction is stored in dX, dY
28 | (dX, dY) = (0, 0)
29 | #Variable to store direction string
30 | direction = ''
31 | #Last pressed variable to detect which key was pressed by pyautogui
32 | last_pressed = ''
33 |
34 | #Start video capture
35 | video_capture = cv2.VideoCapture(0)
36 |
37 | #Sleep for 2 seconds to let camera initialize properly.
38 | time.sleep(2)
39 |
40 | #Use pyautogui function to detect width and height of the screen
41 | width,height = pyautogui.size()
42 |
43 |
44 | #Loop until OpenCV window is not closed
45 | while True:
46 |
47 | '''game_window = pyautogui.locateOnScreen(r'images\SnakeGameWelcomeScreen.png')
48 | game_window_center = pyautogui.center(game_window)
49 | pyautogui.click(game_window_center)'''
50 |
51 |
52 | #Store the readed frame in frame, ret defines return value
53 | ret, frame = video_capture.read()
54 | #Flip the frame to avoid mirroring effect
55 | frame = cv2.flip(frame,1)
56 | #Resize the given frame to a 600*600 window
57 | frame = imutils.resize(frame, width = 600)
58 | #Blur the frame using Gaussian Filter of kernel size 11, to remove excessivve noise
59 | blurred_frame = cv2.GaussianBlur(frame, (11,11), 0)
60 | #Convert the frame to HSV, as HSV allow better segmentation.
61 | hsv_converted_frame = cv2.cvtColor(blurred_frame, cv2.COLOR_BGR2HSV)
62 |
63 | #Create a mask for the frame, showing green values
64 | mask = cv2.inRange(hsv_converted_frame, greenLower, greenUpper)
65 | #Erode the masked output to delete small white dots present in the masked image
66 | mask = cv2.erode(mask, None, iterations = 2)
67 | #Dilate the resultant image to restore our target
68 | mask = cv2.dilate(mask, None, iterations = 2)
69 |
70 | #Find all contours in the masked image
71 | _,cnts,_ = cv2.findContours(mask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
72 |
73 | #Define center of the ball to be detected as None
74 | center = None
75 |
76 | #If any object is detected, then only proceed
77 | if(len(cnts) > 0):
78 | #Find the contour with maximum area
79 | c = max(cnts, key = cv2.contourArea)
80 | #Find the center of the circle, and its radius of the largest detected contour.
81 | ((x, y), radius) = cv2.minEnclosingCircle(c)
82 | #Calculate the centroid of the ball, as we need to draw a circle around it.
83 | M = cv2.moments(c)
84 | center = (int(M['m10'] / M['m00']), int(M['m01'] / M['m00']))
85 |
86 | #Proceed only if a ball of considerable size is detected
87 | if radius > 10:
88 | #Draw circles around the object as well as its centre
89 | cv2.circle(frame, (int(x), int(y)), int(radius), (0,255,255), 2)
90 | cv2.circle(frame, center, 5, (0,255,255), -1)
91 | #Append the detected object in the frame to pts deque structure
92 | pts.appendleft(center)
93 |
94 | #Using numpy arange function for better performance. Loop till all detected points
95 | for i in np.arange(1, len(pts)):
96 | #If no points are detected, move on.
97 | if(pts[i-1] == None or pts[i] == None):
98 | continue
99 |
100 | #If atleast 10 frames have direction change, proceed
101 | if counter >= 10 and i == 1 and pts[-10] is not None:
102 | #Calculate the distance between the current frame and 10th frame before
103 | dX = pts[-10][0] - pts[i][0]
104 | dY = pts[-10][1] - pts[i][1]
105 | (dirX, dirY) = ('', '')
106 |
107 | #If distance is greater than 50 pixels, considerable direction change has occured.
108 | if np.abs(dX) > 50:
109 | dirX = 'West' if np.sign(dX) == 1 else 'East'
110 |
111 | if np.abs(dY) > 50:
112 | dirY = 'North' if np.sign(dY) == 1 else 'South'
113 |
114 | #Set direction variable to the detected direction
115 | direction = dirX if dirX != '' else dirY
116 |
117 | #Draw a trailing red line to depict motion of the object.
118 | thickness = int(np.sqrt(buffer / float(i + 1)) * 2.5)
119 | cv2.line(frame, pts[i - 1], pts[i], (0, 0, 255), thickness)
120 |
121 | #If deteced direction is East, press right button
122 | if direction == 'East':
123 | if last_pressed != 'right':
124 | pyautogui.press('right')
125 | last_pressed = 'right'
126 | print("Right Pressed")
127 | #pyautogui.PAUSE = 2
128 | #If deteced direction is West, press Left button
129 | elif direction == 'West':
130 | if last_pressed != 'left':
131 | pyautogui.press('left')
132 | last_pressed = 'left'
133 | print("Left Pressed")
134 | #pyautogui.PAUSE = 2
135 | #if detected direction is North, press Up key
136 | elif direction == 'North':
137 | if last_pressed != 'up':
138 | last_pressed = 'up'
139 | pyautogui.press('up')
140 | print("Up Pressed")
141 | #pyautogui.PAUSE = 2
142 | #If detected direction is South, press down key
143 | elif direction == 'South':
144 | if last_pressed != 'down':
145 | pyautogui.press('down')
146 | last_pressed = 'down'
147 | print("Down Pressed")
148 | #pyautogui.PAUSE = 2
149 |
150 |
151 | #Write the detected direction on the frame.
152 | cv2.putText(frame, direction, (20,40), cv2.FONT_HERSHEY_SIMPLEX, 1, (0,0,255), 3)
153 |
154 | #Show the output frame.
155 | cv2.imshow('Game Control Window', frame)
156 | key = cv2.waitKey(1) & 0xFF
157 | #Update counter as the direction change has been detected.
158 | counter += 1
159 |
160 | #If pyautogui has not clicked on center, click it once to focus on game window.
161 | if (flag == 0):
162 | #Click on the centre of the screen, game window should be placed here.
163 | pyautogui.click(int(width/2), int(height/2))
164 | flag = 1
165 |
166 | #If q is pressed, close the window
167 | if(key == ord('q')):
168 | break
169 | #After all the processing, release webcam and destroy all windows
170 | video_capture.release()
171 | cv2.destroyAllWindows()
172 |
--------------------------------------------------------------------------------
/images/Game-Play.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/msindev/Gesture-Controlled-Snake-Game/0819c15d9c746fc4d423df6c1ae03ee12e6031c1/images/Game-Play.png
--------------------------------------------------------------------------------
/images/SnakeGameWelcomeScreen.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/msindev/Gesture-Controlled-Snake-Game/0819c15d9c746fc4d423df6c1ae03ee12e6031c1/images/SnakeGameWelcomeScreen.png
--------------------------------------------------------------------------------
/object-detection.py:
--------------------------------------------------------------------------------
1 | ''' This script detects a object of specified object colour from the webcam video feed.
2 | Using OpenCV library for vision tasks and HSV color space for detecting object of given specific color.'''
3 |
4 | #Import necessary modules
5 | import cv2
6 | import imutils
7 | import numpy as np
8 | from collections import deque
9 | import time
10 | import math
11 |
12 | #Define HSV colour range for green colour objects
13 | greenLower = (29, 86, 6)
14 | greenUpper = (64, 255, 255)
15 |
16 | #Used in deque structure to store no. of given buffer points
17 | buffer = 20
18 |
19 | #Points deque structure storing 'buffer' no. of object coordinates
20 | pts = deque(maxlen = buffer)
21 |
22 | #Start video capture
23 | video_capture = cv2.VideoCapture(0)
24 |
25 | #Sleep for 2 seconds to let camera initialize properly.
26 | time.sleep(2)
27 |
28 | #Loop until OpenCV window is not closed
29 | while True:
30 | #Store the readed frame in frame, ret defines return value
31 | ret, frame = video_capture.read()
32 | #Flip the frame to avoid mirroring effect
33 | frame = cv2.flip(frame,1)
34 | #Resize the given frame to a 600*600 window
35 | frame = imutils.resize(frame, width = 600)
36 | #Blur the frame using Gaussian Filter of kernel size 5, to remove excessivve noise
37 | blurred_frame = cv2.GaussianBlur(frame, (5,5), 0)
38 | #Convert the frame to HSV, as HSV allow better segmentation.
39 | hsv_converted_frame = cv2.cvtColor(blurred_frame, cv2.COLOR_BGR2HSV)
40 |
41 | #Create a mask for the frame, showing green values
42 | mask = cv2.inRange(hsv_converted_frame, greenLower, greenUpper)
43 | #Erode the masked output to delete small white dots present in the masked image
44 | mask = cv2.erode(mask, None, iterations = 2)
45 | #Dilate the resultant image to restore our target
46 | mask = cv2.dilate(mask, None, iterations = 2)
47 |
48 | #Display the masked output in a different window
49 | cv2.imshow('Masked Output', mask)
50 |
51 | #Find all contours in the masked image
52 | _,cnts,_ = cv2.findContours(mask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
53 |
54 | #Define center of the ball to be detected as None
55 | center = None
56 |
57 | #If any object is detected, then only proceed
58 | if(len(cnts)) > 0:
59 | #Find the contour with maximum area
60 | c = max(cnts, key = cv2.contourArea)
61 | #Find the center of the circle, and its radius of the largest detected contour.
62 | ((x,y), radius) = cv2.minEnclosingCircle(c)
63 |
64 | #Calculate the centroid of the ball, as we need to draw a circle around it.
65 | M = cv2.moments(c)
66 | center = (int(M['m10'] / M['m00']), int(M['m01'] / M['m00']))
67 |
68 | #Proceed only if a ball of considerable size is detected
69 | if radius > 10:
70 | #Draw circles around the object as well as its centre
71 | cv2.circle(frame, (int(x), int(y)), int(radius), (0,255,255), 2)
72 | cv2.circle(frame, center, 5, (0,255,255), -1)
73 |
74 | #Append the detected object in the frame to pts deque structure
75 | pts.appendleft(center)
76 |
77 | #This function makes the trailing line behind the detected object
78 | for i in range(1, len(pts)):
79 | if pts[i-1] is None or pts[i] is None:
80 | continue
81 |
82 | thickness = int(np.sqrt(buffer / float(i + 1)) * 2.5)
83 | cv2.line(frame, pts[i - 1], pts[i], (0, 0, 255), thickness)
84 |
85 | #Show the output frame
86 | cv2.imshow('Frame', frame)
87 | key = cv2.waitKey(1) & 0xFF
88 |
89 | #If q is pressed, close the window
90 | if(key == ord('q')):
91 | break
92 | #After all the processing, release webcam and destroy all windows
93 | video_capture.release()
94 | cv2.destroyAllWindows()
95 |
--------------------------------------------------------------------------------
/object-tracking-direction-detection.py:
--------------------------------------------------------------------------------
1 | '''This script can detect objects specified by the HSV color and also sense the
2 | direction of their movement. Implemented using OpenCV.'''
3 |
4 | #Import necessary modules
5 | import cv2
6 | import imutils
7 | import numpy as np
8 | from collections import deque
9 | import time
10 |
11 | #Define HSV colour range for green colour objects
12 | greenLower = (29, 86, 6)
13 | greenUpper = (64, 255, 255)
14 |
15 | #Used in deque structure to store no. of given buffer points
16 | buffer = 20
17 |
18 | #Points deque structure storing 'buffer' no. of object coordinates
19 | pts = deque(maxlen = buffer)
20 | #Counts the minimum no. of frames to be detected where direction change occurs
21 | counter = 0
22 | #Change in direction is stored in dX, dY
23 | (dX, dY) = (0, 0)
24 | #Variable to store direction string
25 | direction = ''
26 |
27 | #Start video capture
28 | video_capture = cv2.VideoCapture(0)
29 |
30 | #Sleep for 2 seconds to let camera initialize properly.
31 | time.sleep(2)
32 |
33 | #Loop until OpenCV window is not closed
34 | while True:
35 | #Store the readed frame in frame, ret defines return value
36 | ret, frame = video_capture.read()
37 | #Flip the frame to avoid mirroring effect
38 | frame = cv2.flip(frame,1)
39 | #Resize the given frame to a 600*600 window
40 | frame = imutils.resize(frame, width = 600)
41 | #Blur the frame using Gaussian Filter of kernel size 5, to remove excessivve noise
42 | blurred_frame = cv2.GaussianBlur(frame, (5,5), 0)
43 | #Convert the frame to HSV, as HSV allow better segmentation.
44 | hsv_converted_frame = cv2.cvtColor(blurred_frame, cv2.COLOR_BGR2HSV)
45 |
46 | #Create a mask for the frame, showing green values
47 | mask = cv2.inRange(hsv_converted_frame, greenLower, greenUpper)
48 | #Erode the masked output to delete small white dots present in the masked image
49 | mask = cv2.erode(mask, None, iterations = 2)
50 | #Dilate the resultant image to restore our target
51 | mask = cv2.dilate(mask, None, iterations = 2)
52 |
53 | #Find all contours in the masked image
54 | _,cnts,_ = cv2.findContours(mask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
55 |
56 | #Define center of the ball to be detected as None
57 | center = None
58 |
59 | #If any object is detected, then only proceed
60 | if(len(cnts) > 0):
61 | #Find the contour with maximum area
62 | c = max(cnts, key = cv2.contourArea)
63 | #Find the center of the circle, and its radius of the largest detected contour.
64 | ((x, y), radius) = cv2.minEnclosingCircle(c)
65 | #Calculate the centroid of the ball, as we need to draw a circle around it.
66 | M = cv2.moments(c)
67 | center = (int(M['m10'] / M['m00']), int(M['m01'] / M['m00']))
68 |
69 | #Proceed only if a ball of considerable size is detected
70 | if radius > 10:
71 | #Draw circles around the object as well as its centre
72 | cv2.circle(frame, (int(x), int(y)), int(radius), (0,255,255), 2)
73 | cv2.circle(frame, center, 5, (0,255,255), -1)
74 | #Append the detected object in the frame to pts deque structure
75 | pts.appendleft(center)
76 |
77 | #Using numpy arange function for better performance. Loop till all detected points
78 | for i in np.arange(1, len(pts)):
79 | #If no points are detected, move on.
80 | if(pts[i-1] == None or pts[i] == None):
81 | continue
82 |
83 | #If atleast 10 frames have direction change, proceed
84 | if counter >= 10 and i == 1 and pts[-10] is not None:
85 | #Calculate the distance between the current frame and 10th frame before
86 | dX = pts[-10][0] - pts[i][0]
87 | dY = pts[-10][1] - pts[i][1]
88 | (dirX, dirY) = ('', '')
89 |
90 | #If distance is greater than 100 pixels, considerable direction change has occured.
91 | if np.abs(dX) > 100:
92 | dirX = 'West' if np.sign(dX) == 1 else 'East'
93 |
94 | if np.abs(dY) > 100:
95 | dirY = 'North' if np.sign(dY) == 1 else 'South'
96 |
97 | #Set direction variable to the detected direction
98 | direction = dirX if dirX != '' else dirY
99 |
100 | #Draw a trailing red line to depict motion of the object.
101 | thickness = int(np.sqrt(buffer / float(i + 1)) * 2.5)
102 | cv2.line(frame, pts[i - 1], pts[i], (0, 0, 255), thickness)
103 |
104 | #Write the detected direction on the frame.
105 | cv2.putText(frame, direction, (20,40), cv2.FONT_HERSHEY_SIMPLEX, 1, (0,0,255), 3)
106 |
107 | #Show the output frame.
108 | cv2.imshow('Window- Direction Detection', frame)
109 | key = cv2.waitKey(1) & 0xFF
110 | #Update counter as the direction change has been detected.
111 | counter += 1
112 |
113 | #If q is pressed, close the window
114 | if(key == ord('q')):
115 | break
116 | #After all the processing, release webcam and destroy all windows
117 | video_capture.release()
118 | cv2.destroyAllWindows()
119 |
--------------------------------------------------------------------------------
/requirements.txt:
--------------------------------------------------------------------------------
1 | numpy==1.15.2
2 | imutils==0.5.1
3 | PyAutoGUI==0.9.38
4 | opencv_python==3.4.3.18
5 | pygame==1.9.4
6 |
--------------------------------------------------------------------------------
/settingsSnakeFun.py:
--------------------------------------------------------------------------------
1 | import pygame
2 | from pygame.locals import *
3 |
4 | pygame.init()
5 | pygame.mixer.init()
6 | pygame.font.init()
7 |
8 | FPS = 3
9 | WINDOWWIDTH = 640
10 | WINDOWHEIGHT = 480
11 | CELLSIZE = 20
12 | assert WINDOWHEIGHT % CELLSIZE == 0, "Window Height must be a multiple of Cell Size"
13 | assert WINDOWWIDTH % CELLSIZE == 0, "Window Width must be a multiple of Cell Size"
14 | CELLWIDTH = int(WINDOWWIDTH / CELLSIZE)
15 | CELLHEIGHT = int(WINDOWHEIGHT / CELLSIZE)
16 |
17 | #Colour Codes
18 | # R G B
19 | WHITE = (255, 255, 255)
20 | BLACK = (0, 0, 0)
21 | RED = (255, 0, 0)
22 | GREEN = (0, 255, 0)
23 | DARKGREEN= (0, 155, 0)
24 | DARKGRAY = (40, 40, 40)
25 | YELLOW = (255, 255, 0)
26 |
27 | BGCOLOR = BLACK
28 |
29 | #Control Keys
30 | UP = 'up'
31 | DOWN = 'down'
32 | LEFT = 'left'
33 | RIGHT = 'right'
34 |
35 | HEAD = 0 #Index of the snake's head
36 |
37 | #Game Sounds
38 | APPLEEATSOUND = pygame.mixer.Sound(r"sounds/appleEatSound.wav")
39 | BGMUSIC = pygame.mixer.music.load(r"sounds/bgmusic.mid")
40 |
41 | def levelSelect():
42 | global FPS
43 | if level == "EASY":
44 | FPS = 4
45 | elif level == "MEDIUM":
46 | FPS = 7
47 | elif level == "HARD":
48 | FPS = 10
49 |
--------------------------------------------------------------------------------
/sounds/appleEatSound.wav:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/msindev/Gesture-Controlled-Snake-Game/0819c15d9c746fc4d423df6c1ae03ee12e6031c1/sounds/appleEatSound.wav
--------------------------------------------------------------------------------
/sounds/bgmusic.mid:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/msindev/Gesture-Controlled-Snake-Game/0819c15d9c746fc4d423df6c1ae03ee12e6031c1/sounds/bgmusic.mid
--------------------------------------------------------------------------------