├── .DS_Store ├── LICENSE.txt ├── PyKinect2.pyproj ├── PyKinect2.sln ├── README.md ├── examples ├── PyKinectBodyGame.py ├── __init__.py ├── pointer.png └── right_hand_filtered.png ├── pykinect2 ├── PyKinectRuntime.py ├── PyKinectV2.py └── __init__.py └── setup.py /.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abhirajD/PyKinect/2aed4ff72111f1ec770f8488c820cf84484274f7/.DS_Store -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) Microsoft 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /PyKinect2.pyproj: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | Debug 5 | 2.0 6 | 862156b9-96a7-4f85-bb66-b7ec70418cf7 7 | . 8 | pykinect2/PyKinectBodyGame.py 9 | 10 | 11 | . 12 | . 13 | PyKinect2 14 | PyKinect2 15 | False 16 | 17 | 18 | true 19 | false 20 | 21 | 22 | true 23 | false 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 10.0 35 | $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion)\Python Tools\Microsoft.PythonTools.targets 36 | 37 | 38 | 39 | 42 | 43 | 44 | 45 | 46 | 47 | -------------------------------------------------------------------------------- /PyKinect2.sln: -------------------------------------------------------------------------------- 1 |  2 | Microsoft Visual Studio Solution File, Format Version 12.00 3 | # Visual Studio 14 4 | VisualStudioVersion = 14.0.22609.0 5 | MinimumVisualStudioVersion = 10.0.40219.1 6 | Project("{888888A0-9F3D-457C-B088-3A5042F75D52}") = "PyKinect2", "PyKinect2.pyproj", "{862156B9-96A7-4F85-BB66-B7EC70418CF7}" 7 | EndProject 8 | Global 9 | GlobalSection(SolutionConfigurationPlatforms) = preSolution 10 | Debug|Any CPU = Debug|Any CPU 11 | Release|Any CPU = Release|Any CPU 12 | EndGlobalSection 13 | GlobalSection(ProjectConfigurationPlatforms) = postSolution 14 | {862156B9-96A7-4F85-BB66-B7EC70418CF7}.Debug|Any CPU.ActiveCfg = Debug|Any CPU 15 | {862156B9-96A7-4F85-BB66-B7EC70418CF7}.Release|Any CPU.ActiveCfg = Release|Any CPU 16 | EndGlobalSection 17 | GlobalSection(SolutionProperties) = preSolution 18 | HideSolutionNode = FALSE 19 | EndGlobalSection 20 | EndGlobal 21 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # PyKinect2 2 | 3 | Enables writing Kinect applications, games, and experiences using Python. Inspired by the original [PyKinect project on CodePlex](http://pytools.codeplex.com/wikipage?title=PyKinect). 4 | 5 | Only color, depth, body and body index frames are supported in this version. 6 | PyKinectBodyGame is a sample game. It demonstrates how to use Kinect color and body frames. 7 | 8 | 9 | ## Prerequisites 10 | 11 | The easiest way to get most of the pre-requisites is to use Anaconda which includes NumPy. You'll then need to pip install comtypes. The PyKinectBodyGame sample requires PyGame which needs to be manually installed. 12 | 13 | 1. Download [Anaconda](https://store.continuum.io/cshop/anaconda/) get the 32-bit version. This includes NumPy. 14 | 2. pip install comtypes 15 | 3. Install the [Kinect for Windows SDK v2](http://aka.ms/k4wv2sdk) 16 | 17 | Full List of Dependencies 18 | * [Python 2.7.x or 3.4 and higher](https://www.python.org/) 19 | * [NumPy](http://www.numpy.org/) 20 | * [comtypes](https://github.com/enthought/comtypes/) 21 | * [Kinect for Windows SDK v2](http://aka.ms/k4wv2sdk) 22 | * [Kinect v2 sensor and adapter](http://aka.ms/k4wv2purchase) Note: you can use a Kinect for Xbox One as long as you also have the Kinect Adapter for Windows 23 | * [PyGame](http://www.pygame.org) - for running PyKinectBodyGame sample 24 | 25 | 26 | ![Game](http://www.brekel.com/wp-content/uploads/2013/05/kinect2_5people.jpg) 27 | 28 | 29 | ## Installation 30 | 31 | The package can be installed through pip using the usual means: 32 | ``` 33 | pip install pykinect2 34 | ```` 35 | If you are using a virtual environment, be sure to activate it first. 36 | 37 | For more information, please see https://pip.pypa.io/en/latest/user_guide.html#installing-packages 38 | 39 | 40 | ## Installation (Manual) 41 | 42 | To install the package manually, clone this repository to a local folder and include it in the appropriate python environment. If installing in a virtual environment, be sure to install all required dependencies (above). 43 | 44 | For example: 45 | ``` 46 | cd c:\projects\myproject\env\ 47 | /Scripts/activate.bat 48 | 49 | easy_install -a c:\projects\downloads\PyKinect2 50 | ``` 51 | After installation is complete, you can launch the interactive python shell and `import pykinect2` to ensure everything has been installed properly. 52 | 53 | Core helper classes for working with the Kinect sensor are located in PyKinectRuntime.py. For usage examples, please see /examples/PyKinectBodyGame.py. 54 | -------------------------------------------------------------------------------- /examples/PyKinectBodyGame.py: -------------------------------------------------------------------------------- 1 | from pykinect2 import PyKinectV2 2 | from pykinect2.PyKinectV2 import * 3 | from pykinect2 import PyKinectRuntime 4 | 5 | import ctypes 6 | import _ctypes 7 | import pygame 8 | import sys 9 | 10 | if sys.hexversion >= 0x03000000: 11 | import _thread as thread 12 | else: 13 | import thread 14 | 15 | # colors for drawing different bodies 16 | SKELETON_COLORS = [pygame.color.THECOLORS["red"], 17 | pygame.color.THECOLORS["blue"], 18 | pygame.color.THECOLORS["green"], 19 | pygame.color.THECOLORS["orange"], 20 | pygame.color.THECOLORS["purple"], 21 | pygame.color.THECOLORS["yellow"], 22 | pygame.color.THECOLORS["violet"]] 23 | 24 | 25 | class BodyGameRuntime(object): 26 | def __init__(self): 27 | pygame.init() 28 | 29 | # Used to manage how fast the screen updates 30 | self._clock = pygame.time.Clock() 31 | 32 | # Set the width and height of the screen [width, height] 33 | self._infoObject = pygame.display.Info() 34 | self._screen = pygame.display.set_mode((self._infoObject.current_w >> 1, self._infoObject.current_h >> 1), 35 | pygame.HWSURFACE|pygame.DOUBLEBUF|pygame.RESIZABLE, 32) 36 | 37 | pygame.display.set_caption("Kinect for Windows v2 Body Game") 38 | 39 | # Loop until the user clicks the close button. 40 | self._done = False 41 | 42 | # Used to manage how fast the screen updates 43 | self._clock = pygame.time.Clock() 44 | 45 | # Kinect runtime object, we want only color and body frames 46 | self._kinect = PyKinectRuntime.PyKinectRuntime(PyKinectV2.FrameSourceTypes_Color | PyKinectV2.FrameSourceTypes_Body) 47 | 48 | # back buffer surface for getting Kinect color frames, 32bit color, width and height equal to the Kinect color frame size 49 | self._frame_surface = pygame.Surface((self._kinect.color_frame_desc.Width, self._kinect.color_frame_desc.Height), 0, 32) 50 | 51 | # here we will store skeleton data 52 | self._bodies = None 53 | 54 | 55 | def draw_body_bone(self, joints, jointPoints, color, joint0, joint1): 56 | joint0State = joints[joint0].TrackingState; 57 | joint1State = joints[joint1].TrackingState; 58 | 59 | # both joints are not tracked 60 | if (joint0State == PyKinectV2.TrackingState_NotTracked) or (joint1State == PyKinectV2.TrackingState_NotTracked): 61 | return 62 | 63 | # both joints are not *really* tracked 64 | if (joint0State == PyKinectV2.TrackingState_Inferred) and (joint1State == PyKinectV2.TrackingState_Inferred): 65 | return 66 | 67 | # ok, at least one is good 68 | start = (jointPoints[joint0].x, jointPoints[joint0].y) 69 | end = (jointPoints[joint1].x, jointPoints[joint1].y) 70 | 71 | try: 72 | pygame.draw.line(self._frame_surface, color, start, end, 8) 73 | except: # need to catch it due to possible invalid positions (with inf) 74 | pass 75 | 76 | def draw_body(self, joints, jointPoints, color): 77 | # Torso 78 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_Head, PyKinectV2.JointType_Neck); 79 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_Neck, PyKinectV2.JointType_SpineShoulder); 80 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_SpineShoulder, PyKinectV2.JointType_SpineMid); 81 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_SpineMid, PyKinectV2.JointType_SpineBase); 82 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_SpineShoulder, PyKinectV2.JointType_ShoulderRight); 83 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_SpineShoulder, PyKinectV2.JointType_ShoulderLeft); 84 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_SpineBase, PyKinectV2.JointType_HipRight); 85 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_SpineBase, PyKinectV2.JointType_HipLeft); 86 | 87 | # Right Arm 88 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_ShoulderRight, PyKinectV2.JointType_ElbowRight); 89 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_ElbowRight, PyKinectV2.JointType_WristRight); 90 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_WristRight, PyKinectV2.JointType_HandRight); 91 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_HandRight, PyKinectV2.JointType_HandTipRight); 92 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_WristRight, PyKinectV2.JointType_ThumbRight); 93 | 94 | # Left Arm 95 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_ShoulderLeft, PyKinectV2.JointType_ElbowLeft); 96 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_ElbowLeft, PyKinectV2.JointType_WristLeft); 97 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_WristLeft, PyKinectV2.JointType_HandLeft); 98 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_HandLeft, PyKinectV2.JointType_HandTipLeft); 99 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_WristLeft, PyKinectV2.JointType_ThumbLeft); 100 | 101 | # Right Leg 102 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_HipRight, PyKinectV2.JointType_KneeRight); 103 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_KneeRight, PyKinectV2.JointType_AnkleRight); 104 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_AnkleRight, PyKinectV2.JointType_FootRight); 105 | 106 | # Left Leg 107 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_HipLeft, PyKinectV2.JointType_KneeLeft); 108 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_KneeLeft, PyKinectV2.JointType_AnkleLeft); 109 | self.draw_body_bone(joints, jointPoints, color, PyKinectV2.JointType_AnkleLeft, PyKinectV2.JointType_FootLeft); 110 | 111 | 112 | def draw_color_frame(self, frame, target_surface): 113 | target_surface.lock() 114 | address = self._kinect.surface_as_array(target_surface.get_buffer()) 115 | ctypes.memmove(address, frame.ctypes.data, frame.size) 116 | del address 117 | target_surface.unlock() 118 | 119 | def run(self): 120 | # -------- Main Program Loop ----------- 121 | while not self._done: 122 | # --- Main event loop 123 | for event in pygame.event.get(): # User did something 124 | if event.type == pygame.QUIT: # If user clicked close 125 | self._done = True # Flag that we are done so we exit this loop 126 | 127 | elif event.type == pygame.VIDEORESIZE: # window resized 128 | self._screen = pygame.display.set_mode(event.dict['size'], 129 | pygame.HWSURFACE|pygame.DOUBLEBUF|pygame.RESIZABLE, 32) 130 | 131 | # --- Game logic should go here 132 | 133 | # --- Getting frames and drawing 134 | # --- Woohoo! We've got a color frame! Let's fill out back buffer surface with frame's data 135 | if self._kinect.has_new_color_frame(): 136 | frame = self._kinect.get_last_color_frame() 137 | self.draw_color_frame(frame, self._frame_surface) 138 | frame = None 139 | 140 | # --- Cool! We have a body frame, so can get skeletons 141 | if self._kinect.has_new_body_frame(): 142 | self._bodies = self._kinect.get_last_body_frame() 143 | 144 | # --- draw skeletons to _frame_surface 145 | if self._bodies is not None: 146 | for i in range(0, self._kinect.max_body_count): 147 | body = self._bodies.bodies[i] 148 | if not body.is_tracked: 149 | continue 150 | 151 | joints = body.joints 152 | # convert joint coordinates to color space 153 | joint_points = self._kinect.body_joints_to_color_space(joints) 154 | self.draw_body(joints, joint_points, SKELETON_COLORS[i]) 155 | 156 | # --- copy back buffer surface pixels to the screen, resize it if needed and keep aspect ratio 157 | # --- (screen size may be different from Kinect's color frame size) 158 | h_to_w = float(self._frame_surface.get_height()) / self._frame_surface.get_width() 159 | target_height = int(h_to_w * self._screen.get_width()) 160 | surface_to_draw = pygame.transform.scale(self._frame_surface, (self._screen.get_width(), target_height)); 161 | self._screen.blit(surface_to_draw, (0,0)) 162 | surface_to_draw = None 163 | pygame.display.update() 164 | 165 | # --- Go ahead and update the screen with what we've drawn. 166 | pygame.display.flip() 167 | 168 | # --- Limit to 60 frames per second 169 | self._clock.tick(60) 170 | 171 | # Close our Kinect sensor, close the window and quit. 172 | self._kinect.close() 173 | pygame.quit() 174 | 175 | 176 | __main__ = "Kinect v2 Body Game" 177 | game = BodyGameRuntime(); 178 | game.run(); 179 | 180 | -------------------------------------------------------------------------------- /examples/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abhirajD/PyKinect/2aed4ff72111f1ec770f8488c820cf84484274f7/examples/__init__.py -------------------------------------------------------------------------------- /examples/pointer.png: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | PyKubed/pointer.png at master · abhirajD/PyKubed 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | 107 |
108 | Skip to content 109 |
110 | 111 | 112 | 113 | 114 | 115 | 116 | 117 | 118 | 282 | 283 | 284 | 285 |
286 | 287 |
288 | 289 |
290 |
291 | 292 | 293 | 294 |
295 |
296 |
297 | 298 | 299 | 300 | 301 | 302 | 303 | 304 | 305 |
306 |
307 | 308 |
    309 |
  • 310 |
    311 | 312 |
    313 | 325 | 330 | 331 |
    332 |
    333 |
    334 | 335 | Notifications 336 |
    337 | 338 | 380 | 381 |
    382 |
    383 |
    384 |
    385 |
  • 386 | 387 |
  • 388 | 389 |
    390 |
    391 | 392 | 400 | 404 |
    405 |
    406 | 407 | 415 | 419 |
    420 | 421 |
  • 422 | 423 |
  • 424 |
    425 | 434 |
    435 | 439 |
  • 440 |
441 | 442 |

443 | 444 | /PyKubed 447 | 448 |

449 | 450 |
451 | 452 | 500 | 501 | 502 |
503 | 504 |
505 |
506 | 507 | 508 | Permalink 509 | 510 | 511 | 512 |
513 | 514 |
515 | 521 | 522 |
523 | 524 |
525 |
526 | 527 | Switch branches/tags 528 |
529 | 530 |
531 |
532 | 533 |
534 |
535 | 543 |
544 |
545 | 546 | 574 | 575 |
576 |
577 | 578 | 579 |
580 | 581 |
Nothing to show
582 |
583 | 584 |
585 |
586 |
587 | 588 |
589 | 593 | Find file 594 | 595 | 596 |
597 | 600 |
601 | 602 | 603 | 604 |
605 | 606 | 607 | 3f14818 608 | 609 | Feb 19, 2016 610 | 611 |
612 | @abhirajD 613 | 614 | Reorganized stuff 615 |
616 | 617 |
618 | 622 | 623 |
624 | 625 | 634 |
635 | 636 | 637 |
638 |
639 |
640 | 641 |
642 | Download 643 | History 644 |
645 | 646 | 650 | 651 | 652 | 653 |
654 | 658 |
659 | 660 |
661 | 532 Bytes 662 |
663 |
664 | 665 | 666 | 667 |
668 |
669 | pointer.png 670 |
671 |
672 | 673 |
674 | 675 | 676 | 681 | 682 | 683 |
684 | 685 |
686 | 687 |
688 |
689 | 690 |
691 | 692 | 693 | 718 | 719 | 720 | 721 |
722 | 723 | 726 | You can't perform that action at this time. 727 |
728 | 729 | 730 | 731 | 732 | 733 | 734 | 735 | 736 | 737 | 738 |
739 | 740 | You signed in with another tab or window. Reload to refresh your session. 741 | You signed out in another tab or window. Reload to refresh your session. 742 |
743 | 752 | 753 | 754 | 755 | 756 | 757 | 758 | -------------------------------------------------------------------------------- /examples/right_hand_filtered.png: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | PyKubed/right_hand_filtered.png at master · abhirajD/PyKubed 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | 107 |
108 | Skip to content 109 |
110 | 111 | 112 | 113 | 114 | 115 | 116 | 117 | 118 | 282 | 283 | 284 | 285 |
286 | 287 |
288 | 289 |
290 |
291 | 292 | 293 | 294 |
295 |
296 |
297 | 298 | 299 | 300 | 301 | 302 | 303 | 304 | 305 |
306 |
307 | 308 |
    309 |
  • 310 |
    311 | 312 |
    313 | 325 | 330 | 331 |
    332 |
    333 |
    334 | 335 | Notifications 336 |
    337 | 338 | 380 | 381 |
    382 |
    383 |
    384 |
    385 |
  • 386 | 387 |
  • 388 | 389 |
    390 |
    391 | 392 | 400 | 404 |
    405 |
    406 | 407 | 415 | 419 |
    420 | 421 |
  • 422 | 423 |
  • 424 |
    425 | 434 |
    435 | 439 |
  • 440 |
441 | 442 |

443 | 444 | /PyKubed 447 | 448 |

449 | 450 |
451 | 452 | 500 | 501 | 502 |
503 | 504 |
505 |
506 | 507 | 508 | Permalink 509 | 510 | 511 | 512 |
513 | 514 |
515 | 521 | 522 |
523 | 524 |
525 |
526 | 527 | Switch branches/tags 528 |
529 | 530 |
531 |
532 | 533 |
534 |
535 | 543 |
544 |
545 | 546 | 574 | 575 |
576 |
577 | 578 | 579 |
580 | 581 |
Nothing to show
582 |
583 | 584 |
585 |
586 |
587 | 588 |
589 | 593 | Find file 594 | 595 | 596 |
597 | 600 |
601 | 602 | 603 | 604 |
605 | 606 | 607 | 3f14818 608 | 609 | Feb 19, 2016 610 | 611 |
612 | @abhirajD 613 | 614 | Reorganized stuff 615 |
616 | 617 |
618 | 622 | 623 |
624 | 625 | 634 |
635 | 636 | 637 |
638 |
639 |
640 | 641 |
642 | Download 643 | History 644 |
645 | 646 | 650 | 651 | 652 | 653 |
654 | 658 |
659 | 660 |
661 | 467 Bytes 662 |
663 |
664 | 665 | 666 | 667 |
668 |
669 | right_hand_filtered.png 670 |
671 |
672 | 673 |
674 | 675 | 676 | 681 | 682 | 683 |
684 | 685 |
686 | 687 |
688 |
689 | 690 |
691 | 692 | 693 | 718 | 719 | 720 | 721 |
722 | 723 | 726 | You can't perform that action at this time. 727 |
728 | 729 | 730 | 731 | 732 | 733 | 734 | 735 | 736 | 737 | 738 |
739 | 740 | You signed in with another tab or window. Reload to refresh your session. 741 | You signed out in another tab or window. Reload to refresh your session. 742 |
743 | 752 | 753 | 754 | 755 | 756 | 757 | 758 | -------------------------------------------------------------------------------- /pykinect2/PyKinectRuntime.py: -------------------------------------------------------------------------------- 1 | from pykinect2 import PyKinectV2 2 | from pykinect2.PyKinectV2 import * 3 | 4 | import ctypes 5 | import _ctypes 6 | from _ctypes import COMError 7 | import comtypes 8 | import sys 9 | import numpy 10 | import time 11 | 12 | import importlib 13 | 14 | if sys.hexversion >= 0x03000000: 15 | import _thread as thread 16 | else: 17 | import thread 18 | 19 | KINECT_MAX_BODY_COUNT = 6 20 | 21 | class PyKinectRuntime(object): 22 | """manages Kinect objects and simplifying access to them""" 23 | def __init__(self, frame_source_types): 24 | # recipe to get address of surface: http://archives.seul.org/pygame/users/Apr-2008/msg00218.html 25 | is_64bits = sys.maxsize > 2**32 26 | if not is_64bits: 27 | self.Py_ssize_t = ctypes.c_int 28 | else: 29 | self.Py_ssize_t = ctypes.c_int64 30 | 31 | self._PyObject_AsWriteBuffer = ctypes.pythonapi.PyObject_AsWriteBuffer 32 | self._PyObject_AsWriteBuffer.restype = ctypes.c_int 33 | self._PyObject_AsWriteBuffer.argtypes = [ctypes.py_object, 34 | ctypes.POINTER(ctypes.c_void_p), 35 | ctypes.POINTER(self.Py_ssize_t)] 36 | 37 | #self._color_frame_ready = PyKinectV2._event() 38 | #self._depth_frame_ready = PyKinectV2._event() 39 | #self._body_frame_ready = PyKinectV2._event() 40 | #self._body_index_frame_ready = PyKinectV2._event() 41 | #self._infrared_frame_ready = PyKinectV2._event() 42 | #self._long_exposure_infrared_frame_ready = PyKinectV2._event() 43 | #self._audio_frame_ready = PyKinectV2._event() 44 | 45 | self._close_event = ctypes.windll.kernel32.CreateEventW(None, False, False, None) 46 | 47 | self._color_frame_arrived_event = 0 48 | self._depth_frame_arrived_event = 0 49 | self._body_frame_arrived_event = 0 50 | self._body_index_frame_arrived_event = 0 51 | self._infrared_frame_arrived_event = 0 52 | self._long_exposure_infrared_frame_arrived_event = 0 53 | self._audio_frame_arrived_event = 0 54 | 55 | self._color_frame_lock = thread.allocate() 56 | self._depth_frame_lock = thread.allocate() 57 | self._body_frame_lock = thread.allocate() 58 | self._body_index_frame_lock = thread.allocate() 59 | self._infrared_frame_lock = thread.allocate() 60 | self._long_exposure_infrared_frame_lock = thread.allocate() 61 | self._audio_frame_lock = thread.allocate() 62 | 63 | #initialize sensor 64 | self._sensor = ctypes.POINTER(PyKinectV2.IKinectSensor)() 65 | hres = ctypes.windll.kinect20.GetDefaultKinectSensor(ctypes.byref(self._sensor)) 66 | hres = self._sensor.Open() 67 | 68 | self._mapper = self._sensor.CoordinateMapper 69 | 70 | self.frame_source_types = frame_source_types 71 | self.max_body_count = KINECT_MAX_BODY_COUNT 72 | 73 | self._handles = (ctypes.c_voidp * 8)() 74 | self._handles[0] = self._close_event 75 | self._handles[1] = self._close_event 76 | self._handles[2] = self._close_event 77 | self._handles[3] = self._close_event 78 | self._handles[4] = self._close_event 79 | self._handles[5] = self._close_event 80 | self._handles[6] = self._close_event 81 | self._handles[7] = self._close_event 82 | 83 | self._waitHandleCount = 1 84 | 85 | self._color_source = self._sensor.ColorFrameSource 86 | self.color_frame_desc = self._color_source.FrameDescription 87 | self._depth_source = self._sensor.DepthFrameSource 88 | self.depth_frame_desc = self._depth_source.FrameDescription 89 | self._body_index_source = self._sensor.BodyIndexFrameSource 90 | self.body_index_frame_desc = self._body_index_source.FrameDescription 91 | self._body_source = self._sensor.BodyFrameSource 92 | self._body_frame_data = ctypes.POINTER(ctypes.POINTER(IBody)) 93 | self.max_body_count = self._body_source.BodyCount 94 | 95 | self._color_frame_data = None 96 | self._depth_frame_data = None 97 | self._body_frame_data = None 98 | self._body_index_frame_data = None 99 | self._infrared_frame_data = None 100 | self._long_exposure_infrared_frame_data = None 101 | self._audio_frame_data = None 102 | 103 | if(self.frame_source_types & FrameSourceTypes_Color): 104 | self._color_frame_data = ctypes.POINTER(ctypes.c_ubyte) 105 | self._color_frame_data_capacity = ctypes.c_uint(self.color_frame_desc.Width * self.color_frame_desc.Height * 4) 106 | self._color_frame_data_type = ctypes.c_ubyte * self._color_frame_data_capacity.value 107 | self._color_frame_data = ctypes.cast(self._color_frame_data_type(), ctypes.POINTER(ctypes.c_ubyte)) 108 | self._color_frame_reader = self._color_source.OpenReader() 109 | self._color_frame_arrived_event = self._color_frame_reader.SubscribeFrameArrived() 110 | self._handles[self._waitHandleCount] = self._color_frame_arrived_event 111 | self._waitHandleCount += 1 112 | 113 | if(self.frame_source_types & FrameSourceTypes_Depth): 114 | self._depth_frame_data = ctypes.POINTER(ctypes.c_ushort) 115 | self._depth_frame_data_capacity = ctypes.c_uint(self.depth_frame_desc.Width * self.depth_frame_desc.Height) 116 | self._depth_frame_data_type = ctypes.c_ushort * self._depth_frame_data_capacity.value 117 | self._depth_frame_data = ctypes.cast(self._depth_frame_data_type(), ctypes.POINTER(ctypes.c_ushort)) 118 | self._depth_frame_reader = self._depth_source.OpenReader() 119 | self._depth_frame_arrived_event = self._depth_frame_reader.SubscribeFrameArrived() 120 | self._handles[self._waitHandleCount] = self._depth_frame_arrived_event 121 | self._waitHandleCount += 1 122 | 123 | if(self.frame_source_types & FrameSourceTypes_BodyIndex): 124 | self._body_index_frame_data = ctypes.POINTER(ctypes.c_ubyte) 125 | self._body_index_frame_data_capacity = ctypes.c_uint(self.body_index_frame_desc.Width * self.body_index_frame_desc.Height) 126 | self._body_index_frame_data_type = ctypes.c_ubyte * self._body_index_frame_data_capacity.value 127 | self._body_index_frame_data = ctypes.cast(self._body_index_frame_data_type(), ctypes.POINTER(ctypes.c_ubyte)) 128 | self._body_index_frame_reader = self._body_index_source.OpenReader() 129 | self._body_index_frame_arrived_event = self._body_index_frame_reader.SubscribeFrameArrived() 130 | self._handles[self._waitHandleCount] = self._body_index_frame_arrived_event 131 | self._waitHandleCount += 1 132 | 133 | self._body_frame_data = None 134 | if(self.frame_source_types & FrameSourceTypes_Body): 135 | self._body_frame_data_capacity = ctypes.c_uint(self.max_body_count) 136 | self._body_frame_data_type = ctypes.POINTER(IBody) * self._body_frame_data_capacity.value 137 | self._body_frame_data = ctypes.cast(self._body_frame_data_type(), ctypes.POINTER(ctypes.POINTER(IBody))) 138 | self._body_frame_reader = self._body_source.OpenReader() 139 | self._body_frame_arrived_event = self._body_frame_reader.SubscribeFrameArrived() 140 | self._body_frame_bodies = None 141 | self._handles[self._waitHandleCount] = self._body_frame_arrived_event 142 | self._waitHandleCount += 1 143 | 144 | thread.start_new_thread(self.kinect_frame_thread, ()) 145 | 146 | self._last_color_frame = None 147 | self._last_depth_frame = None 148 | self._last_body_frame = None 149 | self._last_body_index_frame = None 150 | self._last_infrared_frame = None 151 | self._last_long_exposure_infrared_frame = None 152 | self._last_audio_frame = None 153 | 154 | start_clock = time.clock() 155 | self._last_color_frame_access = self._last_color_frame_time = start_clock 156 | self._last_body_frame_access = self._last_body_frame_time = start_clock 157 | self._last_body_index_frame_access = self._last_body_index_frame_time = start_clock 158 | self._last_depth_frame_access = self._last_depth_frame_time = start_clock 159 | self._last_infrared_frame_access = self._last_infrared_frame_time = start_clock 160 | self._last_long_exposure_infrared_frame_access = self._last_long_exposure_infrared_frame_time = start_clock 161 | self._last_audio_frame_access = self._last_audio_frame_time = start_clock 162 | 163 | def close(self): 164 | if self._sensor is not None: 165 | ctypes.windll.kernel32.SetEvent(self._close_event) 166 | ctypes.windll.kernel32.CloseHandle(self._close_event) 167 | 168 | self._color_frame_reader = None 169 | self._depth_frame_reader = None 170 | self._body_index_frame_reader = None 171 | self._body_frame_reader = None 172 | 173 | self._color_source = None 174 | self._depth_source = None 175 | self._body_index_source = None 176 | self._body_source = None 177 | 178 | self._body_frame_data = None 179 | 180 | self._sensor.Close() 181 | self._sensor = None 182 | 183 | def __del__(self): 184 | self.close() 185 | 186 | def __enter__(self): 187 | return self 188 | 189 | def __exit__(self, *args): 190 | self.close() 191 | 192 | def surface_as_array(self, surface_buffer_interface): 193 | address = ctypes.c_void_p() 194 | size = self.Py_ssize_t() 195 | self._PyObject_AsWriteBuffer(surface_buffer_interface, 196 | ctypes.byref(address), ctypes.byref(size)) 197 | bytes = (ctypes.c_byte * size.value).from_address(address.value) 198 | bytes.object = surface_buffer_interface 199 | return bytes 200 | 201 | def has_new_color_frame(self): 202 | has = (self._last_color_frame_time > self._last_color_frame_access) 203 | return has 204 | 205 | def has_new_depth_frame(self): 206 | has = (self._last_depth_frame_time > self._last_depth_frame_access) 207 | return has 208 | 209 | def has_new_body_frame(self): 210 | has = (self._last_body_frame_time > self._last_body_frame_access) 211 | return has 212 | 213 | def has_new_body_index_frame(self): 214 | has = (self._last_body_index_frame_time > self._last_body_index_frame_access) 215 | return has 216 | 217 | def has_new_infrared_frame(self): 218 | has = (self._last_infrared_frame_time > self._last_infrared_frame_access) 219 | return has 220 | 221 | def has_new_long_exposure_infrared_frame(self): 222 | has = (self._last_long_exposure_infrared_frame_time > self._last_long_exposure_infrared_frame_access) 223 | return has 224 | 225 | def has_new_audio_frame(self): 226 | has = (self._last_audio_frame_time > self._last_audio_frame_access) 227 | return has 228 | 229 | 230 | def get_last_color_frame(self): 231 | with self._color_frame_lock: 232 | if self._color_frame_data is not None: 233 | data = numpy.copy(numpy.ctypeslib.as_array(self._color_frame_data, shape=(self._color_frame_data_capacity.value,))) 234 | _last_color_frame_access = time.clock() 235 | return data 236 | else: 237 | return None 238 | 239 | def get_last_depth_frame(self): 240 | with self._depth_frame_lock: 241 | if self._depth_frame_data is not None: 242 | data = numpy.copy(numpy.ctypeslib.as_array(self._depth_frame_data, shape=(self._depth_frame_data_capacity.value,))) 243 | _last_color_frame_access = time.clock() 244 | return data 245 | else: 246 | return None 247 | 248 | def get_last_body_index_frame(self): 249 | with self._body_index_frame_lock: 250 | if self._body_index_frame_data is not None: 251 | data = numpy.copy(numpy.ctypeslib.as_array(self._body_index_frame_data, shape=(self._body_index_frame_data_capacity.value,))) 252 | _last_color_frame_access = time.clock() 253 | return data 254 | else: 255 | return None 256 | 257 | def get_last_body_frame(self): 258 | with self._body_frame_lock: 259 | if self._body_frame_bodies is not None: 260 | _last_body_frame_access = time.clock() 261 | return self._body_frame_bodies.copy() 262 | else: 263 | return None 264 | 265 | 266 | def body_joint_to_color_space(self, joint): 267 | return self._mapper.MapCameraPointToColorSpace(joint.Position) 268 | 269 | 270 | def body_joints_to_color_space(self, joints): 271 | joint_points = numpy.ndarray((PyKinectV2.JointType_Count), dtype=numpy.object) 272 | 273 | for j in range(0, PyKinectV2.JointType_Count): 274 | joint_points[j] = self.body_joint_to_color_space(joints[j]) 275 | 276 | return joint_points 277 | 278 | 279 | def kinect_frame_thread(self): 280 | while 1: 281 | wait = ctypes.windll.kernel32.WaitForMultipleObjects(self._waitHandleCount, self._handles, False, PyKinectV2._INFINITE) 282 | 283 | if wait == 0: 284 | break 285 | 286 | if self._handles[wait] == self._color_frame_arrived_event: 287 | self.handle_color_arrived(wait) 288 | elif self._handles[wait] == self._depth_frame_arrived_event: 289 | self.handle_depth_arrived(wait) 290 | elif self._handles[wait] == self._body_frame_arrived_event: 291 | self.handle_body_arrived(wait) 292 | elif self._handles[wait] == self._body_index_frame_arrived_event: 293 | self.handle_body_index_arrived(wait) 294 | elif self._handles[wait] == self._infrared_frame_arrived_event: 295 | self.handle_infrared_arrived(wait) 296 | elif self._handles[wait] == self._long_exposure_infrared_frame_arrived_event: 297 | self.handle_long_exposure_infrared_arrived(wait) 298 | elif self._handles[wait] == self._audio_frame_arrived_event: 299 | self.handle_audio_arrived(wait) 300 | else: 301 | break 302 | 303 | 304 | def handle_color_arrived(self, handle_index): 305 | colorFrameEventData = self._color_frame_reader.GetFrameArrivedEventData(self._handles[handle_index]) 306 | colorFrameRef = colorFrameEventData.FrameReference 307 | try: 308 | colorFrame = colorFrameRef.AcquireFrame() 309 | try: 310 | with self._color_frame_lock: 311 | colorFrame.CopyConvertedFrameDataToArray(self._color_frame_data_capacity, self._color_frame_data, PyKinectV2.ColorImageFormat_Bgra) 312 | self._last_color_frame_time = time.clock() 313 | except: 314 | pass 315 | colorFrame = None 316 | except: 317 | pass 318 | colorFrameRef = None 319 | colorFrameEventData = None 320 | 321 | 322 | def handle_depth_arrived(self, handle_index): 323 | depthFrameEventData = self._depth_frame_reader.GetFrameArrivedEventData(self._handles[handle_index]) 324 | depthFrameRef = depthFrameEventData.FrameReference 325 | try: 326 | depthFrame = depthFrameRef.AcquireFrame() 327 | try: 328 | with self._depth_frame_lock: 329 | depthFrame.CopyFrameDataToArray(self._depth_frame_data_capacity, self._depth_frame_data) 330 | self._last_depth_frame_time = time.clock() 331 | except: 332 | pass 333 | depthFrame = None 334 | except: 335 | pass 336 | depthFrameRef = None 337 | depthFrameEventData = None 338 | 339 | 340 | def handle_body_arrived(self, handle_index): 341 | bodyFrameEventData = self._body_frame_reader.GetFrameArrivedEventData(self._handles[handle_index]) 342 | bofyFrameRef = bodyFrameEventData.FrameReference 343 | try: 344 | bodyFrame = bofyFrameRef.AcquireFrame() 345 | 346 | try: 347 | with self._body_frame_lock: 348 | bodyFrame.GetAndRefreshBodyData(self._body_frame_data_capacity, self._body_frame_data) 349 | self._body_frame_bodies = KinectBodyFrameData(bodyFrame, self._body_frame_data, self.max_body_count) 350 | self._last_body_frame_time = time.clock() 351 | 352 | # need these 2 lines as a workaround for handling IBody referencing exception 353 | self._body_frame_data = None 354 | self._body_frame_data = ctypes.cast(self._body_frame_data_type(), ctypes.POINTER(ctypes.POINTER(IBody))) 355 | 356 | except: 357 | pass 358 | 359 | bodyFrame = None 360 | except: 361 | pass 362 | bofyFrameRef = None 363 | bodyFrameEventData = None 364 | 365 | 366 | def handle_body_index_arrived(self, handle_index): 367 | bodyIndexFrameEventData = self._body_index_frame_reader.GetFrameArrivedEventData(self._handles[handle_index]) 368 | bodyIndexFrameRef = bodyIndexFrameEventData.FrameReference 369 | try: 370 | bodyIndexFrame = bodyIndexFrameRef.AcquireFrame() 371 | try: 372 | with self._body_index_frame_lock: 373 | bodyIndexFrame.CopyFrameDataToArray(self._body_index_frame_data_capacity, self._body_index_frame_data) 374 | self._last_body_index_frame_time = time.clock() 375 | except: 376 | pass 377 | bodyIndexFrame = None 378 | except: 379 | pass 380 | bodyIndexFrame = None 381 | bodyIndexFrameEventData = None 382 | 383 | def handle_infrared_arrived(self, handle_index): 384 | pass 385 | 386 | def handle_long_exposure_infrared_arrived(self, handle_index): 387 | pass 388 | 389 | def handle_audio_arrived(self, handle_index): 390 | pass 391 | 392 | 393 | 394 | class KinectBody(object): 395 | def __init__(self, body = None): 396 | self.is_restricted = False 397 | self.tracking_id = -1 398 | 399 | self.is_tracked = False 400 | 401 | if body is not None: 402 | self.is_tracked = body.IsTracked 403 | 404 | if self.is_tracked: 405 | self.is_restricted = body.IsRestricted 406 | self.tracking_id = body.TrackingId 407 | self.engaged = body.Engaged 408 | self.lean = body.Lean 409 | self.lean_tracking_state = body.LeanTrackingState 410 | self.hand_left_state = body.HandLeftState 411 | self.hand_left_confidence = body.HandLeftConfidence 412 | self.hand_right_state = body.HandRightState 413 | self.hand_right_confidence = body.HandRightConfidence 414 | self.clipped_edges = body.ClippedEdges 415 | 416 | joints = ctypes.POINTER(PyKinectV2._Joint) 417 | joints_capacity = ctypes.c_uint(PyKinectV2.JointType_Count) 418 | joints_data_type = PyKinectV2._Joint * joints_capacity.value 419 | joints = ctypes.cast(joints_data_type(), ctypes.POINTER(PyKinectV2._Joint)) 420 | body.GetJoints(PyKinectV2.JointType_Count, joints) 421 | self.joints = joints 422 | 423 | joint_orientations = ctypes.POINTER(PyKinectV2._JointOrientation) 424 | joint_orientations_data_type = PyKinectV2._JointOrientation * joints_capacity.value 425 | joint_orientations = ctypes.cast(joint_orientations_data_type(), ctypes.POINTER(PyKinectV2._JointOrientation)) 426 | body.GetJointOrientations(PyKinectV2.JointType_Count, joint_orientations) 427 | self.joint_orientations = joint_orientations 428 | 429 | 430 | class KinectBodyFrameData(object): 431 | def __init__(self, bodyFrame, body_frame_data, max_body_count): 432 | self.bodies = None 433 | self.floor_clip_plane = None 434 | if bodyFrame is not None: 435 | self.floor_clip_plane = bodyFrame.FloorClipPlane 436 | 437 | self.bodies = numpy.ndarray((max_body_count), dtype=numpy.object) 438 | for i in range(0, max_body_count): 439 | self.bodies[i] = KinectBody(body_frame_data[i]) 440 | 441 | def copy(self): 442 | res = KinectBodyFrameData(None, None, 0) 443 | res.floor_clip_plane = self.floor_clip_plane 444 | res.bodies = numpy.copy(self.bodies) 445 | return res 446 | 447 | 448 | -------------------------------------------------------------------------------- /pykinect2/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abhirajD/PyKinect/2aed4ff72111f1ec770f8488c820cf84484274f7/pykinect2/__init__.py -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup, find_packages 2 | 3 | setup(name='pykinect2', 4 | version='0.1.0', 5 | description='Wrapper to expose Kinect for Windows v2 API in Python', 6 | license='MIT', 7 | author='Microsoft Corporation', 8 | author_email='k4w@microsoft.com', 9 | url='https://github.com/Kinect/PyKinect2/', 10 | classifiers=[ 11 | 'Development Status :: 4 - Beta', 12 | 'Programming Language :: Python', 13 | 'Programming Language :: Python :: 2.7', 14 | 'Programming Language :: Python :: 3.4', 15 | 'License :: OSI Approved :: MIT License'], 16 | packages=find_packages(), 17 | install_requires=['numpy>=1.9.2', 18 | 'comtypes>=1.1.1'] 19 | ) 20 | --------------------------------------------------------------------------------