35 | Minimal tutorials on how to use a Kinect in C++, with OpenGL (GLUT or SDL).
36 | These tutorials are for the v1.8 SDK, which is compatible with the structured
37 | light Kinect sensors: The XBox 360 Kinect and the Kinect for Windows.
38 |
55 | Minimal tutorials on how to use a Kinect in C++, with OpenGL (GLUT or SDL).
56 | These tutorials are for the v2 SDK, which is compatible with the time-of-flight
57 | Kinect sensor, i.e. the XBox One Kinect.
58 |
73 | While playing with various device APIs, I found that many of
74 | the tools I was using were poorly supported. As a reference, I wrote
75 | up a set of tutorials. My guiding principle is focus.
76 |
77 |
78 | Explanation only where necessary - I will take
79 | time to explain the stuff the tutorial is on - how to use the API in
80 | question. However, I will only give a high-level overview of the
81 | remaining code, for windowing and display.
82 | If you want to learn more about e.g.
83 | OpenGL or C++, there are plenty of tutorials that will focus on
84 | those aspects. I won't confuse you by trying to teach you three APIs
85 | and a language simultaneously.
86 |
87 |
88 | Concise code - I'll have exactly enough code to make it
89 | work, and little else. I will sacrifice a bit of structure just to
90 | save you the trouble of sorting through twice as many header files,
91 | so that you can focus on the relevant functionality.
92 |
32 | Have a Kinect v2? Head over to the Kinect v2 Tutorial 0. This tutorial is for the v1 Kinect SDK.
33 |
34 |
35 |
36 |
37 | This series of tutorials is intended for C++ programmers who want to use Microsoft's Kinect SDK.
38 | There will be as little Windows code as possible. We will be using OpenGL for graphics.
39 |
40 | Goals: Ensure you have the necessary components for coding with Kinect. Set up a Visual Studio
41 | project with the proper build configurations.
42 | Source:1_Basics.zip
43 |
58 | Most windows systems do not come with GLUT. GLUT is an old but widely
59 | used system for windowing and handling events that is commonly used with
60 | OpenGL.
61 |
62 |
Copy the contents of the Include/ and Lib/ directories you just
64 | unzipped into the appropriate Windows SDK directories. e.g.
65 |
66 |
C:/Program Files/Microsoft SDKs/Windows/v7.0A/Include/ and
67 | C:/Program Files/Microsoft SDKs/Windows/v7.0A/Lib/ for Visual Studio 2010
68 |
C:/Program Files/Windows Kits (x86)/8.1/Include/um/ and
69 | C:/Program Files (x86)/Windows Kits/8.1/Lib/winv6.3/um/ for Visual Studio 2012+
70 |
71 |
72 |
Copy bin/x64/freeglut.dll into C:/Windows/System32 and bin/x86/freeglut.dll into C:/Windows/SysWOW64. If
73 | you have a 32-bit system, just move bin/x86/freeglut.dll into C:/Windows/System32
74 |
75 |
Creating a Kinect Project
76 | Open up Visual Studio and start a new Empty Project. Click
77 | File > New > Project...; then select C++ > General > Empty Project.
78 | Name it appropriately.
79 |
80 | To configure the build rules (i.e. includes and libs) we will use the Property Manager. This will
81 | allow us to save the configurations to separate files so that we can use them again later.
82 |
83 | Go to View > Property Manager. This will open up
84 | a pane in one of the subwindows in Visual Studio containing build
85 | rules.
86 |
87 |
88 | In the Property Manager, right click on the project name and select Add New Project Property Sheet; name it "OpenGL". Create another one named "KinectSDK".
89 | You should see several new items appear under "Debug" and "Release"
90 | with the names of your new Property Sheets. Property Sheets are files
91 | (extension .props) that contain build configuration data. For
92 | modularity, we will create one for our OpenGL info and one for our
93 | Kinect SDK info.
94 |
95 |
96 | Right-click the Kinect Property Sheet and click Properties.
97 |
98 |
Under C/C++ > Additional Include Directories, add "$(KINECTSDK10_DIR)/include".
99 |
Under Linker > Additional Library Directories, add "$(KINECTSDK10_DIR)/lib/x64" (or /x86 for 32-bit systems).
100 |
Under Linker > Input > Additional Dependencies, add "kinect10.lib".
101 |
102 |
103 | Note that to add Include and Library Directories, you can either just
104 | type into the box, or you can click the dropdown, select <Edit>,
105 | and enter the location in the resulting dialog. This method lets you
106 | manage multiple directories easily, as well as lets you browse for the
107 | exact directory (if you don't use environment variables like
108 | KINECTSDK10_DIR)
109 |
110 |
111 | Right-click the OpenGL Property Sheet and click Properties. Under Linker > Input > Additional Dependencies, add the following:
112 |
32 | Have a Kinect v2? Head over to the Kinect v2 Tutorial 0. This tutorial is for the v1 Kinect SDK.
33 |
34 |
35 |
36 |
37 | This series of tutorials is intended for C++ programmers who want to use Microsoft's Kinect SDK.
38 | There will be as little Windows code as possible. We will be using OpenGL for graphics.
39 |
40 | Goals: Ensure you have the necessary components for coding with Kinect. Set up a Visual Studio
41 | project with the proper build configurations.
42 | Source:View SourceDownload:1_Basics.zip
43 |
59 | Download the latest version of the Visual C++ SDL Development Libraries
60 | from libsdl.org; at the time of
61 | writing this was version 1.2.15
62 |
63 |
64 | Unzip the contents to a suitable place (e.g. C:\).
65 |
66 | To make our lives easier we will do two things:
67 |
68 |
Copy the SDL.dll file to our system directory. Go to
69 | C:\SDL-1.2.15\lib\x86 (or the equivalent) and copy SDL.dll. Paste
70 | it into C:\Windows\SysWOW64 if that folder exists (i.e. you have
71 | a 64-bit version of Windows). Otherwise, paste it into
72 | C:\Windows\System32
73 |
Add the SDL folder as an environment variable.
74 | Open up the start menu, right click Computer, and select
75 | Properties. Click "Advanced System Settings" on the left
76 | side, go to the Advanced tab in the resulting dialog, and click
77 | the button labelled "Environment Variables".
78 |
79 | Under User variables (the top set of buttons) click "New...".
80 | Enter "SDL_DIR" (no quotes) for the variable name and the
81 | path to the SDL directory (e.g. "C:\SDL-1.2.15") as the value.
82 |
83 |
84 |
85 |
86 |
Creating a Kinect Project
87 | Open up Visual Studio and start a new Empty Project. Click
88 | File > New > Project...; then select C++ > General > Empty Project.
89 | Name it appropriately.
90 |
91 | To configure the build rules (i.e. includes and libs) we will use the Property Manager. This will
92 | allow us to save the configurations to separate files so that we can use them again later.
93 |
94 | Go to View > Property Manager. This will open up
95 | a pane in one of the subwindows in Visual Studio containing build
96 | rules.
97 |
98 |
99 | In the Property Manager, right click on the project name and select Add New Project Property Sheet; name it "OpenGL". Create another one named "KinectSDK".
100 | You should see several new items appear under "Debug" and "Release"
101 | with the names of your new Property Sheets. Property Sheets are files
102 | (extension .props) that contain build configuration data. For
103 | modularity, we will create one for our OpenGL info and one for our
104 | Kinect SDK info.
105 |
106 |
107 | Right-click the Kinect Property Sheet and click Properties.
108 |
109 |
Under C/C++ > Additional Include Directories, add "$(KINECTSDK10_DIR)/include".
110 |
Under Linker > Additional Library Directories, add "$(KINECTSDK10_DIR)/lib/x64" (or /x86 for 32-bit systems).
111 |
Under Linker > Input > Additional Dependencies, add "kinect10.lib".
112 |
113 |
114 | Note that to add Include and Library Directories, you can either just
115 | type into the box, or you can click the dropdown, select <Edit>,
116 | and enter the location in the resulting dialog. This method lets you
117 | manage multiple directories easily, as well as lets you browse for the
118 | exact directory (if you don't use environment variables like
119 | KINECTSDK10_DIR)
120 |
121 |
122 | Right-click the OpenGL Property Sheet and click Properties. Under Linker > Input > Additional Dependencies, add the following:
123 |
124 |
Under C/C++ > Additional Include Directories, add "$(SDL_DIR)\include".
125 |
Under Linker > Additional Library Directories, add "$(SDL_DIR)\lib\x86".
126 |
Under Linker > Input > Additional Dependencies, add the following:
50 | To get the depth data from the kinect, simply change the arguments
51 | to NuiImageStreamOpen(). The first argument is now
52 | NUI_IMAGE_TYPE_DEPTH, telling the Kinect that we now
53 | want depth images instead of RGB images. (For clarity we also changed
54 | the name of the Handle to reflect this)
55 |
56 | When you're using the Kinect for Windows sensor (as opposed to the XBox
57 | Kinect) you can also enable Near Mode. Near Mode allows the Kinect to
58 | be more sensitive to closer objects (say from 50cm to 200cm), whereas
59 | without near mode it is sensitive from 80cm to 400cm.
60 | To enable near mode, pass in the flag NUI_IMAGE_STREAM_FLAG_ENABLE_NEAR_MODE as the third argument.
61 |
62 | // NEW VARIABLE
63 | HANDLE depthStream;
64 |
65 | bool initKinect() {
66 | // Get a working kinect sensor
67 | int numSensors;
68 | if (NuiGetSensorCount(&numSensors) < 0 || numSensors < 1) return false;
69 | if (NuiCreateSensorByIndex(0, &sensor) < 0) return false;
70 |
71 | // Initialize sensor
72 | sensor->NuiInitialize(NUI_INITIALIZE_FLAG_USES_DEPTH | NUI_INITIALIZE_FLAG_USES_COLOR);
73 |
74 | // --------------- START CHANGED CODE -----------------
75 | sensor->NuiImageStreamOpen(
76 | NUI_IMAGE_TYPE_DEPTH, // Depth camera or rgb camera?
77 | NUI_IMAGE_RESOLUTION_640x480, // Image resolution
78 | NUI_IMAGE_STREAM_FLAG_ENABLE_NEAR_MODE, // Image stream flags, e.g. near mode
79 | 2, // Number of frames to buffer
80 | NULL, // Event handle
81 | &depthStream);
82 | // --------------- END CHANGED CODE -----------------
83 | return sensor;
84 | }
85 |
88 | We'll display the depth image from the kinect in grayscale. Each pixel
89 | will just be the pixel's distance from the kinect (in millimeters) mod
90 | 256.
91 | The only thing to note here is the NuiDepthPixelToDepth
92 | function - each pixel in the actual depth map has both depth information
93 | and player information (i.e. which player can be associated with that
94 | pixel). Calling this function returns the depth in millimeters at that
95 | pixel.
96 |
The depth data is 16 bits, so we use a USHORT to read it in.
97 |
98 | const USHORT* curr = (const USHORT*) LockedRect.pBits;
99 | const USHORT* dataEnd = curr + (width*height);
100 |
101 | while (curr < dataEnd) {
102 | // Get depth in millimeters
103 | USHORT depth = NuiDepthPixelToDepth(*curr++);
104 |
105 | // Draw a grayscale image of the depth:
106 | // B,G,R are all set to depth%256, alpha set to 1.
107 | for (int i = 0; i < 3; ++i)
108 | *dest++ = (BYTE) depth%256;
109 | *dest++ = 0xff;
110 | }
111 |
112 |
113 |
114 | That's all the Kinect code! The rest is just how to get it onscreen.
115 |
116 |
117 | I refactored the code so that the glut and SDL specifics are bundled
118 | up into an init() function, a draw() function,
119 | and an execute() function.
120 |
To use one or the other, simply
121 | change the include at the top of main.cpp to use either
122 | sdl.h or glut.h. Make sure you have the
123 | appropriate includes and links in the Project Properties!
124 |
125 | The End! Build and run, making sure that your Kinect is plugged in.
126 | You should see a window containing a video stream of what your Kinect
127 | sees, in grayscale.
128 |
129 |
25 | This series of tutorials is intended for C++ programmers who want to use Microsoft's Kinect SDK.
26 | There will be as little Windows code as possible. We will be using OpenGL for graphics.
27 |
28 | Goals: Ensure you have the necessary components for coding with Kinect. Set up a Visual Studio
29 | project with the proper build configurations.
30 | Source:1_Basics.zip
31 |
46 | Most windows systems do not come with GLUT. GLUT is an old but widely
47 | used system for windowing and handling events that is commonly used with
48 | OpenGL.
49 |
50 |
Copy the contents of the Include/ and Lib/ directories you just
52 | unzipped into the appropriate Windows SDK directories. e.g.
53 |
54 |
C:/Program Files/Microsoft SDKs/Windows/v7.0A/Include/ and
55 | C:/Program Files/Microsoft SDKs/Windows/v7.0A/Lib/ for Visual Studio 2010
56 |
C:/Program Files/Windows Kits (x86)/8.1/Include/um/ and
57 | C:/Program Files (x86)/Windows Kits/8.1/Lib/winv6.3/um/ for Visual Studio 2012+
58 |
59 |
60 |
Copy bin/x64/freeglut.dll into C:/Windows/System32 and bin/x86/freeglut.dll into C:/Windows/SysWOW64. If
61 | you have a 32-bit system, just move bin/x86/freeglut.dll into C:/Windows/System32
62 |
63 |
Creating a Kinect Project
64 | Open up Visual Studio and start a new Empty Project. Click
65 | File > New > Project...; then select C++ > General > Empty Project.
66 | Name it appropriately.
67 |
68 | To configure the build rules (i.e. includes and libs) we will use the Property Manager. This will
69 | allow us to save the configurations to separate files so that we can use them again later.
70 |
71 | Go to View > Property Manager. This will open up
72 | a pane in one of the subwindows in Visual Studio containing build
73 | rules.
74 |
75 |
76 | In the Property Manager, right click on the project name and select Add New Project Property Sheet; name it "OpenGL". Create another one named "KinectSDK".
77 | You should see several new items appear under "Debug" and "Release"
78 | with the names of your new Property Sheets. Property Sheets are files
79 | (extension .props) that contain build configuration data. For
80 | modularity, we will create one for our OpenGL info and one for our
81 | Kinect SDK info.
82 |
83 |
84 | Right-click the Kinect Property Sheet and click Properties.
85 |
86 |
Under C/C++ > Additional Include Directories, add "$(KINECTSDK20_DIR)/include".
87 |
Under Linker > Additional Library Directories, add "$(KINECTSDK20_DIR)/lib/x64" (or /x86 for 32-bit systems).
88 |
Under Linker > Input > Additional Dependencies, add "kinect20.lib".
89 |
90 |
91 | Note that to add Include and Library Directories, you can either just
92 | type into the box, or you can click the dropdown, select <Edit>,
93 | and enter the location in the resulting dialog. This method lets you
94 | manage multiple directories easily, as well as lets you browse for the
95 | exact directory (if you don't use environment variables like
96 | KINECTSDK20_DIR)
97 |
98 |
99 | Right-click the OpenGL Property Sheet and click Properties. Under Linker > Input > Additional Dependencies, add the following:
100 |
25 | This series of tutorials is intended for C++ programmers who want to use Microsoft's Kinect SDK.
26 | There will be as little Windows code as possible. We will be using OpenGL for graphics.
27 |
28 | Goals: Ensure you have the necessary components for coding with Kinect. Set up a Visual Studio
29 | project with the proper build configurations.
30 | Source:View SourceDownload:1_Basics.zip
31 |
47 | Download the latest version of the Visual C++ SDL Development Libraries
48 | from libsdl.org; at the time of
49 | writing this was version 1.2.15
50 |
51 |
52 | Unzip the contents to a suitable place (e.g. C:\).
53 |
54 | To make our lives easier we will do two things:
55 |
56 |
Copy the SDL.dll file to our system directory. Go to
57 | C:\SDL-1.2.15\lib\x86 (or the equivalent) and copy SDL.dll. Paste
58 | it into C:\Windows\SysWOW64 if that folder exists (i.e. you have
59 | a 64-bit version of Windows). Otherwise, paste it into
60 | C:\Windows\System32
61 |
Add the SDL folder as an environment variable.
62 | Open up the start menu, right click Computer, and select
63 | Properties. Click "Advanced System Settings" on the left
64 | side, go to the Advanced tab in the resulting dialog, and click
65 | the button labelled "Environment Variables".
66 |
67 | Under User variables (the top set of buttons) click "New...".
68 | Enter "SDL_DIR" (no quotes) for the variable name and the
69 | path to the SDL directory (e.g. "C:\SDL-1.2.15") as the value.
70 |
71 |
72 |
73 |
74 |
Creating a Kinect Project
75 | Open up Visual Studio and start a new Empty Project. Click
76 | File > New > Project...; then select C++ > General > Empty Project.
77 | Name it appropriately.
78 |
79 | To configure the build rules (i.e. includes and libs) we will use the Property Manager. This will
80 | allow us to save the configurations to separate files so that we can use them again later.
81 |
82 | Go to View > Property Manager. This will open up
83 | a pane in one of the subwindows in Visual Studio containing build
84 | rules.
85 |
86 |
87 | In the Property Manager, right click on the project name and select Add New Project Property Sheet; name it "OpenGL". Create another one named "KinectSDK".
88 | You should see several new items appear under "Debug" and "Release"
89 | with the names of your new Property Sheets. Property Sheets are files
90 | (extension .props) that contain build configuration data. For
91 | modularity, we will create one for our OpenGL info and one for our
92 | Kinect SDK info.
93 |
94 |
95 | Right-click the Kinect Property Sheet and click Properties.
96 |
97 |
Under C/C++ > Additional Include Directories, add "$(KINECTSDK20_DIR)/include".
98 |
Under Linker > Additional Library Directories, add "$(KINECTSDK20_DIR)/lib/x64" (or /x86 for 32-bit systems).
99 |
Under Linker > Input > Additional Dependencies, add "kinect20.lib".
100 |
101 |
102 | Note that to add Include and Library Directories, you can either just
103 | type into the box, or you can click the dropdown, select <Edit>,
104 | and enter the location in the resulting dialog. This method lets you
105 | manage multiple directories easily, as well as lets you browse for the
106 | exact directory (if you don't use environment variables like
107 | KINECTSDK20_DIR)
108 |
109 |
110 | Right-click the OpenGL Property Sheet and click Properties. Under Linker > Input > Additional Dependencies, add the following:
111 |
112 |
Under C/C++ > Additional Include Directories, add "$(SDL_DIR)\include".
113 |
Under Linker > Additional Library Directories, add "$(SDL_DIR)\lib\x86".
114 |
Under Linker > Input > Additional Dependencies, add the following:
38 | To get the depth data from the kinect, simply change the types of the
39 | framesource, framereader, and frame. Note also that the resolution of
40 | the depth camera is different from that of the color camera: 512*424
41 | instead of 1920*1080.
42 |
65 | We'll display the depth image from the kinect in grayscale. Each pixel
66 | will just be the pixel's distance from the kinect (in millimeters) mod
67 | 256.
68 |
69 | void getKinectData(GLubyte* dest) {
70 | IDepthFrame* frame = NULL;
71 | if (SUCCEEDED(reader->AcquireLatestFrame(&frame))) {
72 | unsigned int sz;
73 | unsigned short* buf;
74 | frame->AccessUnderlyingBuffer(&sz, &buf);
75 |
76 | const unsigned short* curr = (const unsigned short*)buf;
77 | const unsigned short* dataEnd = curr + (width*height);
78 |
79 | while (curr < dataEnd) {
80 | // Get depth in millimeters
81 | unsigned short depth = (*curr++);
82 |
83 | // Draw a grayscale image of the depth:
84 | // B,G,R are all set to depth%256, alpha set to 1.
85 | for (int i = 0; i < 3; ++i)
86 | *dest++ = (BYTE)depth % 256;
87 | *dest++ = 0xff;
88 | }
89 | }
90 | if (frame) frame->Release();
91 | }
92 |
93 | Note that, unlike with the color frame, we just want to access the raw
94 | data from the frame. To do this, we use frame->AccessUnderlyingBuffer
95 | to get a pointer to the data.
96 |
97 | Then we convert the depth data and write it to our image array.
98 |
99 | That's all the Kinect code! The rest is just how to get it onscreen.
100 |
101 |
102 | I refactored the code so that the glut and SDL specifics are bundled
103 | up into an init() function, a draw() function,
104 | and an execute() function.
105 |
To use one or the other, simply
106 | change the include at the top of main.cpp to use either
107 | sdl.h or glut.h. Make sure you have the
108 | appropriate includes and links in the Project Properties!
109 |
110 | The End! Build and run, making sure that your Kinect is plugged in.
111 | You should see a window containing a video stream of what your Kinect
112 | sees, in grayscale.
113 |
114 |
48 | 
68 | 
30 | 
87 | 
95 | 
119 |