├── positionCalc.m ├── LICENSE └── README.md /positionCalc.m: -------------------------------------------------------------------------------- 1 | function [depthIJK, depthXYZ, colorIJK, colorXYZ] = positionCalc(depthPosition, depthIm) 2 | colorKmat = [607.92271, 0, 314.78337; 0, 607.88192, 236.42484; 0, 0, 1]; 3 | depthKmat = [475.62768, 0, 336.41179; 0, 474.77709, 238.77962; 0, 0, 1]; 4 | stereoOm = [0.00531 -0.01196 0.00301]; stereoT = [-24.0381 -0.4563 -1.2326]; 5 | stereoR = rodrigues(stereoOm); 6 | depthIJK(1:2) = depthPosition; 7 | depthIJK(3) = 1; 8 | depthIJK = depthIJK'; depthXYZ = depthKmat \ depthIJK; 9 | depthXYZ = depthXYZ * depthIm(int16(depthIJK(2)),int16(depthIJK(1))); 10 | colorXYZ = stereoR \ (depthXYZ - stereoT); colorIJK = colorKmat * colorXYZ; 11 | colorIJK = colorIJK / colorIJK(3); 12 | end 13 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 zhjwustc 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | This stereo hand pose tracking dataset is described in the paper: 2 | Jiawei Zhang, Jianbo Jiao, Mingliang Chen, Liangqiong Qu, Xiaobin Xu and Qingxiong Yang, "A hand pose tracking benchmark from stereo matching", icip 2017. 3 | 4 | You can download the dataset from 5 | 6 | Onedrive: https://portland-my.sharepoint.com/:f:/g/personal/jiawzhang8-c_my_cityu_edu_hk/EkznohE3YL1Jk8igTCQ4HqYBl8wiPIHNwL7U9X5b3p607Q?e=3aOPez 7 | 8 | BaiduPan: https://pan.baidu.com/s/1qXBpBg4#list/path=%2F 9 | 10 | Our stereo hand pose benchmark contains sequences with 6 different backgrounds and every background has two sequences with counting and random poses. Every sequence has 1500 frames, so there are totally 18000 frames in our benchmark. 11 | Stereo and depth images were captured from a Point Grey Bumblebee2 stereo camera and an Intel Real Sense F200 active depth camera simultaneously. 12 | 13 | 1. Camera parameters 14 | (1) Point Grey Bumblebee2 stereo camera: 15 | base line = 120.054 16 | fx = 822.79041 17 | fy = 822.79041 18 | tx = 318.47345 19 | ty = 250.31296 20 | (2) Intel Real Sense F200 active depth camera: 21 | fx color = 607.92271 22 | fy color = 607.88192 23 | tx color = 314.78337 24 | ty color = 236.42484 25 | fx depth = 475.62768 26 | fy depth = 474.77709 27 | tx depth = 336.41179 28 | ty depth = 238.77962 29 | rotation vector = [0.00531 -0.01196 0.00301] (use Rodrigues' rotation formula to transform it into rotation matrix) 30 | translation vector = [-24.0381 -0.4563 -1.2326] 31 | (rotation and translation vector can transform the coordinates relative to color camera to those relative to depth camera) 32 | 33 | 2. images 34 | All the images are in folder '.\images\' which contains 12 subfolders. The left/right images for Point Grey Bumblebee2 stereo camera and color/depth images for Intel Real Sense F200 active depth camera have prefix BB_left_, BB_right_, SK_color_ and SK_depth_ respectively. 35 | Since the value of depth is usually larger than 255, we use 3 channels to store depth images and depth = r channel + g channel*256. 36 | 37 | 3. labels 38 | All the labels are in folder '.\labels\'. The labels for Point Grey Bumblebee2 stereo camera and color/depth images for Intel Real Sense F200 active depth camera have suffix _BB and _SK respectively. For each mat file in this folder, it contains an array named 'handPara' with size 3*21*1500 which stores the 3D positions (x,y,z) in Millimeter of palm center and finger joints (totally 21 joints) of all 1500 the images in this sequence. 39 | The sequence of 21 joints are: palm center(not wrist or hand center), little_mcp, little_pip, little_dip, little_tip, ring_mcp, ring_pip, ring_dip, ring_tip, middle_mcp, middle_pip, middle_dip, middle_tip, index_mcp, index_pip, index_dip, index_tip, thumb_mcp, thumb_pip, thumb_dip, thumb_tip. 40 | 41 | 42 | -------------------------------------------------------------- 43 | Someone asks about how to project the point cloud into color and depth images for Intel Real Sense F200 active depth camera. You can use the following MATLAB script: positionCalc.m 44 | --------------------------------------------------------------------------------