├── Images ├── 2d.PNG └── 3d.PNG └── README.md /Images/2d.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenMonkeyStudio/OMS_Data/f7c251000128f3c1a4fcb6997ac92aee2e2c7d73/Images/2d.PNG -------------------------------------------------------------------------------- /Images/3d.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenMonkeyStudio/OMS_Data/f7c251000128f3c1a4fcb6997ac92aee2e2c7d73/Images/3d.PNG -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | OpenMonkeyStudio Model and Dataset 2 | ============================================= 3 | 4 | ## Introduction 5 | The OpenMonkeyStudio repository is a work in progress and will be updated on a regular basis. 2D and 3D label data along with the images used for training are available at https://z.umn.edu/OMS_data_link [3.8 GB] 6 | 7 | ## Data folder contents and structure 8 | The structure of the zip file is as follows: 9 | ```sh 10 | +-- Batch7 11 | | +-- camera.txt 12 | | +-- coords_3D.mat 13 | | +-- crop_para.mat 14 | | +-- intrinsic.txt 15 | +-- Batch9 16 | +-- Batch9a 17 | +-- Batch9b 18 | +-- Batch10 19 | +-- Batch11 20 | +-- Data.mat 21 | +-- image_label_visualizer.py 22 | +-- Images 23 | | +-- batch0_000000001_00000002.jpg 24 | | +-- batch0_000000001_00000004.jpg 25 | | +-- . 26 | +-- Readme.pdf 27 | +-- reprojection.py 28 | 29 | ``` 30 | The contents of the zip file are explained below: 31 | 1. Images are stored in the `Images` folder. The images have the following format: `batch(batch#)_(frame#)_(cameraID).jpg`, Eg. `batch7_000003120_18064162.jpg` 32 | 33 | 2. 2D annotations are stored in `Data.mat` file\ 34 | The file comprises of image filenames and image information. It also comprises of labels for 13 2D joints of the subject in the image.\ 35 | `T.name` (194518x1): Image filenames\ 36 | `T.data` (194518x30): Labels for the 2D pose in respective images. The data is stored in the following format: \ 37 | `Top-Left point` `Image_shape` `nose position` `head position` `neck position` `R.Shoulder position` `R.Hand position` `L.Shoulder position` `L.Hand position` `Hip position` `R.Knee position` `R.Foot position` `L.Knee position` `L.Foot position` `Tail`\ 38 | Example: `0 0 Image_height Image_width y1 x1 y2 x2 ... y13 x13` 39 | 40 | 3. 3D information has been uploaded for the following batches: `7, 9, 9a, 9b, 10, 11`. The folder for each batch comprises of the extrinsics and intrinsics of the cameras used during the capture of the respective batch. The folder also comprises of cropping parameters for the images in each batch, along with the 3D information for each frame.\ 41 | The files available in each folder have the following format: 42 | ### File formats 43 | 1. `camera.txt:` The file comprises of the rotation matrices and camera centers for each camera in the batch 44 | ```sh 45 | NumCams 62 46 | NumFrames 62 47 | NumP 62 48 | 0 camID 49 | C (1x3) … Camera center 50 | R (3x3) … Rotation matrix 51 | . 52 | . 53 | ``` 54 | Example: 55 | ```sh 56 | NumCams 62 57 | NumFrames 62 58 | NumP 62 59 | 0 18064162 60 | 7.566247e-01 -3.652013e-02 -3.071504e+00 61 | 9.962194e-01 -7.379663e-02 -4.583673e-02 62 | 6.721757e-02 9.890431e-01 -1.314361e-01 63 | 5.503405e-02 1.278582e-01 9.902644e-01 64 | 0 18260132 65 | 1.887207e+00 -7.456747e-02 -3.053667e+00 66 | 9.922224e-01 -4.990314e-02 1.140367e-01 67 | 6.738378e-02 9.856155e-01 -1.549887e-01 68 | -1.046620e-01 1.614675e-01 9.813125e-01 69 | . 70 | . 71 | ``` 72 | 2. `intrinsic.txt:` The file comprises of the intrinsic matrices and distortion parameters for each camera. The model used for calibration is `RADIAL` 73 | ```sh 74 | 0 camID 75 | K (3x3) … Intrinsic parameters 76 | D (1x2) … Distortion parameters 77 | . 78 | . 79 | ``` 80 | Example: 81 | ```sh 82 | 0 18064162 83 | 7.873470e+02 0 640 84 | 0 7.873470e+02 512 85 | 0 0 1 86 | -3.531380e-01 1.173270e-01 87 | 0 18260132 88 | 7.932990e+02 0 640 89 | 0 7.932990e+02 512 90 | 0 0 1 91 | -3.480520e-01 1.167730e-01 92 | . 93 | . 94 | ``` 95 | 3. 3D annotations are stored in `coords_3D.mat` file:\ 96 | The file comprises of 3D information for 13 joints of the subject in the particular frame\ 97 | `coords` (nx4): First column gives information about the frame number. The next 3 columns provide information about the X, Y and Z coordinates (`n = 13xn’` where `n’` is the no. of frames) 98 | - Each row comprises of 3D information for each joint of each frame 99 | - The format of each row is as follows: `Frame#` `X` `Y` `Z`, Eg. `3120` `3.6364` `0.2717` `1.8471` 100 | - The order for the joints is as follows: `Nose-Head-Neck-RShoulder-RHand-LShoulder-LHand-Hip-RKnee-RFoot-LKnee-LFoot-Tail` 101 | 102 | 4. The cropping parameters are stored in `crop_para.mat` file:\ 103 | The file comprises of cropping parameters for the images (that have 3D information available)\ 104 | `crop` (nx6): The array stores information for frame#, cameraID, top-left position of the cropped image in the original image (x, y) and image size (Image_width Image_Height) 105 | - The format of each row is as follows: `Frame#` `cameraID` `x` `y` `w` `h`, Eg. `3120` `18064162` `789` `327` `381` `382` 106 | 107 | ## Visualization codes 108 | The zip file contains codes for visualization of labeled data 109 | ### image_label_visualizer.py 110 | This code helps to visualize the 2D labels for images 111 | ```sh 112 | python3 image_label_visualizer.py -i 10 113 | ``` 114 | The above command will show the visualization for the 10th image (default is 0th image)\ 115 | ![OMS demo](Images/2d.PNG) 116 | 117 | ### reprojection.py 118 | This code helps to visualize the 2D reprojections of the 3D coordinates for any chosen frame 119 | ```sh 120 | python3 reprojection.py -b 7 -f 10 121 | ``` 122 | The above command will show the visualization for the 10th frame in batch 7 (default is batch 7 frame 0)\ 123 | ![OMS demo](Images/3d.PNG) 124 | --------------------------------------------------------------------------------