├── environment.yml ├── .gitignore └── README.md /environment.yml: -------------------------------------------------------------------------------- 1 | name: contactpose_ml 2 | 3 | channels: 4 | - pytorch 5 | - open3d-admin 6 | 7 | dependencies: 8 | - python=3.7 9 | - pytorch=1.2.0 10 | - torchvision 11 | - matplotlib 12 | - ignite 13 | - open3d 14 | - scikit-learn 15 | - tqdm 16 | - pip 17 | - ipython 18 | - pip: 19 | - visdom 20 | - transforms3d 21 | - hdbscan 22 | - chumpy 23 | - trimesh[easy] 24 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | 2 | # Created by https://www.toptal.com/developers/gitignore/api/macos,linux,vim,code,python 3 | # Edit at https://www.toptal.com/developers/gitignore?templates=macos,linux,vim,code,python 4 | 5 | ### Code ### 6 | .vscode/* 7 | !.vscode/settings.json 8 | !.vscode/tasks.json 9 | !.vscode/launch.json 10 | !.vscode/extensions.json 11 | *.code-workspace 12 | 13 | ### Linux ### 14 | *~ 15 | 16 | # temporary files which can be created if a process still has a handle open of a deleted file 17 | .fuse_hidden* 18 | 19 | # KDE directory preferences 20 | .directory 21 | 22 | # Linux trash folder which might appear on any partition or disk 23 | .Trash-* 24 | 25 | # .nfs files are created when an open file is removed but is still being accessed 26 | .nfs* 27 | 28 | ### macOS ### 29 | # General 30 | .DS_Store 31 | .AppleDouble 32 | .LSOverride 33 | 34 | # Icon must end with two \r 35 | Icon 36 | 37 | # Thumbnails 38 | ._* 39 | 40 | # Files that might appear in the root of a volume 41 | .DocumentRevisions-V100 42 | .fseventsd 43 | .Spotlight-V100 44 | .TemporaryItems 45 | .Trashes 46 | .VolumeIcon.icns 47 | .com.apple.timemachine.donotpresent 48 | 49 | # Directories potentially created on remote AFP share 50 | .AppleDB 51 | .AppleDesktop 52 | Network Trash Folder 53 | Temporary Items 54 | .apdisk 55 | 56 | ### Python ### 57 | # Byte-compiled / optimized / DLL files 58 | __pycache__/ 59 | *.py[cod] 60 | *$py.class 61 | 62 | # C extensions 63 | *.so 64 | 65 | # Distribution / packaging 66 | .Python 67 | build/ 68 | develop-eggs/ 69 | dist/ 70 | downloads/ 71 | eggs/ 72 | .eggs/ 73 | lib/ 74 | lib64/ 75 | parts/ 76 | sdist/ 77 | var/ 78 | wheels/ 79 | pip-wheel-metadata/ 80 | share/python-wheels/ 81 | *.egg-info/ 82 | .installed.cfg 83 | *.egg 84 | MANIFEST 85 | 86 | # PyInstaller 87 | # Usually these files are written by a python script from a template 88 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 89 | *.manifest 90 | *.spec 91 | 92 | # Installer logs 93 | pip-log.txt 94 | pip-delete-this-directory.txt 95 | 96 | # Unit test / coverage reports 97 | htmlcov/ 98 | .tox/ 99 | .nox/ 100 | .coverage 101 | .coverage.* 102 | .cache 103 | nosetests.xml 104 | coverage.xml 105 | *.cover 106 | *.py,cover 107 | .hypothesis/ 108 | .pytest_cache/ 109 | pytestdebug.log 110 | 111 | # Translations 112 | *.mo 113 | *.pot 114 | 115 | # Django stuff: 116 | *.log 117 | local_settings.py 118 | db.sqlite3 119 | db.sqlite3-journal 120 | 121 | # Flask stuff: 122 | instance/ 123 | .webassets-cache 124 | 125 | # Scrapy stuff: 126 | .scrapy 127 | 128 | # Sphinx documentation 129 | docs/_build/ 130 | doc/_build/ 131 | 132 | # PyBuilder 133 | target/ 134 | 135 | # Jupyter Notebook 136 | .ipynb_checkpoints 137 | 138 | # IPython 139 | profile_default/ 140 | ipython_config.py 141 | 142 | # pyenv 143 | .python-version 144 | 145 | # pipenv 146 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 147 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 148 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 149 | # install all needed dependencies. 150 | #Pipfile.lock 151 | 152 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow 153 | __pypackages__/ 154 | 155 | # Celery stuff 156 | celerybeat-schedule 157 | celerybeat.pid 158 | 159 | # SageMath parsed files 160 | *.sage.py 161 | 162 | # Environments 163 | .env 164 | .venv 165 | env/ 166 | venv/ 167 | ENV/ 168 | env.bak/ 169 | venv.bak/ 170 | 171 | # Spyder project settings 172 | .spyderproject 173 | .spyproject 174 | 175 | # Rope project settings 176 | .ropeproject 177 | 178 | # mkdocs documentation 179 | /site 180 | 181 | # mypy 182 | .mypy_cache/ 183 | .dmypy.json 184 | dmypy.json 185 | 186 | # Pyre type checker 187 | .pyre/ 188 | 189 | # pytype static type analyzer 190 | .pytype/ 191 | 192 | ### Vim ### 193 | # Swap 194 | [._]*.s[a-v][a-z] 195 | !*.svg # comment out if you don't need vector files 196 | [._]*.sw[a-p] 197 | [._]s[a-rt-v][a-z] 198 | [._]ss[a-gi-z] 199 | [._]sw[a-p] 200 | 201 | # Session 202 | Session.vim 203 | Sessionx.vim 204 | 205 | # Temporary 206 | .netrwhist 207 | # Auto-generated tag files 208 | tags 209 | # Persistent undo 210 | [._]*.un~ 211 | 212 | # End of https://www.toptal.com/developers/gitignore/api/macos,linux,vim,code,python 213 | data 214 | logs 215 | thirdparty 216 | .vscode -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # [ContactPose](https://contactpose.cc.gatech.edu) 2 | 3 | [PyTorch](https://pytorch.org) implementation of the **data-driven hand-object contact modeling** experiments presented in the paper: 4 | 5 | [ContactPose: A Dataset of Grasps with Object Contact and Hand Pose]() - 6 | 7 | [Samarth Brahmbhatt](https://samarth-robo.github.io/), 8 | [Chengcheng Tang](https://scholar.google.com/citations?hl=en&user=WbG27wQAAAAJ), 9 | [Christopher D. Twigg](https://scholar.google.com/citations?hl=en&user=aN-lQ0sAAAAJ), 10 | [Charles C. Kemp](http://charliekemp.com/), and 11 | [James Hays](https://www.cc.gatech.edu/~hays/), 12 | 13 | **ECCV 2020**. 14 | 15 | Please visit [http://contactpose.cc.gatech.edu](http://contactpose.cc.gatech.edu) to explore the dataset. 16 | 17 | **Note**: This is the ML code for the ECCV 2020 paper. The ContactPose 18 | dataset API is [here](https://github.com/facebookresearch/ContactPose). 19 | 20 | ## Citation 21 | ``` 22 | @InProceedings{Brahmbhatt_2020_ECCV, 23 | author = {Brahmbhatt, Samarth and Tang, Chengcheng and Twigg, Christopher D. and Kemp, Charles C. and Hays, James}, 24 | title = {{ContactPose}: A Dataset of Grasps with Object Contact and Hand Pose}, 25 | booktitle = {The European Conference on Computer Vision (ECCV)}, 26 | month = {August}, 27 | year = {2020} 28 | } 29 | ``` 30 | 31 | # Getting Started 32 | 33 | - Clone this repository 34 | ```bash 35 | $ git clone git@github.com:samarth-robo/ContactPose-ML.git contactpose-ml 36 | $ cd contactpose-ml 37 | ``` 38 | - Install [Miniconda](https://docs.conda.io/en/latest/miniconda.html). Create the `contactpose_ml` conda environment: 39 | `conda env create -f environment.yml`. Activate it: 40 | ```bash 41 | $ source activate contactpose_ml 42 | ``` 43 | - Tested on **PyTorch 1.2.0** (as mentioned in `environment.yml`). 44 | You will probably be able to use later versions, but no guarantees. Please create an issue if you run into problems. 45 | - Install `pytorch-geometric` [from source code](https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.html#installation-from-source). 46 | Pip wheels can unfortunately not be used because of the outdated version of PyTorch. 47 | - Checkout the appropriate branch for the features used to predict contact: 48 | - [mesh](https://github.com/samarth-robo/ContactPose-ML/tree/mesh) 49 | - [skeleton](https://github.com/samarth-robo/ContactPose-ML/tree/skeleton) 50 | - [relative-joints](https://github.com/samarth-robo/ContactPose-ML/tree/relative-joints) 51 | - [simple-joints](https://github.com/samarth-robo/ContactPose-ML/tree/simple-joints) 52 | - [images](https://github.com/samarth-robo/ContactPose-ML/tree/images) 53 | 54 | # Download Links 55 | These links are provided only for reference, you should not need to download them manually. 56 | `get_data.py` in each branch will download everything for you. 57 | 58 | ## Trained Models, Training Logs, and `result.json` files 59 | 60 | | **Learner** | **Features** | **Split** | **Link** | 61 | |---------------------|-----------------|-------------|------| 62 | | MLP | simple-joints | objects | [link](https://www.dropbox.com/sh/diu3ceafm2d29f7/AAB23ugU_1oWQ1kk6lNAKZyya?dl=1) | 63 | | MLP | relative-joints | objects | [link](https://www.dropbox.com/sh/ifb37j6h8ni8851/AAA7JLz96cKmZwzTRwI6G9Vza?dl=1)| 64 | | MLP | skeleton | objects | [link](https://www.dropbox.com/sh/jszbnc5txyp1lny/AABKT8Z9DeHlgZyP7hRxilToa?dl=1) | 65 | | MLP | mesh | objects | [link](https://www.dropbox.com/sh/4dt5rk8ker3hx56/AABni1Czr6RfQ_6r4pTf18oWa?dl=1) | 66 | | MLP | simple-joints | participants | [link](https://www.dropbox.com/sh/4im9mm4nluy5vna/AADOGgTwVClXfmLSojhgDfZYa?dl=1) | 67 | | MLP | relative-joints | participants | [link](https://www.dropbox.com/sh/0ztxdonvdhbftoj/AACJCU3FLQFMo9BINwIc-ZLFa?dl=1) | 68 | | MLP | skeleton | participants | [link](https://www.dropbox.com/sh/x6wl7pbj64y3zxa/AAASD_pFlaUVtgD6_lLIO2lQa?dl=1) | 69 | | MLP | mesh | participants | [link](https://www.dropbox.com/sh/z5q92scdcm4vz41/AABH88OJOFzvAG47y5i9HqNva?dl=1) | 70 | | PointNet++ | simple-joints | objects | [link](https://www.dropbox.com/sh/osq52js7v67f86w/AACiTAWVfiYCo5sqh6LLNLTya?dl=1) | 71 | | PointNet++ | relative-joints | objects | [link](https://www.dropbox.com/sh/6qzsu7dfrw29qzn/AABbiLDaKGz0g06xe25cuJEza?dl=1) | 72 | | PointNet++ | skeleton | objects | [link](https://www.dropbox.com/sh/oo2xsjoklnxwfi7/AAB1By9ELXgpcxfxu11zvKFka?dl=1) | 73 | | PointNet++ | mesh | objects | [link](https://www.dropbox.com/sh/sbskyrjffansvkn/AADIoqzUOv2lh8kzUPzdQkNBa?dl=1) | 74 | | PointNet++ | simple-joints | participants | [link](https://www.dropbox.com/sh/6p3incblwvf87e8/AAATyVHlddD-sEURHoS8Dn-ya?dl=1) | 75 | | PointNet++ | relative-joints | participants | [link](https://www.dropbox.com/sh/ejfljimt4dj92tu/AADq-cPe3nFQ5cXXpQHTlO_Wa?dl=1) | 76 | | PointNet++ | skeleton | participants | [link](https://www.dropbox.com/sh/e6gm4mwsbaqml89/AACPnfzQtL1-YqoMpXkIiA4na?dl=1) | 77 | | PointNet++ | mesh | participants | [link](https://www.dropbox.com/sh/rx4pge4m6mpsb86/AACtSocTvxOexVl3VFL_p8Xpa?dl=1) | 78 | | VoxNet | skeleton | objects | [link](https://www.dropbox.com/sh/1ykqz8pddya1zu7/AABXAuwMIaBLncLmq2t_hoRIa?dl=1) | 79 | | VoxNet | skeleton | participants | [link](https://www.dropbox.com/sh/13mygnw3yu70f5u/AADIGFoV_HNBvoRc_iRGypURa?dl=1) | 80 | | Heuristic (10 pose params) | - | objects | [link](https://www.dropbox.com/sh/478b5v3gp6euzom/AABt_-24TBlglf3c_mctklwZa?dl=1) | 81 | | Heuristic (15 pose params) | - | objects | [link](https://www.dropbox.com/sh/8zidjcmxp50cpuu/AAC6Gq4Kx_AwtOREd6Nh5Of_a?dl=1) | 82 | | Heuristic (10 pose params) | - | participants | [link](https://www.dropbox.com/sh/l1erk9cm3h740st/AADp3MG1-L-PdBH6k11v8fxsa?dl=1) | 83 | | Heuristic (15 pose params) | - | participants | [link](https://www.dropbox.com/sh/lob3yszezysj6ni/AADKhDrOhtJGNuqRETBhFds8a?dl=1) | 84 | | enc-dec, PointNet++ | images (3 view) | objects | [link](https://www.dropbox.com/sh/v8gu9ic5ht1f6hj/AAACckYFTRZu-dfJG17ZaVgLa?dl=1) | 85 | | enc-dec, PointNet++ | images (1 view) | objects | [link](https://www.dropbox.com/sh/lotm1tas810oiip/AABluumM2UccoGcAsJzFsPOma?dl=1) | 86 | | enc-dec, PointNet++ | images (3 view) | participants | [link](https://www.dropbox.com/sh/arp2ujgj15j0wuk/AACvKquP9-1zhd--199rpHuda?dl=1) | 87 | | enc-dec, PointNet++ | images (1 view) | participants | [link](https://www.dropbox.com/sh/x2csef9nhnuw231/AAAZS7cWh7OFsBXbEuz4R_maa?dl=1) | 88 | 89 | ## Other Data 90 | 91 | - [object model voxelizations](https://www.dropbox.com/sh/zyy9jyo6pzat456/AABwO3cR6uVe0bKMXfXn55XQa?dl=1) 92 | - [3D models of objects](https://www.dropbox.com/sh/l76a01eyx6sxoll/AACrvU_QYRG8A8pevM1QPCs9a?dl=1) 93 | - Pre-computed "prediction data": 94 | - [simple-joints](https://www.dropbox.com/s/a6rydh8y0fl85d6/simple-joints_prediction_data.zip?dl=1) 95 | - [relative-joints](https://www.dropbox.com/s/2y9h66mctofs1cj/relative-joints_prediction_data.zip?dl=1) 96 | - [skeleton](https://www.dropbox.com/s/7xyrafply27efog/skeleton_prediction_data.zip?dl=1) 97 | - [mesh](https://www.dropbox.com/s/fjfc81203u418pw/mesh_prediction_data.zip?dl=1) 98 | - [images](https://www.dropbox.com/s/i6j0e9hxdadun9k/images_prediction_data.zip?dl=1) 99 | --------------------------------------------------------------------------------