├── .gitignore ├── LICENSE ├── MoveNet_tf2onnx.ipynb ├── README.md ├── demo_multipose.py ├── demo_multipose_onnx.py ├── demo_singlepose.py ├── demo_singlepose_onnx.py ├── demo_singlepose_tflite.py ├── onnx ├── movenet_multipose_lightning_1.onnx ├── movenet_singlepose_lightning_4.onnx └── movenet_singlepose_thunder_4.onnx └── tflite ├── lite-model_movenet_singlepose_lightning_tflite_float16_4.tflite ├── lite-model_movenet_singlepose_lightning_tflite_int8_4.tflite ├── lite-model_movenet_singlepose_thunder_tflite_float16_4.tflite └── lite-model_movenet_singlepose_thunder_tflite_int8_4.tflite /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | pip-wheel-metadata/ 24 | share/python-wheels/ 25 | *.egg-info/ 26 | .installed.cfg 27 | *.egg 28 | MANIFEST 29 | 30 | # PyInstaller 31 | # Usually these files are written by a python script from a template 32 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 33 | *.manifest 34 | *.spec 35 | 36 | # Installer logs 37 | pip-log.txt 38 | pip-delete-this-directory.txt 39 | 40 | # Unit test / coverage reports 41 | htmlcov/ 42 | .tox/ 43 | .nox/ 44 | .coverage 45 | .coverage.* 46 | .cache 47 | nosetests.xml 48 | coverage.xml 49 | *.cover 50 | *.py,cover 51 | .hypothesis/ 52 | .pytest_cache/ 53 | 54 | # Translations 55 | *.mo 56 | *.pot 57 | 58 | # Django stuff: 59 | *.log 60 | local_settings.py 61 | db.sqlite3 62 | db.sqlite3-journal 63 | 64 | # Flask stuff: 65 | instance/ 66 | .webassets-cache 67 | 68 | # Scrapy stuff: 69 | .scrapy 70 | 71 | # Sphinx documentation 72 | docs/_build/ 73 | 74 | # PyBuilder 75 | target/ 76 | 77 | # Jupyter Notebook 78 | .ipynb_checkpoints 79 | 80 | # IPython 81 | profile_default/ 82 | ipython_config.py 83 | 84 | # pyenv 85 | .python-version 86 | 87 | # pipenv 88 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 89 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 90 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 91 | # install all needed dependencies. 92 | #Pipfile.lock 93 | 94 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow 95 | __pypackages__/ 96 | 97 | # Celery stuff 98 | celerybeat-schedule 99 | celerybeat.pid 100 | 101 | # SageMath parsed files 102 | *.sage.py 103 | 104 | # Environments 105 | .env 106 | .venv 107 | env/ 108 | venv/ 109 | ENV/ 110 | env.bak/ 111 | venv.bak/ 112 | 113 | # Spyder project settings 114 | .spyderproject 115 | .spyproject 116 | 117 | # Rope project settings 118 | .ropeproject 119 | 120 | # mkdocs documentation 121 | /site 122 | 123 | # mypy 124 | .mypy_cache/ 125 | .dmypy.json 126 | dmypy.json 127 | 128 | # Pyre type checker 129 | .pyre/ 130 | 131 | # bat 132 | *.bat -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /MoveNet_tf2onnx.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "MoveNet-tf2onnx.ipynb", 7 | "provenance": [], 8 | "collapsed_sections": [] 9 | }, 10 | "kernelspec": { 11 | "name": "python3", 12 | "display_name": "Python 3" 13 | }, 14 | "language_info": { 15 | "name": "python" 16 | } 17 | }, 18 | "cells": [ 19 | { 20 | "cell_type": "markdown", 21 | "metadata": { 22 | "id": "XL5ZIKOP_R7_" 23 | }, 24 | "source": [ 25 | "# パッケージインストール(Package install)" 26 | ] 27 | }, 28 | { 29 | "cell_type": "code", 30 | "metadata": { 31 | "colab": { 32 | "base_uri": "https://localhost:8080/" 33 | }, 34 | "id": "K9btfs1u_Htr", 35 | "outputId": "89f32224-06ea-471a-c5bf-e3a2da22343d" 36 | }, 37 | "source": [ 38 | "!pip install -U tf2onnx\n", 39 | "!pip install onnxruntime" 40 | ], 41 | "execution_count": 1, 42 | "outputs": [ 43 | { 44 | "output_type": "stream", 45 | "text": [ 46 | "Collecting tf2onnx\n", 47 | " Downloading tf2onnx-1.9.2-py3-none-any.whl (430 kB)\n", 48 | "\u001b[?25l\r\u001b[K |▊ | 10 kB 24.0 MB/s eta 0:00:01\r\u001b[K |█▌ | 20 kB 27.2 MB/s eta 0:00:01\r\u001b[K |██▎ | 30 kB 11.9 MB/s eta 0:00:01\r\u001b[K |███ | 40 kB 9.7 MB/s eta 0:00:01\r\u001b[K |███▉ | 51 kB 5.2 MB/s eta 0:00:01\r\u001b[K |████▋ | 61 kB 5.7 MB/s eta 0:00:01\r\u001b[K |█████▎ | 71 kB 5.5 MB/s eta 0:00:01\r\u001b[K |██████ | 81 kB 6.1 MB/s eta 0:00:01\r\u001b[K |██████▉ | 92 kB 4.6 MB/s eta 0:00:01\r\u001b[K |███████▋ | 102 kB 5.0 MB/s eta 0:00:01\r\u001b[K |████████▍ | 112 kB 5.0 MB/s eta 0:00:01\r\u001b[K |█████████▏ | 122 kB 5.0 MB/s eta 0:00:01\r\u001b[K |██████████ | 133 kB 5.0 MB/s eta 0:00:01\r\u001b[K |██████████▋ | 143 kB 5.0 MB/s eta 0:00:01\r\u001b[K |███████████▍ | 153 kB 5.0 MB/s eta 0:00:01\r\u001b[K |████████████▏ | 163 kB 5.0 MB/s eta 0:00:01\r\u001b[K |█████████████ | 174 kB 5.0 MB/s eta 0:00:01\r\u001b[K |█████████████▊ | 184 kB 5.0 MB/s eta 0:00:01\r\u001b[K |██████████████▌ | 194 kB 5.0 MB/s eta 0:00:01\r\u001b[K |███████████████▎ | 204 kB 5.0 MB/s eta 0:00:01\r\u001b[K |████████████████ | 215 kB 5.0 MB/s eta 0:00:01\r\u001b[K |████████████████▊ | 225 kB 5.0 MB/s eta 0:00:01\r\u001b[K |█████████████████▌ | 235 kB 5.0 MB/s eta 0:00:01\r\u001b[K |██████████████████▎ | 245 kB 5.0 MB/s eta 0:00:01\r\u001b[K |███████████████████ | 256 kB 5.0 MB/s eta 0:00:01\r\u001b[K |███████████████████▉ | 266 kB 5.0 MB/s eta 0:00:01\r\u001b[K |████████████████████▋ | 276 kB 5.0 MB/s eta 0:00:01\r\u001b[K |█████████████████████▎ | 286 kB 5.0 MB/s eta 0:00:01\r\u001b[K |██████████████████████ | 296 kB 5.0 MB/s eta 0:00:01\r\u001b[K |██████████████████████▉ | 307 kB 5.0 MB/s eta 0:00:01\r\u001b[K |███████████████████████▋ | 317 kB 5.0 MB/s eta 0:00:01\r\u001b[K |████████████████████████▍ | 327 kB 5.0 MB/s eta 0:00:01\r\u001b[K |█████████████████████████▏ | 337 kB 5.0 MB/s eta 0:00:01\r\u001b[K |██████████████████████████ | 348 kB 5.0 MB/s eta 0:00:01\r\u001b[K |██████████████████████████▋ | 358 kB 5.0 MB/s eta 0:00:01\r\u001b[K |███████████████████████████▍ | 368 kB 5.0 MB/s eta 0:00:01\r\u001b[K |████████████████████████████▏ | 378 kB 5.0 MB/s eta 0:00:01\r\u001b[K |█████████████████████████████ | 389 kB 5.0 MB/s eta 0:00:01\r\u001b[K |█████████████████████████████▊ | 399 kB 5.0 MB/s eta 0:00:01\r\u001b[K |██████████████████████████████▌ | 409 kB 5.0 MB/s eta 0:00:01\r\u001b[K |███████████████████████████████▎| 419 kB 5.0 MB/s eta 0:00:01\r\u001b[K |████████████████████████████████| 430 kB 5.0 MB/s eta 0:00:01\r\u001b[K |████████████████████████████████| 430 kB 5.0 MB/s \n", 49 | "\u001b[?25hRequirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from tf2onnx) (2.23.0)\n", 50 | "Requirement already satisfied: flatbuffers~=1.12 in /usr/local/lib/python3.7/dist-packages (from tf2onnx) (1.12)\n", 51 | "Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from tf2onnx) (1.15.0)\n", 52 | "Requirement already satisfied: numpy>=1.14.1 in /usr/local/lib/python3.7/dist-packages (from tf2onnx) (1.19.5)\n", 53 | "Collecting onnx>=1.4.1\n", 54 | " Downloading onnx-1.10.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (12.3 MB)\n", 55 | "\u001b[K |████████████████████████████████| 12.3 MB 42.0 MB/s \n", 56 | "\u001b[?25hRequirement already satisfied: typing-extensions>=3.6.2.1 in /usr/local/lib/python3.7/dist-packages (from onnx>=1.4.1->tf2onnx) (3.7.4.3)\n", 57 | "Requirement already satisfied: protobuf in /usr/local/lib/python3.7/dist-packages (from onnx>=1.4.1->tf2onnx) (3.17.3)\n", 58 | "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->tf2onnx) (2.10)\n", 59 | "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->tf2onnx) (2021.5.30)\n", 60 | "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->tf2onnx) (1.24.3)\n", 61 | "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->tf2onnx) (3.0.4)\n", 62 | "Installing collected packages: onnx, tf2onnx\n", 63 | "Successfully installed onnx-1.10.1 tf2onnx-1.9.2\n", 64 | "Collecting onnxruntime\n", 65 | " Downloading onnxruntime-1.8.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.5 MB)\n", 66 | "\u001b[K |████████████████████████████████| 4.5 MB 5.1 MB/s \n", 67 | "\u001b[?25hRequirement already satisfied: numpy>=1.16.6 in /usr/local/lib/python3.7/dist-packages (from onnxruntime) (1.19.5)\n", 68 | "Requirement already satisfied: protobuf in /usr/local/lib/python3.7/dist-packages (from onnxruntime) (3.17.3)\n", 69 | "Requirement already satisfied: flatbuffers in /usr/local/lib/python3.7/dist-packages (from onnxruntime) (1.12)\n", 70 | "Requirement already satisfied: six>=1.9 in /usr/local/lib/python3.7/dist-packages (from protobuf->onnxruntime) (1.15.0)\n", 71 | "Installing collected packages: onnxruntime\n", 72 | "Successfully installed onnxruntime-1.8.1\n" 73 | ], 74 | "name": "stdout" 75 | } 76 | ] 77 | }, 78 | { 79 | "cell_type": "markdown", 80 | "metadata": { 81 | "id": "tR8xMD6Z_VLS" 82 | }, 83 | "source": [ 84 | "# Saved Model v4 ファイルダウンロード(Saved Model v4 file download)" 85 | ] 86 | }, 87 | { 88 | "cell_type": "code", 89 | "metadata": { 90 | "id": "kx__hWSm_OaE" 91 | }, 92 | "source": [ 93 | "# lightning(v4) singlepose\n", 94 | "!wget -q -O movenet_singlepose_lightning_4.tar.gz https://tfhub.dev/google/movenet/singlepose/lightning/4?tf-hub-format=compressed" 95 | ], 96 | "execution_count": 2, 97 | "outputs": [] 98 | }, 99 | { 100 | "cell_type": "code", 101 | "metadata": { 102 | "id": "hX8dhXPM_oBz", 103 | "colab": { 104 | "base_uri": "https://localhost:8080/" 105 | }, 106 | "outputId": "71bdccb7-c1b6-4c02-feba-42e10ff5fc2d" 107 | }, 108 | "source": [ 109 | "!mkdir movenet_singlepose_lightning_4\n", 110 | "!tar -zxvf movenet_singlepose_lightning_4.tar.gz -C movenet_singlepose_lightning_4/" 111 | ], 112 | "execution_count": 3, 113 | "outputs": [ 114 | { 115 | "output_type": "stream", 116 | "text": [ 117 | "./\n", 118 | "./saved_model.pb\n", 119 | "./variables/\n", 120 | "./variables/variables.index\n", 121 | "./variables/variables.data-00000-of-00001\n", 122 | "./assets/\n" 123 | ], 124 | "name": "stdout" 125 | } 126 | ] 127 | }, 128 | { 129 | "cell_type": "code", 130 | "metadata": { 131 | "id": "qZCmwGlBEFox" 132 | }, 133 | "source": [ 134 | "# thunder(v4) singlepose\n", 135 | "!wget -q -O movenet_singlepose_thunder_4.tar.gz https://tfhub.dev/google/movenet/singlepose/thunder/4?tf-hub-format=compressed" 136 | ], 137 | "execution_count": 4, 138 | "outputs": [] 139 | }, 140 | { 141 | "cell_type": "code", 142 | "metadata": { 143 | "colab": { 144 | "base_uri": "https://localhost:8080/" 145 | }, 146 | "id": "GJ4c5Pfvpl9y", 147 | "outputId": "26c157ad-d855-40a0-cf16-1536137dc353" 148 | }, 149 | "source": [ 150 | "!mkdir movenet_singlepose_thunder_4\n", 151 | "!tar -zxvf movenet_singlepose_thunder_4.tar.gz -C movenet_singlepose_thunder_4/" 152 | ], 153 | "execution_count": 5, 154 | "outputs": [ 155 | { 156 | "output_type": "stream", 157 | "text": [ 158 | "./\n", 159 | "./saved_model.pb\n", 160 | "./variables/\n", 161 | "./variables/variables.index\n", 162 | "./variables/variables.data-00000-of-00001\n", 163 | "./assets/\n" 164 | ], 165 | "name": "stdout" 166 | } 167 | ] 168 | }, 169 | { 170 | "cell_type": "code", 171 | "metadata": { 172 | "id": "mXgaunLoqFD3" 173 | }, 174 | "source": [ 175 | "# lightning(v1) multipose\n", 176 | "!wget -q -O movenet_multipose_lightning_1.tar.gz https://tfhub.dev/google/movenet/multipose/lightning/1?tf-hub-format=compressed" 177 | ], 178 | "execution_count": 6, 179 | "outputs": [] 180 | }, 181 | { 182 | "cell_type": "code", 183 | "metadata": { 184 | "colab": { 185 | "base_uri": "https://localhost:8080/" 186 | }, 187 | "id": "Z60ynPk6qN26", 188 | "outputId": "bca5bdac-0666-4440-939e-1afcb3bbee01" 189 | }, 190 | "source": [ 191 | "!mkdir movenet_multipose_lightning_1\n", 192 | "!tar -zxvf movenet_multipose_lightning_1.tar.gz -C movenet_multipose_lightning_1/" 193 | ], 194 | "execution_count": 7, 195 | "outputs": [ 196 | { 197 | "output_type": "stream", 198 | "text": [ 199 | "./\n", 200 | "./saved_model.pb\n", 201 | "./variables/\n", 202 | "./variables/variables.index\n", 203 | "./variables/variables.data-00000-of-00001\n", 204 | "./assets/\n" 205 | ], 206 | "name": "stdout" 207 | } 208 | ] 209 | }, 210 | { 211 | "cell_type": "markdown", 212 | "metadata": { 213 | "id": "gOSwwicjBkPS" 214 | }, 215 | "source": [ 216 | "# tfliteからONNXへ変換(Convert from tflite to ONNX)" 217 | ] 218 | }, 219 | { 220 | "cell_type": "code", 221 | "metadata": { 222 | "id": "C8qXrQf4Aj8a", 223 | "colab": { 224 | "base_uri": "https://localhost:8080/" 225 | }, 226 | "outputId": "8d0b5d3e-23e3-407b-8cda-38fc3d129d4c" 227 | }, 228 | "source": [ 229 | "# lightning(v4) singlepose\n", 230 | "!python -m tf2onnx.convert --opset 11 \\\n", 231 | " --saved-model movenet_singlepose_lightning_4 \\\n", 232 | " --output movenet_singlepose_lightning_4.onnx" 233 | ], 234 | "execution_count": 9, 235 | "outputs": [ 236 | { 237 | "output_type": "stream", 238 | "text": [ 239 | "/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'tf2onnx.convert' found in sys.modules after import of package 'tf2onnx', but prior to execution of 'tf2onnx.convert'; this may result in unpredictable behaviour\n", 240 | " warn(RuntimeWarning(msg))\n", 241 | "2021-08-23 15:38:32,371 - WARNING - '--tag' not specified for saved_model. Using --tag serve\n", 242 | "2021-08-23 15:38:48,900 - INFO - Signatures found in model: [serving_default].\n", 243 | "2021-08-23 15:38:48,901 - WARNING - '--signature_def' not specified, using first signature: serving_default\n", 244 | "2021-08-23 15:38:48,902 - INFO - Output names: ['output_0']\n", 245 | "WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tf2onnx/tf_loader.py:703: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.\n", 246 | "Instructions for updating:\n", 247 | "Use `tf.compat.v1.graph_util.extract_sub_graph`\n", 248 | "2021-08-23 15:38:50,490 - WARNING - From /usr/local/lib/python3.7/dist-packages/tf2onnx/tf_loader.py:703: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.\n", 249 | "Instructions for updating:\n", 250 | "Use `tf.compat.v1.graph_util.extract_sub_graph`\n", 251 | "2021-08-23 15:38:51,030 - INFO - Using tensorflow=2.6.0, onnx=1.10.1, tf2onnx=1.9.2/0f28b7\n", 252 | "2021-08-23 15:38:51,030 - INFO - Using opset \n", 253 | "2021-08-23 15:38:51,915 - INFO - Computed 0 values for constant folding\n", 254 | "2021-08-23 15:38:54,421 - INFO - Optimizing ONNX model\n", 255 | "2021-08-23 15:38:56,726 - INFO - After optimization: BatchNormalization -45 (55->10), Cast -12 (22->10), Const -254 (456->202), Identity -5 (5->0), Reshape -23 (32->9), Squeeze -2 (6->4), Transpose -284 (288->4), Unsqueeze -4 (16->12)\n", 256 | "2021-08-23 15:38:56,760 - INFO - \n", 257 | "2021-08-23 15:38:56,760 - INFO - Successfully converted TensorFlow model movenet_singlepose_lightning_4 to ONNX\n", 258 | "2021-08-23 15:38:56,760 - INFO - Model inputs: ['input']\n", 259 | "2021-08-23 15:38:56,760 - INFO - Model outputs: ['output_0']\n", 260 | "2021-08-23 15:38:56,761 - INFO - ONNX model is saved at movenet_singlepose_lightning_4.onnx\n" 261 | ], 262 | "name": "stdout" 263 | } 264 | ] 265 | }, 266 | { 267 | "cell_type": "code", 268 | "metadata": { 269 | "colab": { 270 | "base_uri": "https://localhost:8080/" 271 | }, 272 | "id": "zOuJplQFqbiU", 273 | "outputId": "b8a4879c-0abb-4375-c3e0-9d90fa874488" 274 | }, 275 | "source": [ 276 | "# thunder(v4) singlepose\n", 277 | "!python -m tf2onnx.convert --opset 11 \\\n", 278 | " --saved-model movenet_singlepose_thunder_4 \\\n", 279 | " --output movenet_singlepose_thunder_4.onnx" 280 | ], 281 | "execution_count": 10, 282 | "outputs": [ 283 | { 284 | "output_type": "stream", 285 | "text": [ 286 | "/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'tf2onnx.convert' found in sys.modules after import of package 'tf2onnx', but prior to execution of 'tf2onnx.convert'; this may result in unpredictable behaviour\n", 287 | " warn(RuntimeWarning(msg))\n", 288 | "2021-08-23 15:39:06,388 - WARNING - '--tag' not specified for saved_model. Using --tag serve\n", 289 | "2021-08-23 15:39:22,578 - INFO - Signatures found in model: [serving_default].\n", 290 | "2021-08-23 15:39:22,579 - WARNING - '--signature_def' not specified, using first signature: serving_default\n", 291 | "2021-08-23 15:39:22,580 - INFO - Output names: ['output_0']\n", 292 | "WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tf2onnx/tf_loader.py:703: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.\n", 293 | "Instructions for updating:\n", 294 | "Use `tf.compat.v1.graph_util.extract_sub_graph`\n", 295 | "2021-08-23 15:39:24,352 - WARNING - From /usr/local/lib/python3.7/dist-packages/tf2onnx/tf_loader.py:703: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.\n", 296 | "Instructions for updating:\n", 297 | "Use `tf.compat.v1.graph_util.extract_sub_graph`\n", 298 | "2021-08-23 15:39:24,996 - INFO - Using tensorflow=2.6.0, onnx=1.10.1, tf2onnx=1.9.2/0f28b7\n", 299 | "2021-08-23 15:39:24,996 - INFO - Using opset \n", 300 | "2021-08-23 15:39:26,945 - INFO - Computed 0 values for constant folding\n", 301 | "2021-08-23 15:39:30,147 - INFO - Optimizing ONNX model\n", 302 | "2021-08-23 15:39:32,531 - INFO - After optimization: BatchNormalization -45 (55->10), Cast -12 (22->10), Const -254 (456->202), Identity -5 (5->0), Reshape -23 (32->9), Squeeze -2 (6->4), Transpose -284 (288->4), Unsqueeze -4 (16->12)\n", 303 | "2021-08-23 15:39:32,584 - INFO - \n", 304 | "2021-08-23 15:39:32,584 - INFO - Successfully converted TensorFlow model movenet_singlepose_thunder_4 to ONNX\n", 305 | "2021-08-23 15:39:32,584 - INFO - Model inputs: ['input']\n", 306 | "2021-08-23 15:39:32,584 - INFO - Model outputs: ['output_0']\n", 307 | "2021-08-23 15:39:32,584 - INFO - ONNX model is saved at movenet_singlepose_thunder_4.onnx\n" 308 | ], 309 | "name": "stdout" 310 | } 311 | ] 312 | }, 313 | { 314 | "cell_type": "code", 315 | "metadata": { 316 | "colab": { 317 | "base_uri": "https://localhost:8080/" 318 | }, 319 | "id": "W7aM5KZEqgfE", 320 | "outputId": "edb2d6a2-0f69-4327-dedd-8922d5075587" 321 | }, 322 | "source": [ 323 | "# lightning(v1) multipose\n", 324 | "!python -m tf2onnx.convert --opset 11 \\\n", 325 | " --saved-model movenet_multipose_lightning_1 \\\n", 326 | " --output movenet_multipose_lightning_1.onnx" 327 | ], 328 | "execution_count": 11, 329 | "outputs": [ 330 | { 331 | "output_type": "stream", 332 | "text": [ 333 | "/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'tf2onnx.convert' found in sys.modules after import of package 'tf2onnx', but prior to execution of 'tf2onnx.convert'; this may result in unpredictable behaviour\n", 334 | " warn(RuntimeWarning(msg))\n", 335 | "2021-08-23 15:39:35,452 - WARNING - '--tag' not specified for saved_model. Using --tag serve\n", 336 | "2021-08-23 15:39:49,820 - INFO - Signatures found in model: [serving_default].\n", 337 | "2021-08-23 15:39:49,820 - WARNING - '--signature_def' not specified, using first signature: serving_default\n", 338 | "2021-08-23 15:39:49,821 - INFO - Output names: ['output_0']\n", 339 | "WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tf2onnx/tf_loader.py:703: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.\n", 340 | "Instructions for updating:\n", 341 | "Use `tf.compat.v1.graph_util.extract_sub_graph`\n", 342 | "2021-08-23 15:39:51,844 - WARNING - From /usr/local/lib/python3.7/dist-packages/tf2onnx/tf_loader.py:703: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.\n", 343 | "Instructions for updating:\n", 344 | "Use `tf.compat.v1.graph_util.extract_sub_graph`\n", 345 | "2021-08-23 15:39:52,524 - INFO - Using tensorflow=2.6.0, onnx=1.10.1, tf2onnx=1.9.2/0f28b7\n", 346 | "2021-08-23 15:39:52,524 - INFO - Using opset \n", 347 | "2021-08-23 15:39:54,166 - INFO - Computed 0 values for constant folding\n", 348 | "2021-08-23 15:39:55,766 - INFO - Optimizing ONNX model\n", 349 | "2021-08-23 15:40:02,085 - INFO - After optimization: Add -2 (26->24), BatchNormalization -45 (55->10), Cast -27 (72->45), Concat -1 (15->14), Const -421 (658->237), Div -2 (9->7), Gather +5 (0->5), Identity -20 (20->0), Mul -2 (29->27), Reshape -26 (48->22), Shape -4 (11->7), Slice -3 (28->25), Transpose -298 (304->6), Unsqueeze -11 (47->36)\n", 350 | "2021-08-23 15:40:02,151 - INFO - \n", 351 | "2021-08-23 15:40:02,151 - INFO - Successfully converted TensorFlow model movenet_multipose_lightning_1 to ONNX\n", 352 | "2021-08-23 15:40:02,152 - INFO - Model inputs: ['input']\n", 353 | "2021-08-23 15:40:02,152 - INFO - Model outputs: ['output_0']\n", 354 | "2021-08-23 15:40:02,152 - INFO - ONNX model is saved at movenet_multipose_lightning_1.onnx\n" 355 | ], 356 | "name": "stdout" 357 | } 358 | ] 359 | }, 360 | { 361 | "cell_type": "markdown", 362 | "metadata": { 363 | "id": "7idFiuPpBpk6" 364 | }, 365 | "source": [ 366 | "# 入力詳細・出力詳細確認(Check Input details & Output details)" 367 | ] 368 | }, 369 | { 370 | "cell_type": "code", 371 | "metadata": { 372 | "id": "KBeEQlxLBcbY" 373 | }, 374 | "source": [ 375 | "import onnxruntime" 376 | ], 377 | "execution_count": 12, 378 | "outputs": [] 379 | }, 380 | { 381 | "cell_type": "code", 382 | "metadata": { 383 | "colab": { 384 | "base_uri": "https://localhost:8080/" 385 | }, 386 | "id": "pUY9K-amBvqe", 387 | "outputId": "27ca04a7-e1a4-47bf-971d-9fa8c9fe639a" 388 | }, 389 | "source": [ 390 | "onnx_session = onnxruntime.InferenceSession('movenet_singlepose_lightning_4.onnx')\n", 391 | "input_detail = onnx_session.get_inputs()\n", 392 | "output_detail = onnx_session.get_outputs()\n", 393 | "\n", 394 | "print(len(input_detail), len(output_detail))\n", 395 | "print(input_detail[0])\n", 396 | "print(output_detail[0])" 397 | ], 398 | "execution_count": 13, 399 | "outputs": [ 400 | { 401 | "output_type": "stream", 402 | "text": [ 403 | "1 1\n", 404 | "NodeArg(name='input', type='tensor(int32)', shape=[1, 192, 192, 3])\n", 405 | "NodeArg(name='output_0', type='tensor(float)', shape=[1, 1, 17, 3])\n" 406 | ], 407 | "name": "stdout" 408 | } 409 | ] 410 | }, 411 | { 412 | "cell_type": "code", 413 | "metadata": { 414 | "colab": { 415 | "base_uri": "https://localhost:8080/" 416 | }, 417 | "id": "_TkayF1yCFVs", 418 | "outputId": "4a24203a-3cab-4b54-c6bf-2ce955252bd6" 419 | }, 420 | "source": [ 421 | "onnx_session = onnxruntime.InferenceSession('movenet_singlepose_thunder_4.onnx')\n", 422 | "input_detail = onnx_session.get_inputs()\n", 423 | "output_detail = onnx_session.get_outputs()\n", 424 | "\n", 425 | "print(len(input_detail), len(output_detail))\n", 426 | "print(input_detail[0])\n", 427 | "print(output_detail[0])" 428 | ], 429 | "execution_count": 14, 430 | "outputs": [ 431 | { 432 | "output_type": "stream", 433 | "text": [ 434 | "1 1\n", 435 | "NodeArg(name='input', type='tensor(int32)', shape=[1, 256, 256, 3])\n", 436 | "NodeArg(name='output_0', type='tensor(float)', shape=[1, 1, 17, 3])\n" 437 | ], 438 | "name": "stdout" 439 | } 440 | ] 441 | }, 442 | { 443 | "cell_type": "code", 444 | "metadata": { 445 | "id": "EF3lqjR4EjHz", 446 | "colab": { 447 | "base_uri": "https://localhost:8080/" 448 | }, 449 | "outputId": "bcb9fe89-b9d1-41b3-e584-fd49437f1cee" 450 | }, 451 | "source": [ 452 | "onnx_session = onnxruntime.InferenceSession('movenet_multipose_lightning_1.onnx')\n", 453 | "input_detail = onnx_session.get_inputs()\n", 454 | "output_detail = onnx_session.get_outputs()\n", 455 | "\n", 456 | "print(len(input_detail), len(output_detail))\n", 457 | "print(input_detail[0])\n", 458 | "print(output_detail[0])" 459 | ], 460 | "execution_count": 15, 461 | "outputs": [ 462 | { 463 | "output_type": "stream", 464 | "text": [ 465 | "1 1\n", 466 | "NodeArg(name='input', type='tensor(int32)', shape=[1, 'unk__1493', 'unk__1494', 3])\n", 467 | "NodeArg(name='output_0', type='tensor(float)', shape=[1, 6, 56])\n" 468 | ], 469 | "name": "stdout" 470 | } 471 | ] 472 | } 473 | ] 474 | } -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # MoveNet-Python-Example 2 | [MoveNet](https://tfhub.dev/s?q=MoveNet)のPythonでの動作サンプルです。
3 | ONNXに変換したモデルも同梱しています。変換自体を試したい方は[MoveNet_tf2onnx.ipynb](MoveNet_tf2onnx.ipynb)を使用ください。
4 | 5 | ![smjqx-4ndt8](https://user-images.githubusercontent.com/37477845/130482531-5be5f3e6-0dc9-42bb-80a8-4e7544d9ba7e.gif) 6 | 7 | 2021/08/24時点でTensorFlow Hubで提供されている以下モデルを使用しています。 8 | * [movenet/singlepose/lightning(v4)](https://tfhub.dev/google/movenet/singlepose/lightning/4) 9 | * [movenet/singlepose/thunder(v4)](https://tfhub.dev/google/movenet/singlepose/thunder/4) 10 | * [movenet/multipose/lightning(v1)](https://tfhub.dev/google/movenet/multipose/lightning/1) 11 | 12 | # Requirement 13 | * TensorFlow 2.3.0 or later 14 | * tensorflow-hub 0.12.0 or later 15 | * OpenCV 3.4.2 or later 16 | * onnxruntime 1.5.2 or later ※ONNX推論を使用する場合のみ 17 | 18 | 19 | # Demo 20 | デモの実行方法は以下です。 21 | #### SignlePose 22 | ```bash 23 | python demo_singlepose.py 24 | ``` 25 | * --device
26 | カメラデバイス番号の指定
27 | デフォルト：0 28 | * --file
29 | 動画ファイルの指定 ※指定時はカメラデバイスより優先
30 | デフォルト：指定なし 31 | * --width
32 | カメラキャプチャ時の横幅
33 | デフォルト：960 34 | * --height
35 | カメラキャプチャ時の縦幅
36 | デフォルト：540 37 | * --mirror
38 | VideoCapture()取り込みデータを左右反転するか否か
39 | デフォルト：指定なし 40 | * --model_select
41 | 使用モデルの選択
42 | Saved Model, ONNX：0→Lightning　1→Thunder
43 | TFLite：0→Lightning(float16)　1→Thunder(float16)　2→Lightning(int8)　3→Thunder(int8)
44 | デフォルト：0 45 | * --keypoint_score
46 | キーポイント表示の閾値
47 | デフォルト：0.4 48 | 49 | #### MultiPose 50 | ```bash 51 | python demo_multipose.py 52 | ``` 53 | * --device
54 | カメラデバイス番号の指定
55 | デフォルト：0 56 | * --file
57 | 動画ファイルの指定 ※指定時はカメラデバイスより優先
58 | デフォルト：指定なし 59 | * --width
60 | カメラキャプチャ時の横幅
61 | デフォルト：960 62 | * --height
63 | カメラキャプチャ時の縦幅
64 | デフォルト：540 65 | * --mirror
66 | VideoCapture()取り込みデータを左右反転するか否か
67 | デフォルト：指定なし 68 | * --keypoint_score
69 | キーポイント表示の閾値
70 | デフォルト：0.4 71 | * --bbox_score
72 | バウンディングボックス表示の閾値
73 | デフォルト：0.2 74 | 75 | # Reference 76 | * [TensorFlow Hub：MoveNet](https://tfhub.dev/s?q=MoveNet) 77 | 78 | # Author 79 | 高橋かずひと(https://twitter.com/KzhtTkhs) 80 | 81 | # License 82 | MoveNet-Python-Example is under [Apache-2.0 License](LICENSE). 83 | 84 | # License(Movie) 85 | サンプル動画は[NHKクリエイティブ・ライブラリー](https://www.nhk.or.jp/archives/creative/)の[ストリートバスケット](https://www2.nhk.or.jp/archives/creative/material/view.cgi?m=D0002080169_00000)を使用しています。 86 | -------------------------------------------------------------------------------- /demo_multipose.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | import copy 5 | import time 6 | import argparse 7 | 8 | import cv2 as cv 9 | import numpy as np 10 | import tensorflow as tf 11 | import tensorflow_hub as tfhub 12 | 13 | 14 | def get_args(): 15 | parser = argparse.ArgumentParser() 16 | 17 | parser.add_argument("--device", type=int, default=0) 18 | parser.add_argument("--file", type=str, default=None) 19 | parser.add_argument("--width", help='cap width', type=int, default=960) 20 | parser.add_argument("--height", help='cap height', type=int, default=540) 21 | 22 | parser.add_argument('--mirror', action='store_true') 23 | 24 | parser.add_argument("--keypoint_score", type=float, default=0.4) 25 | parser.add_argument("--bbox_score", type=float, default=0.2) 26 | 27 | args = parser.parse_args() 28 | 29 | return args 30 | 31 | 32 | def run_inference(model, input_size, image): 33 | image_width, image_height = image.shape[1], image.shape[0] 34 | 35 | # 前処理 36 | input_image = cv.resize(image, dsize=(input_size, input_size)) # リサイズ 37 | input_image = cv.cvtColor(input_image, cv.COLOR_BGR2RGB) # BGR→RGB変換 38 | input_image = input_image.reshape(-1, input_size, input_size, 3) # リシェイプ 39 | input_image = tf.cast(input_image, dtype=tf.int32) # int32へキャスト 40 | 41 | # 推論 42 | outputs = model(input_image) 43 | 44 | keypoints_with_scores = outputs['output_0'].numpy() 45 | keypoints_with_scores = np.squeeze(keypoints_with_scores) 46 | 47 | # キーポイント、バウンディングボックス、スコア取り出し 48 | keypoints_list, scores_list = [], [] 49 | bbox_list = [] 50 | for keypoints_with_score in keypoints_with_scores: 51 | keypoints = [] 52 | scores = [] 53 | # キーポイント 54 | for index in range(17): 55 | keypoint_x = int(image_width * 56 | keypoints_with_score[(index * 3) + 1]) 57 | keypoint_y = int(image_height * 58 | keypoints_with_score[(index * 3) + 0]) 59 | score = keypoints_with_score[(index * 3) + 2] 60 | 61 | keypoints.append([keypoint_x, keypoint_y]) 62 | scores.append(score) 63 | 64 | # バウンディングボックス 65 | bbox_ymin = int(image_height * keypoints_with_score[51]) 66 | bbox_xmin = int(image_width * keypoints_with_score[52]) 67 | bbox_ymax = int(image_height * keypoints_with_score[53]) 68 | bbox_xmax = int(image_width * keypoints_with_score[54]) 69 | bbox_score = keypoints_with_score[55] 70 | 71 | # 6人分のデータ格納用のリストに追加 72 | keypoints_list.append(keypoints) 73 | scores_list.append(scores) 74 | bbox_list.append( 75 | [bbox_xmin, bbox_ymin, bbox_xmax, bbox_ymax, bbox_score]) 76 | 77 | return keypoints_list, scores_list, bbox_list 78 | 79 | 80 | def main(): 81 | # 引数解析 ################################################################# 82 | args = get_args() 83 | cap_device = args.device 84 | cap_width = args.width 85 | cap_height = args.height 86 | 87 | if args.file is not None: 88 | cap_device = args.file 89 | 90 | mirror = args.mirror 91 | keypoint_score_th = args.keypoint_score 92 | bbox_score_th = args.bbox_score 93 | 94 | # カメラ準備 ############################################################### 95 | cap = cv.VideoCapture(cap_device) 96 | cap.set(cv.CAP_PROP_FRAME_WIDTH, cap_width) 97 | cap.set(cv.CAP_PROP_FRAME_HEIGHT, cap_height) 98 | 99 | # モデルロード ############################################################# 100 | model_url = "https://tfhub.dev/google/movenet/multipose/lightning/1" 101 | input_size = 256 102 | 103 | module = tfhub.load(model_url) 104 | model = module.signatures['serving_default'] 105 | 106 | while True: 107 | start_time = time.time() 108 | 109 | # カメラキャプチャ ##################################################### 110 | ret, frame = cap.read() 111 | if not ret: 112 | break 113 | if mirror: 114 | frame = cv.flip(frame, 1) # ミラー表示 115 | debug_image = copy.deepcopy(frame) 116 | 117 | # 検出実施 ############################################################## 118 | keypoints_list, scores_list, bbox_list = run_inference( 119 | model, 120 | input_size, 121 | frame, 122 | ) 123 | 124 | elapsed_time = time.time() - start_time 125 | 126 | # デバッグ描画 127 | debug_image = draw_debug( 128 | debug_image, 129 | elapsed_time, 130 | keypoint_score_th, 131 | keypoints_list, 132 | scores_list, 133 | bbox_score_th, 134 | bbox_list, 135 | ) 136 | 137 | # キー処理(ESC：終了) ################################################## 138 | key = cv.waitKey(1) 139 | if key == 27: # ESC 140 | break 141 | 142 | # 画面反映 ############################################################# 143 | cv.imshow('MoveNet(multipose) Demo', debug_image) 144 | 145 | cap.release() 146 | cv.destroyAllWindows() 147 | 148 | 149 | def draw_debug( 150 | image, 151 | elapsed_time, 152 | keypoint_score_th, 153 | keypoints_list, 154 | scores_list, 155 | bbox_score_th, 156 | bbox_list, 157 | ): 158 | debug_image = copy.deepcopy(image) 159 | 160 | # 0:鼻 1:左目 2:右目 3:左耳 4:右耳 5:左肩 6:右肩 7:左肘 8:右肘 # 9:左手首 161 | # 10:右手首 11:左股関節 12:右股関節 13:左ひざ 14:右ひざ 15:左足首 16:右足首 162 | for keypoints, scores in zip(keypoints_list, scores_list): 163 | # Line：鼻 → 左目 164 | index01, index02 = 0, 1 165 | if scores[index01] > keypoint_score_th and scores[ 166 | index02] > keypoint_score_th: 167 | point01 = keypoints[index01] 168 | point02 = keypoints[index02] 169 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 170 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 171 | # Line：鼻 → 右目 172 | index01, index02 = 0, 2 173 | if scores[index01] > keypoint_score_th and scores[ 174 | index02] > keypoint_score_th: 175 | point01 = keypoints[index01] 176 | point02 = keypoints[index02] 177 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 178 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 179 | # Line：左目 → 左耳 180 | index01, index02 = 1, 3 181 | if scores[index01] > keypoint_score_th and scores[ 182 | index02] > keypoint_score_th: 183 | point01 = keypoints[index01] 184 | point02 = keypoints[index02] 185 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 186 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 187 | # Line：右目 → 右耳 188 | index01, index02 = 2, 4 189 | if scores[index01] > keypoint_score_th and scores[ 190 | index02] > keypoint_score_th: 191 | point01 = keypoints[index01] 192 | point02 = keypoints[index02] 193 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 194 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 195 | # Line：鼻 → 左肩 196 | index01, index02 = 0, 5 197 | if scores[index01] > keypoint_score_th and scores[ 198 | index02] > keypoint_score_th: 199 | point01 = keypoints[index01] 200 | point02 = keypoints[index02] 201 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 202 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 203 | # Line：鼻 → 右肩 204 | index01, index02 = 0, 6 205 | if scores[index01] > keypoint_score_th and scores[ 206 | index02] > keypoint_score_th: 207 | point01 = keypoints[index01] 208 | point02 = keypoints[index02] 209 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 210 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 211 | # Line：左肩 → 右肩 212 | index01, index02 = 5, 6 213 | if scores[index01] > keypoint_score_th and scores[ 214 | index02] > keypoint_score_th: 215 | point01 = keypoints[index01] 216 | point02 = keypoints[index02] 217 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 218 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 219 | # Line：左肩 → 左肘 220 | index01, index02 = 5, 7 221 | if scores[index01] > keypoint_score_th and scores[ 222 | index02] > keypoint_score_th: 223 | point01 = keypoints[index01] 224 | point02 = keypoints[index02] 225 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 226 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 227 | # Line：左肘 → 左手首 228 | index01, index02 = 7, 9 229 | if scores[index01] > keypoint_score_th and scores[ 230 | index02] > keypoint_score_th: 231 | point01 = keypoints[index01] 232 | point02 = keypoints[index02] 233 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 234 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 235 | # Line：右肩 → 右肘 236 | index01, index02 = 6, 8 237 | if scores[index01] > keypoint_score_th and scores[ 238 | index02] > keypoint_score_th: 239 | point01 = keypoints[index01] 240 | point02 = keypoints[index02] 241 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 242 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 243 | # Line：右肘 → 右手首 244 | index01, index02 = 8, 10 245 | if scores[index01] > keypoint_score_th and scores[ 246 | index02] > keypoint_score_th: 247 | point01 = keypoints[index01] 248 | point02 = keypoints[index02] 249 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 250 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 251 | # Line：左股関節 → 右股関節 252 | index01, index02 = 11, 12 253 | if scores[index01] > keypoint_score_th and scores[ 254 | index02] > keypoint_score_th: 255 | point01 = keypoints[index01] 256 | point02 = keypoints[index02] 257 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 258 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 259 | # Line：左肩 → 左股関節 260 | index01, index02 = 5, 11 261 | if scores[index01] > keypoint_score_th and scores[ 262 | index02] > keypoint_score_th: 263 | point01 = keypoints[index01] 264 | point02 = keypoints[index02] 265 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 266 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 267 | # Line：左股関節 → 左ひざ 268 | index01, index02 = 11, 13 269 | if scores[index01] > keypoint_score_th and scores[ 270 | index02] > keypoint_score_th: 271 | point01 = keypoints[index01] 272 | point02 = keypoints[index02] 273 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 274 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 275 | # Line：左ひざ → 左足首 276 | index01, index02 = 13, 15 277 | if scores[index01] > keypoint_score_th and scores[ 278 | index02] > keypoint_score_th: 279 | point01 = keypoints[index01] 280 | point02 = keypoints[index02] 281 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 282 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 283 | # Line：右肩 → 右股関節 284 | index01, index02 = 6, 12 285 | if scores[index01] > keypoint_score_th and scores[ 286 | index02] > keypoint_score_th: 287 | point01 = keypoints[index01] 288 | point02 = keypoints[index02] 289 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 290 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 291 | # Line：右股関節 → 右ひざ 292 | index01, index02 = 12, 14 293 | if scores[index01] > keypoint_score_th and scores[ 294 | index02] > keypoint_score_th: 295 | point01 = keypoints[index01] 296 | point02 = keypoints[index02] 297 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 298 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 299 | # Line：右ひざ → 右足首 300 | index01, index02 = 14, 16 301 | if scores[index01] > keypoint_score_th and scores[ 302 | index02] > keypoint_score_th: 303 | point01 = keypoints[index01] 304 | point02 = keypoints[index02] 305 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 306 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 307 | 308 | # Circle：各点 309 | for keypoint, score in zip(keypoints, scores): 310 | if score > keypoint_score_th: 311 | cv.circle(debug_image, keypoint, 6, (255, 255, 255), -1) 312 | cv.circle(debug_image, keypoint, 3, (0, 0, 0), -1) 313 | 314 | # バウンディングボックス 315 | for bbox in bbox_list: 316 | if bbox[4] > bbox_score_th: 317 | cv.rectangle(debug_image, (bbox[0], bbox[1]), (bbox[2], bbox[3]), 318 | (255, 255, 255), 4) 319 | cv.rectangle(debug_image, (bbox[0], bbox[1]), (bbox[2], bbox[3]), 320 | (0, 0, 0), 2) 321 | 322 | # 処理時間 323 | cv.putText(debug_image, 324 | "Elapsed Time : " + '{:.1f}'.format(elapsed_time * 1000) + "ms", 325 | (10, 30), cv.FONT_HERSHEY_SIMPLEX, 0.7, (255, 255, 255), 4, 326 | cv.LINE_AA) 327 | cv.putText(debug_image, 328 | "Elapsed Time : " + '{:.1f}'.format(elapsed_time * 1000) + "ms", 329 | (10, 30), cv.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 0), 2, 330 | cv.LINE_AA) 331 | 332 | return debug_image 333 | 334 | 335 | if __name__ == '__main__': 336 | main() -------------------------------------------------------------------------------- /demo_multipose_onnx.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | import copy 5 | import time 6 | import argparse 7 | 8 | import cv2 as cv 9 | import numpy as np 10 | import onnxruntime 11 | import tensorflow as tf 12 | 13 | 14 | def get_args(): 15 | parser = argparse.ArgumentParser() 16 | 17 | parser.add_argument("--device", type=int, default=0) 18 | parser.add_argument("--file", type=str, default=None) 19 | parser.add_argument("--width", help='cap width', type=int, default=960) 20 | parser.add_argument("--height", help='cap height', type=int, default=540) 21 | 22 | parser.add_argument('--mirror', action='store_true') 23 | 24 | parser.add_argument("--keypoint_score", type=float, default=0.4) 25 | parser.add_argument("--bbox_score", type=float, default=0.2) 26 | 27 | args = parser.parse_args() 28 | 29 | return args 30 | 31 | 32 | def run_inference(onnx_session, input_size, image): 33 | image_width, image_height = image.shape[1], image.shape[0] 34 | 35 | # 前処理 36 | input_image = cv.resize(image, dsize=(input_size, input_size)) # リサイズ 37 | input_image = cv.cvtColor(input_image, cv.COLOR_BGR2RGB) # BGR→RGB変換 38 | input_image = input_image.reshape(-1, input_size, input_size, 3) # リシェイプ 39 | input_image = input_image.astype('int32') # int32へキャスト 40 | 41 | # 推論 42 | input_name = onnx_session.get_inputs()[0].name 43 | output_name = onnx_session.get_outputs()[0].name 44 | outputs = onnx_session.run([output_name], {input_name: input_image}) 45 | 46 | keypoints_with_scores = outputs[0] 47 | keypoints_with_scores = np.squeeze(keypoints_with_scores) 48 | 49 | # キーポイント、バウンディングボックス、スコア取り出し 50 | keypoints_list, scores_list = [], [] 51 | bbox_list = [] 52 | for keypoints_with_score in keypoints_with_scores: 53 | keypoints = [] 54 | scores = [] 55 | # キーポイント 56 | for index in range(17): 57 | keypoint_x = int(image_width * 58 | keypoints_with_score[(index * 3) + 1]) 59 | keypoint_y = int(image_height * 60 | keypoints_with_score[(index * 3) + 0]) 61 | score = keypoints_with_score[(index * 3) + 2] 62 | 63 | keypoints.append([keypoint_x, keypoint_y]) 64 | scores.append(score) 65 | 66 | # バウンディングボックス 67 | bbox_ymin = int(image_height * keypoints_with_score[51]) 68 | bbox_xmin = int(image_width * keypoints_with_score[52]) 69 | bbox_ymax = int(image_height * keypoints_with_score[53]) 70 | bbox_xmax = int(image_width * keypoints_with_score[54]) 71 | bbox_score = keypoints_with_score[55] 72 | 73 | # 6人分のデータ格納用のリストに追加 74 | keypoints_list.append(keypoints) 75 | scores_list.append(scores) 76 | bbox_list.append( 77 | [bbox_xmin, bbox_ymin, bbox_xmax, bbox_ymax, bbox_score]) 78 | 79 | return keypoints_list, scores_list, bbox_list 80 | 81 | 82 | def main(): 83 | # 引数解析 ################################################################# 84 | args = get_args() 85 | cap_device = args.device 86 | cap_width = args.width 87 | cap_height = args.height 88 | 89 | if args.file is not None: 90 | cap_device = args.file 91 | 92 | mirror = args.mirror 93 | keypoint_score_th = args.keypoint_score 94 | bbox_score_th = args.bbox_score 95 | 96 | # カメラ準備 ############################################################### 97 | cap = cv.VideoCapture(cap_device) 98 | cap.set(cv.CAP_PROP_FRAME_WIDTH, cap_width) 99 | cap.set(cv.CAP_PROP_FRAME_HEIGHT, cap_height) 100 | 101 | # モデルロード ############################################################# 102 | model_path = "onnx/movenet_multipose_lightning_1.onnx" 103 | input_size = 256 104 | 105 | onnx_session = onnxruntime.InferenceSession( 106 | model_path, 107 | providers=[ 108 | 'CUDAExecutionProvider', 109 | 'CPUExecutionProvider', 110 | ], 111 | ) 112 | 113 | while True: 114 | start_time = time.time() 115 | 116 | # カメラキャプチャ ##################################################### 117 | ret, frame = cap.read() 118 | if not ret: 119 | break 120 | if mirror: 121 | frame = cv.flip(frame, 1) # ミラー表示 122 | debug_image = copy.deepcopy(frame) 123 | 124 | # 検出実施 ############################################################## 125 | keypoints_list, scores_list, bbox_list = run_inference( 126 | onnx_session, 127 | input_size, 128 | frame, 129 | ) 130 | 131 | elapsed_time = time.time() - start_time 132 | 133 | # デバッグ描画 134 | debug_image = draw_debug( 135 | debug_image, 136 | elapsed_time, 137 | keypoint_score_th, 138 | keypoints_list, 139 | scores_list, 140 | bbox_score_th, 141 | bbox_list, 142 | ) 143 | 144 | # キー処理(ESC：終了) ################################################## 145 | key = cv.waitKey(1) 146 | if key == 27: # ESC 147 | break 148 | 149 | # 画面反映 ############################################################# 150 | cv.imshow('MoveNet(multipose) Demo', debug_image) 151 | 152 | cap.release() 153 | cv.destroyAllWindows() 154 | 155 | 156 | def draw_debug( 157 | image, 158 | elapsed_time, 159 | keypoint_score_th, 160 | keypoints_list, 161 | scores_list, 162 | bbox_score_th, 163 | bbox_list, 164 | ): 165 | debug_image = copy.deepcopy(image) 166 | 167 | # 0:鼻 1:左目 2:右目 3:左耳 4:右耳 5:左肩 6:右肩 7:左肘 8:右肘 # 9:左手首 168 | # 10:右手首 11:左股関節 12:右股関節 13:左ひざ 14:右ひざ 15:左足首 16:右足首 169 | for keypoints, scores in zip(keypoints_list, scores_list): 170 | # Line：鼻 → 左目 171 | index01, index02 = 0, 1 172 | if scores[index01] > keypoint_score_th and scores[ 173 | index02] > keypoint_score_th: 174 | point01 = keypoints[index01] 175 | point02 = keypoints[index02] 176 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 177 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 178 | # Line：鼻 → 右目 179 | index01, index02 = 0, 2 180 | if scores[index01] > keypoint_score_th and scores[ 181 | index02] > keypoint_score_th: 182 | point01 = keypoints[index01] 183 | point02 = keypoints[index02] 184 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 185 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 186 | # Line：左目 → 左耳 187 | index01, index02 = 1, 3 188 | if scores[index01] > keypoint_score_th and scores[ 189 | index02] > keypoint_score_th: 190 | point01 = keypoints[index01] 191 | point02 = keypoints[index02] 192 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 193 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 194 | # Line：右目 → 右耳 195 | index01, index02 = 2, 4 196 | if scores[index01] > keypoint_score_th and scores[ 197 | index02] > keypoint_score_th: 198 | point01 = keypoints[index01] 199 | point02 = keypoints[index02] 200 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 201 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 202 | # Line：鼻 → 左肩 203 | index01, index02 = 0, 5 204 | if scores[index01] > keypoint_score_th and scores[ 205 | index02] > keypoint_score_th: 206 | point01 = keypoints[index01] 207 | point02 = keypoints[index02] 208 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 209 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 210 | # Line：鼻 → 右肩 211 | index01, index02 = 0, 6 212 | if scores[index01] > keypoint_score_th and scores[ 213 | index02] > keypoint_score_th: 214 | point01 = keypoints[index01] 215 | point02 = keypoints[index02] 216 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 217 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 218 | # Line：左肩 → 右肩 219 | index01, index02 = 5, 6 220 | if scores[index01] > keypoint_score_th and scores[ 221 | index02] > keypoint_score_th: 222 | point01 = keypoints[index01] 223 | point02 = keypoints[index02] 224 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 225 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 226 | # Line：左肩 → 左肘 227 | index01, index02 = 5, 7 228 | if scores[index01] > keypoint_score_th and scores[ 229 | index02] > keypoint_score_th: 230 | point01 = keypoints[index01] 231 | point02 = keypoints[index02] 232 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 233 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 234 | # Line：左肘 → 左手首 235 | index01, index02 = 7, 9 236 | if scores[index01] > keypoint_score_th and scores[ 237 | index02] > keypoint_score_th: 238 | point01 = keypoints[index01] 239 | point02 = keypoints[index02] 240 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 241 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 242 | # Line：右肩 → 右肘 243 | index01, index02 = 6, 8 244 | if scores[index01] > keypoint_score_th and scores[ 245 | index02] > keypoint_score_th: 246 | point01 = keypoints[index01] 247 | point02 = keypoints[index02] 248 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 249 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 250 | # Line：右肘 → 右手首 251 | index01, index02 = 8, 10 252 | if scores[index01] > keypoint_score_th and scores[ 253 | index02] > keypoint_score_th: 254 | point01 = keypoints[index01] 255 | point02 = keypoints[index02] 256 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 257 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 258 | # Line：左股関節 → 右股関節 259 | index01, index02 = 11, 12 260 | if scores[index01] > keypoint_score_th and scores[ 261 | index02] > keypoint_score_th: 262 | point01 = keypoints[index01] 263 | point02 = keypoints[index02] 264 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 265 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 266 | # Line：左肩 → 左股関節 267 | index01, index02 = 5, 11 268 | if scores[index01] > keypoint_score_th and scores[ 269 | index02] > keypoint_score_th: 270 | point01 = keypoints[index01] 271 | point02 = keypoints[index02] 272 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 273 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 274 | # Line：左股関節 → 左ひざ 275 | index01, index02 = 11, 13 276 | if scores[index01] > keypoint_score_th and scores[ 277 | index02] > keypoint_score_th: 278 | point01 = keypoints[index01] 279 | point02 = keypoints[index02] 280 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 281 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 282 | # Line：左ひざ → 左足首 283 | index01, index02 = 13, 15 284 | if scores[index01] > keypoint_score_th and scores[ 285 | index02] > keypoint_score_th: 286 | point01 = keypoints[index01] 287 | point02 = keypoints[index02] 288 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 289 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 290 | # Line：右肩 → 右股関節 291 | index01, index02 = 6, 12 292 | if scores[index01] > keypoint_score_th and scores[ 293 | index02] > keypoint_score_th: 294 | point01 = keypoints[index01] 295 | point02 = keypoints[index02] 296 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 297 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 298 | # Line：右股関節 → 右ひざ 299 | index01, index02 = 12, 14 300 | if scores[index01] > keypoint_score_th and scores[ 301 | index02] > keypoint_score_th: 302 | point01 = keypoints[index01] 303 | point02 = keypoints[index02] 304 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 305 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 306 | # Line：右ひざ → 右足首 307 | index01, index02 = 14, 16 308 | if scores[index01] > keypoint_score_th and scores[ 309 | index02] > keypoint_score_th: 310 | point01 = keypoints[index01] 311 | point02 = keypoints[index02] 312 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 313 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 314 | 315 | # Circle：各点 316 | for keypoint, score in zip(keypoints, scores): 317 | if score > keypoint_score_th: 318 | cv.circle(debug_image, keypoint, 6, (255, 255, 255), -1) 319 | cv.circle(debug_image, keypoint, 3, (0, 0, 0), -1) 320 | 321 | # バウンディングボックス 322 | for bbox in bbox_list: 323 | if bbox[4] > bbox_score_th: 324 | cv.rectangle(debug_image, (bbox[0], bbox[1]), (bbox[2], bbox[3]), 325 | (255, 255, 255), 4) 326 | cv.rectangle(debug_image, (bbox[0], bbox[1]), (bbox[2], bbox[3]), 327 | (0, 0, 0), 2) 328 | 329 | # 処理時間 330 | cv.putText(debug_image, 331 | "Elapsed Time : " + '{:.1f}'.format(elapsed_time * 1000) + "ms", 332 | (10, 30), cv.FONT_HERSHEY_SIMPLEX, 0.7, (255, 255, 255), 4, 333 | cv.LINE_AA) 334 | cv.putText(debug_image, 335 | "Elapsed Time : " + '{:.1f}'.format(elapsed_time * 1000) + "ms", 336 | (10, 30), cv.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 0), 2, 337 | cv.LINE_AA) 338 | 339 | return debug_image 340 | 341 | 342 | if __name__ == '__main__': 343 | main() -------------------------------------------------------------------------------- /demo_singlepose.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | import sys 4 | import copy 5 | import time 6 | import argparse 7 | 8 | import cv2 as cv 9 | import numpy as np 10 | import tensorflow as tf 11 | import tensorflow_hub as tfhub 12 | 13 | 14 | def get_args(): 15 | parser = argparse.ArgumentParser() 16 | 17 | parser.add_argument("--device", type=int, default=0) 18 | parser.add_argument("--file", type=str, default=None) 19 | parser.add_argument("--width", help='cap width', type=int, default=960) 20 | parser.add_argument("--height", help='cap height', type=int, default=540) 21 | 22 | parser.add_argument('--mirror', action='store_true') 23 | 24 | parser.add_argument("--model_select", type=int, default=0) 25 | parser.add_argument("--keypoint_score", type=float, default=0.4) 26 | 27 | args = parser.parse_args() 28 | 29 | return args 30 | 31 | 32 | def run_inference(model, input_size, image): 33 | image_width, image_height = image.shape[1], image.shape[0] 34 | 35 | # 前処理 36 | input_image = cv.resize(image, dsize=(input_size, input_size)) # リサイズ 37 | input_image = cv.cvtColor(input_image, cv.COLOR_BGR2RGB) # BGR→RGB変換 38 | input_image = input_image.reshape(-1, input_size, input_size, 3) # リシェイプ 39 | input_image = tf.cast(input_image, dtype=tf.int32) # int32へキャスト 40 | 41 | # 推論 42 | outputs = model(input_image) 43 | 44 | keypoints_with_scores = outputs['output_0'].numpy() 45 | keypoints_with_scores = np.squeeze(keypoints_with_scores) 46 | 47 | # キーポイント、スコア取り出し 48 | keypoints = [] 49 | scores = [] 50 | for index in range(17): 51 | keypoint_x = int(image_width * keypoints_with_scores[index][1]) 52 | keypoint_y = int(image_height * keypoints_with_scores[index][0]) 53 | score = keypoints_with_scores[index][2] 54 | 55 | keypoints.append([keypoint_x, keypoint_y]) 56 | scores.append(score) 57 | 58 | return keypoints, scores 59 | 60 | 61 | def main(): 62 | # 引数解析 ################################################################# 63 | args = get_args() 64 | cap_device = args.device 65 | cap_width = args.width 66 | cap_height = args.height 67 | 68 | if args.file is not None: 69 | cap_device = args.file 70 | 71 | mirror = args.mirror 72 | model_select = args.model_select 73 | keypoint_score_th = args.keypoint_score 74 | 75 | # カメラ準備 ############################################################### 76 | cap = cv.VideoCapture(cap_device) 77 | cap.set(cv.CAP_PROP_FRAME_WIDTH, cap_width) 78 | cap.set(cv.CAP_PROP_FRAME_HEIGHT, cap_height) 79 | 80 | # モデルロード ############################################################# 81 | if model_select == 0: 82 | model_url = "https://tfhub.dev/google/movenet/singlepose/lightning/4" 83 | input_size = 192 84 | elif model_select == 1: 85 | model_url = "https://tfhub.dev/google/movenet/singlepose/thunder/4" 86 | input_size = 256 87 | else: 88 | sys.exit( 89 | "*** model_select {} is invalid value. Please use 0-1. ***".format( 90 | model_select)) 91 | 92 | module = tfhub.load(model_url) 93 | model = module.signatures['serving_default'] 94 | 95 | while True: 96 | start_time = time.time() 97 | 98 | # カメラキャプチャ ##################################################### 99 | ret, frame = cap.read() 100 | if not ret: 101 | break 102 | if mirror: 103 | frame = cv.flip(frame, 1) # ミラー表示 104 | debug_image = copy.deepcopy(frame) 105 | 106 | # 検出実施 ############################################################## 107 | keypoints, scores = run_inference( 108 | model, 109 | input_size, 110 | frame, 111 | ) 112 | 113 | elapsed_time = time.time() - start_time 114 | 115 | # デバッグ描画 116 | debug_image = draw_debug( 117 | debug_image, 118 | elapsed_time, 119 | keypoint_score_th, 120 | keypoints, 121 | scores, 122 | ) 123 | 124 | # キー処理(ESC：終了) ################################################## 125 | key = cv.waitKey(1) 126 | if key == 27: # ESC 127 | break 128 | 129 | # 画面反映 ############################################################# 130 | cv.imshow('MoveNet(singlepose) Demo', debug_image) 131 | 132 | cap.release() 133 | cv.destroyAllWindows() 134 | 135 | 136 | def draw_debug( 137 | image, 138 | elapsed_time, 139 | keypoint_score_th, 140 | keypoints, 141 | scores, 142 | ): 143 | debug_image = copy.deepcopy(image) 144 | 145 | # 0:鼻 1:左目 2:右目 3:左耳 4:右耳 5:左肩 6:右肩 7:左肘 8:右肘 # 9:左手首 146 | # 10:右手首 11:左股関節 12:右股関節 13:左ひざ 14:右ひざ 15:左足首 16:右足首 147 | # Line：鼻 → 左目 148 | index01, index02 = 0, 1 149 | if scores[index01] > keypoint_score_th and scores[ 150 | index02] > keypoint_score_th: 151 | point01 = keypoints[index01] 152 | point02 = keypoints[index02] 153 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 154 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 155 | # Line：鼻 → 右目 156 | index01, index02 = 0, 2 157 | if scores[index01] > keypoint_score_th and scores[ 158 | index02] > keypoint_score_th: 159 | point01 = keypoints[index01] 160 | point02 = keypoints[index02] 161 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 162 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 163 | # Line：左目 → 左耳 164 | index01, index02 = 1, 3 165 | if scores[index01] > keypoint_score_th and scores[ 166 | index02] > keypoint_score_th: 167 | point01 = keypoints[index01] 168 | point02 = keypoints[index02] 169 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 170 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 171 | # Line：右目 → 右耳 172 | index01, index02 = 2, 4 173 | if scores[index01] > keypoint_score_th and scores[ 174 | index02] > keypoint_score_th: 175 | point01 = keypoints[index01] 176 | point02 = keypoints[index02] 177 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 178 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 179 | # Line：鼻 → 左肩 180 | index01, index02 = 0, 5 181 | if scores[index01] > keypoint_score_th and scores[ 182 | index02] > keypoint_score_th: 183 | point01 = keypoints[index01] 184 | point02 = keypoints[index02] 185 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 186 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 187 | # Line：鼻 → 右肩 188 | index01, index02 = 0, 6 189 | if scores[index01] > keypoint_score_th and scores[ 190 | index02] > keypoint_score_th: 191 | point01 = keypoints[index01] 192 | point02 = keypoints[index02] 193 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 194 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 195 | # Line：左肩 → 右肩 196 | index01, index02 = 5, 6 197 | if scores[index01] > keypoint_score_th and scores[ 198 | index02] > keypoint_score_th: 199 | point01 = keypoints[index01] 200 | point02 = keypoints[index02] 201 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 202 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 203 | # Line：左肩 → 左肘 204 | index01, index02 = 5, 7 205 | if scores[index01] > keypoint_score_th and scores[ 206 | index02] > keypoint_score_th: 207 | point01 = keypoints[index01] 208 | point02 = keypoints[index02] 209 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 210 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 211 | # Line：左肘 → 左手首 212 | index01, index02 = 7, 9 213 | if scores[index01] > keypoint_score_th and scores[ 214 | index02] > keypoint_score_th: 215 | point01 = keypoints[index01] 216 | point02 = keypoints[index02] 217 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 218 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 219 | # Line：右肩 → 右肘 220 | index01, index02 = 6, 8 221 | if scores[index01] > keypoint_score_th and scores[ 222 | index02] > keypoint_score_th: 223 | point01 = keypoints[index01] 224 | point02 = keypoints[index02] 225 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 226 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 227 | # Line：右肘 → 右手首 228 | index01, index02 = 8, 10 229 | if scores[index01] > keypoint_score_th and scores[ 230 | index02] > keypoint_score_th: 231 | point01 = keypoints[index01] 232 | point02 = keypoints[index02] 233 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 234 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 235 | # Line：左股関節 → 右股関節 236 | index01, index02 = 11, 12 237 | if scores[index01] > keypoint_score_th and scores[ 238 | index02] > keypoint_score_th: 239 | point01 = keypoints[index01] 240 | point02 = keypoints[index02] 241 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 242 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 243 | # Line：左肩 → 左股関節 244 | index01, index02 = 5, 11 245 | if scores[index01] > keypoint_score_th and scores[ 246 | index02] > keypoint_score_th: 247 | point01 = keypoints[index01] 248 | point02 = keypoints[index02] 249 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 250 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 251 | # Line：左股関節 → 左ひざ 252 | index01, index02 = 11, 13 253 | if scores[index01] > keypoint_score_th and scores[ 254 | index02] > keypoint_score_th: 255 | point01 = keypoints[index01] 256 | point02 = keypoints[index02] 257 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 258 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 259 | # Line：左ひざ → 左足首 260 | index01, index02 = 13, 15 261 | if scores[index01] > keypoint_score_th and scores[ 262 | index02] > keypoint_score_th: 263 | point01 = keypoints[index01] 264 | point02 = keypoints[index02] 265 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 266 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 267 | # Line：右肩 → 右股関節 268 | index01, index02 = 6, 12 269 | if scores[index01] > keypoint_score_th and scores[ 270 | index02] > keypoint_score_th: 271 | point01 = keypoints[index01] 272 | point02 = keypoints[index02] 273 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 274 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 275 | # Line：右股関節 → 右ひざ 276 | index01, index02 = 12, 14 277 | if scores[index01] > keypoint_score_th and scores[ 278 | index02] > keypoint_score_th: 279 | point01 = keypoints[index01] 280 | point02 = keypoints[index02] 281 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 282 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 283 | # Line：右ひざ → 右足首 284 | index01, index02 = 14, 16 285 | if scores[index01] > keypoint_score_th and scores[ 286 | index02] > keypoint_score_th: 287 | point01 = keypoints[index01] 288 | point02 = keypoints[index02] 289 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 290 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 291 | 292 | # Circle：各点 293 | for keypoint, score in zip(keypoints, scores): 294 | if score > keypoint_score_th: 295 | cv.circle(debug_image, keypoint, 6, (255, 255, 255), -1) 296 | cv.circle(debug_image, keypoint, 3, (0, 0, 0), -1) 297 | 298 | # 処理時間 299 | cv.putText(debug_image, 300 | "Elapsed Time : " + '{:.1f}'.format(elapsed_time * 1000) + "ms", 301 | (10, 30), cv.FONT_HERSHEY_SIMPLEX, 0.7, (255, 255, 255), 4, 302 | cv.LINE_AA) 303 | cv.putText(debug_image, 304 | "Elapsed Time : " + '{:.1f}'.format(elapsed_time * 1000) + "ms", 305 | (10, 30), cv.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 0), 2, 306 | cv.LINE_AA) 307 | 308 | return debug_image 309 | 310 | 311 | if __name__ == '__main__': 312 | main() -------------------------------------------------------------------------------- /demo_singlepose_onnx.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | import sys 4 | import copy 5 | import time 6 | import argparse 7 | 8 | import cv2 as cv 9 | import numpy as np 10 | import onnxruntime 11 | import tensorflow as tf 12 | 13 | 14 | def get_args(): 15 | parser = argparse.ArgumentParser() 16 | 17 | parser.add_argument("--device", type=int, default=0) 18 | parser.add_argument("--file", type=str, default=None) 19 | parser.add_argument("--width", help='cap width', type=int, default=960) 20 | parser.add_argument("--height", help='cap height', type=int, default=540) 21 | 22 | parser.add_argument('--mirror', action='store_true') 23 | 24 | parser.add_argument("--model_select", type=int, default=0) 25 | parser.add_argument("--keypoint_score", type=float, default=0.4) 26 | 27 | args = parser.parse_args() 28 | 29 | return args 30 | 31 | 32 | def run_inference(onnx_session, input_size, image): 33 | image_width, image_height = image.shape[1], image.shape[0] 34 | 35 | # 前処理 36 | input_image = cv.resize(image, dsize=(input_size, input_size)) # リサイズ 37 | input_image = cv.cvtColor(input_image, cv.COLOR_BGR2RGB) # BGR→RGB変換 38 | input_image = input_image.reshape(-1, input_size, input_size, 3) # リシェイプ 39 | input_image = input_image.astype('int32') # int32へキャスト 40 | 41 | # 推論 42 | input_name = onnx_session.get_inputs()[0].name 43 | output_name = onnx_session.get_outputs()[0].name 44 | outputs = onnx_session.run([output_name], {input_name: input_image}) 45 | 46 | keypoints_with_scores = outputs[0] 47 | keypoints_with_scores = np.squeeze(keypoints_with_scores) 48 | 49 | # キーポイント、スコア取り出し 50 | keypoints = [] 51 | scores = [] 52 | for index in range(17): 53 | keypoint_x = int(image_width * keypoints_with_scores[index][1]) 54 | keypoint_y = int(image_height * keypoints_with_scores[index][0]) 55 | score = keypoints_with_scores[index][2] 56 | 57 | keypoints.append([keypoint_x, keypoint_y]) 58 | scores.append(score) 59 | 60 | return keypoints, scores 61 | 62 | 63 | def main(): 64 | # 引数解析 ################################################################# 65 | args = get_args() 66 | cap_device = args.device 67 | cap_width = args.width 68 | cap_height = args.height 69 | 70 | if args.file is not None: 71 | cap_device = args.file 72 | 73 | mirror = args.mirror 74 | model_select = args.model_select 75 | keypoint_score_th = args.keypoint_score 76 | 77 | # カメラ準備 ############################################################### 78 | cap = cv.VideoCapture(cap_device) 79 | cap.set(cv.CAP_PROP_FRAME_WIDTH, cap_width) 80 | cap.set(cv.CAP_PROP_FRAME_HEIGHT, cap_height) 81 | 82 | # モデルロード ############################################################# 83 | if model_select == 0: 84 | model_path = "onnx/movenet_singlepose_lightning_4.onnx" 85 | input_size = 192 86 | elif model_select == 1: 87 | model_path = "onnx/movenet_singlepose_thunder_4.onnx" 88 | input_size = 256 89 | else: 90 | sys.exit( 91 | "*** model_select {} is invalid value. Please use 0-1. ***".format( 92 | model_select)) 93 | 94 | onnx_session = onnxruntime.InferenceSession( 95 | model_path, 96 | providers=[ 97 | 'CUDAExecutionProvider', 98 | 'CPUExecutionProvider', 99 | ], 100 | ) 101 | 102 | while True: 103 | start_time = time.time() 104 | 105 | # カメラキャプチャ ##################################################### 106 | ret, frame = cap.read() 107 | if not ret: 108 | break 109 | if mirror: 110 | frame = cv.flip(frame, 1) # ミラー表示 111 | debug_image = copy.deepcopy(frame) 112 | 113 | # 検出実施 ############################################################## 114 | keypoints, scores = run_inference( 115 | onnx_session, 116 | input_size, 117 | frame, 118 | ) 119 | 120 | elapsed_time = time.time() - start_time 121 | 122 | # デバッグ描画 123 | debug_image = draw_debug( 124 | debug_image, 125 | elapsed_time, 126 | keypoint_score_th, 127 | keypoints, 128 | scores, 129 | ) 130 | 131 | # キー処理(ESC：終了) ################################################## 132 | key = cv.waitKey(1) 133 | if key == 27: # ESC 134 | break 135 | 136 | # 画面反映 ############################################################# 137 | cv.imshow('MoveNet(singlepose) Demo', debug_image) 138 | 139 | cap.release() 140 | cv.destroyAllWindows() 141 | 142 | 143 | def draw_debug( 144 | image, 145 | elapsed_time, 146 | keypoint_score_th, 147 | keypoints, 148 | scores, 149 | ): 150 | debug_image = copy.deepcopy(image) 151 | 152 | # 0:鼻 1:左目 2:右目 3:左耳 4:右耳 5:左肩 6:右肩 7:左肘 8:右肘 # 9:左手首 153 | # 10:右手首 11:左股関節 12:右股関節 13:左ひざ 14:右ひざ 15:左足首 16:右足首 154 | # Line：鼻 → 左目 155 | index01, index02 = 0, 1 156 | if scores[index01] > keypoint_score_th and scores[ 157 | index02] > keypoint_score_th: 158 | point01 = keypoints[index01] 159 | point02 = keypoints[index02] 160 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 161 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 162 | # Line：鼻 → 右目 163 | index01, index02 = 0, 2 164 | if scores[index01] > keypoint_score_th and scores[ 165 | index02] > keypoint_score_th: 166 | point01 = keypoints[index01] 167 | point02 = keypoints[index02] 168 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 169 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 170 | # Line：左目 → 左耳 171 | index01, index02 = 1, 3 172 | if scores[index01] > keypoint_score_th and scores[ 173 | index02] > keypoint_score_th: 174 | point01 = keypoints[index01] 175 | point02 = keypoints[index02] 176 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 177 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 178 | # Line：右目 → 右耳 179 | index01, index02 = 2, 4 180 | if scores[index01] > keypoint_score_th and scores[ 181 | index02] > keypoint_score_th: 182 | point01 = keypoints[index01] 183 | point02 = keypoints[index02] 184 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 185 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 186 | # Line：鼻 → 左肩 187 | index01, index02 = 0, 5 188 | if scores[index01] > keypoint_score_th and scores[ 189 | index02] > keypoint_score_th: 190 | point01 = keypoints[index01] 191 | point02 = keypoints[index02] 192 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 193 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 194 | # Line：鼻 → 右肩 195 | index01, index02 = 0, 6 196 | if scores[index01] > keypoint_score_th and scores[ 197 | index02] > keypoint_score_th: 198 | point01 = keypoints[index01] 199 | point02 = keypoints[index02] 200 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 201 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 202 | # Line：左肩 → 右肩 203 | index01, index02 = 5, 6 204 | if scores[index01] > keypoint_score_th and scores[ 205 | index02] > keypoint_score_th: 206 | point01 = keypoints[index01] 207 | point02 = keypoints[index02] 208 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 209 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 210 | # Line：左肩 → 左肘 211 | index01, index02 = 5, 7 212 | if scores[index01] > keypoint_score_th and scores[ 213 | index02] > keypoint_score_th: 214 | point01 = keypoints[index01] 215 | point02 = keypoints[index02] 216 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 217 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 218 | # Line：左肘 → 左手首 219 | index01, index02 = 7, 9 220 | if scores[index01] > keypoint_score_th and scores[ 221 | index02] > keypoint_score_th: 222 | point01 = keypoints[index01] 223 | point02 = keypoints[index02] 224 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 225 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 226 | # Line：右肩 → 右肘 227 | index01, index02 = 6, 8 228 | if scores[index01] > keypoint_score_th and scores[ 229 | index02] > keypoint_score_th: 230 | point01 = keypoints[index01] 231 | point02 = keypoints[index02] 232 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 233 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 234 | # Line：右肘 → 右手首 235 | index01, index02 = 8, 10 236 | if scores[index01] > keypoint_score_th and scores[ 237 | index02] > keypoint_score_th: 238 | point01 = keypoints[index01] 239 | point02 = keypoints[index02] 240 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 241 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 242 | # Line：左股関節 → 右股関節 243 | index01, index02 = 11, 12 244 | if scores[index01] > keypoint_score_th and scores[ 245 | index02] > keypoint_score_th: 246 | point01 = keypoints[index01] 247 | point02 = keypoints[index02] 248 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 249 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 250 | # Line：左肩 → 左股関節 251 | index01, index02 = 5, 11 252 | if scores[index01] > keypoint_score_th and scores[ 253 | index02] > keypoint_score_th: 254 | point01 = keypoints[index01] 255 | point02 = keypoints[index02] 256 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 257 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 258 | # Line：左股関節 → 左ひざ 259 | index01, index02 = 11, 13 260 | if scores[index01] > keypoint_score_th and scores[ 261 | index02] > keypoint_score_th: 262 | point01 = keypoints[index01] 263 | point02 = keypoints[index02] 264 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 265 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 266 | # Line：左ひざ → 左足首 267 | index01, index02 = 13, 15 268 | if scores[index01] > keypoint_score_th and scores[ 269 | index02] > keypoint_score_th: 270 | point01 = keypoints[index01] 271 | point02 = keypoints[index02] 272 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 273 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 274 | # Line：右肩 → 右股関節 275 | index01, index02 = 6, 12 276 | if scores[index01] > keypoint_score_th and scores[ 277 | index02] > keypoint_score_th: 278 | point01 = keypoints[index01] 279 | point02 = keypoints[index02] 280 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 281 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 282 | # Line：右股関節 → 右ひざ 283 | index01, index02 = 12, 14 284 | if scores[index01] > keypoint_score_th and scores[ 285 | index02] > keypoint_score_th: 286 | point01 = keypoints[index01] 287 | point02 = keypoints[index02] 288 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 289 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 290 | # Line：右ひざ → 右足首 291 | index01, index02 = 14, 16 292 | if scores[index01] > keypoint_score_th and scores[ 293 | index02] > keypoint_score_th: 294 | point01 = keypoints[index01] 295 | point02 = keypoints[index02] 296 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 297 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 298 | 299 | # Circle：各点 300 | for keypoint, score in zip(keypoints, scores): 301 | if score > keypoint_score_th: 302 | cv.circle(debug_image, keypoint, 6, (255, 255, 255), -1) 303 | cv.circle(debug_image, keypoint, 3, (0, 0, 0), -1) 304 | 305 | # 処理時間 306 | cv.putText(debug_image, 307 | "Elapsed Time : " + '{:.1f}'.format(elapsed_time * 1000) + "ms", 308 | (10, 30), cv.FONT_HERSHEY_SIMPLEX, 0.7, (255, 255, 255), 4, 309 | cv.LINE_AA) 310 | cv.putText(debug_image, 311 | "Elapsed Time : " + '{:.1f}'.format(elapsed_time * 1000) + "ms", 312 | (10, 30), cv.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 0), 2, 313 | cv.LINE_AA) 314 | 315 | return debug_image 316 | 317 | 318 | if __name__ == '__main__': 319 | main() -------------------------------------------------------------------------------- /demo_singlepose_tflite.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | import sys 4 | import copy 5 | import time 6 | import argparse 7 | 8 | import cv2 as cv 9 | import numpy as np 10 | import tensorflow as tf 11 | 12 | 13 | def get_args(): 14 | parser = argparse.ArgumentParser() 15 | 16 | parser.add_argument("--device", type=int, default=0) 17 | parser.add_argument("--file", type=str, default=None) 18 | parser.add_argument("--width", help='cap width', type=int, default=960) 19 | parser.add_argument("--height", help='cap height', type=int, default=540) 20 | 21 | parser.add_argument('--mirror', action='store_true') 22 | 23 | parser.add_argument("--model_select", type=int, default=0) 24 | parser.add_argument("--keypoint_score", type=float, default=0.4) 25 | 26 | args = parser.parse_args() 27 | 28 | return args 29 | 30 | 31 | def run_inference(interpreter, input_size, image): 32 | image_width, image_height = image.shape[1], image.shape[0] 33 | 34 | # 前処理 35 | input_image = cv.resize(image, dsize=(input_size, input_size)) # リサイズ 36 | input_image = cv.cvtColor(input_image, cv.COLOR_BGR2RGB) # BGR→RGB変換 37 | input_image = input_image.reshape(-1, input_size, input_size, 3) # リシェイプ 38 | input_image = tf.cast(input_image, dtype=tf.uint8) # uint8へキャスト 39 | 40 | # 推論 41 | input_details = interpreter.get_input_details() 42 | interpreter.set_tensor(input_details[0]['index'], input_image.numpy()) 43 | interpreter.invoke() 44 | 45 | output_details = interpreter.get_output_details() 46 | keypoints_with_scores = interpreter.get_tensor(output_details[0]['index']) 47 | keypoints_with_scores = np.squeeze(keypoints_with_scores) 48 | 49 | # キーポイント、スコア取り出し 50 | keypoints = [] 51 | scores = [] 52 | for index in range(17): 53 | keypoint_x = int(image_width * keypoints_with_scores[index][1]) 54 | keypoint_y = int(image_height * keypoints_with_scores[index][0]) 55 | score = keypoints_with_scores[index][2] 56 | 57 | keypoints.append([keypoint_x, keypoint_y]) 58 | scores.append(score) 59 | 60 | return keypoints, scores 61 | 62 | 63 | def main(): 64 | # 引数解析 ################################################################# 65 | args = get_args() 66 | cap_device = args.device 67 | cap_width = args.width 68 | cap_height = args.height 69 | 70 | if args.file is not None: 71 | cap_device = args.file 72 | 73 | mirror = args.mirror 74 | model_select = args.model_select 75 | keypoint_score_th = args.keypoint_score 76 | 77 | # カメラ準備 ############################################################### 78 | cap = cv.VideoCapture(cap_device) 79 | cap.set(cv.CAP_PROP_FRAME_WIDTH, cap_width) 80 | cap.set(cv.CAP_PROP_FRAME_HEIGHT, cap_height) 81 | 82 | # モデルロード ############################################################# 83 | if model_select == 0: 84 | model_path = 'tflite/lite-model_movenet_singlepose_lightning_tflite_float16_4.tflite' 85 | input_size = 192 86 | elif model_select == 1: 87 | model_path = 'tflite/lite-model_movenet_singlepose_thunder_tflite_float16_4.tflite' 88 | input_size = 256 89 | elif model_select == 2: 90 | model_path = 'tflite/lite-model_movenet_singlepose_lightning_tflite_int8_4.tflite' 91 | input_size = 192 92 | elif model_select == 3: 93 | model_path = 'tflite/lite-model_movenet_singlepose_thunder_tflite_int8_4.tflite' 94 | input_size = 256 95 | else: 96 | sys.exit( 97 | "*** model_select {} is invalid value. Please use 0-3. ***".format( 98 | model_select)) 99 | 100 | interpreter = tf.lite.Interpreter(model_path=model_path) 101 | interpreter.allocate_tensors() 102 | 103 | while True: 104 | start_time = time.time() 105 | 106 | # カメラキャプチャ ##################################################### 107 | ret, frame = cap.read() 108 | if not ret: 109 | break 110 | if mirror: 111 | frame = cv.flip(frame, 1) # ミラー表示 112 | debug_image = copy.deepcopy(frame) 113 | 114 | # 検出実施 ############################################################## 115 | keypoints, scores = run_inference( 116 | interpreter, 117 | input_size, 118 | frame, 119 | ) 120 | 121 | elapsed_time = time.time() - start_time 122 | 123 | # デバッグ描画 124 | debug_image = draw_debug( 125 | debug_image, 126 | elapsed_time, 127 | keypoint_score_th, 128 | keypoints, 129 | scores, 130 | ) 131 | 132 | # キー処理(ESC：終了) ################################################## 133 | key = cv.waitKey(1) 134 | if key == 27: # ESC 135 | break 136 | 137 | # 画面反映 ############################################################# 138 | cv.imshow('MoveNet(singlepose) Demo', debug_image) 139 | 140 | cap.release() 141 | cv.destroyAllWindows() 142 | 143 | 144 | def draw_debug( 145 | image, 146 | elapsed_time, 147 | keypoint_score_th, 148 | keypoints, 149 | scores, 150 | ): 151 | debug_image = copy.deepcopy(image) 152 | 153 | # 0:鼻 1:左目 2:右目 3:左耳 4:右耳 5:左肩 6:右肩 7:左肘 8:右肘 # 9:左手首 154 | # 10:右手首 11:左股関節 12:右股関節 13:左ひざ 14:右ひざ 15:左足首 16:右足首 155 | # Line：鼻 → 左目 156 | index01, index02 = 0, 1 157 | if scores[index01] > keypoint_score_th and scores[ 158 | index02] > keypoint_score_th: 159 | point01 = keypoints[index01] 160 | point02 = keypoints[index02] 161 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 162 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 163 | # Line：鼻 → 右目 164 | index01, index02 = 0, 2 165 | if scores[index01] > keypoint_score_th and scores[ 166 | index02] > keypoint_score_th: 167 | point01 = keypoints[index01] 168 | point02 = keypoints[index02] 169 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 170 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 171 | # Line：左目 → 左耳 172 | index01, index02 = 1, 3 173 | if scores[index01] > keypoint_score_th and scores[ 174 | index02] > keypoint_score_th: 175 | point01 = keypoints[index01] 176 | point02 = keypoints[index02] 177 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 178 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 179 | # Line：右目 → 右耳 180 | index01, index02 = 2, 4 181 | if scores[index01] > keypoint_score_th and scores[ 182 | index02] > keypoint_score_th: 183 | point01 = keypoints[index01] 184 | point02 = keypoints[index02] 185 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 186 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 187 | # Line：鼻 → 左肩 188 | index01, index02 = 0, 5 189 | if scores[index01] > keypoint_score_th and scores[ 190 | index02] > keypoint_score_th: 191 | point01 = keypoints[index01] 192 | point02 = keypoints[index02] 193 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 194 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 195 | # Line：鼻 → 右肩 196 | index01, index02 = 0, 6 197 | if scores[index01] > keypoint_score_th and scores[ 198 | index02] > keypoint_score_th: 199 | point01 = keypoints[index01] 200 | point02 = keypoints[index02] 201 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 202 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 203 | # Line：左肩 → 右肩 204 | index01, index02 = 5, 6 205 | if scores[index01] > keypoint_score_th and scores[ 206 | index02] > keypoint_score_th: 207 | point01 = keypoints[index01] 208 | point02 = keypoints[index02] 209 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 210 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 211 | # Line：左肩 → 左肘 212 | index01, index02 = 5, 7 213 | if scores[index01] > keypoint_score_th and scores[ 214 | index02] > keypoint_score_th: 215 | point01 = keypoints[index01] 216 | point02 = keypoints[index02] 217 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 218 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 219 | # Line：左肘 → 左手首 220 | index01, index02 = 7, 9 221 | if scores[index01] > keypoint_score_th and scores[ 222 | index02] > keypoint_score_th: 223 | point01 = keypoints[index01] 224 | point02 = keypoints[index02] 225 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 226 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 227 | # Line：右肩 → 右肘 228 | index01, index02 = 6, 8 229 | if scores[index01] > keypoint_score_th and scores[ 230 | index02] > keypoint_score_th: 231 | point01 = keypoints[index01] 232 | point02 = keypoints[index02] 233 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 234 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 235 | # Line：右肘 → 右手首 236 | index01, index02 = 8, 10 237 | if scores[index01] > keypoint_score_th and scores[ 238 | index02] > keypoint_score_th: 239 | point01 = keypoints[index01] 240 | point02 = keypoints[index02] 241 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 242 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 243 | # Line：左股関節 → 右股関節 244 | index01, index02 = 11, 12 245 | if scores[index01] > keypoint_score_th and scores[ 246 | index02] > keypoint_score_th: 247 | point01 = keypoints[index01] 248 | point02 = keypoints[index02] 249 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 250 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 251 | # Line：左肩 → 左股関節 252 | index01, index02 = 5, 11 253 | if scores[index01] > keypoint_score_th and scores[ 254 | index02] > keypoint_score_th: 255 | point01 = keypoints[index01] 256 | point02 = keypoints[index02] 257 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 258 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 259 | # Line：左股関節 → 左ひざ 260 | index01, index02 = 11, 13 261 | if scores[index01] > keypoint_score_th and scores[ 262 | index02] > keypoint_score_th: 263 | point01 = keypoints[index01] 264 | point02 = keypoints[index02] 265 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 266 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 267 | # Line：左ひざ → 左足首 268 | index01, index02 = 13, 15 269 | if scores[index01] > keypoint_score_th and scores[ 270 | index02] > keypoint_score_th: 271 | point01 = keypoints[index01] 272 | point02 = keypoints[index02] 273 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 274 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 275 | # Line：右肩 → 右股関節 276 | index01, index02 = 6, 12 277 | if scores[index01] > keypoint_score_th and scores[ 278 | index02] > keypoint_score_th: 279 | point01 = keypoints[index01] 280 | point02 = keypoints[index02] 281 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 282 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 283 | # Line：右股関節 → 右ひざ 284 | index01, index02 = 12, 14 285 | if scores[index01] > keypoint_score_th and scores[ 286 | index02] > keypoint_score_th: 287 | point01 = keypoints[index01] 288 | point02 = keypoints[index02] 289 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 290 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 291 | # Line：右ひざ → 右足首 292 | index01, index02 = 14, 16 293 | if scores[index01] > keypoint_score_th and scores[ 294 | index02] > keypoint_score_th: 295 | point01 = keypoints[index01] 296 | point02 = keypoints[index02] 297 | cv.line(debug_image, point01, point02, (255, 255, 255), 4) 298 | cv.line(debug_image, point01, point02, (0, 0, 0), 2) 299 | 300 | # Circle：各点 301 | for keypoint, score in zip(keypoints, scores): 302 | if score > keypoint_score_th: 303 | cv.circle(debug_image, keypoint, 6, (255, 255, 255), -1) 304 | cv.circle(debug_image, keypoint, 3, (0, 0, 0), -1) 305 | 306 | # 処理時間 307 | cv.putText(debug_image, 308 | "Elapsed Time : " + '{:.1f}'.format(elapsed_time * 1000) + "ms", 309 | (10, 30), cv.FONT_HERSHEY_SIMPLEX, 0.7, (255, 255, 255), 4, 310 | cv.LINE_AA) 311 | cv.putText(debug_image, 312 | "Elapsed Time : " + '{:.1f}'.format(elapsed_time * 1000) + "ms", 313 | (10, 30), cv.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 0), 2, 314 | cv.LINE_AA) 315 | 316 | return debug_image 317 | 318 | 319 | if __name__ == '__main__': 320 | main() -------------------------------------------------------------------------------- /onnx/movenet_multipose_lightning_1.onnx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kazuhito00/MoveNet-Python-Example/515743a113e49b4f31677b1f6252c0041f237e30/onnx/movenet_multipose_lightning_1.onnx -------------------------------------------------------------------------------- /onnx/movenet_singlepose_lightning_4.onnx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kazuhito00/MoveNet-Python-Example/515743a113e49b4f31677b1f6252c0041f237e30/onnx/movenet_singlepose_lightning_4.onnx -------------------------------------------------------------------------------- /onnx/movenet_singlepose_thunder_4.onnx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kazuhito00/MoveNet-Python-Example/515743a113e49b4f31677b1f6252c0041f237e30/onnx/movenet_singlepose_thunder_4.onnx -------------------------------------------------------------------------------- /tflite/lite-model_movenet_singlepose_lightning_tflite_float16_4.tflite: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kazuhito00/MoveNet-Python-Example/515743a113e49b4f31677b1f6252c0041f237e30/tflite/lite-model_movenet_singlepose_lightning_tflite_float16_4.tflite -------------------------------------------------------------------------------- /tflite/lite-model_movenet_singlepose_lightning_tflite_int8_4.tflite: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kazuhito00/MoveNet-Python-Example/515743a113e49b4f31677b1f6252c0041f237e30/tflite/lite-model_movenet_singlepose_lightning_tflite_int8_4.tflite -------------------------------------------------------------------------------- /tflite/lite-model_movenet_singlepose_thunder_tflite_float16_4.tflite: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kazuhito00/MoveNet-Python-Example/515743a113e49b4f31677b1f6252c0041f237e30/tflite/lite-model_movenet_singlepose_thunder_tflite_float16_4.tflite -------------------------------------------------------------------------------- /tflite/lite-model_movenet_singlepose_thunder_tflite_int8_4.tflite: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kazuhito00/MoveNet-Python-Example/515743a113e49b4f31677b1f6252c0041f237e30/tflite/lite-model_movenet_singlepose_thunder_tflite_int8_4.tflite --------------------------------------------------------------------------------