├── td_yolo.PNG └── README.md /td_yolo.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/furmanlukasz/TouchDesigner_YOLOv3/HEAD/td_yolo.PNG -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # TouchDesigner to Yolo_v3 (object detection) 2 | ![GitHub Logo](https://github.com/furmanlukasz/TouchDesigner_YOLOv3/blob/master/td_yolo.PNG) 3 | 4 | ## Summary 5 | Integration of Yolo<->Tensorflow objectdetection model to TouchDesigner. 6 | ### Big thanks to autors of all below repositories: 7 | * Spout <-> Python please refere [here](https://github.com/spiraltechnica/Spout-for-Python) 8 | * Tensorflow -> Yolo integration and training please refere [here](https://github.com/wizyoung/YOLOv3_TensorFlow) 9 | * To get idea how to train orginal Tensorflow-gpu object detection please refere [here](https://github.com/EdjeElectronics/TensorFlow-Object-Detection-API-Tutorial-Train-Multiple-Objects-Windows-10) 10 | * Socket and subprocess [tutorial](https://matthewragan.com/2019/08/14/touchdesigner-python-and-the-subprocess-module/?fbclid=IwAR3Jg4byLlK7_PwDKc1DTQalAYUBrLykMM_6rXGNjeoi17CC5PExHmlZwWU) 11 | 12 | 13 | ## Installation 14 | 15 | 1. Install and create a Anaconda enviroment for Python = 3.7: 16 | 17 | - [download Anaconda](https://www.anaconda.com/distribution/?gclid=EAIaIQobChMIk8aHlZaQ5gIVyKQYCh3aGgmgEAAYASAAEgLAp_D_BwE) and install it. 18 | - open Anaconda Prompt 19 | - create a new python enviroment named __cnn__ : 20 | `conda create -n cnn python=3.7` 21 | - activate enviroment we just create : `conda activate cnn` 22 | 2. Download and install [Cuda](https://developer.nvidia.com/cuda-10.0-download-archive) 10.0 after installing cuda make shure you have it in system envinroments if not just add seperetly following paths 23 | - `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\lib\x64` 24 | - `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\bin` 25 | - `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\include` 26 | - `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0` 27 | 3. Download and copy [CUDNN](https://developer.nvidia.com/compute/machine-learning/cudnn/secure/7.6.5.32/Production/10.0_20191031/cudnn-10.0-windows10-x64-v7.6.5.32.zip) (cudnn-10.0-windows10-x64-v7.6.5.32) following files 28 | - `\cudnn-10.0-windows10-x64-v7.6.5.32.zip\cuda\bin\cudnn64_7.dll` --> `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\bin` 29 | - `\cudnn-10.0-windows10-x64-v7.6.5.32.zip\cuda\include\cudnn.h` --> `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\include` 30 | - `\cudnn-10.0-windows10-x64-v7.6.5.32.zip\cuda\lib\x64\cudnn.lib` --> `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\lib\x64` 31 | 32 | 4. Open Anaconda Prompt and install dependencies for a new environment("copy->paste" line by line this commands and install it. If it's your first time with a command line and anaconda please watch the cmd line logs to take the right action when it aksing you for permission (y/n)): 33 | - `pip install tensorflow-gpu=1.14` 34 | - `pip install pillow` 35 | - `conda install -c conda-forge lxml` 36 | - `conda install -c anaconda cython` 37 | - `conda install -c anaconda contextlib2` 38 | - `conda install -c conda-forge matplotlib` 39 | - `conda install -c anaconda pandas` 40 | - `pip install opencv-python` 41 | - `conda install -c conda-forge tqdm` 42 | - `pip install pygame` 43 | - `pip install PyOpenGL` 44 | - `pip install python-osc` 45 | 46 | 5. Add site-packages to the python system path, fill the path with your details in places marked with apostrophes and execute it in anaconda prompt as before: 47 | - `set PYTHONPATH=C:\Users\'your_user_name'\.conda\envs\cnn\Lib\site-packages` 48 | 5. [Download](https://code.visualstudio.com/) and install Visual Studio Code 49 | 50 | 6. Setup VSCode: 51 | - download [this](https://drive.google.com/open?id=1aBPk3tAOk6WPLBuQ61MZAV0ajvuwIEQA) project folder (include pre-train models) and unzip it 52 | - open vscode as administrator 53 | - open folder /YOLOv3_TF 54 | - if ask you to install python extension say 'yes' 55 | - activate pop up menu: `ctrl+shift+p` 56 | - Set vscode to use 'cnn' environment as default python interpreter, type: `Python: Select Interpreter` and set it to `Python 3.7.5 64-bit ('cnn':conda) ` 57 | - open TouchDesigner project file: /YOLOv3_TF/TD_Yolo.toe 58 | - in the VScode - go to Explorer (ctrl+shift+e) and find the file named `ObjectDetection.py` right-click on a file and select 'Run Python File In Terminal' 59 | 60 | 7. How to use it: 61 | * Inside `TD_Yolo.toe` project you will find spout output to send the texture. 62 | * There is also little parser witch gives you back 10 objects at the time,(accuracy and bounding box position on screen). The code in python is not limited so you can build your own parser or extend this one. It will be updated in the future. This version is a very experimental level. 63 | * to set different resolution change line 26 and 27 in `ObjectDetection.py` and set the same size to constantTOP named 'set_resolution_here' in `TD_Yolo.toe` _*note, for now, is working only with squared resolution_ 64 | 65 | ### TO DO: 66 | 67 | * ### Training module 68 | * ### parser optimalization 69 | 70 | 71 | Have a fun playing with it! 72 | --------------------------------------------------------------------------------