├── .github
    └── FUNDING.yml
├── README.md
├── demo.png
├── index.html
├── main.js
├── recyclebin.png
└── styles.css


/.github/FUNDING.yml:
--------------------------------------------------------------------------------
 1 | # These are supported funding model platforms
 2 | 
 3 | github: [collidingScopes]
 4 | patreon: # Replace with a single Patreon username
 5 | open_collective: # Replace with a single Open Collective username
 6 | ko_fi: # Replace with a single Ko-fi username
 7 | tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
 8 | community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
 9 | liberapay: # Replace with a single Liberapay username
10 | issuehunt: # Replace with a single IssueHunt username
11 | lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
12 | polar: # Replace with a single Polar username
13 | buy_me_a_coffee: # Replace with a single Buy Me a Coffee username
14 | thanks_dev: # Replace with a single thanks.dev username
15 | custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']
16 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
 1 | # 3D Hand Tracking Demo [Shape Creator]
 2 | 
 3 | A threejs / WebGL / MediaPipe-powered interactive demo that allows you to control create and move 3D shapes using hand gestures in real-time.
 4 | 
 5 | <img src="demo.png">
 6 | 
 7 | ## Demo
 8 | 
 9 | Try the live demo: [https://collidingscopes.github.io/shape-creator-tutorial/](https://collidingscopes.github.io/shape-creator-tutorial/)
10 | 
11 | ## Requirements
12 | 
13 | - Modern web browser with WebGL support
14 | - Camera access
15 | 
16 | ## Technologies
17 | 
18 | - **Three.js** for 3D rendering
19 | - **MediaPipe** for hand tracking and gesture recognition
20 | - **HTML5 Canvas** for visual feedback
21 | - **JavaScript** for real-time interaction
22 | 
23 | ## Setup for Development
24 | 
25 | ```bash
26 | # Clone this repository
27 | git clone https://github.com/collidingScopes/shape-creator-tutorial
28 | 
29 | # Navigate to the project directory
30 | cd shape-creator-tutorial
31 | # Serve with your preferred method (example using Python)
32 | python -m http.server
33 | ```
34 | 
35 | Then navigate to `http://localhost:8000` in your browser.
36 | 
37 | ## License
38 | 
39 | MIT License
40 | 
41 | ## Credits
42 | 
43 | - Three.js - https://threejs.org/
44 | - MediaPipe - https://mediapipe.dev/
45 | 
46 | ## Related Projects
47 | 
48 | You might also like some of my other open source projects:
49 | 
50 | - [Threejs hand tracking tutorial](https://collidingScopes.github.io/threejs-handtracking-101) - Basic hand tracking setup with threejs and MediaPipe computer vision
51 | - [Particular Drift](https://collidingScopes.github.io/particular-drift) - Turn photos into flowing particle animations
52 | - [Liquid Logo](https://collidingScopes.github.io/liquid-logo) - Transform logos and icons into liquid metal animations
53 | - [Video-to-ASCII](https://collidingScopes.github.io/ascii) - Convert videos into ASCII pixel art
54 | 
55 | ## Contact
56 | 
57 | - Instagram: [@stereo.drift](https://www.instagram.com/stereo.drift/)
58 | - Twitter/X: [@measure_plan](https://x.com/measure_plan)
59 | - Email: [stereodriftvisuals@gmail.com](mailto:stereodriftvisuals@gmail.com)
60 | - GitHub: [collidingScopes](https://github.com/collidingScopes)
61 | 
62 | ## Donations
63 | 
64 | If you found this tool useful, feel free to buy me a coffee. 
65 | 
66 | My name is Alan, and I enjoy building open source software for computer vision, games, and more. This would be much appreciated during late-night coding sessions!
67 | 
68 | [![Buy Me A Coffee](https://www.buymeacoffee.com/assets/img/custom_images/yellow_img.png)](https://www.buymeacoffee.com/stereoDrift)


--------------------------------------------------------------------------------
/demo.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/collidingScopes/shape-creator-tutorial/eb3b1be41a72815a6dae5ce6b9168333f7f8d503/demo.png


--------------------------------------------------------------------------------
/index.html:
--------------------------------------------------------------------------------
 1 | <!DOCTYPE html>
 2 | <html lang="en">
 3 | <head>
 4 |   <meta charset="UTF-8" />
 5 |   <meta name="viewport" content="width=device-width, initial-scale=1.0" />
 6 |   <title>MediaPipe / Three.js Shape Creator</title>
 7 |   <script defer src="https://cloud.umami.is/script.js" data-website-id="eb59c81c-27cb-4e1d-9e8c-bfbe70c48cd9"></script>
 8 |   <link rel="stylesheet" href="styles.css">
 9 | 
10 |   <!-- Primary Meta Tags -->
11 |   <meta name="title" content="Shape Creator Tutorial">
12 |   <meta name="description" content="create/control 3D shapes with hand gestures">
13 | 
14 |   <!-- Open Graph / Facebook -->
15 |   <meta property="og:type" content="website">
16 |   <meta property="og:url" content="https://collidingscopes.github.io/shape-creator-tutorial/">
17 |   <meta property="og:title" content="Shape Creator Tutorial">
18 |   <meta property="og:description" content="create/control 3D shapes with hand gestures">
19 |   <meta property="og:image" content="https://raw.githubusercontent.com/collidingScopes/shape-creator-tutorial/main/demo.png">
20 | 
21 |   <!-- Twitter -->
22 |   <meta property="twitter:card" content="summary_large_image">
23 |   <meta property="twitter:url" content="https://collidingscopes.github.io/shape-creator-tutorial/">
24 |   <meta property="twitter:title" content="Shape Creator Tutorial">
25 |   <meta property="twitter:description" content="create/control 3D shapes with hand gestures">
26 |   <meta property="twitter:image" content="https://raw.githubusercontent.com/collidingScopes/shape-creator-tutorial/main/demo.png">
27 |   
28 | </head>
29 | <body>
30 | 
31 |   <video id="webcam" autoplay muted playsinline></video>
32 |   <canvas id="canvas"></canvas>
33 |   <div id="three-canvas"></div>
34 |   <img id="recycle-bin" src="recyclebin.png" alt="Recycle Bin" />
35 |   <div id="instructions">
36 |     Bring hands close and pinch to create a shape<br>
37 |      > Move hands apart to make the shape larger<br>
38 |     Hover over a shape / pinch to move it<br>
39 |     Move a shape into the recycle bin to delete it
40 |   </div>
41 |   <span id="links-para"><a href="https://x.com/measure_plan" target="_blank">Twitter</a> | <a href="https://www.instagram.com/stereo.drift/" target="_blank">Instagram</a> | <a href="https://github.com/collidingScopes/shape-creator-tutorial" target="_blank">Code</a></span>
42 |   <span id="coffee-link"><a href="https://buymeacoffee.com/stereodrift" target="_blank">Support my free tutorials & code ❤️</a></span>
43 | 
44 |   <script src="https://cdn.jsdelivr.net/npm/@mediapipe/hands/hands.js"></script>
45 |   <script src="https://cdn.jsdelivr.net/npm/@mediapipe/camera_utils/camera_utils.js"></script>
46 |   <script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r128/three.min.js"></script>
47 | 
48 | </body>
49 | 
50 | <script src="main.js"></script>
51 | 
52 | </html>


--------------------------------------------------------------------------------
/main.js:
--------------------------------------------------------------------------------
  1 | let video = document.getElementById('webcam');
  2 | let canvas = document.getElementById('canvas');
  3 | let ctx = canvas.getContext('2d');
  4 | let scene, camera, renderer;
  5 | let shapes = [];
  6 | let currentShape = null;
  7 | let isPinching = false;
  8 | let shapeScale = 1;
  9 | let originalDistance = null;
 10 | let selectedShape = null;
 11 | let shapeCreatedThisPinch = false;
 12 | let lastShapeCreationTime = 0;
 13 | const shapeCreationCooldown = 1000;
 14 | 
 15 | const initThree = () => {
 16 |   scene = new THREE.Scene();
 17 |   camera = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, 0.1, 1000);
 18 |   camera.position.z = 5;
 19 |   renderer = new THREE.WebGLRenderer({ alpha: true });
 20 |   renderer.setSize(window.innerWidth, window.innerHeight);
 21 |   document.getElementById('three-canvas').appendChild(renderer.domElement);
 22 |   const light = new THREE.AmbientLight(0xffffff, 1);
 23 |   scene.add(light);
 24 |   animate();
 25 | };
 26 | 
 27 | const animate = () => {
 28 |   requestAnimationFrame(animate);
 29 |   shapes.forEach(shape => {
 30 |     if (shape !== selectedShape) {
 31 |       shape.rotation.x += 0.01;
 32 |       shape.rotation.y += 0.01;
 33 |     }
 34 |   });
 35 |   renderer.render(scene, camera);
 36 | };
 37 | 
 38 | const neonColors = [0xFF00FF, 0x00FFFF, 0xFF3300, 0x39FF14, 0xFF0099, 0x00FF00, 0xFF6600, 0xFFFF00];
 39 | let colorIndex = 0;
 40 | 
 41 | const getNextNeonColor = () => {
 42 |     const color = neonColors[colorIndex];
 43 |     colorIndex = (colorIndex + 1) % neonColors.length;
 44 |     return color;
 45 | };
 46 | 
 47 | const createRandomShape = (position) => {
 48 |   const geometries = [
 49 |     new THREE.BoxGeometry(),
 50 |     new THREE.SphereGeometry(0.5, 32, 32),
 51 |     new THREE.ConeGeometry(0.5, 1, 32),
 52 |     new THREE.CylinderGeometry(0.5, 0.5, 1, 32)
 53 |   ];
 54 |   const geometry = geometries[Math.floor(Math.random() * geometries.length)];
 55 |   const color = getNextNeonColor();
 56 |   const group = new THREE.Group();
 57 | 
 58 |   const material = new THREE.MeshBasicMaterial({ color: color, transparent: true, opacity: 0.5 });
 59 |   const fillMesh = new THREE.Mesh(geometry, material);
 60 | 
 61 |   const wireframeMaterial = new THREE.MeshBasicMaterial({ color: 0xffffff, wireframe: true });
 62 |   const wireframeMesh = new THREE.Mesh(geometry, wireframeMaterial);
 63 | 
 64 |   group.add(fillMesh);
 65 |   group.add(wireframeMesh);
 66 |   group.position.copy(position);
 67 |   scene.add(group);
 68 | 
 69 |   shapes.push(group);
 70 |   return group;
 71 | };
 72 | 
 73 | const get3DCoords = (normX, normY) => {
 74 |   const x = (normX - 0.5) * 10;
 75 |   const y = (0.5 - normY) * 10;
 76 |   return new THREE.Vector3(x, y, 0);
 77 | };
 78 | 
 79 | const isPinch = (landmarks) => {
 80 |   const d = (a, b) => Math.hypot(a.x - b.x, a.y - b.y, a.z - b.z);
 81 |   return d(landmarks[4], landmarks[8]) < 0.06;
 82 | };
 83 | 
 84 | const areIndexFingersClose = (l, r) => {
 85 |   const d = (a, b) => Math.hypot(a.x - b.x, a.y - b.y);
 86 |   return d(l[8], r[8]) < 0.12;
 87 | };
 88 | 
 89 | const findNearestShape = (position) => {
 90 |   let minDist = Infinity;
 91 |   let closest = null;
 92 |   shapes.forEach(shape => {
 93 |     const dist = shape.position.distanceTo(position);
 94 |     if (dist < 1.5 && dist < minDist) {
 95 |       minDist = dist;
 96 |       closest = shape;
 97 |     }
 98 |   });
 99 |   return closest;
100 | };
101 | 
102 | const isInRecycleBinZone = (position) => {
103 |   const vector = position.clone().project(camera);
104 |   const screenX = ((vector.x + 1) / 2) * window.innerWidth;
105 |   const screenY = ((-vector.y + 1) / 2) * window.innerHeight;
106 | 
107 |   const binWidth = 160;
108 |   const binHeight = 160;
109 |   const binLeft = window.innerWidth - 60 - binWidth;
110 |   const binTop = window.innerHeight - 60 - binHeight;
111 |   const binRight = binLeft + binWidth;
112 |   const binBottom = binTop + binHeight;
113 | 
114 |   const adjustedX = window.innerWidth - screenX;
115 | 
116 |   return adjustedX >= binLeft && adjustedX <= binRight && screenY >= binTop && screenY <= binBottom;
117 | };
118 | 
119 | const hands = new Hands({ locateFile: file => `https://cdn.jsdelivr.net/npm/@mediapipe/hands/${file}` });
120 | hands.setOptions({ maxNumHands: 2, modelComplexity: 1, minDetectionConfidence: 0.7, minTrackingConfidence: 0.7 });
121 | 
122 | hands.onResults(results => {
123 |   ctx.clearRect(0, 0, canvas.width, canvas.height);
124 |   const recycleBin = document.getElementById('recycle-bin');
125 | 
126 |   for (const landmarks of results.multiHandLandmarks) {
127 |     const drawCircle = (landmark) => {
128 |       ctx.beginPath();
129 |       ctx.arc(landmark.x * canvas.width, landmark.y * canvas.height, 10, 0, 2 * Math.PI);
130 |       ctx.fillStyle = 'rgba(0, 255, 255, 0.7)';
131 |       ctx.fill();
132 |     };
133 |     drawCircle(landmarks[4]); // Thumb tip
134 |     drawCircle(landmarks[8]); // Index tip
135 |   }
136 | 
137 |   // Existing shape interaction and gesture logic...
138 |   if (results.multiHandLandmarks.length === 2) {
139 |     const [l, r] = results.multiHandLandmarks;
140 |     const leftPinch = isPinch(l);
141 |     const rightPinch = isPinch(r);
142 |     const indexesClose = areIndexFingersClose(l, r);
143 | 
144 |     if (leftPinch && rightPinch) {
145 |       const left = l[8];
146 |       const right = r[8];
147 |       const centerX = (left.x + right.x) / 2;
148 |       const centerY = (left.y + right.y) / 2;
149 |       const distance = Math.hypot(left.x - right.x, left.y - right.y);
150 | 
151 |       if (!isPinching) {
152 |         const now = Date.now();
153 |         if (!shapeCreatedThisPinch && indexesClose && now - lastShapeCreationTime > shapeCreationCooldown) {
154 |           currentShape = createRandomShape(get3DCoords(centerX, centerY));
155 |           lastShapeCreationTime = now;
156 |           shapeCreatedThisPinch = true;
157 |           originalDistance = distance;
158 |         }
159 |       } else if (currentShape && originalDistance) {
160 |         shapeScale = distance / originalDistance;
161 |         currentShape.scale.set(shapeScale, shapeScale, shapeScale);
162 |       }
163 |       isPinching = true;
164 |       recycleBin.classList.remove('active');
165 |       return;
166 |     }
167 |   }
168 | 
169 |   isPinching = false;
170 |   shapeCreatedThisPinch = false;
171 |   originalDistance = null;
172 |   currentShape = null;
173 | 
174 |   if (results.multiHandLandmarks.length > 0) {
175 |     for (const landmarks of results.multiHandLandmarks) {
176 |       const indexTip = landmarks[8];
177 |       const position = get3DCoords(indexTip.x, indexTip.y);
178 | 
179 |       if (isPinch(landmarks)) {
180 |         if (!selectedShape) {
181 |           selectedShape = findNearestShape(position);
182 |         }
183 |         if (selectedShape) {
184 |           selectedShape.position.copy(position);
185 | 
186 |           const inBin = isInRecycleBinZone(selectedShape.position);
187 |           selectedShape.children.forEach(child => {
188 |             if (child.material && child.material.wireframe) {
189 |               child.material.color.set(inBin ? 0xff0000 : 0xffffff);
190 |             }
191 |           });
192 |           if (inBin) {
193 |             recycleBin.classList.add('active');
194 |           } else {
195 |             recycleBin.classList.remove('active');
196 |           }
197 |         }
198 |       } else {
199 |         if (selectedShape && isInRecycleBinZone(selectedShape.position)) {
200 |           scene.remove(selectedShape);
201 |           shapes = shapes.filter(s => s !== selectedShape);
202 |         }
203 |         selectedShape = null;
204 |         recycleBin.classList.remove('active');
205 |       }
206 |     }
207 |   } else {
208 |     if (selectedShape && isInRecycleBinZone(selectedShape.position)) {
209 |       scene.remove(selectedShape);
210 |       shapes = shapes.filter(s => s !== selectedShape);
211 |     }
212 |     selectedShape = null;
213 |     recycleBin.classList.remove('active');
214 |   }
215 | });
216 | 
217 | const initCamera = async () => {
218 |   const stream = await navigator.mediaDevices.getUserMedia({ video: { width: 1280, height: 720 } });
219 |   video.srcObject = stream;
220 |   await new Promise(resolve => video.onloadedmetadata = resolve);
221 |   canvas.width = video.videoWidth;
222 |   canvas.height = video.videoHeight;
223 |   new Camera(video, {
224 |     onFrame: async () => await hands.send({ image: video }),
225 |     width: video.videoWidth,
226 |     height: video.videoHeight
227 |   }).start();
228 | };
229 | 
230 | initThree();
231 | initCamera();


--------------------------------------------------------------------------------
/recyclebin.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/collidingScopes/shape-creator-tutorial/eb3b1be41a72815a6dae5ce6b9168333f7f8d503/recyclebin.png


--------------------------------------------------------------------------------
/styles.css:
--------------------------------------------------------------------------------
 1 | body, html {
 2 |   margin: 0;
 3 |   padding: 0;
 4 |   overflow: hidden;
 5 |   background: #000;
 6 | }
 7 | #webcam, #canvas, #three-canvas {
 8 |   position: absolute;
 9 |   width: 100%;
10 |   height: 100%;
11 |   top: 0;
12 |   left: 0;
13 |   object-fit: cover;
14 |   pointer-events: none;
15 |   transform: scaleX(-1);
16 | }
17 | #recycle-bin {
18 |   position: absolute;
19 |   bottom: 60px;
20 |   right: 60px;
21 |   width: 160px;
22 |   height: 160px;
23 |   z-index: 20;
24 |   pointer-events: none;
25 | }
26 | #recycle-bin.active {
27 |   filter: drop-shadow(0 0 10px #ff0000);
28 |   transform: scale(1.1);
29 |   transition: transform 0.2s, filter 0.2s;
30 | }
31 | #instructions {
32 |   position: absolute;
33 |   top: 5px;
34 |   left: 5px;
35 |   color: white;
36 |   background: rgba(0, 0, 0, 0.5);
37 |   padding: 10px 15px;
38 |   /* border-radius: 10px; */
39 |   font-family: sans-serif;
40 |   font-size: 14px;
41 |   z-index: 30;
42 | }
43 | 
44 | #links-para{
45 |         position: absolute;
46 |         bottom: 5px;
47 |         left: 5px;
48 |         font-family: Helvetica, sans-serif;
49 |         font-size: 16px;
50 |         background-color: rgba(255, 255, 255, 0.5);
51 |         padding: 10px;
52 | }
53 | 
54 | a {
55 |     color: #0000FF !important;
56 | }
57 | 
58 | #coffee-link {
59 |   position: absolute;
60 |   top: 5px;
61 |   right: 5px;
62 |   font-family: Helvetica, sans-serif;
63 |   font-size: 16px;
64 |   background-color: rgba(255, 255, 255, 0.5);
65 |   padding: 10px;
66 | }


--------------------------------------------------------------------------------