├── example_face.jpg ├── requirements.txt ├── .gitignore ├── LICENSE ├── .github └── workflows │ └── python-check.yml ├── FaceTracker.css ├── package.json ├── examples ├── react-basic │ ├── App.jsx │ └── App.css └── README.md ├── FaceTracker.jsx ├── CONTRIBUTING.md ├── generate.sh ├── useGazeTracking.js ├── QUICKSTART.md ├── generate_missing.py ├── main.py └── README.md /example_face.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kylan02/face_looker/HEAD/example_face.jpg -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | # Face Looker Dependencies 2 | # For generating gaze-direction face images using Replicate AI 3 | 4 | # Replicate API client 5 | replicate>=0.22.0 6 | 7 | # Image processing 8 | pillow>=10.0.0 9 | 10 | # Progress bars for generation 11 | tqdm>=4.66.0 12 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Python 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | *.so 6 | .Python 7 | build/ 8 | develop-eggs/ 9 | dist/ 10 | downloads/ 11 | eggs/ 12 | .eggs/ 13 | lib/ 14 | lib64/ 15 | parts/ 16 | sdist/ 17 | var/ 18 | wheels/ 19 | *.egg-info/ 20 | .installed.cfg 21 | *.egg 22 | MANIFEST 23 | 24 | # Virtual environments 25 | .env 26 | .venv 27 | env/ 28 | venv/ 29 | ENV/ 30 | env.bak/ 31 | venv.bak/ 32 | 33 | # IDE 34 | .vscode/ 35 | .idea/ 36 | *.swp 37 | *.swo 38 | *~ 39 | 40 | # OS 41 | .DS_Store 42 | .DS_Store? 43 | ._* 44 | .Spotlight-V100 45 | .Trashes 46 | ehthumbs.db 47 | Thumbs.db 48 | 49 | # Project specific 50 | out/ 51 | *.jpg 52 | *.jpeg 53 | *.png 54 | *.webp 55 | !example_face.jpg 56 | 57 | # API keys and secrets 58 | .env.local 59 | .env.development.local 60 | .env.test.local 61 | .env.production.local 62 | 63 | # Logs 64 | *.log 65 | npm-debug.log* 66 | yarn-debug.log* 67 | yarn-error.log* 68 | 69 | # Node modules (if any) 70 | node_modules/ 71 | 72 | # Generated files 73 | index.csv -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2024 Kylan O'Connor 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | -------------------------------------------------------------------------------- /.github/workflows/python-check.yml: -------------------------------------------------------------------------------- 1 | name: Python Code Check 2 | 3 | on: 4 | push: 5 | branches: [ main ] 6 | pull_request: 7 | branches: [ main ] 8 | 9 | jobs: 10 | check: 11 | runs-on: ubuntu-latest 12 | strategy: 13 | matrix: 14 | python-version: ['3.7', '3.8', '3.9', '3.10', '3.11'] 15 | 16 | steps: 17 | - uses: actions/checkout@v3 18 | 19 | - name: Set up Python ${{ matrix.python-version }} 20 | uses: actions/setup-python@v4 21 | with: 22 | python-version: ${{ matrix.python-version }} 23 | 24 | - name: Install dependencies 25 | run: | 26 | python -m pip install --upgrade pip 27 | pip install -r requirements.txt 28 | 29 | - name: Lint with flake8 30 | run: | 31 | pip install flake8 32 | # Stop build if there are Python syntax errors or undefined names 33 | flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics 34 | # Exit-zero treats all errors as warnings 35 | flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics 36 | 37 | - name: Check imports 38 | run: | 39 | python -c "import main; print('✅ main.py imports successfully')" 40 | 41 | -------------------------------------------------------------------------------- /FaceTracker.css: -------------------------------------------------------------------------------- 1 | .face-tracker { 2 | position: relative; 3 | width: 100%; 4 | height: 100%; 5 | min-height: 300px; 6 | background: #f0f0f0; 7 | border-radius: 8px; 8 | overflow: hidden; 9 | cursor: none; /* Hide cursor for better effect */ 10 | } 11 | 12 | .face-image { 13 | width: 100%; 14 | height: 100%; 15 | object-fit: contain; 16 | transition: opacity 0.1s ease-out; 17 | } 18 | 19 | .face-loading { 20 | position: absolute; 21 | top: 50%; 22 | left: 50%; 23 | transform: translate(-50%, -50%); 24 | color: #666; 25 | font-size: 14px; 26 | } 27 | 28 | .face-tracker-error { 29 | display: flex; 30 | align-items: center; 31 | justify-content: center; 32 | height: 200px; 33 | color: #e74c3c; 34 | background: #fdf2f2; 35 | border: 1px solid #f5c6cb; 36 | border-radius: 4px; 37 | padding: 20px; 38 | } 39 | 40 | .face-debug { 41 | position: absolute; 42 | top: 10px; 43 | left: 10px; 44 | background: rgba(0, 0, 0, 0.8); 45 | color: white; 46 | padding: 8px 12px; 47 | border-radius: 4px; 48 | font-family: monospace; 49 | font-size: 12px; 50 | line-height: 1.4; 51 | } 52 | 53 | /* Responsive adjustments */ 54 | @media (max-width: 768px) { 55 | .face-tracker { 56 | min-height: 250px; 57 | } 58 | } 59 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "face-looker", 3 | "version": "1.0.0", 4 | "description": "Generate AI-powered gaze-tracking face images and use them to create interactive React components that follow the cursor", 5 | "main": "useGazeTracking.js", 6 | "scripts": { 7 | "test": "echo \"No tests yet\" && exit 0" 8 | }, 9 | "repository": { 10 | "type": "git", 11 | "url": "git+https://github.com/yourusername/face_looker.git" 12 | }, 13 | "keywords": [ 14 | "react", 15 | "face-tracking", 16 | "gaze-tracking", 17 | "interactive", 18 | "replicate", 19 | "ai", 20 | "cursor-tracking", 21 | "mouse-tracking", 22 | "animation", 23 | "portfolio" 24 | ], 25 | "author": "Kylan O'Connor", 26 | "license": "MIT", 27 | "bugs": { 28 | "url": "https://github.com/yourusername/face_looker/issues" 29 | }, 30 | "homepage": "https://github.com/yourusername/face_looker#readme", 31 | "peerDependencies": { 32 | "react": "^16.8.0 || ^17.0.0 || ^18.0.0" 33 | }, 34 | "files": [ 35 | "useGazeTracking.js", 36 | "FaceTracker.jsx", 37 | "FaceTracker.css", 38 | "main.py", 39 | "generate_missing.py", 40 | "requirements.txt", 41 | "README.md", 42 | "LICENSE", 43 | "examples/" 44 | ] 45 | } 46 | 47 | -------------------------------------------------------------------------------- /examples/react-basic/App.jsx: -------------------------------------------------------------------------------- 1 | import React from 'react'; 2 | import FaceTracker from '../../FaceTracker'; 3 | import './App.css'; 4 | 5 | /** 6 | * Example: Basic implementation of FaceTracker 7 | * 8 | * This demonstrates the simplest way to use the face tracking component 9 | */ 10 | function App() { 11 | return ( 12 |
13 |
14 |

Face Looker Demo

15 |

Move your mouse around the face below!

16 |
17 | 18 |
19 | {/* Basic usage - face in a container */} 20 |
21 | 22 |
23 | 24 |
25 |

Try These:

26 |
    27 |
  • Move your cursor around the screen
    • 28 |
  • Try on mobile with touch
    • 29 |
  • Move quickly vs slowly
    • 30 |
  • Go to the edges and corners
    • 31 |
32 |
33 | 34 | {/* Debug mode enabled */} 35 |
36 |

Debug Mode

37 | 42 |
43 |
44 |
45 | ); 46 | } 47 | 48 | export default App; 49 | 50 | -------------------------------------------------------------------------------- /FaceTracker.jsx: -------------------------------------------------------------------------------- 1 | import React, { useRef, useState } from 'react'; 2 | import useGazeTracking from './useGazeTracking'; 3 | import './FaceTracker.css'; // Optional styling 4 | 5 | /** 6 | * FaceTracker Component 7 | * Displays a face that follows mouse/touch movement 8 | */ 9 | export default function FaceTracker({ 10 | className = '', 11 | basePath = '/faces/', 12 | showDebug = false 13 | }) { 14 | const containerRef = useRef(null); 15 | const { currentImage, isLoading, error } = useGazeTracking(containerRef, basePath); 16 | const [mousePos, setMousePos] = useState({ x: 0, y: 0 }); 17 | 18 | const handleMouseMove = (e) => { 19 | if (!containerRef.current) return; 20 | 21 | const rect = containerRef.current.getBoundingClientRect(); 22 | setMousePos({ 23 | x: e.clientX - rect.left, 24 | y: e.clientY - rect.top 25 | }); 26 | }; 27 | 28 | if (error) { 29 | return ( 30 |
31 | Error loading face images: {error.message} 32 |
33 | ); 34 | } 35 | 36 | return ( 37 |
42 | {currentImage && ( 43 | Face following gaze 54 | )} 55 | 56 | {isLoading && ( 57 |
58 | Loading face... 59 |
60 | )} 61 | 62 | {showDebug && ( 63 |
64 |
Mouse: ({Math.round(mousePos.x)}, {Math.round(mousePos.y)})
65 |
Image: {currentImage?.split('/').pop()}
66 |
67 | )} 68 |
69 | ); 70 | } 71 | -------------------------------------------------------------------------------- /examples/react-basic/App.css: -------------------------------------------------------------------------------- 1 | .App { 2 | text-align: center; 3 | min-height: 100vh; 4 | background: linear-gradient(135deg, #667eea 0%, #764ba2 100%); 5 | color: white; 6 | font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', sans-serif; 7 | } 8 | 9 | .App-header { 10 | padding: 2rem; 11 | } 12 | 13 | .App-header h1 { 14 | font-size: 3rem; 15 | margin: 0; 16 | font-weight: 800; 17 | } 18 | 19 | .App-header p { 20 | font-size: 1.2rem; 21 | opacity: 0.9; 22 | } 23 | 24 | .App-main { 25 | padding: 2rem; 26 | max-width: 1200px; 27 | margin: 0 auto; 28 | } 29 | 30 | .face-container { 31 | width: 400px; 32 | height: 400px; 33 | margin: 2rem auto; 34 | border-radius: 50%; 35 | overflow: hidden; 36 | box-shadow: 0 20px 60px rgba(0, 0, 0, 0.3); 37 | background: white; 38 | } 39 | 40 | .instructions { 41 | background: rgba(255, 255, 255, 0.1); 42 | border-radius: 20px; 43 | padding: 2rem; 44 | margin: 3rem auto; 45 | max-width: 600px; 46 | backdrop-filter: blur(10px); 47 | } 48 | 49 | .instructions h2 { 50 | margin-top: 0; 51 | } 52 | 53 | .instructions ul { 54 | text-align: left; 55 | list-style: none; 56 | padding: 0; 57 | } 58 | 59 | .instructions li { 60 | padding: 0.5rem 0; 61 | padding-left: 2rem; 62 | position: relative; 63 | } 64 | 65 | .instructions li::before { 66 | content: '👉'; 67 | position: absolute; 68 | left: 0; 69 | } 70 | 71 | .debug-container { 72 | margin-top: 3rem; 73 | } 74 | 75 | .debug-face { 76 | width: 300px; 77 | height: 300px; 78 | margin: 1rem auto; 79 | border-radius: 20px; 80 | overflow: hidden; 81 | box-shadow: 0 10px 30px rgba(0, 0, 0, 0.2); 82 | } 83 | 84 | /* Mobile responsiveness */ 85 | @media (max-width: 768px) { 86 | .App-header h1 { 87 | font-size: 2rem; 88 | } 89 | 90 | .face-container { 91 | width: 300px; 92 | height: 300px; 93 | } 94 | 95 | .debug-face { 96 | width: 250px; 97 | height: 250px; 98 | } 99 | } 100 | 101 | -------------------------------------------------------------------------------- /examples/README.md: -------------------------------------------------------------------------------- 1 | # Face Looker Examples 2 | 3 | This directory contains various implementation examples for the Face Looker project. 4 | 5 | ## 📁 Examples 6 | 7 | ### 1. Basic React Implementation (`react-basic/`) 8 | 9 | The simplest way to implement face tracking in your React app. 10 | 11 | **Features:** 12 | - Basic FaceTracker component usage 13 | - Debug mode demonstration 14 | - Responsive design 15 | - Touch support 16 | 17 | **Files:** 18 | - `App.jsx` - Main application component 19 | - `App.css` - Styling 20 | 21 | ### 2. TypeScript Implementation (`typescript/`) 22 | 23 | TypeScript version with full type safety. 24 | 25 | **Features:** 26 | - Full TypeScript types 27 | - Type-safe props 28 | - Better IDE autocomplete 29 | - Production-ready code 30 | 31 | ### 3. Next.js Implementation (`nextjs/`) 32 | 33 | Optimized for Next.js with server-side rendering considerations. 34 | 35 | **Features:** 36 | - Next.js Image optimization (optional) 37 | - SSR-safe implementation 38 | - App Router compatible 39 | - Dynamic imports 40 | 41 | ### 4. Multiple Faces (`multiple-faces/`) 42 | 43 | Example showing multiple face trackers on one page. 44 | 45 | **Features:** 46 | - Multiple independent face trackers 47 | - Different configurations 48 | - Performance optimization 49 | 50 | ### 5. Custom Styling (`custom-styling/`) 51 | 52 | Advanced styling examples. 53 | 54 | **Features:** 55 | - Circular masks 56 | - Border effects 57 | - Animations 58 | - Hover effects 59 | 60 | ## 🚀 Quick Start 61 | 62 | 1. **Generate your faces first:** 63 | ```bash 64 | cd ../ 65 | python main.py --image ./my_face.jpg --out ./out 66 | ``` 67 | 68 | 2. **Copy faces to your React project:** 69 | ```bash 70 | cp -r ./out/faces /path/to/your-react-app/public/ 71 | ``` 72 | 73 | 3. **Copy the example you want:** 74 | ```bash 75 | # For basic React 76 | cp examples/react-basic/* /path/to/your-react-app/src/ 77 | 78 | # For TypeScript 79 | cp examples/typescript/* /path/to/your-react-app/src/ 80 | ``` 81 | 82 | 4. **Install dependencies** (if needed): 83 | ```bash 84 | npm install 85 | # or 86 | yarn install 87 | ``` 88 | 89 | ## 💡 Tips 90 | 91 | - Start with `react-basic` if you're new to the project 92 | - Use `typescript` for production applications 93 | - Check `custom-styling` for inspiration on making it unique 94 | - `multiple-faces` shows how to handle multiple instances 95 | 96 | ## 🔧 Configuration 97 | 98 | All examples assume: 99 | - Face images in `/public/faces/` 100 | - Default grid parameters (`-15` to `15`, step `3`) 101 | - 256×256 image size 102 | 103 | If you changed generation parameters, update the constants in `useGazeTracking.js`. 104 | 105 | ## 📚 Learn More 106 | 107 | See the main [README.md](../README.md) for full documentation. 108 | 109 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing to Face Looker 2 | 3 | Thanks for your interest in contributing! 🎉 4 | 5 | ## 🐛 Bug Reports 6 | 7 | If you find a bug, please open an issue with: 8 | - Clear description of the problem 9 | - Steps to reproduce 10 | - Expected vs actual behavior 11 | - Python version and OS 12 | - Error messages (if any) 13 | 14 | ## 💡 Feature Requests 15 | 16 | We welcome feature ideas! Please open an issue describing: 17 | - The problem you're trying to solve 18 | - Your proposed solution 19 | - Any alternative solutions you've considered 20 | 21 | ## 🔧 Pull Requests 22 | 23 | ### Setup for Development 24 | 25 | 1. Fork the repository 26 | 2. Clone your fork: 27 | ```bash 28 | git clone https://github.com/YOUR_USERNAME/face_looker.git 29 | cd face_looker 30 | ``` 31 | 32 | 3. Create a virtual environment: 33 | ```bash 34 | python -m venv .venv 35 | source .venv/bin/activate 36 | ``` 37 | 38 | 4. Install dependencies: 39 | ```bash 40 | pip install -r requirements.txt 41 | ``` 42 | 43 | 5. Create a feature branch: 44 | ```bash 45 | git checkout -b feature/your-feature-name 46 | ``` 47 | 48 | ### Code Style 49 | 50 | - Follow PEP 8 for Python code 51 | - Use meaningful variable names 52 | - Add docstrings to functions 53 | - Keep functions focused and small 54 | 55 | ### Testing 56 | 57 | Before submitting a PR: 58 | 1. Test your changes with different parameters 59 | 2. Verify React implementation still works 60 | 3. Check that existing functionality isn't broken 61 | 4. Test on both Python 3.7+ and latest version 62 | 63 | ### Documentation 64 | 65 | - Update README.md if you add features 66 | - Add examples for new functionality 67 | - Include code comments for complex logic 68 | 69 | ### Commit Messages 70 | 71 | Use clear, descriptive commit messages: 72 | ``` 73 | Add support for custom image formats 74 | 75 | - Added PNG and JPEG output options 76 | - Updated save_resized_webp to handle multiple formats 77 | - Added --format argument to CLI 78 | ``` 79 | 80 | ### Submitting 81 | 82 | 1. Push to your fork: 83 | ```bash 84 | git push origin feature/your-feature-name 85 | ``` 86 | 87 | 2. Open a Pull Request with: 88 | - Clear description of changes 89 | - Why the change is needed 90 | - Any breaking changes 91 | - Screenshots (if UI changes) 92 | 93 | ## 📝 Example Contributions 94 | 95 | Good first contributions: 96 | - Add more examples 97 | - Improve documentation 98 | - Fix typos 99 | - Add tests 100 | - Performance improvements 101 | 102 | ## 🤝 Code of Conduct 103 | 104 | - Be respectful and inclusive 105 | - Provide constructive feedback 106 | - Help newcomers 107 | - Focus on the best solution, not winning arguments 108 | 109 | ## ❓ Questions? 110 | 111 | Open an issue with the "question" label or reach out to the maintainers. 112 | 113 | Thank you for contributing! 🙏 114 | 115 | -------------------------------------------------------------------------------- /generate.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # Quick start script for generating face images 3 | 4 | set -e 5 | 6 | echo "🎭 Face Looker - Quick Start Generator" 7 | echo "======================================" 8 | echo "" 9 | 10 | # Check if REPLICATE_API_TOKEN is set 11 | if [ -z "$REPLICATE_API_TOKEN" ]; then 12 | echo "❌ Error: REPLICATE_API_TOKEN not set!" 13 | echo "" 14 | echo "Please set your Replicate API token:" 15 | echo " export REPLICATE_API_TOKEN=your_token_here" 16 | echo "" 17 | echo "Get your token at: https://replicate.com/account/api-tokens" 18 | exit 1 19 | fi 20 | 21 | # Check if virtual environment exists 22 | if [ ! -d ".venv" ]; then 23 | echo "📦 Creating virtual environment..." 24 | python3 -m venv .venv 25 | fi 26 | 27 | # Activate virtual environment 28 | echo "🔌 Activating virtual environment..." 29 | source .venv/bin/activate 30 | 31 | # Install dependencies 32 | echo "📥 Installing dependencies..." 33 | pip install -q -r requirements.txt 34 | 35 | # Check if image is provided 36 | if [ -z "$1" ]; then 37 | echo "❌ Error: No image provided!" 38 | echo "" 39 | echo "Usage: ./generate.sh [output_dir] [step]" 40 | echo "" 41 | echo "Examples:" 42 | echo " ./generate.sh my_face.jpg" 43 | echo " ./generate.sh my_face.jpg ./faces" 44 | echo " ./generate.sh my_face.jpg ./faces 2.5" 45 | exit 1 46 | fi 47 | 48 | IMAGE=$1 49 | OUTPUT=${2:-./out} 50 | STEP=${3:-3} 51 | 52 | # Check if image exists 53 | if [ ! -f "$IMAGE" ]; then 54 | echo "❌ Error: Image not found: $IMAGE" 55 | exit 1 56 | fi 57 | 58 | echo "" 59 | echo "⚙️ Configuration:" 60 | echo " Image: $IMAGE" 61 | echo " Output: $OUTPUT" 62 | echo " Step: $STEP" 63 | echo "" 64 | 65 | # Calculate number of images 66 | IMAGES=$(echo "scale=0; ((15 - (-15)) / $STEP + 1) ^ 2" | bc) 67 | echo "📊 Will generate approximately $IMAGES images" 68 | echo "" 69 | 70 | # Confirm 71 | read -p "Continue? (y/n) " -n 1 -r 72 | echo 73 | if [[ ! $REPLY =~ ^[Yy]$ ]]; then 74 | echo "Cancelled." 75 | exit 0 76 | fi 77 | 78 | # Run generation 79 | echo "" 80 | echo "🚀 Starting generation..." 81 | echo "" 82 | 83 | python main.py \ 84 | --image "$IMAGE" \ 85 | --out "$OUTPUT" \ 86 | --step "$STEP" \ 87 | --skip-existing 88 | 89 | echo "" 90 | echo "✅ Done! Images saved to: $OUTPUT" 91 | echo "" 92 | echo "📋 Next steps:" 93 | echo " 1. Copy faces to your React project:" 94 | echo " cp -r $OUTPUT /path/to/your-react-app/public/faces" 95 | echo "" 96 | echo " 2. Copy React components:" 97 | echo " cp useGazeTracking.js /path/to/your-react-app/src/hooks/" 98 | echo " cp FaceTracker.jsx /path/to/your-react-app/src/components/" 99 | echo "" 100 | echo " 3. Use in your app:" 101 | echo " import FaceTracker from './components/FaceTracker'" 102 | echo "" 103 | 104 | -------------------------------------------------------------------------------- /useGazeTracking.js: -------------------------------------------------------------------------------- 1 | import { useState, useEffect, useCallback } from 'react'; 2 | 3 | // Grid configuration (must match your generation parameters) 4 | const P_MIN = -15; 5 | const P_MAX = 15; 6 | const STEP = 3; 7 | const SIZE = 256; 8 | 9 | /** 10 | * Converts normalized coordinates [-1, 1] to grid coordinates 11 | */ 12 | function quantizeToGrid(val) { 13 | const raw = P_MIN + (val + 1) * (P_MAX - P_MIN) / 2; // [-1,1] -> [-15,15] 14 | const snapped = Math.round(raw / STEP) * STEP; 15 | return Math.max(P_MIN, Math.min(P_MAX, snapped)); 16 | } 17 | 18 | /** 19 | * Converts grid coordinates to filename format 20 | */ 21 | function gridToFilename(px, py) { 22 | const sanitize = (val) => val.toString().replace('-', 'm').replace('.', 'p'); 23 | return `gaze_px${sanitize(px)}_py${sanitize(py)}_${SIZE}.webp`; 24 | } 25 | 26 | /** 27 | * Custom hook for gaze tracking 28 | * @param {React.RefObject} containerRef - Reference to the container element 29 | * @param {string} basePath - Base path to face images (default: '/faces/') 30 | * @returns {Object} { currentImage, isLoading, error } 31 | */ 32 | export function useGazeTracking(containerRef, basePath = '/faces/') { 33 | const [currentImage, setCurrentImage] = useState(null); 34 | const [isLoading, setIsLoading] = useState(false); 35 | const [error, setError] = useState(null); 36 | 37 | const updateGaze = useCallback((clientX, clientY) => { 38 | if (!containerRef.current) return; 39 | 40 | const rect = containerRef.current.getBoundingClientRect(); 41 | const centerX = rect.left + rect.width / 2; 42 | const centerY = rect.top + rect.height / 2; 43 | 44 | // Convert to normalized coordinates [-1, 1] 45 | const nx = (clientX - centerX) / (rect.width / 2); 46 | const ny = (clientY - centerY) / (rect.height / 2); 47 | 48 | // Clamp to [-1, 1] range 49 | const clampedX = Math.max(-1, Math.min(1, nx)); 50 | const clampedY = Math.max(-1, Math.min(1, ny)); 51 | 52 | // Convert to grid coordinates 53 | const px = quantizeToGrid(clampedX); 54 | const py = quantizeToGrid(clampedY); 55 | 56 | // Generate filename 57 | const filename = gridToFilename(px, py); 58 | const imagePath = `${basePath}${filename}`; 59 | 60 | setCurrentImage(imagePath); 61 | }, [basePath]); 62 | 63 | const handleMouseMove = useCallback((e) => { 64 | updateGaze(e.clientX, e.clientY); 65 | }, [updateGaze]); 66 | 67 | const handleTouchMove = useCallback((e) => { 68 | if (e.touches.length > 0) { 69 | const touch = e.touches[0]; 70 | updateGaze(touch.clientX, touch.clientY); 71 | } 72 | }, [updateGaze]); 73 | 74 | useEffect(() => { 75 | const container = containerRef.current; 76 | if (!container) return; 77 | 78 | // Add event listeners 79 | container.addEventListener('mousemove', handleMouseMove); 80 | container.addEventListener('touchmove', handleTouchMove, { passive: true }); 81 | 82 | // Set initial center gaze 83 | const rect = container.getBoundingClientRect(); 84 | const centerX = rect.left + rect.width / 2; 85 | const centerY = rect.top + rect.height / 2; 86 | updateGaze(centerX, centerY); 87 | 88 | return () => { 89 | container.removeEventListener('mousemove', handleMouseMove); 90 | container.removeEventListener('touchmove', handleTouchMove); 91 | }; 92 | }, [handleMouseMove, handleTouchMove, updateGaze]); 93 | 94 | return { currentImage, isLoading, error }; 95 | } 96 | 97 | export default useGazeTracking; 98 | 99 | -------------------------------------------------------------------------------- /QUICKSTART.md: -------------------------------------------------------------------------------- 1 | # ⚡ Quick Start Guide 2 | 3 | Get up and running with Face Looker in 5 minutes! 4 | 5 | ## 🎯 Goal 6 | 7 | Create an interactive face that follows your cursor in a React app. 8 | 9 | ## 📝 Prerequisites 10 | 11 | - [ ] Python 3.7+ installed 12 | - [ ] Node.js installed (for React) 13 | - [ ] Replicate account (free tier works!) 14 | - [ ] A 512×512 photo of a face 15 | 16 | ## 🚀 Steps 17 | 18 | ### 1. Get Your Replicate API Token (2 minutes) 19 | 20 | 1. Go to https://replicate.com/ 21 | 2. Sign up (free) 22 | 3. Go to https://replicate.com/account/api-tokens 23 | 4. Copy your API token 24 | 5. Save it: 25 | 26 | ```bash 27 | export REPLICATE_API_TOKEN=your_token_here 28 | ``` 29 | 30 | ### 2. Clone and Setup (1 minute) 31 | 32 | ```bash 33 | # Clone the repo 34 | git clone https://github.com/yourusername/face_looker.git 35 | cd face_looker 36 | 37 | # Quick setup (auto-installs dependencies) 38 | ./generate.sh my_face.jpg 39 | ``` 40 | 41 | Or manual setup: 42 | 43 | ```bash 44 | python -m venv .venv 45 | source .venv/bin/activate 46 | pip install -r requirements.txt 47 | ``` 48 | 49 | ### 3. Generate Face Images (2-5 minutes) 50 | 51 | **Automatic (recommended):** 52 | ```bash 53 | ./generate.sh my_face.jpg 54 | ``` 55 | 56 | **Manual:** 57 | ```bash 58 | python main.py --image my_face.jpg --out ./out 59 | ``` 60 | 61 | ⏰ This takes 2-5 minutes depending on your settings. Grab a coffee! 62 | 63 | ### 4. Use in React (30 seconds) 64 | 65 | ```bash 66 | # Copy faces to your React project 67 | cp -r ./out /path/to/your-react-app/public/faces 68 | 69 | # Copy React files 70 | cp useGazeTracking.js /path/to/your-react-app/src/hooks/ 71 | cp FaceTracker.jsx /path/to/your-react-app/src/components/ 72 | cp FaceTracker.css /path/to/your-react-app/src/components/ 73 | ``` 74 | 75 | ### 5. Add to Your App (30 seconds) 76 | 77 | ```jsx 78 | import FaceTracker from './components/FaceTracker'; 79 | 80 | function App() { 81 | return ( 82 |
83 | 84 |
85 | ); 86 | } 87 | ``` 88 | 89 | ## ✅ Done! 90 | 91 | Your face should now follow the cursor! 🎉 92 | 93 | ## 🎛️ Customization 94 | 95 | ### Smoother transitions (more images): 96 | ```bash 97 | python main.py --image my_face.jpg --out ./out --step 2.5 98 | ``` 99 | 100 | ### Faster generation (fewer images): 101 | ```bash 102 | python main.py --image my_face.jpg --out ./out --step 5 103 | ``` 104 | 105 | ### Circular face: 106 | ```css 107 | .face-tracker { 108 | border-radius: 50%; 109 | overflow: hidden; 110 | } 111 | ``` 112 | 113 | ## 🐛 Troubleshooting 114 | 115 | ### "REPLICATE_API_TOKEN not set" 116 | ```bash 117 | export REPLICATE_API_TOKEN=your_actual_token 118 | ``` 119 | 120 | ### Images not loading in React 121 | - Check that faces are in `public/faces/` 122 | - Verify `basePath="/faces/"` matches your structure 123 | - Check browser console for 404 errors 124 | 125 | ### Face not following cursor 126 | - Click on the face area first (might need focus) 127 | - Check that configuration matches in `useGazeTracking.js` 128 | - Verify all images generated successfully 129 | 130 | ## 📚 Next Steps 131 | 132 | - Read the full [README.md](./README.md) for advanced usage 133 | - Check [examples/](./examples/) for more implementations 134 | - Customize styling in `FaceTracker.css` 135 | - Try different grid densities 136 | 137 | ## 💰 Cost 138 | 139 | - Default settings: ~$0.01 (121 images) 140 | - Smoother (step 2.5): ~$0.02 (169 images) 141 | - Free tier usually covers initial testing! 142 | 143 | ## 🆘 Need Help? 144 | 145 | - Check the [README.md](./README.md) 146 | - Open an issue on GitHub 147 | - See [CONTRIBUTING.md](./CONTRIBUTING.md) 148 | 149 | --- 150 | 151 | **Estimated total time:** 5-10 minutes 152 | **Cost:** ~$0.01-0.02 153 | **Difficulty:** Easy ⭐ 154 | 155 | Ready? Let's go! 🚀 156 | 157 | -------------------------------------------------------------------------------- /generate_missing.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | """ 3 | Generate only the missing face images for the portfolio. 4 | This script identifies which images are missing and generates only those. 5 | """ 6 | 7 | import os 8 | import sys 9 | from pathlib import Path 10 | from main import build_gaze_grid, filename_for, run_expression_editor, save_resized_webp, is_url 11 | 12 | def find_missing_images(portfolio_dir, vmin=-15, vmax=15, step=2.5, size=256): 13 | """Find which images are missing from the portfolio directory.""" 14 | portfolio_path = Path(portfolio_dir) 15 | if not portfolio_path.exists(): 16 | print(f"ERROR: Portfolio directory not found: {portfolio_path}") 17 | return [] 18 | 19 | # Generate expected filenames for step=2.5 20 | grid = build_gaze_grid(vmin, vmax, step) 21 | missing_files = [] 22 | 23 | print(f"Checking {len(grid)} expected files in {portfolio_path}") 24 | 25 | for gp in grid: 26 | fname = filename_for(gp.px, gp.py, size) 27 | file_path = portfolio_path / fname 28 | if not file_path.exists(): 29 | missing_files.append((gp.px, gp.py, fname)) 30 | print(f" MISSING: {fname}") 31 | else: 32 | print(f" EXISTS: {fname}") 33 | 34 | return missing_files 35 | 36 | def generate_missing_images(image_path, portfolio_dir, vmin=-15, vmax=15, step=2.5, size=256): 37 | """Generate only the missing images.""" 38 | 39 | # Check for API token 40 | if not os.getenv("REPLICATE_API_TOKEN"): 41 | print("ERROR: REPLICATE_API_TOKEN not set in environment.") 42 | print("Please set your Replicate API token:") 43 | print("export REPLICATE_API_TOKEN=your_token_here") 44 | return False 45 | 46 | # Find missing images 47 | print("🔍 Scanning for missing images...") 48 | missing = find_missing_images(portfolio_dir, vmin, vmax, step, size) 49 | 50 | if not missing: 51 | print("✅ All images are present! No missing files found.") 52 | return True 53 | 54 | print(f"📊 Found {len(missing)} missing images out of {13*13} total expected.") 55 | print("Missing images:") 56 | for _, _, fname in missing: 57 | print(f" - {fname}") 58 | 59 | # Prepare image input for Replicate 60 | if is_url(image_path): 61 | image_input = image_path 62 | else: 63 | p = Path(image_path) 64 | if not p.exists(): 65 | print(f"ERROR: Image path not found: {p}") 66 | return False 67 | image_input = open(p, "rb") 68 | 69 | # Generate missing images 70 | print(f"\n🎨 Generating {len(missing)} missing images...") 71 | success_count = 0 72 | 73 | for i, (px, py, fname) in enumerate(missing, 1): 74 | print(f"[{i}/{len(missing)}] Generating {fname} (px={px}, py={py})...") 75 | 76 | try: 77 | output_files = run_expression_editor(image_input, px, py) 78 | if not output_files: 79 | print(f" ⚠️ WARNING: No output for ({px}, {py}). Skipping.") 80 | continue 81 | 82 | # Save to portfolio directory 83 | target_path = Path(portfolio_dir) / fname 84 | save_resized_webp(output_files[0], target_path, size=size, quality=95) 85 | success_count += 1 86 | print(f" ✅ Saved: {fname}") 87 | 88 | except Exception as e: 89 | print(f" ❌ ERROR generating {fname}: {e}") 90 | continue 91 | 92 | print(f"\n🎉 Generation complete!") 93 | print(f"✅ Successfully generated: {success_count}/{len(missing)} images") 94 | print(f"📁 Images saved to: {portfolio_dir}") 95 | 96 | return success_count == len(missing) 97 | 98 | def main(): 99 | """Main function to run the missing image generation.""" 100 | import argparse 101 | 102 | parser = argparse.ArgumentParser(description="Generate missing face images for portfolio") 103 | parser.add_argument("--image", default="./my_face.jpg", help="Path to source image") 104 | parser.add_argument("--portfolio", default="/Users/kylanoconnor/Desktop/projects/kylan_portfolio_site/public/faces", help="Portfolio faces directory") 105 | parser.add_argument("--min", dest="vmin", type=float, default=-15.0, help="Minimum value for pupil_x/y") 106 | parser.add_argument("--max", dest="vmax", type=float, default=15.0, help="Maximum value for pupil_x/y") 107 | parser.add_argument("--step", type=float, default=2.5, help="Step size for grid sampling") 108 | parser.add_argument("--size", type=int, default=256, help="Resize dimension for outputs") 109 | 110 | args = parser.parse_args() 111 | 112 | print("🚀 Missing Face Image Generator") 113 | print("=" * 50) 114 | print(f"📸 Source image: {args.image}") 115 | print(f"📁 Portfolio dir: {args.portfolio}") 116 | print(f"📐 Grid: {args.vmin} to {args.vmax}, step {args.step}") 117 | print(f"🖼️ Size: {args.size}x{args.size}") 118 | print() 119 | 120 | success = generate_missing_images( 121 | args.image, 122 | args.portfolio, 123 | args.vmin, 124 | args.vmax, 125 | args.step, 126 | args.size 127 | ) 128 | 129 | if success: 130 | print("\n🎯 All missing images generated successfully!") 131 | sys.exit(0) 132 | else: 133 | print("\n⚠️ Some images failed to generate. Check the errors above.") 134 | sys.exit(1) 135 | 136 | if __name__ == "__main__": 137 | main() 138 | -------------------------------------------------------------------------------- /main.py: -------------------------------------------------------------------------------- 1 | 2 | #!/usr/bin/env python3 3 | """ 4 | Generate a grid of gaze images using Replicate's fofr/expression-editor model, 5 | varying only pupil_x and pupil_y (range: -15..15), then resize outputs to 256px. 6 | Outputs are saved as WebP files with easy-to-parse filenames and a CSV index. 7 | 8 | Usage examples: 9 | export REPLICATE_API_TOKEN=your_token_here 10 | pip install replicate pillow tqdm 11 | python main.py --image ./me_512.jpg --out ./out --min -15 --max 15 --step 3 12 | 13 | Notes: 14 | - Only pupil_x and pupil_y are changed. All other model inputs use defaults. 15 | - Input image can be a local file path or a URL. 16 | - Each output is resized to 256x256 using Lanczos filter. 17 | - A CSV index (index.csv) mapping filenames to (pupil_x,pupil_y) is created. 18 | """ 19 | 20 | import argparse 21 | import io 22 | import math 23 | import os 24 | import sys 25 | from dataclasses import dataclass 26 | from pathlib import Path 27 | from typing import Iterable, Tuple, List 28 | 29 | import replicate 30 | from PIL import Image 31 | from tqdm import tqdm 32 | 33 | MODEL_VERSION = "bf913bc90e1c44ba288ba3942a538693b72e8cc7df576f3beebe56adc0a92b86" 34 | 35 | @dataclass(frozen=True) 36 | class GazePoint: 37 | px: float 38 | py: float 39 | 40 | def frange(start: float, stop: float, step: float) -> Iterable[float]: 41 | """Like range() but for floats, inclusive of stop when it lands exactly.""" 42 | x = start 43 | # Protect against infinite loops from improper step sign 44 | if step == 0: 45 | raise ValueError("step must be non-zero") 46 | cmp = (lambda a, b: a <= b) if step > 0 else (lambda a, b: a >= b) 47 | while cmp(x, stop + (1e-9 if step > 0 else -1e-9)): 48 | yield round(x, 6) # avoid ugly float strings 49 | x += step 50 | 51 | def clamp(v: float, lo: float, hi: float) -> float: 52 | return max(lo, min(hi, v)) 53 | 54 | def build_gaze_grid(xmin: float, xmax: float, ymin: float, ymax: float, step: float) -> List[GazePoint]: 55 | """Build a grid of (pupil_x, pupil_y) with separate ranges for X and Y.""" 56 | xs = list(frange(xmin, xmax, step)) 57 | ys = list(frange(ymin, ymax, step)) 58 | grid = [GazePoint(px=x, py=y) for y in ys for x in xs] 59 | return grid 60 | 61 | def is_url(s: str) -> bool: 62 | return s.startswith("http://") or s.startswith("https://") or s.startswith("data:") 63 | 64 | def run_expression_editor(image_input, pupil_x: float, pupil_y: float): 65 | """ 66 | Calls Replicate model with pupil_x, pupil_y, rotate_pitch, and rotate_yaw adjusted. 67 | Returns a list of file-like objects per Replicate SDK (we expect 1). 68 | """ 69 | # Clamp pupil values 70 | clamped_px = float(clamp(pupil_x, -15, 15)) 71 | clamped_py = float(clamp(pupil_y, -15, 15)) 72 | 73 | # Add pitch and yaw based on pupil position for more exaggerated look 74 | # Scale factor: convert pupil range (-15 to 15) to rotation range 75 | # Yaw follows horizontal pupil movement (left/right) 76 | rotate_yaw = (clamped_px / 15.0) * 10.0 # Max ±10 degrees 77 | # Pitch follows vertical pupil movement (up/down) - INVERTED 78 | rotate_pitch = -(clamped_py / 15.0) * 10.0 # Max ±10 degrees, inverted 79 | 80 | input_payload = { 81 | "image": image_input, 82 | "pupil_x": clamped_px, 83 | "pupil_y": clamped_py, 84 | "rotate_yaw": float(clamp(rotate_yaw, -20, 20)), 85 | "rotate_pitch": float(clamp(rotate_pitch, -20, 20)), 86 | } 87 | return replicate.run( 88 | f"fofr/expression-editor:{MODEL_VERSION}", 89 | input=input_payload 90 | ) 91 | 92 | def save_resized_webp(file_like, out_path: Path, size: int = 256, quality: int = 95): 93 | """ 94 | Read the webp bytes from Replicate, resize to square `size`, and save as webp. 95 | """ 96 | raw = file_like.read() 97 | img = Image.open(io.BytesIO(raw)).convert("RGBA") 98 | img = img.resize((size, size), Image.Resampling.LANCZOS) 99 | out_path.parent.mkdir(parents=True, exist_ok=True) 100 | img.save(out_path, format="WEBP", quality=quality, method=6) 101 | 102 | def sanitize_val(v: float) -> str: 103 | """ 104 | Make a safe string piece for filename from a float, preserving sign and major decimals. 105 | E.g., -12.5 -> "m12p5"; 3 -> "3"; 0.0 -> "0" 106 | """ 107 | s = f"{v}".replace("-", "m").replace(".", "p") 108 | return s 109 | 110 | def filename_for(px: float, py: float, size: int = 256) -> str: 111 | """Create consistent filename embedding gaze values and size.""" 112 | return f"gaze_px{sanitize_val(px)}_py{sanitize_val(py)}_{size}.webp" 113 | 114 | def write_index(csv_path: Path, rows: List[Tuple[str, float, float]]): 115 | csv_path.parent.mkdir(parents=True, exist_ok=True) 116 | with csv_path.open("w", encoding="utf-8") as f: 117 | f.write("filename,pupil_x,pupil_y\n") 118 | for name, px, py in rows: 119 | f.write(f"{name},{px},{py}\n") 120 | 121 | def main(): 122 | parser = argparse.ArgumentParser(description="Generate gaze images with varying pupil_x/y using Replicate.") 123 | parser.add_argument("--image", required=True, help="Path to local image (512x512 recommended) or URL") 124 | parser.add_argument("--out", default="./out", help="Output directory") 125 | parser.add_argument("--xmin", type=float, default=-15.0, help="Minimum value for pupil_x (left)") 126 | parser.add_argument("--xmax", type=float, default=15.0, help="Maximum value for pupil_x (right)") 127 | parser.add_argument("--ymin", type=float, default=-15.0, help="Minimum value for pupil_y (down)") 128 | parser.add_argument("--ymax", type=float, default=15.0, help="Maximum value for pupil_y (up)") 129 | parser.add_argument("--step", type=float, default=3.0, help="Step size for grid sampling") 130 | parser.add_argument("--size", type=int, default=256, help="Resize dimension (square) for outputs") 131 | parser.add_argument("--skip-existing", action="store_true", help="Skip generation if target file already exists") 132 | args = parser.parse_args() 133 | 134 | # Validate token presence early 135 | if not os.getenv("REPLICATE_API_TOKEN"): 136 | print("ERROR: REPLICATE_API_TOKEN not set in environment.", file=sys.stderr) 137 | sys.exit(1) 138 | 139 | out_dir = Path(args.out) 140 | out_dir.mkdir(parents=True, exist_ok=True) 141 | 142 | # Prepare image input for Replicate 143 | if is_url(args.image): 144 | image_input = args.image 145 | else: 146 | p = Path(args.image) 147 | if not p.exists(): 148 | print(f"ERROR: image path not found: {p}", file=sys.stderr) 149 | sys.exit(1) 150 | image_input = open(p, "rb") 151 | 152 | # Build grid and iterate 153 | grid = build_gaze_grid(args.xmin, args.xmax, args.ymin, args.ymax, args.step) 154 | index_rows: List[Tuple[str, float, float]] = [] 155 | 156 | for gp in tqdm(grid, desc="Generating", unit="img"): 157 | fname = filename_for(gp.px, gp.py, args.size) 158 | target = out_dir / fname 159 | if args.skip_existing and target.exists(): 160 | index_rows.append((fname, gp.px, gp.py)) 161 | continue 162 | 163 | try: 164 | output_files = run_expression_editor(image_input, gp.px, gp.py) 165 | if not output_files: 166 | print(f"WARNING: No output for ({gp.px}, {gp.py}). Skipping.", file=sys.stderr) 167 | continue 168 | 169 | # Replicate returns a list; we take first 170 | save_resized_webp(output_files[0], target, size=args.size, quality=95) 171 | index_rows.append((fname, gp.px, gp.py)) 172 | except Exception as e: 173 | print(f"ERROR generating ({gp.px}, {gp.py}): {e}", file=sys.stderr) 174 | 175 | # Write CSV index 176 | write_index(out_dir / "index.csv", index_rows) 177 | 178 | # Helpful note for selecting nearest gaze later 179 | print(f"Done. Wrote {len(index_rows)} images to {out_dir.resolve()} and index.csv") 180 | 181 | if __name__ == "__main__": 182 | main() 183 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Face Looker 👁️ 2 | 3 | 4 | 5 | https://github.com/user-attachments/assets/c8684ae4-3009-47cb-8ad4-0e7af65a608f 6 | 7 | 8 | 9 | Generate a grid of face images with different gaze directions using AI, then use them to create an interactive face that follows the user's cursor in real-time. 10 | 11 | ![Demo](demo.gif) 12 | 13 | ## 🎯 Overview 14 | 15 | This project has two parts: 16 | 1. **Generate Face Images** - Create hundreds of face images looking in different directions using either: 17 | - **Replicate Model** (easiest) - Upload your image and get everything you need in one click 18 | - **Python Script** (advanced) - More control over generation parameters 19 | 2. **React Hook** - Makes those images interactive by displaying the right image based on cursor position 20 | 21 | Perfect for creating engaging portfolio headers, interactive avatars, or fun UI elements! 22 | 23 | ## 📋 Prerequisites 24 | 25 | - A 512×512 photo of a face (your photo or any portrait) 26 | - Node.js (for React implementation) 27 | - [Replicate API account](https://replicate.com/) (free tier available) 28 | 29 | ## 🚀 Part 1: Generate Face Images 30 | 31 | You have two options to generate the face images: 32 | 33 | ### Option 1: Use the Replicate Model (Easiest) ✨ 34 | 35 | The easiest way to get started is using the dedicated [Replicate model](https://replicate.com/kylan02/face-looker) created in collaboration with [fofr](https://x.com/fofrAI). Simply upload your face image and it will automatically generate: 36 | 37 | - ✅ All the face images looking in different directions 38 | - ✅ A sprite/grid sheet that combines all faces into a single image 39 | - ✅ A ZIP file containing vanilla HTML, JS and CSS to render the effect 40 | - ✅ A preview video showing how the animation will look 41 | 42 | **To use it:** 43 | 1. Visit [https://replicate.com/kylan02/face-looker](https://replicate.com/kylan02/face-looker) 44 | 2. Upload your 512×512 face image 45 | 3. Download the generated ZIP file 46 | 4. Extract and use the images/code in your project 47 | 48 | The model automatically uses the Expression Editor model to generate all the images you need. See the [model page](https://replicate.com/kylan02/face-looker) for more details. 49 | 50 | ### Option 2: Use the Python Script (Advanced) 🔧 51 | 52 | If you prefer more control over the generation process or want to customize the parameters, you can use the Python script described below. 53 | 54 | --- 55 | 56 | ### Step 1: Setup 57 | 58 | ```bash 59 | # Clone or download this repository 60 | cd face_looker 61 | 62 | # Create a virtual environment (recommended) 63 | python -m venv .venv 64 | source .venv/bin/activate # On Windows: .venv\Scripts\activate 65 | 66 | # Install dependencies 67 | pip install -r requirements.txt 68 | ``` 69 | 70 | ### Step 2: Get Your Replicate API Token 71 | 72 | 1. Sign up at [replicate.com](https://replicate.com/) 73 | 2. Go to your [account settings](https://replicate.com/account/api-tokens) 74 | 3. Copy your API token 75 | 4. Set it as an environment variable: 76 | 77 | ```bash 78 | export REPLICATE_API_TOKEN=your_token_here 79 | ``` 80 | 81 | Or add it to your `.bashrc`/`.zshrc` for permanent use: 82 | 83 | ```bash 84 | echo 'export REPLICATE_API_TOKEN=your_token_here' >> ~/.zshrc 85 | source ~/.zshrc 86 | ``` 87 | 88 | ### Step 3: Prepare Your Face Image 89 | 90 | **Image Requirements:** 91 | - Use a **512×512 pixel** image for best results 92 | - Face should be centered and looking straight ahead 93 | - Good lighting, clear features 94 | - Neutral expression works best 95 | 96 | **Where to put your image:** 97 | 1. **Any filename works!** You can use any name like `me.jpg`, `portrait.png`, `selfie.jpeg`, etc. 98 | 2. **Any location works!** You can put it in the project folder or anywhere on your computer 99 | 3. **Example file structure:** 100 | ``` 101 | face_looker/ 102 | ├── main.py 103 | ├── my_face.jpg ← Your image can go here 104 | ├── requirements.txt 105 | └── README.md 106 | ``` 107 | 108 | **Or anywhere else:** 109 | ``` 110 | /Users/you/Pictures/portrait.png 111 | /path/to/any/folder/selfie.jpg 112 | ``` 113 | 114 | **Before running the script, make sure you have:** 115 | - ✅ Set your `REPLICATE_API_TOKEN` environment variable 116 | - ✅ Activated your virtual environment (if using one) 117 | - ✅ Installed dependencies with `pip install -r requirements.txt` 118 | - ✅ Your face image is in the project directory 119 | 120 | ### Step 4: Generate Your Face Grid 121 | 122 | **📊 How many images will be created?** 123 | 124 | The default settings create **121 images** (11×11 grid), but you can customize this: 125 | 126 | | Step Size | Grid Size | Total Images | Generation Time | Smoothness | 127 | |-----------|-----------|--------------|-----------------|------------| 128 | | `--step 5` | 7×7 | **49 images** | ~2-3 minutes | Basic | 129 | | `--step 3` (default) | 11×11 | **121 images** | ~5-8 minutes | Good | 130 | | `--step 2.5` | 13×13 | **169 images** | ~8-12 minutes | Smooth | 131 | | `--step 2` | 16×16 | **256 images** | ~12-18 minutes | Very smooth | 132 | 133 | **Basic usage:** 134 | ```bash 135 | # Any filename works! 136 | python main.py --image ./my_face.jpg --out ./out 137 | python main.py --image ./portrait.png --out ./out 138 | python main.py --image /Users/you/Pictures/selfie.jpg --out ./out 139 | ``` 140 | 141 | **Custom grid density:** 142 | ```bash 143 | # Smoother transitions (more images, longer generation time) 144 | python main.py --image ./my_face.jpg --out ./out --step 2.5 145 | 146 | # Faster generation (fewer images, less smooth) 147 | python main.py --image ./my_face.jpg --out ./out --step 5 148 | ``` 149 | 150 | **Full options:** 151 | ```bash 152 | python main.py \ 153 | --image ./my_face.jpg \ # Input image path or URL 154 | --out ./out \ # Output directory 155 | --min -15 \ # Minimum gaze value 156 | --max 15 \ # Maximum gaze value 157 | --step 3 \ # Step size (smaller = more images) 158 | --size 256 \ # Output image size (256x256) 159 | --skip-existing # Skip already generated images 160 | ``` 161 | 162 | ### Understanding the Parameters 163 | 164 | | Parameter | Default | Description | 165 | |-----------|---------|-------------| 166 | | `--min` | -15 | Minimum pupil position (left/up) | 167 | | `--max` | 15 | Maximum pupil position (right/down) | 168 | | `--step` | 3 | Grid spacing (smaller = smoother, more images) | 169 | | `--size` | 256 | Output image dimensions (256×256) | 170 | 171 | **Image count formula:** `((max - min) / step + 1)²` 172 | 173 | Examples: 174 | - `step=3` (default): 121 images (11×11 grid) 175 | - `step=2.5`: 169 images (13×13 grid) 176 | - `step=5`: 49 images (7×7 grid) 177 | 178 | ### Step 5: Output 179 | 180 | The script generates: 181 | - **Images:** `gaze_px{X}_py{Y}_256.webp` files 182 | - Example: `gaze_px0_py0_256.webp` (looking at center) 183 | - Example: `gaze_px15_py0_256.webp` (looking right) 184 | - Example: `gaze_px0_pym15_256.webp` (looking up) 185 | - **CSV Index:** `index.csv` mapping filenames to coordinates 186 | 187 | ``` 188 | out/ 189 | ├── gaze_px-15_py-15_256.webp 190 | ├── gaze_px-15_py-12_256.webp 191 | ├── ... 192 | ├── gaze_px15_py15_256.webp 193 | └── index.csv 194 | ``` 195 | 196 | ## 🎨 Part 2: React Implementation 197 | 198 | ### Setup in Your React Project 199 | 200 | 1. **Copy the generated faces to your public folder:** 201 | 202 | ```bash 203 | # Copy all face images to your React public folder 204 | cp -r ./out/faces /path/to/your-react-app/public/faces 205 | ``` 206 | 207 | 2. **Copy the React files to your project:** 208 | 209 | ```bash 210 | # Copy the hook and component 211 | cp useGazeTracking.js /path/to/your-react-app/src/hooks/ 212 | cp FaceTracker.jsx /path/to/your-react-app/src/components/ 213 | cp FaceTracker.css /path/to/your-react-app/src/components/ 214 | ``` 215 | 216 | ### Basic Usage 217 | 218 | ```jsx 219 | import FaceTracker from './components/FaceTracker'; 220 | 221 | function App() { 222 | return ( 223 |
224 |

My Portfolio

225 | 226 | {/* Basic usage */} 227 | 228 | 229 | {/* With custom styling */} 230 | 234 |
235 | ); 236 | } 237 | ``` 238 | 239 | ### Advanced Usage 240 | 241 | ```jsx 242 | import FaceTracker from './components/FaceTracker'; 243 | 244 | function Header() { 245 | return ( 246 |
252 |
253 | 257 |
258 |
259 | ); 260 | } 261 | ``` 262 | 263 | ### Using the Hook Directly 264 | 265 | For more control, use the `useGazeTracking` hook directly: 266 | 267 | ```jsx 268 | import { useRef, useEffect } from 'react'; 269 | import { useGazeTracking } from './hooks/useGazeTracking'; 270 | 271 | function CustomFaceComponent() { 272 | const containerRef = useRef(null); 273 | const { currentImage, isLoading, error } = useGazeTracking( 274 | containerRef, 275 | '/faces/' 276 | ); 277 | 278 | return ( 279 |
283 | {currentImage && ( 284 | Following face 294 | )} 295 | {isLoading &&

Loading...

} 296 | {error &&

Error: {error.message}

} 297 |
298 | ); 299 | } 300 | ``` 301 | 302 | ### Configuration 303 | 304 | If you change the generation parameters, update these constants in `useGazeTracking.js`: 305 | 306 | ```javascript 307 | // Must match your generation parameters! 308 | const P_MIN = -15; // Same as --min 309 | const P_MAX = 15; // Same as --max 310 | const STEP = 3; // Same as --step 311 | const SIZE = 256; // Same as --size 312 | ``` 313 | 314 | ## 🎛️ Customization 315 | 316 | ### Changing Image Directory 317 | 318 | ```jsx 319 | 320 | ``` 321 | 322 | ### Adding Custom Styling 323 | 324 | ```css 325 | /* FaceTracker.css */ 326 | .face-tracker { 327 | width: 100%; 328 | height: 100%; 329 | position: relative; 330 | overflow: hidden; 331 | background: #f0f0f0; 332 | border-radius: 50%; /* Circular face */ 333 | box-shadow: 0 10px 30px rgba(0,0,0,0.2); 334 | } 335 | 336 | .face-image { 337 | user-select: none; 338 | pointer-events: none; 339 | } 340 | 341 | .face-debug { 342 | position: absolute; 343 | top: 10px; 344 | left: 10px; 345 | background: rgba(0, 0, 0, 0.7); 346 | color: white; 347 | padding: 10px; 348 | border-radius: 5px; 349 | font-family: monospace; 350 | font-size: 12px; 351 | } 352 | ``` 353 | 354 | ### Performance Optimization 355 | 356 | 1. **Preload images** (optional): 357 | ```jsx 358 | useEffect(() => { 359 | // Preload all face images 360 | const images = []; 361 | for (let py = -15; py <= 15; py += 3) { 362 | for (let px = -15; px <= 15; px += 3) { 363 | const img = new Image(); 364 | img.src = `/faces/gaze_px${px}_py${py}_256.webp`; 365 | images.push(img); 366 | } 367 | } 368 | }, []); 369 | ``` 370 | 371 | 2. **Use fewer images** - Generate with larger `--step` value (e.g., `--step 5`) 372 | 373 | ## 📱 Mobile Support 374 | 375 | The component automatically supports touch events! The face will follow finger movement on mobile devices. 376 | 377 | ## 🔧 Troubleshooting 378 | 379 | ### Images not generating? 380 | 381 | **Check your API token:** 382 | ```bash 383 | echo $REPLICATE_API_TOKEN 384 | ``` 385 | 386 | **Verify Replicate credits:** 387 | - Check your account at [replicate.com/account](https://replicate.com/account) 388 | - Free tier includes some credits, but may require payment for large batches 389 | 390 | **Resume interrupted generation:** 391 | ```bash 392 | python main.py --image ./my_face.jpg --out ./out --skip-existing 393 | ``` 394 | 395 | ### Face not following cursor in React? 396 | 397 | 1. **Verify images are in the correct location:** 398 | - Check `public/faces/` directory exists 399 | - Confirm images have correct naming pattern 400 | 401 | 2. **Check browser console for errors:** 402 | - Missing images will show 404 errors 403 | - Path issues will show in network tab 404 | 405 | 3. **Verify configuration matches:** 406 | - `P_MIN`, `P_MAX`, `STEP` in `useGazeTracking.js` must match generation parameters 407 | 408 | ### Performance issues? 409 | 410 | - Generate fewer images (use `--step 5` or higher) 411 | - Reduce image size (use `--size 128`) 412 | - Preload images (see Performance Optimization above) 413 | 414 | ## 📊 Cost Estimation 415 | 416 | Replicate charges per second of GPU time, it should only be a couple of cents. 417 | 418 | Check current pricing at [replicate.com/pricing](https://replicate.com/pricing) 419 | 420 | ## 🎯 Use Cases 421 | 422 | - **Portfolio headers** - Make your about page more engaging 423 | - **Interactive avatars** - Add personality to chat interfaces 424 | - **Product demos** - Draw attention to important elements 425 | - **Educational content** - Create attention-grabbing tutorials 426 | - **Games** - Use as character faces or NPCs 427 | 428 | ## 📝 Examples 429 | 430 | See the [examples](./examples) folder for: 431 | - Full Next.js implementation 432 | - TypeScript version 433 | - Circular face mask 434 | - Multiple faces on one page 435 | 436 | ## 🤝 Contributing 437 | 438 | Contributions welcome! Feel free to: 439 | - Add new features 440 | - Improve documentation 441 | - Share your implementations 442 | - Report bugs 443 | 444 | ## 📄 License 445 | 446 | MIT License - feel free to use in personal and commercial projects! 447 | 448 | ## 🙏 Credits 449 | 450 | - Face generation powered by [Replicate](https://replicate.com/) 451 | - Uses [fofr/expression-editor](https://replicate.com/fofr/expression-editor) model 452 | - Created with ❤️ by Kylan 453 | 454 | ## 🔗 Links 455 | 456 | - [Face Looker Replicate Model](https://replicate.com/kylan02/face-looker) - Generate all images with one click 457 | - [Replicate API Docs](https://replicate.com/docs) 458 | - [Expression Editor Model](https://replicate.com/fofr/expression-editor) 459 | - [Live Demo](https://kylanoconnor.com) 460 | 461 | --- 462 | 463 | **Questions?** Open an issue or contact [@kylancodes](https://x.com/kylancodes) on X 464 | 465 | **Like this project?** Give it a ⭐ on GitHub! 466 | --------------------------------------------------------------------------------