├── .eslintrc ├── .gitignore ├── README.md ├── app.js ├── package.json └── sample_images ├── README.md ├── adult-beard-boy-casual-220453.jpg ├── black-iphone-7-on-brown-table-699122.jpg ├── blur-blurred-background-bokeh-cellphone-1156684.jpg ├── high-angle-shot-of-laptop-and-smartphone-257923.jpg ├── iphone-technology-iphone-6-plus-apple-17663.jpg ├── man-having-a-phone-call-in-front-of-a-laptop-859264.jpg ├── man-wearing-black-zip-jacket-holding-smartphone-surrounded-775091.jpg ├── person-holding-silver-iphone-7-887751.jpg ├── person-in-black-shorts-walking-on-beach-4142815.jpg ├── person-using-iphone-1194760.jpg ├── photo-of-man-in-white-dress-shirt-holding-phone-near-window-859265.jpg ├── photography-of-a-guy-wearing-green-shirt-1222271.jpg ├── silver-iphone-6-987585.jpg ├── silver-iphone-6-beside-click-pen-and-card-843266.jpg ├── silver-iphone-x-with-airpods-788946.jpg ├── white-apple-iphone-on-wooden-table-48605.jpg ├── woman-in-white-t-shirt-holding-smartphone-in-front-of-laptop-914931.jpg ├── woman-sitting-on-sofa-while-looking-at-phone-with-laptop-on-920382.jpg └── women-s-white-and-black-button-up-collared-shirt-774909.jpg /.eslintrc: -------------------------------------------------------------------------------- 1 | { 2 | "extends": "google", 3 | "env": { 4 | "commonjs": true, 5 | "es2017": true, 6 | "node": true 7 | } 8 | } 9 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Logs 2 | logs 3 | *.log 4 | npm-debug.log* 5 | yarn-debug.log* 6 | yarn-error.log* 7 | 8 | # Dependency directories 9 | node_modules/ 10 | 11 | # Optional npm cache directory 12 | .npm 13 | 14 | # Optional eslint cache 15 | .eslintcache 16 | 17 | # Optional REPL history 18 | .node_repl_history 19 | 20 | # Yarn Integrity file 21 | .yarn-integrity 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Teachable Machine Node Example 2 | A demonstration of using [Teachable Machine](https://teachablemachine.withgoogle.com/), which is intended to run in the browser, in Node. Based on the clever approach by [tr7zw](https://github.com/tr7zw) in their [original demo](https://github.com/tr7zw/teachablemachine-node-example). 3 | 4 | ## How It Works 5 | 1. A workaround is used to simulate in Node just enough of the browser parts required for Teachable Machine to work. 6 | * [jsdom](https://github.com/jsdom/jsdom) to emulate a subset of browser APIs. 7 | * [node-canvas](https://github.com/Automattic/node-canvas) to simulate the browser's canvas. 8 | * [node-fetch](https://github.com/node-fetch/node-fetch) to polyfill [fetch](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) for Node. 9 | 2. The script makes Teachable Machine models available through API endpoints defined in the script using [Express](https://expressjs.com/). 10 | 3. An image posted to an endpoint is parsed and passed to the model. Predictions are made and the endpoint responds to the request with JSON formatted prediction results. 11 | 12 | ## Usage 13 | 1. Clone this repo, `cd` into it, and run `npm install` 14 | 2. Edit app.js and add create endpoints associated with a trained model. 15 | 3. Run app server either with `npm start` or `node app.js` 16 | 4. Post an image to the endpoint. **This has the potential to be a source of frustration** — if POSTing the image is not done correctly the application may crash of malfunction. See "Image POST Tips and Notes" and "Post Examples" below. 17 | 18 | ## Image POST Tips and Notes 19 | * Ensure you are pointing to the right endpoint name on the right port. (e.g. localhost:3000 if posting from the same computer the app is running on, or myhostname:3000 if posting from another computer) 20 | * Ensure you are supplying the correct path to the image being posted. 21 | 22 | ### POST Examples 23 | **Linux/OSX** 24 | `curl -X POST -H "Content-Type: image/jpeg" --data-binary '@./sample_images/person-using-iphone-1194760.jpg' http://localhost:3000/test` 25 | view a [breakdown of this command](https://bit.ly/3aEZ4t5) 26 | 27 | **Windows** 28 | `PowerShell.exe Invoke-WebRequest -uri http://localhost:3000/test -Method Post -Infile "./sample_images/person-using-iphone-1194760.jpg" -ContentType 'image/jpeg'` 29 | 30 | ## Notes 31 | * The sample model used makes a prediction on whether a supplied image contains a Solo Human, a Solo Phone, or a Human and Phone. **I am not the creator of this model**. It is used here with permission from the author, [lachlanjc](https://github.com/lachlanjc), who uses it [in this post](https://ima.lachlanjc.me/2019-11-17_cc_week_12_project/). 32 | + Use [the model's URL](https://teachablemachine.withgoogle.com/models/AI5i76oG/) to explore interactively in the Teachable Machine UI or use it in the code for this demo by passing the URL to the `addEndpoint()` method. 33 | * [node-canvas is a jsdom dependency](https://github.com/jsdom/jsdom#canvas-support) for canvas/img support 34 | + See system dependency info at https://bit.ly/3bI4V2a 35 | * Notable changes from the original demo on which this is based: 36 | + Removes requirement in original demo to monkey patch (comment out) part of the Teachable Machine npm package. 37 | + Removes custom _arrayBufferToBase64 method in favor of built-in conversion. 38 | + Dynamically allows for different image Content-Types (i.e. image/jpeg, image/png). 39 | + Uses eslint with [Google style guide](https://google.github.io/styleguide/jsguide.html) configuration for linting. 40 | + Adds additional comments and JSDOCs. 41 | -------------------------------------------------------------------------------- /app.js: -------------------------------------------------------------------------------- 1 | /** 2 | * A demonstration of using Teachable Machine, which is intended to run in the 3 | * browser, in Node. The main entry point of the application is the `init()` 4 | * method, which is called at the end of this script. Configure your endpoints 5 | * in the `configureEndPoints()` method. 6 | */ 7 | const canvas = require('canvas'); 8 | const express = require('express'); 9 | const JSDOM = require('jsdom').JSDOM; 10 | require('@tensorflow/tfjs-node'); 11 | const tmImage = require('@teachablemachine/image'); 12 | 13 | const app = express(); 14 | 15 | /** 16 | * Performs application initialization tasks including configuration of browser 17 | * API polyfills and associating models generated in Teachable Machine with HTTP 18 | * accessible endpoints. 19 | */ 20 | function init() { 21 | configureBodyParser(); 22 | configureEndPoints(); 23 | configureBrowserPolyFills(); 24 | 25 | // Configure the server to begin listening for requests. 26 | app.listen(3000, () => { 27 | console.log('Server running on port 3000'); 28 | }); 29 | } 30 | 31 | /** 32 | * Creates an application endpoint (route) and a associates it with the model it 33 | * should handle. 34 | * @param {string} name The route name to be used for the endpoint. 35 | * @param {string} baseUrl The base URL from where the model files can be 36 | * loaded. `baseUrl` should end with a trailing slash 37 | * (e.g. 'https://teachablemachine.withgoogle.com/models/AI5i76oG/') 38 | * Note that local filesystem paths are not supported. 39 | */ 40 | async function addEndpoint(name, baseUrl) { 41 | const modelURL = baseUrl + 'model.json'; 42 | const metadataURL = baseUrl + 'metadata.json'; 43 | const model = await tmImage.load(modelURL, metadataURL); 44 | app.post('/' + name, (request, response) => { 45 | const base64Image = Buffer.from(request.body).toString('base64'); 46 | const contentType = request.get('Content-Type'); 47 | getPrediction(model, base64Image, contentType, (output) => { 48 | response.send(output); 49 | }); 50 | }); 51 | } 52 | 53 | /** 54 | * Configure body-parser's raw parser (@see https://bit.ly/2y70YFE) to handle 55 | * requests where the `Content-Type` header matches the type specified (in this 56 | * case, 'image/jpeg'). A new, parsed body object will be available on request 57 | * object (i.e. req.body). The body will be a Buffer object. 58 | */ 59 | function configureBodyParser() { 60 | app.use(require('body-parser').raw({type: 'image/jpeg', limit: '3MB'})); 61 | } 62 | 63 | /** 64 | * Simulates in Node just enough of the browser parts required for Teachable 65 | * Machine to work. 66 | */ 67 | function configureBrowserPolyFills() { 68 | global.window = new JSDOM(` 69 | 70 | 73 | `).window; 74 | global.document = window.document; 75 | global.fetch = require('node-fetch'); 76 | global.HTMLVideoElement = class HTMLVideoElement { }; 77 | } 78 | 79 | /** 80 | * Create HTTP accessible named endpoints, each associated with a Teachable 81 | * Machine generated model. Add as many as you'd like. 82 | */ 83 | function configureEndPoints() { 84 | addEndpoint('test', 85 | 'https://teachablemachine.withgoogle.com/models/AI5i76oG/'); 86 | } 87 | 88 | /** 89 | * Forms a prediction for the supplied image data against a supplied model. Once 90 | * a prediction is formed, a supplied callback function is called, with the 91 | * prediction as its argument. A supplied function receives the prediction and 92 | * completes the request cycle by sending the prediction as the response. 93 | * @param {!CustomMobileNet} model An instance of the model. (for type info, 94 | * @see https://bit.ly/2yOoM12) 95 | * @param {string} imageData A Base64 encoded representation of the image to be 96 | * evaluated against the model. 97 | * @param {string} contentType The content type of the supplied imageData. 98 | * @param {!Function} responseFunction A function that receives the prediction 99 | * and completes the request cycle by sending the prediction as a JSON 100 | * formatted response. 101 | */ 102 | async function getPrediction(model, imageData, contentType, responseFunction) { 103 | const imageCanvas = canvas.createCanvas(64, 64); 104 | const canvasContext = imageCanvas.getContext('2d'); 105 | 106 | const canvasImage = new canvas.Image(); 107 | canvasImage.onload = async () => { 108 | canvasContext.drawImage(canvasImage, 0, 0, 64, 64); 109 | 110 | const prediction = await model.predict(imageCanvas); 111 | console.log(prediction); 112 | responseFunction(prediction); 113 | }; 114 | 115 | canvasImage.onerror = (error) => { 116 | throw error; 117 | }; 118 | 119 | canvasImage.src = `data:${contentType};base64,` + imageData; 120 | } 121 | 122 | // Main application entry point. 123 | init(); 124 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "teachable_machine_node_demo", 3 | "version": "1.0.0", 4 | "description": "A demonstration of using Teachable Machine, which is intended to run in the browser, in Node", 5 | "main": "app.js", 6 | "scripts": { 7 | "start": "node app.js" 8 | }, 9 | "author": "skyebot", 10 | "license": "DWYWL", 11 | "dependencies": { 12 | "@teachablemachine/image": "^0.8.4", 13 | "@tensorflow/tfjs": "^1.3.1", 14 | "@tensorflow/tfjs-node": "^0.1.9", 15 | "body-parser": "^1.19.0", 16 | "canvas": "^2.6.0", 17 | "express": "^4.17.1", 18 | "jsdom": "^15.2.1", 19 | "node-fetch": "^2.6.0" 20 | }, 21 | "repository": { 22 | "type": "git", 23 | "url": "https://github.com/drinkspiller/teachablemachine-node-example.git" 24 | }, 25 | "devDependencies": { 26 | "eslint": "^6.8.0", 27 | "eslint-config-google": "^0.14.0" 28 | } 29 | } 30 | -------------------------------------------------------------------------------- /sample_images/README.md: -------------------------------------------------------------------------------- 1 | All photos are from pexels.com 2 | See license at https://www.pexels.com/license/ 3 | -------------------------------------------------------------------------------- /sample_images/adult-beard-boy-casual-220453.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/drinkspiller/teachablemachine-node-example/090c89d3989252fb0e5a0ff318870844b0b911a4/sample_images/adult-beard-boy-casual-220453.jpg -------------------------------------------------------------------------------- /sample_images/black-iphone-7-on-brown-table-699122.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/drinkspiller/teachablemachine-node-example/090c89d3989252fb0e5a0ff318870844b0b911a4/sample_images/black-iphone-7-on-brown-table-699122.jpg -------------------------------------------------------------------------------- /sample_images/blur-blurred-background-bokeh-cellphone-1156684.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/drinkspiller/teachablemachine-node-example/090c89d3989252fb0e5a0ff318870844b0b911a4/sample_images/blur-blurred-background-bokeh-cellphone-1156684.jpg -------------------------------------------------------------------------------- /sample_images/high-angle-shot-of-laptop-and-smartphone-257923.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/drinkspiller/teachablemachine-node-example/090c89d3989252fb0e5a0ff318870844b0b911a4/sample_images/high-angle-shot-of-laptop-and-smartphone-257923.jpg -------------------------------------------------------------------------------- /sample_images/iphone-technology-iphone-6-plus-apple-17663.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/drinkspiller/teachablemachine-node-example/090c89d3989252fb0e5a0ff318870844b0b911a4/sample_images/iphone-technology-iphone-6-plus-apple-17663.jpg -------------------------------------------------------------------------------- /sample_images/man-having-a-phone-call-in-front-of-a-laptop-859264.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/drinkspiller/teachablemachine-node-example/090c89d3989252fb0e5a0ff318870844b0b911a4/sample_images/man-having-a-phone-call-in-front-of-a-laptop-859264.jpg -------------------------------------------------------------------------------- /sample_images/man-wearing-black-zip-jacket-holding-smartphone-surrounded-775091.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/drinkspiller/teachablemachine-node-example/090c89d3989252fb0e5a0ff318870844b0b911a4/sample_images/man-wearing-black-zip-jacket-holding-smartphone-surrounded-775091.jpg -------------------------------------------------------------------------------- /sample_images/person-holding-silver-iphone-7-887751.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/drinkspiller/teachablemachine-node-example/090c89d3989252fb0e5a0ff318870844b0b911a4/sample_images/person-holding-silver-iphone-7-887751.jpg -------------------------------------------------------------------------------- /sample_images/person-in-black-shorts-walking-on-beach-4142815.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/drinkspiller/teachablemachine-node-example/090c89d3989252fb0e5a0ff318870844b0b911a4/sample_images/person-in-black-shorts-walking-on-beach-4142815.jpg -------------------------------------------------------------------------------- /sample_images/person-using-iphone-1194760.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/drinkspiller/teachablemachine-node-example/090c89d3989252fb0e5a0ff318870844b0b911a4/sample_images/person-using-iphone-1194760.jpg -------------------------------------------------------------------------------- /sample_images/photo-of-man-in-white-dress-shirt-holding-phone-near-window-859265.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/drinkspiller/teachablemachine-node-example/090c89d3989252fb0e5a0ff318870844b0b911a4/sample_images/photo-of-man-in-white-dress-shirt-holding-phone-near-window-859265.jpg -------------------------------------------------------------------------------- /sample_images/photography-of-a-guy-wearing-green-shirt-1222271.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/drinkspiller/teachablemachine-node-example/090c89d3989252fb0e5a0ff318870844b0b911a4/sample_images/photography-of-a-guy-wearing-green-shirt-1222271.jpg -------------------------------------------------------------------------------- /sample_images/silver-iphone-6-987585.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/drinkspiller/teachablemachine-node-example/090c89d3989252fb0e5a0ff318870844b0b911a4/sample_images/silver-iphone-6-987585.jpg -------------------------------------------------------------------------------- /sample_images/silver-iphone-6-beside-click-pen-and-card-843266.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/drinkspiller/teachablemachine-node-example/090c89d3989252fb0e5a0ff318870844b0b911a4/sample_images/silver-iphone-6-beside-click-pen-and-card-843266.jpg -------------------------------------------------------------------------------- /sample_images/silver-iphone-x-with-airpods-788946.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/drinkspiller/teachablemachine-node-example/090c89d3989252fb0e5a0ff318870844b0b911a4/sample_images/silver-iphone-x-with-airpods-788946.jpg -------------------------------------------------------------------------------- /sample_images/white-apple-iphone-on-wooden-table-48605.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/drinkspiller/teachablemachine-node-example/090c89d3989252fb0e5a0ff318870844b0b911a4/sample_images/white-apple-iphone-on-wooden-table-48605.jpg -------------------------------------------------------------------------------- /sample_images/woman-in-white-t-shirt-holding-smartphone-in-front-of-laptop-914931.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/drinkspiller/teachablemachine-node-example/090c89d3989252fb0e5a0ff318870844b0b911a4/sample_images/woman-in-white-t-shirt-holding-smartphone-in-front-of-laptop-914931.jpg -------------------------------------------------------------------------------- /sample_images/woman-sitting-on-sofa-while-looking-at-phone-with-laptop-on-920382.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/drinkspiller/teachablemachine-node-example/090c89d3989252fb0e5a0ff318870844b0b911a4/sample_images/woman-sitting-on-sofa-while-looking-at-phone-with-laptop-on-920382.jpg -------------------------------------------------------------------------------- /sample_images/women-s-white-and-black-button-up-collared-shirt-774909.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/drinkspiller/teachablemachine-node-example/090c89d3989252fb0e5a0ff318870844b0b911a4/sample_images/women-s-white-and-black-button-up-collared-shirt-774909.jpg --------------------------------------------------------------------------------