├── .gitignore ├── README.md ├── app.js ├── images ├── drop.png ├── load.png └── play.png ├── lib └── video.js ├── package.json ├── public ├── css │ └── style.css ├── index.html └── js │ ├── lib │ ├── binary.js │ ├── common.js │ ├── jquery.js │ └── video.js │ └── main.js └── samples ├── mp4-sample.mp4 ├── ogg-sample.ogv └── webm-sample.webm /.gitignore: -------------------------------------------------------------------------------- 1 | *.swp 2 | videos/ 3 | node_modules/ 4 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # File Uploading and Streaming with Node.js 2 | 3 | A while back, if you wanted to stream binary data via JavaScript - such as 4 | audio/video content, you'd be sore out of luck :( 5 | 6 | You'd have to rely on either Flash, Java applets or 3rd party plugins that 7 | provided similar functionality. Uggh. 8 | 9 | Over the past few years, advancements in JavaScript on both fronts: server-side 10 | and client-side, now allow you to do so without having to resort to otherwise 11 | tedious workarounds. 12 | 13 | In this post, I'll show you how to upload and stream video files - yup, you heard 14 | that right :) 15 | 16 | How exactly do you ask? By using an awesome Node module called BinaryJS, and 17 | some good ol' client-side Javascripting! 18 | 19 | ## What We'll Need 20 | 21 | Before we can get started writing code to stream binary data, we need to install 22 | some modules. We'll only need two: `express` and `binaryjs`. 23 | 24 | ### Express 25 | 26 | The defacto Node.js web framework! My framework of choice, and that of many fellow 27 | Node developers out there. It's fast, easy-to-use and well-documented. 28 | 29 | To familiarize yourself with the `express` API, if you haven't already done so, 30 | check out the official [ExpressJS API documentation](http://www.expressjs.com/api.html) 31 | 32 | If `express` is not your cup of tea, you're most welcome to opt it out for something 33 | you're more comfortable with. 34 | 35 | ### BinaryJS 36 | 37 | The heart of our video streaming web app! This module uses WebSockets and the 38 | BinaryPack serialization scheme to stream binary content back-and-forth between 39 | the server and the client. 40 | 41 | Want to find out more? Here's the official [BinaryJS Website](http://www.binaryjs.com/), 42 | and here's the [API documentation](https://github.com/binaryjs/binaryjs/tree/master/doc) 43 | for good measure. 44 | 45 | ## The Workflow 46 | 47 | First off, I'll outline the workflow for both the server and client portions of 48 | the video server we're building. 49 | 50 | ### Server-side 51 | 52 | 1. Create an instance of the BinaryJS server 53 | 2. Register custom events and handlers for: 54 | 55 | * uploading videos 56 | * requesting for a video 57 | * listing available videos 58 | 59 | ### Client-side 60 | 61 | 1. Create an instance of the BinaryJS client 62 | 2. Upon connecting to the BinaryJS server, retrieve a list of available videos and present it 63 | 3. Clicking a link in the video list should load the affected video 64 | 4. Add a means to upload video files: 65 | 66 | * use **Drag n Drop** for a better UX experience 67 | * refresh the list of available videos 68 | 69 | ## Installation 70 | 71 | First off, install the modules in your project directory via npm. I've added 72 | the version numbers that were installed for me: 73 | 74 | * Express v3.4.6 75 | * BinaryJS v0.2.1 76 | 77 | ``` 78 | $ npm install express 79 | $ npm install binaryjs 80 | ``` 81 | 82 | Next up, bootstrap your web app using express: 83 | 84 | ``` 85 | $ node_modules/express/bin/express . 86 | ``` 87 | 88 | Or if you have express installed globally: 89 | 90 | ``` 91 | $ express . 92 | ``` 93 | 94 | Remove the directories we don't need: 95 | 96 | ``` 97 | $ rm -rf routes/ views/ 98 | ``` 99 | 100 | Replace the generated copy of `package.json` with this: 101 | 102 | ``` 103 | { 104 | "name": "binaryjs-upload-stream", 105 | "version": "0.1.0", 106 | "private": true, 107 | "scripts": { 108 | "start": "node app.js" 109 | }, 110 | "dependencies": { 111 | "express": "3.4.6", 112 | "binaryjs": "0.2.1" 113 | } 114 | } 115 | ``` 116 | 117 | Also, don't forget to clear out irrelevant code in `app.js`: 118 | 119 | ``` 120 | // remove these lines 121 | app.set('views', path.join(__dirname, 'views')); 122 | app.set('view engine', 'jade'); 123 | 124 | // we won't need routing too 125 | app.get('/', routes.index); 126 | app.get('/users', user.list); 127 | ``` 128 | 129 | For the finishing touch, I renamed some directories in `public/`. This is a 130 | matter of preference and therefore entirely optional. 131 | 132 | ``` 133 | $ cd public/ 134 | $ ls 135 | 136 | images javascript stylesheets 137 | 138 | $ mv javascript/ js/ 139 | $ mv stylesheets/ css/ 140 | ``` 141 | 142 | ## Coding The Backend 143 | 144 | ### app.js 145 | 146 | Open up `app.js` and start coding! You'll need to create an 147 | instance of `BinaryServer`, which the `binaryjs` module provides. 148 | 149 | Also, add a reference to the `video` library for later. 150 | 151 | ``` 152 | // add these two lines near the variable declarations at the top 153 | BinaryServer = require('binaryjs').BinaryServer; 154 | video = require('./lib/video'); 155 | ``` 156 | 157 | I set my instance to run on port `9000`. If you don't specify a custom port, 158 | it'll piggyback on whatever port you've set on `express` after which you'll 159 | need to set a custom endpoint. 160 | 161 | ``` 162 | // add this after the call to server.listen() 163 | bs = new BinaryServer({ port: 9000 }); 164 | ``` 165 | 166 | Now we set the `connection` handler for the `binaryjs` server. 167 | It provides a `client` object which is of type `binaryjs.BinaryClient` 168 | 169 | The client's `stream` event returns both a `stream` object as well as a `meta` object, configurable from the client-side. 170 | 171 | Add handlers for the following meta events: 172 | 173 | * `list` 174 | * `request` 175 | * `upload` 176 | 177 | ``` 178 | bs.on('connection', function (client) { 179 | client.on('stream', function (stream, meta) { 180 | switch(meta.event) { 181 | // list available videos 182 | case 'list': 183 | video.list(stream, meta); 184 | break; 185 | 186 | // request for a video 187 | case 'request': 188 | video.request(client, meta); 189 | break; 190 | 191 | // attempt an upload 192 | case 'upload': 193 | default: 194 | video.upload(stream, meta); 195 | } 196 | }); 197 | }); 198 | ``` 199 | 200 | ### video.js 201 | 202 | Create a source file for managing the videos, I put mine in `lib/video.js`. This 203 | file will house the implementations for the following capabilities: 204 | 205 | * listing of available videos 206 | * requesting of a video for playback 207 | * uploading of a video to the server 208 | 209 | ``` 210 | var fs, uploadPath, supportedTypes; 211 | 212 | fs = require('fs'); 213 | uploadPath = __dirname + '/../videos'; 214 | supportedTypes = [ 215 | 'video/mp4', 216 | 'video/webm', 217 | 'video/ogg' 218 | ]; 219 | 220 | module.exports = { 221 | list : list, 222 | request : request, 223 | upload : upload 224 | }; 225 | ``` 226 | 227 | The `list` function does the simple task of reading filenames in 228 | the `videos/` directory and streaming back a list of it to the client. 229 | 230 | ``` 231 | function list(stream, meta) { 232 | fs.readdir(uploadPath, function (err, files) { 233 | stream.write({ files : files }); 234 | }); 235 | } 236 | ``` 237 | 238 | `request` creates a read stream for the requested video file, and streams 239 | it in chunks back to the client. 240 | 241 | ``` 242 | function request(client, meta) { 243 | var file = fs.createReadStream(uploadPath + '/' + meta.name); 244 | 245 | client.send(file); 246 | } 247 | ``` 248 | 249 | The file upload implementation in `upload` checks if the file is of a supported video type. 250 | 251 | If the type matches, the function proceeds - otherwise, it returns an error. 252 | 253 | For the sake of convenience, the function informs the client of the upload 254 | status as it writes the video to disk, chunk by chunk. 255 | 256 | ``` 257 | function upload(stream, meta) { 258 | if (!~supportedTypes.indexOf(meta.type)) { 259 | stream.write({ err: 'Unsupported type: ' + meta.type }); 260 | stream.end(); 261 | return; 262 | } 263 | 264 | var file = fs.createWriteStream(uploadPath + '/' + meta.name); 265 | stream.pipe(file); 266 | 267 | stream.on('data', function (data) { 268 | stream.write({ rx: data.length / meta.size }); 269 | }); 270 | 271 | stream.on('end', function () { 272 | stream.write({ end: true }); 273 | }); 274 | } 275 | ``` 276 | 277 | ## Coding the Frontend 278 | 279 | ### index.html 280 | 281 | Add the following HTML to your landing page's `` tag. 282 | 283 | ``` 284 |

BinaryJS File Upload and Streaming

285 | 286 |
287 |
288 | Drag n Drop 289 | 292 |
293 | 294 |
295 | Select a Link 296 | 297 | 298 |
299 | 300 |
301 | Play the Video 302 | 303 |
304 | 305 |
306 |
307 |
308 | ``` 309 | 310 | Insert the following ` 315 | 316 | 317 | 318 | 319 | ``` 320 | 321 | ### common.js 322 | 323 | Before anything else can work client-side, make sure to create an 324 | instance of `BinaryClient` with a port of `9000` - or whichever port 325 | you have changed it to - and save this to `js/lib/common.js`. 326 | 327 | ``` 328 | var hostname, client; 329 | 330 | hostname = window.location.hostname; 331 | client = new BinaryClient('ws://' + hostname + ':9000'); 332 | ``` 333 | 334 | The `common.js` file also includes helper functions like `fizzle`, 335 | used to prevent event propagation in JavaScript ... 336 | 337 | ``` 338 | function fizzle(e) { 339 | e.preventDefault(); 340 | e.stopPropagation(); 341 | } 342 | ``` 343 | 344 | And `emit`, which is essentially a wrapper to the 345 | `BinaryClient` method `send`. 346 | 347 | `client.send` takes two arguments: tle file to be 348 | streamed over to the video server, and the 349 | accompanying meta data - in that order. 350 | 351 | ``` 352 | function emit(event, data, file) { 353 | file = file || {}; 354 | data = data || {}; 355 | data.event = event; 356 | 357 | return client.send(file, data); 358 | } 359 | ``` 360 | 361 | ### video.js 362 | 363 | For `js/lib/video.js`, add functions that implement: 364 | 365 | * retrieving of video listings from the video server 366 | * uploading of a video file to the video server 367 | * requesting of a video file from the video server 368 | * downloading of a requested video file from the video server 369 | 370 | ``` 371 | function list(cb) { 372 | var stream = emit('list'); 373 | 374 | stream.on('data', function (data) { 375 | cb(null, data.files); 376 | }); 377 | 378 | stream.on('error', cb); 379 | } 380 | ``` 381 | 382 | The `upload` method facilitates the uploading 383 | of a file - the streaming, and the resulting 384 | feedback of the upload as it progresses. 385 | 386 | ``` 387 | function upload(file, cb) { 388 | var stream = emit('upload', { 389 | name : file.name, 390 | size : file.size, 391 | type : file.type 392 | }, file); 393 | 394 | stream.on('data', function (data) { 395 | cb(null, data); 396 | }); 397 | 398 | stream.on('error', cb); 399 | } 400 | ``` 401 | 402 | The `request` function is nothing more than a 403 | wrapper function for the `request` event: 404 | 405 | ``` 406 | function request(name) { 407 | emit('request', { name : name }); 408 | } 409 | ``` 410 | 411 | In order to get downloading to work, the chunks of video data that 412 | get streamed in as `ArrayBuffer` objects need to be stitched together 413 | in a `Blob` instance. 414 | 415 | The `src` object, containing the newly formed `Blob`, can then be returned 416 | in a callback. 417 | 418 | ``` 419 | function download(stream, cb) { 420 | var parts = []; 421 | 422 | stream.on('data', function (data) { 423 | parts.push(data); 424 | }); 425 | 426 | stream.on('error', function (err) { 427 | cb(err); 428 | }); 429 | 430 | stream.on('end', function () { 431 | var src = (window.URL || window.webkitURL).createObjectURL(new Blob(parts)); 432 | 433 | cb(null, src); 434 | }); 435 | } 436 | ``` 437 | 438 | ### main.js 439 | 440 | The final file, `js/main.js` ties the presentation layer with application logic. 441 | 442 | Once the connection is up, as denoted by the `open` event, add handling for 443 | video listings and Drag n' Drop. 444 | 445 | ``` 446 | client.on('open', function () { 447 | video.list(setupList); 448 | $box.on('drop', setupDragDrop); 449 | }); 450 | ``` 451 | 452 | In the `stream` event, we assume that anything that gets streamed back 453 | without initiation from the client-side (list, video request, etc) is 454 | undoubtedly a video file. 455 | 456 | ``` 457 | client.on('stream', function (stream) { 458 | video.download(stream, function (err, src) { 459 | $video.attr('src', src); 460 | }); 461 | }); 462 | ``` 463 | 464 | `setupList` refreshes the file listing visuals everytime a list request 465 | is sent. 466 | 467 | ``` 468 | function setupList(err, files) { 469 | var $ul, $li; 470 | 471 | $list.empty(); 472 | $ul = $('