├── LICENSE ├── README.md ├── gst-jack-janus.sh ├── images ├── image1.png ├── image2.png └── image3.png ├── web ├── css │ ├── demo.css │ └── piano.css ├── favicon.ico ├── index.html ├── index.js └── janus.js └── webrtc-piano.lua /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Lorenzo Miniero 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | webrtc-piano 2 | ============ 3 | 4 | This is the code I originally wrote for the "Dangerous Demo" session at ClueCon '19 a few months ago. 5 | 6 | The short demo (about 3 minutes) can be seen on [YouTube](https://youtu.be/8Hzg4hSJMsQ?t=790), and was basically an attempt to use WebRTC and Jack together for music purposes. More specifically: 7 | 8 | * I used this cool CSS [piano keyboard](https://codepen.io/felipefialho/pen/oDEki) snippet by [Felipe Fialho](https://piano.felipefialho.com/) to allow multiple web participants to "play" the same keyboard. 9 | * I then hooked the key hits to a data channel, connected to the [Janus WebRTC Server](https://github.com/meetecho/janus-gateway/). 10 | * A Lua script in Janus translates the key hits to MIDI events, which are passed to "Midi Through" in Jack. 11 | * A custom Gstreamer pipeline then listens for the output, encodes it to Opus, and sends it via RTP to a [Streaming mountpoint](https://janus.conf.meetecho.com/docs/streaming) in Janus. 12 | * All web users connected to the demo subscribe to the mountpoint, and so can hear the output via WebRTC in real-time. 13 | 14 | While of course not perfect, I thought this was a cool experiment, so I finally decided to update it and clean it up a bit, and share it here. Hopefully, it will help foster more usages of WebRTC for music-related purposes, which as I wrote in [this post](https://linuxmusicians.com/viewtopic.php?f=28&t=21617) is something I'd really love to see happening more. Feel free to do with it whatever you want! 15 | 16 | ## Requirements 17 | 18 | The demo requires a working installation of Janus, and of the Lua plugin: please refer to the Janus README for instructions on how to install both properly. Besides, you'll need to install the [midialsa](https://www.pjb.com.au/comp/lua/midialsa.html) library for Lua, as the script provided in this repo makes use of it. 19 | 20 | You'll also need an installation of gstreamer that supports [jackaudiosrc](https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-good/html/gst-plugins-good-plugins-jackaudiosrc.html), besides Opus and RTP, since the script provided in this repo uses them all. 21 | 22 | Of course, you'll need a working [Jack](https://jackaudio.org/) setup as well. How you'll configure that (e.g., what to use to render the MIDI data) will be up to you, and explained later. 23 | 24 | ## Configuring Janus 25 | 26 | Besides the usual configuration, you'll need to do two things: 27 | 28 | 1. configure the Lua plugin to use the script provided here: 29 | 2. create a Streaming plugin mountpoint that can receive the stream encoded by the gstreamer pipeline in this repo. 30 | 31 | The former is quite easy, and can be done by editing the `janus.plugin.lua.jcfg` configuration file, e.g.: 32 | 33 | ``` 34 | general: { 35 | path = "/opt/janus/share/janus/lua" 36 | script = "/home/lminiero/webrtc-piano/webrtc-piano.lua" 37 | } 38 | ``` 39 | 40 | If everything's working, you should see something like this in the logs when you launch Janus: 41 | 42 | ``` 43 | Loading plugin 'libjanus_lua.so'... 44 | [webrtc-piano.lua] Loading... 45 | [webrtc-piano.lua] Loaded 46 | [webrtc-piano.lua] Initialized 47 | Janus Lua plugin initialized! 48 | ``` 49 | 50 | Creating the Streaming plugin mountpoint is quite straightforward too. All you need to do is creating an audio-only mountpoint that listens on the port the gstreamer pipeline is configured to send to: of course, feel free to choose a different port and update the gstreamer pipeline accordingly. This is a simple example you can add to the `janus.plugin.streaming.jcfg` configuration file: 51 | 52 | ``` 53 | webrtc-piano: { 54 | type = "rtp" 55 | id = "2019" 56 | audio = true 57 | audioport = 20190 58 | audiopt = 111 59 | audiortpmap = "opus/48000/2" 60 | secret = "adminpwd" 61 | } 62 | ``` 63 | 64 | If everything's working, when you start the gstreamer script (`gst-jack-janus.sh`) you should see something like this in the Janus logs: 65 | 66 | ``` 67 | [webrtc-piano] New audio stream! (ssrc=1980195523) 68 | ``` 69 | 70 | which means Janus is correctly receiving data from it. 71 | 72 | ## Playing with Jack 73 | 74 | After you start Janus and launch the gstreamer script, your Jack graph should look something like this: 75 | 76 | ![Janus and Jack](images/image1.png) 77 | 78 | As you can see in the screenshot above, the gstreamer pipeline should be ready to receive incoming audio via Jack, while the Lua script in Janus (which registers as "Janus WebRTC Piano client") should automatically connect to the MIDI through node via ALSA. I chose to have it connected there to make it easier to switch instruments dynamically, but you're of course free to edit the Lua script to implement a different behaviour, or change the connections to your liking after the fact. 79 | 80 | After that, it should be straightforward to figure out the next steps: all you need to do is launch whatever you want to be able to receive the MIDI input and render it to audio, and connect things accordingly. In the screenshot below, I used a yoshimi instance for the purpose, but it should work with anything else. In case the application you want to send MIDI to doesn't support ALSA, but only Jack, then you'll have to use something like [a2jmidid](https://github.com/linuxaudio/a2jmidid/) for bridging them. 81 | 82 | ![Hooking a Jack application](images/image2.png) 83 | 84 | I simply connected the MIDI through node to Yoshimi (so that it will receive the notes played by the web user), and then connected the output of yoshimi itself to the gstreamer pipeline (which means that whaterver yoshimi plays will encoded and sent in real-time to Janus, and from there to users). At this point, the demo should be ready to be tested. 85 | 86 | ## Using the demo 87 | 88 | To try the demo, you need to serve the contents of the `web` folder with a web server. Notice that, while locally (localhost) it's not needed, you may have to serve the page via HTTPS if the user is on a different machine: in that case, make sure you follow the [deployment guidelines](https://janus.conf.meetecho.com/docs/deploy) for Janus, to serve the Janus API and the pages through the same server (e.g., via nginx); notice you'll need to update the `server` variable in `index.js` accordingly, in case. 89 | 90 | **IMPORTANT:** browsers currently don't support Jack, which means that if you launch the demo on the same machine Janus and Jack are started on, you will NOT hear the output coming via WebRTC through the browser. You'll need a browser launched on a separate machine for this. 91 | 92 | That said, using the demo should be quite straightforward. After opening the page, you should be prompted for a name, which will be how you'll be seen in the demo: a random color is picked for you as well, and the same will happen for other users joining the session. 93 | 94 | ![Playing in the browser](images/image3.png) 95 | 96 | Playing on the keyboard should result in the audio (in my case rendered by yoshimi) through your browser. When you hit a key, the event is relayed to all the other users in the session as well: this means that, if another participant plays a note, you'll see the related key changing color to the one associated to the participant, and so will them when it's you playing. Good chances this is very buggy (I didn't test it much), but for demo purposes this is more than enough. 97 | 98 | Hope you'll enjoy this little toy! Please feel free to join the [Janus community](https://groups.google.com/forum/#!forum/meetecho-janus) in case you have trouble installing Janus or getting it to run as you want to play with this. 99 | -------------------------------------------------------------------------------- /gst-jack-janus.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | gst-launch-1.0 \ 4 | jackaudiosrc name="Janus Streaming mountpoint" connect=0 ! \ 5 | audioconvert ! opusenc bitrate=20000 ! rtpopuspay ! \ 6 | udpsink host=127.0.0.1 port=20190 7 | -------------------------------------------------------------------------------- /images/image1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lminiero/webrtc-piano/dbcb9329d1626dfdf354355c1f7c4619a66bec07/images/image1.png -------------------------------------------------------------------------------- /images/image2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lminiero/webrtc-piano/dbcb9329d1626dfdf354355c1f7c4619a66bec07/images/image2.png -------------------------------------------------------------------------------- /images/image3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lminiero/webrtc-piano/dbcb9329d1626dfdf354355c1f7c4619a66bec07/images/image3.png -------------------------------------------------------------------------------- /web/css/demo.css: -------------------------------------------------------------------------------- 1 | .rounded { 2 | border-radius: 5px; 3 | } 4 | 5 | .centered { 6 | display: block; 7 | margin: auto; 8 | } 9 | 10 | .relative { 11 | position: relative; 12 | } 13 | 14 | .navbar-brand { 15 | margin-left: 0px !important; 16 | } 17 | 18 | .navbar-default { 19 | -webkit-box-shadow: 0px 3px 5px rgba(100, 100, 100, 0.49); 20 | -moz-box-shadow: 0px 3px 5px rgba(100, 100, 100, 0.49); 21 | box-shadow: 0px 3px 5px rgba(100, 100, 100, 0.49); 22 | } 23 | 24 | .navbar-header { 25 | padding-left: 40px; 26 | } 27 | 28 | .margin-sm { 29 | margin: 5px !important; 30 | } 31 | .margin-md { 32 | margin: 10px !important; 33 | } 34 | .margin-xl { 35 | margin: 20px !important; 36 | } 37 | .margin-bottom-sm { 38 | margin-bottom: 5px !important; 39 | } 40 | .margin-bottom-md { 41 | margin-bottom: 10px !important; 42 | } 43 | .margin-bottom-xl { 44 | margin-bottom: 20px !important; 45 | } 46 | 47 | .divider { 48 | width: 100%; 49 | text-align: center; 50 | } 51 | 52 | .divider hr { 53 | margin-left: auto; 54 | margin-right: auto; 55 | width: 45%; 56 | } 57 | 58 | .fa-2 { 59 | font-size: 2em !important; 60 | } 61 | .fa-3 { 62 | font-size: 4em !important; 63 | } 64 | .fa-4 { 65 | font-size: 7em !important; 66 | } 67 | .fa-5 { 68 | font-size: 12em !important; 69 | } 70 | .fa-6 { 71 | font-size: 20em !important; 72 | } 73 | 74 | div.no-video-container { 75 | position: relative; 76 | } 77 | 78 | .no-video-icon { 79 | width: 100%; 80 | height: 240px; 81 | text-align: center; 82 | } 83 | 84 | .no-video-text { 85 | text-align: center; 86 | position: absolute; 87 | bottom: 0px; 88 | right: 0px; 89 | left: 0px; 90 | font-size: 24px; 91 | } 92 | 93 | .meetecho-logo { 94 | padding: 12px !important; 95 | } 96 | 97 | .meetecho-logo > img { 98 | height: 26px; 99 | } 100 | 101 | pre { 102 | white-space: pre-wrap; 103 | white-space: -moz-pre-wrap; 104 | white-space: -pre-wrap; 105 | white-space: -o-pre-wrap; 106 | word-wrap: break-word; 107 | } 108 | 109 | .progress { 110 | position: relative; 111 | cursor: pointer; 112 | } 113 | 114 | .progress span { 115 | position: absolute; 116 | display: block; 117 | width: 100%; 118 | color: black; 119 | } 120 | 121 | textarea { 122 | font-family: monospace; 123 | background-color: black; 124 | color: white; 125 | font-size: large; 126 | width: 100%; 127 | } 128 | -------------------------------------------------------------------------------- /web/css/piano.css: -------------------------------------------------------------------------------- 1 | .piano { 2 | background: -webkit-linear-gradient(-65deg, #000, #222, #000, #666, #222 75%); 3 | background: -moz-linear-gradient(-65deg, #000, #222, #000, #666, #222 75%); 4 | background: -o-linear-gradient(-65deg, #000, #222, #000, #666, #222 75%); 5 | background: linear-gradient(-65deg, #000, #222, #000, #666, #222 75%); 6 | border-top: 2px solid #111; 7 | box-shadow: inset 0 -1px 1px rgba(255, 255, 255, 0.5), inset 0 -4px 5px #000; 8 | margin: 0; 9 | padding: 0 1% 3%; 10 | text-align: center; 11 | } 12 | .key { 13 | display: inline-block; 14 | position: relative; 15 | margin: 0 2px; 16 | width: 6%; 17 | max-width: 85px; 18 | } 19 | .key:active .black-key, 20 | .key.active .black-key { 21 | top: -5px; 22 | } 23 | .key .white-key { 24 | background: -webkit-linear-gradient(-30deg, #f8f8f8, #fff); 25 | background: -moz-linear-gradient(-30deg, #f8f8f8, #fff); 26 | background: -o-linear-gradient(-30deg, #f8f8f8, #fff); 27 | background: linear-gradient(-30deg, #f8f8f8, #fff); 28 | box-shadow: inset 0 1px 0px #ffffff, inset 0 -1px 0px #ffffff, inset 1px 0px 0px #ffffff, inset -1px 0px 0px #ffffff, 0 4px 3px rgba(0, 0, 0, 0.7), inset 0 -1px 0px #ffffff, inset 1px 0px 0px #ffffff, inset -1px -1px 15px rgba(0, 0, 0, 0.5), -3px 4px 6px rgba(0, 0, 0, 0.5); 29 | display: block; 30 | height: 300px; 31 | } 32 | .key .white-key:active, 33 | .key .white-key.active { 34 | box-shadow: inset 0 1px 0px #ffffff, inset 0 -1px 0px #ffffff, inset 1px 0px 0px #ffffff, inset -1px 0px 0px #ffffff, 0 4px 3px rgba(0, 0, 0, 0.7), inset 0 -1px 0px #ffffff, inset 1px 0px 0px #ffffff, inset -1px -1px 15px #000000, -3px 4px 6px rgba(0, 0, 0, 0.5); 35 | position: relative; 36 | top: -5px; 37 | height: 295px; 38 | } 39 | .key .black-key { 40 | content: ""; 41 | box-shadow: inset 0px -1px 2px rgba(255, 255, 255, 0.4), 0 2px 3px rgba(0, 0, 0, 0.4); 42 | background: -webkit-linear-gradient(-20deg, #222, #000, #222); 43 | background: -moz-linear-gradient(-20deg, #222, #000, #222); 44 | background: -o-linear-gradient(-20deg, #222, #000, #222); 45 | background: linear-gradient(-20deg, #222, #000, #222); 46 | border-width: 1px 3px 8px; 47 | border-style: solid; 48 | border-color: #666 #222 #111 #555; 49 | height: 160px; 50 | position: absolute; 51 | top: 0px; 52 | right: -40%; 53 | width: 70%; 54 | z-index: 10; 55 | } 56 | .key .black-key:active, 57 | .key .black-key.active { 58 | border-bottom-width: 3px; 59 | top: 0; 60 | } 61 | @media (max-width: 767px) { 62 | .key { 63 | width: 8%; 64 | } 65 | .key:nth-child(1), 66 | .key:nth-child(2), 67 | .key:nth-child(3) { 68 | display: none; 69 | } 70 | } 71 | @media (max-width: 480px) { 72 | .key { 73 | width: 12%; 74 | } 75 | .key:nth-child(11), 76 | .key:nth-child(12), 77 | .key:nth-child(13), 78 | .key:nth-child(14) { 79 | display: none; 80 | } 81 | } 82 | -------------------------------------------------------------------------------- /web/favicon.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lminiero/webrtc-piano/dbcb9329d1626dfdf354355c1f7c4619a66bec07/web/favicon.ico -------------------------------------------------------------------------------- /web/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | Janus WebRTC Piano Demo 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 47 | 48 |
49 |
50 |
51 | 56 |
57 |
58 |
59 |

Demo details

60 |

Originally written as a Dangerous Demo for ClueCon 2019 61 | a few months ago: having fun with WebRTC and music!

62 |

Press the Start button above to launch the demo.

63 |
64 |
65 |
66 |
67 | 68 |
69 | 70 |
71 |
72 |

Panel title

73 |
74 |
75 |
76 |
77 |
78 | 79 |
80 | 81 |
    82 |
  • 83 | 84 | 85 |
  • 86 |
  • 87 | 88 | 89 |
  • 90 |
  • 91 | 92 |
  • 93 |
  • 94 | 95 | 96 |
  • 97 |
  • 98 | 99 | 100 |
  • 101 |
  • 102 | 103 | 104 |
  • 105 |
  • 106 | 107 |
  • 108 |
  • 109 | 110 | 111 |
  • 112 |
  • 113 | 114 | 115 |
  • 116 |
  • 117 | 118 |
  • 119 |
  • 120 | 121 | 122 |
  • 123 |
  • 124 | 125 | 126 |
  • 127 |
  • 128 | 129 | 130 |
  • 131 |
  • 132 | 133 |
  • 134 |
135 | 136 |
137 | 138 |
139 |
140 |
141 | 142 |
143 | 146 |
147 | 148 | 149 | 150 | -------------------------------------------------------------------------------- /web/index.js: -------------------------------------------------------------------------------- 1 | // We make use of this 'server' variable to provide the address of the 2 | // REST Janus API. By default, in this example we assume that Janus is 3 | // co-located with the web server hosting the HTML pages but listening 4 | // on a different port (8088, the default for HTTP in Janus), which is 5 | // why we make use of the 'window.location.hostname' base address. Since 6 | // Janus can also do HTTPS, and considering we don't really want to make 7 | // use of HTTP for Janus if your demos are served on HTTPS, we also rely 8 | // on the 'window.location.protocol' prefix to build the variable, in 9 | // particular to also change the port used to contact Janus (8088 for 10 | // HTTP and 8089 for HTTPS, if enabled). 11 | // In case you place Janus behind an Apache frontend (as we did on the 12 | // online demos at http://janus.conf.meetecho.com) you can just use a 13 | // relative path for the variable, e.g.: 14 | // 15 | // var server = "/janus"; 16 | // 17 | // which will take care of this on its own. 18 | // 19 | // 20 | // If you want to use the WebSockets frontend to Janus, instead, you'll 21 | // have to pass a different kind of address, e.g.: 22 | // 23 | // var server = "ws://" + window.location.hostname + ":8188"; 24 | // 25 | // Of course this assumes that support for WebSockets has been built in 26 | // when compiling the server. WebSockets support has not been tested 27 | // as much as the REST API, so handle with care! 28 | // 29 | // 30 | // If you have multiple options available, and want to let the library 31 | // autodetect the best way to contact your server (or pool of servers), 32 | // you can also pass an array of servers, e.g., to provide alternative 33 | // means of access (e.g., try WebSockets first and, if that fails, fall 34 | // back to plain HTTP) or just have failover servers: 35 | // 36 | // var server = [ 37 | // "ws://" + window.location.hostname + ":8188", 38 | // "/janus" 39 | // ]; 40 | // 41 | // This will tell the library to try connecting to each of the servers 42 | // in the presented order. The first working server will be used for 43 | // the whole session. 44 | // 45 | var server = null; 46 | if(window.location.protocol === 'http:') 47 | server = "http://" + window.location.hostname + ":8088/janus"; 48 | else 49 | server = "https://" + window.location.hostname + ":8089/janus"; 50 | 51 | var janus = null; 52 | var streaming = null, controller = null; 53 | var opaqueId = "webrtc-piano-"+Janus.randomString(12); 54 | 55 | var myname = null, mycolor = createRandomColor(); 56 | 57 | var streamId = 2019; 58 | var notes = {}, bgs = {}; 59 | 60 | $(document).ready(function() { 61 | // Initialize the library (all console debuggers enabled) 62 | Janus.init({debug: "all", callback: function() { 63 | // Use a button to start the demo 64 | $('#start').one('click', function() { 65 | $(this).attr('disabled', true).unbind('click'); 66 | // Make sure the browser supports WebRTC 67 | if(!Janus.isWebrtcSupported()) { 68 | bootbox.alert("No WebRTC support... "); 69 | return; 70 | } 71 | // Create session 72 | janus = new Janus( 73 | { 74 | server: server, 75 | success: function() { 76 | // Prompt for a name 77 | askForName(); 78 | }, 79 | error: function(error) { 80 | Janus.error(error); 81 | bootbox.alert(error, function() { 82 | window.location.reload(); 83 | }); 84 | }, 85 | destroyed: function() { 86 | window.location.reload(); 87 | } 88 | }); 89 | }); 90 | }}); 91 | }); 92 | 93 | function askForName() { 94 | bootbox.prompt("What's your name?", function(result) { 95 | if(!result || result === "") { 96 | askForName(); 97 | return; 98 | } 99 | myname = result; 100 | // Create the MIDI controller first 101 | createController(); 102 | // Start the streaming mountpoint as well 103 | receiveAudio(); 104 | }); 105 | } 106 | 107 | function createController() { 108 | // Attach to the Lua plugin 109 | janus.attach( 110 | { 111 | plugin: "janus.plugin.lua", 112 | opaqueId: opaqueId, 113 | success: function(pluginHandle) { 114 | controller = pluginHandle; 115 | Janus.log("Plugin attached! (" + controller.getPlugin() + ", id=" + controller.getId() + ")"); 116 | // Setup the DataChannel 117 | var body = { request: "setup" }; 118 | Janus.debug("Sending message (" + JSON.stringify(body) + ")"); 119 | controller.send({ message: body }); 120 | }, 121 | error: function(error) { 122 | console.error(" -- Error attaching plugin...", error); 123 | bootbox.alert("Error attaching plugin... " + error); 124 | }, 125 | webrtcState: function(on) { 126 | Janus.log("Janus says our controller WebRTC PeerConnection is " + (on ? "up" : "down") + " now"); 127 | }, 128 | onmessage: function(msg, jsep) { 129 | Janus.debug(" ::: Got a message :::", msg); 130 | if(msg["error"]) { 131 | bootbox.alert(msg["error"]); 132 | } 133 | if(jsep) { 134 | // Answer 135 | controller.createAnswer( 136 | { 137 | jsep: jsep, 138 | media: { audio: false, video: false, data: true }, // We only use datachannels 139 | success: function(jsep) { 140 | Janus.debug("Got SDP!", jsep); 141 | var body = { request: "ack" }; 142 | controller.send({ message: body, jsep: jsep }); 143 | }, 144 | error: function(error) { 145 | Janus.error("WebRTC error:", error); 146 | bootbox.alert("WebRTC error... " + error.message); 147 | } 148 | }); 149 | } 150 | }, 151 | ondataopen: function(data) { 152 | Janus.log("The DataChannel is available!"); 153 | $('#details').remove(); 154 | $('#start').removeAttr('disabled').html("Stop") 155 | .click(function() { 156 | $(this).attr('disabled', true); 157 | janus.destroy(); 158 | }); 159 | // Register the name+color 160 | sendDataMessage({ action: "register", name: myname, color: mycolor }); 161 | // Show the piano 162 | $('#piano').removeClass('hide'); 163 | $('.white-key, .black-key') 164 | .on('mousedown', function(ev) { 165 | notes[ev.target.dataset.key] = true; 166 | sendDataMessage({ action: "play", note: parseInt(ev.target.dataset.key) }); 167 | }) 168 | .on('mouseup mouseleave', function(ev) { 169 | if(notes[ev.target.dataset.key] === true) { 170 | delete notes[ev.target.dataset.key]; 171 | sendDataMessage({ action: "stop", note: parseInt(ev.target.dataset.key) }); 172 | } 173 | }); 174 | }, 175 | ondata: function(data) { 176 | Janus.debug("We got data from the DataChannel!", data); 177 | handleResponse(JSON.parse(data)); 178 | }, 179 | oncleanup: function() { 180 | Janus.log(" ::: Got a cleanup notification :::"); 181 | } 182 | }); 183 | } 184 | 185 | function sendDataMessage(note) { 186 | controller.data({ 187 | text: JSON.stringify(note), 188 | error: function(reason) { 189 | bootbox.alert(reason); 190 | }, 191 | success: function() { 192 | // TODO 193 | } 194 | }); 195 | } 196 | 197 | function handleResponse(data) { 198 | Janus.debug(data); 199 | if(data["response"] === "error") { 200 | bootbox.alert(data["error"]); 201 | } else if(data["event"]) { 202 | if(data["event"] === "join") { 203 | var id = data["id"]; 204 | var name = data["name"]; 205 | var color = data["color"]; 206 | $('#players').append('

' + name + '

'); 207 | } else if(data["event"] === "leave") { 208 | var id = data["id"]; 209 | $('#p' + id).remove(); 210 | } else if(data["event"] === "play") { 211 | var note = data["note"]; 212 | var name = data["name"]; 213 | var color = data["color"]; 214 | if(!bgs[note]) 215 | bgs[note] = { original: $('[data-key=' + note + ']').css('background'), count: 0, colors: [] }; 216 | bgs[note].count++; 217 | bgs[note].colors.push(color); 218 | $('[data-key=' + note + ']').css('background', color); 219 | console.log(bgs); 220 | } else if(data["event"] === "stop") { 221 | var note = data["note"]; 222 | var name = data["name"]; 223 | if(!bgs[note]) 224 | return; 225 | var index = bgs[note].colors.indexOf(color); 226 | if(index > -1) 227 | bgs[note].colors.splice(index, 1); 228 | if(bgs[note].colors.length === 0) 229 | $('[data-key=' + note + ']').css('background', bgs[note].original); 230 | else 231 | $('[data-key=' + note + ']').css('background', bgs[note].colors[bgs[note].colors.length-1]); 232 | bgs[note].count--; 233 | if(bgs[note].count === 0) { 234 | $('[data-key=' + note + ']').css('background', bgs[note].original); 235 | delete bgs[note]; 236 | } 237 | console.log(bgs); 238 | } 239 | } 240 | } 241 | 242 | function receiveAudio() { 243 | // Attach to streaming plugin 244 | janus.attach( 245 | { 246 | plugin: "janus.plugin.streaming", 247 | opaqueId: opaqueId, 248 | success: function(pluginHandle) { 249 | streaming = pluginHandle; 250 | var body = { request: "watch", id: streamId }; 251 | streaming.send({ message: body }); 252 | }, 253 | error: function(error) { 254 | Janus.error(" -- Error attaching plugin... ", error); 255 | bootbox.alert("Error attaching plugin... " + error); 256 | }, 257 | webrtcState: function(on) { 258 | Janus.log("Janus says our streaming WebRTC PeerConnection is " + (on ? "up" : "down") + " now"); 259 | }, 260 | onmessage: function(msg, jsep) { 261 | Janus.debug(" ::: Got a message :::", msg); 262 | if(msg["error"]) { 263 | bootbox.alert(msg["error"]); 264 | return; 265 | } 266 | if(jsep) { 267 | Janus.debug("Handling SDP as well...", jsep); 268 | // Offer from the plugin, let's answer 269 | streaming.createAnswer( 270 | { 271 | jsep: jsep, 272 | // We only want recvonly audio 273 | media: { audioSend: false, videoSend: false, data: false }, 274 | success: function(jsep) { 275 | Janus.debug("Got SDP!"); 276 | Janus.debug(jsep); 277 | var body = { request: "start" }; 278 | streaming.send({ message: body, jsep: jsep }); 279 | }, 280 | error: function(error) { 281 | Janus.error("WebRTC error:", error); 282 | bootbox.alert("WebRTC error... " + error.message); 283 | } 284 | }); 285 | } 286 | }, 287 | onremotestream: function(stream) { 288 | Janus.debug(" ::: Got a remote stream :::"); 289 | Janus.debug(stream); 290 | if($('#remoteaudio').length === 0) { 291 | // Add the video element 292 | $('#piano').append('