├── .gitignore ├── LICENSE.txt ├── README.md ├── SCML.scd ├── WekinatorProjectFiles ├── VocWhistle_inpHelp.inputproj └── VocWhistle_ld-mfcc-pitch_1 │ ├── VocWhistle_ld-mfcc-pitch_1.wekproj │ ├── current │ ├── currentData.arff │ └── models │ │ ├── model0.xml │ │ ├── model1.xml │ │ ├── model2.xml │ │ └── model3.xml │ ├── inputConfig.xml │ └── outputConfig.xml └── dev └── SCML_dev.scd /.gitignore: -------------------------------------------------------------------------------- 1 | MachineListeningClassesOfInterest.scd 2 | MLstnEx-Loudness_1.scd 3 | MLstnEx-Onsets_1.scd 4 | NickCollinsMLcode 5 | Scratch 6 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | SCML: SuperCollider Machine Listening 2 | 3 | Copyright (c) 2016, Scott Tooby 4 | 5 | This program is free software: you can redistribute it and/or modify it under 6 | the terms of the GNU General Public License as published by the Free Software 7 | Foundation, either version 3 of the License, or (at your option) any later version. 8 | 9 | This program is distributed in the hope that it will be useful, but WITHOUT 10 | ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS 11 | FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. 12 | 13 | You should have received a copy of the GNU General Public License along with 14 | this program. If not, see . -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # SCML - SuperCollider Machine Listening 2 | --- 3 | * SCML is an audio feature extractor made with SuperCollider that converts audio signal feature data into OSC message output for interfacing with external programming environments or software. 4 | 5 | * This is intended to be a convenient toolbox of various SC machine listening UGens wrapped into SynthDefs that can be configured and triggered by the user to analyze audio signals in real-time. 6 | 7 | * Additionally, this is also a template for designing SuperCollider instruments that can be controlled via OSC input from external sources. Example instruments featured in this project are designed to be controlled via machine learning models created with [Wekinator](http://www.wekinator.org/). 8 | - For example, included is a prototype of a basic auto-harmonizer & auto-tuner instrument. [Video demo available here.](https://vimeo.com/185095490) 9 | - When used in conjunction with the included Wekinator project files, analyzed features of a mono audio signal input (singing & whistling voice) can be interpreted by machine learning algorithms implemented within Wekinator to modulate the auto-harmonizer & auto-tuner control parameters in real-time. 10 | - In this particular example, the timbre and pitch of streaming audio can modulate such parameters as: auto-harmonizer & auto-tuner level balancing, auto-harmonizer chord presets, auto-tuner pitch mapping, and reverb wetness. 11 | - Keep in mind the models contained in the Wekinator project file included with this project were all trained with my male tenor singing voice (w/ lots of falsetto training data) as well as whistling. If you cannot whistle or are not a male tenor, you may want to consider re-training the models to suit your purpose. 12 | 13 | 14 | ### What you will need: 15 | * [SuperCollider](http://supercollider.github.io/) 16 | * A microphone (a laptop’s built-in mic will do) 17 | 18 | If you want to run the machine-learning-controlled instrument examples: 19 | * [Wekinator](http://www.wekinator.org/downloads/) 20 | * [WekiInputHelper](http://www.wekinator.org/input-helper/) 21 | 22 | 23 | ### Operating Instructions: 24 | 1. Connect a microphone to your computer (recommended for higher quality sound, but a laptop’s internal mic will work fine too) 25 | 26 | 2. Open the "SCML.scd" file with SuperCollider and evaluate chunk #1 to boot the SC server using your system’s default audio input and outputs. 27 | - Alternatively (if you’re using an external mic), you may need to edit the “o.sampleRate,” “o.inDevice,” “o.outDevice,” and “s.latency” values first to configure the server’s I/O settings to suit your audio hardware setup. See the comments and SuperCollider’s help documentation for further assistance. 28 | 29 | * A. If you want to run the example instrument in conjunction with the Wekinator machine learning software: 30 | - RECOMMENDED: Monitor your system’s audio output with headphones to avoid audio feedback. 31 | 32 | A1. With the SC server booted, evaluate chunk #2. 33 | 34 | A2. Launch WekiInputHelper and open the “VocWhistle_inpHelp.inputproj” file included in the “WekinatorProjectFiles” folder. Click “Start Listening” and click on the “Send and Monitor” tab. 35 | 36 | A3. Launch Wekinator and open the “VocWhistle_ld-mfcc-pitch_1.wekproj” project file included in the “WekinatorProjectFiles” folder. Click “Start Listening” and “Run.” 37 | 38 | A4. Within the “SCML.scd” file in SuperCollider, evaluate item #3 to activate the auto-harmonizer & auto-tuner instrument example. 39 | 40 | - You should hear the auto-harmonizer processing your voice when you sing into the mic. Singing different vowels (AH, EH, EE, OH, OO) will change the harmonizer chord presets. 41 | 42 | - Whistling will switch the instrument into ‘auto-tune’ mode. 43 | 44 | A5. When you’re done, evaluate chunk #4 to shutdown this instrument example. (You can re-evaluate item #3 to restart the instrument after it’s been shutdown.) 45 | 46 | * B. If you don’t want to run the example instrument in conjunction with Wekinator, and simply want to output audio feature data via OSC: 47 | 48 | B1. Configure the OSC Output Host and Port address values in the first line of code at the top of chunk #2 to match the OSC input address and port of whatever you’re sending OSC data to. (Alternatively, leave this as is and configure the OSC input settings in your external programming environment / software to match these values). 49 | 50 | B2. Evaluate chunk #2. 51 | 52 | B3. Skip over chunks #3 and #4 and evaluate chunk “B” within the “Custom Configuration” portion of the code. This will activate input monitoring of your system’s first audio input channel as well as a variety of audio feature extractor “synths.” 53 | 54 | B4. Below chunk “B” within the “Output Audio Features via OSC” section, evaluate individual “TRANSMIT” chunks to output various combinations of audio feature data via OSC. Be sure to evaluate corresponding “STOP” chunks before attempting to transmit alternate feature sets. 55 | 56 | B5. When you’re finished, evaluate the “CLEANUP” chunk and shutdown the server before quitting SuperCollider. 57 | 58 | ### Notes about Wekinator and the included Wekinator project files: 59 | 60 | * [Wekinator](http://www.wekinator.org/) is a free, open source machine learning software created by [Rebecca Fiebrink](https://github.com/fiebrink1) that allows users to build interactive systems with minimal amounts of training data and without having to write any code. 61 | 62 | * SCML is designed to work with Wekinator to integrate machine learning as part of a larger instrument system. The included Wekinator project files have been trained to classify various sound events based on extracted audio feature data provided by SuperCollider. 63 | 64 | * The “VocWhistle_inpHelp.inputproj” file is a WekiInputHelper file designed to smooth the raw OSC output data from “SCML.scd” (SuperCollider) before passing it on to the “VocWhistle_ld-mfcc-pitch_1” Wekinator project file. Eventually this functionality will be implemented in the SCML code itself, but for now it serves as a convenient bridge. 65 | 66 | * Within the “VocWhistle_ld-mfcc-pitch_1” project file, several different machine learning algorithms are being implemented to classify audio input signals in terms of timbre and pitch: 67 | - Output 1: kNN algorithm (k = 1) classifies timbre (singing vs. whistling) based on 13 MFCC and 1 pitch tracker feature inputs (“singing detected” == 1, “whistling detected” == 2) 68 | 69 | - Output 2: neural network (1 hidden layer) derives a relative pitch measure respective of voice timbre from 13 MFCC and 1 pitch tracker feature inputs (lowest pitch range extremes of both singing and whistling voices == 0 and highest extremes == 1, with floating point interpolation between). 70 | 71 | - Output 3: kNN algorithm (k = 1) classifies five formants/vowels of a singing male tenor voice (AH == 1, EH == 2, EE == 3, OH == 4, OO ==5) using 13 MFCC feature inputs. 72 | 73 | - Output 4: neural network (1 hidden layer) with the same functionality as the ‘Output 3’ model detailed above. Using 13 MFCC feature inputs, the neural network identifies five different formants/vowels of a singing male tenor voice (1.0 - 5.0 | AH - OO). (This output was not used with this version of SCML, but it could be useful for controlling some other sound synthesis parameters in the future. At this point it mainly serves as a way of comparing the behavior of a neural network to a kNN algorithm using the same training data). 74 | 75 | * You may find you’ll need to add training data or delete all examples and re-train the models outright in the Wekinator project file so they better respond to your voice. [Here’s a detailed walkthrough](http://www.wekinator.org/walkthrough/) on how to do this. 76 | 77 | --- 78 | This is a work in progress. It will be enhanced in the future and eventually merged into the [Sonic Mirror](https://github.com/stooby/sonic-mirror) project. 79 | 80 | “SCML.scd” is the stable and more user-friendly version of this project. The ‘dev’ folder is where new features and experimental code will live before being merged into “SCML.scd” 81 | 82 | st, 2016 -------------------------------------------------------------------------------- /SCML.scd: -------------------------------------------------------------------------------- 1 | //************************ SCML (SuperCollider Machine Listening) ************************ 2 | 3 | //SEE THE README FOR OPERATING INSTRUCTIONS 4 | 5 | ( //#1 ======INITIALIZE SC SERVER ====== 6 | //if server fails to start and error encountered: try, try again...it's stubborn sometimes... 7 | o = Server.local.options; 8 | o.sampleRate = 44100; 9 | o.blockSize = 512; 10 | o.inDevice = nil; //use system default audio input (change value to manually configure input) 11 | o.outDevice = nil; //use system default audio output (change value to manually configure output) 12 | s.latency = 0.05; //0.2 is SC default 13 | s.boot; 14 | ) 15 | ServerOptions.devices; //evaluate this to see what audio input/output devices are available (if you want to customize your audio I/O configuration) 16 | 17 | ( //#2 ======SETUP====== 18 | 19 | //OSC In/Out ---------- 20 | ~oscOut_weki = NetAddr("127.0.0.1", 6448); //<---OSC OUTPUT HOST & PORT CONFIG SETTINGS (currently set to default Wekinator input port) 21 | 22 | ~oscIn_weki_1 = {|numIn, defKey, port = 12000| //<---OSC INPUT CONFIG SETTINGS (currently set to default Wekinator output port) 23 | var outBus; 24 | outBus = Bus.control(s, numIn); 25 | 26 | OSCdef.new(key: defKey.asSymbol, func: { |msg, time, addr, recvPort| 27 | var outArray = Array.newClear(msg.size - 1); //(# of Weki outputs - 1) b/c msg[0] is msg name 28 | for (1, msg.size - 1, {|i| outArray[i - 1] = msg[i];} ); //collect OSC msg vals into an array 29 | outBus.setn(outArray); 30 | //outBus.setnSynchronous(outArray); //would this yield better timing accuracy??? 31 | }, path: '/wek/outputs', recvPort: port); 32 | 33 | outBus; //return bus for receiving all msg vals from Weki OSC output 34 | }; 35 | 36 | //BUSSES----------- 37 | 38 | ~bus_input1 = Bus.audio(s, 1); 39 | ~bus_loudness = Bus.control(s, 1); 40 | ~bus_rms = Bus.control(s, 1); 41 | ~bus_mfcc = Bus.control(s, 13); 42 | ~bus_pitch = Bus.control(s, 2); 43 | ~bus_pitch2 = Bus.control(s, 3); 44 | ~bus_specCent = Bus.control(s, 1); 45 | ~bus_specFlat = Bus.control(s, 1); 46 | ~bus_onsets = Bus.control(s, 1); 47 | 48 | ~bus_dryIn1Amp = Bus.control(s, 1); //dry input level bus 49 | ~bus_vocAmp = Bus.control(s, 1); //voc proc synth sum level bus 50 | ~bus_wislAmp = Bus.control(s, 1); //whistle proc synth sum level bus 51 | ~bus_vocwislMix = Bus.control(s, 1); //mix balance between vocal FX synths and whistle FX synths 52 | ~bus_accompAmp = Bus.control(s, 1); //level of summed, mixed accomp synths 53 | ~bus_verbMix = Bus.control(s, 1); 54 | ~bus_verbRoom = Bus.control(s, 1); 55 | 56 | ~bus_vocSynthSum = Bus.audio(s, 2); 57 | ~bus_wislSynthSum = Bus.audio(s, 2); 58 | 59 | ~bus_dryIn1Amp.set(0.3); 60 | ~bus_accompAmp.set(1.0); 61 | ~bus_vocAmp.set(0.6); //two voc accomp synths, adjusting levels by ear and monitoring out meter 62 | ~bus_wislAmp.set(1.0); 63 | ~bus_vocwislMix.set(0.0); //0 = voc accomp only, 1 = whistle accomp only. Set via control routine 64 | ~bus_verbMix.set(0.3); 65 | ~bus_verbRoom.set(0.4); 66 | 67 | //SYNTH DEFS ---------- 68 | SynthDef.new(\monAudioInput, { 69 | arg inChannel = 0, amp = 1, outBus; 70 | var sig = SoundIn.ar(bus: inChannel, mul: amp); //defaults to first channel of system audio input 71 | OffsetOut.ar(outBus, sig); 72 | }).add; 73 | 74 | SynthDef.new(\Loudness, { 75 | //arg input, buffer, outBus; 76 | arg input, outBus; 77 | var sigIn, fft, loudness; 78 | sigIn = In.ar(input); 79 | //fft = FFT(buffer, sigIn, wintype: 1); 80 | fft = FFT(LocalBuf(1024), sigIn, wintype: 1); //using LocalBuf per HelpFile recommendation, 1024 frame size recommended for samp rates 44100 and 48000 and 2048 for rates 88200 and 96000 81 | loudness = Loudness.kr(fft); 82 | //loudness.poll(trig: 10, label: "Sones"); 83 | Out.kr(outBus, loudness); 84 | }).add; 85 | 86 | SynthDef.new(\RMS, { 87 | arg input, numSamp = 40, outBus; 88 | var sigIn, rms; 89 | sigIn = In.ar(input); 90 | rms = (RunningSum.kr(sigIn.squared, numSamp) / numSamp).sqrt; 91 | //rms.poll(trig: 10, label: "RMS"); 92 | Out.kr(outBus, rms); 93 | }).add; 94 | 95 | SynthDef.new(\Pitch, { 96 | arg input, ampThresh = 0.04, median = 7, minFreq = 60, maxFreq = 4000, outBus; 97 | var sigIn, freq, hasFreq; 98 | sigIn = In.ar(input); 99 | # freq, hasFreq = Pitch.kr(sigIn, ampThreshold: ampThresh, median: median, minFreq: minFreq, maxFreq: maxFreq); 100 | //freq = Lag.kr(freq.cpsmidi.round(1).midicps, 0.05); 101 | Out.kr(outBus, [freq, hasFreq]); 102 | //freq.poll(trig: 10, label: "Freq"); 103 | //hasFreq.poll(trig: 10, label: "HasFreq"); 104 | }).add; 105 | 106 | SynthDef.new(\Pitch2, { 107 | arg input, ampThresh = 0.04, median = 7, minFreq = 60, maxFreq = 4000, outBus; 108 | var sigIn, freq, hasFreq, midinote; 109 | sigIn = In.ar(input); 110 | # freq, hasFreq = Pitch.kr(sigIn, ampThreshold: ampThresh, median: median, minFreq: minFreq, maxFreq: maxFreq); 111 | //midinote = Lag.kr(freq.cpsmidi.round(1), 0.05); 112 | midinote = freq.cpsmidi.round(1); 113 | Out.kr(outBus, [freq, hasFreq, midinote]); 114 | //freq.poll(trig: 10, label: "Freq"); 115 | //hasFreq.poll(trig: 10, label: "HasFreq"); 116 | }).add; 117 | 118 | SynthDef.new(\MFCC, { 119 | //arg input, buffer, outBus; 120 | arg input, outBus; 121 | var sigIn, fft, array; 122 | sigIn = In.ar(input); 123 | //fft = FFT(buffer, sigIn, wintype: 1); 124 | fft = FFT(LocalBuf(1024), sigIn, wintype: 1); 125 | array = MFCC.kr(fft); //outputs 13 coefficients by default 126 | Out.kr(outBus, array); 127 | //array.poll(trig: 10, label: "MFCCs"); 128 | }).add; 129 | 130 | SynthDef.new(\specCent, { 131 | arg input, outBus; 132 | var sigIn, fft, centroid; 133 | sigIn = In.ar(input); 134 | fft = FFT(LocalBuf(2048), sigIn, wintype: 1); 135 | centroid = SpecCentroid.kr(fft); 136 | Out.kr(outBus, centroid); 137 | }).add; 138 | 139 | SynthDef.new(\specFlat, { 140 | arg input, outBus; 141 | var sigIn, fft, flatness, flatdb, flatdbScaled; 142 | sigIn = In.ar(input); 143 | fft = FFT(LocalBuf(2048), sigIn, wintype: 1); 144 | flatness = SpecFlatness.kr(fft); 145 | 146 | //flatdb = 10 * flat.log; //convert flatness to decibels 147 | //flatdbScaled = LinLin.kr(flatdb, -45, -1.6, 0, 1).max(-10); // Rescale db roughly to 0...1 148 | 149 | Out.kr(outBus, flatness); 150 | }).add; 151 | 152 | SynthDef.new(\onsets, { 153 | arg input, outBus, threshold = 0.5, trigtime = 0.1, odtype = \rcomplex, reltime = 1.0, floor = 0.1; 154 | var sigIn, fft, onsets, trigger; 155 | sigIn = In.ar(input); 156 | fft = FFT(LocalBuf(1024), sigIn, wintype: 1); 157 | onsets = Onsets.kr(fft, threshold, odtype, reltime, floor); 158 | trigger = Trig1.kr(onsets, trigtime); //if onset detected, send output 1 for trigtime seconds to control bus 159 | //trigger.poll(10, label: 'trig'); //uncomment this to monitor trigger val constantly 160 | Out.kr(outBus, trigger); 161 | }).add; 162 | 163 | //--- Synth Defs for Auto-Harmonizer-Tuner example instrument --- 164 | SynthDef.new(\mono2stereo, { //pass mono input to stereo output 165 | |in, amp = 1, pan = 0.0, out| 166 | var sig; 167 | sig = In.ar(in) * amp; 168 | //sig = InFeedback.ar(in) * amp; 169 | sig = Pan2.ar(sig, pan); 170 | //OffsetOut.ar(out, sig); 171 | Out.ar(out, sig); 172 | } ).add; 173 | 174 | SynthDef.new(\reverb2x2, {|outBus, mix = 0.25, room = 0.15, damp = 0.2, amp = 1.0| 175 | var sigIn, sigOut; 176 | sigIn = In.ar(outBus, 2); 177 | sigOut = FreeVerb2.ar(sigIn[0], sigIn[1], mix, room, damp, amp); 178 | ReplaceOut.ar(outBus, sigOut); 179 | } ).add; 180 | 181 | ~wekiSyn_voc1 = {//|oscInBus, audioIn, audioOut, interval = 5.0, pan = 0.0, /*scaleBuf,*/ amp = 1.0, mix = 1.0| //FUNCTION ARGS AREN'T BEING TRANSFERED TO SYNTHDEF ARGS !!!!! 182 | |oscInBus, audioIn, audioOut| 183 | var numCh, inMsg, sigIn, sigOut, returnSynth, pitchRatio, pitchDisp, panMod; 184 | 185 | numCh = oscInBus.numChannels; 186 | 187 | returnSynth = SynthDef(\vocHarm_weki1, {|interval = 5.0, pan = 0.0, amp = 0.5, mix = 1.0| 188 | sigIn = In.ar(audioIn, 1); 189 | inMsg = In.kr(oscInBus, numCh); 190 | 191 | pitchRatio = interval.asFloat.round(0.01).midiratio; //set pitch transposition (semi-tone intervals) 192 | //pitchDisp = LinLin.kr(inMsg[1], 0, 1, 0, 0.12).round(0.01); //scale weki out1 for pitchDisp val 193 | 194 | sigOut = SelectX.ar(mix, [sigIn, PitchShift.ar(sigIn, 0.1, pitchRatio, 0, 0.002)]); 195 | //sigOut = SelectX.ar(mix, [sigIn, PitchShift.ar(sigIn, 0.1, pitchRatio, pitchDisp, 0.002)]); 196 | 197 | //panMod = LinLin.kr(inMsg[1], 0, 1, -0.35, 0.35).round(0.01); //consider 198 | //pan = Clip.kr(pan.asFloat + panMod, -0.9, 0.9); //consider 199 | 200 | sigOut = Pan2.ar(sigOut, pan.asFloat); //?? yes? 201 | 202 | Out.ar(audioOut, sigOut * amp); 203 | }).play(addAction: \addToTail); 204 | 205 | returnSynth; 206 | }; 207 | 208 | ~wekiSyn_wisl1 = {//|oscInBus, audioIn, audioOut, scaleBuf, amp = 1.0, mix = 1.0| 209 | |oscInBus, audioIn, audioOut, scaleBuf| //add arg for initial interval scale spread of weki output1?? (12 == (-12, 12) | 5 == (-5, 5) | etc...) 210 | var numCh, inMsg, sigIn, sigOut, returnSynth, pitchRatio, pitchDisp; 211 | 212 | numCh = oscInBus.numChannels; 213 | 214 | returnSynth = SynthDef(\wislHarm_weki1, {|amp = 1.0, mix = 1.0| //can't use scaleBuf as arg for this synthDef :( 215 | sigIn = In.ar(audioIn, 1); 216 | inMsg = In.kr(oscInBus, numCh); 217 | 218 | pitchRatio = DegreeToKey.kr(scaleBuf.bufnum, LinLin.kr(inMsg[1], 0, 1, 5, -5).round(1)).midiratio; //convert Weki out 1 into a transposition ratio from mapped scale degrees 219 | //pitchRatio.poll(2); 220 | 221 | //pitchDisp = LinLin.kr(inMsg[1], 0, 1, 0, 0.12).round(0.01); //scale weki out1 for pitchDisp val 222 | 223 | sigOut = SelectX.ar(mix, [sigIn, PitchShift.ar(sigIn, 0.1, pitchRatio, 0, 0.002)]); //no pDisp 224 | //sigOut = SelectX.ar(mix, [sigIn, PitchShift.ar(sigIn, 0.1, pitchRatio, pitchDisp, 0.002)]); //pDisp 225 | 226 | sigOut = Pan2.ar(sigOut, 0); 227 | //sigOut = Pan2.ar(sigOut, LinLin.kr(inMsg[1], 0, 1, -0.35, 0.35)); //mod pan w/ weki out?? 228 | 229 | Out.ar(audioOut, sigOut * amp); 230 | }).play(addAction: \addToTail); 231 | 232 | returnSynth; 233 | }; 234 | 235 | SynthDef.new(\XFadeStereo, { 236 | |in1, in2, out, amountControlBus, amp = 1.0| 237 | var inArray, mixSignal, sig; 238 | inArray = [In.ar(in1, 2), In.ar(in2, 2)]; 239 | //inArray = [InFeedback.ar(in1), InFeedback.ar(in2)]; 240 | mixSignal = SelectX.ar(amountControlBus, inArray); 241 | sig = mixSignal * amp; 242 | //OffsetOut.ar(out, sig); //should I use this instead? 243 | Out.ar(out, sig); 244 | } ).add; 245 | 246 | SynthDef(\LinInterp_num, {//imported from SonicMirror 247 | arg startVal, stopVal, duration, mult = 1, out; 248 | Out.kr(out, Line.kr(startVal, stopVal, duration, mul: mult, doneAction: 2)); 249 | } ).add; 250 | 251 | SynthDef.new(\subMix, {//imported from SonicMirror 252 | |in, amp = 1, out| 253 | var sig; 254 | sig = In.ar(in) * amp; 255 | //sig = InFeedback.ar(in) * amp; 256 | //OffsetOut.ar(out, sig); 257 | Out.ar(out, sig); 258 | } ).add; 259 | 260 | //FUNCS------------------------ 261 | ~sumFunc_xin = {arg busArray; //bus val concat function for variable size arg array of feat busses 262 | var numCh = 0, busCol, outArray, outBus; 263 | busArray.do({|item| numCh = numCh + item.numChannels}); //calculate total # bus channels 264 | outBus = Bus.control(s, numCh); 265 | busCol = Array.newClear(busArray.size); 266 | 267 | ~sumSyn_xin = SynthDef(\sumSyn_xin, { 268 | busArray.do({|item, i| busCol[i] = In.kr(item, item.numChannels); }); //input bus vals to var array 269 | busCol.do({|item| //concat all bus vals into single out array 270 | if (item.numChannels == 1, 271 | {outArray = outArray ++ [item]}, 272 | {outArray = outArray ++ (item.numChannels.collect( {|i| item[i]} ))} 273 | ); 274 | }); 275 | Out.kr(outBus, outArray); 276 | }).play(addAction: \addToTail); 277 | 278 | outBus; //return new bus w/ concat values of all input busses in arg array 279 | }; 280 | 281 | ~getSend_busSum = {|bus, netAdr, oscMsg, post = 0| 282 | var numCh = bus.numChannels; //needed for message posting... 283 | bus.get( {arg val; { //<---changing to getSynchronous prevents OSC output for some reason...? 284 | netAdr.sendMsg(*[oscMsg] ++ val); //transmit feature vals via OSC 285 | if (post == 1, { //monitor feature values in SC post window 286 | switch (bus, 287 | ~busSum_ldns_mfcc, { 288 | ("Loudness:" + val[0].round(0.0001)).postln; 289 | ("MFCC:" + val[1..13].round(0.0001)).postln; 290 | ("-------").postln; 291 | }, 292 | ~busSum_ldns_sFlat_sCent, { 293 | ("Loudness:" + val[0].round(0.0001)).postln; 294 | ("Flatness:" + val[1].round(0.0001)).postln; 295 | ("Centroid:" + val[2].round(0.0001)).postln; 296 | ("-------").postln; 297 | }, 298 | ~busSum_ldns_pitch, { 299 | ("Loudness:" + val[0].round(0.0001)).postln; 300 | ("Pitch-freq:" + val[1].round(0.01)).postln; 301 | ("Pitch-hasFreq?:" + val[2]).postln; 302 | ("Pitch-MIDInote:" + val[3]).postln; 303 | ("-------").postln; 304 | }, 305 | ~busSum_ldns_sFlat_onset, { 306 | ("Loudness:" + val[0].round(0.0001)).postln; 307 | ("Flatness:" + val[1].round(0.0001)).postln; 308 | ("Onset Detected:" + val[2]).postln; 309 | ("-------").postln; 310 | }, 311 | {//all other busses (cases) 312 | if (numCh > 1, 313 | { 314 | numCh.do({arg item; ("Feature_" ++ (item + 1) ++ ":" + val[item].round(0.0001)).postln;}); 315 | ("-------").postln; 316 | }, 317 | {("Feature_1:" + val.round(0.0001)).postln;} //else 318 | ); 319 | } 320 | ); 321 | } ); 322 | }.defer; } ); 323 | }; 324 | 325 | ~streamFeatures = { 326 | |bus, netAdr, oscMsg, rate = 0.04, post = 0| 327 | Routine ( { 328 | ~getSend_busSum.value(bus, netAdr, oscMsg, post); 329 | rate.yieldAndReset; 330 | } ); 331 | }; 332 | 333 | //===========ACCOMP SYNTH AND FX CONTROL ROUTINE=========== 334 | ~flag_wekSyn_prevIn = 1; //langside flag storing last input from WekiOut1, 1 == voc, 2 == whistle 335 | ~flag_wekSyn_prevIn_change = false; //flag for debouncing sudden changes due to input noise 336 | ~flag_wekSyn_prevVow = 1; //flag for storing last input from WekiOut3, 1 == ah, 2 == eh, 3 == ee, 4 == oh, 5 == oo 337 | ~flag_wekSyn_prevVow_change = false; //flag for debouncing sudden changes due to input noise 338 | ~yieldTime_wsc1 = 0.1; 339 | ~debounceTime = 0.3; //(in seconds) increase or reduce this value for less or more jittery parameter change response, respectively (higher value causes more delay in parameter changes after detected changes in sound, but may yield more accurate performance...the accuracy of trained Wekinator models is also a factor) 340 | 341 | ~routine_wekiSynthControl1 = {|oscInBus, rate = 0.1| //<---new version w/ debounce for vowel detection 342 | Routine ({ 343 | oscInBus.get({ |msg| 344 | if (msg[0] == 1, { //"voc detected" 345 | if (~flag_wekSyn_prevIn == 1, { //sound source has not changed since prev bus.get 346 | var x; 347 | ~flag_wekSyn_prevIn_change = false; //reset debounce flag in case it was previously triggered 348 | //"msg[0] = 1 - NO CHANGE".postln; 349 | 350 | x = case //MONITOR CHANGES IN msg[2] <<<<<<<< 351 | {msg[2] == 1} {//if msg[2] == 1 - 'AH' DETECTED 352 | if (~flag_wekSyn_prevVow == 1, { //msg[2] has not changed since prev bus.get 353 | ~flag_wekSyn_prevVow_change = false; //reset flag if it was triggered by noise 354 | ~yieldTime_wsc1 = rate.asFloat; //<--- 355 | } , { //else if ~flag_prevVow != 1, maybe we're changing vowels 356 | if (~flag_wekSyn_prevVow_change == false, { //DEBOUNCE 357 | ~flag_wekSyn_prevVow_change = true; 358 | //"msg[2] = 1 | prev msg[2] = ? - STATE CHANGE TEST".postln; 359 | ~yieldTime_wsc1 = ~debounceTime; 360 | }, { //else if ~flag_stateChange == true, state has changed and remained same long enough to rule out noise, and we can proceed with triggered changes 361 | ~flag_wekSyn_prevVow_change = false; 362 | ~flag_wekSyn_prevVow = 1; //set flag to remember last state before resetting routine 363 | //"msg[2] = 1 | prev msg[2] = ? - STATE CHANGE CONFIRMED".postln; 364 | "SINGING DETECTED: AH | msg[2] = 1".postln; 365 | ~vocA1.set(\interval, -5, \pan, -0.75); 366 | ~vocA2.set(\interval, 4.25, \pan, 0.75); 367 | ~yieldTime_wsc1 = rate.asFloat; //<--- 368 | } ); 369 | } ); 370 | } 371 | {msg[2] == 2} {//if msg[2] == 2 - 'EH' DETECTED 372 | if (~flag_wekSyn_prevVow == 2, { //msg[2] has not changed since prev bus.get 373 | ~flag_wekSyn_prevVow_change = false; //reset flag if it was triggered by noise 374 | ~yieldTime_wsc1 = rate.asFloat; //<--- 375 | } , { //else if ~flag_prevVow != 2, maybe we're changing vowels 376 | if (~flag_wekSyn_prevVow_change == false, { //DEBOUNCE 377 | ~flag_wekSyn_prevVow_change = true; 378 | //"msg[2] = 2 | prev msg[2] = ? - STATE CHANGE TEST".postln; 379 | ~yieldTime_wsc1 = ~debounceTime; 380 | }, { //else if ~flag_stateChange == true, state has changed and remained same long enough to rule out noise, and we can proceed with triggered changes 381 | ~flag_wekSyn_prevVow_change = false; 382 | ~flag_wekSyn_prevVow = 2; //set flag to remember last state before resetting routine 383 | //"msg[2] = 2 | prev msg[2] = ? - STATE CHANGE CONFIRMED".postln; 384 | "SINGING DETECTED: EH | msg[2] = 2".postln; 385 | ~vocA1.set(\interval, -3, \pan, -0.6); 386 | ~vocA2.set(\interval, 5, \pan, 0.6); 387 | ~yieldTime_wsc1 = rate.asFloat; //<--- 388 | } ); 389 | } ); 390 | } 391 | {msg[2] == 3} {//if msg[2] == 3 - 'EE' 392 | if (~flag_wekSyn_prevVow == 3, { //msg[2] has not changed since prev bus.get 393 | ~flag_wekSyn_prevVow_change = false; //reset flag if it was triggered by noise 394 | ~yieldTime_wsc1 = rate.asFloat; //<--- 395 | } , { //else if ~flag_prevVow != 3, maybe we're changing vowels 396 | if (~flag_wekSyn_prevVow_change == false, { //DEBOUNCE 397 | ~flag_wekSyn_prevVow_change = true; 398 | //"msg[2] = 3 | prev msg[2] = ? - STATE CHANGE TEST".postln; 399 | ~yieldTime_wsc1 = ~debounceTime; 400 | }, { //else if ~flag_stateChange == true, state has changed and remained same long enough to rule out noise, and we can proceed with triggered changes 401 | ~flag_wekSyn_prevVow_change = false; 402 | ~flag_wekSyn_prevVow = 3; //set flag to remember last state before resetting routine 403 | //"msg[2] = 3 | prev msg[2] = ? - STATE CHANGE CONFIRMED".postln; 404 | "SINGING DETECTED: EE | msg[2] = 3".postln; 405 | ~vocA1.set(\interval, -4, \pan, -0.45); 406 | ~vocA2.set(\interval, 10, \pan, 0.45); 407 | ~yieldTime_wsc1 = rate.asFloat; //<--- 408 | } ); 409 | } ); 410 | } 411 | {msg[2] == 4} {//if msg[2] == 4 - 'OH' 412 | if (~flag_wekSyn_prevVow == 4, { //msg[2] has not changed since prev bus.get 413 | ~flag_wekSyn_prevVow_change = false; //reset flag if it was triggered by noise 414 | ~yieldTime_wsc1 = rate.asFloat; //<--- 415 | } , { //else if ~flag_prevVow != 4, maybe we're changing vowels 416 | if (~flag_wekSyn_prevVow_change == false, { //DEBOUNCE 417 | ~flag_wekSyn_prevVow_change = true; 418 | //"msg[2] = 4 | prev msg[2] = ? - STATE CHANGE TEST".postln; 419 | ~yieldTime_wsc1 = ~debounceTime; 420 | }, { //else if ~flag_stateChange == true, state has changed and remained same long enough to rule out noise, and we can proceed with triggered changes 421 | ~flag_wekSyn_prevVow_change = false; 422 | ~flag_wekSyn_prevVow = 4; //set flag to remember last state before resetting routine 423 | //"msg[2] = 4 | prev msg[2] = ? - STATE CHANGE CONFIRMED".postln; 424 | "SINGING DETECTED: OH | msg[2] = 4".postln; 425 | ~vocA1.set(\interval, -3, \pan, -0.6); 426 | ~vocA2.set(\interval, 7, \pan, 0.6); 427 | ~yieldTime_wsc1 = rate.asFloat; //<--- 428 | } ); 429 | } ); 430 | } 431 | {msg[2] == 5} {//if msg[2] == 5 'OO' 432 | if (~flag_wekSyn_prevVow == 5, { //msg[2] has not changed since prev bus.get 433 | ~flag_wekSyn_prevVow_change = false; //reset flag if it was triggered by noise 434 | ~yieldTime_wsc1 = rate.asFloat; //<--- 435 | } , { //else if ~flag_prevVow != 5, maybe we're changing vowels 436 | if (~flag_wekSyn_prevVow_change == false, { //DEBOUNCE 437 | ~flag_wekSyn_prevVow_change = true; 438 | //"msg[2] = 5 | prev msg[2] = ? - STATE CHANGE TEST".postln; 439 | ~yieldTime_wsc1 = ~debounceTime; 440 | }, { //else if ~flag_stateChange == true, state has changed and remained same long enough to rule out noise, and we can proceed with triggered changes 441 | ~flag_wekSyn_prevVow_change = false; 442 | ~flag_wekSyn_prevVow = 5; //set flag to remember last state before resetting routine 443 | //"msg[2] = 5 | prev msg[2] = ? - STATE CHANGE CONFIRMED".postln; 444 | "SINGING DETECTED: OO | msg[2] = 5".postln; 445 | ~vocA1.set(\interval, -4, \pan, -0.25); 446 | ~vocA2.set(\interval, 5, \pan, 0.25); 447 | ~yieldTime_wsc1 = rate.asFloat; //<--- 448 | } ); 449 | } ); 450 | }; 451 | //~yieldTime_wsc1 = rate.asFloat; //<--- 452 | } , { //else ~flag_wekSyn_prevIn == 2, we're changing from "2 - whistling" to "1 - voc" 453 | //CHECK TO SEE IF TIME THRESHOLD SURPASSED BEFORE CONFIRMING STATE CHANGE 454 | if (~flag_wekSyn_prevIn_change == false, { 455 | ~flag_wekSyn_prevIn_change = true; 456 | //"msg[0] = 1 | prev msg[0] = 2 - STATE CHANGE TEST".postln; 457 | ~yieldTime_wsc1 = ~debounceTime; //"debounce" time 458 | }, { //else if ~flag_stateChange == true, state has changed and remained same long enough to rule out noise, and we can proceed with triggered changes 459 | var interpSynth, interpVerbRoom, interpVerbMix; 460 | ~flag_wekSyn_prevIn_change = false; 461 | //XFADE ~mixAmount.bus from whistle to voc processing synth busses (1.0 - 0.0) 462 | interpSynth = Synth(\LinInterp_num, [\startVal, 1, \stopVal, 0, \duration, 0.3, \out, ~bus_vocwislMix]); 463 | interpVerbRoom = Synth(\LinInterp_num, [\startVal, 0.7, \stopVal, 0.4, \duration, 0.3, \out, ~bus_verbRoom]); 464 | interpVerbMix = Synth(\LinInterp_num, [\startVal, 0.6, \stopVal, 0.3, \duration, 0.3, \out, ~bus_verbMix]); 465 | ~flag_wekSyn_prevIn = 1; //set flag to remember last state before resetting routine 466 | //"msg[0] = 1 | prev msg[0] = 1 - STATE CHANGE CONFIRMED".postln; 467 | "SINGING DETECTED--AUTO-HARMONIZE MODE ACTIVE".postln; 468 | ~yieldTime_wsc1 = rate.asFloat; 469 | //~yieldTime_wsc1 = 0.31; //precautionary yield time to allow LinInterp synth to finish before resuming routine...thought this helped avoid audio clicks, but don't think it does & it adds latency.... 470 | } ); 471 | } ); 472 | } , { //ELSE======= if msg[0] == 2 "whistling detected" 473 | if (~flag_wekSyn_prevIn == 2, { //sound source has not changed since prev bus.get 474 | ~flag_wekSyn_prevIn_change = false; //reset debounce flag 475 | //"msg[0] = 2 - NO CHANGE".postln; 476 | ~yieldTime_wsc1 = rate.asFloat; //<---- 477 | } , { //else flag must have changed recently from "1 - voc" to "2 - whistling" 478 | //CHECK TO SEE IF TIME THRESHOLD SURPASSED BEFORE CONFIRMING STATE CHANGE 479 | if (~flag_wekSyn_prevIn_change == false, { 480 | ~flag_wekSyn_prevIn_change = true; 481 | //"msg[0] = 2 | prev msg[0] = 1 - STATE CHANGE TEST".postln; 482 | ~yieldTime_wsc1 = ~debounceTime; //"debounce" time <--- 483 | }, { //else if ~flag_stateChange == true, state has changed and remained same long enough to rule out noise, and we can proceed with triggered changes 484 | var interpSynth, interpVerbRoom, interpVerbMix; 485 | ~flag_wekSyn_prevIn_change = false; 486 | //XFADE ~mixAmount.bus from voc to whistle processing synth busses (0.0 - 1.0) 487 | interpSynth = Synth(\LinInterp_num, [\startVal, 0, \stopVal, 1, \duration, 0.3, \out, ~bus_vocwislMix]); 488 | interpVerbRoom = Synth(\LinInterp_num, [\startVal, 0.4, \stopVal, 0.7, \duration, 0.3, \out, ~bus_verbRoom]); 489 | interpVerbMix = Synth(\LinInterp_num, [\startVal, 0.3, \stopVal, 0.6, \duration, 0.3, \out, ~bus_verbMix]); 490 | ~flag_wekSyn_prevIn = 2; //set flag to remember last state before resetting routine 491 | //"msg[0] = 2 | prev msg[0] = 2 - STATE CHANGE CONFIRMED".postln; 492 | "WHISTLING DETECTED--AUTO-TUNE MODE ACTIVE".postln; 493 | ~yieldTime_wsc1 = rate.asFloat; //<--- 494 | //~yieldTime_wsc1 = 0.31; //precautionary yield time to allow LinInterp synth to finish before resuming routine...thought this helped avoid audio clicks, but don't think it does & it adds latency.... 495 | } ); 496 | } ); 497 | } ); 498 | }); 499 | //"Out of .get, before slight .yield".postln; 500 | 0.02.yield; //<-----ANY OTHER WAY W/OUT INDUCING LATENCY!?????? 501 | // ("yieldTime_wsc1 = " ++ ~yieldTime_wsc1).postln; 502 | ~yieldTime_wsc1.yieldAndReset; 503 | }); 504 | }; 505 | 506 | ~initAutoHT = Routine({ //===initialize supporting synths and OSC In/Out functions for SC-Wekinator Auto-Harmonizer-Tuner example instrument=== 507 | ~scale1 = Scale.majorPentatonic.degrees; //[0, 2, 4, 7, 9] 508 | ~scaleBuf = Buffer.alloc(s, ~scale1.size, 1, {|b| b.setnMsg(0, ~scale1) } ); 509 | 510 | i = Synth(\monAudioInput, [\outBus, ~bus_input1]); 511 | l = Synth(\Loudness, [\input, ~bus_input1, \outBus, ~bus_loudness]).moveAfter(i); 512 | m = Synth(\MFCC, [\input, ~bus_input1, \outBus, ~bus_mfcc]).moveAfter(i); 513 | p = Synth(\Pitch, [\input, ~bus_input1, \outBus, ~bus_pitch]).moveAfter(i); //outputs two values (freq) 514 | 515 | //"AUDIO IN & FEATURE EXTRACTOR SYNTHS ACTIVE...".postln; 516 | 517 | //0.15.wait; 518 | 519 | //TRANSMIT FEATURES 520 | ~busSum_xin = ~sumFunc_xin.value([~bus_loudness, ~bus_mfcc, ~bus_pitch]); //collect loudness, mfcc, and pitch features to output via OSC to Wekinator 521 | ~transmit = ~streamFeatures.value(~busSum_xin, ~oscOut_weki, "/wek/inputs", 0.08); 522 | SystemClock.play(~transmit); 523 | //"TRANSMITTING FEATURES (OSC OUT)...".postln; 524 | 525 | //0.15.wait; 526 | 527 | //RECEIVE WEKI OSC OUTPUT 528 | ~bus_oscIn_1 = ~oscIn_weki_1.value(numIn: 4, defKey: "oscDef1", port: 12000); //open port to receive OSC output from Wekinator and send to a control bus 529 | 530 | //"OSC-IN Port Ready...".postln; 531 | 532 | 0.15.wait; //voodoo wait... not necessary, but including this seems to prevent audio clicks 533 | 534 | ~dryIn1 = Synth(\mono2stereo, [\in, ~bus_input1, \amp, ~bus_dryIn1Amp.asMap, \out, 0]).moveAfter(i); //pipe dry mono input to stereo output 535 | 536 | ~wisA1 = ~wekiSyn_wisl1.value(oscInBus: ~bus_oscIn_1, audioIn: ~bus_input1, audioOut: ~bus_wislSynthSum, scaleBuf: ~scaleBuf).moveAfter(~dryIn1); //whistle harmonizer 1 537 | 538 | ~vocA1 = ~wekiSyn_voc1.value(oscInBus: ~bus_oscIn_1, audioIn: ~bus_input1, audioOut: ~bus_vocSynthSum).moveAfter(~dryIn1); //voc harmonizer 1 539 | 540 | ~vocA2 = ~wekiSyn_voc1.value(oscInBus: ~bus_oscIn_1, audioIn: ~bus_input1, audioOut: ~bus_vocSynthSum).moveAfter(~dryIn1); //voc harmonizer 2 541 | 542 | //"AutoHarmonizer and AutoTune Synths Active...".postln; 543 | 0.15.wait; //necessary wait 544 | 545 | ~dryIn1.set(\amp, ~bus_dryIn1Amp.asMap); 546 | ~wisA1.set(\amp, ~bus_wislAmp.asMap); 547 | ~vocA1.set(\amp, ~bus_vocAmp.asMap); 548 | ~vocA2.set(\amp, ~bus_vocAmp.asMap); 549 | //"Configure Synths...".postln; 550 | 551 | 0.15.wait; //voodoo wait... not necessary, but including this seems to prevent audio clicks 552 | 553 | ~accomp_mix = Synth(\XFadeStereo, [\in1, ~bus_vocSynthSum, \in2, ~bus_wislSynthSum, \out, 0, \amountControlBus, ~bus_vocwislMix.asMap, \amp, ~bus_accompAmp.asMap], addAction: \addToTail); 554 | 555 | ~fx_reverb = Synth(\reverb2x2, [\outBus, 0, \mix, ~bus_verbMix.asMap, \room, ~bus_verbRoom.asMap], addAction: \addToTail); //stereo reverb processing sum output 556 | 557 | //--Activate OSC input streaming-- 558 | ~wekiSynthControl1 = ~routine_wekiSynthControl1.value(oscInBus: ~bus_oscIn_1, rate: 0.1); //create routine to get latest values from OSC input control bus 559 | SystemClock.play(~wekiSynthControl1); //start OSC input bus monitoring routine 560 | //"OSC Input Streaming...".postln; 561 | "--AUTOHARMTUNE ACTIVE--".postln; 562 | }); 563 | ) 564 | 565 | // #3 ====== ACTIVATE WEKINATOR INSTRUMENT EXAMPLE (Auto-harmonize & Auto-tune) ====== 566 | SystemClock.play(~initAutoHT); 567 | 568 | // #4 ==== STOP & CLEANUP WEKINATOR INSTRUMENT EXAMPLE (Auto-harmonize & Auto-tune) ==== 569 | ( 570 | ~initAutoHT.stop; 571 | ~initAutoHT.reset; 572 | ~transmit.stop; 573 | ~transmit.free; 574 | ~sumSyn_xin.free; 575 | ~busSum_xin.free; 576 | 577 | ~scaleBuf.free; 578 | i.free; 579 | l.free; 580 | m.free; 581 | p.free; 582 | ~dryIn1.free; 583 | ~wisA1.free; 584 | ~vocA1.free; 585 | ~vocA2.free; 586 | ~accomp_mix.free; 587 | ~fx_reverb.free; 588 | 589 | ~bus_oscIn_1.free; 590 | ~wekiSynthControl1.stop; 591 | ~wekiSynthControl1.free; 592 | OSCdef(\oscDef1).free; 593 | ) 594 | 595 | ///////////////////////////// --- CUSTOM CONFIGURATION --- /////////////////////////////////// 596 | 597 | 598 | (// B. ====== ACTIVATE AUDIO INPUT AND FEATURE EXTRACTOR SYNTHS ====== 599 | 600 | i = Synth(\monAudioInput, [\outBus, ~bus_input1]); //stream your system's first audio input channel to an SC audio bus (NOTE: THIS SYNTH MUST BE ACTIVE BEFORE ENABLING ANY OTHER FEATURE EXTRACTOR SYNTH BELOW) 601 | 602 | //LOUDNESS (volume - perceptual measure in Sones) ------ 603 | l = Synth(\Loudness, [\input, ~bus_input1, \outBus, ~bus_loudness]).moveAfter(i); 604 | 605 | //RMS (volume) ------ 606 | r = Synth(\RMS, [\input, ~bus_input1, \numSamp, 40, \outBus, ~bus_rms]).moveAfter(i); 607 | 608 | //MFCC 609 | m = Synth(\MFCC, [\input, ~bus_input1, \outBus, ~bus_mfcc]).moveAfter(i); 610 | 611 | //PITCH TRACKERS 612 | p = Synth(\Pitch, [\input, ~bus_input1, \outBus, ~bus_pitch]).moveAfter(i); //outputs two values (freq, hasPitch) 613 | 614 | ~p2 = Synth(\Pitch2, [\input, ~bus_input1, \outBus, ~bus_pitch2]).moveAfter(i); //3 output vals (freq, hasPitch, MIDInote) 615 | 616 | //SPECTRAL CENTROID 617 | c = Synth(\specCent, [\input, ~bus_input1, \outBus, ~bus_specCent]).moveAfter(i); 618 | 619 | //SPECTRAL FLATNESS 620 | f = Synth(\specFlat, [\input, ~bus_input1, \outBus, ~bus_specFlat]).moveAfter(i); 621 | 622 | //ONSETS 623 | n = Synth(\onsets, [\input, ~bus_input1, \outBus, ~bus_onsets, \threshold, 0.7]).moveAfter(i); 624 | //n.set(\threshold, 0.5); //uncomment evaluate to change onset detection threshold (between 0 - 1) 625 | ) 626 | 627 | // ====== OUTPUT AUDIO FEATURES VIA OSC ====== 628 | 629 | // Evaluate one chunk below at a time to output different feature sets via OSC. 630 | // Evaluate the corresponding STOP chunk before attempting to transmit alternate feature combinations. 631 | 632 | // NOTE: You may need to edit the "netAdr" and "oscMsg" args suited to your needs if you're trying to output OSC messages to something other than the Wekinator application, which is currently the default configuration of this code. 633 | 634 | // Default OSC output (~oscOut_weki): IP - 127.0.0.1, Port # 6448 635 | 636 | (//TRANSMIT LOUDNESS AND MFCCs - (14 features) 637 | ~busSum_ldns_mfcc = ~sumFunc_xin.value( [~bus_loudness, ~bus_mfcc] ); 638 | ~transmit = ~streamFeatures.value(bus: ~busSum_ldns_mfcc, netAdr: ~oscOut_weki, oscMsg: "/wek/inputs", rate: 0.1, post: 1); //rate arg adjusts rate OSC messages are output (0.04 or about 25 times/sec is default) 639 | SystemClock.play(~transmit); 640 | ) 641 | ( //STOP transmitting (NOTE: evaluate this chunk before transmitting other feature sets) 642 | ~transmit.stop; 643 | ~sumSyn_xin.free; 644 | ~busSum_ldns_mfcc.free; 645 | ) 646 | 647 | 648 | (//TRANSMIT LOUDNESS, SPECTRAL FLATNESS, AND SPECTRAL CENTROID - (3 features) 649 | ~busSum_ldns_sFlat_sCent = ~sumFunc_xin.value( [~bus_loudness, ~bus_specFlat, ~bus_specCent] ); 650 | ~transmit = ~streamFeatures.value(bus: ~busSum_ldns_sFlat_sCent, netAdr: ~oscOut_weki, oscMsg: "/wek/inputs", rate: 0.1, post: 1); 651 | SystemClock.play(~transmit); 652 | ) 653 | ( //STOP transmitting 654 | ~transmit.stop; 655 | ~sumSyn_xin.free; 656 | ~busSum_ldns_sFlat_sCent.free; 657 | ) 658 | 659 | 660 | (//TRANSMIT LOUDNESS AND PITCH 661 | //~busSum_ldns_pitch = ~sumFunc_xin.value( [~bus_loudness, ~bus_pitch] ); //3 features [loudness, freq, hasFreq] 662 | ~busSum_ldns_pitch = ~sumFunc_xin.value( [~bus_loudness, ~bus_pitch2] ); //4 features [loudness, freq, hasFreq, MIDI note] 663 | ~transmit = ~streamFeatures.value(bus: ~busSum_ldns_pitch, netAdr: ~oscOut_weki, oscMsg: "/wek/inputs", rate: 0.1, post: 1); 664 | SystemClock.play(~transmit); 665 | ) 666 | ( //STOP transmitting 667 | ~transmit.stop; 668 | ~sumSyn_xin.free; 669 | ~busSum_ldns_pitch.free; 670 | ) 671 | 672 | 673 | (//TRANSMIT LOUDNESS, SPECTRAL FLATNESS, ONSETS - (3 features) 674 | ~busSum_ldns_sFlat_onset = ~sumFunc_xin.value( [~bus_loudness, ~bus_specFlat, ~bus_onsets] ); 675 | ~transmit = ~streamFeatures.value(bus: ~busSum_ldns_sFlat_onset, netAdr: ~oscOut_weki, oscMsg: "/wek/inputs", rate: 0.1, post: 1); 676 | ~transmit.play; 677 | ) 678 | (//STOP transmitting 679 | ~transmit.stop; 680 | ~sumSyn_xin.free; 681 | ~busSum_ldns_sFlat_onset.free; 682 | ) 683 | 684 | (//TRANSMIT FEATURES (MISC FEATURE SET) - Loudness, Flatness, Pitch 685 | ~busSum_xin = ~sumFunc_xin.value([~bus_loudness, ~bus_specFlat, ~bus_pitch]); //input desired feature busses in array (separate w/ comma) 686 | ~transmit = ~streamFeatures.value(bus: ~busSum_xin, netAdr: ~oscOut_weki, oscMsg: "/wek/inputs", rate: 0.1, post: 1); 687 | SystemClock.play(~transmit); 688 | ) 689 | ( //STOP transmitting 690 | ~transmit.stop; 691 | ~sumSyn_xin.free; 692 | ~busSum_xin.free; 693 | ) 694 | 695 | 696 | // ======CLEANUP====== 697 | //evaluate below code before quitting SuperCollider 698 | ( 699 | s.freeAll; //free all active synths on server 700 | 701 | ~bus_input1.free; 702 | ~bus_loudness.free; 703 | ~bus_rms.free; 704 | ~bus_mfcc.free; 705 | ~bus_pitch.free; 706 | ~bus_pitch2.free; 707 | ~bus_specCent.free; 708 | ~bus_specFlat.free; 709 | ~bus_onsets.free; 710 | 711 | ~oscOut_weki.disconnect; 712 | OSCdef.freeAll; 713 | ) 714 | 715 | s.quit; //shutdown the SC server -------------------------------------------------------------------------------- /WekinatorProjectFiles/VocWhistle_inpHelp.inputproj: -------------------------------------------------------------------------------- 1 | 2 | 3 | 6448 4 | 6449 5 | localhost 6 | /wek/inputs 7 | 8 | /wek/inputs 9 | 16 10 | Inputs 11 | 12 | inputs_1 13 | inputs_2 14 | inputs_3 15 | inputs_4 16 | inputs_5 17 | inputs_6 18 | inputs_7 19 | inputs_8 20 | inputs_9 21 | inputs_10 22 | inputs_11 23 | inputs_12 24 | inputs_13 25 | inputs_14 26 | inputs_15 27 | inputs_16 28 | 29 | 30 | 31 | 32 | 33 | inputs_1_Avg10 34 | 0 35 | 10 36 | 37 | 38 | 39 | inputs_2_Avg10 40 | 1 41 | 10 42 | 43 | 44 | 45 | inputs_3_Avg10 46 | 2 47 | 10 48 | 49 | 50 | 51 | inputs_4_Avg10 52 | 3 53 | 10 54 | 55 | 56 | 57 | inputs_5_Avg10 58 | 4 59 | 10 60 | 61 | 62 | 63 | inputs_6_Avg10 64 | 5 65 | 10 66 | 67 | 68 | 69 | inputs_7_Avg10 70 | 6 71 | 10 72 | 73 | 74 | 75 | inputs_8_Avg10 76 | 7 77 | 10 78 | 79 | 80 | 81 | inputs_9_Avg10 82 | 8 83 | 10 84 | 85 | 86 | 87 | inputs_10_Avg10 88 | 9 89 | 10 90 | 91 | 92 | 93 | inputs_11_Avg10 94 | 10 95 | 10 96 | 97 | 98 | 99 | inputs_12_Avg10 100 | 11 101 | 10 102 | 103 | 104 | 105 | inputs_13_Avg10 106 | 12 107 | 10 108 | 109 | 110 | 111 | inputs_14_Avg10 112 | 13 113 | 10 114 | 115 | 116 | 117 | inputs_15_Avg10 118 | 14 119 | 10 120 | 121 | 122 | 123 | 15 124 | 15 125 | false 126 | 127 | 128 | 129 | GREATER_THAN 130 | REPEAT 131 | 15 132 | 0.0 133 | INPUT 134 | 135 | 136 | false 137 | -------------------------------------------------------------------------------- /WekinatorProjectFiles/VocWhistle_ld-mfcc-pitch_1/VocWhistle_ld-mfcc-pitch_1.wekproj: -------------------------------------------------------------------------------- 1 | 2 | 3 | 6449 4 | VocWhistle_ld-mfcc-pitch_1 5 | -------------------------------------------------------------------------------- /WekinatorProjectFiles/VocWhistle_ld-mfcc-pitch_1/current/models/model0.xml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/stooby/SCML/3120c5cc68818474fc77b35de68663ff15fbcc05/WekinatorProjectFiles/VocWhistle_ld-mfcc-pitch_1/current/models/model0.xml -------------------------------------------------------------------------------- /WekinatorProjectFiles/VocWhistle_ld-mfcc-pitch_1/current/models/model1.xml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/stooby/SCML/3120c5cc68818474fc77b35de68663ff15fbcc05/WekinatorProjectFiles/VocWhistle_ld-mfcc-pitch_1/current/models/model1.xml -------------------------------------------------------------------------------- /WekinatorProjectFiles/VocWhistle_ld-mfcc-pitch_1/current/models/model2.xml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/stooby/SCML/3120c5cc68818474fc77b35de68663ff15fbcc05/WekinatorProjectFiles/VocWhistle_ld-mfcc-pitch_1/current/models/model2.xml -------------------------------------------------------------------------------- /WekinatorProjectFiles/VocWhistle_ld-mfcc-pitch_1/current/models/model3.xml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/stooby/SCML/3120c5cc68818474fc77b35de68663ff15fbcc05/WekinatorProjectFiles/VocWhistle_ld-mfcc-pitch_1/current/models/model3.xml -------------------------------------------------------------------------------- /WekinatorProjectFiles/VocWhistle_ld-mfcc-pitch_1/inputConfig.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | /wek/inputs 4 | 15 5 | Inputs 6 | 7 | inputs-1 8 | inputs-2 9 | inputs-3 10 | inputs-4 11 | inputs-5 12 | inputs-6 13 | inputs-7 14 | inputs-8 15 | inputs-9 16 | inputs-10 17 | inputs-11 18 | inputs-12 19 | inputs-13 20 | inputs-14 21 | inputs-15 22 | 23 | -------------------------------------------------------------------------------- /WekinatorProjectFiles/VocWhistle_ld-mfcc-pitch_1/outputConfig.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | /wek/outputs 4 | localhost 5 | 12000 6 | 7 | 8 | outputs-1 9 | 2 10 | false 11 | 12 | 13 | outputs-2 14 | 0.0 15 | 1.0 16 | REAL 17 | HARD 18 | ENFORCE_LIMITS 19 | DONT_ENFORCE_INTEGER 20 | 21 | 22 | outputs-3 23 | 5 24 | false 25 | 26 | 27 | outputs-4 28 | 1.0 29 | 5.0 30 | REAL 31 | HARD 32 | ENFORCE_LIMITS 33 | DONT_ENFORCE_INTEGER 34 | 35 | 36 | -------------------------------------------------------------------------------- /dev/SCML_dev.scd: -------------------------------------------------------------------------------- 1 | //************************ SCML (SuperCollider Machine Listening)_dev ************************ 2 | //new features and experimental code goes here before being merged into "SCML" 3 | 4 | ( //#1 ======INITIALIZE SC SERVER ====== 5 | o = Server.local.options; 6 | //o.sampleRate = 44100; 7 | o.sampleRate = 48000; 8 | o.blockSize = 512; 9 | o.inDevice = "iConnectAudio4+"; //use system default audio input (change value to manually configure input) 10 | o.outDevice = "iConnectAudio4+"; //use system default audio output (change value to manually configure output) 11 | s.latency = 0.05; //0.2 is SC default 12 | s.boot; 13 | ) 14 | ServerOptions.devices; //evaluate this to see what audio input/output devices are available (if you want to customize your audio I/O configuration) 15 | 16 | ( //#2 ======SETUP====== 17 | 18 | //OSC In/Out ---------- 19 | ~oscOut_weki = NetAddr("127.0.0.1", 6448); //<---OSC OUTPUT HOST & PORT CONFIG SETTINGS (currently set to default Wekinator input port) 20 | 21 | ~oscIn_weki_1 = {|numIn, defKey, port = 12000| //<---OSC INPUT CONFIG SETTINGS (currently set to default Wekinator output port) 22 | var outBus; 23 | outBus = Bus.control(s, numIn); 24 | 25 | OSCdef.new(key: defKey.asSymbol, func: { |msg, time, addr, recvPort| 26 | var outArray = Array.newClear(msg.size - 1); //(# of Weki outputs - 1) b/c msg[0] is msg name 27 | for (1, msg.size - 1, {|i| outArray[i - 1] = msg[i];} ); //collect OSC msg vals into an array 28 | outBus.setn(outArray); 29 | //outBus.setnSynchronous(outArray); //would this yield better timing accuracy??? 30 | }, path: '/wek/outputs', recvPort: port); 31 | 32 | outBus; //return bus for receiving all msg vals from Weki OSC output 33 | }; 34 | 35 | //================= NEW VOC/WHISTLE ACCOMP INST CODE =================// 36 | 37 | //-- Accompaniment (Auto-Harmonizer & Auto-tuner) synth level control implementation -- 38 | //a. Control synth group volumes w/ Line.kr sends to control busses (~bus_wislAmp, etc.) 39 | //b. connect synth outputs to respective sub busses (~bus_vocSynthSum, etc...) 40 | //c. Use \XFadeStereo synthDef w/ SelectX ugen, and an additional control bus to control balance between voc and whistle accomp synths being sent to final output (e.g. if ~bus_vocwislMix == 0.0, only voc accomp synths heard, if ~bus_vocwislMix == 1.0, only whistle synth(s) heard 41 | 42 | //BUSSES----------- 43 | 44 | ~bus_input1 = Bus.audio(s, 1); 45 | ~bus_loudness = Bus.control(s, 1); 46 | ~bus_rms = Bus.control(s, 1); 47 | ~bus_mfcc = Bus.control(s, 13); 48 | ~bus_pitch = Bus.control(s, 2); 49 | ~bus_pitch2 = Bus.control(s, 3); 50 | ~bus_specCent = Bus.control(s, 1); 51 | ~bus_specFlat = Bus.control(s, 1); 52 | ~bus_onsets = Bus.control(s, 1); 53 | 54 | ~bus_dryIn1Amp = Bus.control(s, 1); //dry input level bus 55 | ~bus_vocAmp = Bus.control(s, 1); //voc proc synth sum level bus 56 | ~bus_wislAmp = Bus.control(s, 1); //whistle proc synth sum level bus 57 | ~bus_vocwislMix = Bus.control(s, 1); //mix balance between vocal FX synths and whistle FX synths 58 | ~bus_accompAmp = Bus.control(s, 1); //level of summed, mixed accomp synths 59 | ~bus_verbMix = Bus.control(s, 1); 60 | ~bus_verbRoom = Bus.control(s, 1); 61 | 62 | ~bus_vocSynthSum = Bus.audio(s, 2); 63 | ~bus_wislSynthSum = Bus.audio(s, 2); 64 | 65 | ~bus_dryIn1Amp.set(0.3); 66 | ~bus_accompAmp.set(1.0); 67 | ~bus_vocAmp.set(0.6); //two voc accomp synths, adjusting levels by ear and monitoring out meter 68 | ~bus_wislAmp.set(1.0); 69 | ~bus_vocwislMix.set(0.0); //0 = voc accomp only, 1 = whistle accomp only. Set via control routine 70 | ~bus_verbMix.set(0.3); 71 | ~bus_verbRoom.set(0.4); 72 | 73 | //SYNTH DEFS ---------- 74 | SynthDef.new(\mono2stereo, { //pass mono input to stereo output 75 | |in, amp = 1, pan = 0.0, out| 76 | var sig; 77 | sig = In.ar(in) * amp; 78 | //sig = InFeedback.ar(in) * amp; 79 | sig = Pan2.ar(sig, pan); 80 | //OffsetOut.ar(out, sig); 81 | Out.ar(out, sig); 82 | } ).add; 83 | 84 | SynthDef.new(\reverb2x2, {|outBus, mix = 0.25, room = 0.15, damp = 0.2, amp = 1.0| 85 | var sigIn, sigOut; 86 | sigIn = In.ar(outBus, 2); 87 | sigOut = FreeVerb2.ar(sigIn[0], sigIn[1], mix, room, damp, amp); 88 | ReplaceOut.ar(outBus, sigOut); 89 | } ).add; 90 | 91 | ~wekiSyn_voc1 = {//|oscInBus, audioIn, audioOut, interval = 5.0, pan = 0.0, /*scaleBuf,*/ amp = 1.0, mix = 1.0| //FUNCTION ARGS AREN'T BEING TRANSFERED TO SYNTHDEF ARGS !!!!! 92 | |oscInBus, audioIn, audioOut| 93 | var numCh, inMsg, sigIn, sigOut, returnSynth, pitchRatio, pitchDisp, panMod; 94 | 95 | numCh = oscInBus.numChannels; 96 | 97 | returnSynth = SynthDef(\vocHarm_weki1, {|interval = 5.0, pan = 0.0, amp = 0.5, mix = 1.0| 98 | sigIn = In.ar(audioIn, 1); 99 | inMsg = In.kr(oscInBus, numCh); 100 | 101 | pitchRatio = interval.asFloat.round(0.01).midiratio; //set pitch transposition (semi-tone intervals) 102 | //pitchDisp = LinLin.kr(inMsg[1], 0, 1, 0, 0.12).round(0.01); //scale weki out1 for pitchDisp val 103 | 104 | sigOut = SelectX.ar(mix, [sigIn, PitchShift.ar(sigIn, 0.1, pitchRatio, 0, 0.002)]); 105 | //sigOut = SelectX.ar(mix, [sigIn, PitchShift.ar(sigIn, 0.1, pitchRatio, pitchDisp, 0.002)]); 106 | 107 | //panMod = LinLin.kr(inMsg[1], 0, 1, -0.35, 0.35).round(0.01); //consider 108 | //pan = Clip.kr(pan.asFloat + panMod, -0.9, 0.9); //consider 109 | 110 | sigOut = Pan2.ar(sigOut, pan.asFloat); //?? yes? 111 | 112 | Out.ar(audioOut, sigOut * amp); 113 | }).play(addAction: \addToTail); 114 | 115 | returnSynth; 116 | }; 117 | 118 | ~wekiSyn_voc1b = {|oscInBus, audioIn, audioOut, interval = 5.0, pan = 0.0, amp = 1.0| //testing...don't use... 119 | var numCh, inMsg, sigIn, sigOut, returnSynth, pitchRatio, pitchDisp, panMod; 120 | 121 | numCh = oscInBus.numChannels; 122 | 123 | returnSynth = SynthDef(\vocHarm_weki1, {|interval, pan, amp, mix = 1.0| 124 | sigIn = In.ar(audioIn, 1); 125 | inMsg = In.kr(oscInBus, numCh); 126 | 127 | pitchRatio = interval.asFloat.round(0.01).midiratio; //set pitch transposition (semi-tone intervals) 128 | //pitchDisp = LinLin.kr(inMsg[1], 0, 1, 0, 0.12).round(0.01); //scale weki out1 for pitchDisp val 129 | 130 | sigOut = SelectX.ar(mix, [sigIn, PitchShift.ar(sigIn, 0.1, pitchRatio, 0, 0.002)]); 131 | //sigOut = SelectX.ar(mix, [sigIn, PitchShift.ar(sigIn, 0.1, pitchRatio, pitchDisp, 0.002)]); 132 | 133 | //panMod = LinLin.kr(inMsg[1], 0, 1, -0.35, 0.35).round(0.01); //consider 134 | //pan = Clip.kr(pan.asFloat + panMod, -0.9, 0.9); //consider 135 | 136 | sigOut = Pan2.ar(sigOut, pan.asFloat); //?? yes? 137 | 138 | Out.ar(audioOut, sigOut * amp); 139 | }).play(addAction: \addToTail); 140 | 141 | returnSynth; 142 | }; 143 | 144 | ~wekiSyn_wisl1 = {//|oscInBus, audioIn, audioOut, scaleBuf, amp = 1.0, mix = 1.0| 145 | |oscInBus, audioIn, audioOut, scaleBuf| //add arg for initial interval scale spread of weki output1?? (12 == (-12, 12) | 5 == (-5, 5) | etc...) 146 | var numCh, inMsg, sigIn, sigOut, returnSynth, pitchRatio, pitchDisp; 147 | 148 | numCh = oscInBus.numChannels; 149 | 150 | returnSynth = SynthDef(\wislHarm_weki1, {|amp = 1.0, mix = 1.0| //can't use scaleBuf as arg for this synthDef :( 151 | sigIn = In.ar(audioIn, 1); 152 | inMsg = In.kr(oscInBus, numCh); 153 | 154 | pitchRatio = DegreeToKey.kr(scaleBuf.bufnum, LinLin.kr(inMsg[1], 0, 1, 5, -5).round(1)).midiratio; //convert Weki out 1 into a transposition ratio from mapped scale degrees 155 | //pitchRatio.poll(2); 156 | 157 | //pitchDisp = LinLin.kr(inMsg[1], 0, 1, 0, 0.12).round(0.01); //scale weki out1 for pitchDisp val 158 | 159 | sigOut = SelectX.ar(mix, [sigIn, PitchShift.ar(sigIn, 0.1, pitchRatio, 0, 0.002)]); //no pDisp 160 | //sigOut = SelectX.ar(mix, [sigIn, PitchShift.ar(sigIn, 0.1, pitchRatio, pitchDisp, 0.002)]); //pDisp 161 | 162 | sigOut = Pan2.ar(sigOut, 0); 163 | //sigOut = Pan2.ar(sigOut, LinLin.kr(inMsg[1], 0, 1, -0.35, 0.35)); //mod pan w/ weki out?? 164 | 165 | Out.ar(audioOut, sigOut * amp); 166 | }).play(addAction: \addToTail); 167 | 168 | returnSynth; 169 | }; 170 | 171 | SynthDef.new(\XFadeStereo, { 172 | |in1, in2, out, amountControlBus, amp = 1.0| 173 | var inArray, mixSignal, sig; 174 | inArray = [In.ar(in1, 2), In.ar(in2, 2)]; 175 | //inArray = [InFeedback.ar(in1), InFeedback.ar(in2)]; 176 | mixSignal = SelectX.ar(amountControlBus, inArray); 177 | sig = mixSignal * amp; 178 | //OffsetOut.ar(out, sig); //should I use this instead? 179 | Out.ar(out, sig); 180 | } ).add; 181 | 182 | SynthDef(\LinInterp_num, {//imported from SonicMirror 183 | arg startVal, stopVal, duration, mult = 1, out; 184 | Out.kr(out, Line.kr(startVal, stopVal, duration, mul: mult, doneAction: 2)); 185 | } ).add; 186 | 187 | SynthDef.new(\subMix, {//imported from SonicMirror 188 | |in, amp = 1, out| 189 | var sig; 190 | sig = In.ar(in) * amp; 191 | //sig = InFeedback.ar(in) * amp; 192 | //OffsetOut.ar(out, sig); 193 | Out.ar(out, sig); 194 | } ).add; 195 | 196 | //---ORIG SYNTH DEFS from SCML --- 197 | 198 | SynthDef.new(\monAudioInput, { 199 | arg inChannel = 0, amp = 1, outBus; 200 | var sig = SoundIn.ar(bus: inChannel, mul: amp); //defaults to first channel of system audio input 201 | OffsetOut.ar(outBus, sig); 202 | }).add; 203 | 204 | SynthDef.new(\monAudioInFFT, {//considering this as a way to use only one buffer to store FFT analysis, and have other feature extraction synths receive input from said FFT buffer. Could be useful to increase CPU efficiency, but helpfile advises against this... 205 | arg inChannel = 0, amp = 1, buffer, outBus; 206 | var sig, fft; 207 | sig = SoundIn.ar(bus: inChannel, mul: amp); 208 | fft = FFT(buffer, sig, wintype: 1); //<--and then how should other synths access this fft buffer??? 209 | OffsetOut.ar(outBus, sig); 210 | }).add; 211 | 212 | SynthDef.new(\Loudness, { 213 | //arg input, buffer, outBus; 214 | arg input, outBus; 215 | var sigIn, fft, loudness; 216 | sigIn = In.ar(input); 217 | //fft = FFT(buffer, sigIn, wintype: 1); 218 | fft = FFT(LocalBuf(1024), sigIn, wintype: 1); //using LocalBuf per HelpFile recommendation, 1024 frame size recommended for samp rates 44100 and 48000 and 2048 for rates 88200 and 96000 219 | loudness = Loudness.kr(fft); 220 | //loudness.poll(trig: 10, label: "Sones"); 221 | Out.kr(outBus, loudness); 222 | }).add; 223 | 224 | SynthDef.new(\RMS, { 225 | arg input, numSamp = 40, outBus; 226 | var sigIn, rms; 227 | sigIn = In.ar(input); 228 | rms = (RunningSum.kr(sigIn.squared, numSamp) / numSamp).sqrt; 229 | //rms.poll(trig: 10, label: "RMS"); 230 | Out.kr(outBus, rms); 231 | }).add; 232 | 233 | SynthDef.new(\Pitch, { 234 | arg input, ampThresh = 0.04, median = 7, minFreq = 60, maxFreq = 4000, outBus; 235 | var sigIn, freq, hasFreq; 236 | sigIn = In.ar(input); 237 | # freq, hasFreq = Pitch.kr(sigIn, ampThreshold: ampThresh, median: median, minFreq: minFreq, maxFreq: maxFreq); 238 | //freq = Lag.kr(freq.cpsmidi.round(1).midicps, 0.05); 239 | Out.kr(outBus, [freq, hasFreq]); 240 | //freq.poll(trig: 10, label: "Freq"); 241 | //hasFreq.poll(trig: 10, label: "HasFreq"); 242 | }).add; 243 | 244 | SynthDef.new(\Pitch2, { 245 | arg input, ampThresh = 0.04, median = 7, minFreq = 60, maxFreq = 4000, outBus; 246 | var sigIn, freq, hasFreq, midinote; 247 | sigIn = In.ar(input); 248 | # freq, hasFreq = Pitch.kr(sigIn, ampThreshold: ampThresh, median: median, minFreq: minFreq, maxFreq: maxFreq); 249 | //midinote = Lag.kr(freq.cpsmidi.round(1), 0.05); 250 | midinote = freq.cpsmidi.round(1); 251 | Out.kr(outBus, [freq, hasFreq, midinote]); 252 | //freq.poll(trig: 10, label: "Freq"); 253 | //hasFreq.poll(trig: 10, label: "HasFreq"); 254 | }).add; 255 | 256 | SynthDef.new(\MFCC, { 257 | //arg input, buffer, outBus; 258 | arg input, outBus; 259 | var sigIn, fft, array; 260 | sigIn = In.ar(input); 261 | //fft = FFT(buffer, sigIn, wintype: 1); 262 | fft = FFT(LocalBuf(1024), sigIn, wintype: 1); 263 | array = MFCC.kr(fft); //outputs 13 coefficients by default 264 | Out.kr(outBus, array); 265 | //array.poll(trig: 10, label: "MFCCs"); 266 | }).add; 267 | 268 | SynthDef.new(\specCent, { 269 | arg input, outBus; 270 | var sigIn, fft, centroid; 271 | sigIn = In.ar(input); 272 | fft = FFT(LocalBuf(2048), sigIn, wintype: 1); 273 | centroid = SpecCentroid.kr(fft); 274 | Out.kr(outBus, centroid); 275 | }).add; 276 | 277 | SynthDef.new(\specFlat, { 278 | arg input, outBus; 279 | var sigIn, fft, flatness, flatdb, flatdbScaled; 280 | sigIn = In.ar(input); 281 | fft = FFT(LocalBuf(2048), sigIn, wintype: 1); 282 | flatness = SpecFlatness.kr(fft); 283 | 284 | //flatdb = 10 * flat.log; //convert flatness to decibels 285 | //flatdbScaled = LinLin.kr(flatdb, -45, -1.6, 0, 1).max(-10); // Rescale db roughly to 0...1 286 | 287 | Out.kr(outBus, flatness); 288 | }).add; 289 | 290 | SynthDef.new(\onsets, { 291 | arg input, outBus, threshold = 0.5, trigtime = 0.1, odtype = \rcomplex, reltime = 1.0, floor = 0.1; 292 | var sigIn, fft, onsets, trigger; 293 | sigIn = In.ar(input); 294 | fft = FFT(LocalBuf(1024), sigIn, wintype: 1); 295 | onsets = Onsets.kr(fft, threshold, odtype, reltime, floor); 296 | trigger = Trig1.kr(onsets, trigtime); //if onset detected, send output 1 for trigtime seconds to control bus 297 | //trigger.poll(10, label: 'trig'); //uncomment this to monitor trigger val constantly 298 | Out.kr(outBus, trigger); 299 | }).add; 300 | 301 | //FUNCS------------------------ 302 | ~sumFunc_xin = {arg busArray; //bus val concat function for variable size arg array of feat busses 303 | var numCh = 0, busCol, outArray, outBus; 304 | busArray.do({|item| numCh = numCh + item.numChannels}); //calculate total # bus channels 305 | outBus = Bus.control(s, numCh); 306 | busCol = Array.newClear(busArray.size); 307 | 308 | ~sumSyn_xin = SynthDef(\sumSyn_xin, { //<----is it possible to dynamically generate synths that are freed when this function is terminated w/out assigning to environment variable??? 309 | busArray.do({|item, i| busCol[i] = In.kr(item, item.numChannels); }); //input bus vals to var array 310 | busCol.do({|item| //concat all bus vals into single out array 311 | if (item.numChannels == 1, 312 | {outArray = outArray ++ [item]}, 313 | {outArray = outArray ++ (item.numChannels.collect( {|i| item[i]} ))} 314 | ); 315 | }); 316 | Out.kr(outBus, outArray); 317 | }).play(addAction: \addToTail); 318 | 319 | outBus; //return new bus w/ concat values of all input busses in arg array 320 | }; 321 | 322 | ~getSend_busSum = {|bus, netAdr, oscMsg, post = 0| 323 | var numCh = bus.numChannels; //needed for message posting... 324 | bus.get( {arg val; { //<---changing to getSynchronous prevents OSC output for some reason...? 325 | netAdr.sendMsg(*[oscMsg] ++ val); //transmit feature vals via OSC 326 | if (post == 1, { //monitor feature values in SC post window 327 | switch (bus, 328 | ~busSum_ldns_mfcc, { 329 | ("Loudness:" + val[0].round(0.0001)).postln; 330 | ("MFCC:" + val[1..13].round(0.0001)).postln; 331 | ("-------").postln; 332 | }, 333 | ~busSum_ldns_sFlat_sCent, { 334 | ("Loudness:" + val[0].round(0.0001)).postln; 335 | ("Flatness:" + val[1].round(0.0001)).postln; 336 | ("Centroid:" + val[2].round(0.0001)).postln; 337 | ("-------").postln; 338 | }, 339 | ~busSum_ldns_pitch, { 340 | ("Loudness:" + val[0].round(0.0001)).postln; 341 | ("Pitch-freq:" + val[1].round(0.01)).postln; 342 | ("Pitch-hasFreq?:" + val[2]).postln; 343 | ("Pitch-MIDInote:" + val[3]).postln; 344 | ("-------").postln; 345 | }, 346 | ~busSum_ldns_sFlat_onset, { 347 | ("Loudness:" + val[0].round(0.0001)).postln; 348 | ("Flatness:" + val[1].round(0.0001)).postln; 349 | ("Onset Detected:" + val[2]).postln; 350 | ("-------").postln; 351 | }, 352 | {//all other busses (cases) 353 | if (numCh > 1, 354 | { 355 | numCh.do({arg item; ("Feature_" ++ (item + 1) ++ ":" + val[item].round(0.0001)).postln;}); 356 | ("-------").postln; 357 | }, 358 | {("Feature_1:" + val.round(0.0001)).postln;} //else 359 | ); 360 | } 361 | ); 362 | } ); 363 | }.defer; } ); 364 | }; 365 | 366 | ~streamFeatures = { 367 | |bus, netAdr, oscMsg, rate = 0.04, post = 0| 368 | Routine ( { 369 | ~getSend_busSum.value(bus, netAdr, oscMsg, post); 370 | rate.yieldAndReset; 371 | } ); 372 | }; 373 | 374 | ~transmitFeatures = { //works, but not sure if busses are being permanently allocated? 375 | arg busOut, busInArray, netAdr, oscMsg, rate = 0.04, act = true; 376 | if (act == true, 377 | { 378 | busOut = ~sumFunc_xin.value(busInArray); 379 | ~transmit = ~streamFeatures.value(busOut, netAdr, oscMsg, rate); 380 | ~transmit.play; 381 | }, 382 | {//else 383 | ~transmit.stop; 384 | ~sumSyn_xin.free; 385 | busOut.free; 386 | } 387 | ); 388 | }; 389 | 390 | //===========ACCOMP SYNTH AND FX CONTROL ROUTINE=========== 391 | ~flag_wekSyn_prevIn = 1; //langside flag storing last input from WekiOut1, 1 == voc, 2 == whistle 392 | ~flag_wekSyn_prevIn_change = false; //flag for debouncing sudden changes due to input noise 393 | ~flag_wekSyn_prevVow = 1; //flag for storing last input from WekiOut3, 1 == ah, 2 == eh, 3 == ee, 4 == oh, 5 == oo 394 | ~flag_wekSyn_prevVow_change = false; //flag for debouncing sudden changes due to input noise 395 | ~yieldTime_wsc1 = 0.1; //in seconds 396 | ~debounceTime = 0.3; //(in seconds) increase or reduce this value for less or more jittery parameter change response, respectively (higher value causes more delay in parameter changes after detected changes in sound, but may yield more accurate performance...the accuracy of trained Wekinator models is also a factor) 397 | 398 | ~routine_wekiSynthControl1 = {|oscInBus, rate = 0.1| //<---new version w/ debounce for vowel detection 399 | Routine ({ 400 | oscInBus.get({ |msg| 401 | if (msg[0] == 1, { //"voc detected" 402 | if (~flag_wekSyn_prevIn == 1, { //sound source has not changed since prev bus.get 403 | var x; 404 | ~flag_wekSyn_prevIn_change = false; //reset debounce flag in case it was previously triggered 405 | //"msg[0] = 1 - NO CHANGE".postln; 406 | 407 | x = case //MONITOR CHANGES IN msg[2] <<<<<<<< 408 | {msg[2] == 1} {//if msg[2] == 1 - 'AH' DETECTED 409 | if (~flag_wekSyn_prevVow == 1, { //msg[2] has not changed since prev bus.get 410 | ~flag_wekSyn_prevVow_change = false; //reset flag if it was triggered by noise 411 | ~yieldTime_wsc1 = rate.asFloat; //<--- 412 | } , { //else if ~flag_prevVow != 1, maybe we're changing vowels 413 | if (~flag_wekSyn_prevVow_change == false, { //DEBOUNCE 414 | ~flag_wekSyn_prevVow_change = true; 415 | "msg[2] = 1 | prev msg[2] = ? - STATE CHANGE TEST".postln; 416 | ~yieldTime_wsc1 = ~debounceTime; 417 | }, { //else if ~flag_stateChange == true, state has changed and remained same long enough to rule out noise, and we can proceed with triggered changes 418 | ~flag_wekSyn_prevVow_change = false; 419 | ~flag_wekSyn_prevVow = 1; //set flag to remember last state before resetting routine 420 | "msg[2] = 1 | prev msg[2] = ? - STATE CHANGE CONFIRMED".postln; 421 | "SINGING DETECTED: AH | msg[2] = 1".postln; 422 | ~vocA1.set(\interval, -5, \pan, -0.75); 423 | ~vocA2.set(\interval, 4.25, \pan, 0.75); 424 | ~yieldTime_wsc1 = rate.asFloat; //<--- 425 | } ); 426 | } ); 427 | } 428 | {msg[2] == 2} {//if msg[2] == 2 - 'EH' DETECTED 429 | if (~flag_wekSyn_prevVow == 2, { //msg[2] has not changed since prev bus.get 430 | ~flag_wekSyn_prevVow_change = false; //reset flag if it was triggered by noise 431 | ~yieldTime_wsc1 = rate.asFloat; //<--- 432 | } , { //else if ~flag_prevVow != 2, maybe we're changing vowels 433 | if (~flag_wekSyn_prevVow_change == false, { //DEBOUNCE 434 | ~flag_wekSyn_prevVow_change = true; 435 | "msg[2] = 2 | prev msg[2] = ? - STATE CHANGE TEST".postln; 436 | ~yieldTime_wsc1 = ~debounceTime; 437 | }, { //else if ~flag_stateChange == true, state has changed and remained same long enough to rule out noise, and we can proceed with triggered changes 438 | ~flag_wekSyn_prevVow_change = false; 439 | ~flag_wekSyn_prevVow = 2; //set flag to remember last state before resetting routine 440 | "msg[2] = 2 | prev msg[2] = ? - STATE CHANGE CONFIRMED".postln; 441 | "SINGING DETECTED: EH | msg[2] = 2".postln; 442 | ~vocA1.set(\interval, -3, \pan, -0.6); 443 | ~vocA2.set(\interval, 5, \pan, 0.6); 444 | ~yieldTime_wsc1 = rate.asFloat; //<--- 445 | } ); 446 | } ); 447 | } 448 | {msg[2] == 3} {//if msg[2] == 3 - 'EE' 449 | if (~flag_wekSyn_prevVow == 3, { //msg[2] has not changed since prev bus.get 450 | ~flag_wekSyn_prevVow_change = false; //reset flag if it was triggered by noise 451 | ~yieldTime_wsc1 = rate.asFloat; //<--- 452 | } , { //else if ~flag_prevVow != 3, maybe we're changing vowels 453 | if (~flag_wekSyn_prevVow_change == false, { //DEBOUNCE 454 | ~flag_wekSyn_prevVow_change = true; 455 | "msg[2] = 3 | prev msg[2] = ? - STATE CHANGE TEST".postln; 456 | ~yieldTime_wsc1 = ~debounceTime; 457 | }, { //else if ~flag_stateChange == true, state has changed and remained same long enough to rule out noise, and we can proceed with triggered changes 458 | ~flag_wekSyn_prevVow_change = false; 459 | ~flag_wekSyn_prevVow = 3; //set flag to remember last state before resetting routine 460 | "msg[2] = 3 | prev msg[2] = ? - STATE CHANGE CONFIRMED".postln; 461 | "SINGING DETECTED: EE | msg[2] = 3".postln; 462 | ~vocA1.set(\interval, -4, \pan, -0.45); 463 | ~vocA2.set(\interval, 10, \pan, 0.45); 464 | ~yieldTime_wsc1 = rate.asFloat; //<--- 465 | } ); 466 | } ); 467 | } 468 | {msg[2] == 4} {//if msg[2] == 4 - 'OH' 469 | if (~flag_wekSyn_prevVow == 4, { //msg[2] has not changed since prev bus.get 470 | ~flag_wekSyn_prevVow_change = false; //reset flag if it was triggered by noise 471 | ~yieldTime_wsc1 = rate.asFloat; //<--- 472 | } , { //else if ~flag_prevVow != 4, maybe we're changing vowels 473 | if (~flag_wekSyn_prevVow_change == false, { //DEBOUNCE 474 | ~flag_wekSyn_prevVow_change = true; 475 | "msg[2] = 4 | prev msg[2] = ? - STATE CHANGE TEST".postln; 476 | ~yieldTime_wsc1 = ~debounceTime; 477 | }, { //else if ~flag_stateChange == true, state has changed and remained same long enough to rule out noise, and we can proceed with triggered changes 478 | ~flag_wekSyn_prevVow_change = false; 479 | ~flag_wekSyn_prevVow = 4; //set flag to remember last state before resetting routine 480 | "msg[2] = 4 | prev msg[2] = ? - STATE CHANGE CONFIRMED".postln; 481 | "SINGING DETECTED: OH | msg[2] = 4".postln; 482 | ~vocA1.set(\interval, -3, \pan, -0.6); 483 | ~vocA2.set(\interval, 7, \pan, 0.6); 484 | ~yieldTime_wsc1 = rate.asFloat; //<--- 485 | } ); 486 | } ); 487 | } 488 | {msg[2] == 5} {//if msg[2] == 5 'OO' 489 | if (~flag_wekSyn_prevVow == 5, { //msg[2] has not changed since prev bus.get 490 | ~flag_wekSyn_prevVow_change = false; //reset flag if it was triggered by noise 491 | ~yieldTime_wsc1 = rate.asFloat; //<--- 492 | } , { //else if ~flag_prevVow != 5, maybe we're changing vowels 493 | if (~flag_wekSyn_prevVow_change == false, { //DEBOUNCE 494 | ~flag_wekSyn_prevVow_change = true; 495 | "msg[2] = 5 | prev msg[2] = ? - STATE CHANGE TEST".postln; 496 | ~yieldTime_wsc1 = ~debounceTime; 497 | }, { //else if ~flag_stateChange == true, state has changed and remained same long enough to rule out noise, and we can proceed with triggered changes 498 | ~flag_wekSyn_prevVow_change = false; 499 | ~flag_wekSyn_prevVow = 5; //set flag to remember last state before resetting routine 500 | "msg[2] = 5 | prev msg[2] = ? - STATE CHANGE CONFIRMED".postln; 501 | "SINGING DETECTED: OO | msg[2] = 5".postln; 502 | ~vocA1.set(\interval, -4, \pan, -0.25); 503 | ~vocA2.set(\interval, 5, \pan, 0.25); 504 | ~yieldTime_wsc1 = rate.asFloat; //<--- 505 | } ); 506 | } ); 507 | }; 508 | //~yieldTime_wsc1 = rate.asFloat; //<--- 509 | } , { //else ~flag_wekSyn_prevIn == 2, we're changing from "2 - whistling" to "1 - voc" 510 | //CHECK TO SEE IF TIME THRESHOLD SURPASSED BEFORE CONFIRMING STATE CHANGE 511 | if (~flag_wekSyn_prevIn_change == false, { 512 | ~flag_wekSyn_prevIn_change = true; 513 | //"msg[0] = 1 | prev msg[0] = 2 - STATE CHANGE TEST".postln; 514 | ~yieldTime_wsc1 = ~debounceTime; //"debounce" time 515 | }, { //else if ~flag_stateChange == true, state has changed and remained same long enough to rule out noise, and we can proceed with triggered changes 516 | var interpSynth, interpVerbRoom, interpVerbMix; 517 | ~flag_wekSyn_prevIn_change = false; 518 | //XFADE ~mixAmount.bus from whistle to voc processing synth busses (1.0 - 0.0) 519 | interpSynth = Synth(\LinInterp_num, [\startVal, 1, \stopVal, 0, \duration, 0.3, \out, ~bus_vocwislMix]); 520 | interpVerbRoom = Synth(\LinInterp_num, [\startVal, 0.7, \stopVal, 0.4, \duration, 0.3, \out, ~bus_verbRoom]); 521 | interpVerbMix = Synth(\LinInterp_num, [\startVal, 0.6, \stopVal, 0.3, \duration, 0.3, \out, ~bus_verbMix]); 522 | ~flag_wekSyn_prevIn = 1; //set flag to remember last state before resetting routine 523 | //"msg[0] = 1 | prev msg[0] = 1 - STATE CHANGE CONFIRMED".postln; 524 | "SINGING DETECTED--AUTO-HARMONIZE MODE ACTIVE".postln; 525 | ~yieldTime_wsc1 = rate.asFloat; 526 | //~yieldTime_wsc1 = 0.31; //precautionary yield time to allow LinInterp synth to finish before resuming routine...thought this helped avoid audio clicks, but don't think it does & it adds latency.... 527 | } ); 528 | } ); 529 | } , { //ELSE======= if msg[0] == 2 "whistling detected" 530 | if (~flag_wekSyn_prevIn == 2, { //sound source has not changed since prev bus.get 531 | ~flag_wekSyn_prevIn_change = false; //reset debounce flag 532 | //"msg[0] = 2 - NO CHANGE".postln; 533 | ~yieldTime_wsc1 = rate.asFloat; //<---- 534 | } , { //else flag must have changed recently from "1 - voc" to "2 - whistling" 535 | //CHECK TO SEE IF TIME THRESHOLD SURPASSED BEFORE CONFIRMING STATE CHANGE 536 | if (~flag_wekSyn_prevIn_change == false, { 537 | ~flag_wekSyn_prevIn_change = true; 538 | //"msg[0] = 2 | prev msg[0] = 1 - STATE CHANGE TEST".postln; 539 | ~yieldTime_wsc1 = ~debounceTime; //"debounce" time <--- 540 | }, { //else if ~flag_stateChange == true, state has changed and remained same long enough to rule out noise, and we can proceed with triggered changes 541 | var interpSynth, interpVerbRoom, interpVerbMix; 542 | ~flag_wekSyn_prevIn_change = false; 543 | //XFADE ~mixAmount.bus from voc to whistle processing synth busses (0.0 - 1.0) 544 | interpSynth = Synth(\LinInterp_num, [\startVal, 0, \stopVal, 1, \duration, 0.3, \out, ~bus_vocwislMix]); 545 | interpVerbRoom = Synth(\LinInterp_num, [\startVal, 0.4, \stopVal, 0.7, \duration, 0.3, \out, ~bus_verbRoom]); 546 | interpVerbMix = Synth(\LinInterp_num, [\startVal, 0.3, \stopVal, 0.6, \duration, 0.3, \out, ~bus_verbMix]); 547 | ~flag_wekSyn_prevIn = 2; //set flag to remember last state before resetting routine 548 | //"msg[0] = 2 | prev msg[0] = 2 - STATE CHANGE CONFIRMED".postln; 549 | "WHISTLING DETECTED--AUTO-TUNE MODE ACTIVE".postln; 550 | ~yieldTime_wsc1 = rate.asFloat; //<--- 551 | //~yieldTime_wsc1 = 0.31; //precautionary yield time to allow LinInterp synth to finish before resuming routine...thought this helped avoid audio clicks, but don't think it does & it adds latency.... 552 | } ); 553 | } ); 554 | } ); 555 | }); 556 | //"Out of .get, before slight .yield".postln; 557 | 0.02.yield; //<-----ANY OTHER WAY W/OUT INDUCING LATENCY!?????? 558 | // ("yieldTime_wsc1 = " ++ ~yieldTime_wsc1).postln; 559 | ~yieldTime_wsc1.yieldAndReset; 560 | }); 561 | }; 562 | 563 | /*~routine_wekiSynthControl1 = {|oscInBus, rate = 0.1| 564 | Routine ({ 565 | oscInBus.get({ |msg| 566 | if (msg[0] == 1, { //"voc detected" 567 | if (~flag_wekSyn_prevIn == 1, { //sound source has not changed since prev bus.get 568 | var x; 569 | ~flag_wekSyn_prevIn_change = false; //reset debounce flag in case it was previously triggered 570 | //CARRY ON... do anything??? 571 | //"msg[0] = 1 - NO CHANGE".postln; 572 | 573 | //MONITOR CHANGES IN msg[2] <<<<<<<< 574 | x = case 575 | {msg[2] == 1} {//if msg[2] == 1 - 'AH' DETECTED 576 | //"msg[2] = 1 - AH".postln; 577 | ~vocA1.set(\interval, -5, \pan, -0.75); 578 | ~vocA2.set(\interval, 4.25, \pan, 0.75); 579 | /* 580 | if (~flag_wekSyn_prevVow == 1, { //msg[2] has not changed since prev bus.get 581 | 582 | ~flag_wekSyn_prevVow_change = false; //reset flag if it was triggered by noise 583 | 584 | //CARRY ON W/ 'AH' COMMANDS 585 | 586 | ~yieldTime_wsc1 = rate.asFloat; //0.1 <----necessary??? 587 | } , { //else if ~flag_prevVow != 1, maybe we're changing vowels 588 | //DEBOUNCE 589 | if (~flag_wekSyn_prevVow_change == false, { 590 | ~flag_wekSyn_prevVow_change = true; 591 | "msg[2] = 1 | prev msg[2] = ? - STATE CHANGE TEST".postln; 592 | ~yieldTime_wsc1 = ~debounceTime; //"debounce" time 593 | }, { //else if ~flag_stateChange == true, state has changed and remained same long enough to rule out noise, and we can proceed with triggered changes 594 | ~flag_wekSyn_prevVow_change = false; 595 | 596 | //DO STUFF WHEN 'AH' FIRST DETECTED 597 | 598 | ~flag_wekSyn_prevVow = 1; //set flag to remember last state before resetting routine 599 | "msg[2] = 1 | prev msg[2] = ? - STATE CHANGE CONFIRMED".postln; 600 | ~yieldTime_wsc1 = rate; //<---yea? 601 | 602 | } ); 603 | } ); 604 | */ 605 | } 606 | {msg[2] == 2} {//if msg[2] == 2 - 'EH' DETECTED 607 | //DEBOUNCE? 608 | 609 | //"msg[2] = 1 - EH".postln; 610 | ~vocA1.set(\interval, -3, \pan, -0.6); 611 | ~vocA2.set(\interval, 5, \pan, 0.6); 612 | 613 | //ANYTHING ELSE? (OPTIONAL EFFECTS PER VOWEL? 614 | 615 | //~yieldTime_wsc1 = rate.asFloat; //actually not here? 616 | } 617 | {msg[2] == 3} {//if msg[2] == 3 - 'EE' 618 | //DEBOUNCE? 619 | 620 | //"msg[2] = 1 - EE".postln; 621 | ~vocA1.set(\interval, -4, \pan, -0.45); 622 | ~vocA2.set(\interval, 10, \pan, 0.45); 623 | //~yieldTime_wsc1 = rate.asFloat; 624 | } 625 | {msg[2] == 4} {//if msg[2] == 4 - 'OH' 626 | //DEBOUNCE? 627 | 628 | //"msg[2] = 1 - OH".postln; 629 | ~vocA1.set(\interval, -3, \pan, -0.6); 630 | ~vocA2.set(\interval, 7, \pan, 0.6); 631 | //~yieldTime_wsc1 = rate.asFloat; 632 | } 633 | {msg[2] == 5} {//if msg[2] == 5 'OO' 634 | //DEBOUNCE? 635 | 636 | //"msg[2] = 1 - OO".postln; 637 | ~vocA1.set(\interval, -4, \pan, -0.25); 638 | ~vocA2.set(\interval, 5, \pan, 0.25); 639 | //~yieldTime_wsc1 = rate.asFloat; 640 | }; 641 | ~yieldTime_wsc1 = rate.asFloat; //<--- 642 | } , { //else ~flag_wekSyn_prevIn == 2, we're changing from "2 - whistling" to "1 - voc" 643 | 644 | //CHECK TO SEE IF TIME THRESHOLD SURPASSED BEFORE CONFIRMING STATE CHANGE 645 | if (~flag_wekSyn_prevIn_change == false, { 646 | ~flag_wekSyn_prevIn_change = true; 647 | //"msg[0] = 1 | prev msg[0] = 2 - STATE CHANGE TEST".postln; 648 | ~yieldTime_wsc1 = ~debounceTime; //"debounce" time 649 | }, { //else if ~flag_stateChange == true, state has changed and remained same long enough to rule out noise, and we can proceed with triggered changes 650 | var interpSynth, interpVerbRoom, interpVerbMix; 651 | ~flag_wekSyn_prevIn_change = false; 652 | //XFADE ~mixAmount.bus from whistle to voc processing synth busses (1.0 - 0.0) 653 | interpSynth = Synth(\LinInterp_num, [\startVal, 1, \stopVal, 0, \duration, 0.3, \out, ~bus_vocwislMix]); 654 | interpVerbRoom = Synth(\LinInterp_num, [\startVal, 0.7, \stopVal, 0.4, \duration, 0.3, \out, ~bus_verbRoom]); 655 | interpVerbMix = Synth(\LinInterp_num, [\startVal, 0.6, \stopVal, 0.3, \duration, 0.3, \out, ~bus_verbMix]); 656 | //0.2.yield; //<---CAUSES ERROR b/c "Not in Routine" - worried I'd need this for stability? otherwise routine might just race away and fire off lots of these synths on edge cases? 657 | 658 | ~flag_wekSyn_prevIn = 1; //set flag to remember last state before resetting routine 659 | //"msg[0] = 1 | prev msg[0] = 1 - STATE CHANGE CONFIRMED".postln; 660 | ~yieldTime_wsc1 = rate.asFloat; 661 | //~yieldTime_wsc1 = 0.31; //precautionary yield time to allow LinInterp synth to finish before resuming routine...thought this helped avoid audio clicks, but don't think it does & it adds latency.... 662 | } ); 663 | } ); 664 | } , { //ELSE======= if msg[0] == 2 "whistling detected" 665 | if (~flag_wekSyn_prevIn == 2, { //sound source has not changed since prev bus.get 666 | ~flag_wekSyn_prevIn_change = false; //reset debounce flag 667 | 668 | //CARRY ON.... do anything? 669 | //"msg[0] = 2 - NO CHANGE".postln; 670 | 671 | ~yieldTime_wsc1 = rate.asFloat; //<---- 672 | } , { //else flag must have changed recently from "1 - voc" to "2 - whistling" 673 | 674 | //CHECK TO SEE IF TIME THRESHOLD SURPASSED BEFORE CONFIRMING STATE CHANGE 675 | if (~flag_wekSyn_prevIn_change == false, { 676 | ~flag_wekSyn_prevIn_change = true; 677 | //"msg[0] = 2 | prev msg[0] = 1 - STATE CHANGE TEST".postln; 678 | ~yieldTime_wsc1 = ~debounceTime; //"debounce" time <--- 679 | }, { //else if ~flag_stateChange == true, state has changed and remained same long enough to rule out noise, and we can proceed with triggered changes 680 | var interpSynth, interpVerbRoom, interpVerbMix; 681 | ~flag_wekSyn_prevIn_change = false; 682 | 683 | //XFADE ~mixAmount.bus from voc to whistle processing synth busses (0.0 - 1.0) 684 | interpSynth = Synth(\LinInterp_num, [\startVal, 0, \stopVal, 1, \duration, 0.3, \out, ~bus_vocwislMix]); 685 | interpVerbRoom = Synth(\LinInterp_num, [\startVal, 0.4, \stopVal, 0.7, \duration, 0.3, \out, ~bus_verbRoom]); 686 | interpVerbMix = Synth(\LinInterp_num, [\startVal, 0.3, \stopVal, 0.6, \duration, 0.3, \out, ~bus_verbMix]); 687 | //0.2.yield; //<---CAUSES ERROR b/c "Not in Routine" - worried I'd need this for stability? otherwise routine might just race away and fire off lots of these synths on edge cases? 688 | 689 | ~flag_wekSyn_prevIn = 2; //set flag to remember last state before resetting routine 690 | //"msg[0] = 2 | prev msg[0] = 2 - STATE CHANGE CONFIRMED".postln; 691 | ~yieldTime_wsc1 = rate.asFloat; //<--- 692 | //~yieldTime_wsc1 = 0.31; //precautionary yield time to allow LinInterp synth to finish before resuming routine...thought this helped avoid audio clicks, but don't think it does & it adds latency.... 693 | } ); 694 | } ); 695 | } ); 696 | }); 697 | //"Out of .get, before slight .yield".postln; 698 | 0.02.yield; //<-----ANY OTHER WAY W/OUT INDUCING LATENCY!?????? 699 | // ("yieldTime_wsc1 = " ++ ~yieldTime_wsc1).postln; 700 | ~yieldTime_wsc1.yieldAndReset; //<---necessary or simply leave as rate.yieldAndReset??? 701 | }); 702 | }; */ 703 | 704 | ~initAutoHT = Routine({ 705 | ~scale1 = Scale.majorPentatonic.degrees; //[0, 2, 4, 7, 9] 706 | ~scaleBuf = Buffer.alloc(s, ~scale1.size, 1, {|b| b.setnMsg(0, ~scale1) } ); 707 | 708 | i = Synth(\monAudioInput, [\outBus, ~bus_input1]); 709 | l = Synth(\Loudness, [\input, ~bus_input1, \outBus, ~bus_loudness]).moveAfter(i); 710 | m = Synth(\MFCC, [\input, ~bus_input1, \outBus, ~bus_mfcc]).moveAfter(i); 711 | p = Synth(\Pitch, [\input, ~bus_input1, \outBus, ~bus_pitch]).moveAfter(i); //outputs two values (freq) 712 | 713 | //"AUDIO IN & FEATURE EXTRACTOR SYNTHS ACTIVE...".postln; 714 | 715 | //0.15.wait; 716 | 717 | //TRANSMIT FEATURES 718 | ~busSum_xin = ~sumFunc_xin.value([~bus_loudness, ~bus_mfcc, ~bus_pitch]); //collect loudness, mfcc, and pitch features to output via OSC to Wekinator 719 | ~transmit = ~streamFeatures.value(~busSum_xin, ~oscOut_weki, "/wek/inputs", 0.08); 720 | SystemClock.play(~transmit); 721 | //"TRANSMITTING FEATURES (OSC OUT)...".postln; 722 | 723 | //0.15.wait; 724 | 725 | //RECEIVE WEKI OSC OUTPUT 726 | ~bus_oscIn_1 = ~oscIn_weki_1.value(numIn: 4, defKey: "oscDef1", port: 12000); //open port to receive OSC output from Wekinator and send to a control bus 727 | 728 | //"OSC-IN Port Ready...".postln; 729 | 730 | 0.15.wait; //voodoo wait... not critical, but including this seems to prevent audio clicks 731 | 732 | ~dryIn1 = Synth(\mono2stereo, [\in, ~bus_input1, \amp, ~bus_dryIn1Amp.asMap, \out, 0]).moveAfter(i); //pipe dry mono input to stereo output 733 | 734 | ~wisA1 = ~wekiSyn_wisl1.value(oscInBus: ~bus_oscIn_1, audioIn: ~bus_input1, audioOut: ~bus_wislSynthSum, scaleBuf: ~scaleBuf).moveAfter(~dryIn1); //whistle harmonizer 1 735 | 736 | ~vocA1 = ~wekiSyn_voc1.value(oscInBus: ~bus_oscIn_1, audioIn: ~bus_input1, audioOut: ~bus_vocSynthSum).moveAfter(~dryIn1); //voc harmonizer 1 737 | 738 | ~vocA2 = ~wekiSyn_voc1.value(oscInBus: ~bus_oscIn_1, audioIn: ~bus_input1, audioOut: ~bus_vocSynthSum).moveAfter(~dryIn1); //voc harmonizer 2 739 | 740 | //"AutoHarmonizer and AutoTune Synths Active...".postln; 741 | 0.15.wait; //necessary wait 742 | 743 | ~dryIn1.set(\amp, ~bus_dryIn1Amp.asMap); 744 | ~wisA1.set(\amp, ~bus_wislAmp.asMap); 745 | ~vocA1.set(\amp, ~bus_vocAmp.asMap); 746 | ~vocA2.set(\amp, ~bus_vocAmp.asMap); 747 | //"Configure Synths...".postln; 748 | 749 | 0.15.wait; //voodoo wait... not critical, but including this seems to prevent audio clicks 750 | 751 | ~accomp_mix = Synth(\XFadeStereo, [\in1, ~bus_vocSynthSum, \in2, ~bus_wislSynthSum, \out, 0, \amountControlBus, ~bus_vocwislMix.asMap, \amp, ~bus_accompAmp.asMap], addAction: \addToTail); 752 | 753 | ~fx_reverb = Synth(\reverb2x2, [\outBus, 0, \mix, ~bus_verbMix.asMap, \room, ~bus_verbRoom.asMap], addAction: \addToTail); //stereo reverb processing sum output 754 | 755 | //--Activate OSC input streaming-- 756 | ~wekiSynthControl1 = ~routine_wekiSynthControl1.value(oscInBus: ~bus_oscIn_1, rate: 0.1); //create routine to get latest values from OSC input control bus 757 | SystemClock.play(~wekiSynthControl1); //start OSC input bus monitoring routine 758 | //"OSC Input Streaming...".postln; 759 | "--AUTOHARMTUNE ACTIVE--".postln; 760 | }); 761 | 762 | ) 763 | 764 | // #3 ======ACTIVATE WEKI INSTRUMENT EXAMPLE #1 (Auto-harmonize & Auto-tune) ====== 765 | SystemClock.play(~initAutoHT); 766 | 767 | // #4 --- STOP / CLEANUP INSTRUMENT EXAMPLE #1 --- 768 | ( 769 | ~initAutoHT.stop; 770 | ~initAutoHT.reset; 771 | ~transmit.stop; 772 | ~transmit.free; 773 | ~sumSyn_xin.free; 774 | ~busSum_xin.free; 775 | 776 | ~scaleBuf.free; 777 | i.free; 778 | l.free; 779 | m.free; 780 | p.free; 781 | ~dryIn1.free; 782 | ~wisA1.free; 783 | ~vocA1.free; 784 | ~vocA2.free; 785 | ~accomp_mix.free; 786 | ~fx_reverb.free; 787 | 788 | ~bus_oscIn_1.free; 789 | ~wekiSynthControl1.stop; 790 | ~wekiSynthControl1.free; 791 | OSCdef(\oscDef1).free; 792 | 793 | s.freeAll; //don't use when integrated into main SCML .scd....otherwise feature Xtractor synths will be freed too 794 | ) 795 | 796 | 797 | ////////////////////////////////////////////////////////////// 798 | // ======ACTIVATE AUDIO INPUT ANALYSIS SYNTHS====== 799 | i = Synth(\monAudioInput, [\outBus, ~bus_input1]); 800 | 801 | //LOUDNESS (volume - perceptual measure in Sones) ------ 802 | l = Synth(\Loudness, [\input, ~bus_input1, \outBus, ~bus_loudness]).moveAfter(i); 803 | 804 | //RMS (volume) ------ 805 | r = Synth(\RMS, [\input, ~bus_input1, \numSamp, 40, \outBus, ~bus_rms]).moveAfter(i); 806 | 807 | //MFCC 808 | m = Synth(\MFCC, [\input, ~bus_input1, \outBus, ~bus_mfcc]).moveAfter(i); 809 | 810 | //PITCH TRACKERs 811 | p = Synth(\Pitch, [\input, ~bus_input1, \outBus, ~bus_pitch]).moveAfter(i); //2 output vals (freq, hasPitch) 812 | 813 | ~p2 = Synth(\Pitch2, [\input, ~bus_input1, \outBus, ~bus_pitch2]).moveAfter(i); //3 output vals (freq, hasPitch, MIDInote) 814 | 815 | //SPECTRAL CENTROID 816 | c = Synth(\specCent, [\input, ~bus_input1, \outBus, ~bus_specCent]).moveAfter(i); 817 | 818 | //SPECTRAL FLATNESS 819 | f = Synth(\specFlat, [\input, ~bus_input1, \outBus, ~bus_specFlat]).moveAfter(i); 820 | 821 | //ONSETS 822 | n = Synth(\onsets, [\input, ~bus_input1, \outBus, ~bus_onsets, \threshold, 0.7]).moveAfter(i); 823 | n.set(\threshold, 0.5); //evaluate to change detection threshold (between 0 - 1) 824 | 825 | s.plotTree; 826 | //#4 ======OUTPUT AUDIO FEATURES VIA OSC====== 827 | 828 | // Default OSC output (~oscOut_weki): IP - 127.0.0.1, Port # 6448 829 | 830 | (//TRANSMIT LOUDNESS AND MFCCs - (14 features) 831 | ~busSum_ldns_mfcc = ~sumFunc_xin.value( [~bus_loudness, ~bus_mfcc] ); 832 | ~transmit = ~streamFeatures.value(~busSum_ldns_mfcc, ~oscOut_weki, "/wek/inputs", rate: 0.5, post: 1); //rate arg adjusts rate OSC messages are output (0.04 or about 25 times/sec is default) 833 | SystemClock.play(~transmit); 834 | ) 835 | ( //STOP transmitting (NOTE: evaluate this chunk before transmitting other feature sets) 836 | ~transmit.stop; 837 | ~sumSyn_xin.free; 838 | ~busSum_ldns_mfcc.free; 839 | ) 840 | 841 | 842 | (//TRANSMIT LOUDNESS, SPECTRAL FLATNESS, AND SPECTRAL CENTROID - (3 features) 843 | ~busSum_ldns_sFlat_sCent = ~sumFunc_xin.value( [~bus_loudness, ~bus_specFlat, ~bus_specCent] ); 844 | ~transmit = ~streamFeatures.value(~busSum_ldns_sFlat_sCent, ~oscOut_weki, "/wek/inputs", 0.1, 1); 845 | SystemClock.play(~transmit); 846 | ) 847 | ( //STOP transmitting 848 | ~transmit.stop; 849 | ~sumSyn_xin.free; 850 | ~busSum_ldns_sFlat_sCent.free; 851 | ) 852 | 853 | (//TRANSMIT LOUDNESS AND PITCH 854 | ~busSum_ldns_pitch = ~sumFunc_xin.value( [~bus_loudness, ~bus_pitch] ); //3 features [loudness, freq, hasFreq] 855 | //~busSum_ldns_pitch = ~sumFunc_xin.value( [~bus_loudness, ~bus_pitch2] ); //4 features [loudness, freq, hasFreq, MIDI note] 856 | ~transmit = ~streamFeatures.value(~busSum_ldns_pitch, ~oscOut_weki, "/wek/inputs", rate: 0.1, post: 1); 857 | SystemClock.play(~transmit); 858 | ) 859 | ( //STOP transmitting 860 | ~transmit.stop; 861 | ~sumSyn_xin.free; 862 | ~busSum_ldns_pitch.free; 863 | ) 864 | 865 | (//TRANSMIT LOUDNESS, SPECTRAL FLATNESS, ONSETS - (3 features) 866 | ~busSum_ldns_sFlat_onset = ~sumFunc_xin.value( [~bus_loudness, ~bus_specFlat, ~bus_onsets] ); 867 | ~transmit = ~streamFeatures.value(~busSum_ldns_sFlat_onset, ~oscOut_weki, "/wek/inputs", 0.04); 868 | SystemClock.play(~transmit); 869 | ) 870 | (//STOP transmitting 871 | ~transmit.stop; 872 | ~sumSyn_xin.free; 873 | ~busSum_ldns_sFlat_onset.free; 874 | ) 875 | 876 | (//TRANSMIT FEATURES - Loudness, Flatness, Pitch[2] (variable size feature bus set) 877 | ~busSum_xin = ~sumFunc_xin.value([~bus_loudness, ~bus_specFlat, ~bus_pitch2]); //input desired feature busses in array to monitor (separate w/ comma) 878 | ~transmit = ~streamFeatures.value(~busSum_xin, ~oscOut_weki, "/wek/inputs", 0.1, 1); 879 | SystemClock.play(~transmit); 880 | ) 881 | ( //STOP transmitting 882 | ~transmit.stop; 883 | ~sumSyn_xin.free; 884 | ~busSum_xin.free; 885 | ) 886 | 887 | /////////testing new convenience function...in the works...not huge priority now.... 888 | ~transmitFeatures.value(~busSum_ldns_mfcc, [~bus_loudness, ~bus_mfcc], ~oscOut_weki, "/wek/inputs", 0.5, act: false); 889 | s.plotTree; 890 | s.queryAllNodes; 891 | 892 | // ======FINAL CLEANUP====== 893 | ( 894 | s.freeAll; //free all active synths 895 | 896 | ~bus_input1.free; 897 | ~bus_loudness.free; 898 | ~bus_rms.free; 899 | ~bus_mfcc.free; 900 | ~bus_pitch.free; 901 | ~bus_pitch2.free; 902 | ~bus_specCent.free; 903 | ~bus_specFlat.free; 904 | ~bus_onsets.free; 905 | 906 | ~oscOut_weki.disconnect; 907 | OSCdef.freeAll; 908 | ) 909 | 910 | s.quit; 911 | 912 | 913 | //============================================== 914 | //=======SIMPLE TESTS of Voc and Whistle Auto-Harmonize Synths===========// 915 | ~testSynth = ~wekiSyn_vocSimp.value; 916 | ~testSynth.set(\interval, -2.0, \pan, 0.0, \mix, 0.2); 917 | ~testSynth.free; 918 | 919 | ~wekiSyn_vocSimp = {|interval = 5.0, pan = 0.0, /*scaleBuf,*/ amp = 1.0, mix = 1.0| //add arg for initial interval scale spread of weki output1???? (12 == (-12, 12) | 5 == (-5, 5) | etc...) 920 | var sigIn, sigOut, returnSynth, pitchRatio; 921 | 922 | returnSynth = SynthDef(\vocHarm_weki1, {|interval = 5.0, pan = 0.0, /*scaleBuf,*/ amp = 1.0, mix = 1.0| //<--needed to control synth after initial creation w/ ~wekiSyn_wisl1 execution??? 923 | sigIn = SoundIn.ar(0); 924 | 925 | pitchRatio = interval.asFloat.round(0.01).midiratio; //set pitch transposition (semi-tone intervals) 926 | sigOut = SelectX.ar(mix, [sigIn, PitchShift.ar(sigIn, 0.1, pitchRatio, 0, 0.002)]); //<----not sure if I want this dry/wet signal mixing functionality.....decide...? 927 | //sigOut = PitchShift.ar(sigIn, 0.1, pitchRatio, 0, 0.002); 928 | 929 | //sigOut = FreeVerb.ar(sigOut, mix: 0.33, room: LinLin.kr(inMsg[1], 0, 1, 0.01, 0.25)); //?? yes? 930 | sigOut = Pan2.ar(sigOut, pan.asFloat); //?? yes? 931 | //sigOut = Splay.ar(sigOut, LinLin.kr(inMsg[1], 0, 1, 0, 0.2)); //yea / no? 932 | 933 | Out.ar(0, sigOut * amp); 934 | }).play(addAction: \addToTail); 935 | 936 | returnSynth; 937 | }; 938 | 939 | ~scale1 = Scale.majorPentatonic.degrees; //[0, 2, 4, 7, 9] 940 | ~scaleBuf = Buffer.alloc(s, ~scale1.size, 1, {|b| b.setnMsg(0, ~scale1) } ); 941 | ~scaleBuf.free; 942 | ~testSynth = ~wekiSyn_wislSimp.value(controlIn: ~bus_MouseX, scaleBuf: ~scaleBuf); 943 | ~testSynth.set(\room, 0.65, \rmix, 0.3, \mix, 0.6, \pan, 0.0); 944 | ~testSynth.free; 945 | 946 | SynthDef.new(\mousey, {|out| 947 | var sig = MouseX.kr(0, 1); 948 | Out.kr(out, sig); 949 | } ).add; 950 | 951 | ~bus_MouseX = Bus.control(s, 1); 952 | ~mousey = Synth(\mousey, [~bus_MouseX]); 953 | 954 | ~wekiSyn_wislSimp = {|controlIn, scaleBuf, amp = 1.0, mix = 1.0, room = 0.65, rmix = 0.3, pan = 0.0| //add arg for initial interval scale spread of weki output1???? (12 == (-12, 12) | 5 == (-5, 5) | etc...) 955 | var sigIn, sigOut, controlVal, returnSynth, pitchRatio; 956 | 957 | returnSynth = SynthDef(\wislHarm_weki1, {|pitchInt = 3, amp = 1.0, mix = 1.0, room = 0.65, rmix = 0.3, pan = 0.0| //<--needed to control synth after initial creation w/ ~wekiSyn_wisl1 execution??? 958 | controlVal = In.kr(controlIn, 1); 959 | sigIn = SoundIn.ar(0); 960 | //pitchRatio = DegreeToKey.kr(scaleBuf.bufnum, pitchInt.round(0.01)).midiratio; 961 | pitchRatio = DegreeToKey.kr(scaleBuf.bufnum, LinLin.kr(controlVal, 0, 1, 5, -5).round(1)).midiratio; 962 | sigOut = SelectX.ar(mix, [sigIn, PitchShift.ar(sigIn, 0.1, pitchRatio, 0, 0.002)]); //<----not sure if I want this dry/wet signal mixing functionality.....decide...? 963 | 964 | sigOut = FreeVerb.ar(sigOut, mix: rmix, room: room); //?? yes? 965 | sigOut = Pan2.ar(sigOut, pan); //?? yes? 966 | //sigOut = Splay.ar(sigOut, splay); //not stereo spreader I believe (at least w/ one input...?) 967 | 968 | Out.ar(0, sigOut * amp); 969 | }).play(addAction: \addToTail); 970 | 971 | returnSynth; 972 | }; 973 | //========^^^^^^^^^====SIMPLE TESTS======^^^^^^^^^===============// 974 | 975 | 976 | /* 977 | //PITCH SHIFTING VOC / WHISTLING <------TRY THESE PARAMETERS 978 | play({ 979 | PitchShift.ar( 980 | AudioIn.ar([1,2]), // stereo audio input 981 | 0.1, // grain size 982 | MouseX.kr(0.25, 3, 1), //pitchRatio: mod by Weki_Output2 (Relative Pitch Follower) 983 | MouseY.kr(0, 0.15), // pitch dispersion: mod/switch case different vals based on WekiOut1, 3, 4? 984 | 0.002 // time dispersion 985 | ) 986 | }) 987 | */ 988 | 989 | //NOT NEEDED for current implementation of Auto-Harm-Tune example, but may be useful, more granular mix control scheme in the future.... 990 | ~bus_vocSynthSub = Bus.audio(s, 2); //NOT NEEDED if not using subMix synths... 991 | ~bus_wislSynthSub = Bus.audio(s, 2); //NOT NEEDED if not using subMix synths... 992 | ~bus_outMain = Bus.audio(s, 2); //NOT NEEDED 993 | 994 | ~vocSynthSum_mix = Synth(\subMix, [\inBus, ~bus_vocSynthSum, \amp, ~bus_vocAmp.asMap, \outBus, ~bus_outMain/*or just 0???*/]); //voc accomp sum mix synth. NOT NEEDED IF: controlling subSynth levels w/ ~bus_vocAmp and outputting to ~bus_vocSynthSum, OTHERWISE - create additional bus to output to from this synth to use as input bus for ~accomp_mix XFADE synth 995 | 996 | ~wislSynthSum_mix = Synth(\subMix, [\inBus, ~bus_wislSynthSum, \amp, ~bus_wislAmp.asMap, \outBus, ~bus_outMain]); //whistle accomp sum mix synth. NOT NEEDED IF: controlling subSynth levels w/ ~bus_wislAmp and outputting to ~bus_wislSynthSum, OTHERWISE - create additional bus to output to from this synth to use as input bus for ~accomp_mix XFADE synth 997 | 998 | ~masterOut = Synth(\subMix, [in: ~bus_outMain, out: 0]); //NEEDED? 999 | //NOT NEEDED -------^^^^^^^^^^^^^^^ --------------------------------------------------------------------------------