├── .gitignore ├── .travis.yml ├── 8bit8k.wav ├── LICENSE ├── README.rst ├── include └── README ├── lib └── README ├── networkLoopback.py ├── platformio.ini └── src └── main.cpp /.gitignore: -------------------------------------------------------------------------------- 1 | .pio 2 | .vscode 3 | 4 | # Prerequisites 5 | *.d 6 | 7 | # Compiled Object files 8 | *.slo 9 | *.lo 10 | *.o 11 | *.obj 12 | 13 | # Precompiled Headers 14 | *.gch 15 | *.pch 16 | 17 | # Compiled Dynamic libraries 18 | *.so 19 | *.dylib 20 | *.dll 21 | 22 | # Fortran module files 23 | *.mod 24 | *.smod 25 | 26 | # Compiled Static libraries 27 | *.lai 28 | *.la 29 | *.a 30 | *.lib 31 | 32 | # Executables 33 | *.exe 34 | *.out 35 | *.app 36 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | # Continuous Integration (CI) is the practice, in software 2 | # engineering, of merging all developer working copies with a shared mainline 3 | # several times a day < https://docs.platformio.org/page/ci/index.html > 4 | # 5 | # Documentation: 6 | # 7 | # * Travis CI Embedded Builds with PlatformIO 8 | # < https://docs.travis-ci.com/user/integration/platformio/ > 9 | # 10 | # * PlatformIO integration with Travis CI 11 | # < https://docs.platformio.org/page/ci/travis.html > 12 | # 13 | # * User Guide for `platformio ci` command 14 | # < https://docs.platformio.org/page/userguide/cmd_ci.html > 15 | # 16 | # 17 | # Please choose one of the following templates (proposed below) and uncomment 18 | # it (remove "# " before each line) or use own configuration according to the 19 | # Travis CI documentation (see above). 20 | # 21 | 22 | 23 | # 24 | # Template #1: General project. Test it using existing `platformio.ini`. 25 | # 26 | 27 | # language: python 28 | # python: 29 | # - "2.7" 30 | # 31 | # sudo: false 32 | # cache: 33 | # directories: 34 | # - "~/.platformio" 35 | # 36 | # install: 37 | # - pip install -U platformio 38 | # - platformio update 39 | # 40 | # script: 41 | # - platformio run 42 | 43 | 44 | # 45 | # Template #2: The project is intended to be used as a library with examples. 46 | # 47 | 48 | # language: python 49 | # python: 50 | # - "2.7" 51 | # 52 | # sudo: false 53 | # cache: 54 | # directories: 55 | # - "~/.platformio" 56 | # 57 | # env: 58 | # - PLATFORMIO_CI_SRC=path/to/test/file.c 59 | # - PLATFORMIO_CI_SRC=examples/file.ino 60 | # - PLATFORMIO_CI_SRC=path/to/test/directory 61 | # 62 | # install: 63 | # - pip install -U platformio 64 | # - platformio update 65 | # 66 | # script: 67 | # - platformio ci --lib="." --board=ID_1 --board=ID_2 --board=ID_N 68 | -------------------------------------------------------------------------------- /8bit8k.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/andymule/ESP32DuplexAudioStream/67403230ebf3cb30dd9fd24ab3f7b80ac9d1906c/8bit8k.wav -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | This is free and unencumbered software released into the public domain. 2 | 3 | Anyone is free to copy, modify, publish, use, compile, sell, or 4 | distribute this software, either in source code form or as a compiled 5 | binary, for any purpose, commercial or non-commercial, and by any 6 | means. 7 | 8 | In jurisdictions that recognize copyright laws, the author or authors 9 | of this software dedicate any and all copyright interest in the 10 | software to the public domain. We make this dedication for the benefit 11 | of the public at large and to the detriment of our heirs and 12 | successors. We intend this dedication to be an overt act of 13 | relinquishment in perpetuity of all present and future rights to this 14 | software under copyright law. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 17 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 18 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 19 | IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR 20 | OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, 21 | ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR 22 | OTHER DEALINGS IN THE SOFTWARE. 23 | 24 | For more information, please refer to 25 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | ESP32 full duplex audio w UDP broadcasting 2 | 3 | uses both cores, so won't work on single core variants of ESP32 4 | 5 | Pins used (look up your pinouts for proper GPIO): 6 | 7 | DAC_CHANNEL_2 -- generate test noise -- LOLIN32 GPIO26 8 | 9 | DAC_CHANNEL_1 -- Acutual Speaker data analog output -- LOLIN32 GPIO25 10 | 11 | ADC1_CHANNEL_0 -- Read mic input, analog in -- LOLIN32 GPIO36, labelled VP pin 12 | 13 | Currently, shit noise is generated on DAC_CHANNEL_2. I connect this to my ADC1_CHANNEL_0 for simulating a mic input. This is broadcast to 192.168.1.255 subgroup on port 4444. I include python file that listens to this data and broadcasts it straight back to 192.168.1.255 port 4445. The ESP32 takes this data, fills an audio buffer, and plays it on DAC_CHANNEL_1 14 | 15 | Audio is all 8bit 8khz. We should be able to get to 12bit eventually. 16 | 17 | ==BUILD== 18 | 19 | Install PlatformIO into VSCode. Open this folder, build and upload from there very easily. 20 | 21 | -OR- 22 | 23 | Copy/Paste main.cpp into a new Arduino sketch, you should be able to just upload and run (assuming you've already installed ESP32 libraries) 24 | 25 | 26 | -OR- 27 | 28 | How to build PlatformIO based project 29 | ===================================== 30 | 31 | 1. `Install PlatformIO Core `_ 32 | 2. Download `development platform with examples `_ 33 | 3. Extract ZIP archive 34 | 4. Run these commands: 35 | 36 | .. code-block:: bash 37 | 38 | # Change directory to this directory 39 | 40 | # Build project 41 | > platformio run 42 | 43 | # Upload firmware 44 | > platformio run --target upload 45 | 46 | # Build specific environment 47 | > platformio run -e esp32dev 48 | 49 | # Upload firmware for the specific environment 50 | > platformio run -e esp32dev --target upload 51 | 52 | # Clean build files 53 | > platformio run --target clean 54 | -------------------------------------------------------------------------------- /include/README: -------------------------------------------------------------------------------- 1 | 2 | This directory is intended for project header files. 3 | 4 | A header file is a file containing C declarations and macro definitions 5 | to be shared between several project source files. You request the use of a 6 | header file in your project source file (C, C++, etc) located in `src` folder 7 | by including it, with the C preprocessing directive `#include'. 8 | 9 | ```src/main.c 10 | 11 | #include "header.h" 12 | 13 | int main (void) 14 | { 15 | ... 16 | } 17 | ``` 18 | 19 | Including a header file produces the same results as copying the header file 20 | into each source file that needs it. Such copying would be time-consuming 21 | and error-prone. With a header file, the related declarations appear 22 | in only one place. If they need to be changed, they can be changed in one 23 | place, and programs that include the header file will automatically use the 24 | new version when next recompiled. The header file eliminates the labor of 25 | finding and changing all the copies as well as the risk that a failure to 26 | find one copy will result in inconsistencies within a program. 27 | 28 | In C, the usual convention is to give header files names that end with `.h'. 29 | It is most portable to use only letters, digits, dashes, and underscores in 30 | header file names, and at most one dot. 31 | 32 | Read more about using header files in official GCC documentation: 33 | 34 | * Include Syntax 35 | * Include Operation 36 | * Once-Only Headers 37 | * Computed Includes 38 | 39 | https://gcc.gnu.org/onlinedocs/cpp/Header-Files.html 40 | -------------------------------------------------------------------------------- /lib/README: -------------------------------------------------------------------------------- 1 | 2 | This directory is intended for project specific (private) libraries. 3 | PlatformIO will compile them to static libraries and link into executable file. 4 | 5 | The source code of each library should be placed in a an own separate directory 6 | ("lib/your_library_name/[here are source files]"). 7 | 8 | For example, see a structure of the following two libraries `Foo` and `Bar`: 9 | 10 | |--lib 11 | | | 12 | | |--Bar 13 | | | |--docs 14 | | | |--examples 15 | | | |--src 16 | | | |- Bar.c 17 | | | |- Bar.h 18 | | | |- library.json (optional, custom build options, etc) https://docs.platformio.org/page/librarymanager/config.html 19 | | | 20 | | |--Foo 21 | | | |- Foo.c 22 | | | |- Foo.h 23 | | | 24 | | |- README --> THIS FILE 25 | | 26 | |- platformio.ini 27 | |--src 28 | |- main.c 29 | 30 | and a contents of `src/main.c`: 31 | ``` 32 | #include 33 | #include 34 | 35 | int main (void) 36 | { 37 | ... 38 | } 39 | 40 | ``` 41 | 42 | PlatformIO Library Dependency Finder will find automatically dependent 43 | libraries scanning project source files. 44 | 45 | More information about PlatformIO Library Dependency Finder 46 | - https://docs.platformio.org/page/librarymanager/ldf.html 47 | -------------------------------------------------------------------------------- /networkLoopback.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import struct 3 | 4 | import numpy as np 5 | import time 6 | 7 | import wave 8 | # import simpleaudio as sa 9 | 10 | sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP) 11 | sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) 12 | sock.bind(('', 4444)) 13 | mreq = struct.pack("=4sl", socket.inet_aton("224.51.105.104"), socket.INADDR_ANY) 14 | 15 | sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq) 16 | 17 | cs = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) 18 | 19 | counter = 0 20 | 21 | # only used if we want to send noise data as a test 22 | noise = [] 23 | while len(noise) < 800: 24 | noise.append(np.uint8(0)) 25 | noise.append(np.uint8(20)) 26 | 27 | # This code can directly send generated noise to ESP32 instead 28 | if False: 29 | print(bytearray(noise)) 30 | while True: 31 | time.sleep(.1) #send 10 packets or 800 samples per second, 8khz 8bit audio wow 32 | cs.sendto(bytearray(noise), ('192.168.1.137', 4445)) 33 | 34 | # This code will stream a looping test 8k8bit wav file 35 | if True: 36 | wav = wave.Wave_read("8bit8k.wav") 37 | while True: 38 | if wav.tell() + 800 > wav.getnframes(): #might drop some frames, but is a test 39 | wav.rewind() 40 | wavbytes = wav.readframes(800) 41 | # play_obj = sa.play_buffer(wavbytes, 1, 1, 8000) #uncomment to play test wav locally 42 | cs.sendto(wavbytes, ('192.168.1.137', 4445)) 43 | time.sleep(.1) #send 10 packets or 800 samples per second, 8khz 8bit audio wow 44 | 45 | # will take data from ESP and loopback back to it 46 | while True: 47 | a = sock.recv(1600) 48 | cs.sendto(a, ('192.168.1.137', 4445)) 49 | # print(len(a)) 50 | print(counter) 51 | counter += 1 52 | if counter % 10 == 0: 53 | print("+SECOND") 54 | -------------------------------------------------------------------------------- /platformio.ini: -------------------------------------------------------------------------------- 1 | ; PlatformIO Project Configuration File 2 | ; 3 | ; Build options: build flags, source filter, extra scripting 4 | ; Upload options: custom port, speed and extra flags 5 | ; Library options: dependencies, extra library storages 6 | ; 7 | ; Please visit documentation for the other options and examples 8 | ; http://docs.platformio.org/page/projectconf.html 9 | 10 | ; [env:esp wrover kit] 11 | ; platform = espressif32 12 | ; framework = arduino 13 | ; board = esp-wrover-kit 14 | ; monitor_speed = 115200 15 | ; build_flags = 16 | ; ; https://docs.espressif.com/projects/esp-idf/en/latest/get-started/get-started-wrover-kit.html#rgb-led 17 | ; -D LED_BUILTIN=2 18 | 19 | ; [env:esp32doit-devkit-v1] 20 | ; platform = espressif32 21 | ; framework = arduino 22 | ; board = esp32doit-devkit-v1 23 | ; monitor_speed = 115200 24 | 25 | [env:lolin32] 26 | platform = espressif32 27 | framework = arduino 28 | board = lolin32 29 | monitor_speed = 115200 30 | upload_speed = 921600 31 | upload_port=COM7 32 | monitor_port=COM7 -------------------------------------------------------------------------------- /src/main.cpp: -------------------------------------------------------------------------------- 1 | /* Duplex ESP32 audio streamer 2 | */ 3 | 4 | // TODO would be great to read and fill mic buffer from the ULP just bc its sitting there and should be fast enough 5 | 6 | #include 7 | #include 8 | #include 9 | #include 10 | #include 11 | 12 | #define AUDIO_BUFFER_MAX 8000 // maximum allowable audio playback buffer 13 | #define AUDIO_BUFFER_TRANSMIT_PACKET_SIZE 800 // how many samples to capture before transmitting 14 | #define AUDIO_BUFFER_SIZE_WAIT_TO_PLAY 800 // how many packets to buffer before playback starts 15 | 16 | uint8_t audioMicCollectBuffer[AUDIO_BUFFER_TRANSMIT_PACKET_SIZE]; 17 | uint8_t audioMicTransmitBuffer[AUDIO_BUFFER_TRANSMIT_PACKET_SIZE]; 18 | int audioMicBufferIndex = 0; 19 | int debugpacketcounter = 0; 20 | 21 | const char *ssid = "Peas For Our Time"; 22 | const char *password = "lovenotwar"; 23 | const char *host = "192.168.1.255"; //broadcast to subnet audio data to 24 | #define portsend 4444 25 | #define portrecv 4445 26 | 27 | bool audioMicTransmitNow = false; 28 | 29 | uint8_t audioOutputNetworkBuffer[AUDIO_BUFFER_TRANSMIT_PACKET_SIZE]; 30 | uint8_t audioOutputPlaybackBuffer[AUDIO_BUFFER_MAX]; 31 | bool recieveBufferFull = false; 32 | int audioOutputReadIndex = 0, audioOutputWriteIndex = 1; 33 | int audioDataInPlaybackBuffer = 0; 34 | bool play = false; 35 | 36 | AsyncUDP udpSend; 37 | AsyncUDP udpRec; 38 | 39 | hw_timer_t *mictimer = NULL; // our microphone timer 40 | hw_timer_t *playbacktimer = NULL; // our playback timer 41 | portMUX_TYPE timerMux = portMUX_INITIALIZER_UNLOCKED; 42 | 43 | /////////// Audio Code -- runs on seperate core /////////// 44 | TaskHandle_t AudioTask; 45 | 46 | void ReadMicInput() 47 | { 48 | uint16_t adcVal = adc1_get_raw(ADC1_CHANNEL_0); // reads the ADC, LOLIN32 pinout marked VP on board, GPIO36 49 | uint8_t value = map(adcVal, 0, 4096, 0, 255); // converts the value to 0..255 (8bit) 50 | // TODO transmit all 12bits for great great audio 51 | 52 | audioMicCollectBuffer[audioMicBufferIndex] = value; // stores the value 53 | audioMicBufferIndex++; 54 | 55 | if (audioMicBufferIndex == AUDIO_BUFFER_TRANSMIT_PACKET_SIZE) 56 | { // when the buffer is full 57 | portENTER_CRITICAL_ISR(&timerMux); // says that we want to run critical code and don't want to be interrupted 58 | audioMicBufferIndex = 0; 59 | memcpy(audioMicTransmitBuffer, audioMicCollectBuffer, AUDIO_BUFFER_TRANSMIT_PACKET_SIZE); // copy buffer into a second buffer 60 | audioMicTransmitNow = true; // sets the value true so we know that we can transmit now 61 | portEXIT_CRITICAL_ISR(&timerMux); // says that we have run our critical code 62 | } 63 | } 64 | void CopyToOutputBuffer(int length) // copies from network to speaker buffer 65 | { 66 | if (audioDataInPlaybackBuffer + length < AUDIO_BUFFER_MAX) // if our buffer isn't already full TODO drop oldest data instead 67 | { 68 | for (int i = 0; i < length; i++) 69 | { 70 | audioOutputPlaybackBuffer[audioOutputWriteIndex] = audioOutputNetworkBuffer[i]; 71 | audioOutputWriteIndex++; 72 | if (audioOutputWriteIndex == AUDIO_BUFFER_MAX) 73 | audioOutputWriteIndex = 0; 74 | } 75 | audioDataInPlaybackBuffer += length; 76 | } 77 | else 78 | { 79 | Serial.println("Buffer overflow? TOO MUCH DATA WOW"); 80 | } 81 | 82 | if (audioDataInPlaybackBuffer > AUDIO_BUFFER_SIZE_WAIT_TO_PLAY) 83 | play = true; 84 | } 85 | void PlaybackAudio() 86 | { 87 | if (recieveBufferFull) 88 | { 89 | portENTER_CRITICAL_ISR(&timerMux); // says that we want to run critical code and don't want to be interrupted 90 | CopyToOutputBuffer(AUDIO_BUFFER_TRANSMIT_PACKET_SIZE); 91 | recieveBufferFull = false; 92 | portEXIT_CRITICAL_ISR(&timerMux); // says that we have run our critical code 93 | } 94 | if (play) 95 | { 96 | dac_output_voltage(DAC_CHANNEL_1, audioOutputPlaybackBuffer[audioOutputReadIndex]); // DAC 1 is GPIO 25 on Lolin32 97 | 98 | audioOutputReadIndex++; 99 | if (audioOutputReadIndex == AUDIO_BUFFER_MAX) 100 | { 101 | audioOutputReadIndex = 0; 102 | } 103 | 104 | audioDataInPlaybackBuffer -= 1; 105 | if (audioDataInPlaybackBuffer == 0) 106 | { 107 | // TODO this still happens rarely, should switch to a TCP protocol? Or bad parallel work? 108 | Serial.print("Buffer underrun!!! writeP,readp,packets:"); 109 | Serial.print(audioOutputWriteIndex); 110 | Serial.print(" "); 111 | Serial.print(audioOutputReadIndex); 112 | Serial.print(" "); 113 | Serial.println(debugpacketcounter); 114 | debugpacketcounter = 0; 115 | play = false; 116 | } 117 | } 118 | } 119 | void IRAM_ATTR MicInterupt() 120 | { 121 | ReadMicInput(); 122 | } 123 | void IRAM_ATTR PlaybackInterupt() 124 | { 125 | PlaybackAudio(); 126 | } 127 | void AudioCore(void *pvParameters) 128 | { 129 | mictimer = timerBegin(0, 80, true); // 80 Prescaler, hw timer 1. makes timer have 1us scale 130 | playbacktimer = timerBegin(1, 80, true); // 80 Prescaler, hw timer 0. makes timer have 1us scale? 131 | timerAttachInterrupt(mictimer, &MicInterupt, true); // binds the handling function to our timer 132 | timerAttachInterrupt(playbacktimer, &PlaybackInterupt, true); // binds the handling function to our timer 133 | timerAlarmWrite(mictimer, 125, true); // wake every 125us AKA 8khz sampling rate 134 | timerAlarmWrite(playbacktimer, 125, true); // same playback rate as capture -- could be independent though 135 | timerAlarmEnable(mictimer); 136 | timerAlarmEnable(playbacktimer); 137 | 138 | while (true) 139 | { 140 | delay(4900); // keeps watchdog from triggering but essentially useless since we're all on interupts on the audio core 141 | } 142 | } 143 | 144 | // int DEBUGCOUNTER = 0; 145 | /////////// END Audio Code /////////// 146 | void setup() 147 | { 148 | 149 | Serial.begin(115200); 150 | 151 | WiFi.mode(WIFI_STA); 152 | WiFi.begin(ssid, password); 153 | 154 | pinMode(LED_BUILTIN, OUTPUT); 155 | digitalWrite(LED_BUILTIN, LOW); 156 | 157 | dac_output_enable(DAC_CHANNEL_1); 158 | dac_output_enable(DAC_CHANNEL_2); 159 | 160 | Serial.println("connecting to wifi"); 161 | while (WiFi.status() != WL_CONNECTED) 162 | { 163 | delay(500); 164 | Serial.print("."); 165 | } 166 | 167 | Serial.println("MY IP address: "); 168 | Serial.println(WiFi.localIP()); 169 | 170 | adc1_config_width(ADC_WIDTH_12Bit); // configure the analogue to digital converter 171 | adc1_config_channel_atten(ADC1_CHANNEL_0, ADC_ATTEN_0db); // connects the ADC 1 with channel 0 (GPIO 36, VP pin LOLIN32) 172 | 173 | udpSend.connect(IPAddress(192, 168, 1, 255), portsend); // have to initialize to use 174 | 175 | udpRec.listen(portrecv); 176 | 177 | // long gross recvieve function 178 | 179 | udpRec.onPacket([](AsyncUDPPacket packet) { // CANNOT insert a function here that references packet directly, will cause dereft bug/crash 180 | size_t length = packet.length(); 181 | uint8_t *data = packet.data(); 182 | ++debugpacketcounter; 183 | // portENTER_CRITICAL_ISR(&timerMux); // says that we want to run critical code and don't want to be interrupted 184 | if (recieveBufferFull) 185 | Serial.println("Network backup detected -- packet dropped"); 186 | else 187 | { 188 | memcpy(audioOutputNetworkBuffer, data, length); 189 | recieveBufferFull = true; 190 | } 191 | // portEXIT_CRITICAL_ISR(&timerMux); // says that we have run our critical code 192 | // Serial.println(DEBUGCOUNTER); 193 | // ++DEBUGCOUNTER; 194 | // if (DEBUGCOUNTER % 10 == 0) 195 | // { 196 | // // DEBUGCOUNTER = 0; 197 | // Serial.println("second"); 198 | // } 199 | // CopyToOutputBuffer(length); 200 | }); 201 | /// end recv function 202 | 203 | xTaskCreatePinnedToCore( 204 | AudioCore, /* Task function. */ 205 | "AudioCoreTask", /* name of task. */ 206 | 10000, /* Stack size of task */ 207 | NULL, /* parameter of the task */ 208 | 1, /* priority of the task */ 209 | &AudioTask, /* Task handle to keep track of created task */ 210 | 0); /* pin task to core 0 */ 211 | } 212 | 213 | // stupid little function to let us test the ADC as a "real audio signal" to send out 214 | bool noiseflip = true; 215 | void GenNoiseDAC2() 216 | { 217 | noiseflip = !noiseflip; 218 | if (noiseflip) 219 | dac_output_voltage(DAC_CHANNEL_2, 0); // DAC 2 is GPIO 26 on Lolin32 220 | else 221 | dac_output_voltage(DAC_CHANNEL_2, 255); // DAC 2 is GPIO 26 on Lolin32, 255 is 8 bit max signal 222 | } 223 | 224 | void loop() 225 | { 226 | if (audioMicTransmitNow) 227 | { // checks if the buffer is full and sends if so 228 | udpSend.broadcastTo(audioMicTransmitBuffer, AUDIO_BUFFER_TRANSMIT_PACKET_SIZE, portsend); 229 | audioMicTransmitNow = false; 230 | // Serial.print("Sent. Buffsize:"); 231 | // Serial.println(audioDataInPlaybackBuffer); 232 | } 233 | GenNoiseDAC2(); // test tool for closed loop noise test 234 | } 235 | --------------------------------------------------------------------------------