├── .github
└── FUNDING.yml
├── README.md
├── firmware
├── .gitignore
├── .vscode
│ └── extensions.json
├── README.md
├── data
│ └── index.html
├── include
│ └── README
├── lib
│ ├── README
│ └── audio_input
│ │ └── src
│ │ ├── ADCSampler.cpp
│ │ ├── ADCSampler.h
│ │ ├── I2SMEMSSampler.cpp
│ │ ├── I2SMEMSSampler.h
│ │ ├── I2SSampler.cpp
│ │ └── I2SSampler.h
├── platformio.ini
├── src
│ ├── Application.cpp
│ ├── Application.h
│ ├── config.cpp
│ ├── config.h
│ ├── main.cpp
│ └── transports
│ │ ├── TCPSocketTransport.cpp
│ │ ├── TCPSocketTransport.h
│ │ ├── Transport.h
│ │ ├── WebSocketTransport.cpp
│ │ └── WebSocketTransport.h
└── test
│ └── README
└── player
├── .gitignore
├── .python-version
├── README.md
├── main.py
└── requirements.txt
/.github/FUNDING.yml:
--------------------------------------------------------------------------------
1 | # These are supported funding model platforms
2 |
3 | github: [atomic14]
4 | ko_fi: atomic14
5 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Intro
2 |
3 | You can watch a video of this working here: https://youtu.be/0jR-QNTfydA at around 4:23
4 |
5 | [](https://www.youtube.com/watch?v=0jR-QNTfydA)
6 |
7 | The `firmware` folder contains the ESP32 code that will stream audio over websockets or TCP sockets.
8 |
9 | The `player` folder contains a simple Python program that will receive audio from the ESP32 and either play it or record it to a WAV file.
10 |
--------------------------------------------------------------------------------
/firmware/.gitignore:
--------------------------------------------------------------------------------
1 | .pio
2 | .vscode/.browse.c_cpp.db*
3 | .vscode/c_cpp_properties.json
4 | .vscode/launch.json
5 | .vscode/ipch
6 |
--------------------------------------------------------------------------------
/firmware/.vscode/extensions.json:
--------------------------------------------------------------------------------
1 | {
2 | // See http://go.microsoft.com/fwlink/?LinkId=827846
3 | // for the documentation about the extensions.json format
4 | "recommendations": [
5 | "platformio.platformio-ide"
6 | ]
7 | }
8 |
--------------------------------------------------------------------------------
/firmware/README.md:
--------------------------------------------------------------------------------
1 | # Intro
2 |
3 | [](https://ko-fi.com/Z8Z734F5Y)
4 |
5 | # Intro
6 |
7 | You can watch a video of this working here: https://youtu.be/0jR-QNTfydA at around 4:23
8 |
9 | [](https://www.youtube.com/watch?v=0jR-QNTfydA)
10 |
11 | # Getting going
12 |
13 | Open this folder using Platform IO and upload the firmware. You'll also need to upload the filesystem image.
14 |
15 | Once it's running, you can either go to http://microphone.local to get the audio over websockets. Or you can use the Python script in the `player` folder.
16 |
--------------------------------------------------------------------------------
/firmware/data/index.html:
--------------------------------------------------------------------------------
1 |
2 |
3 |
60 |
61 |
62 |
63 |
64 |
65 |
66 |
67 |
68 |
69 |
70 |
71 |
298 |
299 |
300 |
--------------------------------------------------------------------------------
/firmware/include/README:
--------------------------------------------------------------------------------
1 |
2 | This directory is intended for project header files.
3 |
4 | A header file is a file containing C declarations and macro definitions
5 | to be shared between several project source files. You request the use of a
6 | header file in your project source file (C, C++, etc) located in `src` folder
7 | by including it, with the C preprocessing directive `#include'.
8 |
9 | ```src/main.c
10 |
11 | #include "header.h"
12 |
13 | int main (void)
14 | {
15 | ...
16 | }
17 | ```
18 |
19 | Including a header file produces the same results as copying the header file
20 | into each source file that needs it. Such copying would be time-consuming
21 | and error-prone. With a header file, the related declarations appear
22 | in only one place. If they need to be changed, they can be changed in one
23 | place, and programs that include the header file will automatically use the
24 | new version when next recompiled. The header file eliminates the labor of
25 | finding and changing all the copies as well as the risk that a failure to
26 | find one copy will result in inconsistencies within a program.
27 |
28 | In C, the usual convention is to give header files names that end with `.h'.
29 | It is most portable to use only letters, digits, dashes, and underscores in
30 | header file names, and at most one dot.
31 |
32 | Read more about using header files in official GCC documentation:
33 |
34 | * Include Syntax
35 | * Include Operation
36 | * Once-Only Headers
37 | * Computed Includes
38 |
39 | https://gcc.gnu.org/onlinedocs/cpp/Header-Files.html
40 |
--------------------------------------------------------------------------------
/firmware/lib/README:
--------------------------------------------------------------------------------
1 |
2 | This directory is intended for project specific (private) libraries.
3 | PlatformIO will compile them to static libraries and link into executable file.
4 |
5 | The source code of each library should be placed in a an own separate directory
6 | ("lib/your_library_name/[here are source files]").
7 |
8 | For example, see a structure of the following two libraries `Foo` and `Bar`:
9 |
10 | |--lib
11 | | |
12 | | |--Bar
13 | | | |--docs
14 | | | |--examples
15 | | | |--src
16 | | | |- Bar.c
17 | | | |- Bar.h
18 | | | |- library.json (optional, custom build options, etc) https://docs.platformio.org/page/librarymanager/config.html
19 | | |
20 | | |--Foo
21 | | | |- Foo.c
22 | | | |- Foo.h
23 | | |
24 | | |- README --> THIS FILE
25 | |
26 | |- platformio.ini
27 | |--src
28 | |- main.c
29 |
30 | and a contents of `src/main.c`:
31 | ```
32 | #include
33 | #include
34 |
35 | int main (void)
36 | {
37 | ...
38 | }
39 |
40 | ```
41 |
42 | PlatformIO Library Dependency Finder will find automatically dependent
43 | libraries scanning project source files.
44 |
45 | More information about PlatformIO Library Dependency Finder
46 | - https://docs.platformio.org/page/librarymanager/ldf.html
47 |
--------------------------------------------------------------------------------
/firmware/lib/audio_input/src/ADCSampler.cpp:
--------------------------------------------------------------------------------
1 | #include "ADCSampler.h"
2 |
3 | ADCSampler::ADCSampler(adc_unit_t adcUnit, adc1_channel_t adcChannel, const i2s_config_t &i2s_config) : I2SSampler(I2S_NUM_0, i2s_config)
4 | {
5 | m_adcUnit = adcUnit;
6 | m_adcChannel = adcChannel;
7 | }
8 |
9 | void ADCSampler::configureI2S()
10 | {
11 | //init ADC pad
12 | i2s_set_adc_mode(m_adcUnit, m_adcChannel);
13 | // enable the adc
14 | i2s_adc_enable(m_i2sPort);
15 | }
16 |
17 | void ADCSampler::unConfigureI2S()
18 | {
19 | // make sure ot do this or the ADC is locked
20 | i2s_adc_disable(m_i2sPort);
21 | }
22 |
23 | int ADCSampler::read(int16_t *samples, int count)
24 | {
25 | // read from i2s
26 | size_t bytes_read = 0;
27 | i2s_read(m_i2sPort, samples, sizeof(int16_t) * count, &bytes_read, portMAX_DELAY);
28 | int samples_read = bytes_read / sizeof(int16_t);
29 | for (int i = 0; i < samples_read; i++)
30 | {
31 | samples[i] = (2048 - (uint16_t(samples[i]) & 0xfff)) * 15;
32 | }
33 | return samples_read;
34 | }
35 |
--------------------------------------------------------------------------------
/firmware/lib/audio_input/src/ADCSampler.h:
--------------------------------------------------------------------------------
1 | #pragma once
2 |
3 | #include "I2SSampler.h"
4 |
5 | class ADCSampler : public I2SSampler
6 | {
7 | private:
8 | adc_unit_t m_adcUnit;
9 | adc1_channel_t m_adcChannel;
10 |
11 | protected:
12 | void configureI2S();
13 | void unConfigureI2S();
14 |
15 | public:
16 | ADCSampler(adc_unit_t adc_unit, adc1_channel_t adc_channel, const i2s_config_t &i2s_config);
17 | virtual int read(int16_t *samples, int count);
18 | };
19 |
--------------------------------------------------------------------------------
/firmware/lib/audio_input/src/I2SMEMSSampler.cpp:
--------------------------------------------------------------------------------
1 | #include "I2SMEMSSampler.h"
2 | #include "soc/i2s_reg.h"
3 |
4 | I2SMEMSSampler::I2SMEMSSampler(
5 | i2s_port_t i2s_port,
6 | i2s_pin_config_t &i2s_pins,
7 | i2s_config_t i2s_config,
8 | bool fixSPH0645) : I2SSampler(i2s_port, i2s_config)
9 | {
10 | m_i2sPins = i2s_pins;
11 | m_fixSPH0645 = fixSPH0645;
12 | }
13 |
14 | void I2SMEMSSampler::configureI2S()
15 | {
16 | if (m_fixSPH0645)
17 | {
18 | // FIXES for SPH0645
19 | REG_SET_BIT(I2S_TIMING_REG(m_i2sPort), BIT(9));
20 | REG_SET_BIT(I2S_CONF_REG(m_i2sPort), I2S_RX_MSB_SHIFT);
21 | }
22 |
23 | i2s_set_pin(m_i2sPort, &m_i2sPins);
24 | }
25 |
26 | int I2SMEMSSampler::read(int16_t *samples, int count)
27 | {
28 | // read from i2s
29 | int32_t *raw_samples = (int32_t *)malloc(sizeof(int32_t) * count);
30 | size_t bytes_read = 0;
31 | i2s_read(m_i2sPort, raw_samples, sizeof(int32_t) * count, &bytes_read, portMAX_DELAY);
32 | int samples_read = bytes_read / sizeof(int32_t);
33 | for (int i = 0; i < samples_read; i++)
34 | {
35 | samples[i] = (raw_samples[i] & 0xFFFFFFF0) >> 11;
36 | }
37 | free(raw_samples);
38 | return samples_read;
39 | }
40 |
--------------------------------------------------------------------------------
/firmware/lib/audio_input/src/I2SMEMSSampler.h:
--------------------------------------------------------------------------------
1 | #pragma once
2 |
3 | #include "I2SSampler.h"
4 |
5 | class I2SMEMSSampler : public I2SSampler
6 | {
7 | private:
8 | i2s_pin_config_t m_i2sPins;
9 | bool m_fixSPH0645;
10 |
11 | protected:
12 | void configureI2S();
13 |
14 | public:
15 | I2SMEMSSampler(
16 | i2s_port_t i2s_port,
17 | i2s_pin_config_t &i2s_pins,
18 | i2s_config_t i2s_config,
19 | bool fixSPH0645 = false);
20 | virtual int read(int16_t *samples, int count);
21 | };
22 |
--------------------------------------------------------------------------------
/firmware/lib/audio_input/src/I2SSampler.cpp:
--------------------------------------------------------------------------------
1 |
2 | #include "I2SSampler.h"
3 | #include "driver/i2s.h"
4 |
5 | I2SSampler::I2SSampler(i2s_port_t i2sPort, const i2s_config_t &i2s_config) : m_i2sPort(i2sPort), m_i2s_config(i2s_config)
6 | {
7 | }
8 |
9 | void I2SSampler::start()
10 | {
11 | //install and start i2s driver
12 | i2s_driver_install(m_i2sPort, &m_i2s_config, 0, NULL);
13 | // set up the I2S configuration from the subclass
14 | configureI2S();
15 | }
16 |
17 | void I2SSampler::stop()
18 | {
19 | // clear any I2S configuration
20 | unConfigureI2S();
21 | // stop the i2S driver
22 | i2s_driver_uninstall(m_i2sPort);
23 | }
24 |
--------------------------------------------------------------------------------
/firmware/lib/audio_input/src/I2SSampler.h:
--------------------------------------------------------------------------------
1 | #pragma once
2 |
3 | #include
4 | #include
5 |
6 | /**
7 | * Base Class for both the ADC and I2S sampler
8 | **/
9 | class I2SSampler
10 | {
11 | protected:
12 | i2s_port_t m_i2sPort = I2S_NUM_0;
13 | i2s_config_t m_i2s_config;
14 | virtual void configureI2S() = 0;
15 | virtual void unConfigureI2S(){};
16 | virtual void processI2SData(void *samples, size_t count){
17 | // nothing to do for the default case
18 | };
19 |
20 | public:
21 | I2SSampler(i2s_port_t i2sPort, const i2s_config_t &i2sConfig);
22 | void start();
23 | virtual int read(int16_t *samples, int count) = 0;
24 | void stop();
25 | int sample_rate()
26 | {
27 | return m_i2s_config.sample_rate;
28 | }
29 | };
30 |
--------------------------------------------------------------------------------
/firmware/platformio.ini:
--------------------------------------------------------------------------------
1 | ; PlatformIO Project Configuration File
2 | ;
3 | ; Build options: build flags, source filter
4 | ; Upload options: custom upload port, speed and extra flags
5 | ; Library options: dependencies, extra library storages
6 | ; Advanced options: extra scripting
7 | ;
8 | ; Please visit documentation for the other options and examples
9 | ; https://docs.platformio.org/page/projectconf.html
10 |
11 | [env:esp32dev]
12 | platform = espressif32
13 | board = esp32dev
14 | framework = arduino
15 | upload_port = /dev/cu.SLAB_USBtoUART
16 | monitor_port = /dev/cu.SLAB_USBtoUART
17 | monitor_speed = 115200
18 | monitor_filters = esp32_exception_decoder
--------------------------------------------------------------------------------
/firmware/src/Application.cpp:
--------------------------------------------------------------------------------
1 | #include
2 | #include "Application.h"
3 | #include
4 | #include
5 | #include "transports/WebSocketTransport.h"
6 | #include "transports/TCPSocketTransport.h"
7 | #include "config.h"
8 |
9 | void Application::begin()
10 | {
11 | #ifdef USE_I2S_MIC_INPUT
12 | this->input = new I2SMEMSSampler(I2S_NUM_0, i2s_mic_pins, i2s_mic_Config);
13 | #else
14 | this->input = new ADCSampler(ADC_UNIT_1, ADC1_CHANNEL_7, i2s_adc_config);
15 | #endif
16 |
17 | this->input = input;
18 | this->transport1 = new WebSocketTransport();
19 | this->transport2 = new TCPSocketTransport();
20 | this->input->start();
21 | this->transport1->begin();
22 | this->transport2->begin();
23 | TaskHandle_t task_handle;
24 | xTaskCreate(Application::streamer_task, "task", 8192, this, 0, &task_handle);
25 | }
26 |
27 | void Application::streamer_task(void *param)
28 | {
29 | Application *app = (Application *)param;
30 | // now just read from the microphone and send to the clients
31 | int16_t *samples = (int16_t *)malloc(sizeof(int16_t) * 1024);
32 | while (true)
33 | {
34 | // read from the microphone
35 | int samples_read = app->input->read(samples, 1024);
36 | // send to the two transports
37 | app->transport1->send(samples, samples_read * sizeof(int16_t));
38 | app->transport2->send(samples, samples_read * sizeof(int16_t));
39 | }
40 | }
41 |
--------------------------------------------------------------------------------
/firmware/src/Application.h:
--------------------------------------------------------------------------------
1 | #pragma once
2 |
3 | class Transport;
4 | class I2SSampler;
5 |
6 | class Application
7 | {
8 | private:
9 | Transport *transport1 = NULL;
10 | Transport *transport2 = NULL;
11 | I2SSampler *input = NULL;
12 |
13 | public:
14 | void begin();
15 | static void streamer_task(void *param);
16 | };
--------------------------------------------------------------------------------
/firmware/src/config.cpp:
--------------------------------------------------------------------------------
1 | #include "config.h"
2 |
3 | // i2s config for using the internal ADC
4 | i2s_config_t i2s_adc_config = {
5 | .mode = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_RX | I2S_MODE_ADC_BUILT_IN),
6 | .sample_rate = SAMPLE_RATE,
7 | .bits_per_sample = I2S_BITS_PER_SAMPLE_16BIT,
8 | .channel_format = I2S_CHANNEL_FMT_ONLY_LEFT,
9 | .communication_format = I2S_COMM_FORMAT_I2S_LSB,
10 | .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1,
11 | .dma_buf_count = 10,
12 | .dma_buf_len = 1024,
13 | .use_apll = false,
14 | .tx_desc_auto_clear = false,
15 | .fixed_mclk = 0};
16 |
17 | // i2s config for reading from I2S
18 | i2s_config_t i2s_mic_Config = {
19 | .mode = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_RX),
20 | .sample_rate = SAMPLE_RATE,
21 | .bits_per_sample = I2S_BITS_PER_SAMPLE_32BIT,
22 | .channel_format = I2S_CHANNEL_FMT_ONLY_LEFT,
23 | .communication_format = I2S_COMM_FORMAT_I2S,
24 | .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1,
25 | .dma_buf_count = 10,
26 | .dma_buf_len = 1024,
27 | .use_apll = false,
28 | .tx_desc_auto_clear = false,
29 | .fixed_mclk = 0};
30 |
31 | // i2s microphone pins
32 | i2s_pin_config_t i2s_mic_pins = {
33 | .bck_io_num = I2S_MIC_SERIAL_CLOCK,
34 | .ws_io_num = I2S_MIC_LEFT_RIGHT_CLOCK,
35 | .data_out_num = I2S_PIN_NO_CHANGE,
36 | .data_in_num = I2S_MIC_SERIAL_DATA};
37 |
--------------------------------------------------------------------------------
/firmware/src/config.h:
--------------------------------------------------------------------------------
1 | #pragma once
2 |
3 | #include
4 | #include
5 |
6 | // network config
7 | #define WIFI_SSID <>
8 | #define WIFI_PASSWORD <>
9 | #define MDNS_DOMAIN "microphone"
10 |
11 | // save to SPIFFS instead of SD Card?
12 | // #define USE_SPIFFS 1
13 |
14 | // sample rate for the system
15 | #define SAMPLE_RATE 44100
16 |
17 | // are you using an I2S microphone - comment this out if you want to use an analog mic and ADC input
18 | #define USE_I2S_MIC_INPUT
19 |
20 | // I2S Microphone Settings
21 | // Which channel is the I2S microphone on? I2S_CHANNEL_FMT_ONLY_LEFT or I2S_CHANNEL_FMT_ONLY_RIGHT
22 | // Generally they will default to LEFT - but you may need to attach the L/R pin to GND
23 | #define I2S_MIC_CHANNEL I2S_CHANNEL_FMT_ONLY_LEFT
24 | // #define I2S_MIC_CHANNEL I2S_CHANNEL_FMT_ONLY_RIGHT
25 | #define I2S_MIC_SERIAL_CLOCK GPIO_NUM_26
26 | #define I2S_MIC_LEFT_RIGHT_CLOCK GPIO_NUM_22
27 | #define I2S_MIC_SERIAL_DATA GPIO_NUM_21
28 |
29 | // Analog Microphone Settings - ADC1_CHANNEL_7 is GPIO35
30 | #define ADC_MIC_CHANNEL ADC1_CHANNEL_7
31 |
32 | // i2s config for using the internal ADC
33 | extern i2s_config_t i2s_adc_config;
34 | // i2s config for reading from of I2S
35 | extern i2s_config_t i2s_mic_Config;
36 | // i2s microphone pins
37 | extern i2s_pin_config_t i2s_mic_pins;
38 |
--------------------------------------------------------------------------------
/firmware/src/main.cpp:
--------------------------------------------------------------------------------
1 | #include
2 | #include
3 | #include
4 | #include "Application.h"
5 | #include "config.h"
6 |
7 | void setup()
8 | {
9 | Serial.begin(115200);
10 | Serial.println("Starting up");
11 | delay(1000);
12 | WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
13 | while (WiFi.status() != WL_CONNECTED)
14 | {
15 | delay(500);
16 | Serial.print(".");
17 | }
18 | // disable WiFi sleep mode
19 | WiFi.setSleep(WIFI_PS_NONE);
20 |
21 | Serial.println("");
22 | Serial.println("WiFi connected");
23 | Serial.println("IP address: ");
24 | Serial.print(WiFi.localIP());
25 | Serial.println("");
26 |
27 | // startup MDNS
28 | if (!MDNS.begin(MDNS_DOMAIN))
29 | {
30 | Serial.println("MDNS.begin failed");
31 | }
32 | Serial.println("Creating microphone");
33 | Application *application = new Application();
34 | application->begin();
35 | }
36 |
37 | void loop()
38 | {
39 | vTaskDelay(pdMS_TO_TICKS(1000));
40 | }
--------------------------------------------------------------------------------
/firmware/src/transports/TCPSocketTransport.cpp:
--------------------------------------------------------------------------------
1 | #include
2 | #include "TCPSocketTransport.h"
3 |
4 | void TCPSocketTransport::begin()
5 | {
6 | Serial.println("Connect to TCP socket microphone.local:9090 to try out TCP socket streaming");
7 | server = new WiFiServer(9090);
8 | server->begin();
9 | }
10 |
11 | void TCPSocketTransport::send(void *data, size_t len)
12 | {
13 | // get any new connections
14 | WiFiClient client = server->available();
15 | if (client)
16 | {
17 | Serial.println("New Client");
18 | // add to the list of clients
19 | for (int i = 0; i < MAX_CLIENTS; i++)
20 | {
21 | if (NULL == clients[i])
22 | {
23 | clients[i] = new WiFiClient(client);
24 | break;
25 | }
26 | }
27 | }
28 | // send the audio data to any clients
29 | for (int i = 0; i < MAX_CLIENTS; i++)
30 | {
31 | if (clients[i] != NULL && (*clients[i]))
32 | {
33 | // send the samples to the client
34 | clients[i]->write((uint8_t *)data, len);
35 | }
36 | else
37 | {
38 | // client has gone away, remove it from the list
39 | clients[i] = NULL;
40 | }
41 | }
42 | }
--------------------------------------------------------------------------------
/firmware/src/transports/TCPSocketTransport.h:
--------------------------------------------------------------------------------
1 | #pragma once
2 | #include "Transport.h"
3 |
4 | class WiFiClient;
5 | class WiFiServer;
6 | #define MAX_CLIENTS 10
7 |
8 | class TCPSocketTransport : public Transport
9 | {
10 | private:
11 | WiFiServer *server = NULL;
12 | WiFiClient *clients[MAX_CLIENTS] = {NULL};
13 |
14 | public:
15 | void begin();
16 | void send(void *data, size_t size) override;
17 | };
--------------------------------------------------------------------------------
/firmware/src/transports/Transport.h:
--------------------------------------------------------------------------------
1 | #pragma once
2 |
3 | class Transport
4 | {
5 | public:
6 | virtual void begin() = 0;
7 | virtual void send(void *data, size_t bytes) = 0;
8 | };
--------------------------------------------------------------------------------
/firmware/src/transports/WebSocketTransport.cpp:
--------------------------------------------------------------------------------
1 | #include
2 | #include
3 | #include "WebSocketTransport.h"
4 |
5 | void WebSocketTransport::begin()
6 | {
7 | Serial.printf("Connect to http://microphone.local to try out web socket streaming");
8 | server = new AsyncWebServer(80);
9 | ws = new AsyncWebSocket("/audio_stream");
10 | // start off spiffs so we can serve the static files
11 | if (!SPIFFS.begin())
12 | {
13 | Serial.println("SPIFFS.begin failed");
14 | }
15 | else
16 | {
17 | Serial.println("SPIFFS mounted");
18 | }
19 | // setup cors to allow connections from anywhere - handy for testing from localhost
20 | DefaultHeaders::Instance().addHeader("Access-Control-Allow-Origin", "*");
21 | DefaultHeaders::Instance().addHeader("Access-Control-Allow-Methods", "GET, PUT");
22 | DefaultHeaders::Instance().addHeader("Access-Control-Allow-Headers", "*");
23 | // serve up the index.html file
24 | server->serveStatic("/", SPIFFS, "/").setDefaultFile("index.html");
25 | // setup ws server
26 | ws->onEvent([this](AsyncWebSocket *server, AsyncWebSocketClient *client, AwsEventType type, void *arg, uint8_t *data, size_t len)
27 | {
28 | if (type == WS_EVT_CONNECT)
29 | {
30 | Serial.printf("ws[%s][%u] connect\n", server->url(), client->id());
31 | this->connected_client = client;
32 | }
33 | else if (type == WS_EVT_DISCONNECT)
34 | {
35 | Serial.printf("ws[%s][%u] disconnect\n", server->url(), client->id());
36 | this->connected_client = NULL;
37 | }
38 | else if (type == WS_EVT_ERROR)
39 | {
40 | Serial.printf("ws[%s][%u] error(%u): %s\n", server->url(), client->id(), *((uint16_t *)arg), (char *)data);
41 | }
42 | });
43 | server->addHandler(ws);
44 | // handle options requests for cors or not found
45 | server->onNotFound([](AsyncWebServerRequest *request)
46 | {
47 | if (request->method() == HTTP_OPTIONS)
48 | {
49 | request->send(200);
50 | }
51 | else
52 | {
53 | Serial.println("Not found");
54 | request->send(404, "Not found");
55 | }
56 | });
57 |
58 | server->begin();
59 | }
60 |
61 | void WebSocketTransport::send(void *data, size_t len)
62 | {
63 | // This seems to have a lot of issues...
64 | // ws->binaryAll((uint8_t *)data, len);
65 | // so we'll stick to just the one client...
66 | if (connected_client && connected_client->canSend())
67 | {
68 | connected_client->binary(reinterpret_cast(data), len);
69 | }
70 | }
--------------------------------------------------------------------------------
/firmware/src/transports/WebSocketTransport.h:
--------------------------------------------------------------------------------
1 | #pragma once
2 | #include "Transport.h"
3 |
4 | class AsyncWebServer;
5 | class AsyncWebSocket;
6 | class AsyncWebSocketClient;
7 |
8 | class WebSocketTransport : public Transport
9 | {
10 | private:
11 | AsyncWebServer *server = NULL;
12 | AsyncWebSocket *ws = NULL;
13 | // allow for one client at a time
14 | AsyncWebSocketClient *connected_client = NULL;
15 |
16 | public:
17 | void begin();
18 | void send(void *data, size_t size) override;
19 | };
--------------------------------------------------------------------------------
/firmware/test/README:
--------------------------------------------------------------------------------
1 |
2 | This directory is intended for PlatformIO Unit Testing and project tests.
3 |
4 | Unit Testing is a software testing method by which individual units of
5 | source code, sets of one or more MCU program modules together with associated
6 | control data, usage procedures, and operating procedures, are tested to
7 | determine whether they are fit for use. Unit testing finds problems early
8 | in the development cycle.
9 |
10 | More information about PlatformIO Unit Testing:
11 | - https://docs.platformio.org/page/plus/unit-testing.html
12 |
--------------------------------------------------------------------------------
/player/.gitignore:
--------------------------------------------------------------------------------
1 | # Byte-compiled / optimized / DLL files
2 | __pycache__/
3 | *.py[cod]
4 | *$py.class
5 |
6 | # C extensions
7 | *.so
8 |
9 | # Distribution / packaging
10 | .Python
11 | build/
12 | develop-eggs/
13 | dist/
14 | downloads/
15 | eggs/
16 | .eggs/
17 | lib/
18 | lib64/
19 | parts/
20 | sdist/
21 | var/
22 | wheels/
23 | share/python-wheels/
24 | *.egg-info/
25 | .installed.cfg
26 | *.egg
27 | MANIFEST
28 |
29 | # PyInstaller
30 | # Usually these files are written by a python script from a template
31 | # before PyInstaller builds the exe, so as to inject date/other infos into it.
32 | *.manifest
33 | *.spec
34 |
35 | # Installer logs
36 | pip-log.txt
37 | pip-delete-this-directory.txt
38 |
39 | # Unit test / coverage reports
40 | htmlcov/
41 | .tox/
42 | .nox/
43 | .coverage
44 | .coverage.*
45 | .cache
46 | nosetests.xml
47 | coverage.xml
48 | *.cover
49 | *.py,cover
50 | .hypothesis/
51 | .pytest_cache/
52 | cover/
53 |
54 | # Translations
55 | *.mo
56 | *.pot
57 |
58 | # Django stuff:
59 | *.log
60 | local_settings.py
61 | db.sqlite3
62 | db.sqlite3-journal
63 |
64 | # Flask stuff:
65 | instance/
66 | .webassets-cache
67 |
68 | # Scrapy stuff:
69 | .scrapy
70 |
71 | # Sphinx documentation
72 | docs/_build/
73 |
74 | # PyBuilder
75 | .pybuilder/
76 | target/
77 |
78 | # Jupyter Notebook
79 | .ipynb_checkpoints
80 |
81 | # IPython
82 | profile_default/
83 | ipython_config.py
84 |
85 | # pyenv
86 | # For a library or package, you might want to ignore these files since the code is
87 | # intended to run in multiple environments; otherwise, check them in:
88 | # .python-version
89 |
90 | # pipenv
91 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
92 | # However, in case of collaboration, if having platform-specific dependencies or dependencies
93 | # having no cross-platform support, pipenv may install dependencies that don't work, or not
94 | # install all needed dependencies.
95 | #Pipfile.lock
96 |
97 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow
98 | __pypackages__/
99 |
100 | # Celery stuff
101 | celerybeat-schedule
102 | celerybeat.pid
103 |
104 | # SageMath parsed files
105 | *.sage.py
106 |
107 | # Environments
108 | .env
109 | .venv
110 | env/
111 | venv/
112 | ENV/
113 | env.bak/
114 | venv.bak/
115 |
116 | # Spyder project settings
117 | .spyderproject
118 | .spyproject
119 |
120 | # Rope project settings
121 | .ropeproject
122 |
123 | # mkdocs documentation
124 | /site
125 |
126 | # mypy
127 | .mypy_cache/
128 | .dmypy.json
129 | dmypy.json
130 |
131 | # Pyre type checker
132 | .pyre/
133 |
134 | # pytype static type analyzer
135 | .pytype/
136 |
137 | # Cython debug symbols
138 | cython_debug/
--------------------------------------------------------------------------------
/player/.python-version:
--------------------------------------------------------------------------------
1 | 3.9.4
2 |
--------------------------------------------------------------------------------
/player/README.md:
--------------------------------------------------------------------------------
1 | # Intro
2 |
3 | [](https://ko-fi.com/Z8Z734F5Y)
4 |
5 | # Intro
6 |
7 | You can watch a video of this working here: https://youtu.be/0jR-QNTfydA at around 4:23
8 |
9 | [](https://www.youtube.com/watch?v=0jR-QNTfydA)
10 |
11 | # You'll need:
12 |
13 | Python3
14 |
15 | and
16 |
17 | ```
18 | brew install portaudio
19 | brew install brew install blackhole-2ch
20 | ```
21 |
22 | To get setup run
23 |
24 | ```
25 | python3 -m venv venv
26 | . ./venv/bin/activate
27 | pip install -r requirements.txt
28 | ```
29 |
30 | Then just run:
31 |
32 | ```
33 | python main.py
34 | ```
35 |
--------------------------------------------------------------------------------
/player/main.py:
--------------------------------------------------------------------------------
1 | import pyaudio
2 | import socket
3 | import threading
4 | import wave
5 | from PyInquirer import prompt
6 | from time import sleep
7 |
8 |
9 | # ask the user if they want to record or play
10 | def main_menu():
11 | questions = [
12 | {
13 | "type": "list",
14 | "name": "mode",
15 | "message": "What do you want to do?",
16 | "choices": ["Record", "Play"],
17 | }
18 | ]
19 | answers = prompt(questions)
20 | return answers["mode"]
21 |
22 |
23 | # select the output audio device using pyinquirer
24 | def get_output_audio_device_idx(p: pyaudio.PyAudio):
25 | audio_device_list = []
26 | for i in range(p.get_device_count()):
27 | device = p.get_device_info_by_index(i)
28 | if device["maxOutputChannels"] > 0:
29 | audio_device_list.append({"name": device["name"], "checked": True})
30 | questions = [
31 | {
32 | "type": "list",
33 | "name": "audio_device",
34 | "message": "Select the audio device to use (I recommend you use a virtual audio device!)",
35 | "choices": audio_device_list,
36 | }
37 | ]
38 | answers = prompt(questions)
39 | # select the user's preffered audio device
40 | output_device_idx = [
41 | i
42 | for i in range(p.get_device_count())
43 | if answers["audio_device"] == p.get_device_info_by_index(i)["name"]
44 | ][0]
45 | return output_device_idx
46 |
47 |
48 | # this will run in a thread reading audio from the tcp socket and buffering it
49 | buffer = []
50 | buffering = False
51 | buffer_audio = True
52 |
53 |
54 | def read_audio_from_socket():
55 | global buffering, buffer, buffer_audio
56 | # connect to the esp32 socket
57 | sock = socket.socket()
58 | sock.connect(("microphone.local", 9090))
59 | sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
60 | while buffer_audio:
61 | data = sock.recv(4096)
62 | if data == b"":
63 | raise RuntimeError("Lost connection")
64 | buffer.append(data)
65 | if len(buffer) > 50 and buffering:
66 | print("Finished buffering")
67 | buffering = False
68 |
69 |
70 | def main():
71 | global buffer, buffering, buffer_audio
72 | # initiaslise pyaudio
73 | p = pyaudio.PyAudio()
74 | mode = main_menu()
75 | # kick off the audio buffering thread
76 | thread = threading.Thread(target=read_audio_from_socket)
77 | thread.daemon = True
78 | thread.start()
79 | if mode == "Record":
80 | input("Recording to output.wav - hit any key to stop")
81 | buffer_audio = False
82 | # write the buffered audio to a wave file
83 | with wave.open("output.wav", "wb") as wave_file:
84 | wave_file.setnchannels(1)
85 | wave_file.setsampwidth(p.get_sample_size(pyaudio.paInt16))
86 | wave_file.setframerate(44100)
87 | wave_file.writeframes(b"".join(buffer))
88 | else:
89 | output_device_idx = get_output_audio_device_idx(p)
90 | # set up the audio stream
91 | stream = pyaudio.Stream(
92 | p,
93 | output=True,
94 | rate=44100,
95 | channels=1,
96 | format=pyaudio.paInt16,
97 | output_device_index=output_device_idx,
98 | frames_per_buffer=1024,
99 | )
100 | # this will write the buffered audio to the audio stream
101 | while True:
102 | if not buffering and len(buffer) > 0:
103 | data = buffer.pop(0)
104 | stream.write(data)
105 | if len(buffer) == 0:
106 | print("Buffering...")
107 | buffering = True
108 | else:
109 | sleep(0.001)
110 |
111 |
112 | if __name__ == "__main__":
113 | main()
114 |
--------------------------------------------------------------------------------
/player/requirements.txt:
--------------------------------------------------------------------------------
1 | pyaudio
2 | PyInquirer
3 | wave
--------------------------------------------------------------------------------