├── .gitignore ├── .vscode └── extensions.json ├── README.md ├── include └── README ├── lib └── README ├── platformio.ini ├── src ├── SDCard.cpp ├── SDCard.h ├── SDCardArduino.cpp ├── SDCardArduino.h ├── SDCardIdf.cpp ├── SDCardIdf.h ├── SDCardLazyWrite.cpp ├── SDCardLazyWrite.h ├── SDCardMultiSector.cpp ├── SDCardMultiSector.h └── main.cpp ├── test.sh └── test └── README /.gitignore: -------------------------------------------------------------------------------- 1 | .pio 2 | .vscode/.browse.c_cpp.db* 3 | .vscode/c_cpp_properties.json 4 | .vscode/launch.json 5 | .vscode/ipch 6 | -------------------------------------------------------------------------------- /.vscode/extensions.json: -------------------------------------------------------------------------------- 1 | { 2 | // See http://go.microsoft.com/fwlink/?LinkId=827846 3 | // for the documentation about the extensions.json format 4 | "recommendations": [ 5 | "platformio.platformio-ide" 6 | ], 7 | "unwantedRecommendations": [ 8 | "ms-vscode.cpptools-extension-pack" 9 | ] 10 | } 11 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Accompanying Video 2 | 3 | [![WiFi Streaming](https://img.youtube.com/vi/ocXs1yxsux4/0.jpg)](https://www.youtube.com/watch?v=ocXs1yxsux4) 4 | 5 | # ESP32-S3 SD Card Performance Tests - USBMSC and Direct From ESP32 6 | 7 | The ESP32-S3 can act as a Mass Storage Class device (USBMSC). This allows us to connect and SD Card to the ESP32 and then access it from a PC. 8 | 9 | # SD Card Performance Directly Connected to PC 10 | 11 | This is just a baseline to see what the SD Card is actually capable of when connected directly to the SD Card reader of my Mac Book Pro. 12 | 13 | | | Average Speed (B/s) | Average Speed (Mbytes/s) | 14 | |---|---------------------|--------------------------| 15 | | **Write** | 27,297,357 | 26.03 | 16 | | **Read** | 94,395,970 | 90.02 | 17 | 18 | 19 | # ESP32 <-> SD Card Performance 20 | 21 | This test write 100MB of raw sectors to the SD Card and then reads it back. Enable this mode ny uncommenting the line in `main.cpp` 22 | 23 | ```cpp 24 | // #define SD_CARD_SPEED_TEST 25 | ``` 26 | 27 | All values are in MBytes/s 28 | 29 | | Operation | 1Bit SPI 20 MHz | 4Bit SDIO 40 MHz | 30 | | --------- | ---------------- | ---------------- | 31 | | Write | 1.01 | 2.34 | 32 | | Read | 1.67 | 8.39 | 33 | 34 | 35 | # USB <-> ESP32 <-> SD Card: Using USBMSC 36 | 37 | There are three different configurations in this repo. The basic Arduino code uses the SD class to connect to the SD Card using SPI. This only exposes methods for writing/reading one sector at a time which has a considerable amount of overhead. Using IDF code we can write/read multiple sectors at once. And if we move writing to the SD Card to a background task we can get even better performance - but this comes at the cost of no error handling... 38 | 39 | You can switch between these different modes by changing the class used for the SDCard 40 | 41 | ```cpp 42 | #ifdef USE_SDIO 43 | card = new SDCardLazyWrite(Serial, "/sd", SD_CARD_CLK, SD_CARD_CMD, SD_CARD_DAT0, SD_CARD_DAT1, SD_CARD_DAT2, SD_CARD_DAT3); 44 | #else 45 | card = new SDCardLazyWrite(Serial, "/sd", SD_CARD_MISO, SD_CARD_MOSI, SD_CARD_CLK, SD_CARD_CS); 46 | #endif 47 | ``` 48 | 49 | You can use the following classes: `SDCardLazyWrite`, `SDCardMultiWrite`, `SDCardArduino` - NOTE - `SDCardArduino` cannot be used in SDIO mode as the readRAW and writeRAW functions don't exist on the SDMMC class. 50 | 51 | ## Arduino SD Card Code - Single Sector Writing 52 | 53 | 54 | | Type | SPI 20MHz | 4Bit SDIO 40 MHz | 55 | | ----------- | ------------------ | ---------------- | 56 | | Write Speed | 0.278 | NA | 57 | | Read Speed | 0.507 | NA | 58 | 59 | 60 | ## IDF SD Card Code - Multi Sector Writing 61 | 62 | 63 | | Type | SPI 20MHz | 4Bit SDIO 40 MHz | 64 | | ----------- | ------------------ |------------------ | 65 | | Write Speed | 0.481 |0.66 | 66 | | Read Speed | 0.671 |1.00 | 67 | 68 | 69 | ## IDF SD Card Code - Multi Sector Lazy Writing 70 | 71 | | Type | SPI 20MHz | 4Bit SDIO 40 MHz | 72 | | ----------- | ------------------ | ------------------ | 73 | | Write Speed | 0.915 | 0.92 | 74 | | Read Speed | 0.672 | 1.00 | 75 | 76 | # Summary 77 | 78 | Getting up to almost 1MByte/s on both reading and writing is pretty impressive but it probably the best that can be done given the limitations of USB1.1 full speed mode. -------------------------------------------------------------------------------- /include/README: -------------------------------------------------------------------------------- 1 | 2 | This directory is intended for project header files. 3 | 4 | A header file is a file containing C declarations and macro definitions 5 | to be shared between several project source files. You request the use of a 6 | header file in your project source file (C, C++, etc) located in `src` folder 7 | by including it, with the C preprocessing directive `#include'. 8 | 9 | ```src/main.c 10 | 11 | #include "header.h" 12 | 13 | int main (void) 14 | { 15 | ... 16 | } 17 | ``` 18 | 19 | Including a header file produces the same results as copying the header file 20 | into each source file that needs it. Such copying would be time-consuming 21 | and error-prone. With a header file, the related declarations appear 22 | in only one place. If they need to be changed, they can be changed in one 23 | place, and programs that include the header file will automatically use the 24 | new version when next recompiled. The header file eliminates the labor of 25 | finding and changing all the copies as well as the risk that a failure to 26 | find one copy will result in inconsistencies within a program. 27 | 28 | In C, the usual convention is to give header files names that end with `.h'. 29 | It is most portable to use only letters, digits, dashes, and underscores in 30 | header file names, and at most one dot. 31 | 32 | Read more about using header files in official GCC documentation: 33 | 34 | * Include Syntax 35 | * Include Operation 36 | * Once-Only Headers 37 | * Computed Includes 38 | 39 | https://gcc.gnu.org/onlinedocs/cpp/Header-Files.html 40 | -------------------------------------------------------------------------------- /lib/README: -------------------------------------------------------------------------------- 1 | 2 | This directory is intended for project specific (private) libraries. 3 | PlatformIO will compile them to static libraries and link into executable file. 4 | 5 | The source code of each library should be placed in a an own separate directory 6 | ("lib/your_library_name/[here are source files]"). 7 | 8 | For example, see a structure of the following two libraries `Foo` and `Bar`: 9 | 10 | |--lib 11 | | | 12 | | |--Bar 13 | | | |--docs 14 | | | |--examples 15 | | | |--src 16 | | | |- Bar.c 17 | | | |- Bar.h 18 | | | |- library.json (optional, custom build options, etc) https://docs.platformio.org/page/librarymanager/config.html 19 | | | 20 | | |--Foo 21 | | | |- Foo.c 22 | | | |- Foo.h 23 | | | 24 | | |- README --> THIS FILE 25 | | 26 | |- platformio.ini 27 | |--src 28 | |- main.c 29 | 30 | and a contents of `src/main.c`: 31 | ``` 32 | #include 33 | #include 34 | 35 | int main (void) 36 | { 37 | ... 38 | } 39 | 40 | ``` 41 | 42 | PlatformIO Library Dependency Finder will find automatically dependent 43 | libraries scanning project source files. 44 | 45 | More information about PlatformIO Library Dependency Finder 46 | - https://docs.platformio.org/page/librarymanager/ldf.html 47 | -------------------------------------------------------------------------------- /platformio.ini: -------------------------------------------------------------------------------- 1 | ; PlatformIO Project Configuration File 2 | ; 3 | ; Build options: build flags, source filter 4 | ; Upload options: custom upload port, speed and extra flags 5 | ; Library options: dependencies, extra library storages 6 | ; Advanced options: extra scripting 7 | ; 8 | ; Please visit documentation for the other options and examples 9 | ; https://docs.platformio.org/page/projectconf.html 10 | 11 | [env:esp32-s3-devkitc-1] 12 | platform = espressif32 13 | board = esp32-s3-devkitc-1 14 | framework = arduino 15 | board_build.arduino.memory_type = qio_opi 16 | lib_deps = 17 | SD 18 | FS 19 | build_flags = 20 | -DBOARD_HAS_PSRAM 21 | -DSD_CARD_MISO=GPIO_NUM_3 22 | -DSD_CARD_MOSI=GPIO_NUM_8 23 | -DSD_CARD_CLK=GPIO_NUM_46 24 | -DSD_CARD_CS=GPIO_NUM_18 25 | ; make sure serial output works 26 | -DARDUINO_USB_MODE 27 | -DARDUINO_USB_CDC_ON_BOOT 28 | ; prevent hardware serial being used - we'll use USB serial 29 | ; -DNO_GLOBAL_SERIAL 30 | ; decode exceptions 31 | monitor_filters = esp32_exception_decoder 32 | monitor_speed = 115200 33 | 34 | 35 | [env:TinyPICO] 36 | platform = espressif32 37 | board = um_tinys3 38 | framework = arduino 39 | lib_deps = 40 | SD 41 | FS 42 | unexpectedmaker/UMS3 Helper 43 | build_flags = 44 | ; -DSD_CARD_MISO=GPIO_NUM_8 45 | ; -DSD_CARD_MOSI=GPIO_NUM_7 46 | ; -DSD_CARD_CLK=GPIO_NUM_6 47 | ; -DSD_CARD_CS=GPIO_NUM_4 48 | -DUSE_SDIO=1 49 | -DSD_CARD_DAT0=GPIO_NUM_8 50 | -DSD_CARD_DAT1=GPIO_NUM_21 51 | -DSD_CARD_DAT2=GPIO_NUM_5 52 | -DSD_CARD_DAT3=GPIO_NUM_4 53 | -DSD_CARD_CMD=GPIO_NUM_7 54 | -DSD_CARD_CLK=GPIO_NUM_6 55 | ; make sure serial output works 56 | -DARDUINO_USB_MODE 57 | -DARDUINO_USB_CDC_ON_BOOT 58 | ; prevent hardware serial being used - we'll use USB serial 59 | ; -DNO_GLOBAL_SERIAL 60 | ; decode exceptions 61 | monitor_filters = esp32_exception_decoder 62 | monitor_speed = 115200 -------------------------------------------------------------------------------- /src/SDCard.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include "SDCard.h" 3 | 4 | SDCard::SDCard(Stream &debug, const char *mount_point) : m_debug(debug), m_mount_point(mount_point) 5 | { 6 | } 7 | 8 | SDCard::~SDCard() 9 | { 10 | } 11 | -------------------------------------------------------------------------------- /src/SDCard.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include 6 | 7 | #include 8 | 9 | class Stream; 10 | class SDCard 11 | { 12 | protected: 13 | std::string m_mount_point; 14 | int m_sector_size = 0; 15 | int m_sector_count = 0; 16 | Stream &m_debug; 17 | public: 18 | SDCard(Stream &debug, const char *mount_point); 19 | virtual ~SDCard(); 20 | virtual bool writeSectors(uint8_t *src, size_t start_sector, size_t sector_count) = 0; 21 | virtual bool readSectors(uint8_t *dst, size_t start_sector, size_t sector_count) = 0; 22 | virtual void printCardInfo() = 0; 23 | size_t getSectorSize() { return m_sector_size; } 24 | size_t getSectorCount() { return m_sector_count; } 25 | const std::string &get_mount_point() { return m_mount_point; } 26 | }; 27 | -------------------------------------------------------------------------------- /src/SDCardArduino.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | 5 | #include "SDCardArduino.h" 6 | 7 | SDCardArduino::SDCardArduino(Stream &debug, const char *mount_point, gpio_num_t miso, gpio_num_t mosi, gpio_num_t clk, gpio_num_t cs) 8 | : SDCard(debug, mount_point) 9 | { 10 | static SPIClass spi(HSPI); 11 | spi.begin(clk, miso, mosi, cs); 12 | if (SD.begin(cs, spi, 80000000, mount_point)) 13 | { 14 | debug.println("SD card initialized"); 15 | } 16 | else 17 | { 18 | debug.println("SD card initialization failed"); 19 | } 20 | m_sector_size = SD.sectorSize(); 21 | m_sector_count = SD.numSectors(); 22 | } 23 | 24 | SDCardArduino::~SDCardArduino() 25 | { 26 | SD.end(); 27 | } 28 | 29 | void SDCardArduino::printCardInfo() 30 | { 31 | m_debug.printf("Card type: %d\n", SD.cardType()); 32 | if (SD.cardType() == CARD_NONE) 33 | { 34 | m_debug.println("No SD card attached"); 35 | return; 36 | } 37 | m_debug.printf("Card size: %lluMB\n", SD.cardSize() / (1024 * 1024)); 38 | } 39 | 40 | bool SDCardArduino::writeSectors(uint8_t *src, size_t start_sector, size_t sector_count) 41 | { 42 | digitalWrite(GPIO_NUM_2, HIGH); 43 | bool res = true; 44 | for (int i = 0; i < sector_count; i++) 45 | { 46 | res = SD.writeRAW((uint8_t *)src, start_sector + i); 47 | if (!res) 48 | { 49 | break; 50 | } 51 | src += m_sector_size; 52 | } 53 | digitalWrite(GPIO_NUM_2, LOW); 54 | return res; 55 | } 56 | 57 | bool SDCardArduino::readSectors(uint8_t *dst, size_t start_sector, size_t sector_count) 58 | { 59 | digitalWrite(GPIO_NUM_2, HIGH); 60 | bool res = true; 61 | for (int i = 0; i < sector_count; i++) 62 | { 63 | res = SD.readRAW((uint8_t *)dst, start_sector + i); 64 | if (!res) 65 | { 66 | break; 67 | } 68 | dst += m_sector_size; 69 | } 70 | digitalWrite(GPIO_NUM_2, LOW); 71 | return res; 72 | } 73 | -------------------------------------------------------------------------------- /src/SDCardArduino.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "SDCard.h" 4 | 5 | class SDCardArduino: public SDCard 6 | { 7 | protected: 8 | public: 9 | SDCardArduino(Stream &debug, const char *mount_point, gpio_num_t miso, gpio_num_t mosi, gpio_num_t clk, gpio_num_t cs); 10 | ~SDCardArduino(); 11 | bool writeSectors(uint8_t *src, size_t start_sector, size_t sector_count); 12 | bool readSectors(uint8_t *dst, size_t start_sector, size_t sector_count); 13 | void printCardInfo(); 14 | }; 15 | -------------------------------------------------------------------------------- /src/SDCardIdf.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include "esp_err.h" 8 | #include "esp_log.h" 9 | #include "esp_vfs_fat.h" 10 | #include "driver/sdmmc_host.h" 11 | #include "driver/sdspi_host.h" 12 | #include "sdmmc_cmd.h" 13 | 14 | #include "SDCardIdf.h" 15 | 16 | #define SPI_DMA_CHAN SPI_DMA_CH_AUTO 17 | 18 | SDCardIdf::SDCardIdf(Stream &debug, const char *mount_point, gpio_num_t clk, gpio_num_t cmd, gpio_num_t d0, gpio_num_t d1, gpio_num_t d2, gpio_num_t d3) 19 | : SDCard(debug, mount_point) 20 | { 21 | // a mutex to prevent read and write overlapping 22 | m_mutex = xSemaphoreCreateMutex(); 23 | m_host.max_freq_khz = SDMMC_FREQ_52M; 24 | m_host.flags = SDMMC_HOST_FLAG_4BIT; 25 | esp_err_t ret; 26 | // Options for mounting the filesystem. 27 | // If format_if_mount_failed is set to true, SD card will be partitioned and 28 | // formatted in case when mounting fails. 29 | esp_vfs_fat_sdmmc_mount_config_t mount_config = { 30 | .format_if_mount_failed = false, 31 | .max_files = 5, 32 | .allocation_unit_size = 16 * 1024}; 33 | 34 | m_debug.println("Initializing SD card"); 35 | 36 | sdmmc_slot_config_t slot_config = SDMMC_SLOT_CONFIG_DEFAULT(); 37 | slot_config.flags = SDMMC_SLOT_FLAG_INTERNAL_PULLUP; 38 | slot_config.width = 4; 39 | slot_config.clk = clk; 40 | slot_config.cmd = cmd; 41 | slot_config.d0 = d0; 42 | slot_config.d1 = d1; 43 | slot_config.d2 = d2; 44 | slot_config.d3 = d3; 45 | ret = esp_vfs_fat_sdmmc_mount(m_mount_point.c_str(), &m_host, &slot_config, &mount_config, &m_card); 46 | if (ret != ESP_OK) 47 | { 48 | if (ret == ESP_FAIL) 49 | { 50 | m_debug.println("Failed to mount filesystem. " 51 | "If you want the card to be formatted, set the EXAMPLE_FORMAT_IF_MOUNT_FAILED menuconfig option."); 52 | } 53 | else 54 | { 55 | m_debug.printf("Failed to initialize the card (%s). " 56 | "Make sure SD card lines have pull-up resistors in place.\n", 57 | esp_err_to_name(ret)); 58 | } 59 | return; 60 | } 61 | m_debug.printf("SDCard mounted at: %s\n", m_mount_point.c_str()); 62 | m_sector_count = m_card->csd.capacity; 63 | m_sector_size = m_card->csd.sector_size; 64 | m_debug.printf("SDCard sector count: %d, size: %d\n", m_sector_count, m_sector_size); 65 | // Card has been initialized, print its properties 66 | sdmmc_card_print_info(stdout, m_card); 67 | } 68 | 69 | SDCardIdf::SDCardIdf(Stream &debug, const char *mount_point, gpio_num_t miso, gpio_num_t mosi, gpio_num_t clk, gpio_num_t cs) 70 | : SDCard(debug, mount_point) 71 | { 72 | // a mutex to prevent read and write overlapping 73 | m_mutex = xSemaphoreCreateMutex(); 74 | m_host.max_freq_khz = SDMMC_FREQ_52M; 75 | esp_err_t ret; 76 | // Options for mounting the filesystem. 77 | // If format_if_mount_failed is set to true, SD card will be partitioned and 78 | // formatted in case when mounting fails. 79 | esp_vfs_fat_sdmmc_mount_config_t mount_config = { 80 | .format_if_mount_failed = false, 81 | .max_files = 5, 82 | .allocation_unit_size = 16 * 1024}; 83 | 84 | m_debug.println("Initializing SD card"); 85 | 86 | spi_bus_config_t bus_cfg = { 87 | .mosi_io_num = mosi, 88 | .miso_io_num = miso, 89 | .sclk_io_num = clk, 90 | .quadwp_io_num = -1, 91 | .quadhd_io_num = -1, 92 | .max_transfer_sz = 16384, 93 | .flags = 0, 94 | .intr_flags = 0 95 | }; 96 | ret = spi_bus_initialize(spi_host_device_t(m_host.slot), &bus_cfg, SPI_DMA_CHAN); 97 | if (ret != ESP_OK) 98 | { 99 | m_debug.println("Failed to initialize bus."); 100 | return; 101 | } 102 | 103 | // This initializes the slot without card detect (CD) and write protect (WP) signals. 104 | // Modify slot_config.gpio_cd and slot_config.gpio_wp if your board has these signals. 105 | sdspi_device_config_t slot_config = SDSPI_DEVICE_CONFIG_DEFAULT(); 106 | slot_config.gpio_cs = cs; 107 | slot_config.host_id = spi_host_device_t(m_host.slot); 108 | 109 | ret = esp_vfs_fat_sdspi_mount(m_mount_point.c_str(), &m_host, &slot_config, &mount_config, &m_card); 110 | if (ret != ESP_OK) 111 | { 112 | if (ret == ESP_FAIL) 113 | { 114 | m_debug.println("Failed to mount filesystem. " 115 | "If you want the card to be formatted, set the EXAMPLE_FORMAT_IF_MOUNT_FAILED menuconfig option."); 116 | } 117 | else 118 | { 119 | m_debug.printf("Failed to initialize the card (%s). " 120 | "Make sure SD card lines have pull-up resistors in place.\n", 121 | esp_err_to_name(ret)); 122 | } 123 | return; 124 | } 125 | m_debug.printf("SDCard mounted at: %s\n", m_mount_point.c_str()); 126 | m_sector_count = m_card->csd.capacity; 127 | m_sector_size = m_card->csd.sector_size; 128 | m_debug.printf("SDCard sector count: %d, size: %d\n", m_sector_count, m_sector_size); 129 | // Card has been initialized, print its properties 130 | sdmmc_card_print_info(stdout, m_card); 131 | } 132 | 133 | SDCardIdf::~SDCardIdf() 134 | { 135 | // lock the SD card 136 | xSemaphoreTake(m_mutex, portMAX_DELAY); 137 | // All done, unmount partition and disable SDMMC or SPI peripheral 138 | esp_vfs_fat_sdcard_unmount(m_mount_point.c_str(), m_card); 139 | m_debug.println("Card unmounted"); 140 | //deinitialize the bus after all devices are removed 141 | spi_bus_free(spi_host_device_t(m_host.slot)); 142 | // unlock the SD card 143 | xSemaphoreGive(m_mutex); 144 | } 145 | 146 | void SDCardIdf::printCardInfo() 147 | { 148 | sdmmc_card_print_info(stdout, m_card); 149 | } 150 | -------------------------------------------------------------------------------- /src/SDCardIdf.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "SDCard.h" 4 | 5 | #include 6 | #include 7 | #include 8 | #include 9 | 10 | class SDCardIdf: public SDCard 11 | { 12 | protected: 13 | sdmmc_card_t *m_card; 14 | #ifdef USE_SDIO 15 | sdmmc_host_t m_host = SDMMC_HOST_DEFAULT(); 16 | #else 17 | sdmmc_host_t m_host = SDSPI_HOST_DEFAULT(); 18 | #endif 19 | // control access to the SD card 20 | SemaphoreHandle_t m_mutex; 21 | public: 22 | SDCardIdf(Stream &debug, const char *mount_point, gpio_num_t miso, gpio_num_t mosi, gpio_num_t clk, gpio_num_t cs); 23 | SDCardIdf(Stream &debug, const char *mount_point, gpio_num_t clk, gpio_num_t cmd, gpio_num_t d0, gpio_num_t d1, gpio_num_t d2, gpio_num_t d3); 24 | ~SDCardIdf(); 25 | void printCardInfo(); 26 | }; 27 | -------------------------------------------------------------------------------- /src/SDCardLazyWrite.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include "esp_err.h" 8 | #include "esp_log.h" 9 | #include "esp_vfs_fat.h" 10 | #include "driver/sdmmc_host.h" 11 | #include "driver/sdspi_host.h" 12 | #include "sdmmc_cmd.h" 13 | 14 | #include "SDCardLazyWrite.h" 15 | 16 | static const char *TAG = "SDC"; 17 | 18 | #define SPI_DMA_CHAN SPI_DMA_CH_AUTO 19 | 20 | enum class RequestType { 21 | READ, 22 | WRITE 23 | }; 24 | 25 | class Request { 26 | public: 27 | Request(RequestType type, void *data, size_t start_sector, size_t sector_count) 28 | : m_type(type), m_start_sector(start_sector), m_sector_count(sector_count) { 29 | if (type == RequestType::WRITE) { 30 | m_data = malloc(sector_count * 512); 31 | memcpy(m_data, data, sector_count * 512); 32 | } else if(type == RequestType::READ) { 33 | m_data = data; 34 | } else { 35 | m_data = NULL; 36 | } 37 | } 38 | ~Request() { 39 | if (m_type == RequestType::WRITE) { 40 | free(m_data); 41 | } 42 | } 43 | RequestType m_type; 44 | void *m_data; 45 | size_t m_start_sector; 46 | size_t m_sector_count; 47 | }; 48 | 49 | SDCardLazyWrite::SDCardLazyWrite(Stream &debug, const char *mount_point, gpio_num_t miso, gpio_num_t mosi, gpio_num_t clk, gpio_num_t cs) 50 | : SDCardIdf(debug, mount_point, miso, mosi, clk, cs) 51 | { 52 | // a queue to hold requests (to read or write) 53 | m_request_queue = xQueueCreate(10, sizeof(Request *)); 54 | // a queue to hold the results of read requests 55 | m_read_queue = xQueueCreate(10, sizeof(Request *)); 56 | // create a task to drain the write queue 57 | xTaskCreate([](void *param) { 58 | SDCardLazyWrite *card = (SDCardLazyWrite *)param; 59 | card->drainQueue(); 60 | } 61 | , "SDCardWriter", 4096, this, 1, NULL); 62 | } 63 | 64 | SDCardLazyWrite::SDCardLazyWrite(Stream &debug, const char *mount_point, gpio_num_t clk, gpio_num_t cmd, gpio_num_t d0, gpio_num_t d1, gpio_num_t d2, gpio_num_t d3) 65 | : SDCardIdf(debug, mount_point, clk, cmd, d0, d1, d2, d3) 66 | { 67 | // a queue to hold requests (to read or write) 68 | m_request_queue = xQueueCreate(10, sizeof(Request *)); 69 | // a queue to hold the results of read requests 70 | m_read_queue = xQueueCreate(10, sizeof(Request *)); 71 | // create a task to drain the write queue 72 | xTaskCreate([](void *param) { 73 | SDCardLazyWrite *card = (SDCardLazyWrite *)param; 74 | card->drainQueue(); 75 | } 76 | , "SDCardWriter", 4096, this, 1, NULL); 77 | } 78 | 79 | bool SDCardLazyWrite::writeSectors(uint8_t *src, size_t start_sector, size_t sector_count) { 80 | xSemaphoreTake(m_mutex, portMAX_DELAY); 81 | // push the write request onto the queue 82 | Request *req = new Request(RequestType::WRITE, src, start_sector, sector_count); 83 | xQueueSend(m_request_queue, &req, portMAX_DELAY); 84 | xSemaphoreGive(m_mutex); 85 | return true; 86 | } 87 | 88 | void SDCardLazyWrite::drainQueue() { 89 | Request *req; 90 | while (xQueueReceive(m_request_queue, &req, portMAX_DELAY) == pdTRUE) { 91 | // lock the SD card 92 | xSemaphoreTake(m_mutex, portMAX_DELAY); 93 | digitalWrite(GPIO_NUM_2, HIGH); 94 | if (req->m_type == RequestType::WRITE) { 95 | esp_err_t res = sdmmc_write_sectors(m_card, req->m_data, req->m_start_sector, req->m_sector_count); 96 | delete req; 97 | } else if (req->m_type == RequestType::READ) { 98 | esp_err_t res = sdmmc_read_sectors(m_card, req->m_data, req->m_start_sector, req->m_sector_count); 99 | xQueueSend(m_read_queue, &req, portMAX_DELAY); 100 | } 101 | digitalWrite(GPIO_NUM_2, LOW); 102 | xSemaphoreGive(m_mutex); 103 | } 104 | } 105 | 106 | bool SDCardLazyWrite::readSectors(uint8_t *dst, size_t start_sector, size_t sector_count) { 107 | xSemaphoreTake(m_mutex, portMAX_DELAY); 108 | // check to see if the queue has any pending writes 109 | if (uxQueueMessagesWaiting(m_request_queue) > 0) { 110 | // push our read request onto the queue and wait for it to complete 111 | Request *req = new Request(RequestType::READ, dst, start_sector, sector_count); 112 | xQueueSend(m_request_queue, &req, portMAX_DELAY); 113 | // wait for the read to complete 114 | xQueueReceive(m_read_queue, &req, portMAX_DELAY); 115 | delete req; 116 | } else { 117 | digitalWrite(GPIO_NUM_2, HIGH); 118 | // no pending writes, so we can just read directly 119 | esp_err_t res = sdmmc_read_sectors(m_card, dst, start_sector, sector_count); 120 | digitalWrite(GPIO_NUM_2, LOW); 121 | } 122 | xSemaphoreGive(m_mutex); 123 | return true; 124 | } 125 | -------------------------------------------------------------------------------- /src/SDCardLazyWrite.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "SDCardIdf.h" 4 | #include 5 | 6 | class SDCardLazyWrite: public SDCardIdf 7 | { 8 | private: 9 | // queue up requests 10 | QueueHandle_t m_request_queue; 11 | // results of reading data 12 | QueueHandle_t m_read_queue; 13 | void drainQueue(); 14 | public: 15 | SDCardLazyWrite(Stream &debug, const char *mount_point, gpio_num_t miso, gpio_num_t mosi, gpio_num_t clk, gpio_num_t cs); 16 | SDCardLazyWrite(Stream &debug, const char *mount_point, gpio_num_t clk, gpio_num_t cmd, gpio_num_t d0, gpio_num_t d1, gpio_num_t d2, gpio_num_t d3); 17 | bool writeSectors(uint8_t *src, size_t start_sector, size_t sector_count); 18 | bool readSectors(uint8_t *dst, size_t start_sector, size_t sector_count); 19 | }; -------------------------------------------------------------------------------- /src/SDCardMultiSector.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include "esp_vfs_fat.h" 3 | #include "driver/sdmmc_host.h" 4 | #include "driver/sdspi_host.h" 5 | #include "sdmmc_cmd.h" 6 | 7 | #include "SDCardMultiSector.h" 8 | 9 | SDCardMultiSector::SDCardMultiSector(Stream &debug, const char *mount_point, gpio_num_t miso, gpio_num_t mosi, gpio_num_t clk, gpio_num_t cs) 10 | : SDCardIdf(debug, mount_point, miso, mosi, clk, cs) 11 | { 12 | } 13 | 14 | SDCardMultiSector::SDCardMultiSector(Stream &debug, const char *mount_point, gpio_num_t clk, gpio_num_t cmd, gpio_num_t d0, gpio_num_t d1, gpio_num_t d2, gpio_num_t d3) 15 | : SDCardIdf(debug, mount_point, clk, cmd, d0, d1, d2, d3) 16 | { 17 | } 18 | 19 | bool SDCardMultiSector::writeSectors(uint8_t *src, size_t start_sector, size_t sector_count) { 20 | xSemaphoreTake(m_mutex, portMAX_DELAY); 21 | digitalWrite(GPIO_NUM_2, HIGH); 22 | esp_err_t res = sdmmc_write_sectors(m_card, src, start_sector, sector_count); 23 | digitalWrite(GPIO_NUM_2, LOW); 24 | xSemaphoreGive(m_mutex); 25 | return res == ESP_OK; 26 | } 27 | 28 | bool SDCardMultiSector::readSectors(uint8_t *dst, size_t start_sector, size_t sector_count) { 29 | xSemaphoreTake(m_mutex, portMAX_DELAY); 30 | digitalWrite(GPIO_NUM_2, HIGH); 31 | esp_err_t res = sdmmc_read_sectors(m_card, dst, start_sector, sector_count); 32 | digitalWrite(GPIO_NUM_2, LOW); 33 | xSemaphoreGive(m_mutex); 34 | return res == ESP_OK; 35 | } 36 | -------------------------------------------------------------------------------- /src/SDCardMultiSector.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "SDCardIdf.h" 4 | 5 | class SDCardMultiSector: public SDCardIdf 6 | { 7 | protected: 8 | public: 9 | SDCardMultiSector(Stream &debug, const char *mount_point, gpio_num_t miso, gpio_num_t mosi, gpio_num_t clk, gpio_num_t cs); 10 | SDCardMultiSector(Stream &debug, const char *mount_point, gpio_num_t clk, gpio_num_t cmd, gpio_num_t d0, gpio_num_t d1, gpio_num_t d2, gpio_num_t d3); 11 | bool writeSectors(uint8_t *src, size_t start_sector, size_t sector_count); 12 | bool readSectors(uint8_t *dst, size_t start_sector, size_t sector_count); 13 | }; -------------------------------------------------------------------------------- /src/main.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include "USB.h" 3 | #include "USBMSC.h" 4 | 5 | #include "SDCardArduino.h" 6 | #include "SDCardMultiSector.h" 7 | #include "SDCardLazyWrite.h" 8 | 9 | #define BOOT_BUTTON 0 10 | 11 | // #define SD_CARD_SPEED_TEST 12 | #define SPEED_TEST_BUFFER_SIZE 4096 13 | #define SPEED_TEST_NUMBER_SECTORS (SPEED_TEST_BUFFER_SIZE / 512) 14 | 15 | #ifndef SD_CARD_SPEED_TEST 16 | USBMSC msc; 17 | USBCDC Serial; 18 | #endif 19 | SDCard *card; 20 | 21 | void log(const char *str) 22 | { 23 | Serial.println(str); 24 | } 25 | 26 | static int32_t onWrite(uint32_t lba, uint32_t offset, uint8_t *buffer, uint32_t bufsize) 27 | { 28 | // Serial.printf("Writing %d bytes to %d at offset\n", bufsize, lba, offset); 29 | // this writes a complete sector so we should return sector size on success 30 | if (card->writeSectors(buffer, lba, bufsize / card->getSectorSize())) 31 | { 32 | return bufsize; 33 | } 34 | return bufsize; 35 | // return -1; 36 | } 37 | 38 | static int32_t onRead(uint32_t lba, uint32_t offset, void *buffer, uint32_t bufsize) 39 | { 40 | // Serial.printf("Reading %d bytes from %d at offset %d\n", bufsize, lba, offset); 41 | // this reads a complete sector so we should return sector size on success 42 | if (card->readSectors((uint8_t *)buffer, lba, bufsize / card->getSectorSize())) 43 | { 44 | return bufsize; 45 | } 46 | return -1; 47 | } 48 | 49 | static bool onStartStop(uint8_t power_condition, bool start, bool load_eject) 50 | { 51 | Serial.printf("StartStop: %d %d %d\n", power_condition, start, load_eject); 52 | if (load_eject) 53 | { 54 | #ifndef SD_CARD_SPEED_TEST 55 | msc.end(); 56 | #endif 57 | } 58 | return true; 59 | } 60 | 61 | bool isBootButtonClicked() 62 | { 63 | return digitalRead(BOOT_BUTTON) == LOW; 64 | } 65 | 66 | void setup() 67 | { 68 | pinMode(GPIO_NUM_2, OUTPUT); 69 | 70 | #ifdef USE_SDIO 71 | card = new SDCardMultiSector(Serial, "/sd", SD_CARD_CLK, SD_CARD_CMD, SD_CARD_DAT0, SD_CARD_DAT1, SD_CARD_DAT2, SD_CARD_DAT3); 72 | #else 73 | card = new SDCardLazyWrite(Serial, "/sd", SD_CARD_MISO, SD_CARD_MOSI, SD_CARD_CLK, SD_CARD_CS); 74 | #endif 75 | 76 | #ifdef SD_CARD_SPEED_TEST 77 | #warning "SD_CARD_SPEED_TEST is enabled - this will potentially corrupt your SD Card!" 78 | Serial.begin(115200); 79 | // wait a bit of time so we can connect the serial monitor 80 | for (int i = 0; i < 10; i++) 81 | { 82 | Serial.printf("Waiting %d\n", i); 83 | delay(1000); 84 | } 85 | card->printCardInfo(); 86 | Serial.printf("Sector Size=%d\n", card->getSectorSize()); 87 | // allocate a buffer of SPEED_TEST_BUFFER_SIZE bytes and fill it with random numbers 88 | uint8_t *buffer = (uint8_t *)malloc(SPEED_TEST_BUFFER_SIZE); 89 | for (int i = 0; i < SPEED_TEST_BUFFER_SIZE; i++) 90 | { 91 | buffer[i] = random(0, 255); 92 | } 93 | Serial.printf("Starting test: %d, %d\n", SPEED_TEST_BUFFER_SIZE, SPEED_TEST_NUMBER_SECTORS); 94 | // write 400MBytes of data to the SD Card using the writeSectors method 95 | int total_write_bytes = 0; 96 | uint32_t start = millis(); 97 | for (int times = 0; times < 100; times++) 98 | { 99 | for (int i = 0; i < 1024 * 1024 / SPEED_TEST_BUFFER_SIZE; i++) 100 | { 101 | if (card->writeSectors(buffer, 100 + i * SPEED_TEST_NUMBER_SECTORS, SPEED_TEST_NUMBER_SECTORS)) 102 | { 103 | total_write_bytes += SPEED_TEST_BUFFER_SIZE; 104 | // Serial.printf("#"); 105 | } 106 | else 107 | { 108 | Serial.printf("X"); 109 | } 110 | } 111 | Serial.printf("."); 112 | } 113 | Serial.println(); 114 | uint32_t end = millis(); 115 | Serial.printf("Write %dBytes took %dms\n", total_write_bytes, end - start); 116 | // read 400MBytes of data from the SD Card using the readSectors method 117 | int total_read_bytes = 0; 118 | uint8_t *read_buffer = (uint8_t *)malloc(SPEED_TEST_BUFFER_SIZE); 119 | start = millis(); 120 | for (int times = 0; times < 100; times++) 121 | { 122 | for (int i = 0; i < 1024 * 1024 / SPEED_TEST_BUFFER_SIZE; i++) 123 | { 124 | if (card->readSectors(read_buffer, 100 + i * SPEED_TEST_NUMBER_SECTORS, SPEED_TEST_NUMBER_SECTORS)) 125 | { 126 | total_read_bytes += SPEED_TEST_BUFFER_SIZE; 127 | // check the numbers match 128 | // for(int j = 0; j<4096; j++) { 129 | // if (read_buffer[j] != buffer[j]) { 130 | // Serial.printf("Mismatch at %d\n", j); 131 | // break; 132 | // } 133 | // } 134 | // Serial.printf("."); 135 | } 136 | else 137 | { 138 | Serial.printf("X"); 139 | } 140 | } 141 | Serial.printf("."); 142 | } 143 | Serial.println(); 144 | end = millis(); 145 | Serial.printf("Read %dBytes took %dms\n", total_read_bytes, end - start); 146 | #else 147 | // keyboard.begin(); 148 | msc.vendorID("ESP32"); 149 | msc.productID("USB_MSC"); 150 | msc.productRevision("1.0"); 151 | msc.onRead(onRead); 152 | msc.onWrite(onWrite); 153 | msc.onStartStop(onStartStop); 154 | msc.mediaPresent(true); 155 | 156 | msc.begin(card->getSectorCount(), card->getSectorSize()); 157 | Serial.begin(115200); 158 | USB.begin(); 159 | #endif 160 | } 161 | 162 | void loop() 163 | { 164 | // put your main code here, to run repeatedly: 165 | delay(200); 166 | // if (isBootButtonClicked()) 167 | // { 168 | // if (MySD.cardType() == CARD_NONE) 169 | // { 170 | // log("No SD card"); 171 | // } 172 | // } 173 | } -------------------------------------------------------------------------------- /test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Number of tests to run 4 | num_tests=5 5 | 6 | # Size of the file to be written/read (in MB) 7 | filesize=1024 # 1GB 8 | 9 | blocksize=100K # 100MB in total 10 | 11 | # File to write/read 12 | testfile="tempfile" 13 | 14 | # Arrays to store results 15 | write_results=() 16 | read_results=() 17 | 18 | # 1. Write Test Loop 19 | for ((i=1; i<=num_tests; i++)) 20 | do 21 | echo "Write test $i..." 22 | result=$(dd if=/dev/zero of=$testfile$i bs=$blocksize count=$filesize conv=notrunc 2>&1 | awk -F'[()]' '{print $2}' | awk '{print $1}') 23 | speed=${result% B/s} 24 | write_results+=($speed) 25 | echo "Write Speed: $speed B/s" 26 | done 27 | 28 | # 2. Read Test Loop 29 | for ((i=1; i<=num_tests; i++)) 30 | do 31 | echo "Read test $i..." 32 | sudo purge 33 | result=$(dd if=$testfile$i of=/dev/null bs=$blocksize count=$filesize 2>&1 | awk -F'[()]' '{print $2}' | awk '{print $1}') 34 | speed=${result% B/s} 35 | read_results+=($speed) 36 | echo "Read Speed: $speed B/s" 37 | done 38 | 39 | # 3. Cleanup 40 | rm $testfile? 41 | 42 | # 4. Analyze Results (Calculate Average and Standard Deviation) 43 | # You might want to do this analysis part using a tool like Python for simplicity and accuracy. 44 | 45 | echo "Write Speeds: ${write_results[@]} B/s" 46 | echo "Read Speeds: ${read_results[@]} B/s" 47 | 48 | echo ${write_results[@]} > write_results.txt 49 | echo ${read_results[@]} > read_results.txt 50 | 51 | # Optionally, you can call a Python script here to analyze the results 52 | # echo ${write_results[@]} | python3 your_analysis_script.py 53 | # echo ${read_results[@]} | python3 your_analysis_script.py 54 | 55 | -------------------------------------------------------------------------------- /test/README: -------------------------------------------------------------------------------- 1 | 2 | This directory is intended for PlatformIO Test Runner and project tests. 3 | 4 | Unit Testing is a software testing method by which individual units of 5 | source code, sets of one or more MCU program modules together with associated 6 | control data, usage procedures, and operating procedures, are tested to 7 | determine whether they are fit for use. Unit testing finds problems early 8 | in the development cycle. 9 | 10 | More information about PlatformIO Unit Testing: 11 | - https://docs.platformio.org/en/latest/advanced/unit-testing/index.html 12 | --------------------------------------------------------------------------------