Explanation for 32 bit sample resolution and verifying audio

[william-green]

The mic does 24 bit sample resolution that is left justified. The minimum 16-bit word alignment configuration is 32 bit. Shifting 8 bit will get it to 24 bit. In my case, the python capture scripts had to shift 16 bit to downscale it to 16 bit since wav encoding in Python doesn't support 24 bit sampling.

I was able to modify some code to output the audio data as binary encoded over serial USB. A python script captured the serial USB stream and wrote to a .wav file. The audio does indeed sound normal. Note that serial USB has limitations and audio fidelity won't be great. However, it is good enough to verify the mic is working. You need a high baud rate and a limited sample frequency to remain within serial USB limitations.

I remember increasing the DMA buffer sizing. This drastically reduced CPU usage on both the ESP and the host computer.

ESP code:

/*#include <Arduino.h>

void setup() {
  Serial.begin(115200);
  pinMode(2, OUTPUT);
}

void loop() {
  Serial.println("ESP32 is working!");
  digitalWrite(2, HIGH);
  delay(500);
  digitalWrite(2, LOW);
  delay(500);
}*/

/** 
 * ESP32 I2S Serial Plotter Example.
 * 
 * This example is based on the espressif-idf example and Public Domain.
 * @author maspetsberger 
 */

#include <Arduino.h>
#include <driver/i2s.h>

const i2s_port_t I2S_PORT = I2S_NUM_0;

void setup() {
  Serial.begin(460800);
  Serial.println("Configuring I2S...");
  esp_err_t err;

  // The I2S config as per the example
  const i2s_config_t i2s_config = {
      .mode = i2s_mode_t(I2S_MODE_MASTER | I2S_MODE_RX), // Receive, not transfer
      .sample_rate = 16000,                         // 16KHz
      .bits_per_sample = I2S_BITS_PER_SAMPLE_32BIT, // could only get it to work with 32bits
      .channel_format = I2S_CHANNEL_FMT_ONLY_RIGHT, // although the SEL config should be left, it seems to transmit on right
      .communication_format = i2s_comm_format_t(I2S_COMM_FORMAT_I2S | I2S_COMM_FORMAT_I2S_MSB),
      .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1,     // Interrupt level 1
      .dma_buf_count = 4,                           // number of buffers
      .dma_buf_len = 64                              // 64 samples per buffer (minimum)
  };

  // The pin config as per the setup
  const i2s_pin_config_t pin_config = {
      .bck_io_num = 14,   // BCKL
      .ws_io_num = 15,    // LRCL
      .data_out_num = -1, // not used (only for speakers)
      .data_in_num = 32   // DOUT
  };

  // Configuring the I2S driver and pins.
  // This function must be called before any I2S driver read/write operations.
  err = i2s_driver_install(I2S_PORT, &i2s_config, 0, NULL);
  if (err != ESP_OK) {
    Serial.printf("Failed installing driver: %d\n", err);
    while (true);
  }
  err = i2s_set_pin(I2S_PORT, &pin_config);
  if (err != ESP_OK) {
    Serial.printf("Failed setting pin: %d\n", err);
    while (true);
  }
  Serial.println("I2S driver installed.");
}

void loop() {

  // Read a single sample and log it for the Serial Plotter.
  int32_t sample = 0;
  size_t bytes_read = 0;
  esp_err_t result = i2s_read(I2S_PORT, &sample, sizeof(sample), &bytes_read, portMAX_DELAY);//i2s_pop_sample(I2S_PORT, (char *)&sample, portMAX_DELAY); // no timeout
  if(result == ESP_OK && bytes_read == sizeof(sample)){
    //sample >>= 16;//remove the padded 8 bits and downscale to 16 bit audio for wav compatibility
    sample >>= 8;
    int16_t out = (int16_t)(sample >> 8);
    Serial.write((uint8_t*)&out, sizeof(out));

    //Serial.println(sample);
    //Serial.write((uint8_t*)&sample, sizeof(sample));  // sends 4 bytes (32 bits)
    //Serial.write((uint8_t *)&sample, 2);  // for 16-bit output
  }
  
  /*if (bytes_read > 0) {
    Serial.println(sample);
  }*/
  
}

// actually we would need to call i2s_driver_uninstall(I2S_PORT) upon exit.

Python script 1 (capture the binary stream and write to file). This runs on host computer:

import serial

ser = serial.Serial("/dev/cu.usbserial-110", 460800)  # replace with your port

try:
    with open("mic_dump.raw", "wb") as f:
        while True:
            data = ser.read(4096)
            f.write(data)
except KeyboardInterrupt:
    print("capture stopped")
    ser.close()

Python script 2 (convert the raw binary data into a wav file). This runs on the host computer.

import numpy as np
from scipy.io import wavfile

with open("mic_dump.raw", 'rb') as f:
    raw = f.read()
    print(len(raw))

    #samples_32 = np.frombuffer(raw, dtype=np.int32)


    #samples_24 = samples_32 >> 8

    #samples_16 = (samples_24 >> 8).astype(np.int16)

    samples_16 = np.frombuffer(raw, dtype=np.int16)

    import matplotlib.pyplot as plt
    plt.plot(samples_16[:1000])
    plt.show()


    wavfile.write("mic_output.wav", 16000, samples_16)

[maspetsberger]

Hi William, thanks for your investigation!

As for the padding, from what I remember back then, I had it hooked up to the Arduino Serial Plotter it looked reasonably well. Do you know if the padding is from ESP32 IDF or from the MEMS microphone itself?
Probably the best would be to directly read I2S_BITS_PER_SAMPLE_16BIT if that's what's needed on the output (although looking at my comment it didn't work back in the days).

As for cpu efficiency, apart from an increased DMA buffer, you might also want to read multiple bytes in one go in your event loop, and pipe that to Serial.

Feel free to send a merge request if anything should be updated (my example is 7 years old 😆 ) or additional elements (e.g. Readme / python snippet) should be added to the examples.