Oído

Oído provides a streaming recognition variant with a latency of 7.2–22.4 seconds in full-sentence mode and a WER of 4.86% / 11.03%.

With the addition of an on-chip language model with 1.3 million parameters, Oído's WER drops to 3.33% / 7.15%.

After switching to a transducer head structure, Oído can achieve a WER of 2.99% / 6.67%.

Oído's int4 variant can be compressed to a size of 8.3 MB.

On the same chip, Oído's error rate is 2.3 to 2.8 times lower than the chip vendor's built-in recognizer.

Under conditions containing real noise and reverberation, Oído's recognition accuracy outperforms Whisper tiny.en and Moonshine tiny running at full precision on a laptop.

Oído's Spanish-language model reduces errors by 34% to 60% compared to the multilingual Whisper tiny.

Oído's firmware runs on a single core at 1.97 times the speed of audio duration, meaning a 4-second instruction can only be transcribed approximately 8.7 seconds after it is finished being spoken.

Oído's dual-core scheduling scheme produces incorrect results on actual silicon and has therefore been disabled.

Oído is a complete inference engine capable of running open-vocabulary English speech recognition on the ESP32-S3.

The core of Oído is a Conformer-CTC model with 13 million parameters.

Oído employs a quantization strategy of weight int8, per-frame dynamic int8 for activations, int8 precision for the attention mechanism, and leverages the chip's SIMD instructions.

On the LibriSpeech test-clean/test-other datasets, Oído's on-device computation achieves a word error rate of 3.70% / 8.23%, differing from the full-precision version by only 0.12 percentage points.

Oído is an open-source speech recognition project by the Lokutor team that runs on the ESP32-S3 chip, with recognition accuracy exceeding that of Whisper-tiny.en running on a laptop.

Oído uses NVIDIA's Conformer-CTC Small model, which has approximately 13 million (13M) parameters and has been processed with int8 quantization.

Oído's runtime platform is the ESP32-S3 with 8MB PSRAM, priced at approximately $5, with no discrete GPU and no dedicated NPU.

On the LibriSpeech benchmark, Oído's word error rate (WER) is 3.7 (clean) and 8.2 (other), while Whisper tiny.en on a laptop achieves 6.3 and 15.9.

In DEMAND noise environment testing, Oído's average WER is 8.4, while Whisper tiny.en's is 12.1.

The comparative data for Oído comes from the project team's own claims, and independent replication has not yet been conducted.

The Oído project has been open-sourced on GitHub at lokutor-ai/oido, and the team provides a live_demo.py that allows users to run the same inference pipeline used on the chip via a laptop microphone.