Oído: Open-Vocabulary Speech Recognition on a $5 Microcontroller

Squeezing Speech Recognition into a $5 Chip

Running speech recognition on low-cost microcontrollers has long been limited to keyword wake-words or fixed command sets — your device can understand "lights on" or "lights off," but transcribing a full, freely spoken English sentence word by word has been essentially out of reach. The research project Oído breaks that boundary: it's a complete inference engine capable of running open-vocabulary English speech recognition on a single ESP32-S3.

The chip's specs are decidedly modest: a 240 MHz dual-core Xtensa LX7, 512 KB SRAM, 8 MB PSRAM, 16 MB flash — and no neural network accelerator of any kind. In other words, all compute comes from general-purpose CPU cores and their SIMD instruction sets. Getting end-to-end speech transcription to run on hardware like this is a genuine engineering feat.

Model & Quantization: Fitting 13M Parameters onto a Microcontroller

At the core of Oído is a 13-million-parameter Conformer-CTC model. To make it run on a chip with severely limited memory and bandwidth, the team applied an aggressive yet precise quantization strategy:

• Weights use int8
• Activations use per-frame dynamic int8
• Attention mechanisms also run at int8 precision, directly leveraging the chip's SIMD instructions

Critically, the entire computation schedule is designed around flash and PSRAM bandwidth constraints. Without a dedicated accelerator, memory bandwidth — not raw compute — is typically the real bottleneck. Carefully orchestrating data movement is what makes models like this viable in the first place.

Near-Lossless Accuracy

Quantization usually means accuracy loss, but Oído's results are surprisingly strong. On the LibriSpeech test-clean/test-other benchmarks, on-device inference achieves a WER of 3.70% / 8.23% — only 0.12 percentage points behind the full-precision version.

Adding an on-chip 1.3M-parameter language model further reduces WER to 3.33% / 7.15%; switching to a transducer head yields 2.99% / 6.67%; and an int4 variant compresses the model down to just 8.3 MB.

Conformer-CTC is one of the dominant architectures in modern speech recognition. The Conformer (Convolution-augmented Transformer) adds a convolution module alongside standard Transformer attention to capture local audio features — the two complement each other, enabling the model to handle both long-range dependencies and short-time acoustic detail. CTC (Connectionist Temporal Classification) is a sequence alignment approach that lets the model predict character sequences directly without frame-level labels, bypassing the complexity of traditional HMM-DNN pipelines. This combination performs well for both offline batch processing and streaming inference, representing a well-validated end-to-end ASR path alongside Whisper. Choosing Conformer-CTC over Whisper's encoder-decoder architecture has practical advantages for edge deployment: CTC decoding is computationally lighter and doesn't require autoregressive step-by-step generation, making it naturally suited to compute-constrained environments.

WER (Word Error Rate) is the standard evaluation metric in speech recognition — it aligns the hypothesis against the reference text, counts substitution, deletion, and insertion errors, and divides by the total number of reference words. LibriSpeech is a standard English speech dataset built from LibriVox audiobook recordings; the test-clean subset contains clear, low-noise samples, while test-other includes more challenging samples with heavier accents or lower recording quality. By convention, a WER below 5% on test-clean is considered competitive, and maintaining ~8% on test-other typically requires significantly larger models or more training data. Achieving these results on an extremely resource-constrained device means the information loss from quantization has been largely compensated for by careful training and calibration.

How It Compares to the Competition

What makes Oído truly impressive is the head-to-head comparison. On the same chip, its error rate is 2.3 to 2.8× lower than the vendor's built-in recognizer.

The cross-platform comparison is even more striking: under real-world conditions with noise and reverberation, Oído's accuracy outperforms full-precision Whisper tiny.en and Moonshine tiny running on a laptop. A model running on a $5 microcontroller producing fewer errors than a lightweight Whisper variant running on an x86 laptop genuinely overturns the intuition that "small hardware means small capability."

The Spanish language model is equally impressive — compared to multilingual Whisper tiny, it reduces errors by 34% to 60%.

Whisper is OpenAI's open-source speech recognition model family released in 2022, known for large-scale weakly supervised training across multilingual and multitask scenarios. Whisper tiny.en is the smallest English-optimized variant at ~39M parameters, and running it on a laptop CPU still takes several times real-time. Moonshine tiny is a lightweight speech recognition model from Useful Sensors optimized for edge devices and low-latency use cases. Oído's ability to outperform both of these models — running on far more capable hardware — under noisy and reverberant conditions is partly attributable to the Conformer-CTC architecture's precise acoustic feature modeling and training data composition optimized for the target scenario, rather than relying on raw parameter count. This demonstrates that in specific tasks and deployment contexts, architecture fit and training strategy often matter more than absolute model size.

Speed: Accurate, but Not Yet Real-Time

The other side of the engineering story is performance. On a real development board, the firmware runs at 1.97× audio duration on a single core. That means a 4-second voice command takes roughly 8.7 seconds to transcribe after it's spoken — accurate, but not real-time.

The team candidly disclosed one regret: a dual-core scheduling scheme produces incorrect results on actual silicon, so it was disabled. Getting single-core operation working at all is no small feat; parallel dual-core scheduling clearly remains an open problem.

Streaming Mode Mitigates Latency

To improve the user experience, Oído also offers a streaming variant that displays partial text while the speaker is still talking, delivering final results 4.1–11.1 seconds after 3.6–10.6 second segments end (versus 7.2–22.4 seconds in full-sentence mode), with a WER of 4.86% / 11.03%. The streaming approach trades a small accuracy penalty for perceptible real-time feedback.

Why This Matters

Oído's significance goes beyond a benchmark number. It demonstrates that open-vocabulary continuous speech recognition is feasible on cheap, general-purpose microcontrollers with no dedicated AI accelerators. This opens up significant possibilities for embedded, offline, and low-power applications — running entirely locally means no cloud dependency, no voice data uploads, privacy by default, and zero network latency or API costs.

From a technical standpoint, its value lies in pushing the combination of quantization + bandwidth-aware scheduling + SIMD utilization to its limits, rather than relying on hardware brute force. The project's code and models are fully open-sourced, giving developers who want to deploy speech capabilities on edge devices a directly referenceable and reproducible engineering reference.

Of course, the latency issues and silicon-level dual-core scheduling bug mean it's still some distance from a production-ready real-time experience. But as a fusion of research and engineering, Oído clearly marks a new upper bound for low-cost speech AI.