Local Speech-to-Text on Mac: Whisper vs Parakeet Benchmark Comparison

Building a local dictation app on Mac means the choice of default speech recognition engine is a problem you have to solve first — and not by guessing. The author of the open-source dictation tool Plainsay built a reproducible test harness on Apple Silicon and put OpenAI's Whisper large-v3-turbo and NVIDIA's Parakeet TDT 0.6B v3 on the same track, running them twice on the same audio with the same evaluation criteria.

Why this benchmark is worth a look

There's no shortage of claims online about which speech recognition model is faster or more accurate, but very few of them can be re-run by anyone else. Rather than take vendors' or the community's word for it, the author wrote a roughly 240-line Swift command-line tool and committed the audio, scripts, and raw results from every run to the repo. Anyone can clone it and get their own numbers with a single command.

He deliberately avoided testing other commercial dictation apps. Driving someone else's closed-source software with a script is hard to do fairly and tends to turn into relaying unverifiable third-party data. This test only covers his own implementations of the two engines — the boundaries are clear, and it's easy for others to poke holes in it.

Dataset and test harness design

The test uses LibriSpeech's test-clean (CC BY 4.0). The original set contains 40 speakers and 2,620 utterances. The author drew a stratified sample of 50: one out of every four speakers in the sorted speaker list, for 10 speakers total; then 5 utterances from the first chapter of each. The final sample is about 1,002 reference words and 356 seconds of audio. That's small enough to manually inspect each transcription, and it still covers 10 different voices.

A few design details in the harness directly affect the credibility of the results. First, it calls the WhisperKitEngine and ParakeetEngine from the app's core module rather than writing a separate implementation just for testing, and dependency versions are pinned (WhisperKit 1.1.0, FluidAudio 0.15.6). Second, the audio path matches microphone capture: FLAC is first decoded with AVAudioFile, then resampled to 16 kHz mono Float32 with AVAudioConverter; all audio is read into memory ahead of time, so file I/O isn't counted in the timing. Third, only the transcribe stage is timed — model downloads and Core ML preparation happen outside the timing window, and no hidden warm-up is performed, so the slow first inference is honestly counted in the average latency.

Before scoring, a simple normalization is applied: reference text and transcriptions are lowercased, punctuation other than apostrophes is stripped, and whitespace is collapsed. LibriSpeech references are unpunctuated uppercase words, while Whisper output has casing and punctuation, and skipping this step would count formatting differences as recognition errors. WER is computed using word-based Levenshtein alignment.

LibriSpeech test-clean is one of the most widely used public benchmark datasets in speech recognition, derived from audiobook recordings read for the LibriVox project. It's all English, sampled at 16 kHz. The test-clean subset was manually verified to remove samples with obvious noise or unclear pronunciation, so WER tends to be low, and it's often used to compare models' upper bound under "ideal conditions." Its speakers are mostly native English speakers with limited accent diversity — which is exactly why the author emphasizes "English only, clean read speech only": performing similarly on this set doesn't mean performing similarly in real dictation scenarios.

WER (Word Error Rate) is the most central evaluation metric in speech recognition, computed as (substitutions + deletions + insertions) / total reference words. It uses word-based Levenshtein alignment to match the transcription against the reference word by word and count the minimum number of edit operations. WER is extremely sensitive to text formatting: casing, punctuation, abbreviations, and spelling variants all count as substitution errors, so normalization is usually required before evaluation. Here, Whisper's output includes punctuation and casing, and without normalization it would produce a large number of "false errors" — which is precisely why the author stresses the importance of normalization.

Real-Time Factor (RTF) is the ratio of the time needed to process audio to the audio's own duration. RTF below 1 means processing is faster than real time; the smaller, the faster. Here, Parakeet's RTF is 0.014, meaning it takes about 14 milliseconds to process 1 second of audio; Whisper's is 0.095, about 95 milliseconds. RTF reflects a model's scalability across different audio lengths better than absolute latency, and is a key metric for whether local dictation can truly feel like "speak and it's there."

Results: latency gap is clear, accuracy gap is not

Running on a MacBook Air (M4, 10-core CPU, 16 GB) with macOS 26.5.1 produced the following data:

Engine	WER	Average latency	Real-time factor
Parakeet TDT 0.6B v3	2.50% (25/1002)	0.10 s	0.014
Whisper large-v3-turbo	2.89% (29/1002)	0.68 s	0.095

Neither engine failed on any audio clip.

The latency gap is the clearest conclusion from this test. Looking at the per-utterance data, Parakeet's median is 0.088 s with a worst case of 0.19 s; Whisper's median is 0.64 s with a worst case of 1.04 s. On the first inference after preparation, Parakeet took 0.13 s and Whisper took 0.93 s — cold start affects Whisper far more. For push-to-talk dictation, Parakeet's results are nearly instantaneous, while Whisper has a perceptible short pause.

The accuracy gap, however, is not clear, and reading through the errors one by one explains why.

After reading the errors one by one: 0.39 percentage points may be noise

The author committed every transcription to the repo, so you can see exactly which words were counted as errors. A fair number of them aren't recognition mistakes at all:

• British vs. American spelling differences, in both directions. The reference has HONOUR and ENQUIRE, and both engines output honor and inquire; the reference has COLOR and CHECK, and Parakeet outputs colour and cheque. Each is counted as a full word error.
• Abbreviations. Whisper writes SAINT FRANCIS as "St. Francis" and MISTER WATSON as "Mr Watson."
• Quotation marks. In two clips, Whisper produced nested quotes, and because normalization keeps apostrophes to avoid splitting don't, 'we and him' can't be matched to the reference. That one alone accounts for 4 of Whisper's 29 errors; quote-related issues cost it 6 in total.
• Uncommon proper nouns. Margolotte becomes Margolot; Donnithorne becomes donnythorne in Whisper and Donny Thorne in Parakeet (counted as two errors); Silvia becomes Sylvia in both engines.

There are real errors, of course. Both engines hear BOOKKEEPER as "bit keeper," and Whisper turns the French word BADAUDERIE into "Bad-Dot-Grey."

The conclusion: on clean read speech, the 0.39-percentage-point WER gap between the two engines is on the same order as the noise introduced by normalization. The author notes that in an earlier commit's run (August 17), Parakeet had only 23 errors rather than 25 on the same audio. Before concluding that one model is more accurate, look at what the errors actually are. A more complete normalizer (covering spelling variants, abbreviations, edge apostrophes, similar to Whisper's official English text normalizer) should push both numbers down, but the author hasn't committed that version yet, so he doesn't cite recomputed results.

Text normalization is not optional in speech recognition evaluation — it's the precondition for whether a conclusion holds. Whisper's official English evaluation uses a fairly complex normalizer covering numbers, currency, abbreviations, spelling variants, edge apostrophes, and more, with the goal of preventing outputs that are "correct but formatted differently" from being misjudged as errors. This author only did lowercasing, punctuation removal, and whitespace collapsing, so British vs. American spellings (HONOUR vs honor), abbreviations (SAINT vs St.), and nested quotes ('we, him') all counted toward WER. This explains why the 0.39-percentage-point gap shouldn't be read directly as one model being better: it's very likely just a systematic bias from differences in normalizer coverage, not a difference in recognition ability.

What this test does not cover

The author proactively lists the benchmark's boundaries, and this part is more valuable than the results themselves:

• English only. LibriSpeech test-clean says nothing about Polish, German, and so on; Parakeet v3's multilingual capability and Whisper's language breadth are separate, unmeasured claims.
• Clean read speech only. Real dictation faces laptop microphones, room noise, hesitations, and corrections, and both engines' WER will be higher.
• The capture pipeline is skipped. Audio is decoded from files; microphone behavior and voice activity detection aren't tested.
• Only one machine. An M4 MacBook Air. Older M1 machines will be slower, Whisper especially.
• Just a snapshot. The app was later upgraded to FluidAudio 0.15.8, fixing an issue where the ends of some long recordings were dropped; and long dictation isn't in this sample at all, since LibriSpeech clips are all short.

Why Parakeet became the recommended default

For English push-to-talk dictation, latency largely determines the experience: on a few seconds of audio, the difference between 0.1 s and 0.68 s is noticeable. And on this small sample, the accuracy gap falls within the noise, so there's no clear winner. Whisper is still kept, for cases where it performs better on certain languages and accents, and because its punctuation style has its fans.

The author also invites others to run this test on different Macs, is especially eager to see numbers from M1/M2 machines, and welcomes PRs with a better normalizer or a noisier test set.

Run it yourself

git clone https://github.com/conrader/plainsay
cd plainsay
git checkout 2bc1552b19a5ae4094bd81714461baf4ebaa7202
swift build --product BenchmarkCLI
.build/debug/BenchmarkCLI Benchmark/data/librispeech-test-clean-manifest.json

Running it prints per-utterance progress, then gives each engine's WER, latency, and real-time factor, and writes results.json next to the manifest. The full methodology is documented in BENCHMARK.md, and the submitted results live in Benchmark/data/.

One-line Summary

Measured on an M4 Mac, Parakeet's latency is far lower than Whisper's, but the accuracy gap falls within the noise and isn't enough to declare a winner.

Paragraph Summary

The author of the open-source dictation tool Plainsay built a reproducible harness on Apple Silicon and compared Parakeet TDT 0.6B v3 against Whisper large-v3-turbo on the same LibriSpeech audio. Results show Parakeet at 0.10 s average latency and Whisper at 0.68 s — the latency gap is the test's clearest conclusion. Their WERs are 2.50% and 2.89%, just 0.39 percentage points apart, with many errors coming from normalization issues like British/American spelling, abbreviations, and quotes rather than recognition mistakes, so the accuracy gap falls within the noise. The test covers only English clean read speech on a single M4 machine, and doesn't address multilingual, noisy, long-audio, or microphone capture. Based on this, the author recommends Parakeet as the default engine for English push-to-talk scenarios, keeping Whisper for cases where language and accent handling are better.