31.6M-Parameter Turkish Speech Recognition Model: ufakzeka-kulak Open-Sourced

The Turkish speech recognition space has welcomed a lightweight open-source solution. ufakzeka-kulak is an open-source Turkish speech-to-text model with only 31.6M parameters, built with a FastConformer encoder, a token-and-duration transducer (TDT), and a CTC head. It was trained from scratch and released under the Apache-2.0 license. It outputs Turkish text with capitalization and punctuation, with numbers rendered as words. For developers with limited resources who need commercial compliance, this project offers a sample worth studying.

Training Data: Strictly Filtered 4,977 Hours

The model was trained on 4,977 hours of data, and all sources are licensed for commercial use. Of this, 281 hours are human-transcribed, while the training text for the remaining 4,696 hours comes from uploader subtitles or machine transcription—but not adopted directly. A sample is retained only when a second, independent reading of the audio agrees with the text. This cross-validation strategy effectively controls the proportion of noisy labels in the absence of human annotation.

In terms of data scale, nearly 5,000 hours is on the small-to-medium side for speech recognition training, but through strict filtering, the model demonstrates competitiveness on limited data.

What is adversarial filtering (the cross-validation strategy)?

In speech recognition data filtering, when the same audio is transcribed separately by two independent automatic recognition systems (or different decoding configurations of the same system), if the two output texts are highly consistent, it indicates that the audio's acoustic signal is sufficiently clear and the transcription is trustworthy. This method is often called "adversarial filtering" or "self-training filtering." It can remove noisy samples from massive weakly supervised data without human annotation. Probabilistically, the chance that two independent models make the same mistake on noisy audio is far lower than that of a single model, so consistency checks can effectively reduce label noise. In this article, the 4,696 hours of data were retained precisely through this kind of second independent reading cross-validation, allowing nearly 5,000 hours of data to maintain usable quality despite the lack of human annotation.

Evaluation Results: Better Than Whisper base and small

According to the original report, with greedy TDT decoding, the model's normalized word error rate (WER) is 0.0921 (95% confidence interval 0.0792 to 0.1055) on the FLEURS Turkish test set and 0.0916 (0.0873 to 0.0958) on the Common Voice 27.0 Turkish test set. Both are lower than Whisper base and Whisper small, and the paired intervals are entirely below zero—meaning the performance advantage is statistically significant.

Compared with Whisper large-v3 (greedy decoding) at 0.0604 and 0.0767, ufakzeka-kulak still lags behind, but considering the vast difference in parameter scale, this result is impressive. In conversational speech scenarios, WER rises to 0.3807, indicating that colloquial, informal settings remain a challenge for lightweight models.

FLEURS and Common Voice: Characteristics of the Two Benchmark Sets

FLEURS is a benchmark released by Google, the Few-shot Learning Evaluation of Universal Representations of Speech, covering 102 languages. Its Turkish test set contains about 10 hours of read speech, recorded in relatively clean environments, and is often used to measure the zero-shot or fine-tuned performance of multilingual models. Common Voice, by contrast, is crowd-recorded by the Mozilla community; the Turkish 27.0 test set contains a large amount of volunteer speech from different microphones, accents, and background noise, making it closer to real-world scenarios. The two datasets represent "controlled reading" and "crowdsourced diversity" respectively, and reporting results on both provides a more comprehensive reflection of model robustness. Note that WER on FLEURS and Common Voice cannot be directly compared, because text normalization rules and speaker distributions differ.

Paired Confidence Intervals and Statistical Significance

The article mentions that "the paired intervals are entirely below zero," referring to the 95% confidence intervals obtained via paired bootstrap or paired approximate randomization tests. The paired method treats the difference in error rates between two models on the same test samples as the statistic and repeatedly resamples to estimate the distribution of the difference. If the entire confidence interval lies below zero, it indicates that at the 5% significance level, the target model's WER is significantly lower than that of the comparison model. Compared with reporting only point-estimate WER, confidence intervals reflect the uncertainty introduced by test set size—especially when the test set contains only a few thousand utterances, a WER difference of 0.01 may not be significant. The intervals reported here are entirely below zero, providing statistical support for "better than Whisper base and small."

Engineering Practice: Scoring the Test Set Only Once

The project's experimental design deserves attention. The training recipe, checkpoints, annotations, and export scheme were all determined on development data, and the test set was scored only once in a pre-fixed release gate. This approach avoids test set leakage and the optimistic bias caused by repeated tuning, and is worth advocating in both academia and industry.

In addition, the original article reports punctuation and capitalization performance, robustness under degraded audio, and how closely each released export file matches the trained model. These details are crucial for practical deployment, but the abstract does not give specific values.

Release Gating and Test Set Leakage

Release gating is an engineering evaluation discipline: during model development, all hyperparameters, data recipes, checkpoint selection, and export formats are adjusted only on the dev set; the test set is "locked" and run only once before final release according to a predefined single scoring rule, and that score directly determines whether to release. This practice stems from the "test set leakage" problem in machine learning—if the test set is repeatedly evaluated and used for tuning, it effectively becomes a development set, and reported performance suffers from optimistic bias. Pre-fixing the scoring rules (such as fixed decoding methods, normalization procedures, and confidence interval calculation methods) further eliminates room for cherry-picking. For small teams with limited resources, this discipline is more important than large-scale computation for ensuring the credibility of evaluation conclusions.

Application Space for Lightweight Models

The 31.6M parameter scale means the model can run on edge devices or in resource-constrained environments, and the Apache-2.0 license removes obstacles to commercial use. For products that need Turkish speech recognition—such as subtitle generation, voice assistants, and customer service quality inspection—this model provides a localized option that does not rely on large APIs. However, the high WER on conversational speech suggests that it still needs further optimization in complex acoustic scenarios.

According to analysis from Bilibili UP hosts and YouTube channels (single source), community interest in such lightweight multilingual speech models is rising, but the ecosystem for Turkish-specific models remains relatively weak. The open-sourcing of ufakzeka-kulak provides a reproducible baseline for future research.

Conclusion

ufakzeka-kulak demonstrates a path to building a usable speech recognition model with limited data and parameters: strict data filtering, careful evaluation design, and a transparent release process. It surpasses Whisper models of similar size on standard test sets, but there is still room for improvement in conversational speech and in closing the gap with large models. For developers and researchers interested in Turkish speech technology, this project is worth adding to the toolbox.

One-line Summary

A 31.6M-parameter Turkish speech recognition model is open-sourced, surpassing similarly sized Whisper under strict data filtering.

Paragraph Summary

ufakzeka-kulak is an open-source Turkish speech-to-text model with only 31.6M parameters, built with a FastConformer encoder, TDT, and CTC head, trained from scratch and released under Apache-2.0, outputting text with capitalization and punctuation. The training data totals 4,977 hours, of which 281 hours are human-transcribed and the rest filtered through a second independent reading cross-validation to control noisy labels. On the FLEURS and Common Voice 27.0 test sets, WER is 0.0921 and 0.0916 respectively, significantly better than Whisper base and small, though still behind Whisper large-v3; conversational speech WER rises to 0.3807. The project adopts a release-gating design that scores the test set only once, avoiding test set leakage and providing a commercially usable lightweight Turkish recognition baseline for resource-constrained scenarios.