[KongchangAI]
· 2 min read· 1,259 words

TID Team's Speech-COSER Challenge ASR System: LoRA Curriculum Learning + Pruned Ensemble

TID Team's Speech-COSER Challenge ASR System: LoRA Curriculum Learning + Pruned Ensemble

TID uses CTC alignment scores to drive two-stage LoRA curriculum learning, combined with pruning and a seven-system ROVER ensemble, achieving 22.17% normalized WER in the rural elderly spontaneous speech ASR challenge.

Telefónica Innovación Digital (TID) proposed a pragmatic ASR system for spontaneous speech from elderly rural populations in Spain at the Speech-COSER Challenge 2026. The core is using CTC pseudo-forced alignment scores to assess audio-text matching quality and organize two-stage LoRA curriculum learning: stable adaptation on high-matching data first, then gradual introduction of low-matching samples. Post-hoc magnitude pruning of LoRA matrices suppresses overfitting, and a seven-system ROVER ensemble achieves 36.25% non-normalized and 22.17% normalized WER. With no new architecture, the method offers a reusable approach for low-data, high-domain-shift ASR tasks.

TID Team's Speech-COSER Challenge ASR System: LoRA Curriculum Learning + Pruned Ensemble

In the ASR track of the Speech-COSER Challenge 2026, Telefónica Innovación Digital (TID) submitted a recognition system targeting spontaneous speech from elderly rural populations. This type of speech data has long been a challenge for ASR due to heavy accents, irregular speech rates, and limited recording conditions. Rather than stacking complex models, TID's approach focuses on data quality filtering and parameter-efficient fine-tuning, ultimately achieving WERs of 36.25% and 22.17% on the non-normalized and normalized test sets, respectively, with a seven-system ROVER ensemble.

TID system paper source screenshot

Challenge Background: The Difficulty of Recognizing Spontaneous Speech from Rural Elderly Speakers

The Speech-COSER Challenge 2026 focuses on spontaneous speech from elderly speakers in rural Spain. Unlike read speech, spontaneous speech contains numerous pauses, repetitions, and disfluencies, and when combined with pronunciation variation and dialect influence from elderly speakers, it places high demands on ASR system robustness. In their system description, TID notes that these characteristics prompted them to abandon conventional end-to-end fine-tuning in favor of addressing both training data matching quality and model generalization capability.

Core Method: CTC Pseudo-Forced Alignment-Driven Two-Stage LoRA Curriculum

The first layer of TID's design uses CTC-based pseudo-forced alignment to score each audio-text pair, measuring their matching quality. This score is then used to organize two-stage LoRA curriculum learning: first, the model is allowed to stably adapt on data with higher matching quality, and then samples with lower matching quality and greater noise are gradually introduced.

The idea of curriculum learning is not new, but TID bases the curriculum ordering on forced alignment scores between audio and text, rather than simple loss values or sample lengths. This approach essentially uses the CTC model's own capability to perform an initial round of data cleaning, deferring clearly mismatched samples and reducing the interference of low-quality data on LoRA fine-tuning.

CTC (Connectionist Temporal Classification) is a loss function and decoding framework commonly used in speech recognition training, whose core advantage is that it can train sequence models without frame-level alignment annotations. Pseudo-forced alignment, given existing transcript text, uses a CTC model to compute the optimal alignment path between audio and text, from which a matching score for each sample (such as average posterior probability or alignment confidence) is derived. A higher score indicates a more likely correspondence between audio and text; a low score suggests transcription errors, segmentation misalignment, or noise interference. TID uses this score as the curriculum ordering signal, fine-tuning first on high-confidence samples and gradually incorporating low-confidence samples, thereby reducing early interference from noisy data on the LoRA adaptation matrices.

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method that freezes the main weights of a pre-trained model and injects trainable low-rank matrices (A and B) as bypasses only in layers such as the Transformer's attention projections, approximating weight updates through low-rank decomposition. Compared with full fine-tuning, LoRA greatly reduces the number of trainable parameters and memory usage, and the adaptation matrices can be saved and swapped independently. In scenarios with limited challenge data scale and pronounced domain shift, LoRA both avoids catastrophic forgetting and facilitates subsequent post-processing such as pruning of the adaptation matrices.

Post-Processing: LoRA Matrix Magnitude Pruning to Improve Generalization

After two-stage LoRA fine-tuning, TID applied post-hoc magnitude pruning to the LoRA matrices. LoRA itself only trains low-rank adaptation matrices, with far fewer parameters than full fine-tuning, but these matrices may still contain redundancy. By pruning weights with smaller magnitudes, the team hoped to reduce overfitting while maintaining adaptation capability, improving the model's generalization performance on the challenge test set.

Pruning occurs after training is complete and is a post-processing step that does not change the original training pipeline. For challenge scenarios with limited data scale and pronounced domain shift, this lightweight post-processing is a cost-effective regularization technique.

Ensemble Strategy: Seven-System ROVER Fusion

The final submitted system combines seven models through ROVER (Recognizer Output Voting Error Reduction). ROVER generates the final transcription by aligning and voting on the outputs of multiple systems, which can to some extent cancel out random errors from individual systems. TID does not disclose in the abstract the specific differences among the seven systems (such as different random seeds, different curriculum stages, or different pruning ratios), but the benefit of the seven-way ensemble is directly reflected in the final WER: 36.25% non-normalized and 22.17% normalized.

The notable gap between normalized and non-normalized WER indicates that text normalization (such as number and punctuation handling) has a significant impact on this test set, which is also a common phenomenon in spontaneous speech evaluation.

ROVER (Recognizer Output Voting Error Reduction) is a classic ASR system ensemble algorithm. It first constructs a word confusion network by aligning the output transcriptions of multiple systems at the word level, then selects the final word at each alignment position by voting based on occurrence frequency. ROVER is sensitive to complementarity among systems: if the error patterns of the systems differ greatly, voting can effectively cancel out random errors; if the systems are highly homogeneous, the benefit is limited. TID's seven-system ensemble does not disclose the specific sources of diversity, but diversity can typically be created through different random seeds, checkpoints from different curriculum stages, or different pruning ratios.

WER (Word Error Rate) is the core metric for ASR evaluation, computed as (substitutions + deletions + insertions) / total reference words. Normalized WER refers to applying uniform text normalization (such as number transcription, punctuation removal, and case unification) to both the reference text and system output before scoring, while non-normalized WER preserves original formatting differences. A large gap between the two typically reflects that the system makes more errors at the formatting level rather than the semantic level, which is particularly common in spontaneous speech and colloquial text.

Method Insights: Pragmatic Choices in Low-Data Scenarios

TID's system does not introduce new model architectures, but instead focuses on data filtering, parameter-efficient fine-tuning, and post-processing pruning. For ASR tasks targeting specific populations with constrained data collection conditions, this combined strategy offers a reusable approach: first filter data using alignment scores, then use LoRA to control fine-tuning costs, and finally use pruning and ensembling to stabilize generalization. The 36.25% non-normalized WER indicates that the task itself remains highly challenging, but the 22.17% normalized result demonstrates the effectiveness of this pipeline.

(Note: All method details in this article come from the TID system description; the specific configurations of the seven systems in the ensemble are not elaborated in the original material.)

One-line Summary

TID uses CTC alignment scores to drive two-stage LoRA curriculum learning, combined with pruning and a seven-system ROVER ensemble, achieving a normalized WER of 22.17% in the rural elderly spontaneous speech ASR challenge.

Paragraph Summary

Telefónica Innovación Digital (TID) proposed a pragmatic recognition system for spontaneous speech from elderly rural populations in Spain in the ASR track of the Speech-COSER Challenge 2026. The core of the approach is to use CTC pseudo-forced alignment scores to assess the matching quality of each audio-text pair and organize two-stage LoRA curriculum learning accordingly: first stably adapting on high-matching data, then gradually introducing low-matching samples. After training, post-hoc magnitude pruning is applied to the LoRA matrices to suppress overfitting and improve generalization. The final submission ensembles seven models via ROVER, achieving WERs of 36.25% and 22.17% on the non-normalized and normalized test sets, respectively. The method introduces no new architecture, focusing instead on data filtering, parameter-efficient fine-tuning, and lightweight post-processing, offering a reusable approach for ASR tasks with limited data and strong domain shift.

#ASR#LoRA#curriculum learning#CTC pseudo-forced alignment#ROVER ensemble#Speech-COSER Challenge#model pruning#spontaneous speech recognition
Share:

Related articles