Belebele
A multilingual reading comprehension benchmark covering more than 100 languages, with questions written by native speakers. Because of its broad language coverage and human-authored questions, it is also frequently borrowed for retrieval evaluation.
Timeline (last 90 days)
The final model scored an average of 17.18% on the Belebele Dravidian language evaluation, with a best score of 25.00%.
Without fine-tuning, EmbeddingGemma-300M achieves only 51.7% to 60.8% recall@1 on the Belebele dataset for Tajik, Kazakh, Uzbek and Kyrgyz.
On the Belebele dataset, Saidzoda-CA-EmbeddingGemma-300M improved recall@1 for the four languages by 14.4 to 26.0 percentage points, reaching 75.2% to 77.7%.
Belebele is a multilingual reading comprehension dataset covering 122 languages, with each question accompanied by four candidate passages, commonly used to evaluate the cross-lingual retrieval ability of multilingual embedding models.
On the Belebele test set, Saidzoda-Tajik-E5-Large's top-10 retrieval accuracy improved from 92.2% for the baseline to 95.2%, but the confidence intervals overlap.
All Facts (5)
The final model scored an average of 17.18% on the Belebele Dravidian language evaluation, with a best score of 25.00%.
50%UnverifiedWithout fine-tuning, EmbeddingGemma-300M achieves only 51.7% to 60.8% recall@1 on the Belebele dataset for Tajik, Kazakh, Uzbek and Kyrgyz.
50%UnverifiedOn the Belebele dataset, Saidzoda-CA-EmbeddingGemma-300M improved recall@1 for the four languages by 14.4 to 26.0 percentage points, reaching 75.2% to 77.7%.
50%UnverifiedBelebele is a multilingual reading comprehension dataset covering 122 languages, with each question accompanied by four candidate passages, commonly used to evaluate the cross-lingual retrieval ability of multilingual embedding models.
50%UnverifiedOn the Belebele test set, Saidzoda-Tajik-E5-Large's top-10 retrieval accuracy improved from 92.2% for the baseline to 95.2%, but the confidence intervals overlap.
50%