TutlAit v1: A Crowdsourced Speech Dataset for Moroccan Tamazight

An Official Language Left Behind by Technology

Tamazight (also known as Amazigh) shares official language status with Arabic in Morocco, yet it remains severely under-resourced in the field of speech technology. Publicly available annotated audio data is scarce, regional dialect information is often missing, and transcription quality is inconsistent. This reality has left Tamazight largely absent from modern AI applications like automatic speech recognition (ASR) and speech translation.

The TutlAit dataset was created specifically to address this gap. It is a corpus that pairs Moroccan Tamazight speech with Modern Standard Arabic text, complete with explicit regional accent labels — providing a rare foundational resource for speech technology research on low-resource languages.

Tamazight belongs to the Berber branch of the Afroasiatic language family. It is the native tongue of the indigenous Berber people of North Africa (who call themselves Imazighen, meaning "free people") and is spoken across Morocco, Algeria, Tunisia, and beyond. The language comprises several regional varieties that differ significantly in phonology, vocabulary, and grammar, making it more of a dialect continuum than a single unified language. Morocco enshrined Tamazight as an official language in its 2011 constitution, yet Arabic and French continue to dominate in education, government, and media. In the NLP world, "low-resource language" refers to languages lacking large-scale annotated corpora — training modern neural speech models typically requires hundreds or even thousands of hours of high-quality labeled audio, a threshold Tamazight's publicly available data falls far short of. As a result, commercial voice assistants and automatic transcription tools have virtually no support for it.

Technical Architecture of the Crowdsourcing Platform

The dataset was collected through a purpose-built crowdsourcing web application using a fairly standard modern tech stack: React 18 on the frontend, Django 5 with Django REST Framework on the backend, and PostgreSQL for data storage.

The research team recruited native speakers through targeted outreach campaigns on LinkedIn and Instagram. After registering an account, volunteers declared their regional variety (Atlas, Souss, Rif, or Other) along with demographic information, then contributed through one of two workflows.

Two Complementary Collection Workflows

The first is Text-to-Audio: the system displays an Arabic sentence, and the volunteer records their spoken Tamazight equivalent in the browser. The second is Audio-to-Text: the system plays a Tamazight audio clip, and the volunteer types the Arabic transcription. These two bidirectional workflows complement each other — one generates speech from text, the other adds transcriptions to existing audio.

In addition to crowdsourced contributions, the team also extracted supplementary clips from freely available Tamazight audiovisual media, segmented and annotated them using ELAN, and imported them via a batch CSV/ZIP pipeline.

What is ELAN? ELAN (EUDICO Linguistic Annotator) is a multimedia linguistic annotation tool developed by the Max Planck Institute for Psycholinguistics in the Netherlands. It is widely used in fieldwork linguistics and corpus building. ELAN allows researchers to create multi-tiered hierarchical annotation tracks on audio/video timelines, precisely marking start and end timestamps for each utterance, speaker identity, transcription text, and linguistic notes — with export support for multiple common formats. In TutlAit's supplementary clip workflow, ELAN handled the segmentation of continuous media into sentence-level clips with text annotations, a necessary step for transforming raw broadcast or online video into trainable speech samples.

Rigorous Data Quality Control

Data quality management is critical to the usability of any speech corpus. TutlAit applies consistent server-side processing to every upload: audio is converted to 16kHz mono WAV format, deduplicated using SHA-256 hashing (duplicate files are rejected), checked for reasonable duration bounds, and ultimately verified by human administrators.

This pipeline is especially important in a crowdsourcing context, where uneven quality and duplicate submissions are inherent risks. The combination of hash-based deduplication, duration boundary checks, and manual review provides a reasonable guarantee of reliability for the ingested data.

How does SHA-256 deduplication work? SHA-256 is a cryptographic hash function that maps any file's content to a fixed 256-bit digest string. A single bit of difference between two files produces an entirely different hash; conversely, matching hashes indicate identical content with extremely high confidence. In a crowdsourced audio context, a volunteer repeatedly uploading the same recording — or two volunteers coincidentally submitting the same clip — will produce identical hashes, allowing the system to automatically reject duplicates without decoding the audio or performing any manual comparison. Standardizing to 16kHz mono WAV also ensures all audio shares the same sample rate and channel count, preventing format inconsistencies from introducing noise during model training.

Dataset Scale and Regional Distribution

The final TutlAit v1 dataset contains 13,384 audio files with a total duration of 75,231 seconds (approximately 20.9 hours, ~3.01 GB). The regional variety distribution is notably imbalanced:

• Atlas variety: 9,956 files, ~14.08 hours — the dominant portion
• Souss variety: 3,378 files, ~6.75 hours
• Rif variety: only 22 files
• Kabyle variety: only 28 files

Atlas and Souss together account for the vast majority of the data, while the Rif and Kabyle subsets are extremely small. This distribution reflects the practical limitations of crowdsourced recruitment — differences in participation rates across regional communities are directly reflected in data volume. For future models aiming to cover all varieties, the small Rif and Kabyle samples remain a clear shortcoming.

Applications and Significance

As positioned in the paper, the TutlAit corpus can be reused for three types of tasks involving Moroccan Tamazight: speech recognition, speech translation, and accent identification. The inclusion of regional accent labels makes it particularly valuable for accent identification and dialect research — most low-resource speech datasets do not distinguish between regional varieties.

From a broader perspective, TutlAit is a prime example of a "community-driven + engineered pipeline" approach to solving the data problem for low-resource languages. Rather than relying on expensive professional recording sessions, it uses a lightweight combination of social media recruitment, browser-based recording, automated deduplication, and human review to accumulate approximately 21 hours of annotated speech with limited resources. For the many languages around the world that face similar technological barriers, this replicable crowdsourcing methodology may ultimately be more valuable than the dataset itself.