AI Voice Analysis Automatically Estimates Verbal Fluency Index in Motor Neuron Disease Patients

Background: Why Monitor Cognitive Impairment in MND Patients

Patients with Motor Neuron Disease (MND) frequently develop cognitive impairment (CI) as the disease progresses, making timely monitoring essential for guiding treatment and care decisions. However, because MND patients commonly experience dysarthria, traditional cognitive assessment tools are often difficult to administer accurately — speech difficulties and altered speaking rates can distort evaluation results.

The Edinburgh Cognitive and Behavioural ALS Screen (ECAS) is the gold-standard tool for assessing cognitive impairment in MND patients, with the Verbal Fluency Index (VFI) serving as one of its core metrics. VFI measures a patient's ability to generate specific words within a time limit, providing an indirect indicator of cognitive and executive function. This research focuses on how to achieve automated estimation of VFI.

The ECAS was specifically designed to address the unique assessment needs of MND patients. Unlike the MMSE or MoCA scales commonly used for Alzheimer's disease, ECAS deliberately avoids test items that heavily rely on motor function or prolonged verbal expression. It divides cognitive domains into "ALS-specific" (including executive function, language, and fluency) and "ALS non-specific" (including memory and visuospatial ability) categories. The VFI corresponds to a phonemic fluency task — patients are asked to name as many words as possible starting with a given letter within 60 seconds. This task is highly sensitive to frontal lobe executive function; approximately 30–50% of MND patients show notable decline on such tasks, and some eventually progress to ALS combined with frontotemporal dementia (ALS-FTD). Because VFI holds early-warning value for cognitive decline, automating its estimation carries significant clinical importance.

Technical Approach: ASR Alignment and Pause Modelling

The study proposes a system for automatically estimating VFI by combining modern automatic speech recognition (ASR) technology with voice activity detection (VAD), supplemented by refined timestamp processing.

Specifically, the system uses WhisperX as the speech recognition engine to transcribe patient speech into text and provide word-level temporal alignment. Silero is incorporated as the voice activity detection module to precisely capture boundaries between speech segments and pauses. Through fine-grained timestamp processing, the system can identify pause patterns throughout a patient's speech — pause frequency and duration are directly related to cognitive load and lexical retrieval difficulty, making them clinically significant for fluency assessment.

Within this framework, the research team can not only predict VFI scores but also extract multiple clinically interpretable metrics, ensuring that evaluation results are not a black-box output but can instead be cross-referenced with clinical judgment.

WhisperX is an enhanced version of OpenAI's Whisper model, with its key improvement being a forced alignment mechanism that precisely aligns each transcribed word to the audio timeline at millisecond-level accuracy. This is critical for automated fluency task scoring, since counting valid words requires not only recognizing the words themselves but also confirming they were produced within the designated time window. Silero VAD is a lightweight neural network voice activity detection model capable of distinguishing speech frames from non-speech frames (silence, breathing, background noise, etc.), with strong robustness to low signal-to-noise ratios and pathological speech. Combining the two gives the system simultaneous access to both "what words were said" and "when they were said and when the patient paused," providing the foundation for extracting clinically relevant features such as pause frequency, pause duration, and inter-word intervals.

Experimental Results: Clinically Inspired Features Come Out on Top

The study was evaluated on a unique MND speech dataset, comparing multiple feature extraction approaches and regression algorithms.

Results showed that this method significantly outperformed systems based on traditional acoustic features and self-supervised embeddings. Among all feature sets, clinically inspired features demonstrated the most consistent and outstanding performance. The quantitative results are notably impressive:

• P-words (words starting with the letter P): R² of 0.9, with a normalized root mean square error (NRMSE) as low as 0.05;
• S-words (words starting with the letter S): R² of 0.8, with NRMSE of 0.08.

These metrics indicate a strong correlation and low error between model predictions and actual clinical scores, demonstrating the technical feasibility of automated VFI estimation.

Self-supervised embeddings here refer to speech representations pre-trained on large-scale unlabeled speech data — typical examples include high-dimensional feature vectors output by models such as wav2vec 2.0 and HuBERT. Such embeddings can capture rich acoustic and linguistic information from speech and often excel at various speech tasks involving healthy speakers. However, in this study they underperformed relative to clinically inspired features, likely because the acoustic characteristics of MND pathological speech differ substantially from the pre-training data distribution, limiting the embeddings' generalizability. Additionally, these high-dimensional embeddings lack clinical interpretability, making it difficult to align model predictions with clinician judgment. R² (coefficient of determination) measures how much variance in the true scores is explained by model predictions — an R² of 0.9 means 90% of score variation is captured by the model. An NRMSE of 0.05 means the average prediction error is only 5% of the scoring range. Together, these metrics indicate that the model's predictive accuracy is approaching clinically practical levels.

Significance and Limitations of This Research

From an applied perspective, this work demonstrates the potential of speech AI in monitoring neurodegenerative diseases. For conditions like MND, where verbal ability progressively deteriorates, a system capable of automatically and objectively assessing cognitive status can reduce clinical staffing burden and enable more frequent, more consistent follow-up monitoring.

The strong performance of clinically interpretable features also sends an important signal: in medical AI applications, domain knowledge-driven feature engineering need not lag behind general-purpose deep learning embeddings — it may in fact be superior in both interpretability and performance. This is particularly important for applications that require physician trust and clinical review.

That said, as an early-stage study (an arXiv preprint, not yet peer-reviewed), it does have limitations. The research relies on a single MND dataset, and whether the sample size and diversity are sufficient to support generalizability remains to be verified. The performance gap between P-words and S-words tasks (R² 0.9 vs. 0.8) also suggests that different task types pose distinct challenges for the model. Future replication on larger, multi-center datasets will be necessary before this approach can realistically move toward clinical deployment.