WhisperX
An enhanced version of OpenAI's Whisper model that introduces a forced-alignment mechanism, enabling precise word-level alignment of transcribed text to the audio timeline at millisecond accuracy. The system is suited for high-precision applications such as clinical speech analysis.
Timeline (last 90 days)
When running Whisper ASR, a TTS voice model, and nomic-embed-text simultaneously on an RTX 3090 with 24 GB of VRAM, the three resident models occupied 17.8 GB (72%) of VRAM, peaked at 23.4 GB over the course of a week, and never spilled over into system memory.
Whisper ASR (large-v3) averaged approximately 7.7 GB of VRAM over 7 days and peaked at approximately 10.3 GB.
Over the course of one week, six distinct callers hit the Whisper endpoint; Whisper was the largest consumer of GPU time, with the self-built assistant core accounting for 4,970 seconds via the Whisper proxy.
The automated VFI estimation system proposed in the study uses WhisperX as its speech recognition engine to provide word-level temporal alignment.
Forced alignment is a technology that, given audio and its corresponding text transcription, automatically computes the precise start and end timestamps of each word on the audio timeline; it operates in the opposite direction from ASR and is generally faster and more accurate than ASR.
All Facts (5)
When running Whisper ASR, a TTS voice model, and nomic-embed-text simultaneously on an RTX 3090 with 24 GB of VRAM, the three resident models occupied 17.8 GB (72%) of VRAM, peaked at 23.4 GB over the course of a week, and never spilled over into system memory.
50%UnverifiedWhisper ASR (large-v3) averaged approximately 7.7 GB of VRAM over 7 days and peaked at approximately 10.3 GB.
50%UnverifiedOver the course of one week, six distinct callers hit the Whisper endpoint; Whisper was the largest consumer of GPU time, with the self-built assistant core accounting for 4,970 seconds via the Whisper proxy.
50%UnverifiedThe automated VFI estimation system proposed in the study uses WhisperX as its speech recognition engine to provide word-level temporal alignment.
50%UnverifiedForced alignment is a technology that, given audio and its corresponding text transcription, automatically computes the precise start and end timestamps of each word on the audio timeline; it operates in the opposite direction from ASR and is generally faster and more accurate than ASR.
50%