FD-VAD
A semantic voice activity detection method designed for full-duplex spoken dialogue. It frames endpoint detection as a causal audio-language reasoning task, combining a frozen speech encoder, a lightweight modality adapter, and a parameter-efficiently adapted language model to perform streaming semantic turn detection without requiring ASR.
Timeline (last 90 days)
FD-VAD is a method that models semantic endpoint detection as a causal audio-language reasoning task, directly mapping from a limited causal audio window to a binary continue/stop decision without ASR involvement.
The architecture of FD-VAD consists of a frozen speech encoder, a lightweight modality adapter, and a parameter-efficiently adapted language model.
FD-VAD employs a last-chunk training objective specifically designed for streaming inference.
FD-VAD uses a confidence-gated endpoint submission mechanism that balances interruption risk and latency through a confidence threshold.
FD-VAD uses boundary-focused hard negative sampling to strengthen the model's ability to discriminate ambiguous turn boundaries during training.
Under the zero-shot setting on the TurnBench development set, FD-VAD achieves a maximum EOT recall of 0.853 under the constraint of false positive rate FP≤0.10.
FD-VAD outperforms multiple streaming and non-streaming semantic turn classifier baselines in both in-domain and conversational scenario evaluations.
FD-VAD's ASR-free architecture can bring shorter response chains and lower latency for full-duplex interaction scenarios such as voice assistants, real-time simultaneous interpretation, and customer service robots.
All Facts (8)
FD-VAD is a method that models semantic endpoint detection as a causal audio-language reasoning task, directly mapping from a limited causal audio window to a binary continue/stop decision without ASR involvement.
50%UnverifiedThe architecture of FD-VAD consists of a frozen speech encoder, a lightweight modality adapter, and a parameter-efficiently adapted language model.
50%UnverifiedFD-VAD employs a last-chunk training objective specifically designed for streaming inference.
50%UnverifiedFD-VAD uses a confidence-gated endpoint submission mechanism that balances interruption risk and latency through a confidence threshold.
50%UnverifiedFD-VAD uses boundary-focused hard negative sampling to strengthen the model's ability to discriminate ambiguous turn boundaries during training.
50%UnverifiedUnder the zero-shot setting on the TurnBench development set, FD-VAD achieves a maximum EOT recall of 0.853 under the constraint of false positive rate FP≤0.10.
50%UnverifiedFD-VAD outperforms multiple streaming and non-streaming semantic turn classifier baselines in both in-domain and conversational scenario evaluations.
50%UnverifiedFD-VAD's ASR-free architecture can bring shorter response chains and lower latency for full-duplex interaction scenarios such as voice assistants, real-time simultaneous interpretation, and customer service robots.
50%