Cascaded Architecture

The total latency of traditional cascaded voice dialogue architectures is typically above 1–3 seconds.

The benefit of a cascade architecture comes from the long-tail distribution of task difficulty: the vast majority of requests fall into simple categories, so reserving expensive models for the few hard samples lowers average cost with almost no change in quality.

The cost of hierarchical sparse design is increased engineering complexity, and it needs to be tuned for specific tasks, so it is more suitable for scenarios that are quality-sensitive and already have a clear evaluation set.

Cascading is a sequential execution strategy in which a request first goes to a low-cost model, and if its confidence or a verifier determines the result is unreliable, the same request is escalated to a stronger model for a retry.

Traditional cascaded systems consist of three components—a speech recognizer, a large language model, and a separate turn detector—which coordinate poorly with each other and require manual tuning

A cascade classification architecture cuts far more computation than any single round of underlying micro-optimizations can achieve.

Cascade/early-exit architectures use cheap lightweight classifiers that operate in the same latent space as mmBERT to make easy decisions first, routing only uncertain cases to the full model.

Adopting a cascading strategy—using a fast detector first, then collaborating with a VLM—is a viable engineering approach for balancing accuracy and efficiency.

A two-stage cascade architecture (fast detector + VLM collaboration) is an engineering solution to the VLM real-time problem, making the overall pipeline usable by reducing the number of frames and regions that the VLM needs to process.