Alibaba Qwen-Audio-3.1 Launches: Five-Model Full Audio Stack with ASR Pricing Cut by 95%

Alibaba's Qwen team has released the Qwen-Audio-3.1 series, delivering simultaneous upgrades across three core capabilities — Automatic Speech Recognition (ASR), Text-to-Speech (TTS), and real-time interaction (Realtime) — while introducing two new models: TTS-Next and ASR-Next. Together, these five models form a complete audio technology stack, covering everything from understanding and generation to interaction and creation across the full audio processing pipeline.

Beyond the capability upgrades, the most attention-grabbing aspect of this release is the dramatic price cuts: TTS down ~70%, Realtime down ~85%, and ASR slashed by up to 95%. Price adjustments of this magnitude typically signal that Alibaba is aggressively moving to capture developer mindshare in the audio API market.

Five Models, One Complete Audio Stack

Qwen-Audio-3.1 is positioned as "a complete audio stack," with a design logic that maps cleanly onto four distinct directions in audio processing: understanding, generation, interaction, and creation.

• ASR (Automatic Speech Recognition): Core speech-to-text functionality
• ASR-Next (Audio Understanding): Built for more complex audio comprehension tasks
• TTS (Text-to-Speech): Core text-to-audio synthesis
• TTS-Next (Audio Creation): Designed for generative audio content production
• Realtime (Real-Time Interaction): Enables live, simultaneous listen-and-respond conversations

This modular architecture lets developers pick and choose based on their needs, while giving the product line broader scenario coverage overall. Compared to previous single-point capability updates, this feels more like a systematic architectural overhaul.

ASR Upgrades: Multilingual Support, Dialects, and Native Polishing

The upgraded ASR shows meaningful improvements in multilingual and dialect recognition. Perhaps more noteworthy is the newly added "native polishing" feature — the system can automatically remove filler words (such as "um" and "uh") and repetitive phrases, producing cleaner and more coherent transcription output.

For use cases like meeting transcription, interview processing, and subtitle generation, this feature can significantly reduce the manual editing workload. Traditional ASR aims for transcription accuracy; this goes a step further by optimizing for readability.

ASR-Next: From Recognition to Understanding

ASR-Next is the newly introduced audio understanding model, and its capabilities go far beyond conventional speech recognition:

• Multi-speaker identification with speaker labels, timestamps, and aligned transcripts
• Emotion recognition, environmental sound understanding, and machine sound detection, enabling sound captioning and event localization
• Support for audio QA and audio reasoning

This means the model no longer just "turns sound into text" — it can answer questions like "what's happening in this audio clip," "who is speaking," and "what's that background noise?" These capabilities have direct value for content moderation, multimedia search, and intelligent surveillance applications.

Sound Captioning and Event Localization are two core subtasks in audio understanding that go well beyond traditional speech recognition. Sound captioning refers to describing audio content in natural language — for example, "someone is typing on a keyboard with air conditioner white noise in the background." Event localization involves precisely marking the start and end times of specific sound events on a timeline — such as identifying the exact moment a glass breaks in a surveillance recording. Both capabilities typically rely on multimodal models pre-trained on large-scale audio-text pair datasets. Representative work includes Google's AudioCaps dataset and the CLAP (Contrastive Language-Audio Pretraining) framework. Integrating these capabilities into an API means developers can leverage audio semantic understanding for content moderation, security monitoring, podcast search, and other use cases without having to train specialized models themselves.

TTS Upgrades and TTS-Next: From Synthesis to Creation

The updated TTS supports multilingual and dialect synthesis, along with natural cross-lingual voice transfer — the same voice can maintain consistency across different languages. Users can control emotion, speaking pace, and style through simple instructions, lowering the barrier to fine-grained audio tuning.

TTS-Next is the more ambitious new model. Built on a unified Language Model (LM) + Diffusion framework, it can simultaneously generate vocals, sound effects, and background audio in a single inference pass. Target use cases include audiobooks, podcasts, gaming, and advertising.

Combining LM with diffusion for audio creation is one of the leading technical approaches in generative audio today. The ability to produce multi-layered audio content in a single generation pass could significantly compress production workflows and costs for content creators.

The Language Model (LM) + Diffusion framework is one of the dominant technical approaches in generative audio. The language model handles the semantic structure, rhythm, and speaking style of audio at the discrete token level, producing high-level audio representations. The diffusion model then operates in continuous signal space to recover fine-grained detail, iteratively denoising the high-level representation into high-fidelity waveforms or mel spectrograms. The combined strength of this architecture lies in the LM's ability to capture long-range dependencies and temporal relationships across multiple elements (vocals, sound effects, background audio), while the diffusion model excels at generating natural, nuanced audio textures. This framework has been validated across multiple open-source and commercial audio generation projects, including Stability AI's AudioLDM series and Meta's AudioCraft, all sharing the goal of producing structurally complete, multi-layered audio content in a single inference pass — without relying on multi-step post-production mixing.

Realtime: An Interaction That Feels Like a Real Phone Call

The Realtime model emphasizes "listening while speaking," supporting interruptions at any time for an experience closer to a real phone conversation rather than a rigid turn-based exchange.

One interesting detail: when the model detects that a user seems emotionally down, it will proactively slow its speech pace and respond with greater empathy. This kind of emotional state responsiveness is a signal that voice interaction is moving from "utilitarian" toward "human-like" — and it's a meaningful differentiator from purely functional voice assistants.

The Market Logic Behind the Aggressive Price Cuts

The across-the-board price reductions are substantial: TTS down ~70%, Realtime down ~85%, and ASR down by up to 95%. Pricing is one of the key factors in scaling audio API adoption at volume — especially for high-frequency use cases like ASR and real-time interaction, where cost sensitivity is extremely high.

Reducing ASR costs to roughly 5% of their previous level is essentially using price as leverage to lower the barrier to entry for developers, encouraging more applications to treat voice capabilities as a standard feature. Combined with the capability upgrades, Qwen-Audio-3.1's strategy is a two-pronged push: stronger capabilities and lower prices at the same time.

Summary

Qwen-Audio-3.1 builds a complete audio technology stack spanning understanding, generation, interaction, and creation across five models. With ASR-Next's audio comprehension capabilities, TTS-Next's unified audio creation, and Realtime's human-like interaction, this release demonstrates Alibaba's systematic push into audio multimodal AI. Paired with aggressive price cuts, the appeal of this series to developers is worth watching closely. The blog and select model APIs are already live, with more APIs reportedly coming soon.