Qwen2.5-Omni
A native multimodal large model released by Alibaba's Tongyi Qianwen team. It can accept text, image and audio inputs simultaneously within a single model and supports end-to-end multimodal understanding.
Timeline (last 90 days)
Qwen2.5-Omni is a native multimodal model that can receive text, image, and audio inputs simultaneously within the same model, but its GGUF-converted version currently only retains audio input capability, and the output side still requires TTS.
On M4 Max, the text route using Omni 3B + TTS 1.7B (4.61 GB) has a median full-turn latency of 4.27 seconds, with a peak process RSS of 3.86 GiB.
The small model pair halves the download size and reduces peak memory by about 35% compared with the original model pair, but in this trial the full turn was actually slower.
On M4 Max, Omni 3B has a median dialogue generation time of 171 ms and a peak process RSS of 2.57 GiB; Qwen3.5 0.8B has a median dialogue generation time of 96 ms and a peak process RSS of 1.08 GiB; Qwen3-TTS 1.7B has a synthesis time of 1.78 seconds and a peak process RSS of 3.85 GiB.
Sona's offline tokenizer starts from a frozen Qwen2.5-Omni multimodal model and takes two types of input: the mel spectrogram of the first 90 seconds of a track and the track's metadata.
All Facts (5)
Qwen2.5-Omni is a native multimodal model that can receive text, image, and audio inputs simultaneously within the same model, but its GGUF-converted version currently only retains audio input capability, and the output side still requires TTS.
50%UnverifiedOn M4 Max, the text route using Omni 3B + TTS 1.7B (4.61 GB) has a median full-turn latency of 4.27 seconds, with a peak process RSS of 3.86 GiB.
50%UnverifiedThe small model pair halves the download size and reduces peak memory by about 35% compared with the original model pair, but in this trial the full turn was actually slower.
50%UnverifiedOn M4 Max, Omni 3B has a median dialogue generation time of 171 ms and a peak process RSS of 2.57 GiB; Qwen3.5 0.8B has a median dialogue generation time of 96 ms and a peak process RSS of 1.08 GiB; Qwen3-TTS 1.7B has a synthesis time of 1.78 seconds and a peak process RSS of 3.85 GiB.
50%UnverifiedSona's offline tokenizer starts from a frozen Qwen2.5-Omni multimodal model and takes two types of input: the mel spectrogram of the first 90 seconds of a track and the track's metadata.
50%