Yandex's Sona: A Generative Music Recommendation System Without Hand-Crafted Features

Music recommendation systems have long depended on extensive hand-crafted feature engineering. Yandex's research team is challenging that convention with Sona — a generative recommender that learns directly from raw interaction logs without any manually designed features. More notably, it's already running in A/B tests in Yandex's production environment, not just sitting in a paper.

Semantic ID: Turning Music Into Generatable Semantic Codes

The core idea behind Sona is to have a decoder directly generate Semantic IDs assigned to each track, rather than performing traditional similarity-based retrieval over a massive music catalog. These Semantic IDs are pre-generated offline by a tokenizer for every song.

The tokenizer starts from a frozen Qwen2.5-Omni multimodal model. It takes two types of input: the mel-spectrogram of the first 90 seconds of a track, and the track's metadata. In other words, the system understands both the acoustic characteristics of the audio and its structured information — not just one or the other.

On top of Qwen's hidden states, the team stacks an additional 4-layer Transformer to further refine the representation. This "frozen large model + lightweight refinement layers" design leverages the strong generalization of pretrained models while avoiding the enormous cost of training from scratch.

A mel-spectrogram is a time-frequency representation of an audio signal processed through a mel filterbank, where the frequency axis is arranged on the mel scale — matching human auditory perception with higher resolution at low frequencies and lower resolution at high frequencies. Compared to raw waveforms, mel-spectrograms dramatically reduce data dimensionality while preserving acoustic structure critical to music understanding, such as timbre, rhythm, and harmony. As a multimodal large model, Qwen2.5-Omni can simultaneously process audio spectrograms and text metadata (such as artist name, album, genre tags, etc.), fusing these two heterogeneous inputs into a unified hidden state representation — which is precisely why Sona can capture both acoustic semantics and structured attributes.

Building Track Relationships with InfoNCE Contrastive Learning

The tokenizer is trained using an InfoNCE contrastive loss, with the key design choice being how positive sample pairs are defined. Sona uses two types of pairing signals:

• Tracks that co-occur in the same recommendation response from different artists: reflecting real co-occurrence relationships in actual recommendation scenarios;
• Tracks from the same artist with high co-listening NPMI (Normalized Pointwise Mutual Information) within a three-month window: capturing users' genuine continuous listening behavior.

A second loss term is also included to constrain each embedding vector to remain close to the track's own content features. This design prevents embeddings from over-fitting to behavioral signals at the expense of the music's acoustic semantics.

Finally, the system uses Residual K-means (also known as Residual Vector Quantization) to convert continuous embeddings into tuples of three codes, with 32,000 cluster centers at each level. This residual quantization approach compresses each track into a compact, hierarchical discrete ID sequence — perfectly suited for a generative decoder to predict level by level.

NPMI (Normalized Pointwise Mutual Information) is a statistical measure of how often two events co-occur. While raw PMI grows unboundedly with frequency, its normalized form ranges from [-1, 1], where 1 means the two always co-occur, 0 means independence, and -1 means they never co-occur. In Sona's context, if users A and B both listen to songs X and Y in the same session, the NPMI between X and Y increases. Compared to simply counting co-occurrences, NPMI filters out popular songs that "passively co-occur" due to high exposure, more precisely identifying music pairs that users actively and continuously listen to — providing more meaningful positive signals for contrastive learning.

Residual K-means is a variant of Product Quantization, also known as Residual Vector Quantization (RVQ). It works through multiple rounds of quantization: the first round maps a continuous vector to the nearest cluster center, the second round quantizes the residual (error) from the first round, and so on. Sona uses 3 layers with 32,000 cluster centers each, meaning every song is ultimately encoded as a tuple of 3 integers. This theoretically distinguishes 32,000³ ≈ 3.3×10¹³ different ID combinations while maintaining a hierarchical semantic structure — a natural fit for autoregressive decoders that generate tokens one at a time.

Evaluation Methodology and Baseline Comparisons

The paper's approach to evaluating tokenizer quality is refreshingly pragmatic: for each tokenizer variant, a retrieval model is retrained and recall is measured on a held-out request set. In other words, tokenizer quality is ultimately measured by its contribution to downstream recommendation performance, not by intermediate metrics alone.

For the baseline comparison, Sona uses CLMR — an audio-only approach — as the reference point. This directly tests the gains from combining audio, metadata, and behavioral contrastive learning over pure audio representations.

The Gryphon Architecture: Unified Generation and Ranking

Sona's overall architecture follows Gryphon — a design paradigm that unifies generation and ranking. The process works as follows: a decoder proposes candidate Semantic IDs via beam search, and a Ranking Module then scores the tracks mapped to those IDs.

This "generate candidates + rank" combination essentially merges the traditional two-stage recall and ranking pipeline into a more unified generative framework.

Teacher-Student Distillation Under Sparse Supervision

A practical challenge is that user likes and dislikes are extremely sparse, resulting in severely limited ranking supervision signals. To get around this bottleneck, the team trained a separate, larger Teacher Ranker using a full year of log data.

The teacher model's training follows two steps: first next-item prediction, then multi-head ranking. Notably, this teacher model is never served online directly. Its purpose is to distill its scoring knowledge into the online ranking module — with the distillation targets including both the decoder's own generated candidates and the exposed samples recorded in the logs.

This approach effectively transfers the ranking capabilities a large model learns from massive historical data into a lightweight, deployable online module, while also mitigating the sparse feedback problem.

Knowledge Distillation, proposed by Hinton et al. in 2015, centers on having a smaller "student" model fit the output distribution of a larger "teacher" model rather than directly fitting hard labels. The teacher's outputs (typically probability distributions or continuous scores) carry richer "dark knowledge" than one-hot labels: for instance, if the teacher assigns a jazz track a score of 0.7 and a blues track 0.6, that relative relationship provides far more learnable signal than a simple "like/dislike." In Sona's case, the Teacher Ranker is thoroughly trained on a full year of logs and develops strong ranking capabilities, then transfers its scoring knowledge to a lightweight online ranking module — satisfying real-time latency constraints while avoiding the underfitting caused by directly relying on sparse user feedback.

What This System Represents

Sona represents a concrete step toward the generative paradigm shift in recommendation systems. Several of its characteristics are worth noting: breaking free from hand-crafted feature dependencies, using a multimodal large model as a foundational representation, transforming recommendation into a sequence generation task via Semantic IDs, and applying Teacher-Student distillation to address sparse feedback in real-world scenarios.

For practitioners, Sona offers a replicable path: how to integrate large model capabilities into a recommendation pipeline while maintaining production deployability. For technical details, refer to the team's paper at arxiv.org/abs/2608.11015.