Sona

Sona's teacher model is not deployed directly for online serving; instead, its scoring knowledge is distilled into the online ranking module to alleviate the sparse ranking supervision signal caused by scarce user likes and dislikes.

The Yandex research team introduced a generative music recommendation system called Sona that learns directly from raw interaction logs without any hand-engineered features.

Sona has already been A/B tested in Yandex's production environment rather than remaining at the paper stage.

The core idea behind Sona is to have the decoder directly generate Semantic IDs assigned to each track, rather than performing traditional similarity-based retrieval over a massive music library.

Sona's offline tokenizer starts from a frozen Qwen2.5-Omni multimodal model and takes two types of input: the mel spectrogram of the first 90 seconds of a track and the track's metadata.

Sona stacks a 4-layer Transformer on top of Qwen's hidden states to further refine the representations.

Sona's tokenizer is trained with an InfoNCE contrastive loss using two types of positive sample pairing signals: tracks co-occurring within the same recommendation response from different artists, and tracks by the same artist with high co-listening NPMI within a three-month window.

Sona converts continuous embeddings into tuples of three codes via residual K-means, with 32,000 cluster centers at each level.

Sona introduces a second loss term to constrain each embedding vector to remain close to the track's own content features, preventing the embeddings from over-indexing on behavioral signals at the expense of acoustic semantics.

Sona evaluates tokenizer quality by retraining retrieval models for different tokenizer variants and measuring recall on a held-out request set, judging tokenizer quality by downstream recommendation performance.

Sona uses the audio-only CLMR as a comparison baseline to examine the gains of audio-plus-metadata-plus-behavioral contrastive learning over purely audio-based representations.

Sona trains a separate and larger Teacher Ranker using a full year of log data in a two-step process: next-track prediction first, followed by multi-head ranking.

The Yandex Music research team replaced more than 15 candidate generators, pre-ranking models, and ranking models in their production environment with a single Transformer model called Sona.

Sona is capable of reading up to 8,192 user behavior events.

Sona's History Compression mechanism reduces inference costs by roughly half.

The two history segments exchange information through cross-attention and a single self-attention layer covering the full history, while the subsequent 7-layer Transformer stack runs only on the most recent 2,048 events.

Sona's decoder and ranking module read the same encoder output, so the encoder needs to be run only once per request.

Sona generates candidates via beam search, producing outputs in the form of Semantic IDs that are immediately scored.

Sona's A/B test was conducted in the Yandex Music smart speaker scenario, ran for 7 days, and covered 15% of users per experiment group.

In the A/B test, Sona achieved a +4.53% increase in active users and a +6.30% increase in total listening time, with both metrics reaching a significance level of p<0.01.