HASTE: Hierarchical and Attribute Supervision Doubles Generative Recommendation Training Speed

Generative recommendation is becoming a hot direction in recommender system research: encode items into semantic IDs, then autoregressively generate these IDs layer by layer. But most existing methods optimize semantic ID prediction with just a standard causal cross-entropy objective, ignoring an obvious fact—semantic IDs themselves have a hierarchical structure. This both slows down training convergence and leaves data-sparse long-tail items without sufficient supervision. A study called HASTE attempts to solve both problems simultaneously with "hierarchical loss" and "attribute-specific losses."

The Wasted Hierarchical Structure of Semantic IDs

The core idea of generative recommendation is to represent each item as a sequence of semantic IDs, then let the model autoregressively generate this sequence just like generating text. The problem is that the training objective is usually just a generic causal cross-entropy loss—the model has no idea that these IDs have parent-child hierarchical relationships.

To use an analogy: if the prefix of a semantic ID represents coarse-grained categories and subsequent digits represent finer distinctions, then standard cross-entropy won't give partial credit when the model "gets the major category right but errs on the details," nor will it penalize more heavily when the model "gets even the major category wrong." The hierarchical information is flattened into a string of ordinary tokens, and the model can only figure it out from massive amounts of data on its own.

This directly leads to two consequences. First, slow convergence—the paper points out that the standard cross-entropy objective requires a considerable number of training steps to learn effective item representations. Second, weak long-tail performance—data-sparse tail items already lack supervision signals, and under this "one-size-fits-all" loss, it's even harder to learn good representations, naturally leading to worse recommendation performance.

What is a Semantic ID

In generative recommendation, a semantic ID is a representation that maps discrete items to token sequences, typically generated by quantization methods such as RQ-VAE, residual quantization, or hierarchical clustering. The typical approach: first learn a dense vector for each item using content features (text, images, categories), then quantize layer by layer through multi-level codebooks to obtain an ID string like "12-47-8." Each level of the codebook corresponds to a granularity level—the first few digits encode coarse-grained categories, and the later digits encode fine-grained differences. This way, item prediction is transformed into a sequence generation problem, directly reusing the autoregressive architecture and training paradigm of language models. The quality of semantic IDs directly determines the upper bound of generative recommendation, because they are both the input representation and the generation target.

Why Causal Cross-Entropy Loss "Flattens" the Hierarchy

Standard causal cross-entropy treats every token position in the sequence equally: correct prediction gets one point, incorrect gets zero, and gradients only come from whether the current token is right or wrong. It naturally doesn't distinguish which level a token belongs to—getting the first digit of a semantic ID right and everything after wrong, versus getting the first digit wrong, won't reflect the degree of "hierarchical partial correctness" in the loss. This compresses a prediction task that could be decomposed into a tree structure into a string of mutually independent classification problems. The result: the model must learn the prior that "prefixes matter more than suffixes" from massive samples on its own, and with long-tail items where sample sizes are inherently insufficient, learning is even more difficult. Hierarchical loss is precisely the mechanism that explicitly gives this tree-structured prior back to the model by computing supervision separately at each level's prefix.

HASTE's Two Core Losses

HASTE (Hierarchical and Attribute-aware Supervision for generaTive rEcommendation) is a unified training framework that introduces two types of additional supervision beyond the original objective.

Hierarchical Loss explicitly leverages the multi-level structure of semantic IDs, providing supervision signals at every level of the item hierarchy. In other words, the model is not only judged on the final ID sequence but receives feedback at each level's prefix. This makes gradient signals denser and more directional, significantly accelerating training convergence.

Attribute-Specific Losses encourage the model to preserve fine-grained item attributes. It leverages the semantic knowledge already present in the language model itself, injecting attribute information as supervision signals into representation learning, improving the representation quality of tail items and thereby enhancing long-tail recommendation performance.

The combination of these two types of losses constitutes the key differentiator of HASTE from existing generative recommendation approaches.

Why Attribute-Specific Losses Help the Long Tail

The problem with long-tail items is that interaction data is extremely scarce, making it difficult for the model to learn reliable representations from collaborative signals alone. But they are often not sparse in content attributes—fine-grained attributes such as color, fit, material, and use cases are still available. The attribute-specific loss approach is: convert item attribute labels into natural language prompts or category supervision, leverage the semantic associations already embedded in pre-trained language models, and constrain the generative model to preserve this attribute information when predicting semantic IDs. This is equivalent to providing tail items with an additional "content-side" supervision pathway, using the language model's general semantic knowledge to fill the gap left by behavioral data. Compared to relying solely on ID co-occurrence, this attribute supervision is more stable and generalizes better to cold-start items.

Experiments: 2x Convergence Speed and Comprehensive Performance Gains

The research team evaluated on fashion recommendation benchmarks and multiple domains of the Amazon dataset. Results show that HASTE brings recommendation performance improvements across different datasets and different item frequency ranges—not just for head popular items, but the long-tail range benefits as well.

The most striking number is training efficiency: hierarchical loss achieves approximately 2x convergence speed, substantially accelerating training compared to the standard causal cross-entropy objective while maintaining strong recommendation quality. For practical deployment, this means less training compute and shorter iteration cycles without compromising on effectiveness.

It should be noted that these conclusions come from the paper's own experimental setup, and the magnitude of gains in real-world business scenarios will vary depending on data distribution and model scale.

Why This Matters

Generative recommendation is often compared with traditional recommendation paradigms, with the discussion focus mostly on "whether the generative paradigm is stronger." HASTE offers a different angle: the real room for improvement may not lie in the generative paradigm itself, but in the design of supervision signals.

The hierarchical structure of semantic IDs is prior knowledge naturally carried by such methods, and explicitly writing it into the loss function is essentially saving the model from taking unnecessary detours. Attribute supervision, meanwhile, targets the long-standing long-tail challenge in recommender systems, using the language model's semantic knowledge to compensate for data sparsity.

For teams building or tuning generative recommendation systems, HASTE's approach offers directly actionable insights: check whether your training objective is wasting structural information in item representations—this is often the most easily overlooked and most easily improved area.

One-line Summary

HASTE optimizes semantic ID prediction with hierarchical loss and attribute-specific losses, achieving ~2x faster convergence and improved long-tail recommendations.

Paragraph Summary

Generative recommendation encodes items as semantic IDs and generates them autoregressively, but existing methods optimize with only standard causal cross-entropy, ignoring the hierarchical structure of semantic IDs, resulting in slow convergence and insufficient long-tail supervision. HASTE introduces two additional supervisions: hierarchical loss provides feedback at each level's prefix of the item hierarchy, making gradient signals denser and achieving ~2x convergence speed; attribute-specific losses inject fine-grained attribute information via the language model's semantic knowledge, improving sparse tail item representations. On fashion recommendation benchmarks and multi-domain Amazon datasets, HASTE improves recommendation performance across different item frequency ranges.