Verbalized Sampling Lands at ICML: Does Prompt Engineering Belong at Top Conferences?

A Controversial ICML Paper

A paper titled Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity was recently accepted at this year's ICML. What would normally be routine academic good news sparked a fierce debate in Reddit's machine learning community about what qualifies as serious ML research.

The controversy isn't about whether the paper's conclusions are correct — it's about the perceived technical depth of the methodology. The core idea is surprisingly straightforward: by changing how prompts are phrased, you can get large language models to produce significantly more diverse outputs. In other words, it's essentially a pure prompt engineering technique.

One Reddit user put it bluntly: "Even if it works, I'm not sure this kind of prompt engineering belongs at a top ML conference. Some people call this work 'modern machine learning,' but I think it belongs in a lower-tier venue. Am I just being rigid?"

What Are "Mode Collapse" and "Verbalized Sampling"?

To understand the debate, you first need to grasp what problem the paper is trying to solve.

Mode Collapse: The LLM "Boredom" Problem

Mode collapse is a classic challenge in generative modeling. When you ask an LLM the same open-ended question repeatedly, it tends to produce highly similar answers, lacking variety. Ask a model to "write a joke" or "suggest a name," and it may cycle through just a handful of templated responses.

This stems from the inherent dynamics of training — models aligned via RLHF (Reinforcement Learning from Human Feedback) tend to converge toward the "safest" and "highest-probability" outputs, sacrificing creativity and diversity. This is a clear limitation for use cases like brainstorming, creative writing, and data augmentation.

Verbalized Sampling: Getting the Model to "Verbalize" Its Probabilities

The paper's proposed solution is unexpectedly simple. Traditional approaches control diversity by tuning sampling parameters like temperature and Top-p. Verbalized sampling takes a different route: it asks the model — directly in the prompt — to generate a set of candidate responses and explicitly "verbalize" the probability distribution associated with each one.

By embedding the sampling process into the prompt in a verbalized form, the model is guided to explore more of the probability distribution rather than always converging to the peak. This approach requires no changes to model weights, no retraining — just a cleverly designed prompt that significantly boosts output diversity.

The Real Debate: Is Prompt Engineering "Real Research"?

The most valuable part of this discussion is what it reveals about a deeper identity crisis in AI research today.

The Skeptics: Lacking Theoretical Depth

Critics make a clear case. First, this kind of work is hard to support with rigorous theoretical analysis — it's difficult to mathematically prove why rephrasing a prompt works, and its effectiveness relies more on empirical observation. Second, from a technical complexity standpoint, there's no new model architecture, no new training algorithm — it's fundamentally a usage trick.

In the eyes of traditional ML researchers, top conferences should publish technically innovative work with solid theoretical foundations that is reproducible and generalizable. A prompting trick, however effective, looks more like a summary of engineering experience than a scientific discovery.

The Supporters: This Is "Modern Machine Learning"

The other camp argues this represents exactly the kind of paradigm shift happening in ML research. When large language models become inscrutable black boxes, how to interact with them effectively is itself a serious scientific question. Prompt engineering is no longer a minor trick — it's a critical tool for understanding and steering the behavior of massive models.

More importantly, if a simple method can reliably address mode collapse — a long-standing pain point — and holds up under extensive experimental validation, its practical value alone justifies academic recognition. When evaluating research, novelty and impact are both essential; neither can be dismissed.

A Deeper Reflection: Evaluation Standards Are Being Rewritten

This debate has no clear right answer — it reflects the profound transformation AI is undergoing.

Before deep learning took off, machine learning was a highly mathematical discipline where theoretical proofs were the gold standard. The era of large models changed that: many important findings now follow an "experiment-first" approach — observe the phenomenon, then try to explain it — with some phenomena (like emergent abilities and in-context learning) still lacking complete theoretical explanations today.

From this perspective, the questioning Reddit user isn't being "too rigid" — they're defending a traditional standard of academic rigor, and that stance has genuine value. At the same time, dismissing prompt engineering wholesale as "not technical enough" risks its own form of arrogance. After all, a method that gets cited widely and genuinely improves model performance shouldn't have its contribution buried under the simplicity of its approach.

The healthiest attitude may be this: welcome pragmatic, empirical innovations into top venues to advance real-world applications, while also maintaining the pursuit of theoretical depth and encouraging researchers to explain why things work. Simple does not mean shallow; complex does not mean profound. ICML's decision to accept this paper may be the academic community's way of answering that question in action — in the age of large models, practical methods that solve real problems deserve to be seen.

Closing Thoughts

The controversy over Verbalized Sampling is fundamentally a debate about values in AI research. When the pace of technological development far outstrips our ability to explain it theoretically, evaluation frameworks must inevitably adapt. For practitioners, rather than getting caught up in "does this count as real research," it may be more useful to ask a simpler question: can this method actually help your application? If the answer is yes, its value has already been proven.