When Should LLM Agents Ask Humans? The CVD Framework for Optimizing Human Decision Allocation

Large language models are increasingly automating complex tasks, but human decision-making capacity has not scaled alongside model capabilities. In many applications, humans remain indispensable—they provide private context, authority, or judgment that agents cannot access. However, human attention is a scarce resource, and agents cannot consult humans about every detail. How to allocate limited human decision-making capacity to the most critical problems has become an urgent challenge. A recent paper proposes the "human decision capacity allocation" problem and offers a solution called "Consultation Value Distillation" (CVD).

Problem Definition: Human Decision Capacity Allocation

The paper formalizes the problem as follows: given multiple consultation opportunities and a limited human decision budget, at which decision points should LLM agents consult humans? The core challenge lies in estimating "consultation value," i.e., the degree to which human input contributes to the final task outcome. This value is executor-dependent—the same human input may yield vastly different benefits for different downstream agents (executors). For example, an agent skilled at code generation may benefit greatly from human advice on algorithm selection, while an agent skilled at copywriting may value human guidance on tone and style more. Therefore, consultation value cannot be assessed in isolation from the specific executor.

Executor in this paper refers to the downstream LLM agent that actually performs the task, calls tools, or generates the final output. Consultation value is executor-dependent because human suggestions must be understood, adopted, and translated into action by the executor; different executors have different capability distributions, knowledge blind spots, and reasoning styles, so the same suggestion may be absorbed to different degrees. For example, a human hint "use dynamic programming to optimize time complexity" can be directly implemented by an executor with algorithmic knowledge, but may be ignored or misused by an executor lacking relevant training. Therefore, CVD conditions the value function on the executor when estimating consultation value, rather than assuming a universal "value of human input." This requires offline rollouts to be conducted for the target executor, and also means that the value predictor needs to be retrained or at least recalibrated after swapping executors.

The CVD Framework: Offline Learning of Consultation Value

To address the challenge of consultation value estimation, researchers propose Consultation Value Distillation (CVD). Its core idea is: use Monte Carlo executor rollouts to estimate consultation value offline, then distill these values into a lightweight predictor for budget allocation at deployment time. This eliminates the need for rollout-based value estimation during deployment, greatly reducing computational overhead.

CVD's key innovations are joint value learning and continuation-aware value learning. Joint value learning considers the mutual influence between multiple consultation points, while continuation-aware value learning considers the sequential nature of consultation decisions—one consultation may change the value of subsequent decisions. Through these two mechanisms, CVD can more accurately allocate the limited human consultation budget.

Monte Carlo executor rollouts are the fundamental means by which CVD estimates consultation value offline. The approach is: during training, for each candidate consultation point, simulate two branches—"consult human" and "do not consult human"—and let the target executor sample subsequent execution trajectories (i.e., rollouts) multiple times from that point, using the empirical distribution to approximate task success probability or reward. Consultation value can be defined as the difference in expected reward between the two branches. Since LLM generation is stochastic, single rollouts are noisy; Monte Carlo methods reduce variance through multiple samples. The cost is that the number of rollouts grows with the number of consultation points, resulting in high computational overhead—which is precisely why CVD needs to distill values into a lightweight predictor to avoid real-time rollouts at deployment. Reduced rollout supervision means values can be estimated with fewer samples but introduces bias; the paper conducts a robustness analysis on this.

Experimental Setup and Main Results

The paper conducts experiments in five batch settings and three sequential settings, using two frontier executors. Results show that CVD improves allocation through joint and continuation-aware value learning. As a detail, independent consultation value (i.e., value estimation that does not consider mutual influence between consultation points) remains competitive in multiple settings, suggesting that simple methods can also work in some scenarios.

Compared with direct prompting and the Value of Information method applicable to specific scenarios, CVD performs better across a wide range of scenarios. Furthermore, CVD remains robust when rollout supervision is substantially reduced and can achieve near-lossless approximate allocation at low search cost. This means CVD is not only effective but also practical for real-world deployment.

Value of Information (VoI) is a classic concept in decision theory that measures the expected gain from acquiring additional information before making a decision. In this paper's context, VoI methods typically compute the value of "acquiring human input" versus "not acquiring" independently for each consultation point, and allocate budget accordingly. The key difference from CVD is that classic VoI often assumes information values are additive or ignores mutual influence between consultation points, whereas CVD explicitly models the sequential dependencies and interaction effects of consultation decisions through joint value learning and continuation-aware value learning. Additionally, VoI requires a known or assumed probability distribution of human input, while CVD directly estimates executor-dependent empirical value through rollouts, making it better suited to the open-ended task scenarios of LLM agents.

Implications and Outlook

The CVD framework provides a data-driven approach to human consultation strategies for LLM agents. It emphasizes the executor-dependence of consultation value and demonstrates the potential of combining offline learning with lightweight predictors. Future directions worth exploring include further reducing rollout costs, adapting to dynamic environments, and handling scenarios with multiple human consultations.

For developers building practical LLM agents, CVD offers an approach: rather than letting agents consult blindly or act fully autonomously, quantify the value of human input through offline learning to maximize task success rate under a limited human budget. As LLM capabilities continue to improve, the bottleneck effect of human decision-making will become more pronounced, and allocation mechanisms like CVD may become standard components of agent systems.

Batch settings and sequential settings are two ways of organizing tasks for evaluating CVD in the paper. Batch settings refer to the agent facing a set of mutually independent decision points and needing to choose which points to consult under a total consultation budget constraint; consultation order does not affect the value of each point, and the focus is on how joint value learning optimizes combinatorial selection. Sequential settings refer to decision points unfolding in task execution order, where one consultation changes subsequent states and the value of subsequent consultations, and the focus is on how continuation-aware value learning models sequential decisions. The two types of settings cover the spectrum from static budget allocation to dynamic sequential decision-making, and CVD performs well on both, indicating that its framework has some generality across different task structures.

One-line Summary

CVD estimates consultation value through offline rollouts and distills it into a lightweight predictor to optimize LLM agent consultation allocation under a limited human budget.

Paragraph Summary

While LLM agents can automate complex tasks, humans remain indispensable for providing private context, authority, and judgment—yet human attention is scarce. The paper formalizes the "human decision capacity allocation" problem: under a limited consultation budget, at which decision points should agents consult humans? The core challenge is estimating executor-dependent consultation value. To this end, it proposes Consultation Value Distillation (CVD), which uses Monte Carlo executor rollouts to estimate consultation value offline and distills it into a lightweight predictor, eliminating rollouts at deployment. CVD introduces joint value learning and continuation-aware value learning to model mutual influence between consultation points and sequential dependencies, respectively. Experiments across five batch and three sequential settings show that CVD improves allocation, outperforms direct prompting and Value of Information methods, remains robust under reduced rollout supervision, and achieves near-lossless approximate allocation at low search cost.