How to Call Multiple Foundation Models on a Limited Budget? New Framework Uses Q-Learning for Dynamic Selection

Under an average query budget constraint, fixed querying or Q-learning-based dynamic combination of multiple foundation models can improve prediction accuracy.
This article introduces a study on combining multiple foundation models under budget constraints. Foundation models vary in accuracy and query costs, and deployment often faces an average per-input cost cap. The researchers propose two methods: querying a fixed set of models for all inputs and fusing outputs—theoretically proving that multiple less accurate models jointly can surpass the best single model and achieve the minimax rate—and using Q-learning to dynamically decide which models to query per input and when to stop, with finite-sample risk and cost upper bounds. Simulations and a multimodal laptop price prediction application show both methods outperform existing approaches. The article notes that treating model calling as an optimizable decision problem helps achieve better cost-effectiveness within a budget.
Why We Should Care About the Query Costs of Foundation Models
Thanks to large-scale pre-training, foundation models can provide out-of-the-box predictive capabilities for downstream tasks. However, different models vary significantly in accuracy on specific tasks, and the cost of each query is not the same—some models are lightweight and cheap, while others require calling massive parameters or complex reasoning at a high cost.
In real-world deployment scenarios, we often face a budget constraint: there is an upper limit on the average query cost allowed per input. How to combine the outputs of multiple foundation models within a budget—improving prediction accuracy without overspending—has become a key problem. A recent study proposes a systematic framework to resolve this tension. (Source: rss, Screenshot: screenshots/source/54398_0.png)

Two Strategies: Fixed Querying and Adaptive Q-Learning
The researchers designed two methods that use training data to learn "which models to query" and "how to combine outputs" with features that can be obtained without querying.
Method 1: Query the Same Models for Every Input
The first method queries a fixed set of models for all inputs and then fuses their outputs. It seems simple, but the paper proves a counterintuitive conclusion: multiple individually less accurate outputs can jointly surpass the best single model among them. The underlying logic is that different models may have complementary error patterns, and ensembling can offset some of the bias.
At the theoretical level, when the regression function depends only on model outputs within the budget, this method achieves the minimax rate. Conversely, if the regression function depends on models outside the budget, any budget-constrained method is forced into a slower rate, and this method also achieves that slower rate. This means Method 1 is theoretically near-optimal.
Method 2: Dynamically Select Models per Input
The second method allows different sets of models to be queried for each input and uses Q-learning to decide "which model to query next" and "when to stop querying." This is similar to a reinforcement learning agent that, at each step, decides based on currently available information whether it is worth continuing to query.
The paper clarifies the conditions under which varying the queried models per input can strictly outperform querying the same fixed set of models, and provides upper bounds on prediction risk and average cost under finite samples. This provides theoretical guarantees for the effectiveness of the dynamic strategy.
Minimax rate is a core concept in statistical learning theory for measuring the optimality of estimation methods. It characterizes the fastest rate at which the expected error in the worst case converges as the sample size increases, over all possible true data distributions. If a method achieves the minimax rate, it means no other method can significantly outperform it in the worst case, so the method is rate-optimal in a theoretical sense. Method 1 in the paper achieves this rate when the regression function depends only on model outputs within the budget, indicating that its error convergence rate cannot be generally improved; when the regression function depends on models outside the budget, any budget-constrained method can only achieve a slower rate, and Method 1 also matches this lower bound, further confirming its theoretical tightness.
Q-learning is a model-free reinforcement learning algorithm that learns an optimal policy by iteratively updating the state-action value function Q(s, a). In this framework, the "state" encodes the models already queried and their outputs, the "action" corresponds to which model to query next or whether to stop querying, and the "reward" combines the improvement in prediction accuracy with the query cost. Q-learning enables the agent, in an unknown environment dynamic, to balance exploration (trying new queries) and exploitation (making decisions based on existing knowledge) through trial and error, thereby learning an input-adaptive query strategy. Compared with fixed querying, Q-learning does not require a pre-specified query order and can flexibly handle the heterogeneity of different inputs, which explains why it can be strictly superior to static methods in theory.
Experimental Validation: Wins in Both Simulations and Real Applications
The researchers compared the two methods with existing methods through simulation experiments and a multimodal laptop price prediction application. The results show that both methods outperform existing approaches. Laptop price prediction involves multimodal features such as images and text, and model query costs vary, which precisely reflects the practical value of model selection under budget constraints.

Implications for AI Application Development
For AI application developers, this research conveys a pragmatic signal: there is no need to fetishize a single strongest model, nor to call the most expensive model for every input. By learning a query strategy from training data, multiple foundation models can be combined within a budget to achieve better cost-effectiveness. In particular, dynamic querying driven by Q-learning offers a practical approach for allocating computational resources on demand.
However, the paper's theoretical analysis relies on certain assumptions, and practical deployment still requires validation for specific tasks. Nevertheless, the framework itself points in a clear direction: treat model calling as an optimizable decision problem rather than a fixed pipeline.
One-line Summary
Under an average query budget constraint, fixed querying or Q-learning-based dynamic combination of multiple foundation models can improve prediction accuracy.
Paragraph Summary
This article introduces a study on combining multiple foundation models under budget constraints. Foundation models vary in accuracy on specific tasks and have different query costs, and real-world deployment often faces an upper limit on average cost per input. The researchers propose two methods: first, querying a fixed set of models for all inputs and fusing their outputs, with theoretical proof that multiple less accurate models jointly can surpass the best single model and achieve the minimax rate; second, using Q-learning to dynamically decide which models to query per input and when to stop, with finite-sample risk and cost upper bounds. Simulation experiments and a multimodal laptop price prediction application show that both methods outperform existing approaches. The article points out that treating model calling as an optimizable decision problem helps achieve better cost-effectiveness within a budget.
Related articles

Gemini 4 Hydrogen Hands-On: Google's Fastest and Cheapest AI Model
Hands-on with Gemini 4 Hydrogen: the fastest, cheapest model in the Gemini 4 family. We break down the Carbon/Argon/Hydrogen split, API pricing traps, free-tier limits, and the cost trade-offs of four thinking levels—with a real bug-debugging demo.

Gemini 4 Hydrogen vs Sonnet Comparison Test: Uses Only 1% of Weekly Quota, Performance on Par
Hands-on test: Gemini 4 Hydrogen vs Sonnet on the same CRUD task—Hydrogen used only 1% weekly quota with comparable performance and better UI details, while Anthropic's model used 7%. Developer says daily tasks don't require expensive models.

Gemini 4 Neon Released: Beats Opus 5.5 on Multiple Benchmarks, Google Returns to the Top Tier
Google DeepMind releases Gemini 4 Neon: 61 on the Artificial Analysis intelligence index (3rd) and 1st on the coding index, beating Opus 5.5 on multiple benchmarks. $3/$15 per million tokens; available in the Gemini API and AI Studio.