How to Pick the Best LLM Team? A Heterogeneity-Driven Framework for Model Collaboration Selection

Why Multi-Model Collaboration Can't Rely on Individual Strength Alone

As multi-LLM systems become increasingly common, a widespread misconception persists: stack the most capable models together and the team will naturally perform best. A new study published on arXiv argues that this intuition is often wrong.

The researchers point out that a team's performance ceiling depends not only on the ability of individual models, but also on the degree of "error resonance" and predictive divergence among its members. In other words, if several strong models tend to fail on the same types of problems, combining them doesn't enable mutual error correction — they simply fail together. This is precisely why heterogeneous teams frequently outperform homogeneous ones in practice.

The challenge, however, is that while the effectiveness of heterogeneous teams has been widely observed, there has been no computable, interpretable, or optimizable metric for quantifying complementarity. As a result, team composition has long depended on intuition and heuristic rules, without a solid theoretical foundation.

The Framework's Core: Two Signals for Measuring Complementarity

This research proposes a heterogeneity-driven team selection framework built around offline profiling to characterize each model's behavior. Beyond evaluating individual capability, the framework introduces two key complementarity signals:

Error Decorrelation: Reducing Shared Failures

The first signal captures decorrelation in error patterns. It measures which problems different models tend to fail on simultaneously. If two models' errors are highly correlated, they will "fail together" when confronted with similar difficult inputs, limiting the value of pairing them. Conversely, the less correlated their error patterns, the better the team can compensate for each other's blind spots and reduce the overall failure rate.

The concept of error decorrelation has roots in classic Ensemble Learning theory. In methods like Random Forests and Boosting, researchers established early on that independence of errors among weak classifiers is a stronger predictor of ensemble performance than any single classifier's accuracy. Formally, when two models' errors are positively correlated, the ensemble error approaches that of a single model; when errors are negatively correlated or uncorrelated, the ensemble can significantly reduce error. Translating this principle to LLM teams introduces an additional challenge: LLM "errors" cannot be as easily quantified as classification labels — an answer may be semantically close to correct while being subtly wrong in detail. This makes the choice of how to measure error correlation a critical design decision, and it is why the framework relies on offline profiling to build an error profile for each model.

Predictive Behavior Divergence: Capturing Strategic Diversity

The second signal measures divergence in predictive behavior, capturing strategic diversity. Even when two models have similar accuracy rates, they may approach problems in fundamentally different ways, following distinct reasoning paths. This strategy-level diversity is another source of value in heterogeneous teams, helping the team maintain robustness across a broader distribution of tasks.

Formalizing Team Selection as an Optimizable Combinatorial Problem

With individual capability assessments and two complementarity signals in hand, the research formalizes team selection as a standardized quality–complementarity combinatorial objective. This transforms team composition from an intuition-driven decision into an optimization problem with a clear mathematical objective.

Since selecting the optimal small team from a candidate pool is a combinatorially explosive problem, exhaustive search is impractical. The framework employs an efficient greedy search strategy to quickly assemble small teams from a pool of candidate models — a practical balance between computational cost and selection quality that also makes the framework easier to deploy in real-world settings.

The core difficulty of Combinatorial Optimization problems lies in the search space growing exponentially with the number of candidates. Selecting K models from N candidates yields C(N,K) possible combinations — already exceeding 15,000 when N=20 and K=5, making exhaustive search extremely costly. Greedy Search is a standard approximation strategy for such problems: at each step, the model from the remaining candidates that provides the greatest marginal gain to the current objective is added to the team, until the target size is reached. This reduces time complexity from exponential to polynomial, at the cost of not guaranteeing global optimality and potentially settling for local optima. In the multi-LLM team selection context, greedy search is particularly practical because complementarity objective functions typically exhibit a degree of submodularity — the property of diminishing marginal returns — and greedy approximation of submodular functions carries a theoretical optimality guarantee of (1−1/e) ≈ 63%.

Experimental Results: Consistently Outperforming Quality-Only Baselines

The framework was validated across multiple benchmarks. Under controlled candidate pool and team size conditions, it consistently outperformed quality-only baseline methods. This directly supports the core thesis: maximizing individual model capability alone is not the optimal strategy — complementarity is equally critical in determining the team's performance ceiling.

More broadly, this work establishes reusable selection principles for multi-LLM systems, transforming what was once an experience-driven process into a methodology grounded in metrics, objectives, and algorithms.

Practical Implications for Multi-Model Systems

As agentic systems and multi-model collaborative architectures become increasingly prevalent, how to combine different models has become a genuine engineering challenge. The value of this research lies in providing a quantifiable thinking framework:

• When evaluating models, look not only at absolute capability, but also at whether a model's error patterns complement those of other team members;
• Pay attention to strategic diversity among models to avoid the collective blind spots that come with homogeneity;
• Treat team composition as an optimizable problem, reducing reliance on intuition.

For teams building multi-LLM pipelines, these principles offer a more nuanced guide than simply "stack the strongest models." That said, the framework has primarily been validated in controlled benchmark settings. In real production environments — where candidate pools are larger and tasks are more complex — the efficiency and generalizability of its greedy search approach still warrant further investigation.

Overall, this research is the first to systematically transform the intuitive question of "which models work well together" into a measurable, optimizable engineering problem, providing a valuable methodological foundation for multi-model collaboration.