Converge First, Then Diversify: CTD — A New Paradigm for Multi-Objective Bayesian Optimization

Multi-objective Bayesian optimization (MOBO) has long been a go-to approach for optimizing expensive black-box functions — particularly in scenarios where each evaluation carries a high cost, such as hyperparameter tuning, materials design, and engineering simulation. But a new arXiv study identifies a significant blind spot in mainstream MOBO methods when evaluation budgets are extremely tight, and proposes a decoupled approach called Converge-Then-Diversify (CTD).

The Core Tension in MOBO: Convergence vs. Diversity

In multi-objective optimization, the goal isn't to find a single best solution — it's to approximate the entire Pareto front. A high-quality solution set must simultaneously satisfy two conditions: convergence (solutions should lie as close as possible to the true Pareto front) and diversity (solutions should be well-spread and broadly cover the front).

Virtually all existing MOBO methods try to tackle both at once — pushing the search toward the Pareto front while maintaining a diverse set of non-dominated solutions, with the expectation that these solutions will gradually and collectively approximate the full front.

This logic works well when the search budget is generous. But the paper's authors astutely identify a long-overlooked issue: simultaneously pursuing convergence and diversity is non-trivial and requires careful design. Under an extremely limited budget, the system may simply not generate enough solutions to approximate the entire front at once. In other words, when you only have a handful of evaluation opportunities, spreading resources too thin can mean making no real progress in any direction.

CTD's Approach: Split the Two Goals into Two Phases

CTD's core innovation lies in decoupling. Rather than letting convergence and diversity compete throughout the entire search process, it separates them into two sequential, independent phases.

Phase 1: Focus All Resources on Convergence

The first phase focuses exclusively on convergence, with a clear and singular objective — drive the search rapidly to a single point on the Pareto front. The key word here is "single point": no attempt is made to achieve coverage; the only goal is to reach the front as quickly as possible. This focused strategy is especially efficient under tight budgets, because it concentrates all available evaluations on the most fundamental objective: getting there first.

Phase 2: Spread Out for Diversity

Only once the first phase successfully reaches the front does the second phase shift its focus to diversity, expanding solutions along the front to build a broadly distributed solution set. Because a high-quality "anchor point" already exists at this stage, subsequent diversification can grow steadily from that foundation — rather than fumbling across multiple dimensions from scratch.

Notably, CTD is not a complex new framework. It can directly reuse widely adopted acquisition functions already common in the field. The paper presents two straightforward CTD instantiations, both built on existing mature tools, which significantly lowers the barrier to adoption.

Experimental Results: Dominant Performance Across 446 Comparisons

The paper's experimental section presents highly convincing numbers. Across a total of 446 pairwise comparisons:

• CTD was statistically significantly better than state-of-the-art methods in 72.9% of cases
• CTD performed comparably in 21.1% of cases
• CTD was statistically significantly worse in only 6.1% of cases

This means CTD wins in nearly three out of four comparisons, while losing in fewer than one in fifteen. More importantly, it's worth noting where these gains appear — the paper explicitly highlights that CTD's advantages are most pronounced under extremely tight evaluation budgets or in high-dimensional problems. This directly validates its design intent: it is precisely when resources are constrained and traditional methods struggle to pursue both objectives simultaneously that the decoupling strategy delivers the most value.

Why This Idea Is Worth Paying Attention To

From a methodological standpoint, CTD challenges a default assumption in the MOBO field: that convergence and diversity must proceed in parallel. The "divide and conquer" philosophy is not new to optimization, but systematically introducing it into Bayesian optimization — and rigorously validating its superiority under tight budgets through statistical comparisons — carries real practical significance.

For industry practitioners, many real-world scenarios are characterized by expensive evaluations — a physical experiment, a large-scale simulation, or a costly model training run. In these settings, evaluation budgets are naturally scarce, and CTD's "reach the front first, then expand" strategy aligns better with engineering intuition and is easier to tune.

Of course, as a newly published preprint, CTD still has aspects worth watching: how to define the switching point between the two phases, whether single-point convergence could introduce an initial bias that affects final diversity, and whether it still offers marginal gains over traditional methods in standard settings with ample budgets. These questions call for further research and real-world validation. Nevertheless, CTD offers a clean and effective new lens through which to view multi-objective Bayesian optimization.