AMM: Budget-Aware Optimizer Routing via Online Probes

The Optimizer Selection Dilemma in Black-Box Optimization

There is no universally optimal solution for continuous black-box optimization problems—quasi-Newton methods, trust-region methods, coordinate descent, multi-start, and evolutionary strategies each excel on different terrains. Traditional approaches either rely on heuristic rules based on dimensionality and budget, or route based on past improvements or features fitted from a single benchmark; few methods verify during runtime whether an optimizer's implicit assumptions actually hold.

AMM (Adaptive Multi Mode) proposes a different approach: given a known budget, it uses online probes to assess the curvature, plateaus, separability, kinks, basins, and fractal structure of the current terrain in real time, then dynamically allocates budget to the best-matching optimization engine.

Probe-Driven Routing Mechanism

Each of AMM's probes comes with a default value, a dimension-budget validity range, and a probe cost that counts toward the total budget. This means probing is not free—routing decisions must trade off information gain against budget consumption.

Routing targets include five engine classes: quasi-Newton, trust-region, coordinate, multi-start, and evolutionary strategies. Terrain features identified by the probes trigger different engine switches. For example, when separability is detected, coordinate methods may be preferable; when a multi-basin structure is detected, multi-start or evolutionary strategies are more suitable.

The core insight of this design is that an optimizer's performance depends on how well its implicit assumptions match the current terrain, and terrain features can be measured online rather than relying on fits from offline benchmarks.

Online probes are miniature tests actively executed during optimization, designed to obtain structural information about the terrain at the cost of a small number of function evaluations. For example, small-step perturbations along coordinate axes estimate local curvature or separability, or evaluations at multiple random points detect plateaus or basin structures. Unlike offline benchmark fitting, online probes act directly on the current problem instance, so their conclusions are closer to the actual terrain. However, probing itself consumes budget, so information gain must be weighed against computational cost. AMM explicitly accounts for probe cost within the total budget, making routing decisions part of a resource allocation problem.

Pre-Registered Experiments and Held-Out Set Validation

AMM conducted a pre-registered development loop on the bbob benchmark, with 32 of 36 predictions holding in 16 subsequent confirmations. It then ran a one-shot test on held-out bbob instances 101-110 (D=2-40, budget 10D to 1000D), comparing against up to seven competitors.

Across 600 function-dimension-budget cells, AMM held the highest objective share in 382 cells (236 outright, 146 tied). By a descriptive posterior rule, it was credibly behind in only 35 cells, concentrated mainly in the low-budget regimes of 10D and 30D.

At each budget tier, AMM also led control groups built from its own engines—including the single best engine, the best engine selected by dimension and budget, and greedy scheduling—with narrower margins at 10D and 30D; it also maintained a lead over nevergrad's NgIohTuned.

Transformation Robustness and Failure Boundaries

The pre-registered runs showed that AMM's lead over IPOP-CMA-ES holds on affine-transformed and box-constrained bbob, as well as on the merged results for rotated f2-f4.

But there is one clear failure scenario: after rotating the separable f3 and f4, most of AMM's lead there disappears. This suggests that the probe's judgment of separability may become misleading after rotation, weakening the advantage of coordinate-class engines. This boundary condition is informative for practical applications—whether a problem is rotation-invariant affects the effectiveness of routing strategies.

Rotation invariance refers to the property that an optimization algorithm or problem feature remains unchanged after the coordinate system is rotated. For separable functions, if the coordinate axes align with the function's principal axes, algorithms such as coordinate descent can work efficiently; but after random rotation, the function is no longer separable in the original coordinate system, and the resolving power of coordinate-class methods declines. AMM's probes judge separability based on probing actions in the original coordinate system, which may misjudge after rotation, leading to routing to an unsuitable engine. This boundary condition reveals the sensitivity of probe-based routing strategies to problem transformations; practical applications must consider whether the problem is rotation-invariant.

Divergent Performance on MA-BBOB

On held-out MA-BBOB, AMM's final share at the 2000*D budget matches that of IPOP-CMA-ES, but its anytime area (i.e., cumulative performance throughout the optimization process) is lower. This indicates that AMM is not inferior in final solution quality, but does not have an advantage in early progress during optimization.

For scenarios with tight budgets or where a usable solution is needed early, this difference is worth noting; for scenarios concerned only with final accuracy, the tied result is sufficient.

Anytime area is a metric measuring an optimization algorithm's cumulative performance throughout its entire run, typically computed as the area under the curve (or normalized area) of objective function value versus number of evaluations. Unlike focusing solely on final solution quality, anytime area reflects how quickly the algorithm progresses across different budget stages. For scenarios requiring a usable solution early (e.g., online decision-making, real-time optimization), anytime area is more practically meaningful. AMM's final share on MA-BBOB matches IPOP-CMA-ES but its anytime area is lower, indicating slower early convergence, possibly due to budget consumption during the probing phase or delays from routing switches.

Openness and Practical Significance

AMM has been released as an open-source Python package (pip install amm-optimizer, code at github.com/BjBodner/amm-optimizer). Its methodological value lies in turning optimizer selection from static rules into online hypothesis testing: the design that counts probe cost toward the budget forces the routing strategy to make an explicit trade-off between information and compute.

From an engineering perspective, AMM's probing framework can be extended—adding new probes or adjusting validity ranges may improve routing quality on specific problem families. The failure on rotationally separable functions also reminds users that the effectiveness of any router is bound to the coverage of its probing assumptions.

One-line Summary

AMM dynamically routes optimization engines by probing terrain features online, performing excellently on bbob but with a failure boundary on rotationally separable functions.

Paragraph Summary

AMM (Adaptive Multi Mode) addresses the optimizer selection challenge in continuous black-box optimization by using online probes to assess terrain curvature, plateaus, separability, and other features in real time, and dynamically allocating budget to five engine classes: quasi-Newton, trust-region, coordinate, multi-start, and evolutionary strategies. Probe cost counts toward the total budget, forcing routing to trade off information against compute. Across 600 cells in the held-out bbob set, AMM achieved the highest objective share in 382 cells, with credible deficits in only 35. However, its lead over IPOP-CMA-ES largely disappears after rotating the separable f3 and f4, and its anytime area on MA-BBOB is lower. AMM is open-source, and its methodology turns optimizer selection into online hypothesis testing.