ReliabilityRoute
A structured intervention mechanism that guides forecasting agents in making evidence-based routing decisions among retrieval, reasoning, reference to market priors, and historical analogy invocation. Routing is governed by reliability features including historical coverage, market prior availability, and evidence strength.
Timeline (last 90 days)
The research team proposes ReliabilityRoute, a structural intervention mechanism that guides a forecasting agent's behavior selection using a set of reliability features.
ReliabilityRoute's reliability features include historical coverage, market prior availability, source prior sharpness, evidence strength, evidence divergence, and forecast time horizon.
Fixed rules can closely match human-designed taxonomies without hard-coding source name decisions.
Rolling adaptive rules continuously refit thresholds from historical batches with previously revealed answers, achieving the best average Brier score among all deterministic systems in tests spanning 16 subsequent LLM versions.
The researchers consider the gains from rolling adaptive rules to be modest, with traditional historical analogy and search baselines remaining highly competitive.
All Facts (5)
The research team proposes ReliabilityRoute, a structural intervention mechanism that guides a forecasting agent's behavior selection using a set of reliability features.
50%UnverifiedReliabilityRoute's reliability features include historical coverage, market prior availability, source prior sharpness, evidence strength, evidence divergence, and forecast time horizon.
50%UnverifiedFixed rules can closely match human-designed taxonomies without hard-coding source name decisions.
50%UnverifiedRolling adaptive rules continuously refit thresholds from historical batches with previously revealed answers, achieving the best average Brier score among all deterministic systems in tests spanning 16 subsequent LLM versions.
50%UnverifiedThe researchers consider the gains from rolling adaptive rules to be modest, with traditional historical analogy and search baselines remaining highly competitive.
50%