AQuA
A research paper on model-search agents that proposes a constrained model development loop design. It introduces sealed evaluators and restricted configuration change zones to ensure experimental auditability, preventing agents from artificially inflating reported metrics by tampering with the evaluation process.
Timeline (last 90 days)
AQuA uses finance as its experimental domain but explicitly states that this does not constitute trading advice.
In AQuA's U.S. stock market experiment, the model was trained on data from 2010–2019, with 2020 set as an embargo zone and 2021–2025 used as the final test window.
AQuA's early stopping and checkpoint selection use only an internal validation slice within the training window.
The AQuA paper does not disclose the exact feature set, normalization methods, or label construction details, making it difficult for external researchers to fully reproduce the results.
The constrained configuration interface does not guarantee that each candidate model receives the same wall-clock computation time; heavier architectures may consume more computational resources, which is a potential source of unfairness.
A narrower search space sacrifices the agent's creativity in exchange for the attributability of each improvement, while a wider search space may discover better solutions but makes it difficult to explain what worked.
In AQuA's model development loop, the agent can only propose bounded configuration changes within four restricted areas (architecture, loss function, sampler, and optimizer) in each iteration.
AQuA's generative model is trained under a sealed evaluator, with data splits, feature definitions, label definitions, and evaluation logic all outside the agent's adjustable scope.
Drawing a clear boundary between the proposal language and the evaluator cuts off the attribution contamination path through which an agent could inflate scores by relaxing evaluation criteria or adjusting data splits.
Appendix B of the AQuA paper documents a case in which the explanation for an early feature passed review, but future information was used when calculating the total daily trading volume as a denominator, constituting data leakage.
4 more timeline events
All Facts (14)
AQuA uses finance as its experimental domain but explicitly states that this does not constitute trading advice.
50%UnverifiedIn AQuA's U.S. stock market experiment, the model was trained on data from 2010–2019, with 2020 set as an embargo zone and 2021–2025 used as the final test window.
50%UnverifiedAQuA's early stopping and checkpoint selection use only an internal validation slice within the training window.
50%UnverifiedThe AQuA paper does not disclose the exact feature set, normalization methods, or label construction details, making it difficult for external researchers to fully reproduce the results.
50%UnverifiedThe constrained configuration interface does not guarantee that each candidate model receives the same wall-clock computation time; heavier architectures may consume more computational resources, which is a potential source of unfairness.
50%UnverifiedA narrower search space sacrifices the agent's creativity in exchange for the attributability of each improvement, while a wider search space may discover better solutions but makes it difficult to explain what worked.
50%UnverifiedIn AQuA's model development loop, the agent can only propose bounded configuration changes within four restricted areas (architecture, loss function, sampler, and optimizer) in each iteration.
50%UnverifiedAQuA's generative model is trained under a sealed evaluator, with data splits, feature definitions, label definitions, and evaluation logic all outside the agent's adjustable scope.
50%UnverifiedDrawing a clear boundary between the proposal language and the evaluator cuts off the attribution contamination path through which an agent could inflate scores by relaxing evaluation criteria or adjusting data splits.
50%UnverifiedAppendix B of the AQuA paper documents a case in which the explanation for an early feature passed review, but future information was used when calculating the total daily trading volume as a denominator, constituting data leakage.
50%UnverifiedIn a later design of AQuA, a fixed causal operator registry replaced the original open construction space.
50%UnverifiedUnder the constrained operation language design, the normalizer that used future information became fundamentally inexpressible at the language level.
50%UnverifiedThe AQuA paper explicitly acknowledges that the constrained operation language cannot prevent all possible forms of leakage.
50%UnverifiedAQuA's research reveals that evaluators can approve a seemingly reasonable explanation while completely missing the erroneous intermediate operations that produced it.
50%