FrontierPhysics
An end-to-end benchmark for evaluating physics research agents, constructed by physicists on the basis of real research. It contains tasks across multiple domains and evaluates an agent's ability to complete the full scientific research loop, combining deterministic verifiers with expert grading rubrics.
Timeline (last 90 days)
FrontierPhysics is an end-to-end benchmark built by 42 physicists based on their own research, comprising 56 tasks covering nine fields across the three major categories of theoretical, computational and experimental physics.
All 56 FrontierPhysics tasks were designed by 42 physicists based on their own ongoing research rather than drawn from textbooks or question banks.
Each FrontierPhysics task requires the agent to complete the full research loop: reviewing the literature, formulating a research plan, conducting research and experiments, and generating structured outputs and a scientific manuscript.
FrontierPhysics evaluation combines deterministic verifiers with expert-written grading rubrics, checking both the correctness of results and whether the research reaches a publishable standard.
The researchers tested 22 models and 26 configurations, and found that no configuration passed more than 46 of its 168 runs.
FrontierPhysics lists the lack of expert intuition as a primary root cause of failure, suggesting that the evaluation and training of research agents need to introduce open-ended situations without standard answers, letting models practise self-verification and forming scientific judgments under uncertainty.
All Facts (6)
FrontierPhysics is an end-to-end benchmark built by 42 physicists based on their own research, comprising 56 tasks covering nine fields across the three major categories of theoretical, computational and experimental physics.
50%UnverifiedAll 56 FrontierPhysics tasks were designed by 42 physicists based on their own ongoing research rather than drawn from textbooks or question banks.
50%UnverifiedEach FrontierPhysics task requires the agent to complete the full research loop: reviewing the literature, formulating a research plan, conducting research and experiments, and generating structured outputs and a scientific manuscript.
50%UnverifiedFrontierPhysics evaluation combines deterministic verifiers with expert-written grading rubrics, checking both the correctness of results and whether the research reaches a publishable standard.
50%UnverifiedThe researchers tested 22 models and 26 configurations, and found that no configuration passed more than 46 of its 168 runs.
50%UnverifiedFrontierPhysics lists the lack of expert intuition as a primary root cause of failure, suggesting that the evaluation and training of research agents need to introduce open-ended situations without standard answers, letting models practise self-verification and forming scientific judgments under uncertainty.
50%