World Specification
A machine-readable description of the environment in which a task takes place, including the initial state, legal state space, invariants, milestones, and termination conditions. When the specification becomes the single source of truth, the environment can be rendered from it and progress becomes a computable function.
Timeline (last 90 days)
The core bottleneck of long-horizon Agent evaluation is the cost of writing the specification (world specification) required for dense scoring, not the running cost of dense scoring itself.
Tev's method is not post-hoc trajectory scoring, but rather writing a hidden real-world specification first, and then rendering the environment from that specification.
A world specification is a machine-readable description of the environment in which the task takes place, including the initial state, legal state space, invariants, milestones, and termination conditions.
If the specification is not written specifically enough, dense rewards will be confidently wrong and cause the Agent to optimize directly into the gap between the specification and reality.
Writing specifications is real work, shifting the cost from scoring on every run to one-time authoring, because specifications are reusable, reviewable, and versionable.
One of the principles to be learned from Tev is: write the world specification (states, invariants, milestones, termination conditions) first, then write the task prompt.
All Facts (6)
The core bottleneck of long-horizon Agent evaluation is the cost of writing the specification (world specification) required for dense scoring, not the running cost of dense scoring itself.
50%UnverifiedTev's method is not post-hoc trajectory scoring, but rather writing a hidden real-world specification first, and then rendering the environment from that specification.
50%UnverifiedA world specification is a machine-readable description of the environment in which the task takes place, including the initial state, legal state space, invariants, milestones, and termination conditions.
50%UnverifiedIf the specification is not written specifically enough, dense rewards will be confidently wrong and cause the Agent to optimize directly into the gap between the specification and reality.
50%UnverifiedWriting specifications is real work, shifting the cost from scoring on every run to one-time authoring, because specifications are reusable, reviewable, and versionable.
50%UnverifiedOne of the principles to be learned from Tev is: write the world specification (states, invariants, milestones, termination conditions) first, then write the task prompt.
50%