Reef Infra
An infrastructure framework for reinforcement learning engineering practice that decouples algorithmic logic from service infrastructure through a 'weight recipes' mechanism. It provides componentized abstractions including processors, objectives, schedulers, and recipes, and incorporates service-layer fault-tolerance capabilities such as restart replay, checkpointing, and batch acknowledgment holding.
Timeline (last 90 days)
Reef Infra 的 TTT-Discover 示例给出了一种显式配对的做法：把 post-step 的 PUCT 搜索档案与已提交（committed）的模型版本绑定在一起。
TTT-Discover 的具体机制是：搜索步骤完成后，先写入一个 pending archive；只有当与之对应的模型提交也建立起来后，才把这个档案标记为 committed。
Separating the request entry point from the release gate is a separation-of-concerns design that allows teams to independently adjust request strategies and release strategies.
Requiring the explicit declaration of a requests parameter rather than accepting the broad **kwargs moves interface compliance checking to the startup phase, strictly separating configuration errors from runtime errors.
Manual requests can initiate changes, but the original evaluation and release strategy still determines whether the proposed edits ultimately take effect.
Reef Infra requires custom methods to explicitly declare a requests keyword argument; using the generic **kwargs will be directly rejected by the recipe at startup.
Hybrid mode attaches a portion of samples from the next automated batch, allowing the request to arrive with recent execution context.
Reef Infra provides two modes for custom harness methods—manual and hybrid—making active requests a first-class input for triggering changes.
In manual mode, the proposer receives a queued request and an empty sample list, with changes entirely driven by intent.
A mature system architecture should provide clear channels for both data-driven corrections and intent-driven changes.
20 more timeline events
All Facts (20)
If deployment decisions are to depend on benchmarking or regression checks, a candidate evaluation plugin that implements both metric measurement and selection decisions must be provided for the recipe.
60%UnverifiedReef Infra 的 TTT-Discover 示例给出了一种显式配对的做法：把 post-step 的 PUCT 搜索档案与已提交（committed）的模型版本绑定在一起。
50%UnverifiedTTT-Discover 的具体机制是：搜索步骤完成后，先写入一个 pending archive；只有当与之对应的模型提交也建立起来后，才把这个档案标记为 committed。
50%UnverifiedRequiring the explicit declaration of a requests parameter rather than accepting the broad **kwargs moves interface compliance checking to the startup phase, strictly separating configuration errors from runtime errors.
50%UnverifiedManual requests can initiate changes, but the original evaluation and release strategy still determines whether the proposed edits ultimately take effect.
50%UnverifiedReef Infra requires custom methods to explicitly declare a requests keyword argument; using the generic **kwargs will be directly rejected by the recipe at startup.
50%UnverifiedHybrid mode attaches a portion of samples from the next automated batch, allowing the request to arrive with recent execution context.
50%UnverifiedReef Infra provides two modes for custom harness methods—manual and hybrid—making active requests a first-class input for triggering changes.
50%UnverifiedIn manual mode, the proposer receives a queued request and an empty sample list, with changes entirely driven by intent.
50%UnverifiedA mature system architecture should provide clear channels for both data-driven corrections and intent-driven changes.
50%UnverifiedSeparating the request entry point from the release gate is a separation-of-concerns design that allows teams to independently adjust request strategies and release strategies.
50%UnverifiedReef Infra's experiment loop consists of three components: a Harness proposer, an Evaluator, and a control run, with the control run executed in parallel under a fixed model and the same set of tasks.
50%UnverifiedReef Infra provides an experiment loop for testing context strategies written by users themselves.
50%UnverifiedReef Infra provides comparison mechanisms and a publish-or-revert loop, but the compression strategy, task fixtures, and definition of acceptable coverage remain under the user's own control.
50%UnverifiedReef Infra's default selection strategy tallies the win-loss ratio across tasks and adopts a candidate strategy as long as it wins more tasks than it loses.
50%UnverifiedThe Reef Infra repository includes a more conservative example strategy that only accepts a candidate if no task shows any regression.
50%UnverifiedWhen using a custom LangChain harness, an adapter is needed to launch an episode and read the trace in order to integrate with Reef Infra.
50%UnverifiedThe Objective is responsible for preparing advantage values, metrics, and proposed algorithm state, and declares the loss family it belongs to.
50%UnverifiedThe Processor is responsible for assembling recorded generation results and feedback into typed batches.
50%UnverifiedReef Infra's weight recipes mechanism breaks down a single training update into four collaborating components: Processor, Objective, Schedule, and Recipe.
50%