Outcome-Aware Likelihood Comparison
A utility estimation approach that does not rely on additional environment interaction. It evaluates skill utility by computing the difference in log-likelihood between the model reproducing a trajectory with and without the skill, weighted according to outcome labels.
Timeline (last 90 days)
SkillScaffold reuses training trajectories whose results have already been verified and uses outcome-aware likelihood comparison to estimate skill utility, without relying at all on candidate-specific environment interaction.
Outcome-aware likelihood comparison evaluates skill utility by computing the log-likelihood of the model reproducing the trajectory under the two conditions of carrying and not carrying the candidate skill, and weighting and comparing the likelihood difference according to the trajectory outcome label.
All Facts (2)
SkillScaffold reuses training trajectories whose results have already been verified and uses outcome-aware likelihood comparison to estimate skill utility, without relying at all on candidate-specific environment interaction.
50%UnverifiedOutcome-aware likelihood comparison evaluates skill utility by computing the log-likelihood of the model reproducing the trajectory under the two conditions of carrying and not carrying the candidate skill, and weighting and comparing the likelihood difference according to the trajectory outcome label.
50%