WikiSkill
A conceptual agent knowledge evolution system proposed by Google researchers, designed to enable agents to continuously accumulate and evolve knowledge during use. The mechanism draws an analogy to Wikipedia-style self-improvement.
Timeline (last 90 days)
The concept behind the Ledger system shares similarities with the WikiSkill (evolutionary system) proposed by Google researchers, but the Ledger was independently designed and developed by the author.
The WikiSkill ablation experiment tested the effect of allowing the reasoning agent to read the Wiki layer during training.
In the experiment where the 27B model wrote its own skills, it scored 56.3% on LiveMath.
The division-of-labor model in which small models act as skill writers and large models act as skill executors can provide cost optimization ideas for multi-agent systems.
A smaller 4B model can be an excellent 'teacher' for writing skills to a 27B large model, because the smaller model is forced to write highly explicit, step-by-step operation guides.
When an agent can read the Wiki layer, it directly retrieves raw analytical answers, bypassing procedural skills and thereby contaminating the execution trajectory.
When agents can see the raw analysis at the Wiki layer, they directly grab answers and bypass procedural skills, contaminating execution trajectories (the shortcut learning problem).
When designing a teacher-student architecture, using a small model for knowledge distillation and skill extraction while using a large model for actual execution may be a more optimal cost-performance combination.
WikiSkill decouples the three stages of discovery, documentation, and execution, preventing errors from cross-contaminating within a single loop, with each layer independently auditable and optimizable.
The architectural principle given by WikiSkill is that the Wiki layer should be visible only to the skill-proposing agent and not to the execution agent.
16 more timeline events
All Facts (20)
Using small models for skill distillation and large models for actual execution may be a more optimal cost-performance combination
70%UnverifiedSmaller models are forced to produce highly explicit, step-by-step operational instructions, which happen to be exactly the type of input that large models are good at executing.
60%UnverifiedThe concept behind the Ledger system shares similarities with the WikiSkill (evolutionary system) proposed by Google researchers, but the Ledger was independently designed and developed by the author.
50%UnverifiedThe WikiSkill ablation experiment tested the effect of allowing the reasoning agent to read the Wiki layer during training.
50%UnverifiedIn the experiment where the 27B model wrote its own skills, it scored 56.3% on LiveMath.
50%UnverifiedThe division-of-labor model in which small models act as skill writers and large models act as skill executors can provide cost optimization ideas for multi-agent systems.
50%UnverifiedA smaller 4B model can be an excellent 'teacher' for writing skills to a 27B large model, because the smaller model is forced to write highly explicit, step-by-step operation guides.
50%UnverifiedWhen an agent can read the Wiki layer, it directly retrieves raw analytical answers, bypassing procedural skills and thereby contaminating the execution trajectory.
50%UnverifiedWhen agents can see the raw analysis at the Wiki layer, they directly grab answers and bypass procedural skills, contaminating execution trajectories (the shortcut learning problem).
50%UnverifiedWhen the execution agent has access to the Wiki layer, it directly retrieves answers, bypassing procedural skills, which leads to shortcut learning and contaminates the execution trajectories.
50%UnverifiedAblation experiments show that allowing the reasoning agent to read the Wiki layer during training leads to a drop in win rate.
50%UnverifiedWikiSkill decouples the three stages of discovery, documentation, and execution, preventing errors from cross-contaminating within a single loop, with each layer independently auditable and optimizable.
50%UnverifiedThe architectural principle given by WikiSkill is that the Wiki layer should be visible only to the skill-proposing agent and not to the execution agent.
50%UnverifiedWhen designing a teacher-student architecture, using a small model for knowledge distillation and skill extraction while using a large model for actual execution may be a more optimal cost-performance combination.
50%UnverifiedThe ability to write skills and the ability to execute skills are two different capabilities.
50%UnverifiedA smaller 4B model can serve as an excellent skill-writing 'teacher' for a 27B large model.
50%UnverifiedThe WikiSkill paper proposes the idea of completely decoupling 'skill discovery' from 'skill execution'
50%UnverifiedDecoupling the three stages of discovery, documentation, and execution can prevent errors from cross-contaminating within a single loop, and allows each layer to be independently audited and optimized
50%UnverifiedThe architectural principle given by WikiSkill is that the Wiki layer should be visible only to the skill-proposing agent and not to the execution agent
50%UnverifiedWhen the execution agent can read the Wiki layer, it directly retrieves answers and bypasses procedural skills, thereby contaminating the execution trajectories
50%