Task Vector
A vector formed by the difference between model weights before and after fine-tuning. It encodes the knowledge learned on a specific task and can be used for knowledge injection and model merging.
Timeline (last 90 days)
Task Vectors (TVs), extracted by contrasting model hidden states under different tasks, are a lightweight means of model steering.
Recent research proposed that task vectors can be linearly decomposed into functionally interpretable components (the TR component, the TL component, and a residual component).
Experimental results showed that the intervention directions identified by TR/TL-based optimization outperform the original task vectors in performance.
The decomposability of task vectors was cross-validated by the research team's experiments on multiple model families and by independent analysts (such as Bilibili uploaders and YouTube channels).
The linear decomposition of task vectors provides a new tool for understanding the mechanism of in-context learning and is expected to advance the development of explainable AI.
All Facts (5)
Task Vectors (TVs), extracted by contrasting model hidden states under different tasks, are a lightweight means of model steering.
50%UnverifiedRecent research proposed that task vectors can be linearly decomposed into functionally interpretable components (the TR component, the TL component, and a residual component).
50%UnverifiedExperimental results showed that the intervention directions identified by TR/TL-based optimization outperform the original task vectors in performance.
50%UnverifiedThe decomposability of task vectors was cross-validated by the research team's experiments on multiple model families and by independent analysts (such as Bilibili uploaders and YouTube channels).
50%UnverifiedThe linear decomposition of task vectors provides a new tool for understanding the mechanism of in-context learning and is expected to advance the development of explainable AI.
50%