Task Arithmetic
A training-free model merging method that defines the parameter difference between a fine-tuned model and its base model as a 'task vector.' Multiple models' task capabilities can be flexibly combined in parameter space by performing vector addition and subtraction on these task vectors.
Timeline (last 90 days)
The Task Arithmetic method treats the parameter difference between a fine-tuned model and a base model as a task vector, combining multiple task capabilities in parameter space through vector addition and subtraction.
Task Arithmetic, proposed by Ilharco et al. in 2023, obtains task vectors by subtracting pre-trained weights from fine-tuned model weights; adding multiple task vectors together and superimposing them onto the pre-trained weights can combine multi-task capabilities.
The effectiveness of Task Arithmetic is premised on multiple fine-tuned models sharing the same pre-training starting point and cannot be directly applied to two networks independently trained from random initialization.
All Facts (3)
The Task Arithmetic method treats the parameter difference between a fine-tuned model and a base model as a task vector, combining multiple task capabilities in parameter space through vector addition and subtraction.
50%UnverifiedTask Arithmetic, proposed by Ilharco et al. in 2023, obtains task vectors by subtracting pre-trained weights from fine-tuned model weights; adding multiple task vectors together and superimposing them onto the pre-trained weights can combine multi-task capabilities.
50%UnverifiedThe effectiveness of Task Arithmetic is premised on multiple fine-tuned models sharing the same pre-training starting point and cannot be directly applied to two networks independently trained from random initialization.
50%