TIES
A model merging method that reduces parameter interference among multiple fine-tuned updates through a three-step process of trimming, sign election, and merging.
Timeline (last 90 days)
The TIES method fluctuates in accuracy by 6.7 percentage points at the 4B scale, whereas selection by loss effect fluctuates by only 0.7 percentage points.
In single-target expansion scenarios, SSU-Wanda+TIES achieves the best balance between source-language retention and target-language acquisition, outperforming target-only fine-tuning and standard TIES merging.
SSU-Wanda is used to select and protect parameters important to the source language, while TIES reduces interference by merging multiple updates.
TIES (Trim, Elect Sign, Merge) is a model merging method whose procedure is: first trim the largest-magnitude parameters in each update vector, then perform sign election on conflicting parameters according to the majority update direction, and finally merge only the parameters consistent with the elected sign.
Among merging methods, LERP (linear interpolation) leans more toward retaining adult recognition capability, while TIES is better at recovering children's recognition gains.
TIES (Trim, Elect Sign & Merge) resolves directional conflicts during multi-task vector superposition through sign voting.
All Facts (6)
The TIES method fluctuates in accuracy by 6.7 percentage points at the 4B scale, whereas selection by loss effect fluctuates by only 0.7 percentage points.
50%UnverifiedIn single-target expansion scenarios, SSU-Wanda+TIES achieves the best balance between source-language retention and target-language acquisition, outperforming target-only fine-tuning and standard TIES merging.
50%UnverifiedSSU-Wanda is used to select and protect parameters important to the source language, while TIES reduces interference by merging multiple updates.
50%UnverifiedTIES (Trim, Elect Sign, Merge) is a model merging method whose procedure is: first trim the largest-magnitude parameters in each update vector, then perform sign election on conflicting parameters according to the majority update direction, and finally merge only the parameters consistent with the elected sign.
50%UnverifiedAmong merging methods, LERP (linear interpolation) leans more toward retaining adult recognition capability, while TIES is better at recovering children's recognition gains.
50%UnverifiedTIES (Trim, Elect Sign & Merge) resolves directional conflicts during multi-task vector superposition through sign voting.
50%