Concept
KnOTS
A model merging alignment method designed specifically to address orthogonal transformation symmetry in Transformer models. It improves merging quality by resolving parameter misalignments caused by orthogonal transformation symmetry.
Timeline (last 90 days)
Oct 8
KnOTS is specifically designed to address orthogonal transformation symmetry in Transformers with a dedicated alignment strategy.
Unverified50%