ConceptLoRA adapter
LoRA Adapter
A low-rank weight increment module trained with the LoRA method that can be attached to a base model to achieve task-specific fine-tuning. Its parameter count is only 0.1%–1% of the original model.
Timeline (last 90 days)
Nov 4
The researchers computed the principal components of the adapter update matrices and tracked their dynamics as λ varied; when λ causes the teacher to stop thinking, certain directions in the update subspace shift significantly, and this shift point corresponds to the disappearance of the teacher's thinking behavior.
Unverified50%