Knowledge Editing
A subfield of large language model research focused on updating or correcting factual knowledge already learned by a model. Primary approaches include direct parameter modification, retrieval-augmented methods, and fine-tuning.
Timeline (last 90 days)
Mainstream knowledge editing methods include parameter-modification methods (such as ROME), meta-learning methods (such as MEND) and external-memory methods, and the interventions of these methods all occur at the level of weights or model structure.
Knowledge editing directly modifies a model's internal parameters to update or correct specific facts.
Orthogonal complement subspace constraints are a common means of mitigating forgetting in continual learning and knowledge editing.
ROME (Rank-One Model Editing) is a knowledge editing method that changes a model's recall of specific facts by locating and modifying key weights in the feedforward network.
The survey connects mechanistic understanding with safety practice and discusses three types of applications: knowledge editing, steering, and monitoring.
Knowledge editing is a class of techniques that directionally modify a certain piece of factual knowledge in a model without retraining the entire model, with typical methods including locating and modifying specific layers/neurons, and inserting external memory at inference time.
Evaluating knowledge editing typically focuses on three dimensions: edit success rate, generality and locality; sequential editing scenarios additionally examine portability and long-term stability.
A model maintaining factual accuracy on obscure topics reflects its knowledge density and training quality more than writing a sorting algorithm.
All Facts (8)
Mainstream knowledge editing methods include parameter-modification methods (such as ROME), meta-learning methods (such as MEND) and external-memory methods, and the interventions of these methods all occur at the level of weights or model structure.
50%UnverifiedKnowledge editing directly modifies a model's internal parameters to update or correct specific facts.
50%UnverifiedOrthogonal complement subspace constraints are a common means of mitigating forgetting in continual learning and knowledge editing.
50%UnverifiedROME (Rank-One Model Editing) is a knowledge editing method that changes a model's recall of specific facts by locating and modifying key weights in the feedforward network.
50%UnverifiedThe survey connects mechanistic understanding with safety practice and discusses three types of applications: knowledge editing, steering, and monitoring.
50%UnverifiedKnowledge editing is a class of techniques that directionally modify a certain piece of factual knowledge in a model without retraining the entire model, with typical methods including locating and modifying specific layers/neurons, and inserting external memory at inference time.
50%UnverifiedEvaluating knowledge editing typically focuses on three dimensions: edit success rate, generality and locality; sequential editing scenarios additionally examine portability and long-term stability.
50%UnverifiedA model maintaining factual accuracy on obscure topics reflects its knowledge density and training quality more than writing a sorting algorithm.
50%