SAFT
A parameter-efficient adversarial fine-tuning method that identifies robustness-critical parameters (RCPs) through adversarial gradient saliency and performs adversarial training by updating only this sparse parameter subset, thereby maintaining or improving adversarial robustness while reducing the number of trainable parameters.
Timeline (last 90 days)
SAFT (Sparse Adversarial Fine-Tuning) identifies RCPs through adversarial gradient saliency, updates only these parameters, and freezes the rest.
By updating only about 5% of the parameters, SAFT can approach or even surpass full-parameter adversarial training in robustness.
Across multiple network architectures and datasets, SAFT leads LoRA-based methods in both clean accuracy and robust accuracy, while using fewer trainable parameters.
SAFT has significant advantages in memory-constrained scenarios and is expected to promote the practical deployment of adversarial defense for large models, but it still needs to be validated on more complex attacks and in real deployments.
All Facts (4)
SAFT (Sparse Adversarial Fine-Tuning) identifies RCPs through adversarial gradient saliency, updates only these parameters, and freezes the rest.
50%UnverifiedBy updating only about 5% of the parameters, SAFT can approach or even surpass full-parameter adversarial training in robustness.
50%UnverifiedAcross multiple network architectures and datasets, SAFT leads LoRA-based methods in both clean accuracy and robust accuracy, while using fewer trainable parameters.
50%UnverifiedSAFT has significant advantages in memory-constrained scenarios and is expected to promote the practical deployment of adversarial defense for large models, but it still needs to be validated on more complex attacks and in real deployments.
50%