Adversarial Training
A machine learning training technique that incorporates adversarial examples or adversarial perturbations to improve model robustness. It systematically reshapes the internal representation structure of a model and is commonly employed in robustness research and in the design of mechanistic interpretability experiments.
Timeline (last 90 days)
Adversarial examples exploit the fragility of the model's decision boundary in high-dimensional space; the network's robustness mechanisms (such as normalization and Dropout) are effective against random noise but almost ineffective against adversarial perturbations, because the perturbation direction is exactly the gradient ascent direction.
GCD does not require retraining the model, and its deployment cost is far lower than that of adversarial training-type solutions.
At the representation level, adversarially trained robust models are more easily decomposed by SAEs and invoke fewer SAE features in task attribution.
In the high-faithfulness range (90%, 95%), the robust model requires significantly fewer circuit edges.
All Facts (4)
Adversarial examples exploit the fragility of the model's decision boundary in high-dimensional space; the network's robustness mechanisms (such as normalization and Dropout) are effective against random noise but almost ineffective against adversarial perturbations, because the perturbation direction is exactly the gradient ascent direction.
50%UnverifiedGCD does not require retraining the model, and its deployment cost is far lower than that of adversarial training-type solutions.
50%UnverifiedAt the representation level, adversarially trained robust models are more easily decomposed by SAEs and invoke fewer SAE features in task attribution.
50%UnverifiedIn the high-faithfulness range (90%, 95%), the robust model requires significantly fewer circuit edges.
50%