Conditional Autoencoder
A variant of the autoencoder in which the encoder receives an input and a conditioning variable, and the decoder reconstructs the input from the representation and the conditioning variable. In this framework the conditioning variable is a sensitive attribute, and concept erasure is achieved by jointly optimizing the reconstruction loss and an independence constraint.
Timeline (last 90 days)
The paper proposes a framework based on conditional autoencoders that provides the first formal guarantee for nonlinear concept erasure.
The conditional autoencoder's objective function simultaneously constrains reconstruction quality (preserving task-relevant information) and the population-level independence of the sensitive attribute from the representation (removing sensitive information).
By changing the target concept value at inference time within the same conditional autoencoder, the framework can generate representations corresponding to different concept values without additional training cost, achieving representation steering.
Representation steering and erasure appear contradictory, but the conditional autoencoder unifies the two operations by taking the concept value as a conditional input and replacing that conditional value at inference time, sharing the same set of parameters.
All Facts (4)
The paper proposes a framework based on conditional autoencoders that provides the first formal guarantee for nonlinear concept erasure.
50%UnverifiedThe conditional autoencoder's objective function simultaneously constrains reconstruction quality (preserving task-relevant information) and the population-level independence of the sensitive attribute from the representation (removing sensitive information).
50%UnverifiedBy changing the target concept value at inference time within the same conditional autoencoder, the framework can generate representations corresponding to different concept values without additional training cost, achieving representation steering.
50%UnverifiedRepresentation steering and erasure appear contradictory, but the conditional autoencoder unifies the two operations by taking the concept value as a conditional input and replacing that conditional value at inference time, sharing the same set of parameters.
50%