KANSteer
A nonlinear activation steering method based on Kolmogorov-Arnold Networks. It models concept traversal in text-to-image generation models as curves rather than straight lines, enabling more precise control over intermediate semantic states.
Timeline (last 90 days)
The arXiv paper "Beyond the Linear Representation Hypothesis" proposes the KANSteer method.
The study found that when intermediate attribute prompts are fed into text-to-image models, activations rarely fall along the straight-line direction connecting the two endpoints.
The core idea of KANSteer is to model concept traversal as a curve passing through intermediate states, rather than a straight line.
KANSteer uses Kolmogorov-Arnold Networks (KAN) to provide one-dimensional coordinates, with the learned functions defining the trajectory of concepts through the activation space.
The paper validates across multiple concepts and multiple text-to-image diffusion Transformers, finding that activation trajectories deviate significantly from straight lines.
Compared to linear steering, KANSteer more closely follows the true trajectory and achieves smoother traversal between intermediate attributes.
As a preprint paper, KANSteer's generalization capability, computational overhead, and applicability to a broader range of generative tasks remain to be validated in subsequent work.
All Facts (7)
KANSteer uses Kolmogorov-Arnold Networks (KAN) to provide one-dimensional coordinates, with the learned functions defining the trajectory of concepts through the activation space.
50%UnverifiedThe study found that when intermediate attribute prompts are fed into text-to-image models, activations rarely fall along the straight-line direction connecting the two endpoints.
50%UnverifiedThe core idea of KANSteer is to model concept traversal as a curve passing through intermediate states, rather than a straight line.
50%UnverifiedAs a preprint paper, KANSteer's generalization capability, computational overhead, and applicability to a broader range of generative tasks remain to be validated in subsequent work.
50%UnverifiedThe paper validates across multiple concepts and multiple text-to-image diffusion Transformers, finding that activation trajectories deviate significantly from straight lines.
50%UnverifiedCompared to linear steering, KANSteer more closely follows the true trajectory and achieves smoother traversal between intermediate attributes.
50%UnverifiedThe arXiv paper "Beyond the Linear Representation Hypothesis" proposes the KANSteer method.
50%