Kernel Machine
A non-parametric learning method based on kernel functions. It implicitly maps data into a high-dimensional feature space through a kernel function and solves the problem using linear methods.
Timeline (last 90 days)
High probe accuracy is compatible with two geometric structures: a true linear world model (in which a subspace of latent alignment dominates the representation) and a kernel machine (in which latent variables are linearly embedded in a larger curved manifold).
In the kernel machine case, linear probes can still achieve high accuracy because random feature maps can also linearly separate data at sufficiently high dimensions, but the model has not learned a linear world model and its responses to out-of-distribution data may deviate completely from expectations.
In a linear world model, reverse prediction should be symmetric with forward prediction, whereas in a kernel machine, reverse prediction is significantly weaker than forward prediction.
All Facts (3)
High probe accuracy is compatible with two geometric structures: a true linear world model (in which a subspace of latent alignment dominates the representation) and a kernel machine (in which latent variables are linearly embedded in a larger curved manifold).
50%UnverifiedIn the kernel machine case, linear probes can still achieve high accuracy because random feature maps can also linearly separate data at sufficiently high dimensions, but the model has not learned a linear world model and its responses to out-of-distribution data may deviate completely from expectations.
50%UnverifiedIn a linear world model, reverse prediction should be symmetric with forward prediction, whereas in a kernel machine, reverse prediction is significantly weaker than forward prediction.
50%