Kolmogorov-Arnold Networks

Unlike MLPs, which place fixed nonlinear activation functions at the nodes and weights on the edges, KANs place learnable spline functions (usually B-splines) on the edges, and the nodes only perform summation.

The splines on the edges of a KAN are only locally effective within their domain, and this locality makes KAN's changes to existing functions more concentrated when learning a new task, so it is less likely to affect the parameter regions associated with old tasks.

The paper notes that KAN can accomplish tasks of learning physical relationships in control problems with significantly higher parameter efficiency.

KAN was originally proposed in 2024 by researchers from MIT and other institutions (Liu et al., 2024), and typically uses B-splines or other learnable basis functions to parameterize the activation functions on edges.

Experiments show that the relative performance improvement of KAN over MLP during training can reach up to 50%.

Experiments show that KAN is robust to reward noise, with its advantages remaining stable when the reward signal has varying degrees of noise.

The paper's conclusions are primarily built on the Feynman dataset and Gymnasium benchmarks, and whether the 40% sample savings can be reproduced for KAN in more complex, high-dimensional real-world robotic or industrial systems requires further validation.

A study published on arXiv (arXiv:2610.10627) systematically investigates the potential of the KAN architecture to improve sample efficiency in reinforcement learning.

The core conclusion of the paper is that using the KAN architecture can achieve performance comparable to traditional MLPs with a 40% reduction in sample size.

KAN places learnable activation functions on the edges (connections) of the network, whereas traditional MLPs use fixed activation functions at nodes and learnable weights on edges.

The theoretical foundation of KAN derives from the Kolmogorov–Arnold representation theorem.

KAN places learnable activation functions on the edges of the network, with each edge corresponding to a trainable spline function, differing from traditional MLPs which apply fixed activation functions at nodes.

KANSteer uses Kolmogorov-Arnold Networks (KAN) to provide one-dimensional coordinates, with the learned functions defining the trajectory of concepts through the activation space.

Kolmogorov-Arnold Networks (KAN) were proposed in 2024 by MIT and other institutions.

KAN (Kolmogorov-Arnold Networks), proposed in 2024, abandons the traditional fixed weight matrix and places a learnable spline function on each network connection.