SW-KAN: Solving KAN's Domain Mismatch Problem with q-Orthogonal Polynomials

The Rise of KAN and an Overlooked Vulnerability

Kolmogorov-Arnold Networks (KAN) have emerged in recent years as a genuinely paradigm-shifting attempt in deep learning. Unlike traditional neural networks that place fixed activation functions on nodes, KAN places learnable univariate functions on the network's edges. This seemingly small change delivers two immediate benefits: stronger interpretability and higher parameter efficiency. In other words, the model can express more complex relationships with fewer parameters, while remaining human-readable at each edge.

The original KAN relied on B-spline implementations — a flexible but computationally expensive approach. This prompted researchers to propose polynomial-based KAN variants, replacing B-splines with orthogonal polynomials to reduce computational costs. However, these methods introduced a long-underestimated problem: domain mismatch.

Orthogonal polynomials are typically defined over bounded intervals or semi-infinite intervals, while real-world neural network inputs are often unbounded real values. When unbounded inputs are forced onto polynomial bases with bounded support, both numerical stability and expressive power suffer. This new arXiv paper targets exactly this pain point.

A B-spline is a piecewise polynomial curve defined by a set of control points, maintaining smooth continuity between adjacent segments. Its strength lies in local support — changing one control point only affects a local region of the curve, making B-splines both flexible and stable when fitting complex shapes. The original KAN parameterizes the learnable function on each edge with a B-spline, effectively learning an adaptive curve for every input-output connection. However, B-splines come at a significant cost: they require storing and updating large numbers of control points, and their parallel computation on GPUs is far less efficient than matrix multiplication. This is precisely why subsequent researchers turned to orthogonal polynomials (such as Chebyshev and Legendre polynomials) — these can be computed efficiently via clean recurrence relations and are naturally GPU-friendly. But they are typically defined over bounded intervals like [-1, 1], creating a natural rift with the unbounded activation values found in real neural networks: the domain mismatch problem this paper focuses on.

The Core Idea Behind SW-KAN

The paper introduces the Stieltjes-Wigert Kolmogorov-Arnold Network (SW-KAN), which takes a different path: using Stieltjes-Wigert q-orthogonal polynomials, defined over the semi-infinite domain (0, ∞), as basis functions.

This choice is far from arbitrary. Stieltjes-Wigert polynomials carry a log-normal weight structure and a learnable q parameter. The authors argue that this structure provides a unique inductive bias, keeping the model robust under resource-constrained conditions — such as compressed feature dimensions or limited training data.

Bridging the Domain Gap

Connecting unbounded real-valued inputs to the (0, ∞) polynomial support requires a stable mapping. SW-KAN introduces a smooth exponential-of-tanh mapping. The elegance of this mapping lies in two properties: it reliably transports arbitrary real numbers to the positive half-axis, while maintaining well-conditioned gradients — avoiding gradient explosion or vanishing during training.

The essence of domain mismatch is the misalignment between the input distribution and the support of the basis functions. Rather than trimming inputs to fit the polynomial (cutting the foot to fit the shoe), SW-KAN designs a mathematically smooth and numerically stable bridge. This is the core innovation that sets it apart from previous polynomial KAN approaches.

The tanh function compresses any real number to the interval (-1, 1), and taking its exponential (exponential-of-tanh) shifts the output range to (e⁻¹, e¹) ≈ (0.37, 2.72) — a bounded subset of the positive half-axis (0, ∞). SW-KAN's design is more refined: by appropriately scaling the tanh output, it can cover the region where the Stieltjes-Wigert polynomial weight function is concentrated, while the mapping's derivative is everywhere non-zero and bounded, ensuring gradients neither explode nor vanish during backpropagation. This is fundamentally different from the simple clipping or normalization schemes commonly used in earlier polynomial KANs — clipping introduces non-differentiable points, while normalization depends on batch statistics, making it unstable for small batches or during inference. The exponential-of-tanh is a purely element-wise smooth transformation, fully compatible with automatic differentiation frameworks, with virtually zero additional engineering overhead.

Guaranteeing Computational Efficiency

Efficiency is the raison d'être of polynomial KAN. SW-KAN leverages a numerically stable three-term recurrence relation to compute the polynomial expansion. The advantages of this recurrence are: it completes the expansion in O(N) complexity and requires no special-function calls.

This matters greatly in practice. Special function calls are often performance bottlenecks and sources of numerical instability, while a pure recurrence approach makes the entire evaluation process both fast and predictable.

Experimental Results and Applicable Scenarios

The paper presents fairly comprehensive experiments across two task categories: image classification and continuous function approximation. The conclusion is that SW-KAN achieves a superior accuracy-efficiency trade-off across diverse tasks.

More specifically, the authors highlight several key findings:

• On standard benchmarks, SW-KAN outperforms existing polynomial KAN baseline models.
• For complex multivariate function approximation, SW-KAN demonstrates strong representational capacity with very few parameters.
• In resource-constrained settings (low feature dimensions, limited training data), the inductive bias from its log-normal weight structure and learnable q parameter proves robust.

This points to a very clear target audience for SW-KAN: scenarios sensitive to parameter budgets that still require efficient function approximation or classification capabilities — such as edge device deployment and few-shot learning.

Inductive bias refers to the prior assumptions a model relies on when generalizing beyond training data. All machine learning models embody some form of inductive bias: convolutional neural networks assume local translation invariance, while Transformers assume global attention relationships across sequences. SW-KAN's inductive bias derives from the log-normal weight structure of Stieltjes-Wigert polynomials — a structure naturally suited to positively skewed, multiplicative data, which is ubiquitous in nature and engineering systems (e.g., financial returns, certain distributions in particle physics). The learnable q parameter further allows the model to dynamically adjust the strength of this bias during training, enabling fast convergence when the prior matches the data while avoiding over-constraint when it doesn't. This explains why SW-KAN performs relatively robustly in small-data scenarios: a compact yet structured inductive bias exploits limited samples more efficiently.

A Balanced Assessment: Value and Limitations

SW-KAN's contribution lies in explicitly naming a long-overlooked theoretical flaw in polynomial KAN variants — domain mismatch — and offering a solution that balances mathematical rigor with engineering feasibility. The combination of the exponential-of-tanh mapping and three-term recurrence reflects the authors' careful trade-off between stability and efficiency.

That said, a measured perspective is warranted: this is a freshly posted preprint (arXiv:2610.00050v1) that has not yet undergone peer review. The performance advantages claimed in the paper are based on an experimental scope defined by the authors themselves; true generalization capability still requires broader replication and third-party validation. Furthermore, whether the log-normal bias introduced by Stieltjes-Wigert polynomials is beneficial across all tasks remains an open question.

For researchers and engineers tracking KAN's development, SW-KAN offers a noteworthy new direction: rather than patching bounded polynomials, redesign from the ground up with semi-infinite domain basis functions. Whether it will become a mainstream choice for efficient function approximation remains to be seen.