Intrinsic and Post-hoc Interpretability of Gaussian-Based Kolmogorov–Arnold Networks

Standard feedforward nets hang their nonlinearity on the nodes and keep the edges linear. Kolmogorov–Arnold Networks swap the two: each edge is now a learnable one-variable map, each node a plain summation. A small architectural move, but with a payoff we want to test here, namely, that the trained network becomes partly readable, edge by edge, without extra tooling. Our test bed is the shared-basis Gaussian variant of the KAN, recently published, and the task is regression on California Housing. We train a three-layer network with five Gaussian kernels per layer and then look at it from four angles. The first angle is the tensor of first-layer coefficients, summarized into a per-feature importance number. The second is KernelSHAP run globally; the third is LIME, pointwise; the fourth, and the one that only makes sense on a KAN, is the direct drawing of the learned edge activations as curves. Intrinsic and SHAP rankings land on the same two features at the top, MedInc and Latitude, and the same feature at the bottom (Population). They shuffle the middle, and that shuffle is diagnostic: coefficient size reports on encoding strength in layer zero, SHAP on end-to-end marginal effect. The plotted edge activations confirm what domain knowledge would suggest: income acts roughly monotonically on price, geography acts locally, crowding has a threshold effect. Taken together, the four views make the model easier to audit than a comparable black box would be.