Spectral-State Neural Networks: A Mode-Decomposition Architecture for Learned Dynamics

Standard neural networks represent hidden state as unstructured activation vectors evolved by arbitrary weight matrices. We introduce Spectral-State Neural Networks (SSNNs): an architecture where the latent representation is a vector of spectral mode amplitudes, evolved by structured operators with explicit stability, damping, and mode-mixing properties. The state update replaces the standard \(h_{t+1} = \sigma(Wh_t + b)\) with \(z_{t+1} = G_t U_t z_t + P x_t\), where \(U_t\) is a learned near-orthogonal mode-mixing operator, \(G_t\) is a diagonal mode-gate controlling damping and amplification per mode, and \(P\) projects inputs into spectral space. We prove three properties: (1) stability: the spectral norm \(\|G_t U_t\| \leq 1 + \varepsilon\) is enforced by construction, preventing gradient explosion; (2) timescale separation: slow modes (\(|g_k| \approx 1\)) carry long-term memory while fast modes (\(|g_k| \ll 1\)) respond to immediate inputs; (3) spectral optimality: for linear targets, the SSNN with \(K\) modes achieves the Eckart-Young optimal \(K\)-rank approximation error. Experiments on chaotic time-series prediction (Lorenz, Mandelbrot iteration), sequence modeling, and Lean tactic prediction show that SSNNs match or exceed standard RNNs and state-space models while producing interpretable mode decompositions of the learned dynamics.