Machine Learning

We introduce the Structured Latent Basis (SLB) framework, a perspective on supervised learning that unifies feature engineering and modeling as a single problem: selecting a mathematical basis in which the target function is linear.

We introduce the Smooth-Step Spectral Method (S³M), a structured basis regression approach that unifies smooth spectral features with learned threshold features for tabular data.

Large language models are deployed in medicine, law, education, and critical infrastructure — yet we cannot mathematically characterize when their outputs are trustworthy.

We present a unified framework in which a single parameter — the analyticity radius $\rho$ — simultaneously controls five properties of smooth systems: (1) **structural complexity** (the number of interaction grades needed), (2) **computational cost*

We present a unified, machine-verified theory of AI safety in which the eigenvalue decay rate $s$ of the data covariance determines a formal chain: scaling laws, self-improvement ceilings, robustness certificates, and a quantitative safety budget.

We identify a structural property unique to large language model (LLM) agents: when an agent's behavioral policy is specified in natural language (a "skill file"), and its reasoning engine operates in the same modality (natural language), the agent c

The Latent Theorem guarantees that any smooth system has a finite representation whose size depends on regularity and accuracy, not on ambient dimensionality. We extend this result to **families** of smooth systems.

Emergent capabilities in large language models — abilities that appear suddenly as model scale increases — remain poorly understood. We propose that emergence corresponds to the activation of new **universal Latent modes**: directions in the model's representation space that are shared across all domains.

We present the first complete, machine-verified proof chain connecting four fundamental phenomena in deep learning theory: neural scaling laws, attention as spectral filtering, grokking as a spectral phase transition, and holographic entropy bounds on learned representations.

The companion methods paper (Nagy 2026a) defines the Knowledge Artifact — a portable spectral representation of trained model knowledge — and the Knowledge Algebra — exact arithmetic on compatible artifacts.

Modern learning systems appear algorithmically diverse yet empirically convergent toward a small set of training regimes. We propose a universality framework in which optimization dynamics are classified by spectral invariants rather than by optimizer-specific update rules.

We develop a phase-transition theory of generalization based on spectral order parameters. Instead of smooth monotonic scaling, we show that model performance exhibits threshold behavior when mode occupancy, noise-floor coupling, and effective rank cross critical boundaries.

Current AI agent architectures either use static retrieval (RAG) or unstructured agent loops (ReAct, Reflexion) with no formal guarantees on memory utilization, routing optimality, or safe self-adaptation.

LLM agents retrieve context via flat embedding similarity: the query is embedded, the $k$ nearest neighbors are returned, and relevance decays with cosine distance. This approach ignores **structure**: which claims support which, which topics form coherent clusters, where the gaps are, and what timescale each memory operates on.

Standard neural networks represent hidden state as unstructured activation vectors evolved by arbitrary weight matrices.

Modern training optimizes losses that are local, task-specific, and often blind to the amount of recoverable structure present in the data. We propose a complementary view: the spectral quality parameter $\rho > 1$ should be understood as a **structural fitness metric**.

We show that the uncertainty of any smooth learning problem is captured by $K^*$ spectral modes of the data covariance, where $K^* = \Theta(\log(n/\sigma^2)/\log(\rho^2))$ depends only on the dataset size $n$, noise level $\sigma$, and eigenvalue dec

We show that the trainability of variational quantum circuits is determined by a single number: the spectral gap $|\lambda_1|$ of the Lindblad noise generator.

We propose that emergent abilities in large language models are spectral phase transitions: a capability appears when the model's spectral resolution crosses the task's intrinsic complexity.

Low-Rank Adaptation (LoRA; Hu et al., 2021) fine-tunes large language models by adding rank-$r$ updates $\Delta W = AB$ with $r \ll d$. In practice, $r = 4$--$16$ works remarkably well, but no theory explains why or predicts the optimal $r$ for a given task.

Can we *prove* a neural network is safe? Adversarial examples — imperceptible input perturbations that cause misclassification — remain the most persistent failure mode of deep learning. Existing defenses rely on empirical testing, which cannot guarantee safety.

Neural scaling laws — the empirical observation that test loss decreases as a power law in compute budget — are the foundation of modern AI training strategy. Every major laboratory trains billion-dollar models by extrapolating scaling curves, yet *why* these power laws hold remains unexplained.

Knowledge distillation (Hinton et al., 2015) compresses a large teacher model into a smaller student model by training on the teacher's soft outputs. The student is a smaller neural network — still a black box, with no guarantee on how much knowledge was preserved.

We present spectral model compression: any trained model — neural network, tree ensemble, or kernel machine — can be distilled into $K^*$ spectral coefficients via eigendecomposition of its learned function on the data manifold.

Can we *prove* an AI system is safe — not with testing, not with benchmarks, but with mathematical certainty? We present a Lean 4 verification of a unified AI safety certificate — to our knowledge, the first formal verification of a multi-dimensional

We present a method to decompose any trained machine learning model's knowledge into a vector of spectral coefficients, and show that arithmetic on these vectors corresponds to meaningful operations on knowledge: addition combines complementary model

We prove that a single singular value decomposition (SVD) of a neural network's local Jacobian yields three formally verified trustworthiness guarantees simultaneously.

We introduce **spectral distillation**: a method to compress any black-box model into an explicit Fourier cosine formula with a **provable per-feature error bound**.

We introduce a spectral decomposition framework for computational workloads and prove that the von Neumann architecture — where all hardware resources serve a single universal execution mode — is strictly suboptimal for any non-degenerate workload.

Knowledge Artifacts (Nagy 2026a) support algebraic operations in a shared spectral eigenbasis.

We describe SpectralShell, a universal model class that loads, predicts from, and performs algebra on Knowledge Artifacts — portable spectral representations of trained ML models.

A 200-tree Random Forest has 126,074 parameters. Its knowledge? Three numbers and a basis. We introduce the **Knowledge Artifact** — a portable representation of what any ML model has learned — and the **Knowledge Algebra** — provably exact arithmetic on these artifacts.

We derive neural scaling laws, transformer convergence rates, and self-improvement limits from a single principle: the eigenvalue decay of the data covariance matrix. For data with spectral exponent $s$ (eigenvalues $\lambda_k \sim k^{-s}$), we prove: 1.