What Does Your Model Know? Spectral Decomposition and Arithmetic of Machine Learning Knowledge

We present a method to decompose any trained machine learning model's knowledge into a vector of spectral coefficients, and show that arithmetic on these vectors corresponds to meaningful operations on knowledge: addition combines complementary models, subtraction isolates specific signals, and distance quantifies model similarity with per-mode structural diagnosis. The method requires only the model's predictions — no access to weights, architecture, or training data.

In a banking scenario, two credit models trained on non-overlapping features (credit bureau data vs. behavioral data) are combined by adding their spectral vectors: prediction error drops from 5.16 and 3.64 (individually) to 0.22 (combined) — a 23x improvement from one vector addition. Two models trained on similar data are compared mode-by-mode: the spectral distance reveals they are 7.7x more similar than models from different data sources, and the per-mode comparison identifies exactly which patterns each model captured and where they disagree.

We demonstrate the framework on 19 model types (linear, tree, kernel, neural, nearest-neighbor families) from scikit-learn — all are spectrally decomposable. The spectral representation can be serialized (702 KB vs. 5.3 MB for the original Random Forest), loaded without the original model, and operated on algebraically. Applications include: model governance (structural diff between model versions), fairness auditing (subtract bias by one operation), federated learning (share coefficient vectors, not patient data), and continual learning (add new task knowledge without forgetting old).