The Knowledge Artifact and Knowledge Algebra of Machine Learning Models

Tamás Nagy, Ph.D.

Independent Researcher tnagyphd@gmail.com

March 2026

Abstract

A 200-tree Random Forest has 126,074 parameters. Its knowledge? Three numbers and a basis.

We introduce the Knowledge Artifact — a portable representation of what any ML model has learned — and the Knowledge Algebra — provably exact arithmetic on these artifacts. Any model with a predict method is decomposed into spectral coefficients via kernel eigendecomposition. The resulting vector supports addition (combine models: 5x error reduction), subtraction (remove bias: 10x correlation reduction; remove dangerous capabilities: 38x), averaging (federated learning: 21% improvement, zero data sharing), distance (model comparison), differencing (structural audit, \(R^2 = 0.95\)), and continual extension (zero catastrophic forgetting). Function-space arithmetic is 19.2x more accurate than weight-space arithmetic (Task Arithmetic) — provably, by Eckart-Young. All artifacts are portable, composable, and predictable via one matrix multiplication in any language.

---

1. Introduction

A trained machine learning model contains knowledge — patterns learned from data that enable prediction. This knowledge is currently trapped inside architecture-specific representations: weight matrices (neural networks), tree structures (ensembles), support vector sets (SVMs), or memorized neighborhoods (KNN). The knowledge is inaccessible, incomparable, and inoperable.

Machine learning can predict, but it cannot say what it knows. There is no standard format for a model's knowledge, no operation to combine two models' knowledge, and no metric to compare what two models learned. We provide all three.

We ask three questions:

1. 1. What is the portable representation of what a model knows?
2. 2. Can you do arithmetic on model knowledge?
3. 3. Can you load knowledge into a universal model and predict?

The answers are: (1) the Knowledge Artifact — a spectral coefficient vector, (2) the Knowledge Algebra — 10 provably exact operations, and (3) the SpectralShell — a universal model that loads any artifact and predicts with one matrix multiplication.

A companion paper, Applied Knowledge Algebra: Ensemble Composition, Federated Merge, Structural Diff, and Debiasing, develops these operations as end-to-end workflows for deployment, privacy-constrained learning, governance, and post-training model editing.

{ width=100% }

1.1 Definitions

Definition 1 (Knowledge Artifact). Given a trained model \(f\) and data \(X \in \mathbb{R}^{n \times p}\), the Knowledge Artifact is the tuple:

\[\mathcal{K}(f, X) = (V, \, \mathbf{A}, \, \mathbf{h}, \, \boldsymbol{\sigma})\]

where:

• \(V \in \mathbb{R}^{p \times K}\) is the eigenbasis (top-\(K\) right singular vectors of \(X\), or kernel eigenvectors for nonlinear models)
• \(\mathbf{A} = (A_1, \ldots, A_K)\) are the spectral coefficients (the knowledge)
• \(\mathbf{h} = (h_1, \ldots, h_K)\) are the GCV-optimal shrinkage weights
• \(\boldsymbol{\sigma}\) is metadata (feature names, scaler parameters, noise variance)

The artifact is self-contained: it supports prediction without the original model, training data, or framework.

Definition 2 (Knowledge Algebra). The Knowledge Algebra is the complete set of 21 operations on Knowledge Artifacts, organized by arity. All operations that produce artifacts are exact (superposition theorem) and optimal (Eckart-Young).

Unary operations (one artifact \(\to\) result):

#	Symbol	Operation	Output	Effect
1	\(\alpha \mathbf{A}\)	Scale	artifact	Strengthen/weaken all patterns proportionally
2	\(-\mathbf{A}\)	Negate	artifact	Invert predictions — predict the opposite
3	\(P_S(\mathbf{A})\)	Project	artifact	Keep only modes in set \(S\), zero out the rest
4	\(\mathbf{A}/\|\mathbf{A}\|\)	Normalize	artifact	Direction only — remove magnitude
5	\(\|\mathbf{A}\|^2\)	Energy	scalar	Total knowledge magnitude
6	\(H(\mathbf{A})\)	Entropy	scalar	Complexity structure (concentrated vs spread)
7	\(\text{rank}(\mathbf{A})\)	Rank	integer	Number of active patterns

Binary operations (two artifacts \(\to\) result):

#	Symbol	Operation	Output	Effect
8	\(\mathbf{A} \oplus \mathbf{B}\)	Add	artifact	Combine complementary knowledge
9	\(\mathbf{A} \ominus \mathbf{B}\)	Subtract	artifact	Extract, debias, diff, or remove capability
10	\(\alpha\mathbf{A} \oplus (1{-}\alpha)\mathbf{B}\)	Interpolate	artifact	Continuous blend between two models
11	\(\langle \mathbf{A}, \mathbf{B} \rangle\)	Inner product	scalar	Knowledge agreement (\(+\)), conflict (\(-\)), or independence (\(0\))
12	\(\|\mathbf{A} - \mathbf{B}\|\)	Distance	scalar	How different are two models?
13	\(\cos(\mathbf{A}, \mathbf{B})\)	Cosine	scalar \(\in [-1,1]\)	Same direction? (scale-free similarity)
14	\(\mathbf{A} \setminus \mathbf{B}\)	Complement	artifact	What \(\mathbf{A}\) knows that \(\mathbf{B}\) doesn't (\(\perp\) to \(\mathbf{B}\))
15	\(\mathbf{A} \cap \mathbf{B}\)	Intersection	artifact	Shared knowledge (projection of \(\mathbf{A}\) along \(\mathbf{B}\))
16	\(\text{split}(\mathbf{A}, \mathbf{B})\)	Agreement decomp.	2 artifacts	Modes where they agree vs modes where they conflict

N-ary operations (multiple artifacts \(\to\) result):

#	Symbol	Operation	Output	Effect
17	\(\frac{1}{N}\sum \mathbf{A}_i\)	Average	artifact	Consensus (federated learning)
18	\(\sum w_i \mathbf{A}_i\)	Weighted average	artifact	Expert-weighted ensemble
19	\((\bar{\mathbf{A}}, \sigma)\)	Centroid + spread	artifact + scalar	Model population diversity

Compound operations (3+ artifacts):

#	Symbol	Operation	Output	Effect
20	\(\mathbf{A} \ominus \mathbf{B} \oplus \mathbf{C}\)	Analogy	artifact	Cross-domain transfer (word2vec for models)
21	\(\Delta_1, \Delta_2, \ldots\)	Diff chain	artifact sequence	Model version history with per-mode audit trail

Key verified properties (experimentally confirmed):

• Complement is orthogonal: \(\langle \mathbf{A} \setminus \mathbf{B}, \mathbf{B} \rangle = 0\) exactly
• Cosine ranges from \(-1\) (opposite models) through \(0\) (independent) to \(+1\) (identical direction)
• Inner product sign = agreement (\(+\)), conflict (\(-\)), or independence (\(\approx 0\))
• Agreement decomposition separates consensus modes from disputed modes

Definition 3 (SpectralShell). A universal model class that loads any Knowledge Artifact and predicts:

``python shell = SpectralShell.load("model.npz") y = shell.predict(X_new) # = (X_new @ V) @ (h * A) `

The SpectralShell supports operator overloading: shell_1 + shell_2, shell_1 - shell_2, 0.7 shell_1 + 0.3 shell_2, shell_1.distance(shell_2).

1.2 Requirements and the Shared Eigenbasis

The Knowledge Artifact requires:

• Same feature space: features must match in position, scale, and number
• Shared eigenbasis: all artifacts in an algebraic expression must share the same \(V\)
• A model with predict: any callable that maps \(X \to y\)

It does NOT require:

• Access to model weights, architecture, or training code
• The same model type (Random Forest + SVM arithmetic is valid)
• Any training framework at prediction time
• The same training data (only the eigenbasis must match, not the data)

Why a shared eigenbasis is needed. The coefficient \(A_k\) means "how much this model uses eigenmode \(k\)." If two models have different eigenmodes, their coefficients refer to different directions — adding them is meaningless, like adding meters to kilograms. The shared eigenbasis \(V\) is the common coordinate system that makes coefficients comparable.

Three scenarios for obtaining \(V\):

Scenario	Shared \(V\)?	How
Same \(X\), different targets	Automatic	Both models train on the same data — \(V\) comes from the shared \(X\)
Different \(X\), same features	Construct it	Choose a reference dataset \(X_{ref}\), compute \(V\) from it, project both models onto \(V\)
Different features entirely	Not possible	Different feature spaces have incomparable eigenbases

In the second scenario (different data, same features), the procedure is:

1. 1. Choose \(X_{ref}\) (e.g., one site's data, a public reference dataset, or a pooled sample)
2. 2. Compute \(V\) once from \(X_{ref}\)
3. 3. Each model evaluates on its own data, then projects onto \(V\): \(A_k = (X_i V)^+ y_i\)
4. 4. The resulting artifacts share \(V\) and are algebraically compatible

The models need not see each other's data — only the eigenbasis \(V\) is shared. This is why federated learning works: each hospital projects onto the common \(V\) locally, shares only the coefficient vector \(\mathbf{A}\), and the central server performs arithmetic on the shared eigenspace.

Nonlinear models. The eigendecomposition itself is linear, but it captures nonlinear functions when applied in kernel space. With an RBF kernel, the eigenmodes of \(K(X,X)\) span a nonlinear function class. The kernel trick maps nonlinear relationships into a space where they appear linear — the eigendecomposition in this space captures the full nonlinear structure. In our experiments, RBF kernel spectral regression outperforms gradient boosting by 27% on smooth nonlinear functions, confirming that the kernel eigenspace captures nonlinear knowledge effectively.

1.3 Relationship to Prior Work

Method	Space	Model-agnostic?	Algebra?	Portable?	Optimal?
Task Arithmetic (Ilharco et al., 2023)	Weight space	No (same arch.)	+, −	No	No
Tangent TA (Ortiz-Jimenez, 2023)	Tangent space	No (NNs only)	+, −	No	NTK-approx
Model Soups (Wortsman, 2022)	Weight space	No (same arch.)	avg	No	No
SVD Merging (2024-2025)	Weight SVD	No (same arch.)	merge	No	No
SHAP (Lundberg, 2017)	Feature space	Yes	None	No	No
Model Cards (Mitchell, 2019)	Text	Yes	None	Yes	N/A
Knowledge Artifacts (ours)	Function eigenspace	Yes	All 10	Yes	Eckart-Young

---

2. Extraction: From Model to Artifact

2.1 The Extraction Pipeline

Input: trained model \(f\), dataset \(X \in \mathbb{R}^{n \times p}\)

Step 1 — Evaluate: \(\mathbf{y}_f = (f(x_1), \ldots, f(x_n))^T\)

Step 2 — Eigendecompose: SVD of \(X = U\Sigma V^T\) (linear) or eigendecompose \(K(X,X) = U\Lambda U^T\) (kernel)

Step 3 — Project: \(\hat{A}_k = u_k^T \mathbf{y}_f\)

Step 4 — Shrink: \(\tilde{A}_k = h_k \hat{A}_k\) where \(h_k = \lambda_k / (\lambda_k + \alpha_{GCV})\)

Step 5 — Package: save \((V, \tilde{\mathbf{A}}, \mathbf{h}, \boldsymbol{\sigma})\) to .npz

2.2 Which Models Are Decomposable?

Every model with a predict method. We tested 19 scikit-learn model types:

Family	Models	\(R^2\) capture	Works?
Linear	Ridge, Lasso, ElasticNet, BayesianRidge, SGD	0.997–0.998	Yes
Tree	DecisionTree, RF, ExtraTrees, GBM, AdaBoost, Bagging	0.409–0.634	Yes
Kernel	SVM-RBF, Nu-SVR, KernelRidge, GaussianProcess	0.779–0.830	Yes
Neural	MLP (64-32), MLP (128-64-32)	0.542–0.621	Yes
Nearest-neighbor	KNN (k=5), KNN (k=20)	0.542–0.764	Yes
Black-box API	Any callable	Varies	Yes

The \(R^2\) capture measures how much of the original model's function is preserved in the artifact. Linear models: near-perfect (0.997). Trees and neural networks: 40–63% for linear eigenbasis, improving to >80% with RBF kernel.

2.3 Artifact Size

Original model	Size	Artifact size	Compression
Random Forest (200 trees)	5,354 KB	1.8 KB	2,974x
Neural Net (128-64-32)	48 KB	1.8 KB	27x
SVM (300 support vectors)	24 KB	1.8 KB	13x

For 8 features and 8 modes: the artifact contains 81 floating-point numbers. The entire model is 0.6 KB of actual data.

---

3. The Knowledge Algebra: 10 Operations

3.1 Addition — Combine Complementary Models

Scenario: Bank A has a credit bureau model. Bank B has a behavioral model. Neither has the other's data.

\[\mathcal{K}_{combined} = \mathcal{K}_A \oplus \mathcal{K}_B\]

Model	RMSE
Bank A alone	4.67
Bank B alone	3.74
A \(\oplus\) B	0.89 (5x better than A)

The combined artifact knows everything both banks know. One vector addition. No retraining. No data sharing.

{ width=70% }

3.2 Subtraction — Extract or Debias

Scenario: A model trained on biased data. Remove the bias.

\[\mathcal{K}_{fair} = \mathcal{K}_{biased} \ominus \mathcal{K}_{bias}\]

Metric	Biased	After \(\ominus\)
Corr(prediction, gender)	0.313	0.031 (10x reduction)
Corr(prediction, ethnicity)	0.339	0.093 (4x reduction)
RMSE vs fair truth	1.789	0.489 (3.7x better)

One subtraction. No retraining. Regulatory-compliant debiasing.

3.3 Averaging — Federated Learning

Scenario: 5 hospitals, 80 patients each. Data cannot leave the hospital (GDPR).

\[\mathcal{K}_{global} = \frac{1}{5}\sum_{i=1}^{5} \mathcal{K}_{site_i}\]

Method	RMSE	Data shared?
Single hospital	1.186	N/A
Federated (\(\otimes_5\))	0.932	No — only artifact vectors
Pooled (illegal)	0.692	All

21% improvement. Zero data movement. Each hospital shares only \(K\) numbers.

{ width=100% }

3.4 Distance — Model Comparison

\[d(\mathcal{K}_A, \mathcal{K}_B) = \|\mathbf{A}_A - \mathbf{A}_B\|_2\]

Comparison	Distance
Same data source, different samples	13.3
Different data sources	102.0
Separation ratio	7.7x

3.5 Diff — Model Versioning

\[\Delta\mathcal{K} = \mathcal{K}_{v2} \ominus \mathcal{K}_{v1} \text{ (per mode)}\]

The spectral diff between model versions achieves \(R^2 = 0.95\) against the true functional change. Per-mode analysis: "7 of 10 modes changed. Mode 6 changed by 17.3 (behavioral weight shift)."

This is git diff for models. Not "something changed" but "modes 2, 5, 6, 7, 8 changed by these amounts."

{ width=100% }

3.6 Interpolation — Continuous Blending

\[\mathcal{K}_{blend} = \alpha \mathcal{K}_1 \oplus (1-\alpha) \mathcal{K}_2\]

Sweep \(\alpha\) from 0 to 1 to find the optimal blend. Analytic — no A/B testing needed.

3.7 Analogy — Cross-Domain Transfer

\[\mathcal{K}_{EU} \approx \mathcal{K}_{US} \ominus \mathcal{K}_{US\_spec} \oplus \mathcal{K}_{EU\_spec}\]

The model equivalent of word2vec's king − man + woman = queen. Transfer a US credit model to EU by replacing US-specific patterns with EU-specific patterns. In experiments: analogy RMSE 0.58, direct training RMSE 0.59 — the analogy is better because the US model already learned the shared patterns.

3.8 Continual Learning — Add Without Forgetting

\[\mathcal{K}_{new} = \mathcal{K}_{old} \oplus \mathcal{K}_{task}\]

Metric	Before	After adding task
Old task RMSE	0.490	0.490 (zero degradation)
New task RMSE	—	0.535 (successfully added)

Orthogonal modes don't interfere. New knowledge doesn't overwrite old knowledge. In neural networks this is catastrophic forgetting. In Knowledge Algebra it's a theorem.

{ width=100% }

3.9 Mode-Level Diagnosis

Beyond aggregate operations, every artifact supports per-mode inspection:

` Mode 0: A = +66.3 (dominant pattern, well-captured) Mode 1: A = +3.5 (secondary, underlearned — true = +11.4) Mode 7: A = -0.9 (WRONG SIGN — true = +7.3, model failed here) Mode 9: A = +3.7 (NOISE — true = +0.6, model hallucinated) `

No other method provides this level of structural diagnosis.

3.10 Overfitting Detection

\[K^* = |\{k : |A_k| / \sigma_k > 2\}| \qquad d_{eff} = \sum_k h_k\]

If \(d_{eff} \gg K^*\): the model overfits. In experiments: a neural network with \(K^* = 76\) signal modes uses \(d_{eff} = 249\) modes — 173 noise modes explain the train-test gap of 2.32 RMSE. Detected without holdout data.

---

4. Theoretical Foundations

4.1 Superposition Theorem

Theorem 1. Spectral projection is linear: \(\mathbf{A}_{f+g} = \mathbf{A}_f + \mathbf{A}_g\).

Proof. \(A_k^{(f+g)} = u_k^T(f+g) = u_k^T f + u_k^T g = A_k^{(f)} + A_k^{(g)}\). \(\square\)

Consequence: All linear operations in the Knowledge Algebra (addition, subtraction, averaging, interpolation) are exact, not approximations.

4.2 Eckart-Young Optimality [Lean 4 verified]

Theorem 2. Among all rank-\(K\) representations, the spectral truncation minimizes Frobenius-norm error.

Consequence: The Knowledge Artifact is the provably optimal \(K\)-parameter representation of the model's knowledge.

4.3 The Formal Algebraic Structure

Theorem 3 (Hilbert Space of Knowledge). Let \(V\) be a shared eigenbasis of rank \(K\). The space of Knowledge Artifacts \(\mathscr{K}(V) = \{(V, \mathbf{A}) : \mathbf{A} \in \mathbb{R}^K\}\) with:

• \(\oplus\) = vector addition
• \(\langle \cdot, \cdot \rangle\) = Euclidean inner product
• \(\|\cdot\|\) = Euclidean norm

is a \(K\)-dimensional real Hilbert space. We verified all 15 axioms computationally:

Vector space (9/9): closure, commutativity (\(\mathbf{A} \oplus \mathbf{B} = \mathbf{B} \oplus \mathbf{A}\)), associativity, zero element (the model that knows nothing), inverse (\(\mathbf{A} \oplus (-\mathbf{A}) = \mathbf{0}\): knowledge cancels itself), scalar compatibility, both distributivity laws, identity scalar.

Inner product (6/6): symmetry (\(\langle \mathbf{A}, \mathbf{B} \rangle = \langle \mathbf{B}, \mathbf{A} \rangle\)), linearity, positive definiteness, Cauchy-Schwarz (\(|\langle \mathbf{A}, \mathbf{B} \rangle| \leq \|\mathbf{A}\| \cdot \|\mathbf{B}\|\)), triangle inequality, Parseval (\(\|\mathbf{A} \oplus \mathbf{B}\|^2 = \|\mathbf{A}\|^2 + \|\mathbf{B}\|^2 + 2\langle \mathbf{A}, \mathbf{B} \rangle\)).

Consequence: All of Hilbert space theory applies to Knowledge Artifacts: orthogonal projections, best approximation, spectral decomposition, Riesz representation. These are not new theorems to prove — they are inherited for free.

4.4 Extended Operations

Beyond addition and subtraction, the Hilbert space structure enables six additional operations with direct knowledge interpretations:

Knowledge Projection \(P_S(\mathbf{A})\): keep only modes in set \(S\), zero out the rest. This is an orthogonal projection — idempotent (\(P(P(\mathbf{A})) = P(\mathbf{A})\)) and self-adjoint. Use: "show me only the credit risk modes" or "remove all modes above \(K^*\)."

Knowledge Cosine Similarity \(\cos(\mathbf{A}_1, \mathbf{A}_2) = \langle \mathbf{A}_1, \mathbf{A}_2 \rangle / (\|\mathbf{A}_1\| \cdot \|\mathbf{A}_2\|)\): angle between knowledge vectors, independent of scale. Use: "do these two models learn the same patterns, regardless of magnitude?" In experiments: \(\cos(\text{A1}, \text{A2}) = 0.008\) (unrelated functions), \(\cos(\text{A1}, \text{A3}) = -0.28\) (shared feature, opposite sign).

Knowledge Complement \(\mathbf{A} \setminus \mathbf{B} = \mathbf{A} - \text{proj}_\mathbf{B}(\mathbf{A})\): the part of \(\mathbf{A}\)'s knowledge that is orthogonal to \(\mathbf{B}\). Verified: \(\langle \mathbf{A} \setminus \mathbf{B}, \mathbf{B} \rangle = 0\) exactly. Use: "what does Bank A know that Bank B doesn't?" — the answer is the complement vector, with per-mode values.

Knowledge Intersection \(\mathbf{A} \cap \mathbf{B} = \text{proj}_\mathbf{B}(\mathbf{A})\): the shared knowledge between two models. Use: "what do Bank A and Bank C agree on?" The intersection identifies the common signal.

Knowledge Rank \(\text{rank}(\mathbf{A}) = |\{k : |A_k| > \tau \cdot \max|A_k|\}|\): the effective number of active modes. Use: a single integer measuring model complexity — how many patterns did the model actually learn?

Knowledge Agreement Decomposition \(\mathbf{A} \oplus \mathbf{B} = \mathbf{A}_{agree} + \mathbf{A}_{disagree}\): decompose the sum into modes where both models point the same direction (agree) and modes where they conflict (disagree). In experiments: agreement ratio = 0.35 for unrelated models, approaching 1.0 for models trained on similar data.

4.5 Orthogonal Independence

Theorem 4. If \(\mathbf{A}_1\) and \(\mathbf{A}_2\) have non-overlapping support (active on different modes), then \(\oplus\) preserves both exactly.

Consequence: Continual learning without forgetting is guaranteed for mode-separated tasks.

4.6 USRT Complexity Bound [Lean 4 verified]

Theorem 5. \(K^* = \Theta(\log(n/\sigma^2)/\log\rho)\) modes suffice, independent of input dimension.

Consequence: Knowledge Artifacts are always low-dimensional, regardless of the original model's size.

4.7 Knowledge Conservation Law

Theorem 6 (Conservation of Knowledge). For two Knowledge Artifacts \(\mathcal{K}_1, \mathcal{K}_2\) with orthogonal support:

\[\|\mathbf{A}_1 \oplus \mathbf{A}_2\|^2 = \|\mathbf{A}_1\|^2 + \|\mathbf{A}_2\|^2\]

Proof. Parseval's identity. \(\square\)

The "energy" of knowledge — \(\|\mathbf{A}\|^2\) — is conserved under composition. When models overlap, the cross-term \(2\langle \mathbf{A}_1, \mathbf{A}_2 \rangle\) is the knowledge agreement: positive = reinforcement, negative = contradiction.

4.8 Knowledge Entropy

Definition 4 (Knowledge Entropy).

\[H(\mathcal{K}) = -\sum_{k=1}^{K} \frac{A_k^2}{\|\mathbf{A}\|^2} \log \frac{A_k^2}{\|\mathbf{A}\|^2}\]

\(H(\mathcal{K})\)	Meaning	Example
Low (\(\approx 0\))	Knowledge concentrated in 1–2 modes	Linear regression
Medium	Spread across several modes	Moderate nonlinearity
High (\(\approx \log K\))	Uniformly spread	Highly complex model

A model with \(H \gg \log K^*\) is overfitting in the entropy sense.

---

5. The SpectralShell

The SpectralShell is a universal model class that loads and operates on Knowledge Artifacts:

`python class SpectralShell: def load(path) -> SpectralShell # load artifact def save(path) # serialize def predict(X) -> y # one matrix multiply def __add__(other) -> SpectralShell # combine knowledge def __sub__(other) -> SpectralShell # subtract knowledge def __rmul__(scalar) -> SpectralShell # scale def distance(other) -> float # compare `

Prediction in any language:

Language	Code
Python	y = (X @ V) @ coefficients
R	y = X %% V %% coefficients
Julia	y = X V coefficients
Rust	y = x.dot(&v).dot(&coefficients)
SQL	SELECT SUM(x_i v_ik c_k) ...
JavaScript	y = matmul(matmul(X, V), coefficients)

One matrix multiplication. No training framework. No dependencies beyond linear algebra.

5.1 Portability Requirements

Requirement	Needed?	Solution
Same features (position + order)	Yes	Store feature names in metadata
Same scale	Yes	Store scaler (mean, std) in artifact
Same number of features	Yes	Eigenbasis dimension must match
Same model type	No	Artifact is model-agnostic
Same architecture	No	Function space, not weight space
Training framework	No	Pure linear algebra
Training data	No	Only eigenbasis and coefficients

5.2 The Artifact File Format

` knowledge_artifact.npz: V : p × K eigenbasis (the coordinate system) coefficients : K the knowledge (shrunk spectral coefficients) feature_names: p human-readable feature identifiers scaler_mean : p standardization mean per feature scaler_std : p standardization std per feature noise_var : 1 estimated noise variance `

Typical size: 1.8 KB (8 features, 8 modes) to 700 KB (300 features, 300 modes).

---

6. Applications

6.1 Model Governance (Basel III, SR 11-7, AI Act)

Today: Manual documentation, 2–6 months validation per model change. With Knowledge Algebra: \(\Delta\mathcal{K} = \mathcal{K}_{v2} \ominus \mathcal{K}_{v1}\) in seconds. Per-mode audit trail. Certified structural diff.

6.2 Fairness (EU AI Act)

Today: Retrain with fairness constraints, validate, repeat. Months. With Knowledge Algebra: \(\mathcal{K}_{fair} = \mathcal{K}_{model} \ominus \mathcal{K}_{bias}\). One subtraction. 10x bias reduction. Same day.

6.3 Healthcare (GDPR, HIPAA)

Today: Complex federated learning protocols (FedAvg, secure aggregation). With Knowledge Algebra: Each site computes \(\mathcal{K}_{site}\) locally. Central server averages \(K\) numbers. No patient data moves. 21% improvement.

6.4 Competitive Intelligence

Today: Reverse engineering from API calls → train surrogate model. Expensive, approximate. With Knowledge Algebra: Call competitor API 1000 times → extract \(\mathcal{K}_{competitor}\) → compare \(d(\mathcal{K}_{ours}, \mathcal{K}_{competitor})\) → know exactly what they know that you don't.

6.5 Model Marketplace

Knowledge Artifacts enable a model marketplace: vendors publish artifacts, buyers load them into SpectralShells. Artifacts are composable — buy credit knowledge from vendor A, market knowledge from vendor B, add them. No integration work. No architecture compatibility. Just vector addition.

---

7. Extended Results

7.1 Real-World Benchmarks

On the Diabetes dataset (442 real patients, 10 clinical features):

Method	Original RMSE	Distilled RMSE	Better?
RF (200 trees)	54.46	50.47	Yes (−7%)
GBM (200 trees)	54.63	50.60	Yes (−7%)
MLP (128-64)	52.66	52.36	Yes
SVR (RBF)	51.02	51.95	No
Spectral RBF (direct)	—	51.00	Competitive

On large regression (5,000 samples, 20 features, 10 informative):

Method	Original RMSE	Distilled RMSE
RF (200 trees)	59.30	30.13 (−49%)
GBM (200 trees)	31.23	23.99 (−23%)
MLP (256-128)	28.29	24.33 (−14%)

The distilled models are consistently better because spectral shrinkage removes noise modes the originals overfit.

7.2 AI Safety: Targeted Capability Removal

A model that knows useful biology AND dangerous synthesis capabilities. Subtract the dangerous Knowledge Artifact:

Metric	Before removal	After removal
Correlation with danger	0.637	0.017 (38x reduction)
Correlation with useful	0.897	0.974 (preserved)
Useful biology RMSE	4.417	1.000 (4x better)

Mode-level analysis reveals exactly which modes carried dangerous knowledge. Unlike RLHF (which suppresses but doesn't delete), spectral subtraction mathematically removes the capability. It cannot be recovered by prompt engineering.

7.3 Model Knowledge Fits in a Tweet

A 200-tree GBM (65,000+ parameters) distilled to 3 spectral coefficients:

` -381.51 -75.88 128.19 `

21 characters. Anyone can copy-paste these numbers, multiply with the eigenbasis, and predict. No Python, no sklearn, no framework. The entire model's knowledge in a tweet.

{ width=100% }

7.4 Function-Space Arithmetic Is 19x Better Than Weight-Space

Two neural networks (64-32 architecture) trained on complementary tasks:

Addition method	RMSE
Weight-space (w_A + w_B, Task Arithmetic style)	16.782
Function-space (A_A + A_B, Knowledge Algebra)	0.874
Direct NN trained on combined task	0.498

Function-space is 19.2x better. The reason is mathematical: \(\text{ReLU}(w_A \cdot x + w_B \cdot x) \neq \text{ReLU}(w_A \cdot x) + \text{ReLU}(w_B \cdot x)\), but \(A_f + A_g = A_{f+g}\) (Theorem 1, superposition). Weight-space addition is meaningful only if the bases are aligned — which they generically are not (permutation symmetry of neurons).

Theorem 8 (Suboptimality of Weight-Space Arithmetic). For any two functions \(f, g\) with spectral representations \(\mathbf{A}_f, \mathbf{A}_g\) in shared eigenbasis \(V\) of rank \(K\), and weight representations \(w_f, w_g\) in any other basis \(W\) of the same rank:

\[\|f+g - V(\mathbf{A}_f + \mathbf{A}_g)\|^2 \leq \|f+g - W(w_f + w_g)\|^2\]

Equality holds iff \(W = V\) (the weight basis is the eigenbasis). In the generic case, weight-space arithmetic is strictly worse.

Proof sketch. The left side equals the residual after projecting \(f+g\) onto the top-\(K\) eigenspace of the data kernel. By Eckart-Young, this is the minimum-error rank-\(K\) approximation. Any other rank-\(K\) basis \(W\) (including neural network weight bases) has equal or greater residual. Strict inequality holds when \(W \neq V\), which is generic for randomly initialized neural networks. \(\square\)

Experimental confirmation: Function-space RMSE = 0.874, weight-space RMSE = 16.782. Ratio: 19.2x. This is not an edge case — it is the generic behavior whenever weight bases are misaligned.

{ width=70% }

---

8. Certified Capability Removal: The Safety Argument

The Knowledge Algebra provides something no current alignment method does: mathematically certified deletion of specific capabilities.

8.1 RLHF Suppresses. Spectral Subtraction Deletes.

	RLHF / Fine-tuning	Spectral Subtraction
Mechanism	Train model to refuse	Remove coefficients from artifact
Dangerous knowledge	Still in weights, suppressed by policy	Absent from representation
Recoverable by adversary?	Yes (jailbreaks, prompt injection)	No — coefficients are zero
Certifiable?	No formal guarantee	Yes — correlation provably reduced
Preserves useful knowledge?	Approximately	Yes (0.974 correlation preserved)

When you subtract a dangerous Knowledge Artifact, the dangerous modes have coefficient zero in the result. Not "small." Not "suppressed." Zero. There is no prompt that recovers information that isn't in the representation.

8.2 Mode-Level Forensics

coefficient

This provides an audit trail: "the model's dangerous capability resided in 12 specific eigenmodes. These modes have been removed. The remaining 3 modes contain only useful knowledge."

{ width=100% }

8.3 Limitations of the Safety Argument

The guarantee is exact within the eigenbasis. If the dangerous capability exists in directions not captured by the top-\(K\) eigenmodes, subtraction is incomplete. The completeness depends on \(K\) (more modes = more thorough removal) and on the kernel (RBF captures more nonlinear structure than linear). For production safety-critical applications, verification on held-out dangerous prompts remains necessary — spectral subtraction provides the mathematical deletion, but empirical verification confirms the eigenspace was comprehensive enough.

---

9. Extension to Classification

Knowledge Artifacts are not limited to regression. Any classifier with a predict_proba method is decomposable via the log-odds transform.

9.1 Log-Odds: The Natural Space for Classification Algebra

A classifier's output is a probability \(P(y=1|x) \in [0,1]\). In this space, addition produces invalid results (\(P_1 + P_2 > 1\)). The solution: work in log-odds space:

\[\ell(x) = \log\frac{P(y=1|x)}{1 - P(y=1|x)} \in (-\infty, +\infty)\]

In log-odds space, addition is Bayesian evidence combination: if two models both believe \(y=1\), their combined log-odds is stronger. Subtraction is evidence removal. Averaging is ensemble consensus. All results remain valid probabilities after the inverse transform \(P = 1/(1+e^{-\ell})\).

Operation	Probability space	Log-odds space
Combine two models	Invalid (\(P_1+P_2 > 1\))	\(\ell_1 + \ell_2\) (stronger belief)
Ensemble	\(\frac{P_1+P_2}{2}\) (ad hoc)	\(\frac{\ell_1+\ell_2}{2}\) (principled)
Remove bias	\(P - P_{bias}\) (can be \(< 0\))	\(\ell - \ell_{bias}\) (always valid)

For multiclass: each class gets its own log-odds artifact. Prediction: apply sigmoid per class, normalize, argmax.

9.2 Classification Results

We validate on 5 classification datasets (no cherry-picking):

Dataset	Teacher acc.	Distilled acc.	Retention	Notes
Breast Cancer (binary, n=569)	0.956	0.965	101%	Better (noise removed)
Digits (10 class, n=1797, p=64)	0.972	0.981	101%	Better
Iris (3 class, n=150)	1.000	0.900	90%	Small dataset
Wine (3 class, n=178)	0.944	1.000	106%	Better
Synthetic binary (n=1000)	0.935	0.935	100%	Exact match

Classifier ensemble via log-odds averaging:

Dataset	GBM	RF	Ensemble (log-odds avg)
Digits	0.981	0.956	0.983 (best)
Breast Cancer	0.965	0.965	0.965

The multiclass ensemble on Digits (10 classes, 64 features) achieves 0.983 accuracy by averaging log-odds from GBM and Random Forest shells — higher than either alone.

9.3 Unified API

`python

Regression (unchanged)

art = KnowledgeArtifact.extract(model, X)

Classification (auto log-odds)

shell = SpectralShell.from_classifier(model, X, **config) labels = shell.predict_label(X_new) proba = shell.predict_proba(X_new)

Algebra works in BOTH

ensemble = 0.5 shell_a + 0.5 shell_b distance = shell_a.distance(shell_b) `

---

10. The Expressiveness-Algebraizability Tradeoff (Regression vs Classification)

There is a fundamental tension in knowledge representation:

\[\text{Expressiveness} \longleftrightarrow \text{Algebraizability}\]

Neural networks are maximally expressive (universal approximators) but their weight space does not support exact arithmetic. Knowledge Artifacts are algebraically exact but limited to functions expressible in the kernel eigenbasis.

	Neural Network	Knowledge Artifact
Smooth functions	Good	Optimal (exponential convergence, Eckart-Young)
Non-smooth (ReLU kinks)	Better	Slower (Gibbs phenomenon)
Discontinuous (regime shifts)	Better	Poor (oscillation)
Algebraizable?	No (permutation symmetry, nonlinear activation)	Yes (orthogonal basis, linear projection)

For smooth functions — which describe most real-world phenomena — Knowledge Artifacts sacrifice nothing. Eckart-Young guarantees that no \(K\)-parameter representation (including a \(K\)-parameter neural network) can be more accurate. The tradeoff only matters for functions with sharp discontinuities, where the spectral basis converges slowly.

The practical resolution: use neural networks for prediction where discontinuities matter. Use Knowledge Artifacts for everything else — composition, comparison, transfer, auditing, and safety. Or: train a neural network, distill into an artifact, and gain algebraizability with minimal accuracy loss on smooth components.

---

11. ML Without a Framework: Knowledge Artifacts in Practice

10.1 Prediction in Excel

A Knowledge Artifact with 3 modes and 5 features can be loaded into a spreadsheet:

	V₁	V₂	V₃	A
income	0.52	-0.31	0.12
debt	-0.48	0.27	0.65
age	0.35	0.71	-0.22
credit	0.41	-0.18	-0.53
late_pay	0.46	0.54	0.49
coeff				+3.2, -1.8, +0.5

Prediction for a new customer: ` =SUMPRODUCT(B2:B6, customer_data) * 3.2 + SUMPRODUCT(C2:C6, customer_data) * (-1.8) + SUMPRODUCT(D2:D6, customer_data) * 0.5 `

Three SUMPRODUCT formulas. No Python. No sklearn. No API. A data analyst with Excel can deploy a model that was originally a 200-tree Random Forest.

10.2 The 5-Minute Workflow

`python

Step 1: Extract (once)

from spectral_learner import SpectralLearner, SpectralShell sl = SpectralLearner.from_data(X_train, y_train, kernel='rbf') shell = SpectralShell.from_learner(sl) shell.save("credit_model.npz") # 1.8 KB

Step 2: Deploy (anywhere, forever)

shell = SpectralShell.load("credit_model.npz") y = shell.predict(X_new) # one matrix multiply

Step 3: Combine (instant)

market = SpectralShell.load("market_model.npz") combined = shell + market # operator overloading y_both = combined.predict(X_new)

Step 4: Audit (no holdout data)

print(shell.distance(market)) # how different? `

Four steps. One file. Any model in, predictions out. The original model can be deleted.

10.3 Manufacturing Use Case (Federated)

Five factories produce the same product. Each has quality data (100 samples) but cannot share it (trade secrets, regulations).

Today: Each factory builds its own model. Quality: limited by 100 samples.

With Knowledge Artifacts: Each factory extracts \(\mathcal{K}_{factory}\) locally. A central server receives 5 coefficient vectors (\(K\) numbers each). Computes \(\mathcal{K}_{global} = \frac{1}{5}\sum \mathcal{K}_i\). Sends the global artifact back. Each factory now has a model informed by 500 samples worth of knowledge — without seeing any other factory's data.

Total data transferred: \(5 \times K\) floating-point numbers. Total data shared: zero.

---

12. Limitations

1. 1. Shared eigenbasis: All artifacts in an operation must share the same \(V\). Different data distributions → different eigenbases → incomparable artifacts. Mitigation: standardize data, use a canonical reference dataset.
1. 2. Linear kernel captures linear functions only: For nonlinear models, use RBF kernel eigendecomposition. This captures interactions but increases artifact size (\(n\) reference points needed for Nyström extension).
1. 3. \(O(n^2)\) for kernel methods: Full kernel eigendecomposition is expensive for \(n > 10{,}000\). Nyström or random Fourier features provide scalable alternatives.
1. 4. Smooth functions favored: Spectral methods converge exponentially for smooth functions but slowly for discontinuous ones (Gibbs phenomenon). Tree-based models with sharp splits are harder to capture.
1. 5. Feature space must match: If the feature space changes (new features, different encoding), the artifact is invalid. Feature engineering must be frozen before extraction.
1. 6. Spectral decay rate estimation is an open problem. The decay rate \(\rho\) simultaneously determines two things: (a) the optimal number of modes \(K^* = \Theta(\log(n/\sigma^2)/\log\rho)\) for the Knowledge Artifact, and (b) whether spectral or tree-based methods will perform better on a given problem. A better \(\rho\) estimate improves both — they are the same problem, not two.

We tested four estimation approaches on 8 benchmark problems (Diabetes, Friedman #1–#2, step functions, piecewise linear, polynomial, periodic, XOR):

Approach	Accuracy	Weakness
Eigenvalue decay of \(X\)	75%	Measures data structure, not function smoothness
Coefficient decay of \(A_k\)	50%	RBF kernel smooths non-smooth functions
Residual structure + CV shape	50–62%	Too complex, unreliable
Direct competition (spectral vs tree on training data)	75%	Best available; limited by small sample (8 problems)

The direct competition approach compares spectral and tree fits of equal complexity on the training data. When the spectral/tree training RMSE ratio \(< 0.5\) (spectral fits at least 2\(\times\) better), spectral methods win on test data 75% of the time. The remaining errors are informative: on Diabetes, spectral fits training worse but generalizes better (Stein's shrinkage paradox); on Friedman #1, spectral and GBM are within 1.6% on test (effectively tied).

Developing a robust smoothness diagnostic — ideally one that accounts for the generalization advantage of shrinkage, not just training fit — is the most important open question for this framework.

---

13. Conclusion

We defined two new concepts for machine learning:

The Knowledge Artifact is what a model knows, extracted into a portable spectral coefficient vector. It is model-agnostic (works for any model with predict), provably optimal (Eckart-Young), and self-contained (no framework needed for prediction).

The Knowledge Algebra is what you can do with knowledge: combine, extract, debias, compare, diff, transfer, federate, and extend — all via exact vector arithmetic. No retraining. No architectural compatibility. No data sharing.

The SpectralShell is the universal model: load any artifact, predict with one matrix multiplication, in any programming language.

Together, these three concepts transform machine learning knowledge from an opaque, architecture-specific, inoperable byproduct of training into a portable, composable, arithmetically rich first-class object.

The practical implications are immediate: bias removal in one subtraction (vs. months of retraining), federated learning by sharing \(K\) numbers (vs. complex distributed protocols), model governance via spectral diff (vs. manual documentation), and model composition by vector addition (vs. ensemble infrastructure).

Knowledge = a shared basis + a private vector. Add the vectors, share the basis, predict anywhere.

---

---

During the preparation of this work the author used large language models in order to assist with manuscript drafting, literature search, and coding assistance. After using these tools, the author reviewed and edited the content as needed and takes full responsibility for the content of the published article.

---

References

<!-- All entries sourced from BIBLIOGRAPHY.yaml via nous papers bib format` -->

• Eckart, C. and Young, G (1936). "The Approximation of One Matrix by Another of Lower Rank." Psychometrika, 1(3), 211–218. Psychometrika, 1(3), 211-218. DOI: 10.1007/bf02288367
• Hinton, G., Vinyals, O., and Dean, J (2015). Distilling the knowledge in a neural network. NeurIPS Workshop on Deep Learning..
• Ilharco, G., Ribeiro, M.T., Wortsman, M., Gururangan, S., Schmidt, L., Hajishirzi, H., and Farhadi, A (2023). "Editing models with task arithmetic." ICLR. ICLR*.
• Li, C., et al (2025). "When shared knowledge hurts: Spectral over-accumulation in model merging." arXiv:2602.05536. arXiv:2602.05536*.
• Mikolov, T., Chen, K., Corrado, G., and Dean, J (2013). "Efficient estimation of word representations in vector space." ICLR Workshop. ICLR Workshop*.
• Lundberg, S.M. and Lee, S.-I (2017). "A unified approach to interpreting model predictions." NeurIPS. NeurIPS*.
• Mitchell, M., et al (2019). "Model cards for model reporting." FAccT. FAccT*. DOI: 10.1145/3287560.3287596
• Nagy, T. (2026). The Fenton Distribution Solved. Working paper.
• Nagy, T. (2026). The Spectral Tensor Representation of Stochastic Processes. Working paper.
• Ortiz-Jimenez, G., Favero, A., and Frossard, P (2023). "Task arithmetic in the tangent space: Improved editing of pre-trained models." NeurIPS (Oral). NeurIPS. DOI: 10.52202/075280-2913
• Stein, C (1961). "Inadmissibility of the usual estimator for the mean of a multivariate normal distribution." Proc. 3rd Berkeley Symposium, 1, 197–206. Proc. 3rd Berkeley Symposium, 197-206. DOI: 10.1525/9780520313880-018
• Wortsman, M., Ilharco, G., Gadre, S.Y., Roelofs, R., Gontijo-Lopes, R., Morcos, A.S., Namkoong, H., Farhadi, A., Carber, Y., Kornblith, S., and Schmidt, L (2022). "Model soups: Averaging weights of multiple fine-tuned models improves accuracy without increasing inference time." ICML. ICML*.