The model was the test subject; the method was the result. This specimen produced almost nothing shippable (no capability win worth a GGUF), but the instrument was built here, and one empirical regularity, the Σα-law, held up across every later model. Read it for the ruler, not the result.
§0The patient
MiniCPM5-1B is the plainest model
in the lab. Unlike its sibling MiniCPM-V 4.6 (a Qwen3.5 rebase), it's an original release, and the
config says it all: the muP machinery is gone, and it genuinely declares LlamaForCausalLM:
24 dense GQA layers, untied embeddings. Everything downstream is textbook-Llama behaviour, including
the residual-norm pathology it wears so openly.
It is also the model where the engine changed. Every earlier specimen was cut with llama.cpp on a static GGUF: you can duplicate or delete whole blocks, but you can't turn a layer's contribution down to 60%. The interesting surgery on MiniCPM5 is continuous, so the whole sweep moved into a live MLX process where the model's own Python objects get patched between forward passes. That sounds like a footnote, but it's the reason this page exists.
↳ the massive activation is created by layer 0 — the embedding is diffuse; the spike is born, not carried
§1The lever — continuous damping, not duplication
Duplication and pruning are discrete (a layer is either in the stack or out). Damping makes layer surgery continuous: you can turn a layer's contribution down to 60% instead of all-or-nothing. That's what needed the live MLX engine, and it's what the rest of this page is about. The surgery primitive is one line:
# ScaledLayer.__call__
out = x + α · (block(x) − x) = x + α·Δ
Δ is the layer's residual contribution. α = 1 is the untouched model.
0 < α < 1 damps the layer: it still computes, but writes a fraction of its update into the
stream. α = 0 is a clean skip (identity). α > 1 over-amplifies. And
α < 0 is an anti-update (push the stream the opposite way the layer wanted), which is
not a functional inverse, because the block is nonlinear; it's a controlled way to ask "what does fighting
this layer do?"
Configs are composed in original-layer-index space, applied in the order prune → blocks → dups → damp.
A single layer can be duplicated k times, each copy given its own α, and the original pass damped
too. That composition is the entire experimental vocabulary: a split-pass duplicates L and damps both the
original and the copy; a stack is k copies with a per-copy α profile; everything else (ramps,
triples, multi-site chains) is the same trick with more knobs. Nothing here is trainable — it's all
inference-time. The question every config asks is the lab's usual one: did capability move, or did the grader
just get fooled?
§2Building the ruler before trusting it
The first real finding had nothing to do with surgery. The base model has a cliff. On the original wide math set, ~50% of items floored to 0.0 and ~24% saturated to 1.0, leaving only ~8% in a band where a score could move. Baseline 0.33, no resolution. A sweep run on that ruler measures mostly nothing, precisely.
So the ruler got calibrated. A difficulty ladder (16 items per rung, both operands controlled, 8 seeds,
baseline-only) mapped where each operation breaks. Rungs are operand digit-counts: d5 means
both operands run 5 digits; mul/div shapes like 3×2 mean 3-digit × 2-digit. The packed in-band set (load_math_inband, 48
items) all sits at p ∈ 0.28–0.72, baseline ≈ 0.48: headroom in both directions, so a config can win
or lose visibly.
| op | easy → hard (strict rate) | usable band |
|---|---|---|
| add | d2 1.00 → d5 0.48 → d7 0.22 | smooth; bullseye at d5 |
| sub | d2 0.89 → d3 0.60 → d4 0.30 → d5 0.17 | controlled-borrow, near-monotone |
| mul | 2×2 0.80 → 3×2 0.49 → 3×3 0.04 | razor-thin; only 3×2 usable |
| div | 1×2 0.98 → 2×2 0.72 → 2×3 0.28 | the rich, controllable channel |
↳ a shaded band marks the 0.3–0.7 region where a score can actually move — most of the wide set sat outside it
add_d5, mul_3×2) marked. Operation by
marker shape, not colour.Two methodology lessons fell out of building this, and they generalise to every model in the lab.
(1) n=4 items per rung is a lottery. A famous sub_d4 = 0.03 "dip" turned out to be an artifact
of 4-item sampling plus a generator bug (only the minuend's digits were controlled, so each rung was a
difficulty grab-bag). Control all operands; ≥12 items per rung; the dip vanished. (2) The cell cache had a
contamination bug. The harness keyed result cells on (config_id, probe_name, seed) with no content
check — change a probeset while keeping the names (items-per-rung 4→16 shifts the RNG stream, so
cal_sub_d4_i00 is now a different problem) and it silently resumed over stale cells. The fix:
content-hash the battery into the results filename, so any change auto-forks a fresh file. Neither is glamorous;
both would have quietly poisoned every number after them.
§3Split-pass — damping both copies is a real mechanism
The first thing that actually moved the needle: take a layer, duplicate it, and damp
both the original pass and the copy. On L1 and L10 specifically (not the "refine zone" L5–12 we'd
guessed), this buys +0.11–0.12 strict math, confirmed at 24 seeds, t ≈ +6.
split_1_a50_a70 = 0.53 strict, t +6.2.
Don't apply the layer's contribution in one go. Apply a scaled-back version, let the residual stream move, then run the layer again on that moved stream and apply another scaled-back version — two gentle nudges instead of one shove. What makes this its own mechanism is that on the second nudge the layer re-evaluates from the stream's new position, so it contributes a fresh opinion, not a scaled copy of its first.
The arithmetic, precisely — why split ≠ a stronger single pass
α is the per-pass damping and Δ(x) = block(x) − x the layer's residual contribution. A single damped pass
computes the update once, at the original input x. A split pass computes it twice, and
the second time the block sees the already-moved stream:
# single: x + α·Δ(x) # split: x₁ = x + α₀·Δ(x) ← first nudge, evaluated at x x₂ = x₁ + α₁·Δ(x₁) ← second nudge, re-evaluated at x₁
Because the block is nonlinear, Δ(x₁) ≠ Δ(x). So x + α₀·Δ(x) + α₁·Δ(x₁) is not the vector
x + (α₀+α₁)·Δ(x) — which is all a single pass at strength α₀+α₁ could produce. One Euler step versus
two half-steps: the second corrects for where the first landed.
↳ the layer rotates its opinion 42° between sub-steps — a fresh read, not a scaled copy
single − split = (Σα/k)·Σᵢ(Δ(x) − Δ(xᵢ)), which held numerically to 1.6% — the divergence is
provably nothing but the layer's change of opinion.And it isn't only those three controls. We swept a single damped pass of L1 across the whole single-knob range (α = 0.5 → 1.5, the entire envelope a single α can reach). It tops out at +0.028 (α ≈ 0.6) and craters under amplification (α = 1.5 → −0.115); it never clears baseline by more than noise. The best split/stack sits at +0.153. No single α comes close: a +0.125 gap, ~5×. And at the same total dose Σα = 1.5, one pass scores −0.115 and the split scores +0.153, a 0.27 swing that is pure re-evaluation. (Cross-checked across two independent sweep files: L1 at α = 1.30 gives Δ −0.080 in both to four decimals, so the comparison rides on Δ-vs-baseline, where the constant harness offset cancels.)
observation a single α on L1 buys at most +0.03 and is actively harmful when amplified; the split at identical Σα buys +0.15. interpretation a single α can only slide the residual along the Δ(x) line, and that line never gets above baseline — the gain lives off-axis, where only the re-evaluating split can reach. The split isn't a tidier route to a place single-α could reach; it reaches a place single-α can't. This is the behavioural face of the geometry in Fig 1.
↳ same knob budget, opposite outcomes — the split stays above the single at every dose
§4The Σα-law
Take layer L, give it k passes with a profile of alphas, and define Σα = the sum of all
per-pass alphas on that layer.
The claim: Σα is the governing variable — not the copy count, not the ramp shape, not which particular pass carries which α. The cumulative alpha sets the score.
A caveat on the word "law." This is an empirical regularity, not a theorem: strong enough to name, not proven universal. It earns the name here: the pooled regression (§6) fits it at R² = 0.74, and the same Σα-governance recurs on Granite. But it's one layer of one 1B model plus a cross-model echo; the crest drifts by channel; nothing here derives it from first principles. That's the bar this lab holds. Σα graduated to a named law on the strength of the regression and the transfer, where Granite's weaker combo-stacking pattern stayed a "depth-separation heuristic" precisely because it hadn't.
| config (L1 stack-ramp) | profile | Σα | math |
|---|---|---|---|
| stack_1_k2…k8 (conserve) | [1/k] × k | 1.0 | 0.52–0.55 — flat across k=2→8 |
| stack_1_k5decay | [1, .5, .25, .125, .06] | 1.94 | 0.44 (n.s.) — survives |
| stack_1_k8decay | [1, .5, …] | 1.99 | 0.44 (n.s.) — survives |
| stack_1_k5dec | [1, .8, .6, .4, .2] | 3.0 | 0.26 — cratering |
| stack_1_k5inc | [.2, .4, .6, .8, 1] | 3.0 | 0.16 — worse (back-loaded) |
| stack_1_k8dec / k8inc | linspace | 4.8 | 0.000 — dead |
Read down the Σα column and the score tracks it, not the shape. The conserve row (Σα = 1) is flat from k=2 to k=8 — once total update is fixed, adding copies does nothing. A fine scan then sharpened it: the peak isn't at Σα = 1 (just the lowest densely-sampled cell). On L1 it sits at Σα ≈ 1.25, on a broad 1.1–1.5 ridge, fading by ~2.0, cratering by ~3, and reaching full residual-norm collapse around Σα ≈ 4.8.
↳ the signature plot — k spreads the curve vertically (gain), it does not move the peak horizontally
Honesty about the peak. "Σα ≈ 1.25" is the L1 finescan answer, but it's not clean across the board. A
separate distributed sweep found Σ1.50 k6 stacks beating the Σ1.25 band at the very top
(champion shp1_S150_k6_flat, t +7.33), and the pooled regression put
the math optimum nearer Σα ≈ 1.65 versus ~1.5 for the balanced battery. So the truthful statement is:
a broad peak in the ~1.25–1.65 region, drifting higher for the narrow math channel, not a single magic
constant. The ridge is real; its exact crest is channel-dependent. That tension is in the data and stays in the
writeup.
§5One death, two doors
Over-drive (Σα ≫ 1) and negative-α reversal converge on the same failure. Grade the verbatim generations
along the Σα climb and you watch the model die the same way each time: at Σα = 1 clean; at Σα ≈ 2
coherent but with arithmetic slips plus de-narration derail (the model wandering into MMLU-style narration); at
Σα = 3 wounded (operand echo, CJK script leaking); at Σα ≈ 4.8 dead: 500000…0
digit-repetition, special tokens, random scripts.
observation a divergence sandwich diverge_567 (+.7 / −.5 / +.8 on L5/6/7)
collapses to 0.00, while the control (+.7 / +.5 / +.8) holds at 0.27. interpretation it's the
negation, not the perturbation, that kills it: the bracketing positive layers do not absorb
a reversed middle layer. The verbatim of the collapsed run shows the same 0.0000…0 norm-collapse
signature as over-drive: the two doors open onto the same room.
The mechanistic reading, consistent with Fig 1: L1's job is keeping the residual inside the norm band the downstream layernorms expect. You can break that from either side: pour too much in (over-drive) or push it backwards (negation). Either way the stream walks out of the band, the late layernorms see a vector they were never trained on, and the output head spills digits, special tokens, and whatever script is nearest in embedding space. L1 is simultaneously the most split-helpful layer and the most negation-fragile one (at α = −0.1, L1 → 0.03 vs L5 → 0.12). The leverage cuts both ways because it's the same leverage.
§6The degeneration rate — exposure, not capability
This is the result I'm fondest of, and it cost no compute: a metric computed over generations that were already stored. Different probe channels disagreed about whether a config was good: math liked high-k; the emotional-intelligence (eq) channel hated it. observation diffing the verbatim responses showed the disagreement is about length: math answers are ~16 tokens, eq answers ~96. interpretation the explanation is not "math and eq need different surgery"; it's exposure: a low per-token derailment probability is invisible in 16 tokens and compounds into a visible loss across 96.
So define a degeneration rate: the fraction of stored responses that derail (special-token leak + name:number line-loops, the same crater signature, appearing subclinically). It's monotone in both knobs: in k (0 → 5.0% from k4 → k12 at S150) and in Σα (0.8 → 11.7% from S150 → S250 at k6). Math sits near 0%: its short answers just don't expose the risk.
Take the eq channel, throw out only the derailed responses, and score the clean
ones (conditional mean): at k10/k12 the clean-only eq is ≥ baseline (0.478 / 0.475 vs 0.472).
The clean responses hold at or above baseline on every channel, including the one whose
aggregate score the surgery tanked. What the surgery costs isn't content quality; it's the probability of
derailing. The "eq champion" of the early board, dampall_a65, turns out to be a 14.6% derailment
machine; the genuine stable champion, shp1_S150_k6_flat, degenerates 0.4% of the time, more stable
than baseline's own 2.1%.
↳ throw out the derailed responses and the clean ones sit at/above baseline — the cost is risk, not quality
§7It's all one cost surface
The separate observations (flat-beats-ramp, k6-beats-k4 below peak, the dose hump) are not separate. Pool all 35 L1 finescan configs (flats, ramps, k-ladders) and fit:
math = β₀ + β₁·Σα + β₂·Σα² + β₃·Σα²(profile)
→ R² = 0.74, β₃ = −0.041 ± 0.009 (t −4.9)
The Σα² term gives the dose parabola (peak ≈ 1.65 for math); the profile-curvature term, Σa² summed over the per-pass alphas, is a penalty for concentration: spreading the same Σα across flatter, smaller passes costs less than dumping it into one big pass. "Flat beats ramp" and "k6 beats k4" are the same phenomenon: both lower Σa² at fixed Σα.
Where the regression fails is itself informative: the residual-norm wall is not smooth. A k1→k2 rescue at Σ1.5 was observed at +0.252 against a quadratic prediction of ~+0.05: a single concentrated pass breaks beyond what smooth curvature predicts. The model is approximately additive inside its trained envelope and catastrophically nonlinear at the norm walls: a regime-linear residual stream, polite in the middle, lethal at the edges. Per-layer the additivity has a spectrum: L0 is an extreme outlier (−0.19; it wants one concentrated write, corroborating its amplification-site identity), L1 (+0.035) and L23 (+0.049) are the only saturating layers (splitting the dose helps, exactly why L1 rewards high k, a suggestive bookend symmetry), and the mid-stack is near-additive.
§8What did it actually measure?
Mostly not new capability. The throughline of the whole MiniCPM5 board is that the gains are de-collapse /
compliance relief: surgery unclogs a model that was strangling its own output on a stalled valve, rather than
teaching it arithmetic it didn't know. The overall v3-battery champ, damp5_a80_damp23_a80 (L5 semantic
entry + L23 output gate, both α0.8, bal t +4.12), is an additivity bet that paid off, but it's relieving pathology,
not adding a skill. The multiple-choice "winners" (dampall_a64/65/66) are pure A-attractor de-collapse:
the model stops defaulting to answer A.
This is why the next fork was built to falsify the capability story: a chain-of-thought battery. If a
config's win survives when the model is actually reasoning step-by-step, it's capability; if it evaporates, it was
compliance relief. That fork is built but not fully run. The one suggestive smoke result is worth flagging as a
hypothesis, not a finding: observation dampall_a90 beat baseline in
CoT smoke (0.074 vs 0.046). interpretation if a 1B can't sustain chain-of-thought
because it collapses into loops, then de-collapse surgery might be a reasoning prosthesis: not a
capability test but a capability enabler. Unproven. The most interesting thread left on the table when the
model was closed out.
§9Methodology lessons (the durable part)
Read these before adding runs to any model, not just this one.
| # | Lesson |
|---|---|
| 1 | Content-hash the battery into the results filename. Cell caches keyed on (config, probe, seed) silently resume over stale cells when a probeset changes under a stable name. Auto-fork on content change. |
| 2 | Cap batch width explicitly (max_batch_rows). Short uniform prompts (math ~15 tok) balloon to ~500-wide under a token-based cap, past MLX's ~256-row decode knee → throughput collapse. The cap only bites the short channels. |
| 3 | ≥12 items per rung, all operands controlled. Small rungs are hostage to one derail-prone item; uncontrolled generators turn a difficulty rung into a grab-bag. |
| 4 | r is a std-dev ratio, invariant to N. Persistent seed-sensitivity (r = std(per-seed Δ) / std(baseline noise) > 1) is real effect-heterogeneity, not low power — more seeds sharpen the mean (t↑) but don't drop r. Stop chasing r < 1 on the math channel; report the means. |
| 5 | The de-narration derail is the irreducible floor. "Question: In the following…" appears in every config including healthy ones; surgery modulates how much, never eliminates it — the same collapse the project studies, subclinical. |
| 6 | Greedy n=1 fabricates false positives (carried from HRM, re-confirmed here). Always sample, always multi-seed. |
Caveats
Medium confidence, and here is what that means. The Σα-law itself graduated (R² = 0.74 on the
fit, plus a predicted transfer to Granite), but its peak is a channel-dependent ridge, not a constant:
quoting "Σα = 1.25" without the 1.25–1.65 range and the math-vs-balanced drift is over-claiming.
And there is no shippable model win here: the best math specialist (split_1_a50_a70) loses
overall to a de-collapse config, and even that is pathology relief. Read a method and a regularity into
this page, not a capability claim.
- The CoT prosthesis hypothesis is a smoke-test hunch (§8), not a result — flagged because it's the most promising unfinished thread, not because it's settled.
- eq is a null-rate gauge, not a ranker on the v4 battery (r > 1, low signal); use it for derailment exposure, not leaderboards.
MiniCPM5 is the continuous-damping branch of the line — the model where the engine moved from llama.cpp static GGUFs to live MLX in-process patching, which is what made everything above possible. The Σα-law born here transferred, with a predicted downshift, to Granite; the "I built the ruler before trusting it" lesson is shared across the lab. Engine: MLX in-process patching, mlx-zaya venv, mlx-lm 0.31.3. Surgery primitive ScaledLayer + apply_config in shared/mlx_sweep_harness.py; batteries probes_combined_v3 / probes_math_wide (load_math_inband) / probes_eq_bench_v4; all generations stored for regrade. Technique credit and the grader-gaming taxonomy live on the shared Methods & references page. Sibling specimens: Gemma 4, Granite, HRM. Run locally on a 16 GB Mac Mini.