Documentation  /  E1 — holographic composition

HiNet E1 — Holographic Composition: Design & Pre-Registration

Status: pre-registration (commit before running). This is the make-or-break experiment from HiNet-Foundational-Model-Plan.md §8. Hypotheses, metrics, and success margins below are fixed before any composition run so the test is honest. Code lives in ../experiments/e1-holographic-composition/.

Dataset IDs were HF-verified mid-2026. Facts flagged [v] = verified live on the hub this cycle.

1. The question E1 answers

Take a frozen shared anchor. Branch 3 domain-specialist persona experts (frozen-anchor LoRA). Compose them three ways and ask: does a weight-level composition beat the best single persona iCore on a held-out mixed-domain MCQ eval, while opt-out (dropping one expert) is exact?

This is the smallest possible test of the two-part success predicate: (A) the part is a usable LLM and (B) the whole is bigger (more capable than any single part), decomposably. If neither weight-level mode beats the best single iCore by the pre-registered margin, the holographic-assembly thesis is falsified at the smallest scale and we stop before any networking.

2. Fixed design

2.1 Anchor (frozen "shared DNA")

• Qwen/Qwen3-30B-A3B-Instruct-2507 — MoE, 30.5B total / ~3.3B active, 128 experts / 8 active, Apache-2.0 [v].
• On Mac/MLX: mlx-community/Qwen3-30B-A3B-Instruct-2507-4bit (17.2 GB on disk) [v]. Persona experts are QLoRA over this 4-bit MoE.
• Why Instruct, not -Base: a base model has no chat template / no instruction prior, so MCQ letter-output is unreliable zero/few-shot. The Instruct-2507 variant is non-thinking (no <think> blocks) — clean single-letter answers. The FlexOlmo "frozen anchor = a shared public seed everyone trains against" property holds equally for an instruct seed; this is a pragmatic E1 choice for a clean metric, not an architectural change.
• BTX caveat: physical expert surgery (§2.4c) may require the bf16 backbone (Qwen/Qwen3-30B-A3B-Instruct-2507, ~61 GB) since manipulating quantized expert tensors is messy. Committee + MoErging run on the 4-bit anchor. Disk budget: 17 GB (4-bit) + 61 GB (bf16, BTX only) ≈ 78 GB, within the ~232 GB free.

2.2 The three personas (cross-source train/eval for contamination control)

Training source ≠ eval source for every domain — the key methodological control.

Econ/Business eval configs: econometrics, high_school_macroeconomics, high_school_microeconomics, professional_accounting, management, marketing. Never train on cais/mmlu:auxiliary_train (contaminates all MMLU evals). A normalized n-gram dedupe between each persona's SFT questions and the final eval runs before training; any near-duplicate eval item is dropped.

2.3 Mixed-domain eval (the metric)

• 600 items: 200 per domain, balanced; seed=0, shuffle-within-domain, take first 200.
• Uniform 4-choice (A–D), exact-match accuracy. Grading = first-token logit-argmax over the token set {A,B,C,D} at the first generated position — parser-free, perfectly uniform across domains and models. (Free-generation + regex extraction kept only as a fallback diagnostic.)
• Prompt: chat template, system = "Answer with ONLY the single letter of the correct option.", user = question + A./B./C./D. + Answer:.
• Report per-domain accuracy and macro-average (mean of the 3 domain accuracies).

2.4 The three composition modes

All persona experts are trained against the same frozen anchor (FlexOlmo invariant) so they are mutually composable; each expert carries a learned router embedding (centroid of its training-query hidden states).

• (a) Committee / MoA — text/logit level. Run all 3 expert-iCores; aggregate the first-token {A,B,C,D} distributions (mean, and majority-vote as a secondary). "Bigger system, not a bigger model." Cost ≈ 3× inference. Baseline composition.
• (b) MoErging (embedding-routed, train-free) — embed the query (anchor mean-pooled hidden state), route to the nearest persona router-embedding, answer with that expert's adapter. Decomposable: opt-out = remove that router embedding. The default-live mode in the plan. (For a single-letter answer the routing is query-level; token-level routing is the stronger "single model" claim and is the BTX target.)
• (c) BTX (router-finetuned MoE) — the FlexOlmo expert-augmentation: insert the persona-adapted experts as additional physical experts into the MoE layers, expand the gate to route over the union, and run a light router finetune on a small mixed set with the anchor + experts frozen. The unambiguous "bigger single LLM" test. This is the unproven-engineering stretch (FlexOlmo is verified on OLMo-2, not Qwen3-MoE).

2.5 Training config (per persona)

• QLoRA over the 4-bit MoE anchor via mlx_lm.lora. LoRA must target the expert SwitchLinear layers (mlx-lm/mlx-tune auto-detect MoE) + attention q/v. A dense-LoRA config silently trains the wrong tensors — the model passport records the target-module set.
• Starting hyperparameters (tunable during the specialization-gate phase): rank 16, ~16 layers, batch 4, ~600–1500 iters, LR 1e-4. SFT only for E1 (DPO/GRPO deferred).
• Output: one adapter dir per persona under adapters/, plus a router-embedding .npz per persona.

3. Pre-registered hypotheses & success criteria

Specialization gate (prerequisite, run first). Each persona_i must beat the frozen anchor on its own domain eval by ≥ 5 pp (first-token accuracy). If a persona fails (the AU→US law transfer is the flagged risk), swap that persona's training source (candidate: a US legal-QA set) and retrain before composition. Personas that don't specialize make composition meaningless.

Primary success criterion (predicate B). On the 600-item mixed eval (macro-avg), at least one weight-level mode (MoErging or BTX) exceeds the best single persona iCore by ≥ 8 absolute pp. - "Best single iCore" = max macro-avg over the 3 single-LoRA models on the mixed eval. - Rationale for 8 pp: a router that picks the right specialist should approach the per-domain specialist ceiling (each domain answered by its own specialist), typically 15–30 pp above any single specialist's mixed score; 8 pp is a conservative bar clearly above eval noise (±~2 pp at N=600).

Secondary checks (honesty / fidelity). - Exact opt-out: dropping persona_k drops domain_k accuracy to within ±3 pp of the anchor (no-expert) baseline, while the other two domains stay within ±2 pp of the full composition — with zero retraining. - Right-sizing — "more is not better" (per the §2A ownership-and-learning model): add a naive all-experts-always baseline (activate/average all 3 experts on every query, ignoring domain). The contextual router (MoErging) must beat it on the mixed eval — proving the win comes from routing to the right-sized member set, not from piling on capacity. Also: on narrow single-domain queries the router must route to the correct specialist (target ≥ 90% correct top-1 routing); on those queries, smart routing should match the specialist ceiling while all-experts-always underperforms it. If all-experts-always wins, the thesis is "more is better" — the opposite of the design intent — and must be flagged. - Ceiling gap: report composition macro-avg vs the per-domain specialist ceiling (how much routing leaves on the table). - Committee vs weight-level: report whether the weight-level modes match/beat the committee at lower inference cost.

Falsification. If neither MoErging nor BTX clears +8 pp over best-single, E1 fails → the assembly thesis is falsified at the smallest scale; we stop and re-examine before networking (no sunk-cost continuation).

E1b (sovereignty / part-is-a-whole). Fold persona-1's LoRA into a standalone W_1 = B + Δ_1 via mlx_lm.fuse; confirm it loads and answers offline with no router/anchor co-present. Proves the unconditional-ownership face.

4. Run plan (phased; gate between phases)

1. Toolchain smoke test — convert → lora → eval → fuse on a tiny model (Qwen3-0.6B/4B). (done — MLX 0.31.2 / mlx-lm 0.31.3 on the M4 Max GPU; QLoRA-on-quantized-MoE validated, val loss 4.94→1.00; HF_HUB_DISABLE_XET=1 required to avoid a download hang. Anchor Qwen3-30B-A3B-Instruct-2507-4bit downloaded.)
2. Data prep — download + normalize the 3 SFT sets → mlx-lm chat jsonl; build the 600-item mixed eval; run the train/eval dedupe.
3. Atoms + specialization gate — train the 3 persona experts on the 30B-A3B anchor; verify each clears the +5 pp own-domain gate. ← checkpoint.
4. Composition — implement (a) committee, (b) MoErging, (c) BTX, and the (d) all-experts-always control; eval each on the mixed set + the narrow per-domain routing check.
5. Verdict — composed vs best-single (+8 pp), opt-out exactness, E1b fold. Write the scorecard; go/no-go on networking.

Effort: ~2–3 weeks, 1–2 people (plan §8). A convincing internal result (atoms + committee + MoErging) is achievable well before BTX lands.

5. Risks (E1-specific)

• FlexOlmo → Qwen3-MoE retarget (blocks BTX): the frozen-anchor expert-insertion recipe is verified on OLMo-2; ultra-sparse Qwen3-MoE has different expert granularity. Mitigation: MoErging (b) needs no surgery and can satisfy the primary criterion alone; BTX is the stretch; OLMoE is the fallback base if Qwen3-MoE surgery proves intractable.
• AU→US law transfer (specialization gate): mitigated by the gate + a US-law training fallback.
• 4-bit vs bf16 for BTX: expert surgery may force the bf16 backbone; budgeted in disk.
• Eval noise / format: first-token logit-argmax removes parsing noise; N=600 keeps the ±2 pp band well under the 8 pp bar.

6. Results (2026-06-21) — E1 PASSED (with a quantified interference tax)

The experiment went through three testbeds before a valid one; the journey is itself the finding.

Testbed evolution: 1. Public domains (medicine/law/econ) → abandoned. A strong instruct anchor already scores ~0.6–0.86 on public MCQ → negative headroom: persona SFT only degraded it (medicine 0.82→0.56). Composition is untestable when specialists can't beat the base. 2. Synthetic invented "registries" (helix/ledger/relay) → the valid testbed. Anchor ~chance (0.26) on invented facts → clean headroom. Also more faithful to HiNet's real "learn my unseen data" use case. 3. Fact injection: 4-bit FAILED, bf16 full-FT SUCCEEDED. 4-bit QLoRA could not inject facts (degenerate output, 0/8 recall) — confirms Physics-of-LMs int4 capacity collapse. bf16 full-FT of Qwen3-4B-Instruct-2507 (all-linear, diverse aug + full-profile examples + 5% replay, 250 iters, lr 2e-5): helix 0.27→0.97 likelihood, 8/8 generation recall. Full-FT peaked at 26 GB → 7–8B full-FT is comfortable on a 128 GB Mac. Fine-tuning genuinely injects personal data when done right; the earlier "can't" was a setup flaw.

Three specialists (bf16 full-FT, each on its own pack): helix_ft 0.97 / ledger_ft 0.87 / relay_ft 0.97 on their own packs, ~chance on others — clean specialization, no spillover. Best single macro on the mixed test = 0.556; specialist (oracle-routing) ceiling = 0.934.

Composition (task-vector merge of the 3 specialists, graded on all packs):

Verdict: PASS. Weight-level composition (sum, a single merged model) beats the best single iCore by +11.1 pp (≥8 pp bar) and knows all three packs above chance → the whole is a bigger single model than the part. Routing achieves ~ceiling (0.934) as a system of specialists.

The depth↔composability tax (honest): the merged single model (0.667) is well below the ceiling (0.934) — merging three heavily-overfit full-FT deltas (val loss 0.136 → large-magnitude task vectors) loses ~28%/pack to interference, exactly the research boundary. mean dilutes; TIES (this impl) was dominated by the largest delta; sum won. Reducing the tax: lighter/bounded deltas (DoRA, lower rank, less overfit) + re-basing should pull merge toward the ceiling — the next experiment. Routing avoids interference entirely (at the cost of being a system, not one model).

M5-009 sweep update (2026-06-22): tested delta-type (full-FT-heavy/light, DoRA-r64, LoRA-r128) × merge (sum/TIES/DARE). Result: merge-algo buys single digits (best full-FT-heavy+DARE 0.678/73%); LoRA/DoRA under-inject facts (0.478 ceiling) so low-rank bounding backfires for facts; the axes cannot close the gap to 0.934. Conclusion (the real finding): independent facts are the worst case for weight-merging — erased facts don't recover via distill (facts don't generalize from probes), so weight-fact-composition is structurally capped (~73% ≈ small-model recovery band). Facts should compose by RETRIEVAL union (lossless); weight-merge + distill is for SKILLS. The composition layer inherits the storage hybrid (facts→RAG, skills→weights). Full detail: Personalization-Engine §5b–§5c.

M5-011 update (2026-06-22) — refined: the variable is CONFLICT, not fact-vs-skill. 3 substitution-cipher "skill" specialists (directly conflicting maps on the same input) merged catastrophically (sum 0.04 / 7% retention, ≪ facts' 73%), but routing them hit the 0.61 ceiling (~15× the merge). Lesson: merge complementary deltas (literature ~98% on different tasks); ROUTE conflicting ones; RETRIEVAL for facts — the selector picks by conflict. [Personalization-Engine §5c].

Next: The sim-to-real gap →  ·  All documentation →