Update 2026-05-05: v14_branchb_learnable_vocab deployed live — single architectural change from v13: vocab_matrix_global is now an nn.Parameter (~5.6M trainable parameters) trained alongside the brain via AdamW with lr=EMBED_LR/2=2.5e-4. Brain was warm-started from v13 final and frozen for epoch 1 so the vocab could adjust against fixed brain predictions, then unfrozen for epochs 2–5 (joint training). Architecture otherwise identical to v13: 12L / 384D / 12H / RoPE / Pre-LN / AdamW, vocab 14,704, context 32, 5.3 MB corpus. Trained 2026-05-05 on Modal L4, $8.00.

Loss curve (the key evidence): v13 final plateau: 5.04 nats. Epoch 1 end (brain frozen, only vocab moving): 4.7854 — already 0.25 nats below v13 with the brain frozen. Epoch 2 (brain unfrozen, joint training): 4.0939. Epoch 3: 3.5592. Epoch 4: 3.0563. Epoch 5 final: 2.7059 — 2.34 nats below v13, ~2.12 nats above the theoretical floor at INV_TEMP=30 (≈0.59).

Smoking gun — epoch 1 with brain frozen. The fact that loss broke through v13's 5.04 plateau in epoch 1 with the brain still frozen is the clean ablation: vocab geometry was the bottleneck. The sentence-transformer initialization optimizes for semantic similarity ("cat" near "dog"), not syntactic prediction ("cat" followed by "is"). Letting the vocab learn under the cross-entropy gradient unlocked the descent.

Eval verdict — PARTIAL. Qualitatively much improved, not yet fully coherent. Output now produces real English fragments and recognizable conversational/scheduling patterns from the dailydialog corpus. Sample outputs: "hi how about tomorrow coming to account on friday", "okay how about one week user what time bot at ten to p after", "ten minutes walk", "i love you very long". Compare v13's token-soup: "today we who had selling how me user". Replies are not yet fully coherent multi-turn dialogue but this is by far the cleanest signal yet — dialog-shaped text is appearing for the first time.

Significance of Branch B. Branch B is one of engram's core architectural bets: vocab/brain as separable, swappable components. v14-B is the first run to exercise the "learnable" half of "learnable 96-D coordinates" under cross-entropy training. The frozen sentence-transformer geometry optimized for semantic similarity turns out to be hostile to syntactic prediction — these are different tasks, and the gradient needed to move the vocab to find out.

Next — Branches A, C, D from V14_CANDIDATES.md. Branch A (raise halt-gate cap from 3 → 5 + lower ponder cost) is most likely next: avg_ponder is still pegging at 3.00 (the cap), meaning the adaptive-compute lever is not yet engaged. Branch D (surprise-modulated gradient) and Branch C (episodic memory at training time) are the remaining axes. After A/C/D land, plans/FUTURE_RESEARCH.md picks up at v15+ (∇-Reasoner is the recommended first follow-on, zero training cost). Cumulative spend: ~$73 of $150 budget ceiling.

Five architectural ideas from kyegomez/OpenMythos were ported and tested with strict pass/kill criteria from a benchmark harness (bench/run.py). Only one shipped.

Each run tested one hypothesis. Architecture has been stable since v6 (Pre-LN + AdamW + RoPE).

A 16-prompt evaluation across three difficulty buckets (greetings · chitchat · harder Q&A) was run against v4_rope. Result: all 32 generated replies were near-identical sequences of rare/odd words regardless of input prompt. Example prompt/reply pairs:

Diagnosis: the model isn't conditioning on input prompts. The output collapses to the same vocab cluster regardless of context. Two plausible causes:

• Undertraining — 1 epoch on a corpus where ~50% is 19th-century novels means rare archaic vocab dominates the output distribution. Classic sign of insufficient gradient steps.
• Eval bug — the bench harness used torch.randn() embeddings in its eval_cosine_top1, which never moved off 5.0% across any architectural change. The metric may be saturated / noise-dominated, but live eval on real ChromaDB embeddings still produces nonsense — so undertraining is the dominant explanation.

Modal pricing as of 2026-04-26 (Starter plan, $25 included credits). Each tier assumes restart from previous run + bug fixes. Recommendation: don't budget more than $5 until v5 confirms whether undertraining is really the bottleneck.

1. Branch A — raise halt-gate cap from 3 → 5, lower ponder cost. avg_ponder is pegging at exactly 3.00 (the current cap) in v14-B evals, which means the adaptive-compute lever is not engaged — the model is always running to the cap rather than learning to halt earlier on easy tokens. Raising the cap to 5 and reducing the ponder cost will test whether the model can learn differentiated pondering depth. This is a relatively cheap single-variable change. See plans/V14_CANDIDATES.md.
2. Branches C and D — episodic memory and surprise-modulated gradient. Branch C adds episodic memory at training time; Branch D modulates the loss gradient by per-token prediction surprise. Both are independent axes that can be tested after Branch A. Order and priority are documented in plans/V14_CANDIDATES.md.
3. v15+ research backlog. Once Branches A/C/D are resolved, plans/FUTURE_RESEARCH.md picks up at the ∇-Reasoner follow-on (zero training cost) and other v15+ candidates. The core vocab/brain architecture is now validated and coherent dialog fragments are appearing — further work should stack on this foundation rather than revisiting architecture fundamentals.