Recursive, Evidence-Gated Training

“Self-improving” is an appealing phrase and a dangerous specification. A model that trains on its own unchecked output can amplify errors, contaminate its evaluation set, forget rare cases, or optimize an easy proxy. Recursive learning therefore requires more structure than repeatedly launching a training script.

This chapter defines the DOGMA recursive research loop. Gradient descent still trains each neural candidate. Evolutionary search operates one level above it: it chooses bounded changes to data, optimization, or architecture; evaluates the descendants; and decides what may be retained. This separation follows the spirit of population-based training [Jaderberg et al., 2017] and quality-diversity search [Mouret and Clune, 2015], while adding strict data provenance and deployment gates.

12.1 Three Loops, Three Responsibilities

Let a candidate be \[ c=(\theta ,h,d,a), \] where \(\theta \) are learned weights, \(h\) are training hyperparameters, \(d\) is a versioned data manifest, and \(a\) is an architecture declaration. A lineage edge \[ c_{t+1}=M(c_t; m_t) \] must record the mutation \(m_t\), the parent checkpoint, and the code revision. Without this information, a result cannot be reproduced or rolled back.

12.2 The Candidate Lifecycle

The production loop has seven stages.

1.

Observe. Collect failures from immutable held-out tests and operational traces. Do not train on the held-out answers.

2.

Diagnose. Classify each failure: sequence validity, language readability, factuality, calibration, long-range retention, contamination, or resource cost.

3.

Propose. Change exactly one bounded variable in an ordinary descendant cycle. Examples are learning rate, dropout, curriculum temperature, or adaptive-sampling strength.

4.

Train. Produce a new checkpoint from the declared parent and data manifest. Training is isolated from the live service.

5.

Challenge. Run frozen probes with at least two fixed seeds, held-out likelihood, regression tests, and resource measurements.

6.

Archive. Place scientifically valid stepping stones in a behavior-indexed archive even when they are not the global winner.

7.

Promote or reject. Atomically update the live pointer only if all hard gates pass and the global objective improves. Otherwise retain the parent.

12.3 Data Provenance and Model Collapse

Recursive training creates a feedback channel: model outputs can become future inputs. Repeatedly replacing real observations with generated approximations can erase distribution tails and degrade later generations, a failure studied as model collapse [Shumailov et al., 2024]. DOGMA therefore separates real anchors from derived or synthetic material.

For sampling weights \(w_i\), define \[ r=\frac {\sum _{i\in R}w_i}{\sum _i w_i}, \] where \(R\) is the set of records with audited real-data provenance. The canonical policy requires \[ r\ge 0.65,\qquad 1-r\le 0.35. \] This is a policy choice, not a universal constant. It is deliberately conservative for a small research program. If an experiment uses another mixture, it must receive another contract name and be reported separately.

Document hashes detect exact overlap among train, validation, and test splits. Near-duplicate detection and homology-aware splitting remain required for serious genomic benchmarks: exact hashes alone cannot detect highly similar subsequences or related organisms.

12.4 A Multi-Probe Evaluator

Perplexity answers one narrow question: how surprised was the model by held-out next bytes under teacher forcing? Let \[ \operatorname {PPL}=\exp \left ( -\frac {1}{N}\sum _{i=1}^{N}\log p_\theta (x_i\mid x_{<i}) \right ). \] It does not test whether autoregressive sampling remains on a readable trajectory. The frozen dogma-recursive-v2 suite therefore separates three capabilities:

1.

Sequence validity: produce a sufficiently long run containing only canonical \(A,C,G,T\) symbols.

2.

Readable explanation: explain the biological central dogma in plain language without invalid bytes.

3.

Self-critique: state one model limitation and one experiment that could test it.

Each probe runs under two fixed generation seeds. These checks are intentionally small smoke tests, not claims of scientific knowledge. A stronger evaluation program must add exact genomic tasks, out-of-distribution species, reverse complement invariance, length extrapolation, calibration, and matched baselines such as HyenaDNA, Caduceus, and DNABERT-2.

For archive ranking, the current scalar objective is \[ J(c)=P_{\mathrm {test}}+0.25P_{\mathrm {valid}} +\lambda _D+\lambda _E+\lambda _C, \] where each \(\lambda \) is zero for a passed probe and positive for a failed probe. Lower is better. Hard promotion still requires every probe to pass; the scalar cannot trade away a critical failure.

12.5 Why Keep a Quality-Diversity Archive?

A single best-so-far chain can become trapped. A candidate may improve DNA syntax while temporarily losing explanatory language; another may preserve language but explore a different recurrent mechanism. MAP-Elites [Mouret and Clune, 2015] keeps the best candidate in each behavior cell rather than collapsing all behavior into one leaderboard.

The current archive descriptor records: \[ \text {architecture features} : \{\text {DNA},\text {explanation},\text {critique}\}\text { passed}. \] Every third scheduled cycle may branch from an archive elite. The live checkpoint remains unchanged until a descendant wins the complete gate. This provides exploration without converting production into an online experiment.

12.6 Controlled Mutation

Ordinary descendant cycles mutate exactly one scalar: \[ h'_j=\operatorname {clip}(\alpha h_j,\ell _j,u_j). \] The mutation record stores \(j,\alpha ,\ell _j,u_j\), the parent value, and the candidate value. One-variable mutation is slower than unconstrained search but makes failure analysis possible on limited compute.

Architecture changes are separate branches. Adding a layer or changing a state dimension changes tensor shapes, so an optimizer state from the parent may no longer be valid. Such experiments start from a declared compatible checkpoint or from initialization and use matched compute. They never masquerade as an ordinary resumed cycle.

12.7 Promotion Contract

A candidate can replace the live model only when all of the following hold:

1.

the architecture audit confirms no self-attention or transformer layer;

2.

parent and candidate checkpoints are distinct and recorded;

3.

at least 65% of sampling mass remains on real anchors;

4.

train/evaluation overlap is zero under the declared detector;

5.

two distinct generation seeds reproduce every probe;

6.

validation and test perplexity are finite and within the regression budget;

7.

no critical regression is present; and

8.

\(J(c_{\mathrm {candidate}})<J(c_{\mathrm {live}})\).

Promotion creates a pointer to the exact evaluated checkpoint. It does not copy an ambiguous “best” file from another run. The previous live checkpoint is retained for rollback.

12.8 Connection to Self-Improving Systems

Population-based training adapts hyperparameters while a population learns [Jaderberg et al., 2017]. Quality-diversity methods preserve multiple useful behaviors [Mouret and Clune, 2015]. The Darwin Gödel Machine explores code changes and evaluates descendants on coding tasks [Zhang et al., 2025]; AlphaEvolve combines model-generated programs with automated evaluators [Novikov et al., 2025]. These systems support a general lesson: recursion becomes useful when the evaluator is external to the candidate and failures are cheap to reject.

DOGMA adopts that lesson conservatively. It currently evolves bounded training choices and archives checkpoint lineages. Self-editing source code remains a future sandboxed research track, because tests can be incomplete and an agent can overfit the evaluator. The phrase “evolves itself” should therefore mean “produces descendants under an external promotion contract,” not “unilaterally changes the running system.”

12.9 Exercises

12.1.

A corpus has real-anchor weight \(18\) and synthetic weight \(12\). Compute \(r\). Does it satisfy the canonical policy?

12.2.

A model has validation perplexity \(2.8\), test perplexity \(1.4\), and fails only the explanation probe. Compute \(J\) when \(\lambda _E=1.5\). Explain why it still cannot be promoted.

12.3.

Design a behavior descriptor with two architectural and three capability dimensions. How many archive cells are possible?

12.4.

Give an example where a candidate improves test perplexity while generation quality worsens. Why is this possible?

12.5.

Propose a homology-aware split test stronger than exact document hashes. State its computational cost and possible false positives.

12.6.

Write a mutation record for changing dropout from \(0.10\) to \(0.11\). Include enough information to reproduce and reverse it.

12.7.

Design an out-of-distribution probe that distinguishes memorizing DNA motifs from learning a useful sequence rule.

12.8.

Identify one way a candidate could game each of the three smoke probes. Add a counter-test for every weakness.

12.10 Key Takeaways

Suggested Reading

• Jaderberg et al. [2017] on population-based training.
• Mouret and Clune [2015] on MAP-Elites and quality-diversity search.
• Shumailov et al. [2024] on recursive synthetic-data failure.
• Zhang et al. [2025] and Novikov et al. [2025] on evaluator-driven machine discovery.