DOGMA — The Six-Stage Pipeline

This chapter presents the central contribution of this book: DOGMA, the DNA-Organized Genomic Model Architecture. DOGMA is not an incremental improvement to the transformer. It is a fundamentally different way of organizing computation in an AI system—one inspired by three and a half billion years of biological information processing.

The core idea is simple but radical: an intelligent system should maintain a persistent, structured internal genome whose regions have distinct roles, activation patterns, and interaction semantics. Output is not the system itself. Output is the temporary expression of a deeper organized substrate.

In a standard large language model, everything—system instructions, retrieved context, latent plans, working memory, and output tokens—exists in a single undifferentiated token stream, processed by uniform attention blocks. DOGMA rejects this flattening. It proposes that capable intelligence requires the separation of persistent state from transient expression, just as biology separates the genome from the phenotype.

This chapter is organized as follows. We begin with the DOGMA thesis and motivation (§7.1), then define the genomic base—DOGMA’s atomic computational unit (§7.2). The heart of the chapter is a detailed walkthrough of the six-stage pipeline (§7.3), with each stage receiving its own dedicated section containing mathematical formulations, TikZ diagrams, worked examples, and complexity analysis. We then present a full architecture diagram (§7.5), the DogmaBlock pseudocode (§7.6), a detailed comparison with competing architectures (§7.7), and a complete worked example tracing data through all six stages (§7.8).

7.1 The DOGMA Thesis

7.1.1 Why Current LLMs Are Structurally Limited

Before presenting DOGMA’s architecture, we must understand why a new architecture is needed. Current transformer-based LLMs, despite their impressive capabilities [Brown et al., 2020, Wei et al., 2022a], share several structural limitations:

1.

No native region semantics. System prompts, user queries, retrieved documents, chain-of-thought reasoning, and generated output all occupy the same undifferentiated token stream. The model must learn implicit conventions to distinguish these roles—conventions that are fragile and unverifiable.

2.

Weak persistent organization. A transformer’s “knowledge” lives in its parameters (frozen after training) and its context window (bounded and transient). There is no intermediate state that persists across sessions while remaining modifiable.

3.

Dense uniform interaction. Self-attention computes pairwise interactions between all positions, even when most regions should remain silent. This is \(O(T^2)\) in sequence length \(T\)—biologically implausible and computationally wasteful for long sequences.

4.

Output-first bias. The canonical training objective (next-token prediction) optimizes text generation directly. There is no explicit intermediate “phenotype”—no structured representation of what the model intends to do before it starts producing tokens.

5.

No evolutionary self-modification. Once trained, a model’s parameters are fixed. It cannot modify its own computational structure in response to new tasks or environments without full retraining.

DOGMA addresses each limitation by drawing on biological precedent (Table 7.1).

7.2 The Genomic Base: DOGMA’s Atomic Unit

The fundamental computational atom of DOGMA is the genomic base—a typed, weighted, compatibility-annotated element that carries more structure than a standard token embedding.

7.2.1 Anatomy of a Genomic Base

To build intuition, let us examine each component in detail.

The base embedding \(\sigma _i\). This is the primary informational payload of the base. In the simplest case, \(\sigma _i\) is a learned embedding vector of dimension \(d\) (e.g., \(d = 512\) or \(d = 1024\)). Unlike a standard token embedding, \(\sigma _i\) is structured: the first \(d/4\) dimensions encode type-specific features, while the remaining dimensions encode content. This structure allows downstream operations to efficiently extract type information without separate lookups.

The base type \(\tau _i\). The discrete type tag partitions all bases into four functional classes. The type determines which computational pathways the base participates in and which complementary pairing rules apply. The complement mapping is: \begin {equation} \overline {\mathsf {A}} = \mathsf {T}, \quad \overline {\mathsf {T}} = \mathsf {A}, \quad \overline {\mathsf {C}} = \mathsf {G}, \quad \overline {\mathsf {G}} = \mathsf {C}. \label {eq:complement} \end {equation}

The compatibility vector \(\rho _i\). This vector governs inter-base interactions. Given two bases \(b_i\) and \(b_j\), their compatibility score is: \begin {equation} \mathrm {compat}(b_i, b_j) = \rho _i^\top \rho _j + \beta (\tau _i, \tau _j), \label {eq:compat} \end {equation} where \(\beta (\tau _i, \tau _j)\) is a learned type-pair bias. Complementary pairs (\(\mathsf {A}\)–\(\mathsf {T}\) and \(\mathsf {C}\)–\(\mathsf {G}\)) receive a positive bias; mismatches receive a negative bias or zero.

The weight \(\omega _i\). This scalar controls how strongly the base contributes to downstream computation. During the regulation stage (§7.3.3), weights are modulated by context. A weight of zero effectively silences the base.

The following TikZ diagram illustrates a single genomic base with all its components annotated.

ATCGATCG

7.2.2 Genomic Sequences and Regions

Bases are organized into genomic sequences and regions:

7.2.3 Genomic Embedding: From Raw Bytes to the 4-Tuple Base Representation

Before DOGMA can operate on input data, raw bytes must be converted into genomic bases. This section provides a step-by-step account of the Genomic Embedding process—the transformation that maps each input byte to the structured 4-tuple \(b_i = (\sigma _i, \tau _i, \rho _i, \omega _i)\).

Step 1: Byte-to-Base Mapping

Every input byte has a value in \(\{0, 1, \ldots , 255\}\). DOGMA maps each byte value to one of the four bases \(\{\mathsf {A}, \mathsf {C}, \mathsf {G}, \mathsf {T}\}\) using modular arithmetic: \begin {equation} \tau (v) = \begin {cases} \mathsf {A} & \text {if } v \bmod 4 = 0, \\ \mathsf {C} & \text {if } v \bmod 4 = 1, \\ \mathsf {G} & \text {if } v \bmod 4 = 2, \\ \mathsf {T} & \text {if } v \bmod 4 = 3. \end {cases} \label {eq:byte-to-base} \end {equation}

Table 7.3 shows this mapping for representative ASCII values.

Step 2: Strand Identity \(\sigma _i\) — The Base Embedding

The strand identity \(\sigma _i \in \mathbb {R}^d\) is the primary informational payload of each base. It is computed in two stages:

(a) Type-specific learned embedding. Each base type \(\tau \in \{\mathsf {A}, \mathsf {C}, \mathsf {G}, \mathsf {T}\}\) has an associated embedding matrix \(\mathbf {E}_\tau \in \mathbb {R}^{256 \times d}\). The byte value \(v_i\) indexes into the embedding matrix for its assigned type: \begin {equation} \sigma _i^{(\text {raw})} = \mathbf {E}_{\tau _i}[v_i] \in \mathbb {R}^d. \label {eq:sigma-raw} \end {equation}

Having four separate embedding tables (one per type) means that even if two bytes map to the same base type, their embeddings can differ based on the actual byte value. The embedding dimension \(d\) is structured: the first \(d/4\) dimensions encode type-specific features, while the remaining \(3d/4\) encode content.

(b) Positional enrichment. The raw embedding is enriched with positional information from the backbone position \(\rho _i\) (described below) via addition: \begin {equation} \sigma _i = \sigma _i^{(\text {raw})} + \rho _i. \label {eq:sigma-final} \end {equation}

This is analogous to how transformer embeddings add token and positional embeddings, but with a key difference: \(\rho _i\) is not sinusoidal—it encodes backbone geometry.

Step 3: Base Type \(\tau _i\) — Learned Embedding Lookup

The base type \(\tau _i \in \{\mathsf {A}, \mathsf {C}, \mathsf {G}, \mathsf {T}\}\) is determined by the byte-to-base mapping (Equation 7.5). Beyond its discrete role in routing computation, \(\tau _i\) also contributes a learned type embedding \(\mathbf {t}_{\tau _i} \in \mathbb {R}^{d_\tau }\) that is concatenated into downstream representations: \begin {equation} \mathbf {t}_{\tau _i} = \mathbf {E}_{\text {type}}[\tau _i], \quad \mathbf {E}_{\text {type}} \in \mathbb {R}^{4 \times d_\tau }. \label {eq:type-embedding} \end {equation}

The type embedding serves two purposes: (1) it provides a continuous representation of the discrete type tag for use in differentiable operations, and (2) it allows the model to learn type-specific biases that evolve during training. The type embedding dimension \(d_\tau \) is typically \(d/4\), matching the type-specific partition of the base embedding.

Step 4: Backbone Position \(\rho _i\) — Replacing Sinusoidal Encoding

In a standard transformer, positional information is injected via sinusoidal functions of the position index [Vaswani et al., 2017a]: \begin {equation} \text {PE}(i, 2j) = \sin \!\left (\frac {i}{10000^{2j/d}}\right ), \quad \text {PE}(i, 2j+1) = \cos \!\left (\frac {i}{10000^{2j/d}}\right ). \label {eq:sinusoidal-pe} \end {equation}

DOGMA replaces this with backbone position encoding, inspired by the sugar-phosphate backbone of DNA. The backbone position \(\rho _i \in \mathbb {R}^d\) encodes both absolute position and local structural context: \begin {equation} \rho _i = \mathbf {W}_{\text {pos}} \cdot \phi (i) + \mathbf {W}_{\text {local}} \cdot \psi (\tau _{i-1}, \tau _i, \tau _{i+1}), \label {eq:backbone-position} \end {equation} where:

• \(\phi (i) \in \mathbb {R}^{d_p}\) is a learned absolute position embedding (similar to learned positional embeddings in GPT-2 [Radford et al., 2019], but extended to support arbitrary lengths via interpolation).
• \(\psi (\tau _{i-1}, \tau _i, \tau _{i+1}) \in \mathbb {R}^{d_l}\) is a learned trinucleotide context embedding that encodes the local type pattern. There are \(4^3 = 64\) possible trinucleotide contexts, each with its own learned embedding.
• \(\mathbf {W}_{\text {pos}} \in \mathbb {R}^{d \times d_p}\) and \(\mathbf {W}_{\text {local}} \in \mathbb {R}^{d \times d_l}\) are learned projection matrices.

Step 5: Complementarity Weight \(\omega _i\) — Watson-Crick Pairing

The weight \(\omega _i \in \mathbb {R}_{\ge 0}\) controls how strongly each base contributes to downstream computation. At initialization, \(\omega _i\) is derived from Watson-Crick base-pairing statistics: \begin {equation} \omega _i^{(0)} = \frac {1}{|N(i)|} \sum _{j \in N(i)} \mathbb {1}\!\left [\tau _j = \overline {\tau _i}\right ], \label {eq:omega-init} \end {equation} where \(N(i)\) is a local window of positions around \(i\), \(\overline {\tau _i}\) is the Watson-Crick complement of \(\tau _i\) (Equation 7.2), and \(\mathbb {1}[\cdot ]\) is the indicator function. This initial weight is high when a base has many complementary partners nearby (suggesting it participates in double-stranded structure) and low when it is isolated.

During the forward pass, \(\omega _i\) is modulated by the Regulation Gate (Stage 3), which can amplify or suppress individual bases based on context.

Complete Worked Example: Encoding “Hello”

Let us trace the complete genomic embedding for the ASCII string “Hello” with embedding dimension \(d = 8\). The five characters have byte values:

Step A: Base type assignment. Applying Equation 7.5: \[ \tau _1 = \mathsf {A}, \quad \tau _2 = \mathsf {C}, \quad \tau _3 = \mathsf {A}, \quad \tau _4 = \mathsf {A}, \quad \tau _5 = \mathsf {T}. \]

The genomic sequence spells out the base string: A-C-A-A-T.

Step B: Raw embeddings \(\sigma ^{(\text {raw})}\). Each byte value indexes into its type-specific embedding table. Using \(d = 8\), suppose the (randomly initialized, pre-training) embeddings are: \begin {align*} \sigma _1^{(\text {raw})} &= \mathbf {E}_{\mathsf {A}}[72] = (+0.12,\; -0.34,\; +0.56,\; +0.78,\; -0.23,\; +0.45,\; -0.67,\; +0.11), \\ \sigma _2^{(\text {raw})} &= \mathbf {E}_{\mathsf {C}}[101] = (-0.41,\; +0.22,\; +0.63,\; -0.18,\; +0.55,\; -0.31,\; +0.42,\; -0.09), \\ \sigma _3^{(\text {raw})} &= \mathbf {E}_{\mathsf {A}}[108] = (+0.33,\; +0.15,\; -0.28,\; +0.61,\; +0.14,\; -0.52,\; +0.39,\; +0.27), \\ \sigma _4^{(\text {raw})} &= \mathbf {E}_{\mathsf {A}}[108] = (+0.33,\; +0.15,\; -0.28,\; +0.61,\; +0.14,\; -0.52,\; +0.39,\; +0.27), \\ \sigma _5^{(\text {raw})} &= \mathbf {E}_{\mathsf {T}}[111] = (-0.55,\; +0.47,\; +0.19,\; -0.36,\; +0.72,\; +0.08,\; -0.44,\; +0.61). \end {align*}

Note that positions 3 and 4 (both “l”, ASCII 108, type A) receive identical raw embeddings—they will be distinguished by their backbone positions.

Step C: Backbone positions \(\rho _i\). The backbone position combines absolute position and trinucleotide context. For position \(i=3\), the trinucleotide context is \((\tau _2, \tau _3, \tau _4) = (\mathsf {C}, \mathsf {A}, \mathsf {A})\), while for position \(i=4\), it is \((\tau _3, \tau _4, \tau _5) = (\mathsf {A}, \mathsf {A}, \mathsf {T})\). These different contexts produce different local embeddings: \begin {align*} \rho _3 &= \mathbf {W}_{\text {pos}} \cdot \phi (3) + \mathbf {W}_{\text {local}} \cdot \psi (\mathsf {C}, \mathsf {A}, \mathsf {A}) \\ &= (+0.05,\; +0.11,\; -0.08,\; +0.03,\; +0.15,\; -0.02,\; +0.07,\; -0.13), \\ \rho _4 &= \mathbf {W}_{\text {pos}} \cdot \phi (4) + \mathbf {W}_{\text {local}} \cdot \psi (\mathsf {A}, \mathsf {A}, \mathsf {T}) \\ &= (+0.09,\; -0.06,\; +0.12,\; -0.04,\; +0.18,\; +0.10,\; -0.05,\; +0.03). \end {align*}

Step D: Final embeddings \(\sigma _i\). Adding backbone positions to raw embeddings (Equation 7.7): \begin {align*} \sigma _3 &= \sigma _3^{(\text {raw})} + \rho _3 = (+0.38,\; +0.26,\; -0.36,\; +0.64,\; +0.29,\; -0.54,\; +0.46,\; +0.14), \\ \sigma _4 &= \sigma _4^{(\text {raw})} + \rho _4 = (+0.42,\; +0.09,\; -0.16,\; +0.57,\; +0.32,\; -0.42,\; +0.34,\; +0.30). \end {align*}

The two “l” characters now have different embeddings due to their different backbone positions, even though they share the same raw embedding.

Step E: Complementarity weights \(\omega _i\). Using a window of size 3 centered at each position: \begin {align*} \omega _1 &: N(1) = \{2, 3\}. \;\text {Complements of A are T. Neither C nor A is T.} \;\Rightarrow \; \omega _1 = 0/2 = 0.0. \\ \omega _2 &: N(2) = \{1, 3\}. \;\text {Complements of C are G. Neither A nor A is G.} \;\Rightarrow \; \omega _2 = 0/2 = 0.0. \\ \omega _3 &: N(3) = \{2, 4\}. \;\text {Complements of A are T. Neither C nor A is T.} \;\Rightarrow \; \omega _3 = 0/2 = 0.0. \\ \omega _4 &: N(4) = \{3, 5\}. \;\text {Complements of A are T. Position 5 is T!} \;\Rightarrow \; \omega _4 = 1/2 = 0.5. \\ \omega _5 &: N(5) = \{4\}. \;\text {Complements of T are A. Position 4 is A!} \;\Rightarrow \; \omega _5 = 1/1 = 1.0. \end {align*}

Position 4 has moderate weight (\(\omega _4 = 0.5\)) because one of its two neighbors is complementary, while position 5 has full weight (\(\omega _5 = 1.0\)) because its only neighbor is complementary. Positions 1–3 have zero initial weight—they lack nearby complementary partners.

Step F: Final 4-tuple representation. The complete genomic bases for “Hello” are:

The compatibility vector \(\rho _i\) (in its role as the \([0,1]^c\) inter-base interaction affinity) is computed later during the pipeline; at embedding time, \(\rho _i\) serves as the backbone position vector that enriches \(\sigma _i\).

7.3 The Six-Stage Pipeline: Overview

DOGMA processes information through six sequential stages, each corresponding to a step in the biological Central Dogma and its regulatory context. The complete pipeline transforms persistent genomic state into task-specific output:

\begin {equation} \boxed { \text {TetraMemory} \xrightarrow {\text {Stage 1}} \text {DNA Block} \xrightarrow {\text {Stage 2}} \text {Regulate} \xrightarrow {\text {Stage 3}} \text {Translate} \xrightarrow {\text {Stage 4}} \text {Strand Attn} \xrightarrow {\text {Stage 5}} \text {Epi+Allo} \xrightarrow {\text {Stage 6}} \text {Output} } \label {eq:dogma-pipeline} \end {equation}

Each stage receives the output of the previous stage and produces a transformed representation. We now devote a full section to each stage, providing mathematical formulations, diagrams, worked examples, and complexity analysis.

7.3.1 Stage 1: TetraMemory

TetraMemory is DOGMA’s replacement for global self-attention. Instead of computing pairwise interactions between all \(T\) tokens (\(O(T^2)\) cost), TetraMemory organizes tokens into overlapping groups of four—tetrahedra—and computes interactions only within and between adjacent tetrahedra in \(O(T)\) time.

The Tetrahedral Unit: \(K_4\) Complete Interaction

Given a sequence of \(T\) tokens, TetraMemory partitions them into \(\lfloor T/4 \rfloor \) non-overlapping groups of four consecutive tokens. Within each group, all six pairwise interactions are computed, forming a complete graph \(K_4\).

ATCGATCG

Face Propagation

Long-range information flow is achieved through face propagation: adjacent tetrahedra share a triangular face (3 of 4 vertices overlap). This overlap allows information computed in \(\Delta _k\) to flow into \(\Delta _{k+1}\) through shared vertices.

\begin {equation} \text {Face}(\Delta _k, \Delta _{k+1}) = \{x_{4k+2},\; x_{4k+3},\; x_{4k+4}\}. \label {eq:face-sharing} \end {equation}

More precisely, after computing local interactions within \(\Delta _k\), the updated representations of the shared vertices carry information from the entire tetrahedron. When these shared vertices participate in \(\Delta _{k+1}\)’s local computation, information propagates forward. After processing all \(\lfloor T/4 \rfloor \) tetrahedra, information from position 1 can reach position \(T\) through a chain of \(O(T/4)\) propagation steps—but each step involves only \(O(1)\) local computations.

Mathematical Formulation

Let \(\mathbf {X} \in \mathbb {R}^{T \times d}\) be the input sequence. TetraMemory proceeds as follows:

1.

Partition. Divide \(\mathbf {X}\) into groups of 4: \(\mathbf {X}_k = [x_{4k+1}; x_{4k+2}; x_{4k+3}; x_{4k+4}] \in \mathbb {R}^{4 \times d}\).

2.

Local \(K_4\) interaction. For each tetrahedron \(\Delta _k\), compute: \begin {equation} \mathbf {Y}_k = \mathrm {softmax}\!\left (\frac {\mathbf {X}_k \mathbf {W}_Q (\mathbf {X}_k \mathbf {W}_K)^\top }{\sqrt {d_k}}\right ) \mathbf {X}_k \mathbf {W}_V, \label {eq:tetra-local} \end {equation} where \(\mathbf {W}_Q, \mathbf {W}_K, \mathbf {W}_V \in \mathbb {R}^{d \times d_k}\) are learned projections. This is standard attention, but restricted to a \(4 \times 4\) attention matrix—a constant-size operation.

3.

Face propagation. Update shared vertices: \begin {equation} \tilde {x}_{4k+j} = \mathbf {Y}_k[j] + \gamma \cdot \mathbf {Y}_{k-1}[j+1] \quad \text {for } j \in \{1, 2, 3\}, \label {eq:face-prop} \end {equation} where \(\gamma \) is a learned mixing coefficient and the index arithmetic accounts for the face-sharing overlap.

Complexity Analysis

7.3.2 Stage 2: DNA Computing Block

The DNA Computing Block is the core computational engine of DOGMA. It replaces the transformer’s feed-forward + attention block with four parallel computational paths, each inspired by a different mechanism from DNA computing. The four paths are:

1.

Path (i): Strand Displacement — conditional signal routing

2.

Path (ii): Parallel Scan Automaton / SSM — linear-time sequential processing

3.

Path (iii): Stochastic Computing — probabilistic reasoning

4.

Path (iv): Double Strand — bidirectional context

The outputs of all four paths are combined through a learned softmax gate: \begin {equation} \mathbf {y} = \sum _{p=1}^{4} \alpha _p \cdot \mathrm {Path}_p(\mathbf {x}), \qquad \boldsymbol {\alpha } = \mathrm {softmax}(\mathbf {W}_g \mathbf {x} + \mathbf {b}_g) \in \mathbb {R}^4, \label {eq:gated-fusion} \end {equation} where \(\mathbf {W}_g \in \mathbb {R}^{4 \times d}\) and \(\mathbf {b}_g \in \mathbb {R}^4\) are learned parameters. The gate weights \(\alpha _p\) are input-dependent: for some inputs, the strand displacement path dominates; for others, the parallel scan path may carry more weight.

ATCGATCG

We now describe each path in detail.

Path (i): Strand Displacement

Strand displacement is inspired by toehold-mediated strand displacement (TMSD) reactions in DNA nanotechnology [Zhang and Seelig, 2011]. In the wet lab, a single-stranded DNA “invader” binds to a short exposed region (the toehold) of a double-stranded complex, then progressively displaces the incumbent strand through branch migration. DOGMA abstracts this into a computational primitive for conditional signal routing.

Mechanism. Given input \(\mathbf {x} \in \mathbb {R}^{T \times d}\), the strand displacement path computes: \begin {align} \mathbf {e}_{\text {toe}} &= \sigma (\mathbf {x} \mathbf {W}_{\text {toe}}), && \text {(toehold binding energies)} \label {eq:toehold} \\ \mathbf {m}_{\text {gate}} &= \mathbf {1}[\mathbf {e}_{\text {toe}} > \theta ], && \text {(thresholded gate mask)} \\ \mathbf {h}_{\text {migrate}} &= \mathrm {CausalConv1D}(\mathbf {x} \odot \mathbf {m}_{\text {gate}}), && \text {(branch migration via causal conv)} \\ \mathbf {y}_{\text {SD}} &= \mathbf {x} + \mathbf {h}_{\text {migrate}} \mathbf {W}_{\text {out}}, && \text {(residual output)} \end {align}

where \(\theta \) is a learned threshold, \(\sigma \) is the sigmoid function, and \(\odot \) denotes element-wise multiplication. The gate mask \(\mathbf {m}_{\text {gate}}\) selectively activates only those positions where the toehold binding is strong enough—mimicking the thermodynamic threshold required for strand displacement to initiate.

Computational role. Strand displacement excels at conditional routing: “if this pattern is present, activate this pathway.” This is analogous to biological signal transduction cascades.

Path (ii): Parallel Scan Automaton / SSM

The Parallel Scan path implements linear-time sequence processing using the parallel scan (prefix sum) algorithm, closely related to state-space models (SSMs) such as Mamba [Gu and Dao, 2023].

Mechanism. The path defines a linear recurrence: \begin {equation} h_t = \mathbf {A}_t h_{t-1} + \mathbf {B}_t x_t, \qquad y_t = \mathbf {C}_t h_t + \mathbf {D}_t x_t, \label {eq:ch7-ssm-recurrence} \end {equation} where \(\mathbf {A}_t, \mathbf {B}_t, \mathbf {C}_t, \mathbf {D}_t\) are input-dependent (selective) parameters computed from \(x_t\). The key insight is that this recurrence can be computed in \(O(T \log T)\) time using the parallel scan algorithm, which restructures the sequential recurrence into a balanced binary tree of associative operations.

The parallel scan algorithm. Given \(T\) elements with an associative binary operator \(\oplus \), the parallel scan computes all prefix sums \(s_k = a_1 \oplus a_2 \oplus \cdots \oplus a_k\) in \(O(\log T)\) parallel steps using \(O(T)\) total work. For the SSM recurrence, the operator is: \begin {equation} (a_i, b_i) \oplus (a_j, b_j) = (a_j \cdot a_i, \; a_j \cdot b_i + b_j), \label {eq:scan-operator} \end {equation} which allows all hidden states \(h_1, h_2, \ldots , h_T\) to be computed in parallel.

Computational role. The Parallel Scan path handles sequential dependencies efficiently. It captures positional and ordering information that the other paths may miss.

Path (iii): Stochastic Computing

The Stochastic Computing path implements probabilistic reasoning through stochastic bitstream operations, inspired by stochastic computing in hardware design and by the inherent randomness of molecular reactions.

Mechanism. The input is converted to stochastic bitstreams via Bernoulli sampling: \begin {equation} \hat {x}_t^{(s)} \sim \mathrm {Bernoulli}\bigl (\sigma (\mathbf {x}_t \mathbf {W}_s)\bigr ), \qquad s = 1, \ldots , S, \label {eq:stochastic-sample} \end {equation} where \(S\) is the number of stochastic samples. Multiplication of probabilities is implemented as AND operations on bitstreams; addition as OR operations. This provides naturally calibrated uncertainty estimates.

Output. The stochastic path output is the empirical mean over \(S\) samples: \begin {equation} \mathbf {y}_{\text {SC}} = \frac {1}{S} \sum _{s=1}^{S} f(\hat {\mathbf {x}}^{(s)}), \label {eq:stochastic-output} \end {equation} where \(f\) is a learned nonlinear function applied to each sampled bitstream. During training, the Bernoulli sampling is relaxed using the Gumbel–softmax trick for differentiability.

Computational role. Stochastic computing provides uncertainty quantification and robustness to noise. It excels at tasks requiring probabilistic inference or calibrated confidence estimates.

Path (iv): Double Strand

The Double Strand path maintains two coupled representations—a forward strand \(\vec {x}\) and a reverse-complement strand \(\overleftarrow {x}\)—mirroring DNA’s antiparallel double-helix structure.

Mechanism. \begin {align} \vec {h}_t &= \mathrm {ForwardRNN}(\vec {h}_{t-1}, x_t), \label {eq:forward-strand} \\ \overleftarrow {h}_t &= \mathrm {ReverseRNN}(\overleftarrow {h}_{t+1}, \bar {x}_t), \label {eq:reverse-strand} \\ \mathbf {y}_{\text {DS},t} &= \mathbf {W}_{\text {merge}}[\vec {h}_t \| \overleftarrow {h}_t] + \mathbf {b}_{\text {merge}}, \label {eq:ds-merge} \end {align} where \(\bar {x}_t\) is the type-complement of \(x_t\) (applying the complement mapping from Equation 7.2) and \(\|\) denotes concatenation. The forward strand processes the sequence left-to-right; the reverse strand processes the complement right-to-left. Their outputs are concatenated and projected.

Computational role. The Double Strand path provides bidirectional context, similar to bidirectional LSTMs but with the added constraint of complementary pairing. This constraint acts as a regularizer: the reverse strand must maintain consistency with the forward strand’s complement.

7.3.3 Stage 3: Regulation Gate

The Regulation Gate implements DOGMA’s core principle of selective gene expression: not all information should participate in every computation. Given the output of the DNA Computing Block, the Regulation Gate computes a context-dependent express/suppress decision for each position.

Mathematical Formulation

Let \(\mathbf {h} \in \mathbb {R}^{T \times d}\) be the hidden representation from Stage 2 and \(\mathbf {c} \in \mathbb {R}^{d_c}\) be the current context embedding. The Regulation Gate computes: \begin {align} \mathbf {g}_{\text {express}} &= \sigma \bigl (\mathbf {h} \mathbf {W}_e + \mathbf {c} \mathbf {U}_e + \mathbf {b}_e\bigr ) \in [0,1]^{T \times d}, \label {eq:express-gate} \\ \mathbf {g}_{\text {suppress}} &= \sigma \bigl (\mathbf {h} \mathbf {W}_s + \mathbf {c} \mathbf {U}_s + \mathbf {b}_s\bigr ) \in [0,1]^{T \times d}, \label {eq:suppress-gate} \\ \mathbf {r} &= \mathbf {g}_{\text {express}} \odot (1 - \mathbf {g}_{\text {suppress}}), \label {eq:regulation-combined} \\ \mathbf {h}_{\text {reg}} &= \mathbf {r} \odot \mathbf {h}, \label {eq:regulated-output} \end {align}

where \(\sigma \) is the sigmoid function, \(\mathbf {W}_e, \mathbf {W}_s \in \mathbb {R}^{d \times d}\) and \(\mathbf {U}_e, \mathbf {U}_s \in \mathbb {R}^{d_c \times d}\) are learned weight matrices. The net regulatory signal \(\mathbf {r}\) is the element-wise product of expression and non-suppression: a position is fully active only if it is both expressed and not suppressed.

ATCGATCG

Sparsity and Complexity

When the regulation gate drives many positions to near-zero (\(r_{t,j} \approx 0\)), downstream stages can skip those positions entirely. If a fraction \(s\) of positions survive regulation (i.e., \(r_{t,j} > \epsilon \) for some threshold \(\epsilon \)), then downstream computation scales with \(sT\) rather than \(T\).

7.3.4 Stage 4: Codon Translation

Codon Translation implements DOGMA’s analogue of the genetic code: a differentiable mapping from 64 input codons to 21 output symbols (20 amino acids + 1 stop signal). This stage compresses information through a biologically-motivated bottleneck.

The \(64 \to 21\) Mapping

In biology, three consecutive nucleotides (a codon) encode one amino acid according to the standard genetic code. There are \(4^3 = 64\) possible codons but only 20 amino acids plus a stop signal, so the code is degenerate—multiple codons map to the same amino acid.

DOGMA implements this as a learned soft mapping. Given the regulated hidden state \(\mathbf {h}_{\text {reg}} \in \mathbb {R}^{T \times d}\), consecutive triples of positions are grouped into codons: \begin {equation} \mathbf {c}_k = [\mathbf {h}_{\text {reg}, 3k-2} \| \mathbf {h}_{\text {reg}, 3k-1} \| \mathbf {h}_{\text {reg}, 3k}] \in \mathbb {R}^{3d}, \quad k = 1, \ldots , \lfloor T/3 \rfloor . \label {eq:codon-grouping} \end {equation}

Each codon is translated through a differentiable lookup: \begin {equation} \mathbf {a}_k = \sum _{j=1}^{64} \mathrm {softmax}\bigl (\mathbf {c}_k \mathbf {W}_{\text {codon}}\bigr )_j \cdot \mathbf {E}_{\text {aa}}[j], \quad \mathbf {W}_{\text {codon}} \in \mathbb {R}^{3d \times 64}, \label {eq:codon-translate} \end {equation} where \(\mathbf {E}_{\text {aa}} \in \mathbb {R}^{64 \times d_a}\) is a codon embedding table whose rows are initialized to respect the degeneracy structure of the biological genetic code. Multiple codons that map to the same amino acid are initialized with the same embedding, though they are free to diverge during training.

Why a Compression Bottleneck?

The \(3 \to 1\) compression (every 3 positions become 1) serves several purposes:

1.

Information compression. The regulated representation contains redundancy; grouping triples and projecting to a smaller space removes it.

2.

Sequence length reduction. After codon translation, the effective sequence length is \(T/3\), reducing the cost of the subsequent Strand Attention stage from \(O(T^2 d)\) to \(O((T/3)^2 d) = O(T^2 d / 9)\).

3.

Inductive bias. The degeneracy structure of the genetic code provides a useful inductive bias: synonymous codons (mapping to the same amino acid) should carry similar information, encouraging the model to learn robust representations.

7.3.5 Stage 5: Strand Attention

Strand Attention is DOGMA’s analogue of self-attention, but with attention scores derived from thermodynamic binding energies rather than simple dot products. It operates on the codon-translated representation and computes pairwise interactions using the SantaLucia nearest-neighbor model of DNA hybridization.

Thermodynamic Attention Scores

Standard dot-product attention computes scores as \(\mathbf {q}_i^\top \mathbf {k}_j / \sqrt {d}\). Strand Attention replaces this with a binding energy function inspired by DNA thermodynamics: \begin {equation} \mathrm {Attn}_{\text {DNA}}(\mathbf {Q}, \mathbf {K}, \mathbf {V}) = \mathrm {softmax}\!\left (\frac {\Delta G(\mathbf {Q}, \mathbf {K})}{\tau }\right ) \mathbf {V}, \label {eq:dna-attention} \end {equation} where \(\tau \) is a learnable temperature parameter and the binding energy function \(\Delta G\) is: \begin {equation} \Delta G(\mathbf {q}_i, \mathbf {k}_j) = \underbrace {\mathbf {q}_i^\top \mathbf {W}_{\Delta H} \mathbf {k}_j}_{\text {enthalpy (stacking)}} - \tau _{\text {phys}} \cdot \underbrace {\mathbf {q}_i^\top \mathbf {W}_{\Delta S} \mathbf {k}_j}_{\text {entropy (disorder)}} + \underbrace {\beta (\tau _i, \tau _j)}_{\text {type bias}}. \label {eq:delta-g} \end {equation}

Here \(\mathbf {W}_{\Delta H}, \mathbf {W}_{\Delta S} \in \mathbb {R}^{d \times d}\) are learned matrices corresponding to enthalpy and entropy contributions, \(\tau _{\text {phys}}\) is a learned physical temperature, and \(\beta (\tau _i, \tau _j)\) is the type-pair bias from Equation 7.3.

The SantaLucia Connection

In physical chemistry, the SantaLucia nearest-neighbor model computes the free energy of DNA hybridization as: \begin {equation} \Delta G^{\circ } = \Delta H^{\circ } - T \Delta S^{\circ } = \sum _{i=1}^{n-1} \Delta H^{\circ }_{X_i X_{i+1} / Y_i Y_{i+1}} - T \sum _{i=1}^{n-1} \Delta S^{\circ }_{X_i X_{i+1} / Y_i Y_{i+1}}, \label {eq:ch7-santalucia} \end {equation} where the sums are over dinucleotide pairs and \(T\) is the physical temperature in Kelvin. DOGMA’s Strand Attention learns analogous dinucleotide-like parameters in a high-dimensional embedding space. The enthalpy matrix \(\mathbf {W}_{\Delta H}\) captures context-dependent binding strength; the entropy matrix \(\mathbf {W}_{\Delta S}\) captures the disorder cost of binding.

ATCGATCG

Complexity

7.3.6 Stage 6: Epigenetic + Allosteric Mechanisms

The final stage combines two complementary memory mechanisms: epigenetic memory (persistent, slowly-evolving state) and allosteric broadcast (fast, targeted signal distribution). Together they provide DOGMA with both long-term and short-term context integration.

Epigenetic Memory: \(O(TM)\) Persistent State

Epigenetic memory maintains a bank of \(M\) memory slots \(\mathbf {m}_1, \ldots , \mathbf {m}_M \in \mathbb {R}^{d_m}\) that persist across time steps and are updated incrementally: \begin {align} \mathbf {u}_j &= \sigma \bigl (\mathbf {h}_{\text {attn}} \mathbf {W}_u \mathbf {m}_j^\top \bigr ), && \text {(update gate for slot } j\text {)} \label {eq:epi-update-gate} \\ \tilde {\mathbf {m}}_j &= \tanh \bigl (\mathbf {h}_{\text {attn}} \mathbf {W}_m + \mathbf {m}_j \mathbf {W}_r\bigr ), && \text {(candidate update)} \label {eq:epi-candidate} \\ \mathbf {m}_j' &= (1 - \mathbf {u}_j) \odot \mathbf {m}_j + \mathbf {u}_j \odot \tilde {\mathbf {m}}_j, && \text {(gated update)} \label {eq:epi-gated} \end {align}

where \(\mathbf {h}_{\text {attn}} \in \mathbb {R}^{T' \times d}\) is the output of Strand Attention. The read operation retrieves information from memory via attention: \begin {equation} \mathbf {v}_{\text {epi}} = \mathrm {softmax}\bigl (\mathbf {h}_{\text {attn}} \mathbf {W}_q^{\text {epi}} (\mathbf {M} \mathbf {W}_k^{\text {epi}})^\top \bigr ) \mathbf {M} \mathbf {W}_v^{\text {epi}}, \label {eq:epi-read} \end {equation} where \(\mathbf {M} = [\mathbf {m}_1; \ldots ; \mathbf {m}_M] \in \mathbb {R}^{M \times d_m}\).

The complexity of epigenetic memory operations is \(O(T' \cdot M \cdot d_m)\), which is linear in sequence length when \(M\) is fixed.

Allosteric Broadcast: \(O(Tk)\) Signal Distribution

Allosteric regulation, in biology, is the modulation of a protein’s activity by binding at a site other than the active site. In DOGMA, allosteric broadcast selects \(k\) “broadcast sites” and distributes their signals to all other positions: \begin {align} \text {sites} &= \mathrm {TopK}\bigl (\mathbf {h}_{\text {attn}} \mathbf {w}_{\text {select}},\; k\bigr ), \label {eq:allo-select} \\ \mathbf {s}_{\text {broadcast}} &= \frac {1}{k}\sum _{j \in \text {sites}} \mathbf {h}_{\text {attn}, j}, \label {eq:allo-mean} \\ \mathbf {h}_{\text {final}, t} &= \mathbf {h}_{\text {attn}, t} + \mathbf {v}_{\text {epi}, t} + \mathrm {MLP}(\mathbf {s}_{\text {broadcast}}), \label {eq:allo-combine} \end {align}

where \(\mathbf {w}_{\text {select}} \in \mathbb {R}^d\) is a learned selection vector and \(\mathrm {TopK}\) selects the \(k\) positions with highest selection scores. The broadcast signal \(\mathbf {s}_{\text {broadcast}}\) is added to all positions, providing global context from the most salient positions.

The complexity is \(O(T' \cdot k \cdot d)\) for computing the broadcast, where \(k \ll T'\).

ATCGATCG

7.4 The Expression Folder

After the six-stage pipeline produces the final hidden representation \(\mathbf {h}_{\text {final}}\), DOGMA must convert this internal representation into the output space—a process we call the Expression Folder, by analogy with protein folding.

In protein folding, a linear amino acid chain (the primary structure) folds into a complex three-dimensional structure (the tertiary structure) that determines the protein’s function. Similarly, the Expression Folder transforms the linear sequence of hidden states into a structured output representation.

\begin {equation} \Phi = \mathrm {ExpressionFolder}(\mathbf {h}_{\text {final}}) = \mathrm {MLP}_{\text {fold}}\bigl (\mathrm {LayerNorm}(\mathbf {h}_{\text {final}})\bigr ), \label {eq:expression-folder} \end {equation}

where \(\mathrm {MLP}_{\text {fold}}\) is a two-layer feed-forward network with GELU activation: \begin {equation} \mathrm {MLP}_{\text {fold}}(\mathbf {z}) = \mathbf {W}_2 \cdot \mathrm {GELU}(\mathbf {W}_1 \mathbf {z} + \mathbf {b}_1) + \mathbf {b}_2. \label {eq:fold-mlp} \end {equation}

The output \(\Phi \) is not text. It is a phenotype representation—a structured semantic object that can be realized into different modalities:

• Natural language: A language head applies a linear projection followed by softmax to produce token probabilities.
• Code: A syntax-aware head produces abstract syntax tree nodes.
• JSON/Schema: A structured output head generates key-value pairs conforming to a schema.
• Action plans: A tool-use head produces function calls and arguments.
• 3D structure: A TetraMesh head produces spatial coordinates (Chapter 10).

7.5 Full Architecture Diagram

Figure 7.7 presents the complete DOGMA architecture, showing both the six-stage pipeline (left) and the detailed DNA Computing Block structure (right).

ATCGATCG

7.6 The DogmaBlock: Complete Forward Pass

Algorithm 1 presents the complete forward pass of a single DogmaBlock as pseudocode.

7.7 Architecture Comparison: DOGMA vs. Alternatives

Table 7.5 provides a detailed comparison of DOGMA with three major competing architectures: the Transformer [Vaswani et al., 2017a], Mamba (a state-space model) [Gu and Dao, 2023], and RWKV (a linear-attention RNN).

7.7.1 Complexity Summary

To summarize the per-layer complexity of a single DogmaBlock:

\begin {align} \text {Stage 1 (TetraMemory):} \quad & O(T \cdot d), \\ \text {Stage 2 (DNA Block):} \quad & O(T \cdot d) \quad \text {(Parallel Scan dominates)}, \\ \text {Stage 3 (Regulation):} \quad & O(T \cdot d), \\ \text {Stage 4 (Codon Translation):} \quad & O(T \cdot d), \\ \text {Stage 5 (Strand Attention):} \quad & O(T'^2 \cdot d) \quad \text {where } T' = T/3, \\ \text {Stage 6 (Epi + Allo):} \quad & O(T' \cdot M \cdot d_m + T' \cdot k \cdot d). \end {align}

The bottleneck is Stage 5 (Strand Attention) at \(O(T'^2 d) = O(T^2 d / 9)\). This is \(9\times \) cheaper than standard self-attention on the full sequence, and further reduced when regulation sparsity is applied. When \(T' \leq T/3\) after regulation-induced sparsity, the effective complexity can approach linear.

7.8 Worked Example: Tracing Data Through All Six Stages

To make the pipeline concrete, let us trace a small example through all six stages. Suppose we have an input sequence of \(T = 12\) tokens with embedding dimension \(d = 8\) (vastly simplified from production dimensions, but sufficient to illustrate the data flow).

7.9 The Evolutor State

The complete state of a DOGMA system at time \(t\) is captured by the Evolutor state:

The Evolutor state encapsulates everything needed to reproduce the system’s behavior and continue its evolution. It is the DOGMA analogue of a cell’s complete state: genome + epigenome + cellular memory.

7.10 Why DOGMA Is Not Just a Modified Transformer

It is tempting to view DOGMA as “a transformer with extra steps.” This misses the fundamental architectural difference. Consider three levels of distinction:

1.

Ontological: In a transformer, the primary computational object is the token sequence. In DOGMA, the primary computational object is the genome—a persistent, typed, structured state from which tokens are merely one possible output.

2.

Control flow: In a transformer, all tokens interact uniformly through attention. In DOGMA, interaction is regulated—controlled by context-dependent activation maps that determine which regions participate.

3.

Evolutionary: A transformer’s parameters are fixed after training. A DOGMA genome can be evolved—modified through mutation, selection, and synthesis—enabling open-ended improvement without full retraining.

The relationship between DOGMA and transformers is analogous to the relationship between cells and chemical reactions. A cell contains chemical reactions, but a cell is not merely “chemistry with extra steps.” The organizational structure—membranes, organelles, regulatory networks, genetic programs—is what makes a cell fundamentally different from a random mixture of the same chemicals.

Similarly, DOGMA may contain attention mechanisms (within its Strand Attention stage or DNA Computing Block modules), but the organizational structure—TetraMemory, four-path DNA computing, regulation gating, codon translation, thermodynamic attention, epigenetic memory—is what makes it architecturally distinct.

7.11 Exercises

7.1.

[Fundamentals] For each of the six DOGMA stages (TetraMemory, DNA Computing Block, Regulation Gate, Codon Translation, Strand Attention, Epigenetic+Allosteric), identify the corresponding step in the biological Central Dogma and explain the mapping.

7.2.

[Analysis] Consider a DOGMA system processing a sequence of \(T = 4096\) tokens with embedding dimension \(d = 512\).

(a)

How many tetrahedra does Stage 1 create?

(b)

What is the sequence length after Stage 4 (Codon Translation)?

(c)

Compare the cost of Stage 5 Strand Attention (on the compressed sequence) with standard self-attention on the original 4096 tokens.

7.3.

[Design] The DNA Computing Block uses a learned softmax gate to merge four path outputs. Design an experiment to test whether all four paths are necessary. Describe what you would measure when ablating each path individually, and predict which path’s removal would cause the largest performance drop for (a) a language modeling task, (b) a protein structure prediction task.

7.4.

[Coding] Implement the four-equation Regulation Gate defined above in PyTorch. Create a test that verifies:

(a)

The output has the same shape as the input.

(b)

When the suppress gate outputs all ones, the regulated output is all zeros.

(c)

When both gates output 0.5, the net regulation is \(0.5 \times 0.5 = 0.25\) per element.

7.5.

[Theory] Prove that if the regulation stage produces an activation map with at most \(k\) nonzero entries (hard sparsity), then the computational cost of Stage 5 Strand Attention is \(O(k'^2 d)\) where \(k' = k/3\), rather than \(O((T/3)^2 d)\).

7.6.

[Research] Read about Mixture-of-Experts (MoE) architectures [Brown et al., 2020]. Compare the MoE routing mechanism with DOGMA’s Regulation Gate. What are the key similarities and differences? Which approach offers better interpretability, and why?

7.7.

[Analysis] The Codon Translation stage uses a \(64 \to 21\) soft mapping initialized from the biological genetic code. Discuss:

(a)

Why is degeneracy (multiple codons \(\to \) same amino acid) beneficial for robustness?

(b)

Should the codon embeddings be frozen at their biological initialization or fine-tuned? Argue both sides.

7.8.

[Advanced] The Evolutor state \(\mathcal {E}_t = (\Gamma _t, H_t, \mathcal {A}_t, \mathcal {L}_t)\) maintains lineage history \(\mathcal {L}_t\). Discuss the computational and memory tradeoffs of maintaining full lineage vs. compressed lineage (keeping only the most recent \(m\) checkpoints). Under what conditions would compressed lineage lose important information?

7.9.

[Worked Example] Trace a sequence of \(T = 24\) tokens through the six-stage pipeline, showing tensor shapes at each stage. How many codons are produced? What is the final phenotype length?

7.10.

[Comparison] Using Table 7.5, explain why DOGMA is better suited than Mamba for tasks requiring bidirectional context, and why Mamba might be preferred for pure autoregressive generation at very long sequence lengths.

7.12 Key Takeaways

Suggested Reading

• Crick [1970]: The original Central Dogma, DOGMA’s biological inspiration.
• Vaswani et al. [2017a]: The transformer architecture, for comparison.
• Gu and Dao [2023]: State-space models, related to DOGMA’s ParallelScan path.
• Zhang and Seelig [2011]: DNA strand displacement, the basis for DOGMA’s Strand Displacement path.
• ENCODE Project Consortium [2012]: The ENCODE project on gene regulation and expression sparsity.
• Chapters 8–11 of this book: detailed treatments of DOGMA’s subsystems.