The DOGMA Architecture
TetraData Structure and TetraMemory
Computer graphics has long relied on triangles as the fundamental rendering primitive. Triangle meshes approximate surfaces efficiently and map naturally to GPU hardware. But triangles are inherently two-dimensional : they describe surfaces, not volumes. For a computational architecture inspired by three-dimensional...
DOGMA Pipeline
Six local stages replace one monolithic attention operation
Advance through the canonical non-transformer computation from sequence to realized output.
Encode
HelixHash creates structural sequence representations.
G = H(sequence)Computer graphics has long relied on triangles as the fundamental rendering primitive. Triangle meshes approximate surfaces efficiently and map naturally to GPU hardware. But triangles are inherently two-dimensional: they describe surfaces, not volumes. For a computational architecture inspired by three-dimensional molecular biology—where proteins fold into globular structures, DNA coils into superhelical conformations, and tetrahedral carbon bonds define organic chemistry—a volumetric primitive is needed.
This chapter introduces the TetraData Structure (TDS): a computational framework built on tetrahedra rather than triangles. We begin with a patient, step-by-step construction that builds a tetrahedron one vertex at a time, then show how tetrahedra link together into chains that map directly onto DNA sequences. We then develop TetraMemory, a structured memory system that replaces self-attention in DOGMA with local tetrahedral interactions that compose into global computation.
10.1 Building a Tetrahedron from Scratch
Before defining the TetraData Structure formally, let us build our fundamental primitive—the tetrahedron—one vertex at a time. This construction will reveal why the tetrahedron is the minimal solid that carries volume and chirality, and why no simpler primitive will do.
10.1.1 Step 0: A Single Vertex (0D)
We begin with nothing: an empty space. We place a single point \(v_0\) and assign it a value (say, a DNA base, a token embedding, or simply a label).
A single vertex is a 0-dimensional object. It can store a value—but it encodes no relationships. There is no notion of distance, connection, area, or volume. In a sequence model, a single vertex is a single token in isolation, with no context whatsoever.
Remark 10.1. In graph theory terms, we have the graph \(K_1\): one vertex, zero edges. The adjacency matrix is the \(1 \times 1\) zero matrix. This is the least informative data structure possible.
10.1.2 Step 1: Add a Second Vertex (1D — an Edge)
Now add a second vertex \(v_1\) and connect it to \(v_0\) with an edge \(e_{01}\).
An edge is a 1-dimensional object. It encodes exactly one relationship: the pairwise interaction between \(v_0\) and \(v_1\). The edge can carry a weight \(w_{01} \in \mathbb {R}\) representing the strength or character of that interaction.
In the language of neural networks, this is a single attention weight: how much does token \(v_0\) attend to token \(v_1\)? One edge captures one pairwise interaction. This is the graph \(K_2\): the complete graph on 2 vertices.
Remark 10.2. An edge has no area, no volume, and no handedness. You cannot distinguish a “left-handed” line segment from a “right-handed” one. An edge is symmetric under reflection.
10.1.3 Step 2: Add a Third Vertex (2D — a Triangle)
Add a third vertex \(v_2\) and connect it to both existing vertices. We now have three vertices and three edges forming a triangle.
A triangle is a 2-dimensional object. It has area but no volume. Three edges encode all \(\binom {3}{2} = 3\) pairwise interactions among the three vertices, forming the complete graph \(K_3\).
This is the primitive used in computer graphics: triangle meshes tile surfaces. GPUs are optimized for rasterizing triangles. But triangles have two critical limitations for our purposes:
- 1.
- No volume. A triangle is flat. It cannot enclose space. It describes a boundary, not an interior.
- 2.
- No chirality. A triangle in 2D can be reflected into its mirror image by a rotation. In 3D, a triangle embedded in a plane does have two “sides,” but this orientation is not an intrinsic property—it depends on the choice of normal direction.
Why Triangles Are Not Enough
In molecular biology, chirality is not optional—it is essential. All naturally occurring amino acids are L-amino acids (left-handed). DNA’s double helix is always right-handed (B-form). Enzymes distinguish between mirror-image molecules with exquisite specificity. A computational primitive that cannot represent chirality is fundamentally incapable of encoding molecular structure faithfully. A triangle mesh can approximate the surface of a protein, but it cannot capture the handed internal packing that determines function.
10.1.4 Step 3: Add a Fourth Vertex (3D — the Tetrahedron!)
Now the crucial step. Add a fourth vertex \(v_3\) that is not in the plane of the first three, and connect it to all existing vertices. We obtain a tetrahedron: the simplest possible solid.
A tetrahedron is the first object in our construction that possesses:
- Volume: It encloses a region of 3D space. The signed volume is computed from the \(3 \times 3\) determinant of the edge vectors.
- Chirality: The sign of the determinant (\(+1\) or \(-1\)) defines a handedness that cannot be changed by any rotation—only by reflection.
- Complete pairwise interactions: Six edges encode all \(\binom {4}{2} = 6\) pairwise relationships. This is the complete graph \(K_4\).
- Four faces: Each face is a triangle that can serve as an interface for connecting to another tetrahedron.
Definition 10.1 (Tetrahedron). A tetrahedron \(\mathcal {T}\) in \(\mathbb {R}^3\) is the convex hull of four non-coplanar points \(\mathbf {v}_0, \mathbf {v}_1, \mathbf {v}_2, \mathbf {v}_3\): \begin {equation} \mathcal {T} = \left \{ \sum _{i=0}^{3} \lambda _i \mathbf {v}_i \;\middle |\; \lambda _i \geq 0, \; \sum _{i=0}^{3} \lambda _i = 1 \right \}. \end {equation} The signed volume is: \begin {equation} V_{\text {signed}}(\mathcal {T}) = \frac {1}{6} \det \begin {pmatrix} \mathbf {v}_1 - \mathbf {v}_0 & \mathbf {v}_2 - \mathbf {v}_0 & \mathbf {v}_3 - \mathbf {v}_0 \end {pmatrix}, \label {eq:signed-volume} \end {equation} and the chirality is: \begin {equation} \chi (\mathcal {T}) = \operatorname {sign}\!\big (V_{\text {signed}}(\mathcal {T})\big ) \in \{+1, -1\}. \end {equation}
10.1.5 The Construction at a Glance
The following figure summarizes the four steps side by side, highlighting what each new vertex contributes:
Example 10.1 (Counting Combinatorics). At each step \(k\) (adding vertex \(v_k\) to the existing structure), the new vertex connects to all \(k\) existing vertices, adding exactly \(k\) new edges:
| Step | Vertices | New edges | Total edges | Graph |
| 0 | 1 | 0 | 0 | \(K_1\) |
| 1 | 2 | 1 | 1 | \(K_2\) |
| 2 | 3 | 2 | 3 | \(K_3\) |
| 3 | 4 | 3 | 6 | \(K_4\) |
In general, the complete graph \(K_n\) has \(\binom {n}{2} = \frac {n(n-1)}{2}\) edges. For \(K_4\), this gives \(\binom {4}{2} = 6\) edges—exactly the six edges of a tetrahedron.
10.2 Why Tetrahedra? — A Deeper Look
With the construction complete, we can appreciate why DOGMA chooses the tetrahedron over simpler primitives.
10.2.1 Tetrahedra vs. Triangles: A Detailed Comparison
| Property | Triangle Mesh | Tetrahedral Mesh (TDS) |
| Dimension | 2D surface in 3D | 3D volume |
| Vertices per element | 3 | 4 |
| Edges per element | 3 (\(K_3\)) | 6 (\(K_4\)) — \(2\times \) more interactions |
| Faces per element | 1 | 4 triangular faces |
| Interior | None (boundary only) | Enclosed volume |
| Chirality | None | \(\pm 1\) handedness |
| Adjacency | Edge-sharing | Face-sharing (richer interface) |
| DNA mapping | — | 4 vertices \(\leftrightarrow \) {A, C, G, T} |
| DOF per element | 9 (positions only) | 36 (positions + edges + bases + vol + chir) |
| Info. density | 3 edges | 6 edges — \(4\times \) information density |
The last row deserves emphasis. A triangle encodes 3 pairwise interactions. A tetrahedron encodes 6. But the tetrahedron has only one more vertex. In terms of pairwise interactions per vertex, the triangle achieves \(3/3 = 1\) edge per vertex, while the tetrahedron achieves \(6/4 = 1.5\) edges per vertex—a 50% improvement in information density.
10.2.2 The Biological Justification
The biological case for tetrahedra is compelling:
- Carbon chemistry: The carbon atom—the backbone of all organic molecules—forms four bonds arranged tetrahedrally (\(sp^3\) hybridization). Methane (CH\(_4\)) is a perfect tetrahedron. Protein backbone geometry is fundamentally tetrahedral at each \(C_\alpha \) center.
- DNA base count: DNA uses exactly four bases—A, C, G, T. A tetrahedron has exactly four vertices. This numerical coincidence becomes the foundation of DOGMA’s data structure: each vertex of a tetrahedron is assigned one DNA base.
- DNA base stacking: Adjacent base pairs in the double helix create local geometries that are naturally described by tetrahedral coordinates.
- Protein structure: Alpha-carbon (\(C_\alpha \)) geometry in protein backbones forms tetrahedral arrangements that are native to TDS representation.
- Chirality in biology: All naturally occurring amino acids are L-amino acids; DNA forms a right-handed helix. Chirality is information that must be preserved, and only volumetric primitives (3D and above) can encode it.
Four Bases, Four Vertices
The fact that DNA uses exactly four nucleotide bases is one of the most fundamental constraints in molecular biology. This four-fold alphabet maps perfectly onto the four vertices of a tetrahedron. This is not merely a convenient analogy: it is the mathematical foundation of the TetraData Structure. Each vertex carries a base type, and the six edges encode all pairwise interactions among those bases—including the Watson–Crick pairings (A–T and G–C) as geometrically distinguished edges.
10.3 Linking Tetrahedra — The LinkedTetraChain
A single tetrahedron encodes the interactions among four elements. But real sequences are longer than four tokens. How do we handle sequences of arbitrary length? The answer is linking: we chain tetrahedra together by sharing triangular faces, exactly as the PowerPoint construction shows.
10.3.1 Adding Vertex 4: The Second Tetrahedron
We continue the construction from Section 10.1. Having built the first tetrahedron \(\mathcal {T}_1 = (v_0, v_1, v_2, v_3)\), we now add vertex \(v_4\) and connect it to the three most recent vertices: \(v_1, v_2, v_3\). This creates a second tetrahedron: \[ \mathcal {T}_2 = (v_1, v_2, v_3, v_4). \]
The two tetrahedra share the triangular face \((v_1, v_2, v_3)\). The new vertex \(v_4\) adds exactly 3 new edges: \(e_{14}, e_{24}, e_{34}\).
Remark 10.3. The shared face \((v_1, v_2, v_3)\) contains three edges and three vertices that belong to both tetrahedra. This is a much richer interface than the edge-sharing used by triangle meshes: a shared face transmits three pairwise relationships, while a shared edge transmits only one.
10.3.2 Adding Vertex 5: The Third Tetrahedron
Continuing the pattern, vertex \(v_5\) connects to \(v_2, v_3, v_4\), creating: \[ \mathcal {T}_3 = (v_2, v_3, v_4, v_5), \] which shares face \((v_2, v_3, v_4)\) with \(\mathcal {T}_2\).
10.3.3 The General Pattern: LinkedTetraChain
The pattern is now clear. At each step, a new vertex \(v_{k+3}\) connects to the three most recent vertices \((v_k, v_{k+1}, v_{k+2})\), forming tetrahedron: \[ \mathcal {T}_{k+1} = (v_k, v_{k+1}, v_{k+2}, v_{k+3}), \] which shares face \((v_k, v_{k+1}, v_{k+2})\) with the previous tetrahedron \(\mathcal {T}_k = (v_{k-1}, v_k, v_{k+1}, v_{k+2})\).
Definition 10.2 (LinkedTetraChain). A LinkedTetraChain of length \(n\) is an ordered sequence of tetrahedra \((\mathcal {T}_1, \mathcal {T}_2, \ldots , \mathcal {T}_n)\) constructed from a vertex sequence \((v_0, v_1, \ldots , v_{n+2})\) where: \begin {equation} \mathcal {T}_k = (v_{k-1}, v_k, v_{k+1}, v_{k+2}), \quad k = 1, 2, \ldots , n, \end {equation} and consecutive tetrahedra \(\mathcal {T}_k\) and \(\mathcal {T}_{k+1}\) share the triangular face \((v_k, v_{k+1}, v_{k+2})\).
Proposition 10.1 (Edge Count Formula). A LinkedTetraChain of \(n\) tetrahedra built from \(n + 3\) vertices has exactly \(3n + 3\) edges.
Proof. The first tetrahedron \(\mathcal {T}_1\) contributes \(\binom {4}{2} = 6\) edges. Each subsequent tetrahedron \(\mathcal {T}_k\) for \(k \geq 2\) adds one new vertex that connects to 3 existing vertices, contributing exactly 3 new edges. Thus the total is \(6 + 3(n-1) = 3n + 3\). □
The following diagram shows a chain of four linked tetrahedra:
10.3.4 Growth Table
The following table tracks the combinatorics as we add vertices:
| Vertex | Total \(|V|\) | New edges | Total \(|E|\) | Total \(|\mathcal {T}|\) | Shared face | Graph |
| \(v_0\) | 1 | 0 | 0 | 0 | — | \(K_1\) |
| \(v_1\) | 2 | 1 | 1 | 0 | — | \(K_2\) |
| \(v_2\) | 3 | 2 | 3 | 0 | — | \(K_3\) |
| \(v_3\) | 4 | 3 | 6 | 1 (\(\mathcal {T}_1\)) | — | \(K_4\) |
| \(v_4\) | 5 | 3 | 9 | 2 (\(\mathcal {T}_2\)) | \((v_1,v_2,v_3)\) | — |
| \(v_5\) | 6 | 3 | 12 | 3 (\(\mathcal {T}_3\)) | \((v_2,v_3,v_4)\) | — |
| \(v_6\) | 7 | 3 | 15 | 4 (\(\mathcal {T}_4\)) | \((v_3,v_4,v_5)\) | — |
| \(v_{k+3}\) | \(k{+}4\) | 3 | \(3(k{+}1){+}3\) | \(k{+}1\) | \((v_k,v_{k+1},v_{k+2})\) | — |
Notice that after the first tetrahedron is formed, each new vertex adds exactly 3 new edges and exactly 1 new tetrahedron. The growth is perfectly linear.
10.4 DNA Mapping — The Key Insight
We now arrive at the central insight of the TetraData Structure: the mapping between tetrahedra and DNA sequences.
10.4.1 Four Vertices, Four Bases
A tetrahedron has four vertices. DNA has four bases: adenine (A), cytosine (C), guanine (G), and thymine (T). In a LinkedTetraChain, each vertex is assigned one base type from \(\{\mathsf {A}, \mathsf {C}, \mathsf {G}, \mathsf {T}\}\).
TetraData–DNA Correspondence
In the TetraData Structure, each vertex \(v_i\) carries a base assignment \(b_i \in \{\mathsf {A}, \mathsf {C}, \mathsf {G}, \mathsf {T}\}\). A tetrahedron \(\mathcal {T}_k = (v_{k-1}, v_k, v_{k+1}, v_{k+2})\) with base assignments \((b_{k-1}, b_k, b_{k+1}, b_{k+2})\) represents a window of four consecutive bases in a DNA-like sequence.
10.4.2 Worked Example: Mapping \(5'\)-ACGTACGT-\(3'\)
Consider the DNA sequence \(5'\)-ACGTACGT-\(3'\), which has 8 bases. We assign these to vertices \(v_0\) through \(v_7\):
| Vertex | \(v_0\) | \(v_1\) | \(v_2\) | \(v_3\) | \(v_4\) | \(v_5\) | \(v_6\) | \(v_7\) |
| Base | A | C | G | T | A | C | G | T |
This sequence of 8 vertices produces a LinkedTetraChain of \(8 - 3 = 5\) tetrahedra:
| Tetra | Vertices | Bases | Shared face with prev. |
| \(\mathcal {T}_1\) | \((v_0, v_1, v_2, v_3)\) | (A, C, G, T) | — |
| \(\mathcal {T}_2\) | \((v_1, v_2, v_3, v_4)\) | (C, G, T, A) | \((v_1, v_2, v_3) = \) (C, G, T) |
| \(\mathcal {T}_3\) | \((v_2, v_3, v_4, v_5)\) | (G, T, A, C) | \((v_2, v_3, v_4) = \) (G, T, A) |
| \(\mathcal {T}_4\) | \((v_3, v_4, v_5, v_6)\) | (T, A, C, G) | \((v_3, v_4, v_5) = \) (T, A, C) |
| \(\mathcal {T}_5\) | \((v_4, v_5, v_6, v_7)\) | (A, C, G, T) | \((v_4, v_5, v_6) = \) (A, C, G) |
Notice the structure: each tetrahedron is a sliding window of width 4 that advances by one base at a time. The overlap of 3 bases ensures that every pair of adjacent bases is captured within at least one tetrahedron, and in fact every pair of bases separated by at most 2 positions is captured.
10.4.3 Why Not Triangles? An Information Density Argument
If we used triangles instead of tetrahedra, our sliding window would be width 3 with overlap 2. Each triangle would be a \(K_3\) with only 3 edges. Let us compare the two approaches for the same 8-base sequence:
| Triangle mesh (\(K_3\)) | Tetra mesh (\(K_4\)) | |
| Window width | 3 | 4 |
| Overlap | 2 | 3 |
| Elements for 8 bases | 6 triangles | 5 tetrahedra |
| Edges per element | 3 | 6 |
| Total unique edges | 7 | 10 |
| Pairwise interactions | 3 per element | 6 per element |
| Has volume? | No | Yes |
| Has chirality? | No | Yes |
The tetrahedron captures \(2\times \) the pairwise interactions per element, while also encoding volume and chirality. For the same sequence, the tetrahedral representation is strictly richer.
10.4.4 The Six Edges: Pairwise Base Interactions
Within each tetrahedron \(\mathcal {T}_k = (b_{k-1}, b_k, b_{k+1}, b_{k+2})\), the six edges encode all pairwise interactions among the four bases:
The six edges naturally partition into:
- 2 Watson–Crick edges: A–T and C–G. These are the complementary base pairings that form hydrogen bonds in the double helix.
- 4 non-Watson–Crick edges: A–C, A–G, C–T, G–T. These encode stacking interactions, non-canonical base pairing, and other local sequence features.
Each edge carries a learnable weight \(w_{ij} \in \mathbb {R}\) that represents the strength and character of the pairwise interaction. These weights are analogous to attention scores in a transformer, but they are local (within a single tetrahedron) and structured (constrained by the tetrahedral geometry).
10.5 The TetraData Structure (TDS) — Formal Definition
We now give the complete formal definition of the TetraData Structure, incorporating everything we have built up.
Definition 10.3 (TetraData Structure). A TetraData Structure (TDS) element is defined by: \begin {equation} \mathcal {T}_i = (\mathbf {V}_i, \mathbf {E}_i, \mathbf {B}_i, v_i, \chi _i), \end {equation} where:
- \(\mathbf {V}_i \in \mathbb {R}^{4 \times 3}\): four vertex positions in 3D space.
- \(\mathbf {E}_i \in \mathbb {R}^{6}\): six edge weights (forming the complete graph \(K_4\)).
- \(\mathbf {B}_i \in \{\mathsf {A}, \mathsf {C}, \mathsf {G}, \mathsf {T}\}^4\): base type assignments for each vertex.
- \(v_i \in \mathbb {R}\): enclosed volume (computed from vertex positions).
- \(\chi _i \in \{+1, -1\}\): chirality (handedness), defined by the sign of the determinant.
10.5.1 Chirality
Chirality—the property of non-superimposability on a mirror image—is fundamental to molecular biology. Amino acids are left-handed (\(L\)-amino acids), sugars are right-handed (\(D\)-sugars), and the DNA double helix is right-handed. TDS preserves chirality through the sign of the vertex determinant:
\begin {equation} \chi _i = \text {sign}\left (\det \begin {pmatrix} \mathbf {v}_1 - \mathbf {v}_0 & \mathbf {v}_2 - \mathbf {v}_0 & \mathbf {v}_3 - \mathbf {v}_0 \end {pmatrix}\right ). \end {equation}
Chirality-preserving transformations are those that maintain the sign of \(\chi \). This constraint ensures that biological structures retain their handedness through computational operations.
Example 10.2 (Computing Chirality). Consider a tetrahedron with vertices: \[ \mathbf {v}_0 = (0,0,0), \quad \mathbf {v}_1 = (1,0,0), \quad \mathbf {v}_2 = (0,1,0), \quad \mathbf {v}_3 = (0,0,1). \] The edge matrix is: \[ M = \begin {pmatrix} 1-0 & 0-0 & 0-0 \\ 0-0 & 1-0 & 0-0 \\ 0-0 & 0-0 & 1-0 \end {pmatrix} = \begin {pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end {pmatrix} = I_3. \] We have \(\det (M) = +1\), so \(\chi = +1\) (right-handed). The signed volume is \(V_{\text {signed}} = \frac {1}{6}(+1) = +\frac {1}{6}\).
Now reflect \(\mathbf {v}_3\) across the \(xy\)-plane: \(\mathbf {v}_3' = (0,0,-1)\). The new determinant is \(-1\), giving \(\chi = -1\) (left-handed). Reflection reverses chirality, as expected.
10.5.2 Face-Sharing Adjacency
Adjacent tetrahedra share triangular faces, forming tetrahedral chains or tetrahedral meshes. This face-sharing adjacency defines a natural graph structure:
Definition 10.4 (Tetrahedral Mesh). A tetrahedral mesh is a collection of tetrahedra \(\{\mathcal {T}_1, \ldots , \mathcal {T}_M\}\) where adjacent elements share triangular faces. The dual graph has one node per tetrahedron and edges between face-sharing neighbors.
Face-sharing adjacency is richer than the edge-sharing adjacency of triangle meshes: each tetrahedron has exactly four potential neighbors (one per face), creating a bounded-degree graph that enables efficient local computation.
10.6 36 Degrees of Freedom
Each TDS element carries a rich set of information. Let us count the degrees of freedom precisely.
10.6.1 Detailed DOF Breakdown
| DOF | Component | Count | Description |
| 1–12 | Vertex positions \(\mathbf {V}_i \in \mathbb {R}^{4 \times 3}\) | 12 | 4 vertices \(\times \) 3 coordinates each |
| 13–18 | Edge weights \(\mathbf {E}_i \in \mathbb {R}^6\) | 6 | All \(\binom {4}{2} = 6\) pairwise interactions |
| 19–34 | Base encoding \(\text {onehot}(\mathbf {B}_i) \in \mathbb {R}^{4 \times 4}\) | 16 | 4 vertices \(\times \) 4-dim one-hot encoding |
| 35 | Volume \(v_i \in \mathbb {R}\) | 1 | Signed volume of the tetrahedron |
| 36 | Chirality \(\chi _i \in \{+1,-1\}\) | 1 | Handedness (sign of the determinant) |
| Total | 36 |
10.6.2 Comparison: Triangle vs. Tetrahedron DOF
| Component | Triangle | Tetrahedron |
| Vertex positions | \(3 \times 3 = 9\) | \(4 \times 3 = 12\) |
| Edge weights | \(\binom {3}{2} = 3\) | \(\binom {4}{2} = 6\) |
| Base encoding | — | \(4 \times 4 = 16\) |
| Volume | 0 (area only) | 1 |
| Chirality | 0 (no handedness) | 1 |
| Total DOF | 12 | 36 |
| Ratio | \(36/12 = 3\times \) richer
| |
If we count only the positional DOF (no base encoding), the triangle has 9 DOF and the tetrahedron has \(12 + 6 + 1 + 1 = 20\) DOF—still more than \(2\times \) richer. The base encoding adds another 16 DOF that are entirely absent in triangle meshes.
10.6.3 TetraTokenization
For foundation model integration, each TDS element is serialized into a 36-dimensional vector:
\begin {equation} \text {TetraToken}(\mathcal {T}_i) = [\underbrace {\mathbf {V}_i^{\text {flat}}}_{12} \;|\; \underbrace {\mathbf {E}_i}_{6} \;|\; \underbrace {\text {onehot}(\mathbf {B}_i)}_{16} \;|\; \underbrace {v_i}_{1} \;|\; \underbrace {\chi _i}_{1}] \in \mathbb {R}^{36}. \label {eq:tetra-token} \end {equation}
This tokenization enables DOGMA foundation models to ingest and produce 3D structural data alongside text, code, and genomic sequences.
TetraToken Design Study
In the proposed multimodal DOGMA design, the 36-dimensional TetraToken is projected to the model’s hidden dimension \(d_{\text {model}}\) via a learned linear layer: \[ \mathbf {h}_i = \mathbf {W}_{\text {proj}} \cdot \text {TetraToken}(\mathcal {T}_i) + \mathbf {b}_{\text {proj}}, \quad \mathbf {W}_{\text {proj}} \in \mathbb {R}^{d_{\text {model}} \times 36}. \] The proposed reverse direction uses a symmetric linear projection from \(d_{\text {model}}\) back to 36 dimensions, followed by extraction of vertex positions, edge weights, base assignments, volume, and chirality. This path is not part of the measured baseline in Chapter 14.
10.7 TetraMemory: Structured Memory for DOGMA
TetraMemory is DOGMA’s alternative to self-attention. Instead of computing pairwise interactions between all positions (quadratic cost), TetraMemory organizes state elements into a tetrahedral mesh where interactions are local but compose into global computation through propagation.
10.7.1 The Quadratic Attention Problem
Standard self-attention computes: \begin {equation} \text {Attention}(\mathbf {Q}, \mathbf {K}, \mathbf {V}) = \text {softmax}\left (\frac {\mathbf {Q}\mathbf {K}^\top }{\sqrt {d_k}}\right )\mathbf {V}, \quad \text {cost: } O(T^2 d). \end {equation}
For a sequence of length \(T = 100{,}000\) (a modest genomic sequence), the attention matrix has \(10^{10}\) entries—impractical for both memory and computation. Even for \(T = 10{,}000\), storing the \(T \times T\) attention matrix in float32 requires \(\sim \)400 MB per head, per layer.
10.7.2 Face Propagation: Information Flow Through Shared Faces
The key mechanism of TetraMemory is face propagation: information flows from one tetrahedron to its neighbor through their shared triangular face.
In a LinkedTetraChain, \(\mathcal {T}_k\) and \(\mathcal {T}_{k+1}\) share three vertices. When we compute the state of \(\mathcal {T}_{k+1}\), three of its four vertices already carry information from \(\mathcal {T}_k\). Only one vertex is new. This means that 75% of each tetrahedron’s information comes from its predecessor—a natural form of causal propagation.
Definition 10.5 (TetraMemory State Update). At layer \(\ell \), the state of tetrahedron \(\mathcal {T}_i\) is updated by: \begin {equation} \mathbf {h}_i^{(\ell +1)} = \phi \left (\mathbf {h}_i^{(\ell )} + \sum _{j \in \mathcal {N}_4(i)} \alpha _{ij} \cdot \mathbf {W}^{(\ell )} \mathbf {h}_j^{(\ell )}\right ), \label {eq:tetra-update} \end {equation} where:
- \(\mathcal {N}_4(i)\) is the set of (at most 4) face-sharing neighbors of \(\mathcal {T}_i\).
- \(\alpha _{ij}\) are attention weights computed from edge properties and shared-face geometry.
- \(\mathbf {W}^{(\ell )} \in \mathbb {R}^{d \times d}\) is the layer’s learnable weight matrix.
- \(\phi \) is a nonlinear activation function.
The cost per layer is \(O(T \cdot k \cdot d)\) where \(k \leq 4\), making it linear in sequence length.
10.7.3 Volume as Attention Weight
Each tetrahedron’s signed volume provides a natural attention modulator:
\begin {equation} \alpha _{ij} = \sigma \!\left (\frac {\mathbf {q}_i^\top \mathbf {k}_j}{\sqrt {d}} + \beta \cdot V_{\text {signed}}(\mathcal {T}_i)\right ), \end {equation}
where \(\sigma \) is a sigmoid function and \(\beta \) is a learnable temperature parameter. The volume term acts as a geometric bias: tetrahedra with larger volume (more “open” configurations) attend more strongly to their neighbors, while degenerate (near-flat) tetrahedra attend weakly.
Volume as Importance
In molecular biology, the volume of a protein’s active site determines its binding affinity. A large, open active site can accommodate many substrates; a narrow, constrained one is highly specific. TetraMemory’s use of volume as an attention modulator mirrors this: the “openness” of a tetrahedral configuration controls how much information flows through it.
10.7.4 Edge Weights as Pairwise Compatibility
The six edge weights \(\mathbf {E}_i \in \mathbb {R}^6\) within each tetrahedron encode pairwise compatibility scores between the four vertices. These are analogous to the pairwise attention scores in self-attention, but:
- 1.
- They are local: only the 6 pairs within a tetrahedron are computed, not the \(\binom {T}{2}\) global pairs.
- 2.
- They are geometrically constrained: the edge weights must be consistent with a valid 3D tetrahedron (triangle inequality on faces, positive volume).
- 3.
- They are typed: the Watson–Crick edges (A–T, C–G) can carry different learned biases than the non-Watson–Crick edges.
10.7.5 Long-Range Information Flow
How does local interaction enable global computation? Through propagation: after \(L\) layers of TetraMemory updates, information from tetrahedron \(\mathcal {T}_i\) has propagated to all tetrahedra within graph distance \(L\) in the dual graph.
For a LinkedTetraChain of \(n\) tetrahedra, the dual graph is a path graph of length \(n\), so the diameter is \(n\) and full propagation requires \(L = n\) layers. However, for a 3D tetrahedral mesh (not just a chain), the dual graph diameter is \(O(n^{1/3})\), so \(O(n^{1/3})\) layers suffice for global information flow.
Theorem 10.2 (TetraMemory Propagation). For a tetrahedral mesh with \(n\) tetrahedra and dual-graph diameter \(D\), after \(L \geq D\) layers of TetraMemory updates (Equation 10.9), every tetrahedron has received information from every other tetrahedron.
Proof. At each layer, each tetrahedron receives information from its face-sharing neighbors (at most 4). After \(L\) layers, the receptive field of each tetrahedron is the set of all tetrahedra within graph distance \(L\) in the dual graph. When \(L \geq D\), this includes all tetrahedra. □
10.7.6 Complexity Comparison
| Self-Attention | TetraMemory | |
| Cost per layer | \(O(T^2 d)\) | \(O(T \cdot k \cdot d)\), \(k \leq 4\) |
| Memory per layer | \(O(T^2)\) | \(O(T \cdot k)\) |
| Layers for global info | 1 (instant) | \(O(D)\) where \(D\) is mesh diameter |
| Total cost for global | \(O(T^2 d)\) | \(O(D \cdot T \cdot k \cdot d)\) |
| For 3D mesh (\(D = O(T^{1/3})\)) | \(O(T^2 d)\) | \(O(T^{4/3} k d)\) |
| Structural locality | None | Built-in (mesh topology) |
| Chirality encoding | None | Built-in (signed volume) |
10.8 The DoubleHelixMesh
In real DNA, information is stored in two complementary strands running antiparallel to each other, forming the iconic double helix. DOGMA mirrors this with the DoubleHelixMesh: two complementary LinkedTetraChains—the sense strand and the antisense strand—cross-linked by Watson–Crick hydrogen bonds.
10.8.1 Sense and Antisense Strands
Given a sense-strand sequence \(S = s_0 s_1 s_2 \cdots s_{n-1}\), the antisense strand is the reverse complement: \begin {equation} \bar {S} = \overline {s_{n-1}}\;\overline {s_{n-2}}\;\cdots \;\overline {s_1}\;\overline {s_0}, \end {equation} where the complement map is \(\overline {\mathsf {A}} = \mathsf {T}\), \(\overline {\mathsf {T}} = \mathsf {A}\), \(\overline {\mathsf {C}} = \mathsf {G}\), \(\overline {\mathsf {G}} = \mathsf {C}\).
Each strand forms its own LinkedTetraChain:
- Sense chain: \(\mathcal {C}_{\text {sense}} = (\mathcal {T}_1^{+}, \mathcal {T}_2^{+}, \ldots , \mathcal {T}_{n-3}^{+})\)
- Antisense chain: \(\mathcal {C}_{\text {anti}} = (\mathcal {T}_1^{-}, \mathcal {T}_2^{-}, \ldots , \mathcal {T}_{n-3}^{-})\)
The antisense chain runs in the reverse direction (antiparallel), so \(\mathcal {T}_k^{-}\) aligns spatially with \(\mathcal {T}_{n-3-k}^{+}\).
10.8.2 Watson–Crick Cross-Links
The two strands are connected by hydrogen bonds between complementary base pairs:
- A–T pairing: 2 hydrogen bonds (weaker).
- G–C pairing: 3 hydrogen bonds (stronger).
In the DoubleHelixMesh, these cross-links are represented as inter-chain edges with weights proportional to hydrogen bond strength:
\begin {equation} w_{\text {cross}}(b, \bar {b}) = \begin {cases} w_{\text {AT}} & \text {if } (b, \bar {b}) \in \{(\mathsf {A}, \mathsf {T}), (\mathsf {T}, \mathsf {A})\}, \\ w_{\text {GC}} & \text {if } (b, \bar {b}) \in \{(\mathsf {G}, \mathsf {C}), (\mathsf {C}, \mathsf {G})\}, \end {cases} \end {equation}
where \(w_{\text {GC}} > w_{\text {AT}}\) reflects the stronger G–C bond.
10.8.3 Advantages of the Double Helix Structure
The DoubleHelixMesh provides several computational advantages:
- 1.
- Redundancy and error detection: Each base on the sense strand has a complementary base on the antisense strand. Discrepancies can be detected and corrected, mirroring DNA’s own error-correction mechanism.
- 2.
- Bidirectional reading: The sense strand reads \(5' \to 3'\) while the antisense reads \(3' \to 5'\). This provides a natural mechanism for bidirectional sequence processing without the need for separate forward and backward passes.
- 3.
- Cross-strand attention: The hydrogen bond cross-links create additional attention pathways between the two chains, enriching the information flow beyond what a single chain provides.
- 4.
- Melting dynamics: In biology, the double helix can “melt” (denature) at high temperatures, separating the strands. In DOGMA, a learned “temperature” parameter controls the strength of cross-strand interactions, enabling the model to dynamically decouple the strands when independent processing is beneficial.
DoubleHelixMesh Efficiency
Despite having two chains, the DoubleHelixMesh does not double the computational cost. The antisense chain shares vertex positions (by symmetry) with the sense chain, and the cross-strand hydrogen bond weights are computed from the base assignments (which are determined by complementarity). The additional cost is \(O(T)\) for the cross-strand edges, which is dominated by the \(O(T \cdot k \cdot d)\) per-layer cost of TetraMemory updates on each chain.
10.9 Delaunay Tetrahedralization
While the LinkedTetraChain is the primary structure for sequence data, DOGMA also supports general 3D point clouds through Delaunay tetrahedralization.
Given a set of points \(P \subset \mathbb {R}^3\), the Delaunay tetrahedralization partitions the convex hull of \(P\) into tetrahedra such that no point lies inside the circumsphere of any tetrahedron.
Definition 10.6 (Delaunay Tetrahedralization). A tetrahedralization of point set \(P\) is Delaunay if, for every tetrahedron \(\mathcal {T}\), the open circumsphere of \(\mathcal {T}\) contains no point of \(P\).
The Delaunay condition maximizes the minimum solid angle among all tetrahedralizations, producing well-shaped elements that avoid degeneracies. This quality guarantee is important for numerical stability in TetraMemory computations.
Applications relevant to DOGMA:
- Protein structure: Given \(C_\alpha \) coordinates, Delaunay tetrahedralization produces a volumetric mesh capturing local packing geometry.
- Medical imaging: CT/MRI voxels can be converted to Delaunay tetrahedral meshes for volumetric analysis.
- Molecular dynamics: Voronoi tessellation (dual of Delaunay) defines natural atomic neighborhoods.
10.10 Multimodal Applications of TDS
The TDS representation enables DOGMA to handle diverse data modalities through a unified geometric framework:
| Modality | TDS Representation | Application |
| DNA/RNA | LinkedTetraChain | Sequence modeling, variant calling |
| 3D Vision | Volumetric scene \(\to \) tetra mesh | Object recognition, scene understanding |
| Proteins | \(C_\alpha \) coordinates \(\to \) Delaunay | Structure prediction, drug design |
| Medical | CT/MRI voxels \(\to \) tetra mesh | Segmentation, anomaly detection |
| Game engines | Physics bodies \(\to \) deformable tetra | Destruction simulation, soft bodies |
| Video | Temporal tetrahedra | Motion tracking, action recognition |
| Audio | Waveform helix encoding | Speaker recognition, music analysis |
10.11 Worked Examples
Example 10.3 (Building a LinkedTetraChain from a DNA Sequence). Given the sequence \(5'\)-CGTACG-\(3'\) (6 bases), construct the LinkedTetraChain.
Step 1: Assign vertices.
| Vertex | \(v_0\) | \(v_1\) | \(v_2\) | \(v_3\) | \(v_4\) | \(v_5\) |
| Base | C | G | T | A | C | G |
Step 2: Enumerate tetrahedra. The chain has \(6 - 3 = 3\) tetrahedra: \begin {align*} \mathcal {T}_1 &= (v_0, v_1, v_2, v_3) = (\textsf {C}, \textsf {G}, \textsf {T}, \textsf {A}), \\ \mathcal {T}_2 &= (v_1, v_2, v_3, v_4) = (\textsf {G}, \textsf {T}, \textsf {A}, \textsf {C}), \\ \mathcal {T}_3 &= (v_2, v_3, v_4, v_5) = (\textsf {T}, \textsf {A}, \textsf {C}, \textsf {G}). \end {align*}
Step 3: Count combinatorics.
- Vertices: 6
- Edges: \(3(3) + 3 = 12\)
- Shared faces: \((v_1,v_2,v_3)\) between \(\mathcal {T}_1\) and \(\mathcal {T}_2\); \((v_2,v_3,v_4)\) between \(\mathcal {T}_2\) and \(\mathcal {T}_3\).
Step 4: Compute volume and chirality for \(\mathcal {T}_1\). Suppose we embed the vertices at: \(\mathbf {v}_0 = (0,0,0)\), \(\mathbf {v}_1 = (1,0,0)\), \(\mathbf {v}_2 = (0.5, 0.87, 0)\), \(\mathbf {v}_3 = (0.5, 0.29, 0.82)\).
Then: \[ V_{\text {signed}} = \frac {1}{6} \det \begin {pmatrix} 1 & 0.5 & 0.5 \\ 0 & 0.87 & 0.29 \\ 0 & 0 & 0.82 \end {pmatrix} = \frac {1}{6}(1 \cdot 0.87 \cdot 0.82) = \frac {0.7134}{6} \approx 0.119. \] Since \(V_{\text {signed}} > 0\), the chirality is \(\chi = +1\) (right-handed).
Example 10.4 (TetraMemory vs. Self-Attention Cost). Consider a genomic sequence of length \(T = 50{,}000\) with embedding dimension \(d = 512\).
Self-attention cost per layer: \[ O(T^2 d) = 50{,}000^2 \times 512 = 1.28 \times 10^{12} \text { FLOPs}. \]
TetraMemory cost per layer (with \(k = 4\) neighbors): \[ O(T \cdot k \cdot d) = 50{,}000 \times 4 \times 512 = 1.024 \times 10^{8} \text { FLOPs}. \]
Speedup per layer: \(\frac {1.28 \times 10^{12}}{1.024 \times 10^{8}} \approx 12{,}500\times \).
Even accounting for the need for \(O(T^{1/3}) \approx 37\) layers for global propagation in a 3D mesh, the total TetraMemory cost is: \[ 37 \times 1.024 \times 10^{8} = 3.79 \times 10^{9} \text { FLOPs}, \] which is still \(\sim \)\(338\times \) faster than a single self-attention layer.
10.12 Exercises
- 10.1.
- [Fundamentals] Compute the volume and chirality of the tetrahedron with vertices \((0,0,0)\), \((1,0,0)\), \((0,1,0)\), \((0,0,1)\). Verify that reflecting one vertex across a coordinate plane reverses the chirality.
- 10.2.
- [Construction] Build the LinkedTetraChain for the DNA sequence \(5'\)-GATTACA-\(3'\). List all tetrahedra, their base assignments, and the shared faces between consecutive tetrahedra. How many total edges does the chain have?
- 10.3.
- [Analysis] For a tetrahedral mesh with \(M\) elements, each with at most 4 face-sharing neighbors, compare the computational cost of one TetraMemory layer (\(O(4Md)\)) with one self-attention layer (\(O(M^2 d)\)). For what value of \(M\) does TetraMemory become 100\(\times \) faster?
- 10.4.
- [Information Density] Compute the number of unique pairwise interactions captured by (a) a triangle mesh with sliding window of width 3 and overlap 2, and (b) a tetrahedral mesh with sliding window of width 4 and overlap 3, both applied to a sequence of length \(T = 100\). Express the ratio.
- 10.5.
- [DOF Counting] A hexahedron (cube) has 8 vertices, 12 edges, and 6 faces. Compute the number of degrees of freedom for a “HexaData” structure analogous to TDS (vertex positions + edge weights + base encoding for 8 bases + volume + chirality). Is the DOF-per-vertex ratio better or worse than TDS?
- 10.6.
- [Coding] Implement the TetraTokenizer (Equation 10.7) in Python. Given a list of tetrahedra (each specified by 4 vertex coordinates and 4 base types), produce the 36-dimensional token representation for each.
- 10.7.
- [DoubleHelix] Given the sense strand \(5'\)-ATGCATGC-\(3'\), write out the antisense strand. Construct the DoubleHelixMesh and count the total number of cross-strand hydrogen bonds. How many are A–T bonds (2H) and how many are G–C bonds (3H)?
- 10.8.
- [Theory] Prove that the graph diameter of a Delaunay tetrahedralization of \(M\) uniformly distributed points in a unit cube is \(O(M^{1/3})\).
- 10.9.
- [Design] Propose a scheme for encoding a protein backbone (sequence of \(C_\alpha \) coordinates) as a TDS mesh. How would you assign base types \(\{\mathsf {A}, \mathsf {C}, \mathsf {G}, \mathsf {T}\}\) to vertices based on amino acid properties?
- 10.10.
- [Research] Compare TetraMemory with graph neural networks (GNNs). Both use local message passing. What does the tetrahedral structure provide that a generic graph does not? Consider: bounded degree, volume, chirality, and face-sharing.
10.13 Key Takeaways
Key Takeaways
- A tetrahedron is the minimal solid: four vertices, six edges, four faces, and one volume. It is the simplest 3D object with both volume and chirality. No simpler primitive can encode these properties.
- The TetraData Structure uses tetrahedra as computational primitives, mapping four vertices to DNA base types \(\{\mathsf {A}, \mathsf {C}, \mathsf {G}, \mathsf {T}\}\) and encoding 36 degrees of freedom per element.
- The LinkedTetraChain links tetrahedra by shared triangular faces, acting as a sliding window of width 4 with overlap 3 over a sequence. Each new vertex adds exactly 3 edges and 1 tetrahedron.
- DNA sequences map naturally to LinkedTetraChains: each tetrahedron represents a window of 4 consecutive bases, and adjacent tetrahedra share 3 bases through their common face.
- TetraMemory replaces \(O(T^2)\) self-attention with \(O(T \cdot k)\) local tetrahedral interactions (\(k \leq 4\)) that compose into global computation through face propagation. Volume serves as a geometric attention weight; edge weights encode local pairwise compatibility.
- The DoubleHelixMesh pairs a sense and antisense LinkedTetraChain with Watson–Crick cross-links (A–T: 2 hydrogen bonds, G–C: 3 hydrogen bonds), providing redundancy, bidirectional reading, and cross-strand attention.
- Delaunay tetrahedralization extends TDS beyond sequences to general 3D point clouds, enabling multimodal applications from protein structure to medical imaging.
Suggested Reading
- Chapter 7, Section on DNAComputeBlock: TetraMemory’s architectural context.
- Jumper et al. [2021]: AlphaFold’s use of structural representations for proteins.
- Gu and Dao [2023]: Alternative approach to sub-quadratic sequence processing.