Mathematical Foundations

This appendix collects the minimum mathematics needed to read the DOGMA chapters and evaluate the experiments. It is a reference, not a substitute for a course in linear algebra, probability, or statistics.

A.1 Functions, Vectors, and Tensors

A function \(f:\mathcal {X}\to \mathcal {Y}\) maps each input in \(\mathcal {X}\) to one output in \(\mathcal {Y}\). A scalar is one number; a vector \(\mathbf {x}\in \mathbb {R}^d\) is an ordered list of \(d\) numbers; a matrix \(\mathbf {W}\in \mathbb {R}^{m\times d}\) maps a \(d\)-vector to an \(m\)-vector: \[ \mathbf {y}=\mathbf {W}\mathbf {x}+\mathbf {b}. \] A tensor generalizes matrices to more axes. A batch of \(B\) sequences of length \(T\), each with hidden dimension \(d\), commonly has shape \(B\times T\times d\).

The dot product \[ \mathbf {x}^{\top }\mathbf {y}=\sum _{i=1}^{d}x_i y_i \] measures signed alignment. The Euclidean norm \[ \|\mathbf {x}\|_2=\sqrt {\sum _i x_i^2} \] measures magnitude. Always write tensor shapes while deriving a model; many apparent conceptual errors are shape errors.

A.2 Probability and Information

For outcomes \(x\in \mathcal {X}\), a probability distribution satisfies \[ p(x)\ge 0,\qquad \sum _{x\in \mathcal {X}}p(x)=1. \] Conditional probability \(p(y\mid x)\) describes uncertainty about \(y\) after observing \(x\). A causal sequence model factorizes \[ p(x_{1:T})=\prod _{t=1}^{T}p(x_t\mid x_{<t}). \]

The information content of an event is \(-\log p(x)\). Rare events carry more surprise. Cross-entropy for a one-hot target \(y\) and prediction \(p\) is \[ \mathcal {L}_{\mathrm {CE}}=-\log p(y). \] Average sequence loss gives perplexity: \[ \operatorname {PPL}=\exp (\bar {\mathcal {L}}). \] If \(\bar {\mathcal {L}}=\log 4\), then \(\operatorname {PPL}=4\). Perplexity can be compared only when tokenization, data, and evaluation procedure are compatible.

A.3 Gradients and Optimization

For parameters \(\theta \), the gradient \[ \nabla _\theta \mathcal {L} \] contains the partial derivative of loss with respect to every parameter. Gradient descent updates \[ \theta _{t+1}=\theta _t-\eta \nabla _\theta \mathcal {L}(\theta _t), \] where \(\eta >0\) is the learning rate. Modern optimizers such as AdamW adapt the update using moment estimates and weight decay.

A gradient trains one candidate. Evolutionary search chooses among candidate configurations. These are complementary levels of optimization.

A.4 Convolution and Recurrence

A one-dimensional causal convolution with width \(K\) computes \[ y_t=\sum _{k=0}^{K-1}W_k x_{t-k}. \] Only current and earlier inputs are used. Stacking layers expands the receptive field, but a finite stack remains locally bounded.

A recurrence carries a state: \[ h_t=f_\theta (h_{t-1},x_t),\qquad y_t=g_\theta (h_t). \] State-space models use structured linear recurrences whose parameters may depend on the input. Parallel-scan algorithms can evaluate associative recurrences efficiently. Calling a recurrence “DNA-inspired” does not change these mathematical requirements.

A.5 Gates and Bounded Memory

A sigmoid gate \[ g_t=\sigma (Wx_t+b),\qquad \sigma (z)=\frac {1}{1+e^{-z}}, \] lies in \((0,1)\). It can interpolate between old and new state: \[ m_t=(1-g_t)\odot m_{t-1}+g_t\odot \widetilde {m}_t. \] This equation is a useful abstraction for expression or epigenetic memory. It does not imply a biological molecule is present.

A.6 Multi-Objective Evaluation

Let a candidate have metric vector \[ \mathbf {s}(c)=(s_1(c),\ldots ,s_k(c)). \] Candidate \(a\) Pareto-dominates \(b\) when it is no worse on every metric and strictly better on at least one. A scalar objective \[ J(c)=\sum _i w_i s_i(c) \] is convenient for ranking but hides trade-offs in the weights. Hard safety or validity requirements should remain constraints rather than terms that can be paid away by another metric.

A.7 Uncertainty and Reproduction

For measurements \(x_1,\ldots ,x_n\), the sample mean is \[ \bar {x}=\frac {1}{n}\sum _i x_i. \] The sample standard deviation is \[ s=\sqrt {\frac {1}{n-1}\sum _i(x_i-\bar {x})^2}. \] Two generation seeds are enough for a smoke test, not a confidence interval. Research comparisons should repeat independent training runs and report uncertainty. When data are grouped by organism, patient, or chromosome, resampling must preserve those groups.

A.8 Complexity

Big-\(O\) notation describes asymptotic growth. A local width-\(K\) convolution over length \(T\) is \(O(TK)\); if \(K\) is fixed, this is \(O(T)\). Dense all-pairs attention is \(O(T^2)\) in sequence length. These labels omit constants, memory traffic, hardware utilization, and hidden dimension. Wall-clock benchmarking remains necessary.

A.9 Review Problems

A.1.

Verify the shapes in \(\mathbf {Y}=\mathbf {X}\mathbf {W}^{\top }\) for \(\mathbf {X}\in \mathbb {R}^{T\times d}\) and \(\mathbf {W}\in \mathbb {R}^{m\times d}\).

A.2.

Compute the cross-entropy when the correct byte receives probability \(0.8\), then compute perplexity.

A.3.

For a width-5 causal convolution stacked in 4 layers without dilation, determine the receptive field.

A.4.

Give two candidates that are Pareto-incomparable on accuracy and latency.

A.5.

Explain why five samples from one checkpoint are not five independent training replications.