Hermon DNA Encoder
A DNABERT-2-based sequence encoder and task heads. Current code includes deterministic data generation, sequence classification, LoRA/full fine-tuning options, and data gates.
Hermon Research · Genome Intelligence
Hermon DNA
Hermon DNA studies actual nucleotide sequences with genome-native encoders and evidence-grounded reasoning. Unlike DOGMA, it is not an alternative architecture doctrine: transformers, state-space models, and implicit convolutions all compete on the same scientific evaluations.
Long-context research
HyenaDNA and Caduceus are comparison targets for single-nucleotide context, state-space mixing, bidirectionality, and reverse-complement equivariance.
Hermon instruction layer
A separate model turns retrieved evidence and structured predictions into explanations. It reports provenance, confidence, alternatives, and limitations.
Learning Stack
Sequence representation first, explanation second
Separating the encoder from the instruction model makes evaluation sharper: sequence predictions can be measured directly, while natural-language explanations can be checked for evidence fidelity.
Canonical sequence
Validate alphabet, ambiguity codes, provenance, coordinate system, and reverse-complement policy before a sequence enters training or evaluation.
Genome tokenizer
Compare single-nucleotide, fixed k-mer, and BPE representations under equal compute. Tokenization is an empirical decision, not a permanent doctrine.
Bidirectional encoder
Pretrain a DNABERT-style encoder with masked sequence objectives so every position can use upstream and downstream context.
Equivariance and long context
Test reverse-complement consistency and longer sequence mixers. Caduceus and HyenaDNA provide strong external baselines for these properties.
Scientific heads
Attach narrow, calibrated heads for motif, region, variant-effect, retrieval, and sequence-quality tasks with species- and chromosome-held-out splits.
Hermon reasoning bridge
A separate instruction model explains evidence, uncertainty, and limits. It must never convert an encoder score into unsupported biological certainty.
Pretraining Objectives
One model, several complementary signals
Masked sequence modelingRecover hidden spans from bidirectional context.
Reverse-complement consistencyPenalize incompatible states for equivalent strand views.
Contrastive region learningSeparate matched biological regions from GC- and length-matched hard negatives.
Multi-task supervisionShare an encoder while keeping task-specific heads, labels, uncertainty, and split rules explicit.
Retrieval-grounded instructionTrain explanations against cited evidence and structured encoder outputs rather than free-form imitation.
Benchmark Matrix
Accuracy without split discipline is not scientific progress
FamilyTasksRequired splitCritical failure
Representation
Masked nucleotide recovery, retrieval, clustering, linear probes
Species-, chromosome-, and family-held-out
Shortcut learning, tokenizer leakage, or embedding collapse
Invariance
Reverse-complement consistency, shift robustness, ambiguity handling
Paired original/reverse-complement and controlled perturbations
Predictions change when the biological meaning should not
Regulation
Promoter, enhancer, splice, chromatin, and motif benchmarks
Chromosome and cell-type holdouts where labels support them
GC-content or species identity explains the apparent gain
Variant effect
Calibrated ranking and classification of sequence perturbations
Gene-, locus-, and study-held-out with time-aware validation
High aggregate score with poor rare-class recall or calibration
Scientific reasoning
Evidence retrieval, uncertainty explanation, experimental critique
Source-isolated questions and adversarial unsupported claims
The instruction model invents causality from an encoder score
Research Boundary
Scientific assistance with explicit biological safeguards
Public Hermon DNA experiences prioritize sequence understanding, retrieval, simulation, and education. Biological design requests require risk classification and appropriate controls. Model confidence is not experimental validation, clinical evidence, or permission to act.
Core Reading
Encoder and long-context baselines
DNABERT-2ICLR 2024
BPE tokenization, efficient multi-species pretraining, and the Genome Understanding Evaluation benchmark.
Nucleotide TransformerNature Methods 2025
A family of DNA foundation encoders evaluated across human and multi-species genomic tasks.
HyenaDNANeurIPS 2023
Single-nucleotide, long-context genomic modeling using implicit convolutions instead of dense attention.
CaduceusICML 2024
Bidirectional state-space modeling with reverse-complement equivariance for DNA.
MambaCOLM 2024
Selective state-space sequence modeling with linear scaling in sequence length.
EvoScience 2024
A long-context biological sequence model based on StripedHyena, spanning nucleotide to genome scales.