LLMs Cram Textbooks. Genomic Models Discover New Biology
Language models learn what science has written down. Genomic models learn from what evolution has encoded.
Whenever I say that we are training a genomic foundation model, I hear the same question:
Why spend years building a model for DNA? LLMs already know biology, and the next one will know even more.
The conclusion is understandable. Modern LLMs can explain pathways, summarize papers, write analysis code, compare disease mechanisms and propose experiments. Give one a difficult biology question and it can often produce a useful synthesis in seconds.
At first glance, the problem looks solved.
The scientific record has a boundary
Language is a compressed record of what people have observed, measured and understood. Biology papers contain decades of experiments. Databases organize variants, genes, pathways, phenotypes and diseases. An LLM trained on this material can build a remarkably rich representation of established biological knowledge.
That is already transformative. It makes the scientific record searchable through natural language and lets researchers combine facts that previously lived in different papers, databases and codebases.
But it is still a representation of the record.
The literature is a snapshot of what science knows today. It does not contain every relationship that exists in biology. Most possible mutations have never been tested. Most regulatory sequences have never been measured across every cell type. Most combinations of variants, environments and cellular states have never been observed.
A mutation no one has studied
To see the gap, we need a different kind of question.
Take a mutation that has never appeared in a paper. How will it change the activity of an enhancer in a specific cell type?
An enhancer is a stretch of DNA that helps control when and where a gene is active. An LLM can explain that. It can retrieve similar mutations, identify relevant transcription factors, write code and propose an assay.
But there is no sentence containing the answer. The mutation is new.
A useful answer must be inferred from sequence context, regulatory grammar, evolutionary constraint and experimental measurements. It should be quantitative and calibrated: not merely “this mutation may matter,” but how much it is likely to change expression, in which cells, and with what uncertainty.
This is the difference between knowing what humans have written about the genome and learning a predictive model of the genome itself.
Genomes contain the surviving output of billions of years of mutation, recombination and selection. They encode regularities that science has not yet described in language.
Not every phenotype is determined by sequence. Environment and cellular state matter, too. But sequence contains far more information than the literature has extracted from it.
Written knowledge tells us what has already been discovered. A genomic foundation model tries to learn enough structure from sequence to make predictions beyond that boundary.
If the missing information is in DNA, put DNA into the LLM’s training set.
Adding DNA is not enough
In principle, nothing prevents a model from receiving both text and sequence. In practice, adding a modality is not the same as learning it at frontier quality.
We have already seen this with images and video. General AI products can accept multiple media types, but the highest-quality visual understanding and generation still depend on machinery built for pixels and frames: dedicated representations, training mixtures, objectives, architectures and inference procedures.
A text model does not become the best video model because video was added to the batch.
DNA makes the problem more extreme.
Sequence changes the scale of training
Public sequencing archives contain roughly 50 petabases of raw sequence: tens of quadrillions of bases collected from more than 27 million datasets. Counted as symbols, public DNA already dwarfs the text corpora used to train today’s LLMs.
Raw bases are not equivalent to useful training examples. The archives contain redundancy, sequencing errors and uneven sampling. But cleaning and weighting the data does not remove the scale problem. It makes data curation part of the model.
If DNA receives a small share of a general training mixture, the model gets a shallow biological education. If genomic diversity is represented at the scale required for the best performance, sequence can dominate the data and compute budget.
This is not a temporary inconvenience solved by a larger cluster. A model still has to decide what to learn, how often to see it and which errors deserve capacity.
The data also changes on a different schedule. New genomes, assemblies and metagenomic datasets arrive continuously. Functional assays are smaller but disproportionately valuable. A genomic model can be updated when either source improves.
Folding every biological update into a general model creates a slower release cycle and forces unrelated capabilities through the same regression tests.
Genomic context is part of the mechanism
The context requirements are different in both scale and meaning.
A human genome contains about three billion bases. Current LLMs work with inputs up to a million tokens. In the genome, important effects can cross enormous distances. For example, variants on different parts of a chromosome can interact.
In language, a distant passage is often useful context that can be retrieved or summarized. In a genome, a distant element may be part of the mechanism itself. The model may not know which distant region matters until it has represented the surrounding sequence and regulatory state.
Sequence also has structure that text does not: two DNA strands, reverse-complement symmetry, repeated elements, nested regulatory signals and dependencies that span bases, genes, chromosomes and three-dimensional genome organization.
Handling this efficiently is likely to require hierarchical memory, compression, sparse long-range interactions and architectures that represent biological structure directly.
Biological predictions need biological tests
An LLM is optimized to predict and generate language. A genomic model can also begin with next-base prediction, but biological usefulness depends on different outcomes.
Which variant changes expression? Which promoter works in a particular cell? Which mutation disrupts a protein? Which sequence produces the phenotype we want?
Answering these questions requires alignment with expression, epigenetics, perturbation screens, protein measurements, phenotypes and clinical data. These signals are sparse, expensive and context-dependent. They should not be diluted inside an objective dominated by text.
The models also face different standards of success. A fluent answer is useful in conversation. A genomic prediction should hold up when researchers test it experimentally. That requires biological objectives, evaluation and a release cadence suited to the work. It is a different scientific instrument.
One system, different scientific instruments
Imagine a researcher asking one question in ordinary language:
Which mutations are likely to alter this regulatory program, and what experiment should I run next?
The LLM can interpret the question, search the literature, write code, compare hypotheses and explain the result. The genomic model can predict the effects of mutations and propose sequences for experimental testing. Other specialist models may contribute protein structure, cell-state or molecular predictions.
One system can coordinate all of this. That does not require one undifferentiated model to be best at every modality.
Better LLMs will make genomic models easier to use. They will let more researchers formulate precise questions and turn predictions into experiments. But every question that crosses the boundary of written knowledge still needs a model trained to learn from biology itself.
LLMs are becoming powerful representations of scientific knowledge. That is not the same as being a predictive representation of nature.
The literature tells us what biology has revealed so far. Genomes preserve a much larger record of what evolution has tried, retained and rejected. Learning that record requires data volumes, context, objectives and architectures that are specific to sequence.
The next LLM will help us ask better biological questions. Genomic foundation models will help us answer the ones science has never seen before.