Background — everything you need, from zero

Audience: ML researchers — especially NLP — with no biology background. Promise: by the end you can read any paper in this field without getting stuck on vocabulary. Method: build the biology from first principles, anchoring each concept to something an ML reader already knows.

Read this before 10_SOTA_LANDSCAPE.md.

Part 1: What a gene actually is

1.1 The central dogma

$$\text{DNA}\stackrel{\text{transcription}}{\to }\text{RNA}\stackrel{\text{translation}}{\to }\text{protein}$$

• DNA — the stored program. A string over $\{A,C,G,T\}$. ~3 billion characters in a human, ~2.7 billion in a rat. Identical in every cell of the organism.
• Gene — a substring of DNA that codes for something. ~20,000 protein-coding genes. (Plus tens of thousands of non-coding ones that do regulatory things. Ignore those for now.)
• RNA — a working copy of one gene. Disposable. Made on demand.
• Protein — the thing that actually does work. Enzymes, structure, signalling.

The thing that confuses everyone at first: if every cell has identical DNA, why is a liver cell different from a brain cell?

Because they transcribe different subsets of it, at different rates. Same program, different runtime behaviour. A liver cell runs the metabolism modules hard and leaves the neurotransmitter modules idle. Brain does the reverse.

NLP framing: DNA is the codebase. Cell type is which functions actually get called, and how often. Gene expression is the profiler output.

1.2 Gene expression

Expression of gene $g$ = how much RNA for $g$ exists right now. It’s a proxy for “how hard is the cell running module $g$.”

Measure it for all ~20,000 genes at once and you get a transcriptomic profile:

$$\mathbf{x}\in {\mathbb{R}}^{20000},x_g=\text{abundance of RNA for gene }g$$

NLP framing: a dense embedding of the cell’s current state. But — and this is the important difference — every dimension is named and interpretable. Dimension 4,412 is Cyp1a1, and biologists have opinions about it. When they say “the model got Atf3 wrong,” they mean coordinate 8,301 specifically.

This is why interpretability pressure in this field is so much higher than in NLP. Nobody asks what dimension 412 of a BERT embedding means. Here it’s the entire point.

1.3 Why “expression” is a slippery word

Three different things get called expression:

These are not interchangeable, and conflating them is the single most common way to misread this literature. See Part 4.

Part 2: How you measure it

2.1 Microarrays (the old way)

A glass chip with millions of short DNA probes stuck to it in known positions. Each probe is complementary to a bit of one gene.

1. Extract RNA from tissue, convert to cDNA, tag it with fluorescent dye
2. Wash it over the chip
3. cDNA sticks (“hybridizes”) to matching probes
4. Shine a laser, photograph the chip
5. Brightness of spot $g$ ∝ abundance of gene $g$

Analog. Continuous fluorescence intensity. Properties that matter:

• saturation — very bright spots max out; you can’t distinguish 100× from 1000×
• background — the chip glows a bit even with nothing bound
• cross-hybridization — similar sequences stick to the wrong probe
• fixed vocabulary — the chip only has probes for genes it was designed with

In DrugMatrix: CodeLink (8,565 probes, discontinued 1st-gen) and Affymetrix GeneChip Rat 230 2.0 (31,042 probes, 2nd-gen, still used).

NLP framing: a closed-vocabulary bag-of-words counter with lossy analog readout and OOV problems. You can only count words that were in the dictionary when the chip was printed.

2.2 RNA-seq (the new way)

Don’t hybridize. Just read the RNA, letter by letter, using a sequencer. Then count how many reads map to each gene.

Digital. Read counts. Properties:

• huge dynamic range (no saturation)
• open vocabulary — you find genes you weren’t looking for
• Poisson-ish counting noise at low abundance
• depth-dependent — sequence deeper, get more counts for everything

2.3 Targeted sequencing: S1500+ / TempO-Seq

Sequencing everything is expensive. So: pick ~2,700 well-chosen landmark genes, measure only those, then predict the rest.

This works because gene expression is massively redundant — genes move in coordinated modules, so a few hundred well-chosen ones carry most of the information.

In DrugMatrix: BioSpyder S1500+ measures the landmarks; a tool called GeniE extrapolates to ~20,000 genes → “BioSpyderWT” (22,794 probes).

⚠️ File this away. A chunk of “measured” BioSpyderWT data is itself a model prediction. Nobody in the four papers dwells on this. When ToxCompl+ imputes DSMatrix, it is partly imputing from imputations. That’s a soft spot, and an open question — quantifying how much error the extrapolation injects is F2 in 15_FRONTIER.md.

NLP framing: landmark genes are like measuring a document’s embedding from 300 anchor words and predicting the rest. Same trick LINCS L1000 uses (978 landmarks) — which is why L1000 and S1500+ are architecturally cousins. Remember that; it matters in 15_FRONTIER.md.

2.4 The comparability problem

You cannot put a fluorescence intensity and a read count on the same axis. They aren’t the same kind of number.

This is what TransPlatformer exists to solve, and it’s why fold-change matters so much.

Part 3: Single-cell vs bulk

3.1 Bulk

Grind up a whole liver. Extract all the RNA. Measure. You get one vector per sample.

But a liver contains hepatocytes, immune cells, endothelial cells, bile duct cells… So your vector is a weighted average over cell types:

$${\mathbf{x}}_{\text{bulk}}=\sum _k{\pi }_k{\mathbf{x}}_k,\sum _k{\pi }_k=1$$

where ${\pi }_k$ is the proportion of cell type $k$.

The consequence, and it’s ugly: if a drug kills 10% of one cell type, $\mathbf{\pi }$ changes and ${\mathbf{x}}_{\text{bulk}}$ shifts — even if no surviving cell changed its expression at all. You cannot distinguish “cells changed behaviour” from “the population changed composition.”

NLP framing: you’re reading the mean embedding of a document collection. If the topic mixture shifts, the mean moves, and you can’t tell that from every document changing.

3.2 Single-cell (scRNA-seq)

Physically separate the cells first (droplets, wells, barcodes), then sequence each one. Instead of one vector per liver you get 10,000 vectors, one per cell.

The cost is brutal. Each cell contains very little RNA. You capture maybe 10–20% of it. So per cell you detect ~1,000–5,000 genes out of 20,000. The rest are zero — and you cannot tell “not expressed” from “we missed it.”

That’s dropout, and it’s the defining pathology of single-cell data.

NLP framing: every sentence is 70–90% masked, at random, and you don’t get the mask positions. You have millions of sentences but each is mostly holes.

3.3 The trade

Why this matters for you: the entire foundation model literature is single-cell, because that’s where the millions of samples are. the DrugMatrix data is bulk. That is not a small mismatch — it’s one of the two central gaps you’d be bridging. (The other is fold-change vs absolute; Part 4.)

Part 4: Fold change — the concept that decides everything

4.1 The definition

$${\text{FC}}_g=\frac{\text{expression of }g\text{ in treated animals}}{\text{expression of }g\text{ in control animals}},\text{value stored}={\log }_{10}{\text{FC}}_g$$

Log because effects are multiplicative and you want symmetry: doubling is $+0.301$, halving is $-0.301$.

4.2 Why they do it

Reason 1 — it kills the baseline. Raw liver expression is dominated by “this is a liver.” Every profile looks the same. The drug effect is a small perturbation on a huge constant. Dividing by control subtracts the constant and leaves the signal.

Reason 2 — it makes platforms comparable. Fluorescence and read counts are different units. But a ratio is unitless:

$$\frac{{\text{intensity}}_{\text{treated}}}{{\text{intensity}}_{\text{control}}}\text{and}\frac{{\text{counts}}_{\text{treated}}}{{\text{counts}}_{\text{control}}}$$

are both “how much did this gene move,” and the platform-specific scale factor cancels — to first order. It doesn’t fully cancel, because saturation and background are nonlinear. That residual is exactly what TransPlatformer models.

4.3 The three consequences you must hold onto

(a) 92% of the matrix is ≈ 0. Most drugs don’t touch most genes. From TransTissue Table 1:

The ~8% that isn’t zero is the entire biological content. This is class imbalance wearing a regression costume. Keep asking: what does the all-zeros predictor score?

(b) It changes what correlation means. GenTox proves both directions (see 16_MATH_NOTES.md §3):

• On absolute data, two unrelated profiles share the baseline ⟹ $\rho \to 1$. High PCC is meaningless.
• On fold-change, the baseline is gone ⟹ $\rho \to 0$ under noise. High PCC is hard-won.

(c) It breaks foundation models. ⭐ This is the single most important fact in this document.

scGPT’s input is binned absolute expression. Its value encoder takes a count, bins it, embeds the bin. DrugMatrix is already-differenced log ratios.

There is no sensible way to feed a fold-change into a value encoder trained on counts. $-0.7$ is not a count. It isn’t in any bin. The model has never seen a negative number in that slot.

This is the honest, technical, concrete answer to “why don’t you just fine-tune scGPT?” It’s not a shrug. It’s an architectural incompatibility, and naming it precisely is what turns a weak future-work paragraph into a real argument.

NLP framing: it’s like trying to feed word deltas to a model whose embedding layer expects word IDs. The type signature is wrong.

4.4 Levels of processing (a table you’ll want later)

Note L1000 Level 5. Z-scores are signed and centred at zero — structurally like fold-change. That’s not a coincidence, and it’s why L1000 is the natural bridge. See 15_FRONTIER.md F1.

Part 5: Biological structure you need to know exists

5.1 Pathways

Genes don’t act alone. They form pathways — chains where gene A’s protein activates gene B, which regulates C.

Example: a toxicant damages DNA → p53 (encoded by TP53) activates → p53 turns on ~100 downstream genes → cell either repairs itself or dies.

So a “p53 signature” is a coordinated pattern across ~100 genes. You don’t look at one gene; you look for the fingerprint.

This is why every one of these papers insists genes are “not a sequence.” There’s no linear order — there’s a graph. TransPlatformer §2.2 makes this argument to reject Seq2Seq. They’re right that there’s no order; whether that rules out attention is a separate question (it argues for no positional encoding, i.e. a set transformer — which is close to what they built).

5.2 Enrichment analysis — how biologists check your work

Given a predicted profile:

1. Take the top 100 up-regulated genes
2. Ask: are they enriched for any known gene set? (Hypergeometric test.)
3. If “PPARα activation” comes back at $p<0.05$, your prediction is consistent with PPARα activation.

Enrichr is the standard tool. This is how TransTissue §6 validated its predicted kidney profiles — and it’s worth more than any PCC, because a toxicologist looked at the output and said yes, that’s what lead poisoning looks like.

NLP framing: it’s like checking that your generated text has the right topic distribution rather than just low perplexity. A downstream, semantic, human-meaningful check.

5.3 Gene regulatory networks

Build a graph: nodes = genes, edges = co-expression across conditions. Find hubs (master regulators) and modules (functional units).

Crucially: the edges are correlations between rows of $G$ — gene $i_1$ across all treatments vs gene $i_2$ across all treatments.

This is why GenTox §2.3’s row-wise argument is load-bearing. A model perfect column-wise and garbage row-wise gives you profiles that score beautifully and are useless for the analysis biologists actually run.

5.4 Toxicology vocabulary

Genes that keep appearing, and why:

TransTissue’s validation used exactly these: gemfibrozil → PPARα ✓, cisplatin → TP53 ✓, lead → p53 + oxidative stress ✓.

5.5 Why cross-tissue translation is a coherent idea

Dose a rat. The compound enters the bloodstream and reaches every organ. Liver, kidney, heart, brain all see it. Each responds — differently, but to the same systemic event.

So there’s genuinely shared structure:

$${\mathbf{x}}_c^{\tau }=\underset{\text{shared across }\tau }{\underbrace{\text{common systemic response}}}+\underset{\text{depends on }\tau }{\underbrace{\text{organ-specific program}}}$$

If you can measure liver (easy, always done) and infer kidney (harder), you save an animal and a lot of money.

⚠️ And here’s the catch that the whole research agenda turns on. That decomposition has two terms. The common term is the same shape for every drug — only its magnitude changes with severity. A model that learns only the common term looks like it’s translating and has learned nothing drug-specific.

The mean-predictor baseline measures exactly this. That’s why 14_RESEARCH_AGENDA.md A0.1 is first.

Whether translation is possible is open, and TransTissue says so — §5, verbatim: “it is possible that there are simply no (or sufficient) signals for cross-tissue translation.” That honesty is a feature.

Part 6: Cross-species — orthology

Rat and human genes descend from a common ancestor ~90M years ago. Corresponding genes are orthologs.

• ~80% of rat protein-coding genes have a clean 1:1 human ortholog (approximate — verify before citing)
• The rest: many-to-many (gene families expanded differently), or no ortholog at all

The specific problem for toxicology: cytochrome P450s have expanded and diverged differently in rodents. Rats have P450s humans don’t, with different substrate specificities. So the genes that matter most for drug metabolism are the ones where orthology is worst. That’s not bad luck; it’s because those genes are under strong species-specific selection (different diets, different toxins).

NLP framing: orthology is a bilingual dictionary. ~80% coverage, and the missing 20% is concentrated in exactly the domain-specific terminology you need.

And this is why UCE is interesting. It tokenizes genes via ESM2 protein embeddings — a gene’s token comes from its protein sequence, not a vocabulary lookup. So it embeds any protein-coding gene from any species, zero-shot, no dictionary. That’s byte-level/subword tokenization for an unseen language versus a fixed vocab that OOVs everything.

Related, and directly relevant: people have already done rat→human translation of drug-induced expression with deep nets — PLOS One 2020 used a CNN and a bottleneck DNN to translate rat→human primary hepatocytes, explicitly “circumventing the current reliance on orthologs”, and beat classical ML. A 2023 follow-up added transfer learning for rat in vitro → human in vivo. Neither is cited in the four papers, and both are the cross-species analogue of TransTissueFormer. Worth reading and worth raising.

Part 7: The datasets

DrugMatrix’s unique selling point: eight tissues, in vivo. TG-GATEs has two. LINCS and Tahoe are cell lines in dishes — no organs, no systemic exposure, no inter-organ communication.

That is the moat. Cross-tissue in vivo translation cannot be studied on LINCS or Tahoe at all. Whatever else is true, the program has data nobody else has for this specific question.

Part 8: ML concepts specific to this field

8.1 “Foundation model” here

Same pitch as BERT: pretrain self-supervised on lots of unlabelled cells, fine-tune on your small labelled task.

The cell-as-sentence metaphor drives all of it:

The load-bearing weirdness: there is no word order. A cell is a set of (gene, value) pairs. So every one of these models is a set transformer with no positional encoding, and the real design problem is how to encode the value. That’s the main axis of variation between models — see 10_SOTA_LANDSCAPE.md.

8.2 Batch effects

Same biological sample, two labs, two days, two kits → measurably different numbers. Batch effects are often larger than the biological signal you’re chasing.

Standard fixes: ComBat, Harmony, quantile normalization, scVI.

This matters more than you’d think: several 2026 benchmark papers find that FM embeddings encode batch as strongly as biology, and that plain HVG selection beats them at integration. A model can look great and be reading the sequencer’s serial number.

8.3 Observational vs interventional ⭐

This is the deepest idea in the whole field, and it’s the one to actually internalize.

• Observational data: you watched cells sit there. CELLxGENE, most atlases. Tells you what states exist.
• Interventional data: you did something and measured the response. LINCS, Tahoe, DrugMatrix. Tells you what happens when you push.

Ahlmann-Eltze et al.’s explanation for why foundation models underperform on perturbation:

pretraining data is observational. You cannot learn what happens when you push a system by only watching it sit still.

This is Pearl’s ladder of causation, and it’s the single best argument for why scaling observational cell atlases might not deliver perturbation prediction — and why DrugMatrix, which is 100% interventional, is more valuable per sample than its size suggests.

8.4 Matrix completion / collaborative filtering

Netflix Prize. Assume the matrix is low-rank, factor $M\approx PQ$, fit on observed entries only.

• Transductive: can only fill in cells whose row and column were seen. Vanilla Funk-SVD.
• Inductive: can handle a new row/column by using features. GenTox’s Row NN / Col NN.

MAR vs MNAR:

• Missing At Random — missingness independent of the values. Recovery theory assumes this.
• Missing Not At Random — missingness depends on the values. Everything breaks.

TransTissue §5 explicitly assumes MAR. The panel analysis in 16_MATH_NOTES.md §6 suggests that’s false — Table 3 decomposes exactly into study panels, with structural zeros (BR–LI = 0) that no random model produces. This is an open, testable, previously-unstated concern.

8.5 The metrics

Report all five. Always. GenTox §2 argues this from first principles; nobody, including GenTox, consistently does it.

Part 9: The dictionary

Keep this open.

Part 10: Where the two worlds don’t meet

The four gaps between the DrugMatrix data and the FM literature. Everything in 15_FRONTIER.md is an attack on one of these.

Read the last two rows again. On the axes that Ahlmann-Eltze identifies as why FMs fail, DrugMatrix has what the FMs lack. It is small, but it is in vivo and it is interventional.

That asymmetry is the thesis. It’s developed in 15_FRONTIER.md.

Next: 10_SOTA_LANDSCAPE.md — what the field actually looks like in 2026, and why the news is stranger than you’d expect.