Chapter 13 — Connecting the Dots: Four Papers, One Matrix, One Gap

What the program is, what it proved, what it contradicts, and what to do.

Read last. This assumes Chapters 0 → 3.

13.1 The whole program in one picture

There is one matrix. Everything is a different question about it.

                    treatments  (m ≈ 2,700-3,000)
                    ─────────────────────────────►
                    (chemical, dose, duration)

    g   ┌───────────────────────────────────────┐
    e   │  CodeLink   × LI  ░░░░████████░░░░██  │  ← 8,565 rows
    n   │  CodeLink   × KI  ░░░████░░░░░░████░  │
    e   │  CodeLink   × BR  ░░░░░░░░░░█░░░░░░░  │  ← almost empty
    s   │  ...                                   │
        │  Affymetrix × LI  ██████░░░████████░  │  ← 31,042 rows
    n   │  ...                                   │
    ≈   │  BioSpyder  × LI  ░░████████░░░░░░██  │  ← 22,794 rows
   375k │  ...                                   │
        └───────────────────────────────────────┘
              █ = measured (12%)   ░ = missing (88%)

paper	the question	direction	verb
ToxCompl+	fill in the ░ cells	anywhere	transductive
TransPlatformer	CodeLink block → BioSpyder block	↕ across platforms	transductive
TransTissueFormer	liver block → kidney block	↕ across tissues	transductive
GenTox	here’s a drug never tested — make the column	← new column	inductive

That’s the program. Four papers, one object, four axes.

13.2 What each one actually established

ToxCompl+ — the engine


does	$G_{ij} \approx b_{i} + b_{j} + ⟨ P_{i, :}, Q_{j, :} ⟩$ , $r = 300$ . Netflix Prize on genes × treatments.
fix 1	side information (drug name, dose, organ). MaxAE 3.99 → 3.27, MAE 0.03 → 0.05. A trade.
fix 2	attention: $(P \circ Q) \cdot σ (P^{'} \circ Q^{'})$ . MaxAE 3.27 → 0.83. A large win.
breaks	rare signals aren’t low-rank. Target −0.77 → predicted +0.94. Sign flip.
⭐ my finding	~369 obs/gene-row. At $r = 300$ : ratio 1.23 (marginal). At $r = 500$ : 0.74 — underdetermined. That’s why raising $r$ made MaxAE worse — a mechanism the paper doesn’t give.

TransPlatformer — the harmonizer


does	$8, 565 \to 22, 794$ . Same tissue, same drug, different machine.
headline	MAE 0.043 vs ToxCompl 0.09; PCC 0.71 vs MLP 0.37.
Seq2Seq	on a 32GB V100S at length 8,565: 2 layers fit. vs 32.
⭐ §3.3	train {LI,KI,HE,SM}, withhold brain entirely, test brain → 2.42× better than brain’s own model. Then one brain sample improves it more. Zero-shot + one-shot transfer. Not in the abstract.
⭐ best number	harmonize legacy Affy → liver-necrosis classifier: F1 0.636 → 0.718. Twenty years of dead data, made useful, on a real task.

TransTissueFormer — the translator


does	liver → kidney. 8,565 → 8,565, from 425 pairs.
direct	PCC 0.53. LI–KI only 0.40. “no definite conclusion can be drawn whether cross-tissue translation is possible.”
augmented	PCC 0.793, MAE 0.081→0.059, rare MAE 0.27→0.159.
validation	gemfibrozil → PPARα ✓, cisplatin → TP53 ✓, lead → p53 ✓. 3 of 44 reported.
negative	multi-task collapses 0.53 → 0.23.
⭐ my finding	96.6% of the model is the bottleneck — a gene embedding table. The 32-layer transformer is 3.4%.
⭐ my finding	the augmented task is exactly affine — ridge scores $ρ = 1.0000$ , MAE 0.0000, closed-form. Figure 7’s pretrain row measures linear-map approximation.

GenTox — the generalizer


does	predict a whole new column — a drug never tested. Inductive: $Q_{j, :} = f (features (j))$ .
basis	GNN pretrained on 1,000,000 compounds, self-supervised (contrastive = SimCLR; or graph InfoMax).
finding	InfoMax > contrastive ≫ Mordred ≈ Morgan. Learned beats hand-crafted.
⭐ theory	Theorem 1: unrelated profiles correlate at $ρ \to 1$ . Their own baseline: $ρ_{col} = 0.992$ , $ρ_{row} = 0.0084$ . A model that learned nothing, scoring 0.992.
⭐ theory	Theorem 2: on fold-change, $ρ \to 0$ . The two theorems cut opposite ways.
⭐ the tell	“As we do not perform inductive learning on the genes or tissues, the [gene] NN we use is a simply embedding layer.”
status	draft. `Figure ??`, `[?]` citations, §5 and §6 empty section headers.

13.3 The thread: one random table, four times ⭐⭐⭐

paper	the object
ToxCompl	$P$ — $375, 000 \times 300$	gene table, random init
TransPlatformer	$W$ — $31, 042 \times 512$	gene table, random init, 15.9M params from 888 samples
TransTissueFormer	$W$ — $8, 565 \times 512$	gene table, random init, 96.6% of the model, 425 samples
GenTox	col NN — $31, 099 \times 300$	gene lookup, random init

Four papers. Four different problems. The same object at the core of each, and it is random every time.

And GenTox is the sharpest case, because it proves the point and then doesn’t apply it.

GenTox went inductive on drugs, with a GNN pretrained on 1,000,000 compounds as the basis. That is a foundation model — built by this program, in 2024, before TransTissue wrote “we plan to explore the adaptation of these models in future work.” And their own ablation showed learned representations beat hand-crafted ones.

They never applied that argument to the other axis of the same matrix.

And there’s a public checkpoint of exactly the right shape

TransTissueFormer’s $W$ is $8, 565 \times 512$ .
scGPT’s gene embedding table is $\sim 60, 000 \times 512$ . (verified)

Same shape. Load it. No adapter, no dimension mismatch, no architecture change.

And it dodges the type error entirely — scGPT’s fold-change incompatibility lives in its value encoder (11_SC_FOUNDATION_MODELS.md §3), which none of these architectures have. They multiply the raw fold-change straight into the gene embedding. The broken part of scGPT is the part they don’t need.

13.4 The internal contradictions — where the program argues with itself

These are the most useful things in the whole analysis, because the program contains its own corrections.

(a) TransPlatformer proves the transfer; TransTissueFormer needs the token ⭐


TransPlatformer §3.3	withhold brain entirely, test brain → wins by 2.42×
TransTissueFormer §6	multi-task model → collapses 0.53 → 0.23

Not a contradiction. A proof.

TransPlatformer:  input = CodeLink profile of tissue X
                  output = BioSpyder profile of tissue X     ← SAME tissue
                  → tissue is a NUISANCE. Implicit in the input.
                  → no choice to make. Mixing tissues = free data. WORKS.

TransTissueFormer: input = liver profile
                   output = kidney? heart? spleen?           ← YOUR CHOICE
                   → tissue is a FREE PARAMETER, never supplied.
                   → model must choose, blind. Returns E[t|s] = the average.
                   → COLLAPSES.

TransPlatformer proves the shared trunk works. TransTissueFormer proves you need the target token. Add $e_{τ}$ and you get both.

This is no longer an argument imported from NMT. It’s the program’s own two results, read together.

(b) GenTox proves PCC is broken; everyone reports PCC ⭐

GenTox §2 proves — with a theorem and their own trained model scoring $ρ_{col} = 0.992$ / $ρ_{row} = 0.0084$ — that column-wise PCC is unreliable, and that row-wise metrics are mandatory.

TransTissueFormer reports column-wise PCC. Only.

The metric that would settle its central claim is the one the same program already argued for, in another paper.

And it matters more than it did in 2024: “The Metric Picks the Winner” (June 2026) shows model rankings inverting end-to-end with metric choice on drug-response prediction. GenTox was early. The field has caught up to it. GenTox is still a draft with two empty sections.

(c) GenTox proves learned > hand-crafted; genes stay random

§13.3. The argument is theirs. The axis is the only thing that changed.

(d) The augmentation is back-translation; they cite Sennrich and stop

TransTissue’s Algorithm 1 is back-translation. They cite Sennrich et al. 2016 — as [36], in the related-work list. Then stop.

Seven years of follow-up sits unused, and each item addresses a problem the paper reports: tagged BT (Caswell 2019) quarantines the provably linear artifact; noised BT (Edunov 2018) attacks the fact that Funk-SVD output is noiselessly linear; iterative BT (Hoang 2018) dilutes it. (08_AUGMENTATION.md §8.9.)

13.5 What’s genuinely good — and shouldn’t get lost

Being clear about this, because the analysis above is heavy on gaps:


The compute argument is unanswerable.	At $n = 31, 042$ , standard attention needs ~2 TB. On a 32GB V100S, 2 layers fit. There is no version of this work that uses vanilla attention over genes.
Refusing positional encoding is correct.	Genes have no order. Most people would have imposed one.
Imputing the whole matrix is clever.	Heart data creates liver–kidney training pairs. 43% more pairs, and the arithmetic checks out exactly.
The attention aggregator works.	MaxAE 3.99 → 0.83 is a large, real win on the metric that matters.
GenTox §2 is ahead of the field.	They proved the metric problem in 2024. The field rediscovered it in 2026.
§3.3’s zero-shot result is a real finding.	Buried in a paragraph.
The biological validation is the right species of evidence.	Gemfibrozil→PPARα, cisplatin→TP53, lead→p53. A toxicologist looked at a predicted profile and said yes, that’s lead poisoning. Worth more than every PCC.
F1 0.636 → 0.718 is the best number here.	Legacy data, harmonized, made a real classifier better at a real task.

This is good work with a metrics problem — and the metrics problem is diagnosed in their own GenTox paper.

13.6 What needs to be done

Ordered. Each tier gates the next.

TIER 0 — establish the ground truth (days; nothing below matters without it)

#	do	why
0.1	the mean predictor — ignore the source, predict the average target profile	Ahlmann-Eltze (2025): 5 FMs + 2 deep models fail to beat it. In one simulator regime it scores 0.797; TransTissueFormer reports 0.793.
0.2	row-wise PCC, alongside column-wise	Undefined for the mean predictor — that’s what makes it the metric that exposes it. GenTox §2.3 already argues for it.
0.3	Funk-SVD alone — Algorithm 1 line 3 already imputes the test targets. Read them out.	Ten minutes. The number exists inside the pipeline.
0.4	ridge on the augmented pairs, into Figure 7	Verified: scores $ρ = 1.0000$ . It would top that figure.
0.5	the strict protocol — withhold all non-source tissues at test columns	The current protocol lets brain and heart inform the test treatment. Deployment has only liver.
0.6	MAR vs MNAR	Table 3 decomposes exactly into 24 study panels (residual 0.0, all 24 structural zeros). Missingness is study design → plausibly MNAR, contra the stated MAR assumption.

These are not criticism. They’re the field’s central methodological problem — the Virtual Cell Challenge ran 1,200 teams and reported models “not yet consistently outperforming naive baselines.” Running them is the highest-value, lowest-risk contribution available.

TIER 1 — the gene table (days–weeks; touches 96.6% of one model)

#	do
1.1	initialize $W$ / $P$ from scGPT gene embeddings. Same shape ( $\times 512$ ). One tensor load.
1.2	ablate: random / co-expression from DrugMatrix itself ⭐ / scGPT / UCE-ESM2 / shuffled / ortholog-only
1.3	make the gene side inductive — $gene_factor (i) = RowNN (features (i))$ . GenTox’s own argument, other axis.

⚠️ The control is 1.2’s second row, not the first. “Pretrained beats random” is rigged when the system is underdetermined. Does scGPT beat DrugMatrix’s own co-expression? If it loses — consistent with Kedzierska and Souza & Mehta — that’s also a result, and it converts a hedge into evidence. Both outcomes publish.

What 1.3 buys beyond a leaderboard bump:

new platforms — CodeLink’s Cyp1a1 and Affymetrix’s Cyp1a1 are different lookup rows today. With features they’re the same feature. TransPlatformer’s problem, dissolved.
new species — rat→human. A lookup table cannot, ever. UCE tokenizes by protein sequence — no dictionary.
the underdetermined tissues — brain: 65 observations, 300 parameters. A shared function doesn’t need 300 observations per gene.

TIER 2 — the cheap fixes (hours–days each; all compose)

#	do	cost
2.1	target-tissue embedding $e_{τ}$	4,096 params. Fixes the 0.23 collapse by construction. TransPlatformer §3.3 already proved the trunk transfers.
2.2	tag the synthetic data (Caswell 2019)	512 params. Quarantines a proven artifact.
2.3	noise the augmentation (Edunov 2018)	Funk-SVD output is noiselessly linear — the extreme case.
2.4	iterate the augmentation (Hoang 2018)	Round 1 uses the trained model to re-impute → nonlinear → the artifact dilutes.
2.5	slot embedding $P \in R^{s \times r}$	8,192 params (0.18%). Block-1 attention is provably inert without it. Check the code first.
2.6	sweep $s$ and $r$	$s = 128, r = 128$ makes the transformer real. If performance is flat, the transformer isn’t contributing.

TIER 3 — the contributions (weeks–months)

#	do
3.1	⭐ enrichment-consistency as a metric. Score by whether predictions support the same biological conclusions. Robust to Theorem 1. The mean predictor scores zero by construction — identical enrichment for every treatment. TransTissue did this by hand for 3 of 44 and never automated it.
3.2	finish GenTox §5 (Gene Network Analysis). §2 spends three pages proving it’s essential, then leaves it an empty header.
3.3	unify GenTox + TransTissueFormer — one model, tissue embedding + compound embedding. Funk-SVD is the linear special case of CPA; this subsumes rather than replaces.
3.4	zero-shot to the 24 empty tissue pairs. The zeros are structural — no pairwise model can ever reach them. §3.3 proves the mechanism works.
3.5	the fold-change corpus — pseudobulk Tahoe ÷ DMSO + LINCS L5 + TG-GATEs + DrugMatrix ≈ 1.4M interventional differential signatures. Doesn’t exist. Nothing prevents it.

13.7 The one-paragraph version

Four papers, one matrix, four axes. ToxCompl fills holes with a rank-300 factorization — running at obs/param 1.23, marginal, and 0.74 (underdetermined) at the $r = 500$ they tried, which explains their own unexplained failure. TransPlatformer harmonizes machines, proves standard attention is impossible (2 layers fit on a V100S), and buries the best result in the program in §3.3: zero-shot to brain, 2.42× better than brain’s own model. TransTissueFormer translates organs, reports 0.793 with an augmentation that is provably an exact linear map, and reports a multi-task collapse that is textbook missing-target-token — which TransPlatformer §3.3 already showed how to fix. GenTox goes inductive on drugs with a GNN pretrained on 1,000,000 compounds — a foundation model, built here, in 2024 — proves learned beats hand-crafted, proves PCC is broken two years before the field agreed, and then says: “we do not perform inductive learning on the genes.”

All four have a randomly-initialized gene embedding table at their core. scGPT ships one, 512-dimensional, matching, trained on 33 million cells, with the incompatible part in a value encoder none of these architectures have.

Run the mean baseline first. Everything above is conditional on it.

13.8 The honest caveat

Everything in Tiers 1–3 is downstream of Tier 0.1.

If the mean predictor matches 0.793 on real DrugMatrix, then cross-tissue translation is mostly “what does a poisoned kidney look like in general” — and the premise needs rethinking, not extending. Most of 15_FRONTIER.md gets rewritten.

That is not a prediction. It is an unknown, and it is cheap to resolve. Which is exactly why it goes first.

Verified claims are tested in code/. Unverified ones are flagged in 17_SOURCES.md. §13.4(a)’s reading of the TransPlatformer/TransTissueFormer relationship is mine, though it follows directly from the two task definitions. Numbers are verbatim from the papers except where marked ⭐ (my analysis).

Bridging Genomics Foundation Models and Toxicogenomics