Chapter 5 — TransPlatformer: Translating Between Generations of Machines

Why it’s needed, what it does, and the finding buried in §5.5 that matters more than the headline.

Paper: “TransPlatformer – Translating Toxicogenomic Profiles Between Generations of Platforms” (ORNL / Sciome / NIEHS).

Read 04_TOXCOMPL.md first. This is the sibling of TransTissueFormer — same architecture, different axis.

5.1 Why this exists: a very expensive problem

Over 20 years, three generations of machines measured gene expression:

Here’s the problem in one sentence:

Decades of toxicology were run on CodeLink and Affymetrix. Modern work is BioSpyderWT. The old data can’t be pooled with the new. So it sits there, paid for, unusable.

Why can’t they be pooled? The two technologies produce different kinds of number:

• A microarray is a glass chip. Probes glow, and you measure brightness. That’s analog. It saturates, and it has background noise.
• A sequencer reads the RNA letter by letter and counts reads. That’s digital, with a huge dynamic range.

You can’t put a brightness and a count on the same axis. They aren’t the same kind of number.

NLP analogy: it’s the same language, transcribed by three different systems with different phoneme inventories and different error profiles. You have a huge corpus in the old transcription, and your model expects the new one. TransPlatformer is a normalizer.

But wait — doesn’t fold-change fix this?

Partly. And that’s the whole point. From 01_BACKGROUND.md §4.2: dividing treated by control makes the numbers unitless, so the platform’s scale factor cancels — to first order.

It doesn’t cancel all the way, because saturation and background are nonlinear. A gene at the top of CodeLink’s range is clipped; the same gene on BioSpyderWT isn’t. Fold-change doesn’t undo clipping.

That leftover part — the bit fold-change can’t remove — is exactly what TransPlatformer models. Everything else was already handled by the ratio.

5.2 What it actually does

$${\mathbf{x}}^{\text{CodeLink}}\in {\mathbb{R}}^{8,565}⟼{\mathbf{x}}^{\text{BioSpyderWT}}\in {\mathbb{R}}^{22,794}$$

Same tissue. Same treatment. Different machine.

⚠️ Notice the dimensions don’t match. $n=8,565\to m=22,794$. This is not a square problem. TransTissueFormer maps $n\to n$; TransPlatformer maps $n\to m$. Same architecture, but the output side has to be sized differently — hence 45 decoders ($\lceil 22794/512\rceil =45$) instead of TransTissueFormer’s 17.

The architecture — identical to TransTissueFormer

(B, n)  →  [n×s bottleneck]  →  (B, s)  →  [1×r]  →  (B, s, r)
        →  [L = 32 layers]   →  [⌈m/s⌉ = 45 decoders]  →  (B, m, r)  →  [r×1]  →  (B, m)

$s=512$, $r=16$, $L=32$, dropout 0.1, Adam $10^{-3}$, batch 16.

Everything in 07_TRANSTISSUEFORMER.md §7.3–7.6 applies word for word, including:

• The bottleneck is a gene embedding table — row $g$ of $W$ is a 512-dim vector for gene $g$.
• The bottleneck holds the overwhelming majority of the parameters. It’s worse here: on Affymetrix it’s $31,042\times 512=15.9\mathbf{million}$ parameters, against a ~150k transformer stack.
• The transformer never sees a gene.
• The 1×r projection makes the tokens collinear at layer 0.

5.3 The Seq2Seq ablation — a number worth remembering

They tried a plain Seq2Seq transformer. Treat the profile as a sentence, decode it one token at a time, <EOS> and all.

Here’s the result, on an NVIDIA Tesla V100S 32GB, sequence length 8,565, batch 8:

The number of layers that fit: 2.

Two. Against TransPlatformer’s 32.

And on the 2 layers that fit, TransPlatformer is 2.74× better on overall MAE and 2.27× better on rare-signal MAE. Seq2Seq loses to the MLP too.

This is why “we can’t use standard transformers” isn’t hand-waving. It’s not that they’re slow. At $n=31,042$ (Affymetrix) a standard transformer doesn’t run at all — see 07_TRANSTISSUEFORMER.md §7.4: about 2 TB of attention scores.

Their three reasons for rejecting Seq2Seq (their §2.2), all sound:

1. Memory. $O(n^2)$ attention on 31,042 tokens. Dead.
2. Genes aren’t ordered. “the first $s$ genes as tokens may not necessarily hint at the $(s+1)$th token.” Correct — there’s no order; there’s a graph.
3. Profiles are ~90% identical to each other. “two distinct transcriptomic profiles for two different treatments can share more than 90% of their content.” Feed those in as “sentences” and the model drowns.

Point 2 is right that there’s no order, which means no positional encoding — a set transformer. But that doesn’t mean no attention. Their conclusion overshoots their premise a little. Point 1 alone settles it, though.

5.4 Three training strategies — and this is the interesting part

The data is skewed (liver everywhere, brain almost nowhere). So how do you train?

Sample counts: CodeLink→BioSpyderWT = 2,225 pairs. Affymetrix→BioSpyderWT = 888.

Mixed-tissue — the headline

MAE cut by more than half versus ToxCompl (0.043 vs 0.09). PCC doubled versus the MLP (0.71 vs 0.37). Rare-signal MAE under 0.22.

Why compare against ToxCompl at all? Because ToxCompl can also fill in the missing entries — but it’s “agnostic to the relationship between data entries and the platforms.” So the comparison asks one precise question:

“whether additional side information, in this case the correspondence of genes across platforms, can help improve prediction performance.”

It does — by more than half. That’s a clean, well-posed ablation, and it’s the best-designed comparison in the four papers.

The mixed-tissue assumption, stated plainly:

“It assumes that the tissue-identifying signals are latent in the vectors, and with a powerful architecture and plenty of [data]…”

And here’s the thing: that assumption is correct — for this task. Hold that thought; §5.6 is about why.

5.5 Zero-shot transfer, and nobody is talking about it ⭐⭐⭐

This is the most important section in the paper, and it’s not in the abstract.

The problem: brain and intestine have almost no data. Single-tissue models for them fail.

So they tried this:

Train on $S=\{\text{LI, KI, HE, SM}\}$. Test on $T=\{\text{BR}\}$ or $\{\text{IN}\}$.

“Note that data from the target tissues are completely withheld from training.”

Zero brain data. Zero intestine data. Predict brain and intestine.

The result

A model that had never seen a brain beat the model trained on brain’s own data — by 2.42×.

What this means

Read it slowly, because it’s counterintuitive:

• The single-tissue brain model had brain data. It was trained on the actual target.
• The cross-tissue model had zero brain data. It saw liver, kidney, heart, muscle.
• The one with no target data won, by a factor of 2.4.

Why? Because brain’s own data is so scarce that training on it is worse than useless — you overfit ~65 samples. But the translation function itself — “how does CodeLink’s brightness map to BioSpyder’s counts?” — is largely tissue-independent. It’s a property of the machines, not the organ. So liver data teaches you the machine, and the machine is the same for brain.

Their own framing:

“Cross-tissue mode assumes that some fundamental relationships among genes are preserved across tissues and can be leveraged for translation.”

“Figure 6 suggests that the translation mechanism learned from one set of tissues may be applied to another set.”

And then they did few-shot

“To further improve the performance of the model, we include a very small fraction of the data (e.g., one data sample) from the target tissue in training.”

One sample. They call it Cross-tissue*.

NLP framing, and it’s exact:

This is massively multilingual NMT’s central finding. A low-resource language does better inside a multilingual model trained mostly on other languages than in its own bilingual model. Its own corpus is too small; the shared structure carries it.

§5.5 is zero-shot cross-lingual transfer, plus one-shot fine-tuning, in a toxicogenomics paper — reported as a paragraph and a bar chart.

5.6 The contradiction — and why it isn’t one ⭐⭐

Put two facts from the same program side by side:

Same group. Same architecture. Same data. Opposite outcomes. What gives?

The tasks are structurally different, and the difference is exactly conditioning:

TransPlatformer:
    input:   CodeLink profile of tissue X
    output:  BioSpyder profile of tissue X          ← SAME tissue
    → the tissue is a NUISANCE VARIABLE. It's implicit in the input.
    → the model never has to CHOOSE. Mixing tissues just adds data.
    → mixed-tissue WORKS.

TransTissueFormer:
    input:   liver profile
    output:  kidney? heart? spleen?                 ← DIFFERENT tissue, YOUR CHOICE
    → the tissue is a FREE PARAMETER. It is not in the input.
    → the model MUST choose and is never told which.
    → multi-task COLLAPSES.

The two results are perfectly consistent. Together they prove the conditioning argument from inside the program.

TransPlatformer doesn’t need a target token — the target tissue equals the source tissue, so it’s already in the input. Mixing tissues is free data.

TransTissueFormer does need one — the target tissue is a choice the model is asked to make blind. Under squared loss it correctly returns $\mathbb{E}[{\mathbf{t}}_c\mid {\mathbf{s}}_c]$, the average over organs (16_MATH_NOTES.md §4.2). Mush is the right answer to an ill-posed question.

So this is no longer my argument from the NMT literature. It’s the program’s own two results, read together. TransPlatformer proves the shared trunk works. TransTissueFormer proves you need the target token. Add ${\mathbf{e}}_{\tau }$ and you get both.

And §5.5 makes the zero-shot claim in 12_TRANSLATION_TRANSFER.md §12.4 far less speculative than I first rated it. Zero-shot to brain already worked once, on the platform axis, in this program. The tissue axis needs conditioning — but the transfer mechanism is demonstrated.

5.7 Attention interpretability — a real bonus

Translating CodeLink → BioSpyderWT, the model attends most to:

Cmya1, Ca3, Cyp1a1, Ctsh, Sds, Cited4, Atf3, Lcn2, Stac3

Several are well-known toxicology genes:

And per-tissue (heart): Arntl, Timm8a, Rufy1, Kap, Atf3, Sln, Gal, Alas2, Fos — Arntl is a circadian clock gene, Fos an immediate-early stress gene. Plausible for heart.

This is a genuine sanity check. The model attends to genes a toxicologist would name. It’s weak evidence — attention weights aren’t explanations, and that’s a whole literature — but it’s the right kind of evidence, and it costs nothing.

⚠️ Be careful here. “Attention is not explanation” (Jain & Wallace 2019) is a well-known result in your field, and it applies. High attention does not mean high importance. But “the model attends to Cyp1a1 when translating toxicogenomics” is still a reassuring smell test, and they present it as one rather than over-claiming.

5.8 The downstream result — the most convincing number in all four papers ⭐

Everything above is intrinsic evaluation. This is the one that matters.

Task: predict liver necrosis (does this compound damage the liver?) from a BioSpyderWT profile. A simple MLP classifier.

Experiment: train it on BioSpyderWT only. Then translate the legacy Affymetrix data into BioSpyderWT format with TransPlatformer, add it to the training set (nearly doubling it), and retrain the same model with the same hyperparameters.

That’s +12.9% relative F1. All four metrics improve.

Why this is worth more than every PCC in the four papers

PCC 0.71 tells you the predicted vector correlates with the true vector. So what? Nobody deploys a predicted vector.

This says something else: translated legacy data made a real classifier measurably better at a real task. The old CodeLink and Affymetrix data, sitting unusable for 20 years, became useful. That’s the paper’s actual claim, tested end to end.

It’s the same kind of evidence as TransTissueFormer’s enrichment validation (gemfibrozil → PPARα ✓), and it’s the direction 15_FRONTIER.md F6 argues the whole field should move: score models by whether they help the downstream task, not by whether the vectors correlate.

⚠️ One small reporting inconsistency: the abstract says F1 “improves by approximately 8%”; the results section says “12.9% improvement.” Both are right — 8.2 percentage points (0.636 → 0.718) versus 12.9% relative. Worth picking one convention and stating it consistently.

5.9 What’s still open

5.10 Summary

Why it’s needed. Three generations of machines. Microarrays measure brightness; sequencers count. You can’t pool them. Decades of paid-for toxicology data sit unusable. Fold-change cancels the platform to first order — but saturation and background are nonlinear and survive. That leftover is what TransPlatformer models.

What it does. $8,565\to 22,794$. Same tissue, same treatment, different machine. Same architecture as TransTissueFormer, 45 output decoders instead of 17.

The headline. MAE 0.043 vs ToxCompl’s 0.09 (>50% better); PCC 0.71 vs the MLP’s 0.37 (doubled). The ToxCompl comparison is well-posed — it isolates “whether the correspondence of genes across platforms helps.” It does.

The Seq2Seq number. On a 32GB V100S at length 8,565: 2 layers fit, versus 32. TransPlatformer is 2.74× better on MAE. The compute argument is not hand-waving.

The finding nobody mentions (§5.5). Train on {liver, kidney, heart, muscle}, withhold brain entirely, test on brain: 2.42× better than the model trained on brain’s own data. Then one brain sample improves it further. That is zero-shot and one-shot cross-lingual transfer. It’s not in the abstract.

The contradiction that isn’t. TransPlatformer’s mixed-tissue works; TransTissueFormer’s multi-task collapses. Consistent: TransPlatformer’s target tissue equals its source tissue (implicit in the input, no choice to make), while TransTissueFormer’s is a free choice it’s never told. The program’s own two results prove the target-token argument from the inside.

The best number in all four papers. Harmonize legacy Affymetrix into BioSpyderWT, add it to a liver-necrosis classifier: F1 0.636 → 0.718, all metrics up. Twenty years of unusable data, made useful, measured on a real task.

Numbers are verbatim from the paper. §5.6’s reading of the TransPlatformer/TransTissueFormer relationship is mine — it follows from the two task definitions, but I’d want the authors to confirm the framing. Architecture claims inherit the caveats in 07_TRANSTISSUEFORMER.md §7.6.