Chapter 4 — ToxCompl and ToxCompl+, and Matrix Factorization From Zero

This is the engine under the whole program. It assumes nothing. Run code/demo_matrix_factorization.py next to this chapter — it prints every number below.

Read this before Chapters 1–3. ToxCompl is the thing TransTissueFormer’s augmentation runs on. It’s what GenTox generalizes. And it’s where a foundation model plugs in most naturally.

⚠️ Two papers, one lineage. Get the order right.

This confused me at first, so here it is plainly:

		what it adds
1. ToxCompl	bioRxiv 2024, Auerbach co-author	the original: matrix completion, plus hybrid sampling and Bayesian optimization, plus biological validation. Deployed publicly by NIEHS.
2. ToxCompl+	the sequel	built on top of ToxCompl. Adds side-information and attention (§4.8–4.9).

ToxCompl came first. ToxCompl+ is the follow-up. §4.8–4.9 are the sequel’s additions. §4.12 is what the original does — hybrid sampling, a proper rare-signal metric, a real baseline, and biological validation.

The matrix-factorization core (§4.1–4.6) is shared by both, and it’s the part that lasts. Read §4.12 before repeating any criticism — two critiques in this chapter don’t hold against the original paper.

4.1 The magic trick

Four people, four movies, ratings 1–10:

Look at Bob’s row: [4, 1, 4.5, 0.5]. Now Alice’s: [8, 2, 9, 1].

Bob is exactly Alice divided by 2. Every single entry.

Carol is Alice times 1.5. Dave is Alice divided by 4. Exactly, every entry.

So every row is just a multiple of Alice’s row. That is what “rank 1” means. Nothing more.

Write it compactly. Give each person one number, and each movie one number:

$$\mathbf{p}=[1,0.5,1.5,0.25]\text{one number per person}$$ $$\mathbf{q}=[8,2,9,1]\text{one number per movie}$$ $$M_{ij}=p_i\cdot q_j$$

Check: Carol’s RoboCop rating is $p_3\cdot q_3=1.5\times 9=13.5$. ✓

Now count the numbers:

Eight numbers describe sixteen. So you don’t need to see all sixteen to know them.

4.2 So you can fill in holes

Say Bob rated only one movie:

Bob rated Alien 4. Alice rated Alien 8. So Bob’s scale is $4/8=0.5$.

That fixes his whole row: $0.5\times$ Alice’s row = [4, 1, 4.5, 0.5].

predicted: [4.  1.  4.5 0.5]
truth:     [4.  1.  4.5 0.5]
error:     0.0000000000

One observation. Three holes filled. Exactly right.

That is matrix completion. That’s the whole idea. Everything after this is just making it work when the structure is only approximate, not exact.

4.3 Rank 2, rank $r$

Rank 1 is too rigid. It says everyone has the same taste and differs only in how generous they are. But real people differ in kind, not just in degree.

Fix: give each person two numbers, and each movie two numbers.

$${\mathbf{p}}_i=[\text{likes ACTION},\text{likes ROMANCE}],{\mathbf{q}}_j=[\text{has ACTION},\text{has ROMANCE}]$$

$$M_{ij}=p_{i,1}{q}_{j,1}+p_{i,2}{q}_{j,2}\leftarrow \text{a dot product}$$

• Alice on Alien: $9(9)+1(1)=82$ — action fan, action movie. High. ✓
• Bob on Alien: $2(9)+8(1)=26$ — romance fan, action movie. Low. ✓

Rank 1 could not do this. It forced Bob to like whatever Alice liked, only weaker. Rank 2 lets Bob like different things.

The general form

$$\boxed{M_{ij}=\sum _{f=1}^r{P}_{if}{Q}_{jf}=⟨P_{i,:},Q_{j,:}⟩}$$

If $r$ is small, then $(n+m)r$ is far less than $nm$. Fewer unknowns than entries. That gap is what lets you fill holes.

This is the entire mathematical content of ToxCompl.

4.4 Swap the nouns: this is DrugMatrix

Change the words, keep the math:

Here is ToxCompl’s model, straight from the paper:

$$G_{ij}\approx b_i+b_j+\sum _{f=1}^r{P}_{if}{Q}_{jf},r=300$$

Two extra terms:

• $b_i$ is the gene bias — “this gene is generally jumpy.”
• $b_j$ is the treatment bias — “this drug is generally toxic.”

Why believe low rank here?

The paper’s argument, and it’s a good one:

“there are similarities between the drugs, for example, Doxorubicin and Epirubicin, and the genes form co-expression networks”

In plain terms:

• There are 3,000 columns, but only 636 distinct drugs. Many columns repeat.
• Doxorubicin and Epirubicin are nearly the same molecule. So their columns are nearly the same. Redundant.
• Genes move in co-regulated groups. So rows are redundant too.
• The same gene is measured in 8 organs. More redundancy.

Redundant rows and columns are exactly what “low rank” means.

4.5 How you actually fit it

There’s no formula that solves this in one step. The problem is non-convex. So you go downhill:

1. Start $P$ and $Q$ at random values.
2. Measure the error on the entries you did observe.
3. Nudge $P$ and $Q$ in the direction that lowers that error.
4. Repeat.

$$\underset{P,Q,b}{\min }\sum _{(i,j)\in \Omega }(G_{ij}-(b_i+b_j+⟨P_{i,:},Q_{j,:}⟩){)}^2$$

⚠️ The sum runs only over observed entries ($\Omega$). Missing entries add nothing to the loss. They get filled in purely because the low-rank structure forces the whole table to stay consistent. That’s the trick.

Optimizer: Adam, learning rate $10^{-3}$, weight decay $5\times 10^{-4}$.

4.6 The counting that decides everything

Take a 12×10 table of true rank 2. Change how much of it we observe:

That table is the whole lesson. There’s a sharp jump right around obs/param = 1.

Matrix completion isn’t magic. It’s just solving a system of equations. Each observation is an equation. Each entry of $P$ and $Q$ is an unknown.

• Many more equations than unknowns → clean recovery.
• Barely more → fragile.
• Fewer → arbitrary. Garbage.

Now count the real matrix

DSMatrix is 375,000 rows × 3,000 columns = 1.12 billion entries. 12.3% are observed = 138 million.

$$\text{observations per gene row}=\frac{138,000,000}{375,000}\approx 369$$

To fit one gene’s vector $P_{i,:}\in {\mathbb{R}}^r$, you need at least $r$ observations in that row:

This explains a failure the paper reports but doesn’t explain

ToxCompl+ Figure 2 shows something odd: going from $r=300$ to $r=500$ made MAE better but MaxAE worse. The paper calls this surprising and moves on.

Here’s why it happens.

At $r=500$, each gene row has about 369 observations but 500 free parameters. The row is underdetermined. The fit is arbitrary in roughly 131 directions.

And that’s the average row. Brain shows up in only ~65 treatments. So a brain gene has 65 observations for 500 parameters — underdetermined by nearly 8×. Its latent vector is essentially random in most directions.

Even $r=300$ is only marginal (ratio 1.23). ToxCompl is running at the edge of identifiability, and the tissues with the least data fall off that edge.

Remember this. It’s exactly what a foundation model fixes (§4.10).

4.7 Where ToxCompl breaks: the sign flip

ToxCompl+ Figure 2 reports this, moving from DrugMatrix to the bigger DSMatrix:

And at the worst single point:

$$\text{target}=-0.77\text{(gene SUPPRESSED)}\text{predicted}=+0.94\text{(gene ACTIVATED)}$$

The sign flipped. This isn’t “a little off.” It’s the opposite conclusion.

Why it happens

Low rank assumes everything is a mix of $r$ shared patterns. But a rare extreme signal is, by definition, not a shared pattern. It’s the part of the table that isn’t low-rank at all. So the model can’t represent it.

The demo reproduces this. A 200×150 matrix, true rank 4, plus rare spikes on 1% of the entries:

typical |value| in the smooth part : 1.48
spike magnitudes                   : 6 to 10   ← rare, large, NOT low-rank

held-out MAE on NORMAL entries : 0.178
held-out MAE ON SPIKES         : 7.845   ← 44x worse
MaxAE                          : 10.099
SIGN FLIPS on held-out spikes  : 49%     ← the ToxCompl failure

49% sign flips on the rare signals. A coin toss. The average looks fine, but the only entries with toxicological meaning are wrecked.

This is why MAE alone can fool you. The average hides the failure.

✅ But credit where it’s due: the ToxCompl authors saw this exact problem and built for it. The published paper (§4.12) doesn’t hide behind MAE. It reports Mean F1 across the five categories precisely to check whether the rare up/down signals survive, and its main contribution — hybrid sampling — exists to preserve them. So “MAE hides the failure” is a fair point about the metric in general, but not a fair criticism of this paper, which measures the right thing and reports it.

4.8 ToxCompl+ fix 1 — side information

The idea: we know things about the rows and columns that the plain model ignores.

Two treatments labeled “Doxorubicin at 100mg” and “Doxorubicin at 200mg” should end up with similar latent vectors. But plain ToxCompl doesn’t know they share a drug. To it, they’re just column 47 and column 112.

So embed each feature and add the interactions:

$${\hat{G}}_{ij}=b_i+b_j+P_{i,:}\cdot Q_{:,j}+\underset{\text{row}\times \text{col-features}}{\underbrace{\sum _{t=1}^k{P}_{i,:}\cdot C_{:,j}^t}}+\underset{\text{row-features}\times \text{col}}{\underbrace{\sum _{t=1}^k{R}_{:,i}^t\cdot Q_{j,:}}}$$

Result (Figure 3): MaxAE 3.99 → 3.27, but MAE 0.03 → 0.05.

That’s a trade, not a clean win. Better on the extremes, worse on the average.

4.9 ToxCompl+ fix 2 — attention (the signature move)

Plain factorization:

$${\hat{G}}_{ij}=P_{i,:}\cdot Q_{j,:}=\sum _f{P}_{if}{Q}_{jf}$$

Every one of the $r=300$ dimensions counts equally. That’s just what a dot product is — an unweighted sum.

But maybe dimension 7 matters a lot for this gene-treatment pair and not at all for another one. A plain dot product can’t say that.

So ToxCompl+ learns two extra tables, $P^{\prime }$ and $Q^{\prime }$. Their only job is to decide how much each dimension counts:

$$\boxed{{\hat{G}}_{ij}=(P_{i,:}\circ Q_{j,:})\cdot \sigma (P_{i,:}^{\prime }\circ Q_{j,:}^{\prime })}$$

Here $\circ$ is the elementwise product — multiply position by position, don’t sum. And $\sigma$ is softmax.

Worked out, with $r=4$

P[i,:]           = [ 2.0  -1.0   3.0   0.5]
Q[j,:]           = [ 1.0   2.0  -1.0   4.0]
P ∘ Q  (values)  = [ 2.0  -2.0  -3.0   2.0]     ← "what each dim says"

P'[i,:]          = [ 0.1   0.2   3.0   0.1]
Q'[j,:]          = [ 0.5   0.3   2.5   0.2]
P' ∘ Q'          = [0.05  0.06   7.5  0.02]
softmax(P' ∘ Q') = [0.00  0.00  1.00  0.00]     ← "how much each dim counts"

PLAIN dot product  : sum(P ∘ Q)        = -1.000
ATTENTION-weighted : (P ∘ Q) · softmax = -2.994

The plain version gives every dimension weight 1. The attention version gives dimension 3 weight ≈ 1.00 and the rest ≈ 0. So the answer is dominated by dimension 3’s value, which is −3.0.

Why this really is attention

values   = P ∘ Q                (r of them)
weights  = softmax(P' ∘ Q')     (sum to 1)
output   = Σ values × weights

That’s attention with $r$ “positions” and one query. Same three lines as a transformer. The paper is right to call it attention.

Result: MaxAE 3.27 → 0.83, MAE → 0.02. A large win.

The fair objection

Is the win from attention, or just from doubling the parameters ($P,Q\to P,P^{\prime },Q,Q^{\prime }$)?

The paper answers this in part. Raising $r$ from 300 to 500 also adds parameters, and that made things worse. So extra capacity alone doesn’t explain it. Decent evidence.

Not airtight, though. The clean test is to run $P^{\prime },Q^{\prime }$ with $\sigma =\mathrm{id}$ (identity instead of softmax). That would isolate the nonlinearity from the extra capacity. Worth asking whether it was run.

(Note from §4.6: the $r$=500 comparison is confounded anyway — it went underdetermined, which is a different failure from “more capacity didn’t help.” So the ablation is still needed.)

4.10 Where a foundation model plugs in

Look at $P$ again.

$$P\in {\mathbb{R}}^{375,000\times 300},\text{row }i\text{ is a 300-dim vector describing gene }i$$

$$\boxed{\text{P IS A GENE EMBEDDING TABLE.}}$$

Same realization as TransTissueFormer’s bottleneck (07_TRANSTISSUEFORMER.md). And right now it is randomly initialized and learned from about 369 observations per row.

From §4.6: at $r=300$ that’s a ratio of 1.23 — marginal. At $r=500$ it’s underdetermined. For brain genes (~65 observations, 300 parameters) it’s hopeless.

A pretrained gene embedding is a prior. And a prior is the only thing that can fix an underdetermined system.

This isn’t a nice-to-have. When you have 65 observations and 300 unknowns, you must bring in outside information, or the answer is arbitrary. scGPT’s gene embeddings are outside information — learned from 33 million cells.

Demonstrated

Set up genes in three co-regulated modules. Starve the factorization of data so the system is genuinely underdetermined, like the real low-data tissues:

matrix 60x50, true rank 4, observed 10%
observations: 286   free parameters: 440   ratio: 0.65   ← UNDERDETERMINED

                                    held-out MAE     MaxAE
------------------------------------------------------------
random init  (ToxCompl today)             1.9851   13.7905
PRETRAINED init (scGPT-style)             0.6886    5.4533

Better MAE and better MaxAE — and MaxAE is the metric ToxCompl+ was built to fix. The prior does the job the data can’t, because the data isn’t there.

⚠️ The control that actually matters

The real comparison is not “pretrained vs random.” That one is rigged — of course a prior helps when the system is underdetermined.

The real question is: does scGPT beat the co-expression structure you can compute from DrugMatrix itself? DrugMatrix already tells you which genes move together. That’s a free prior, with no foundation model involved.

If scGPT loses to DrugMatrix’s own co-expression, that’s a result too. It’s what Kedzierska et al. (2025) and Souza & Mehta (2026) would predict. It turns “we plan to explore this in future work” into “we tested it, here’s the evidence.”

Either outcome publishes. That’s what makes it the right first experiment.

4.11 ToxCompl vs ToxCompl+ side by side

The last row is the opening. Both versions learn a 375,000 × 300 gene embedding table from scratch, from about 369 observations per row, in a system that’s marginally determined at best.

4.12 What the published paper actually does

§4.8–4.9 covered ToxCompl+, the sequel. This section is the original ToxCompl it was built on: “Completion of the DrugMatrix Toxicogenomics Database using ToxCompl”, Cong, Patton, … Auerbach. It’s the foundational paper of the whole program, and it’s more careful than the chapter above might suggest. Here’s what it really does, and what it corrects.

The real fix: hybrid sampling, not attention

The published version’s answer to the rare-signal problem (§4.7) is not side-information-plus-attention. It’s hybrid sampling guided by Bayesian optimization:

• The 5 categories are wildly imbalanced. 92% of values sit in category 0 (“no change”).
• Left alone, the model just learns to predict 0 and ignores the rare extremes.
• So they over-sample and under-sample the minority categories, and use Bayesian optimization to find the best mix.

Result (their Figure 2): Mean F1 about 2.6× better than plain matrix completion at recovering the rare up/down signals, while keeping MAE about the same.

That’s the same instinct as “class weights for an imbalanced classifier” (12_TRANSLATION_TRANSFER.md §12.8), reached from the completion side.

They do report a real metric — and a baseline

And there is a baseline: Nearest-Neighbor-Mean (NNM, $k=10$). For each missing entry, average its 10 nearest measured neighbours in the same row. ToxCompl beats it per-organ (their Figure 1).

⚠️ Correction to the rest of this book. Two critiques I make elsewhere are too harsh against this paper:

• “they only use MAE” — false. They report F1 and Mean F1, and Mean F1 is the number the method exists to move.
• “no baselines” — partly false. NNM is a real baseline for the completion task. (It is not the cross-tissue mean-predictor from 14_RESEARCH_AGENDA.md §A0 — that’s a different task — so that critique still stands for TransTissueFormer. But for ToxCompl completion, a baseline exists.)

They did the biological validation — the gold standard

Not just numbers. The published paper checks the completed matrix against biology:

• connectivity analysis — do predicted profiles cluster with measured ones the way real biology does?
• pathway / MOA analysis — do the differentially-expressed gene lists point at the right mechanisms?
• transcriptional biomarkers of tissue toxicity
• predicted apical endpoints — does the filled-in data predict actual organ damage?

This is exactly the enrichment-consistency direction 15_FRONTIER.md F6 argues the field should move toward — and ToxCompl is already doing it. It just hasn’t propagated to TransTissueFormer.

And they name their own weakness — which matters to a regulator

The paper flags a positivity bias: when predicting histopathology, the model over-predicts damage. It cries “toxic” too often.

That’s the honest, important kind of failure. A false alarm (“this safe chemical looks toxic”) wastes a real animal study. And in the phase-out world of 02_TOXICOLOGY_CONTEXT.md, a screening tool that over-flags is a specific, named problem that regulators care about. That’s worth more attention than any MAE number.

It’s a deployed resource, not just a paper

The completed DrugMatrix is released publicly by NIEHS: rstudio.niehs.nih.gov/complete_drugmatrix. Gene expression, clinical chemistry, hematology, and histopathology — filled in and browsable.

This confirms 02_TOXICOLOGY_CONTEXT.md’s central point: this is a regulatory toxicology program that uses ML, not an ML program that happens to use tox data. The output is a government resource, not a leaderboard entry.

They know about the GAN competitors

The paper cites ToxGAN and TransOrGAN — the FDA NCTR generative work behind TransTox (03_TOXICOGENOMICS_RESEARCH.md §3.5). So the ORNL group is aware of the FDA group’s methods. That softens the “they ignore TransTox” worry somewhat — though TransTissueFormer specifically still under-cites it.

What still stands

The durable critiques survive:

• The obs/param counting (§4.6) is still the best explanation for the $r$=500 MaxAE failure, and the paper still doesn’t give it.
• The gene table is random-init (§4.10). A pretrained prior is still the natural fix.
• Row-wise metrics. F1 is a real metric, but it’s still per-entry. The gene-network / row-structure argument (16_MATH_NOTES.md §3.5) is untouched.

4.13 Summary

Matrix factorization. Every row is a combination of $r$ shared patterns. So $(n+m)r$ numbers describe $nm$ entries. Fewer unknowns than entries means you can fill holes. ToxCompl: $G_{ij}\approx b_i+b_j+⟨P_{i,:},Q_{j,:}⟩$, $r=300$, fit by Adam on observed entries only.

Why it breaks. Rare extreme signals are the part that isn’t low-rank. MAE improves while MaxAE degrades, and signs flip: target −0.77, predicted +0.94. Reproduced: 49% sign flips on held-out rare signals.

A quantitative reason the paper doesn’t give. About 369 observations per gene row. At $r=300$ the ratio is 1.23 (marginal). At $r=500$ it’s 0.74 (underdetermined). That’s why raising $r$ made MaxAE worse. Brain genes (~65 obs, 300 params) are hopeless.

ToxCompl+ fix 1 — side information. Use drug name, dose, organ. MaxAE 3.99 → 3.27, MAE 0.03 → 0.05. A trade.

ToxCompl+ fix 2 — attention. $(P\circ Q)\cdot \sigma (P^{\prime }\circ Q^{\prime })$. Values from one pair of tables, weights from another. Literally attention with $r$ positions and one query. MaxAE 3.27 → 0.83. A large win.

Where the foundation model goes. $P$ is a gene embedding table, randomly initialized, fit from ~369 observations per row for 300 parameters. That system is barely determined, and for low-data tissues it’s not determined at all. A pretrained gene embedding is a prior — the only thing that can fix an underdetermined system. Control against DrugMatrix’s own co-expression, or the comparison is rigged.

Every number here comes from code/demo_matrix_factorization.py. The obs/param analysis uses the paper’s stated DSMatrix dimensions (375,000 × 3,000, 12.3% observed) and its stated $r$ values. The underdetermination explanation for the $r$=500 failure is my reading, not the paper’s — it follows from the counting, but should be checked against the implementation.