Medical Report Assistant

The brief

There are two phrasings you will hear, and they are not the same question.

“Design a system that reads clinical reports, summarizes them, and highlights the key findings so clinicians spend less time on documentation.”

“A hospital group wants to cut the time their physicians spend reading and writing notes. Build it. They have Epic, they have thirty thousand clinicians, and their general counsel will be in the room.”

The second version is the real one. The mention of counsel is the interviewer telling you what they want to hear about.

In plain terms, a clinician has a patient with a long record and perhaps twelve minutes. The record holds prior visits, discharge summaries, radiology and pathology reports, labs, and medication history. The product reads the record and produces a short, structured, sourced summary: what happened, what the key findings are, what changed since last time, and what is outstanding. Every statement links back to the exact sentence in the exact document it came from. A physician reads it, checks the parts that matter, and signs.

Two things must be said within the first three minutes, because they determine whether the interviewer thinks you have ever been near healthcare.

First, hallucination is categorically unacceptable here. It is not a matter of “we should minimise it.” A fabricated allergy, an invented negative finding, or a dropped “no” in front of “evidence of malignancy” is a patient safety event. So the system must be architected so that ungrounded output is structurally difficult, not merely discouraged by a prompt.

Second, this product assists, it does not diagnose. That distinction is not marketing. It is the difference between a workflow tool and a regulated medical device, and it shapes the interface, the model’s scope, and the entire release process.

What I’d ask first

Who reads the output, and do they sign it? A summary a physician reviews and attests to is a fundamentally different regulatory and design object from text that flows into the chart automatically. If a human signs, you have a human-in-the-loop system with defined accountability. If nothing signs, you have built an unattended clinical system, and almost every downstream answer changes.

Does the output go into the medical record? Text that lands in the chart becomes part of the legal record. It is discoverable in litigation, and it propagates to every downstream reader and every downstream algorithm forever. A summary shown transiently in a side panel has a much smaller blast radius. I would strongly push for the side panel in v1.

Summarise, or interpret? “Patient’s HbA1c was 8.2% on 3 March, up from 7.1% in December” is a summary of recorded facts. “Patient’s diabetes is poorly controlled and warrants escalation” is an interpretation, and it moves you towards being a device. Where this line sits determines the FDA question, so I would ask it explicitly.

What is the deployment environment, and where can PHI go? Protected health information, or PHI, is the identifiable health data HIPAA governs, and it determines your entire infrastructure. The options are on-premises, cloud under a business associate agreement, or a fully isolated tenancy. Each is buildable, and they have very different costs and iteration speeds.

What is the clinical setting? The options include inpatient rounding, outpatient pre-visit review, radiology reading, emergency triage, and oncology tumour board. These have different tempos, different documents, different risk profiles, and different definitions of “key finding.” Emergency medicine has minutes and high acuity. A pre-visit summary has hours and can be prepared in batch overnight. Anyone who designs “a medical summarizer” without picking one is designing nothing.

Who owns clinical validation, and who signs off? There will be a clinical governance committee, and they are the actual gate. Knowing whether they exist and what evidence they want matters more to shipping than any architectural choice.

What I’ll design against

Outpatient specialty clinics in a US hospital group. The use case is pre-visit summarisation of a patient’s record for the treating physician, plus highlighting of findings requiring follow-up. Output renders in a side panel in the EHR. It is not written to the chart, and it is never signed. Summarisation is strictly extractive-first over documented facts, with no diagnostic or treatment recommendations. Cloud deployment under a BAA with an enterprise model provider, with zero retention and no training on customer data. The physician remains the decision-maker at all times. The target is reading time, not decisions.

The design

        SOURCE SYSTEMS                      INGESTION
 ┌────────────────────────┐        ┌────────────────────────────┐
 │ EHR (FHIR / HL7 v2)    │        │ · FHIR client, incremental │
 │ notes, labs, meds,     ├───────▶│ · HL7 listener for feeds   │
 │ problems, allergies    │        │ · consent + purpose check  │
 ├────────────────────────┤        │ · audit log EVERY access   │
 │ Radiology / path (PDF, ├───────▶│ · OCR for scanned docs     │
 │ scanned faxes)         │        └─────────────┬──────────────┘
 └────────────────────────┘                      ▼
                                    ┌────────────────────────────┐
                                    │ DOCUMENT PROCESSING        │
                                    │ · segment into sections    │
                                    │ · sentence-level IDs ◀── the
                                    │ · normalize codes          │  key
                                    │   (LOINC/RxNorm/SNOMED)    │  primitive
                                    │ · timeline construction    │
                                    └─────────────┬──────────────┘
                                                  ▼
   ┌──────────────────────────────────────────────────────────────────┐
   │  PHI-BOUNDARY STORE (encrypted, tenant-isolated, audited)        │
   │  documents · sentence index · structured timeline · embeddings   │
   └───────┬──────────────────────────────────────────┬───────────────┘
           ▼                                          ▼
 ┌────────────────────────┐              ┌──────────────────────────┐
 │ DETERMINISTIC LAYER    │              │ RETRIEVAL / SELECTION    │
 │ · lab trends & deltas  │              │ · which docs matter for  │
 │ · med reconciliation   │              │   this visit + specialty │
 │ · overdue screenings   │              │ · recency + relevance    │
 │ · critical-value flags │              └────────────┬─────────────┘
 └───────────┬────────────┘                           │
             └──────────────────┬────────────────────┘
                                ▼
                 ┌───────────────────────────────────┐
                 │ SUMMARIZATION (LLM)               │
                 │ · every sentence emits source IDs │
                 │ · negation preserved verbatim     │
                 │ · no diagnosis, no recommendation │
                 └────────────────┬──────────────────┘
                                  ▼
                 ┌───────────────────────────────────┐
                 │ GROUNDING VERIFIER (non-optional) │
                 │ · entailment check per sentence   │
                 │ · numbers/dates/meds exact-match  │
                 │ · negation & laterality check     │
                 │ · unsupported → drop or flag      │
                 └────────────────┬──────────────────┘
                                  ▼
                 ┌───────────────────────────────────┐
                 │ CLINICIAN UI (read-only panel)    │
                 │ click any claim → source sentence │
                 │ highlighted in original document  │
                 │ + feedback: wrong / missing / ok  │
                 └───────────────────────────────────┘

Ingestion. Use FHIR for structured data such as labs, medications, problems, allergies, and encounters. Use HL7 v2 feeds where the integration is older, which is most places. Unstructured clinical notes are the substance of the problem. Many are dictated, many are templated with vast boilerplate, and a meaningful fraction of outside records arrive as scanned faxes that need OCR. OCR quality on a third-generation fax of a handwritten note is genuinely poor, and a summariser that confidently summarises garbled OCR is a hazard. So set confidence thresholds on OCR, and exclude low-confidence regions from summarisation rather than guessing at them.

Every single access to a record is authorised against the clinician’s relationship with that patient, and every access is logged. The audit log is not observability. It is a regulatory artifact.

Document processing, the key primitive. Segment each document into sections such as chief complaint, history, assessment, plan, findings, and impression. Then segment into sentences, and give every sentence a stable identifier. Everything else in the system hangs off this. A claim in the summary is a claim plus a set of sentence IDs. Citation is not a feature added at the end. Citation is the data model.

Normalise codes so that “Tylenol,” “acetaminophen,” and “paracetamol” are one entity. Use LOINC for labs, RxNorm for medications, and SNOMED for problems. Normalisation also lets you compare lab values from three institutions with different reference ranges.

Build a timeline with every event dated and ordered. Clinical reasoning is temporal, and most summarisation failures are temporal. The model attributes an old finding to the present, or it loses the sequence of a treatment course.

Deterministic layer. A large fraction of what a clinician wants is computable without a model, and it should be. That includes lab trends and deltas, values outside reference range, medication reconciliation between lists, overdue screenings by guideline, allergy-prescription conflicts, and care gaps. These are rules and queries. They are exact, and they are testable. Compute them in code and pass them to the summariser as facts, exactly as in the finance chapter.

Retrieval and selection. A patient with a twenty-year record has thousands of documents, and you cannot summarise them all. Selection is a real modelling problem. Which documents are relevant to this visit, this specialty, and this question? Recency helps but is not sufficient, because a cardiologist needs the echo from four years ago. Get selection wrong and the summary is confidently incomplete, which is the most dangerous output the system can produce.

Summarisation. Use an LLM, constrained hard. Lean extractive. Prefer the source’s own words, especially for findings, and quote rather than paraphrase where paraphrasing risks meaning. Every generated sentence must carry the sentence IDs it derives from. Preserve negation and uncertainty language verbatim. Phrases like “no evidence of,” “cannot exclude,” and “likely represents” are clinically load-bearing, and compression destroys them. No diagnostic statements, no treatment recommendations, and no inference beyond what is written.

Grounding verifier. This is a separate, non-optional stage, and it is the component that makes the product shippable. For each generated sentence, ask whether the cited source text entails it. Run a natural language inference model. Then run deterministic checks, which are more reliable than any model. Every number, date, dose, and medication name in the output must appear exactly in the cited source. Negation polarity must match. Laterality, meaning left or right, must match. Anything unsupported is dropped or surfaced as unverified. It is never rendered as if it were sourced.

This costs an extra inference pass per summary, and it is the least negotiable spend in the system.

Clinician surface. The clinician sees a read-only panel beside the chart. Every claim is clickable, and it jumps to the highlighted source sentence in the original document. Nothing auto-populates the note. Every claim carries one-tap feedback: wrong, missing, or useful. Clinicians will not write bug reports, but they will tap a thumb.

Where the AI actually is

The model does three things. It selects which documents matter, compresses language while preserving meaning, and phrases the result readably. That is genuinely a language task, and there is no non-LLM substitute for it.

Everything numeric and everything rule-shaped is code. That covers lab deltas, trends, out-of-range flags, medication reconciliation, drug interactions, guideline-based care gaps, and allergy conflicts. These have correct answers, and they must be exactly right. Asking a language model to compute them trades a guarantee for a probability. Never make that trade in a clinical setting.

What I would deliberately not use an LLM for:

• Diagnosis or treatment recommendation. This is both a regulatory decision and a safety one. The moment the system says “consider starting a statin,” you are in device territory, and you have taken on clinical liability the product is not built to carry.
• Risk scoring. Sepsis prediction, readmission risk, and deterioration are classical ML problems with validated approaches and calibration requirements. Under the ONC HTI-1 rule they also carry specific transparency obligations for predictive decision support interventions in certified health IT, including 31 required source attributes covering development data, fairness, validation, and performance (ONC DSI fact sheet). An LLM is the wrong estimator and the wrong compliance posture.
• Coding for billing. Assigning ICD-10 or CPT codes has fraud implications. Use it to suggest to a certified coder, never to submit.
• Anything with a numeric answer, per above.
• Deciding what to hide. The system may rank and it may collapse. However, a design where the model silently withholds information from a clinician is not defensible after an adverse event. Everything is reachable, and only the ordering is modelled.

The honest split here is the starkest of the three chapters. The model work is maybe 15% of the effort. That work is the prompt, the retrieval strategy, the verifier, and the eval set. The other 85% is EHR integration, which is famously the hardest integration work in software. It is also FHIR and HL7 plumbing, OCR pipelines, identity and consent, audit logging, encryption and key management, BAAs and vendor diligence, the clinical validation study, the governance sign-off, IT security review at each hospital, clinician training, and a UI that fits a workflow where the user has eleven minutes and four other tabs open. A candidate who spends the whole interview on the prompt has failed the question.

Compliance: HIPAA, PHI, and the device question

This deserves its own section, because it is where the interview is actually going.

PHI and the BAA. Any vendor that processes PHI on your behalf is a business associate, and it needs a business associate agreement. That includes your model provider. The major providers will sign a BAA under enterprise terms. However, the terms matter more than the signature. You need zero retention, no training on your data, no human review of prompts, defined subprocessors, and defined breach notification. Verify per-vendor and per-product, because BAA coverage often applies to a specific enterprise offering and not to the general API.

Minimum necessary. HIPAA’s minimum necessary standard means you send the least PHI required for the task. In practice, retrieve and send the relevant documents for this summary rather than the whole record. Strip identifiers the summariser does not need. Do not log prompt contents in your general observability stack, which is an extremely common and serious mistake.

De-identification, and what it is actually for. HIPAA recognises two methods. Safe Harbor removes 18 specified identifier categories. Expert Determination has a qualified statistician certify a very small re-identification risk (HHS guidance). The honest engineering point is that de-identification is for your development and evaluation environments, not for production. You cannot summarise a de-identified record for the treating clinician, because they need the real one. So de-identification’s job is to give your team a corpus to build and test against without every engineer touching live PHI. It is also not free. Automated de-identification of free-text notes is imperfect, so the de-identified corpus is treated as PHI-adjacent with restricted access anyway. Assume residual risk rather than assuming safety.

The Security Rule is changing. HHS proposed a substantial overhaul of the HIPAA Security Rule. The proposal moves controls that were “addressable” to mandatory, covering encryption, multifactor authentication, asset inventory and network mapping, segmentation, regular penetration testing, and backup and recovery. As of 2026 the final rule has been pushed to 2027. However, the direction is clear, and building to the proposed controls now is the cheap option (status).

Is it a medical device? This is the question that decides your release process. Under the 21st Century Cures Act, software meeting all four criteria in 21 U.S.C. § 360j(o) is excluded from the device definition. First, it does not acquire, process, or analyse a medical image or a signal from a diagnostic device. Second, it displays, analyses, or prints medical information. Third, it supports or provides recommendations to a healthcare professional. Fourth, and this is the hard one, it enables the professional to independently review the basis for the recommendation, so they are not relying primarily on it (FDA CDS guidance).

That fourth criterion is exactly why the citation architecture above is not a nicety. A summary where every claim links to the source sentence is a system whose basis a clinician can independently review. An opaque summary is not. The design decision and the regulatory position are the same decision.

FDA revisited this guidance in 2026. It notably softened its treatment of single-recommendation outputs and removed the blanket exclusion for time-critical decision support (analysis). The direction is somewhat more permissive. However, the fourth criterion is untouched, and processing images or device signals still puts you squarely in device territory regardless. Say clearly in the interview that you would get a regulatory determination in writing rather than reasoning your way to a conclusion. Say also that the product scope was chosen to sit comfortably on the non-device side, not right on the line.

Key decisions and tradeoffs

What breaks

Negation, and it is the classic. “No evidence of metastatic disease” summarised as “metastatic disease” is a catastrophic error produced by a single dropped token. Clinical text is dense with negation, hedging, and hypotheticals. “Rule out pulmonary embolism” appears in the charts of patients who do not have one. The mitigation is explicit negation-detection checks in the verifier, plus a strong preference for quoting findings verbatim.

Temporal confusion. A history section describes a myocardial infarction from 2011. The summary says the patient has had a heart attack, which is true. Or the summary presents it in the current context, which is dangerously misleading. This is why the timeline is a first-class object, and why every clinical claim in the output carries a date.

The historical-versus-active problem. Problem lists in real EHRs are full of resolved conditions nobody removed. Medication lists contain drugs the patient stopped taking two years ago. The source data is wrong, so summarising it faithfully produces a wrong summary. The honest handling is to present what the record says with its provenance and last-updated date, rather than to silently infer that something is resolved.

Copy-forward and note bloat. Clinicians copy previous notes forward, so the same paragraph appears in forty consecutive encounters. Naive summarisation weights it heavily by repetition, so near-duplicate detection is needed before anything reaches the model. There is a worse consequence. An error copied forward is now in forty documents, and it looks strongly corroborated.

Missing data misread as absent findings. This is the most subtle domain failure. A record with no documented allergies is not a record of a patient with no allergies. A summary that says “no known allergies” when the field is simply empty has asserted a clinical fact from an absence of data. The distinction between “documented as negative” and “not documented” must survive into the output, every time.

Automation bias. Once clinicians trust the summary, they stop reading the source. That is precisely what makes the tool valuable, and precisely what makes an error dangerous. This is well-documented in the clinical decision support literature, and it gets worse as the system gets better. The mitigation is interface design as much as model quality. Show visible confidence, require review of flagged critical findings, keep source access easy, and add periodic deliberate friction on high-stakes items.

Bias and equity. Clinical notes encode documentation practices that vary by patient demographics, language, and insurance status. A summariser trained or tuned on one population may compress differently for another. The ONC transparency requirements exist because this is a known, measured problem. So evaluation must be stratified by demographic group. “We did not measure it” is not an acceptable answer to a clinical governance committee.

Multilingual and interpreter-mediated records. Notes documenting an encounter conducted through an interpreter are already a lossy transcript. Summarising them compounds the loss.

Everything about EHR integration. Different institutions run different EHR versions with different customisations. FHIR support is often nominal rather than real. Rate limits can make a full record pull take hours. Read-only sandboxes do not resemble production. This is where projects die, and it is worth saying so.

How you’d evaluate it

Offline, on a de-identified corpus, against clinician-authored references. Do not use ROUGE. ROUGE measures n-gram overlap and is nearly uncorrelated with clinical correctness, because a summary that inverts a negation scores well.

Three metrics matter:

1. Factual precision. Of the claims in the summary, what fraction are supported by the source? The target is 100%, and anything else is a defect rather than a score. This is measurable automatically via the verifier, and confirmable by clinician review on a sample.
2. Critical-finding recall. Of the findings a clinician panel marked as must-surface, what fraction appeared? This is the safety metric, and it is the one with a hard floor. Measure it against a curated set of cases with adjudicated ground truth. That set is expensive to build, and it is the most valuable asset the project will produce.
3. Clinical acceptability. A blinded panel rates summaries against clinician-written references on accuracy, completeness, and usability. Report inter-rater agreement, because clinicians disagree with each other, and a system inside that disagreement band is doing well.

Include an adversarial set deliberately. It should contain heavy negation, contradictory documents, OCR-garbled inputs, records where the critical finding is buried in a two-year-old outside report, and patients with near-identical names in the same practice.

Clinical validation. This is distinct from model evaluation, and it is what the governance committee actually wants. Run a prospective study. Clinicians use the tool, an independent reviewer compares their decisions and their documentation against the full record, and you measure both time saved and whether anything material was missed. Sign-off comes from a clinical governance committee, typically a chief medical informatics officer, specialty leads, compliance, and legal. It does not come from an ML team’s dashboard.

Online. Run shadow mode first. Generate summaries nobody sees, and have clinicians review a sample retrospectively. Then run a limited pilot with a small volunteer cohort and intensive monitoring. Then do a staged rollout by specialty.

The business metric is documentation and chart-review time per encounter, ideally with clinician burnout instruments alongside it, because that is what the executive sponsor is actually buying. The metric that gates release is critical-finding recall. Two guardrail metrics matter: source-click-through rate, which is your automation-bias early warning, and per-claim negative feedback rate.

Regressions. Every reported error becomes a permanent test case. That corpus is the real safety net. Freeze a golden set and gate any model, prompt, or retrieval change on it, with critical-finding recall as a blocking criterion regardless of aggregate improvement. Treat model version upgrades as clinical changes requiring re-validation, not as dependency bumps. State that plainly, because it is the operational cost that surprises teams most.

For the general machinery, see the sibling agentic-ai-evaluation-guide. That covers judge design and calibration, eval harness structure, CI gating, and drift monitoring. What is specific here is the hard floor on critical-finding recall, and the fact that a governance committee grants release rather than a metric.

Follow-ups they will ask

“You said hallucination is unacceptable. But models hallucinate. So how do you ship?” By making ungrounded output structurally hard rather than merely discouraged. Generation is constrained to cited spans. A separate verifier checks entailment for every sentence, and exact-matches every number, date, dose, and medication name against the cited source. Anything unsupported is dropped or visibly marked unverified. Then the architecture assumes residual failure: display-only output, no chart writing, a physician who reviews and is accountable, and one-click access to the source for every claim. I am not claiming zero hallucination. I am claiming that a hallucination is catchable in one click by the person responsible, and that nothing propagates without their review. That is a defensible safety argument. “Our model is very good” is not.

“Is this a medical device? Walk me through it.” Under the Cures Act carve-out, software avoids the device definition if it meets four criteria. The one that does the work is that it must enable the clinician to independently review the basis for the recommendation. Our design is built for that criterion, because every claim cites its source sentence, and clicking it shows the original document. We also do not process medical images or device signals, and we do not make diagnostic or treatment recommendations, which keeps us clear of the other criteria. FDA updated this guidance in 2026 and became somewhat more permissive around single-output recommendations, but that fourth criterion is unchanged. In practice I would get a written regulatory determination rather than reason my way to a conclusion. I would also deliberately scope the product to sit well inside the line rather than on it, because the cost of being wrong is a 510(k) pathway you did not plan for and a shipped product you have to withdraw.

“A physician relies on your summary, it omits an allergy, and the patient is harmed. What happens?” Clinically and legally the physician is accountable for the decision. That is exactly why display-only, no-signature, human-in-the-loop is the right architecture rather than a limitation. However, the ethical answer is that “the doctor should have checked” is not a defence I would want to give. So the engineering question is why the allergy was omitted. Retrieval failure is the likeliest cause and the most dangerous class, meaning the allergy was in a document we did not select, and the output still looked complete. That is why allergies, active medications, and critical results are pulled deterministically from structured data on every summary, never left to retrieval, and rendered in a fixed section that is always present even when empty. Incident response follows the medical safety process: report, root-cause, add to the permanent regression set, notify affected sites, and decide whether to pull the feature.

“Where does de-identification actually fit? You can’t de-identify a summary for the treating doctor.” Correct, and that is the point people miss. De-identification serves development and evaluation, not production. It gives engineers and eval sets a corpus without live PHI exposure, under either Safe Harbor’s 18 identifier categories or Expert Determination. I would raise two caveats. First, automated de-identification of free-text notes is imperfect, because of rare names, small-town hospital names, and unusual dates, so the de-identified corpus is still handled as restricted. Second, de-identified data can be distributionally different from real data, because aggressive scrubbing changes the text. So I validate on real PHI in a controlled, audited environment before release, rather than trusting de-identified metrics alone.

“How do you handle the model provider? PHI is going to a third party.” BAA first, and the terms matter rather than the signature: zero retention, no training on our data, no human review of prompts, named subprocessors, breach notification, and defined data residency. Then minimisation. We send the selected relevant documents, not the whole record, and we strip identifiers the summariser does not need. Then the boring but critical control: PHI must not leak into our own observability stack. So prompts and completions are not logged to the general telemetry system, and traces store document IDs rather than content. Each hospital’s security review will ask about all of this, and having it documented is the difference between a two-week and a six-month sales cycle.

“Why not fine-tune on the hospital’s own notes?” It is tempting, and I would be cautious. A model fine-tuned on PHI is itself a PHI-containing artifact, with memorisation and extraction risk. The model weights then become subject to the same controls, deletion obligations, and breach exposure as the data. Fine-tuning also fragments your evaluation across per-hospital models and makes upgrades combinatorial. So I would exhaust retrieval, prompting, and a well-built few-shot set from a de-identified corpus first, because most of the perceived need for fine-tuning here is really a need for better document selection and better templates. If fine-tuning genuinely won, I would do it on a de-identified corpus with memorisation testing before release.

“How do you decide which documents to include? A twenty-year record is enormous.” It is a ranking problem with a safety floor. The floor is deterministic. Allergies, active medications, active problems, recent labs, and any result flagged critical are always included from structured data, regardless of ranking. Above that floor, retrieval is scored on recency, on document type weighted by specialty, on semantic relevance to the visit reason, and on explicit clinical linkage. The pathology report attached to this diagnosis is always relevant, however old it is. I evaluate this component separately from generation, because a retrieval miss and a generation error look identical in the output and have entirely different fixes. The UI also always says how many documents were considered, and it lets the clinician expand the scope, because a summary that hides its own boundaries is the failure mode I fear most.

“Clinicians will stop reading the source. Isn’t your safety argument circular?” It is a real tension, and I would not pretend otherwise. Automation bias is well documented, and it gets worse as the tool gets better. I have three responses. On design, critical findings require an explicit acknowledgement rather than a passive read, and the source is one tap away rather than three. On measurement, source click-through rate is a monitored metric, and a decline is a signal to investigate rather than a success. On scope, the tool summarises documented facts rather than making recommendations, so over-reliance means trusting an accurate transcription rather than trusting a judgement. That last one is the strongest, and it is another reason not to drift towards interpretation.

“What if the underlying record is wrong — a stale problem list, a medication the patient stopped?” Then the summary is wrong, and I cannot fix that, so I must not obscure it. Everything carries provenance and a date, such as “Active problems per problem list, last updated 14 months ago.” Where structured data and narrative text disagree, for example the note says the patient stopped metformin but the med list still has it, I surface the conflict explicitly rather than silently picking one. Clinicians are extremely good at resolving these when they can see them, and they have no chance when they cannot. Silent reconciliation is the tempting design, and it is the wrong one.

“How do you evaluate this without a large labelled dataset? Clinician time is expensive.” Accept that clinician annotation is the budget item, and spend it where it is irreplaceable. Automated verification handles factual precision at full volume and needs no clinicians. Clinician time goes to two things only. First, building the adjudicated critical-findings set, which is a few hundred cases built once and reused forever. Second, periodic blinded acceptability review on a sample. Everything else uses the in-production feedback loop, meaning one-tap wrong, missing or useful on every claim. That is cheap for the clinician and generates labelled data continuously. I would also use an LLM judge calibrated against the clinician labels for rapid iteration, while being explicit that it gates development and never gates release.

“How do you handle a model version upgrade?” As a clinical change, not a dependency bump. This is the operational cost people underestimate. You do a full re-run of the golden set with critical-finding recall as a blocking gate. You review every behavioural diff on the adversarial set. You run shadow mode against the current production model with disagreement analysis. Then you do a staged rollout by specialty with rollback ready. Governance is notified. The uncomfortable consequence is that provider model deprecation timelines can be shorter than a validation cycle. So I pin versions contractually where possible, and I keep a continuously-running validation pipeline so a re-validation takes days rather than months.

“How would you extend this to draft notes rather than summarise?” Carefully. I would treat it as a different product with a different risk profile, even though the pipeline looks similar. Drafting means generating text that enters the legal record under a physician’s signature. So the attestation step becomes the entire safety mechanism, and the interface must make skimming-and-signing genuinely harder than reviewing. I would start with the most structured, lowest-risk section, which is the interval history assembled from documented facts, all of it citable. I would stay away from assessment and plan, because that is where clinical judgement lives and where a plausible-sounding draft is most likely to be signed unread. Ambient documentation from the consultation audio is the adjacent product everybody wants, and it adds consent, recording law, and speech-recognition error on top of everything here.

“Which of the eighty-five percent of non-AI work would you do first?” The sentence-level document model and the citation data structure, because every safety property in the design depends on it, and retrofitting it is a rewrite. Then EHR integration and the audit log, because they gate every pilot conversation. Then the deterministic layer covering labs, meds, allergies, and care gaps, which delivers real clinician value with zero hallucination risk and can ship before the summariser exists. The summariser is the last thing I would build. By then the eval harness and the golden set are already in place to receive it.

Say it in one breath

A clinical report assistant is a citation system with a summariser attached. Documents are segmented to sentence-level IDs, and every generated claim carries the IDs it came from. A separate verifier checks entailment and exact-matches every number, date, dose, and medication before anything is displayed, and the physician can reach the source sentence in one click. That click is simultaneously the safety mechanism and the reason the product satisfies the Cures Act criterion that a clinician can independently review the basis for the output, which keeps it out of medical-device territory. Everything numeric or rule-shaped is computed deterministically and never asked of the model, including lab deltas, medication reconciliation, allergy conflicts, and care gaps. The system assists rather than diagnoses, displays rather than writes to the chart, and is never signed by anything but a human. Roughly fifteen percent of this project is the model. The rest is FHIR and HL7 integration, OCR, de-identified development corpora, BAAs and minimum-necessary discipline, audit logging, a prospective clinical validation study, and a governance committee that grants release. The metric with a hard floor is critical-finding recall, not time saved.