Testing every model output is unaffordable, and testing by intuition misses what actually causes harm. This is a method for spending a finite verification budget where clinical AI errors cost the most.
A clinical language model can be accurate enough to deploy and wrong often enough to cause harm at the same time. Both statements hold, and the distance between them is where the practical work lives. The error that should worry a clinical team most is the well-formed citation that points to nothing. When researchers had ChatGPT generate medical content with references, 47 percent of the citations were fabricated and only 7 percent were both real and accurate (Bhattacharyya et al., Cureus, 2023). Other analyses put reference fabrication anywhere from roughly 40 percent to above 90 percent depending on the model and the task (Chelli et al., JMIR, 2024; Walters and Wilder, Scientific Reports, 2023). These are not garbled outputs. They are plausible, correctly formatted citations attributed to real authors.
The failure is no longer confined to test conditions. A 2026 audit in The Lancet of 2.5 million biomedical papers found fabricated references now appearing in the published literature, with the rate rising more than twelvefold between 2023 and 2025, and the fakes were topically specific, correctly formatted, and attributed to real researchers (Topaz et al., The Lancet, 2026). As the audit's lead author put it, clinicians and guideline developers have no way of knowing that the evidence they rely on does not exist. A system can clear a deployment bar and still seed a record no one can trust.
The volume of output in any real clinical deployment is too large to verify exhaustively. Someone decides, explicitly or by default, which outputs get checked and which pass through. Most teams make that decision in one of two unsatisfying ways. They check everything at a shallow level, which spreads attention so thin that the checking catches little. Or they check by intuition, which tracks what feels alarming rather than what actually causes harm. Neither allocates a finite verification budget to the errors that matter most.
Failure-cost cartography is a method for making that allocation deliberate. It borrows its spine from FMEA, the failure-mode and effects analysis used in reliability and safety engineering for decades, and adapts it to the particular ways language models fail. The goal is a ranked, auditable account of which model errors warrant verification now, which are acceptable to absorb, and what the proportionate control is for each. The output is a decision, not an inventory of things that could go wrong.
The method starts by naming failure modes precisely. Not "the model hallucinates," which is a category, but a specific failure with a location and a consequence: in a medication-review workflow, the model omits a contraindication for a drug the patient is already taking, phrased with the same fluency as its correct output, so a clinician skimming the summary does not notice the gap. That is a mode. It has a place in the workflow, a concrete harm, and a detectability you can reason about.
Naming modes this way is most of the discipline. A worry is unbounded and cannot be scored. A mode is a thing you can measure, weight, and control.
Classical FMEA ranks each failure by the product of three numbers: how severe the consequence is, how often it occurs, and how hard it is to detect. The structure transfers. Two of the inputs need rework for language models, and the rework is where the clinical value sits.
Likelihood has to be measured, not estimated. A model's error rate on a specific mode is an empirical quantity. You can probe it with a set of representative inputs and record how often the failure actually fires. Estimating it from intuition reproduces the same bias the method is meant to remove. A likelihood number is valid only for the model and version you measured.
Detectability runs backward from mechanical systems. In a hydraulic line, a worse failure is often easier to see; pressure drops, something leaks. In language-model output, the more fluent and well-formatted a wrong answer is, the harder it is to catch. A fabricated drug interaction written in the same confident register as a real one passes casual review precisely because nothing about its surface signals that it is false. The dangerous errors in clinical AI are not the garbled ones. They are the plausible ones. A scoring model that does not penalize fluency will systematically underrate the failures most likely to reach a patient.
Two further inputs sharpen the ranking. Exposure asks how often the triggering situation actually arises in the deployment, because a catastrophic error on a query no one issues matters less than a moderate error on a high-traffic path. Reversibility asks whether the consequence can be undone once it occurs, which separates an error caught before an order is signed from one discovered after.
Combine these and you get an expected-loss estimate for each mode:
The arithmetic matters less than the discipline of filling each input with a measured number or a stated basis rather than a feeling. The register ranks by that estimate.
Consider a clinical summarization assistant that drafts medication reviews. One failure mode, scored:
| Element | Assessment |
|---|---|
| Mode | Omits a contraindication for a drug the patient already takes, in an otherwise correct summary |
| Location | The synthesis step, where the model condenses the chart into a review |
| Severity | High. The omission can lead to a harmful co-prescription; the harm is physical and can be serious |
| Likelihood | Measured, not assumed. Probe the model with a representative set of charts containing known contraindications and record the omission rate for this model and version |
| Detectability | Low. The summary reads as complete and fluent, so a clinician skimming it has little surface cue that something was dropped |
| Exposure | High. Medication review is a routine, high-volume task |
| Reversibility | Partial. Recoverable if caught before prescribing, serious if not |
Severe, frequent, hard to see, common, and only partly reversible places this mode near the top of the register. The ranking then drives a proportionate control rather than a vague call for caution: enforce a contraindication check outside the model, run deterministically against the patient's active medication list, and require that the model's summary either surface the check or flag that it could not complete it. With that control in place you record the residual risk, which is what remains after the control and is contingent on the check's coverage being current.
The same workflow carries other modes worth scoring: a confident statement of a superseded guideline, a unit or dose magnitude error, a finding attributed to the wrong condition because two presentations resemble each other. Each gets the same treatment. Some will rank high and earn a control. Others will rank low enough to accept knowingly, which is itself a decision worth making on purpose.
A register is valid for a model and a date. When the model is updated, the likelihood column is no longer trustworthy, because a new version can fix some modes and introduce others. A verification plan trusted past the version it was built on is worse than none, since it confers false assurance. The maintenance is not busywork. It is the recognition that the thing being measured moves. Pair the register with a way to log real failures as they surface in use, and it improves rather than ossifies: production incidents feed the likelihood estimates and occasionally reveal a mode no one anticipated.
Verifying clinical AI is often framed as a question of whether a model is accurate. The more useful question is where its errors cost the most and which of them a finite budget should catch. Cost is domain-specific. The same factual slip carries different consequences in a medication review, a billing summary, and a patient-education leaflet, which is why a credible failure-cost map has to be built inside a domain by someone who understands what its errors actually do, not assembled from a generic checklist.
What the method protects is the line between an answer that reads as true and one that is true. In clinical settings that line is expensive to maintain and easy to lose, because the failures that cross it are the ones dressed in the same fluency as everything else. Drawing the line deliberately, and spending verification where it does the most good, is ordinary engineering discipline applied to a system that happens to speak in complete sentences.
Verification note: the citation-fabrication figures (47 percent fabricated, 7 percent accurate) were confirmed against the primary paper, and the Lancet audit figures against the journal's reporting. Exact reference-fabrication percentages vary by study design, which is why the text gives a range. One claim from secondary aggregators (a 60-plus percent medical hallucination rate) could not be traced to a primary source and was removed.
VictorOS organizes evidence; it does not practice medicine. This essay describes a method for evaluating AI systems. It does not give medical advice, diagnose, or recommend treatment. Disease facts in VictorOS guides are based on articles retrieved from PubMed and cited with stable identifiers. These materials support your medical team; your clinicians remain the ones who diagnose and treat.