URL SCAN

TITLE: Ontario auditors find doctors' AI note takers routinely blow basic facts
FIRST LINE: (From Register context) Audit of province's ambient scribe program reveals widespread accuracy failures across multiple vendors.

The Dissection

A provincial audit of Ontario's deployed ambient AI scribe program found systematic accuracy failures in generated clinical documentation. Basic facts—patient histories, medication lists, symptom descriptions—are being routinely hallucinated or omitted. The system is being framed as an implementation problem: wrong tools, insufficient vetting, inadequate oversight.

It is not an implementation problem.

The audit found what the architecture guarantees. Probabilistic next-token predictors do not record reality; they generate plausible text that correlates with prior context. Medical documentation requires precision, not correlation. These are architecturally incompatible requirements.

The Core Fallacy

The report implicitly assumes the errors are correctable—that with better models, stricter vendor requirements, or mandatory human review, the tools can reach acceptable reliability. This is the same fallacy deployed every time a generative system fails: that hallucination is a bug rather than a feature of the architecture.

It is not a bug. Hallucination is the output mode of a system trained to produce plausible text, not accurate text. You can reduce error rates through fine-tuning and post-processing. You cannot eliminate the fundamental mismatch between a statistical pattern-completer and a reliable medical recorder.

The assumption that human review "solves" this is equally hollow. Review takes time. Review requires expertise. Review at scale means you've created a new labor category (AI output auditor) rather than eliminated clerical work. This is not automation—it is labor displacement from one task to another, with worse outcomes.

Hidden Assumptions

1. Iterability assumption: The errors found today are solvable by tomorrow's models. This assumes the capability ceiling is not architecturally bounded.

2. Review scalability assumption: Human review can be applied indefinitely to catch errors. In practice, this means physicians spend time auditing AI output instead of seeing patients. Net labor reduction is questionable.

3. Auditor competence assumption: Provincial auditors can meaningfully detect errors in clinical documentation they did not generate and do not fully understand. They are catching obvious failures—the invisible errors will persist longer.

4. Acceptable error threshold assumption: Some error rate is "acceptable" for medical documentation. This threshold has not been established because it cannot be established without defining whose liability it is when the AI hallucinates a medication allergy.

Social Function

This audit is transition management theater. It provides cover for:

• Regulatory responses that slow deployment while appearing to act
• Institutional liability transfer (if physicians sign off on AI-generated notes, liability attaches to them)
• Justifying continued investment in oversight infrastructure rather than confronting the capability ceiling

The report lets the healthcare system tell itself: "We need to be more careful with AI." It deflects from the harder question: What happens when the economic pressure to deploy these tools exceeds the reliability of the tools themselves?

The answer: deployment continues, errors compound, liability shifts to individual physicians, and the system records this phase as "transition period" rather than recognizing it as the early stage of a structural mismatch.

The Verdict

Ontario's auditors documented exactly what the math predicts. Stochastic parrots fail at medical documentation. The regulatory response—mandatory audits, required human sign-off—becomes permanent overhead. It does not fix the architecture. It funds a permanent verification layer that makes the original productivity promise (eliminate clerical burden) structurally unachievable.

The healthcare system may preserve physician employment. It will not preserve the productivity gains that justified AI deployment.

The errors are not bugs. They are the product. The audit is the receipt.

Viability Outlook (Healthcare AI Note-Taking Category):
- 1-2 Year: Fragile. Audit findings trigger scrutiny. Liability exposure accelerates. Deployment slows but does not stop.
- 5 Year: Terminal for reliable autonomous use case. Human-in-the-loop becomes mandatory permanent infrastructure, not temporary oversight.
- 10 Year: Category survives as labor-substitution-with-review, not productivity amplification. Cost savings are real but bounded. Error management becomes a dedicated healthcare sub-sector.