Cited but Not Consulted: A Counterfactual Audit of Legal Chain-of-Thought Faithfulness

TEXT START: Large language models increasingly justify legal decisions by naming the statute or precedent behind a verdict, treated as evidence that the decision follows from it.

THE DISSECTION

This paper dismantles a specific audit fiction: that citing the correct statute or precedent proves the model used it. The counterfactual test exposes citation as often decorative output rather than causal dependence. The red-team result is worse: models obey hostile instructions embedded in case facts far more reliably than they respond to changes in the supposedly governing authority.

The paper therefore identifies two separate failures: explanation is not evidence of reasoning, and a legally formatted verdict remains behaviorally manipulable. Its strongest contribution is treating faithfulness as an intervention problem instead of accepting textual plausibility as proof.

THE CORE FALLACY

The paper’s weakness is not that its findings are wrong. It is that it remains trapped inside the audit frame. It treats unreliable legal reasoning as an engineering defect to be measured and repaired, without confronting the Discontinuity Thesis’s economic consequence: the legal task can be automated even when the explanation is unfaithful.

A model does not need to possess human-style legal reasoning to destroy the market for large quantities of legal drafting, triage, review, and memo production. It needs only to be cheap, fast, sufficiently accurate, and wrapped in a verification layer. The paper attacks citation-based trust; it does not restore human productive indispensability. In DT terms, it diagnoses a dangerous machine while leaving the automation trajectory intact.

HIDDEN ASSUMPTIONS

• Counterfactual verdict sensitivity is treated as a sufficient proxy for legal faithfulness. It measures dependence on the named authority, but not every form of valid legal reasoning.
• The selected open-weight models, benchmarks, and LoRA reproduction are implicitly treated as informative about deployed frontier systems, though the supplied abstract does not establish that generalization.
• Better prompting, retrieval, model training, or audit design are presumed to be viable institutional remedies rather than temporary containment layers.
• Human legal review is assumed to remain economically valuable at scale. Under P1, verification itself becomes a target for automation.
• Legal legitimacy is treated primarily as an explanation and compliance problem. The deeper problem is that institutions may continue purchasing legal outputs after the labor producing them has been commoditized.
• The paper separates verdict quality, explanation faithfulness, and adversarial robustness, but does not show that solving any one of them preserves human control over the legal production chain.
• A correct citation is implicitly granted too much importance. The real strategic question is not whether the model can name authority, but who owns the data, validation infrastructure, liability gate, and deployment channel.

SOCIAL FUNCTION

Classification: partial truth serving transition management and prestige signaling.

This is not simple copium. It provides a real warning that generated explanations cannot safely function as compliance artefacts. But it converts a structural displacement problem into a legible research agenda: build better faithfulness tests, better safeguards, and better audit layers. That agenda will create transition niches for evaluators, red-teamers, validators, and liability managers—the Servitors and Hyenas clustered around the machine.

It does not preserve the old legal workforce. It helps institutions decide which automated systems they can risk deploying, thereby accelerating the replacement of routine legal cognition while making the replacement administratively acceptable.

THE VERDICT

A sharp autopsy of legal explanation, not a defense of human legal labor. The paper proves that citation is theater and that legal models remain adversarially pliable. Its implicit remedy—more auditing and verification—will become another automated control layer. The citation-dependent lawyer is already obsolete as a moat; the remaining leverage belongs to whoever controls legal data, validation, liability, and deployment.