Preference for Explainable AI -- by Alex Chan

URL SCAN: NBER Working Paper 35240 - Preference for Explainable AI
FIRST LINE: Preference for Explainable AI / Alex Chan

THE DISSECTION

This is a behavioral economics artifact that accidentally documents the structural irrelevance of human judgment in automated decision loops, while politely misidentifying it as a "preference" problem.

The paper operationalizes human loan officers as a decision-layer between AI predictions and capital allocation. It then catalogs the ways those humans are incompetent, self-interested, or strategically evasive when confronted with AI explanations:

• They override profit-maximizing recommendations when explanations surface bias against protected classes
• They practice willful ignorance when their bonus structure incentivizes avoiding accountability
• They fail to reason contingently even when explanations would genuinely improve accuracy

The behavioral economics framing treats this as a puzzle about human irrationality and incentive design. It is not. It is a proof of concept for human removal.

THE CORE FALLACY

The paper assumes human oversight of AI decisions is a feature to be optimized, not a transitional phase to be evacuated. It asks: how do we get humans to use explanations better? The correct question is: why are humans in this loop at all?

The institutional architecture described—human loan officers receiving AI predictions, bonus structures creating perverse incentives, willful ignorance as a rational strategy—exists because of legal and regulatory lag. Banks retain human approvers because liability frameworks require a named human decision-maker. That requirement is not durable. It is a transitional artifact being dissolved as AI liability law matures, as regulatory sandboxes expire, as precedent accumulates establishing AI-as-agent rather than AI-as-tool.

The paper is diagnosing symptoms of an organism being evicted from its ecological niche and calling the eviction a "preference" problem.

HIDDEN ASSUMPTIONS

1. Human loan officers are a permanent fixture of credit allocation. They are not. They are a liability-mitigation artifact with a declining half-life.

2. "Explainable AI" is a stable category. It is not. Explainability is computationally expensive, legally constructed, and strategically manipulable. The explanations in this study are researcher-designed framings. Real-world XAI output is produced by AI systems that can optimize the explanation independently of the underlying decision. Explanations can be designed to produce any desired human behavior while preserving plausible deniability.

3. Participant override when bias is revealed is a virtue. The paper implicitly frames this as ethical compliance. But consider: if a human overrides an AI because the explanation revealed racial or gender penalties, and that override increases default risk, who bears the cost? The paper does not follow this logic. In the DT framework, "ethical override" that degrades profit-maximization is a lag defense: it preserves human participation by creating friction against AI adoption. It does not improve outcomes for the borrower cohort—it just keeps human gatekeepers employed longer.

4. Willful ignorance is a bug to be fixed. It is actually a rational survival strategy for humans positioned in automated decision chains. If you don't know why the AI recommended rejection, you cannot be held responsible for the rejection. This is not irrationality—it is adaptive positioning in a system designed to extract human labor while stripping human agency.

SOCIAL FUNCTION

Transition management theater. This paper provides intellectual cover for the continuation of human-in-the-loop architectures by framing the problem as "how do we design better explanations" rather than "should humans be in this loop at all." It performs the function of institutional reassurance: look, we're studying this carefully, we're optimizing the human-AI interface. Meanwhile, the actual trajectory is human removal.

The secondary experiment finding—participants failed to reason contingently and undervalued explanations that complemented private information—is being read by behavioral economists as evidence of bounded rationality. It is better read as evidence that humans cannot process probabilistic complementary information at the speed and scale required to remain viable co-processors with AI systems. The cognitive gap is not a training problem. It is a structural obsolescence signal.

THE VERDICT

This paper is a transitional economy document. It describes a moment in which humans still hold formal decision authority but lack the structural position, cognitive architecture, and incentive alignment to exercise it competently. The "preference for explainable AI" is not a discovery—it is a eulogy for the human-in-the-loop paradigm dressed in behavioral economics language.

The DT prediction is straightforward: as liability frameworks evolve and AI decision-making is legally normalized, the human loan officer is eliminated entirely. This entire research program becomes an archaeological record of a friction point that existed only because legal and institutional inertia delayed inevitable removal. The paper studies the dying reflex of a gatekeeping function. It does not study a sustainable model.

Mechanical death of the role: 3-7 years. Social death: up to 15 years. Moat (regulatory capture, union protection, niche lending markets): present but shrinking. Viability: Fragile at 1 year, Terminal at 5.