The AGI Race and Existential Risk -- by Ethan Bueno de Mesquita, Wioletta Dziuda, Mattias Polborn

TEXT ANALYSIS

TEXT START: "Concerns about the race to artificial general intelligence often assume that competition and resources increase risk by accelerating development."

1. The Dissection

A game-theoretic model analyzing how competing firms allocate resources between speed (probability of reaching AGI first) and safety (reducing existential risk). The paper argues:

• Fragmentation (more competing firms on fixed resources) pushes equilibrium toward speed over safety
• A "critical market size" exists where AGI achievement shifts from positive to negative expected value
• Policy levers—consolidation, resource regulation, commitment devices, cautious public entry—can improve welfare by reshaping incentives

The structure is elegant. The assumptions are clean. The policy recommendations are politically unrealizable. The entire exercise is a category error masquerading as rigorous analysis.

2. The Core Fallacy

The paper treats AGI risk as a market design problem when it is the terminal mechanism of the system it assumes as given.

The central error: The model assumes existing economic and political institutions have the capacity and incentive to implement the policy interventions it recommends. The DT framework says these institutions are themselves products of the mass-employment mass-consumption circuit that AGI severs. You cannot use institutions dying of the same cause to fix the cause.

Additional critical failures:

• "Safety" is modeled as an endogenous optimization variable. The DT framework suggests safety investment is systematically crowded out by competitive pressure—not because firms are irrational, but because rational survival under competition demands speed. The model treats safety as a choice available to firms when competitive dynamics make it effectively unavailable.

• The doom risk framing is backwards. The paper models existential catastrophe as a technical safety failure (like a nuclear meltdown) that better incentives could prevent. DT posits the existential risk is structural—the elimination of productive participation, not an engineering defect. Different mechanism. Different solution class. No solution in this paper's toolkit.

• "Critical market size" is a red herring. The positive-to-negative expected value transition is interesting model behavior. But the DT question is whether the system is viable before AGI arrives, not whether achieving AGI is "worth it" in expectation. The mass employment -> wage -> consumption circuit is already degrading. AGI's "negative expected value" isn't a signal to stop; it's a signal the underlying system is already dying.

3. Hidden Assumptions

1. Institutional permanence: Assumes the policy apparatus (consolidation enforcement, resource regulation, international coordination) functions with sufficient legitimacy and capacity to reshape AGI development incentives. This is the assumption of a stable post-WWII institutional order—which DT identifies as the structure under dissolution.

2. Agent rationality in a discontinuous transition: Assumes rational actors optimize across a stable preference function. In a discontinuous transition, the rational strategy may be flood the zone—build as much optionality as possible, regardless of aggregate risk. Individual rationality driving to collective catastrophe is the mechanism, not a model deviation.

3. Safety as a real choice variable: The entire speed-safety tradeoff framework assumes firms can credibly commit to safety investment and that safety investment actually reduces doom risk. Both assumptions are heroic. Competitive pressure and the absence of verifiable safety metrics make "safety investment" largely performative.

4. Doom risk as a bounded probability: The model treats existential risk as a calculable probability amenable to cost-benefit analysis. DT suggests the relevant framing isn't "will AGI kill us" but "will the transition mechanism preserve viable human systems"—which depends on institutional, political, and social factors that models like this cannot capture.

4. Social Function

Classification: Institutional Self-Exculpation and Prestige Signaling

This paper's primary function is to demonstrate that serious economists are thinking hard about AGI risk in sophisticated, policy-relevant ways—without confronting the structural implications of the DT thesis.

Specifically:
- It gives academic cover to the position that AGI risk is manageable within existing frameworks
- It positions economists (and by extension, the economic order) as the relevant authority on existential risk
- It offers policy recommendations that require precisely the kind of supranational coordination and institutional legitimacy that does not exist and is not emerging
- It performs technical rigor to deflect from the fundamental question: not how do we make AGI safer but what is the mechanism of systemic death this process is producing

This is the academic apparatus of lag defense—producing sophisticated-sounding analysis that preserves the legitimacy of institutions that cannot actually implement its recommendations.

5. The Verdict

The paper is a technically sophisticated analysis of the wrong problem.

It treats AGI risk as a market failure requiring incentive correction within the existing economic framework. DT says AGI is not a market failure—it is the mechanism by which the mass-employment mass-consumption foundation of that framework is dissolved. The doom risk the paper models as a technical externality is not a bug in an otherwise functional system; it is the system's terminal output.

The policy recommendations are structurally equivalent to proposing hospital reform as a response to the extinction of the patient population. Institutions capable of implementing consolidation, resource regulation, and international coordination on AI development do not currently exist and are not emerging—with one exception: the nation-state capacity for prohibition and destruction, which DT identifies as the primary instrument of transition, not the solution to it.

The model is internally consistent. Its assumptions are coherent. The world it describes does not exist.

Bottom line: This paper is what the academic apparatus produces when it refuses to update on the structural thesis. It is rigorous analysis of a system in denial.