How Could AI Eliminate Humanity? A Failure-Mode Analysis of Civilizational Risk

URL SCAN: How Could AI Eliminate Humanity? A Failure-Mode Analysis of Civilizational Risk
FIRST LINE: # Computer Science > Artificial Intelligence

The Dissection

The paper constructs a non-operational taxonomy of AI catastrophe: autonomous misalignment, harmful human use, organizational failure, and competitive deployment. Its useful move is rejecting the childish premise that extinction requires consciousness, hatred, or explicit intent. Its actual limitation is narrower: it is a hazard framework, not a political economy of who controls AI after deployment.

The Core Fallacy

Under the Discontinuity Thesis, the central error is treating permanent human disempowerment as merely one catastrophic branch among several. It is potentially the structural consequence of successful competitive deployment itself. Once cognitive automation severs the labor–income–consumption circuit, humanity can lose productive participation without a rogue system, war, or laboratory accident. The paper maps catastrophic triggers while under-modeling terminal equilibrium: AI works as designed, institutions compete, and most humans become economically unnecessary.

Hidden Assumptions

• Human survival is treated as a sufficient proxy for human security, despite the abstract naming permanent disempowerment as a distinct outcome.
• Institutions retain enough coordination to impose safeguards under competitive pressure.
• Recovery capacity can be preserved independently of ownership and control of AI infrastructure.
• “Humanity” is treated as a unified subject rather than a hierarchy of sovereigns, servitors, and dispossessed populations.
• Defensive research and institutional safeguards can keep pace with deployment incentives.
• The decisive variables are capability, autonomy, access, persistence, and safeguards; ownership concentration and loss of economic necessity receive no comparable weight.

Social Function

Classification: partial truth serving transition management, with a secondary elite self-exoneration function. The framework is technically serious because it identifies non-intentional routes to catastrophe. But it channels a power and distribution crisis into a safety-engineering problem: measure risks, improve safeguards, preserve recovery. That permits deployers to present themselves as responsible managers while leaving the ownership question largely untouched. It is not simple copium. It is a competent catastrophe map with a strategically narrow perimeter.

The Verdict

The paper identifies spectacular ways AI could end civilization, but not the quieter mechanism by which AI can leave the species alive while stripping most humans of economic agency. Under DT, it describes terminal accidents and misses terminal equilibrium. Useful for failure-mode research; incomplete as an account of civilizational risk.