Open Problems in AI Risk Modeling: Insights from a Workshop on the Technical Foundations of AI Risk Modeling

URL SCAN: Open Problems in AI Risk Modeling: Insights from a Workshop on the Technical Foundations of AI Risk Modeling
FIRST LINE: # Computer Science > Computers and Society

The Dissection

The paper builds an administrative control layer for advanced-AI risk. It translates a civilizational power shift into familiar instruments—probabilistic models, Bayesian networks, thresholds, disclosure rules, expert workshops, and regulatory institutions. Its real project is to make systemic danger legible enough for governance.

The Core Fallacy

It treats better measurement as if it creates control. Under the Discontinuity Thesis, risk modeling does not solve P1, P2, or P3: AI can outcompete cognitive labor, human institutions cannot preserve stable human-only economic domains, and productive participation collapses. A Bayesian network cannot restore the wage–consumption circuit. A regulatory threshold cannot reliably stop actors whose competitive survival depends on crossing it. The paper confuses epistemic improvement with political and economic capacity.

Hidden Assumptions

• AI risks will be observable, quantifiable, and stable enough for calibrated models.
• Historical evidence can validate models for unprecedented regime changes.
• Regulators will remain independent rather than becoming instruments of concentrated AI capital.
• Disclosure will produce accountability instead of strategic opacity or information overload.
• Institutions can coordinate internationally despite competitive incentives to defect.
• Identifying a threshold will cause authorities to act before deployment becomes irreversible.
• Quantitative uncertainty is the main barrier, rather than ownership, coercive power, and incentive structure.

Social Function

Partial truth serving transition management and prestige signaling. The paper correctly recognizes that current risk governance lacks robust methods, independent evaluation, and durable institutional memory. But it converts an ownership-and-coordination crisis into an expert-governance program. That may improve detection, triage, and damage containment. It does not reverse the underlying economic discontinuity.

The Verdict

Technically useful as an instrumentation agenda; strategically inadequate as a theory of systemic AI risk. It may produce better maps of the wreckage and a professional class authorized to narrate it. If P1–P3 hold, it cannot preserve postwar capitalism or restore mass productive participation. Its likely function is not preventing obsolescence, but making the collapse administratively measurable.