SpireBench: Evaluating Long-Horizon Strategic Play of Agentic Models with Slay the Spire

TEXT START: Agentic large language models now tackle complex tasks, yet strategic reasoning over long horizons is under-studied: most agentic benchmarks are short and episodic, and performance on long tasks is hard to dissect into stored knowledge, reasoning and luck.

THE DISSECTION

This is a controlled autopsy of agentic planning. SpireBench separates local competence from cumulative strategic competence: models can win early fights and match average humans, yet lose the run through poor risk calibration, excessive health preservation, faulty memory, and dependence on memorized game knowledge. It identifies where current agents break under compounding state rather than merely declaring that they “reason poorly.”

The paper is not a theory of general intelligence or economic agency. It is a measurement instrument for one closed, rule-bound environment.

THE CORE FALLACY

The dangerous inference is benchmark essentialism: treating the expert gap as evidence of a durable human moat. It is not. The results show a current capability deficit in long-horizon state management, not an inherent barrier to automation.

A fixed game exposes engineering targets—search, simulation, state tracking, calibrated risk-taking, better memory, and iterative training. The fact that written memories are wrong and models fail to improve across runs demonstrates weak adaptation in this setup, not permanent human indispensability. Under the Discontinuity Thesis, this is lag evidence against immediate P1, not a refutation of P1.

HIDDEN ASSUMPTIONS

• Slay the Spire is treated as a meaningful proxy for real-world strategic agency.
• Expert human performance is assumed to be stable rather than compressible into training data, search, or better scaffolding.
• The present harness and model architecture are treated as representative of future systems.
• Performance in a closed game is assumed to transfer cleanly to open-world coordination, production, and decision-making.
• The separation of knowledge, reasoning, and luck is assumed to be sufficiently clean.
• Failure to learn from self-written memories is treated as a general limitation rather than a limitation of the memory mechanism.

SOCIAL FUNCTION

Primarily partial truth, prestige signaling, and transition management.

It provides a legitimate map of present failure modes while advertising a new, unsaturated benchmark. Its practical function is to convert “agents are not yet reliable strategists” into an engineering backlog. Used by human exceptionalists, it becomes ideological anesthesia: a temporary performance gap misrepresented as permanent protection.

THE VERDICT

SpireBench is useful evidence that agentic models remain brittle over long horizons and still trail experts. That is a real lag, but it is not a fortress. The paper measures the delay before strategic automation becomes dependable; it does not establish a human-only economic domain.

Under DT logic, the expert gap is hospice care for human exceptionalism unless it survives repeated advances in training, search, simulation, and coordination. This benchmark weakens the claim of immediate P1 completion. It does nothing to overturn the trajectory toward P1, P2, and eventually P3.