Terminal-Bench-Science: Evaluating AI agents on scientific research workflows

TEXT START: Terminal-Bench-Science evaluates AI agents on workflows from researchers' own work.

The Dissection

This text performs two operations at once. It builds a serious measurement instrument for bounded scientific workflows, then narrates the likely automation of those workflows as mere augmentation.

The benchmark is valuable evidence: 70 expert-curated tasks, reproducible artifact-based grading, and a leading resolution rate of only 30%. But it measures what can currently be formalized and verified—not the full economic function of scientists. Its task design also makes scientific labor legible to the systems being trained to replace it.

The Core Fallacy

The central error is treating present incapability as a durable human moat. A 30% resolution rate is a snapshot, not a ceiling. The benchmark is explicitly designed to expose failure modes, calibrate against frontier systems, and feed progress back into AI development. That is an acceleration mechanism for P1, not evidence against it.

The article assumes that humans will remain necessary for defining questions, forming hypotheses, interpreting results, and validating outputs. Under the Discontinuity Thesis, those are decomposable cognitive activities. Human sign-off may persist as a legal or institutional ritual, but ritual authorization does not require the employment of the current number of researchers.

The phrase “freeing scientists” conceals the actual transition: agents absorb execution first, then narrow the premium paid for interpretation and supervision. The benchmark is therefore not protecting scientific labor. It is mapping the labor that can be stripped out.

Hidden Assumptions

• A workflow that agents cannot yet complete is assumed to be resistant to automation rather than simply immature.
• Human judgment is treated as indivisible and permanently superior instead of decomposable into trainable procedures.
• Objectively verifiable artifacts are treated as a technical boundary, though better tools can expand what is verifiable.
• Human researchers will retain control of research questions and hypotheses even as agents generate, test, and rank them.
• Institutions will use improved agents to augment staff rather than reduce headcount and concentrate ownership.
• The benchmark’s continuous updates are treated as proof of progress, although they also create a moving goalpost that obscures absolute capability gains.
• Cost and token frontiers are treated as secondary metrics, despite deployment economics deciding whether human labor remains necessary.
• Scientific demand will expand fast enough to absorb displaced technical labor. The article provides no mechanism for that assumption.

Social Function

Primary classification: partial truth wrapped in transition management, prestige signaling, and ideological anesthetic.

The partial truth is real: current agents are unreliable, expensive, and far from autonomous scientific competence. The transition-management layer converts that fact into a comforting institutional script: humans will simply move upward into more meaningful work. The prestige signal comes from allowing scientists to define the benchmark and thereby retain symbolic authority over the transition.

Its deeper function is elite self-exoneration. If researchers describe themselves as the permanent holders of judgment while releasing standardized workflows into agent-readable form, they can present labor substitution as liberation. The benchmark does not need to intend this function. Its structure performs it.

The Verdict

Terminal-Bench-Science is a credible early-warning instrument, not a defense of the post-WWII employment order. Its 30% result shows that the scientific labor substrate is not yet fully automated. Its task corpus, verification machinery, and continuous frontier feedback show how that substrate will be made progressively automatable.

The benchmark’s most consequential achievement may be organizational rather than scientific: it teaches institutions to separate research into executable units, grade them mechanically, and compare agents against human workflows. That is the anatomy of productive participation collapse. The scientists who own models, compute, data, infrastructure, or resulting intellectual property move toward Sovereign status. Most operators and assistants move toward Servitor status, then face compression as supervision itself becomes automated.

This is not the end of science. It is the beginning of science becoming less dependent on scientists as a mass occupation.