Show HN: Parsimony – an open-source benchmark for how much code AI models add

TEXT START: Existing benchmarks measure solve rate and cost, but don't punish excess code, which makes a codebase harder to work with in the long term.

The Dissection

Parsimony is a measurement correction, not a theory of AI productivity. It exposes a real defect in coding benchmarks: models can receive credit for producing bloated patches, while the downstream maintenance burden is externalized onto humans. Net coding units make that waste visible and offer a useful secondary ranking signal.

But the benchmark remains trapped inside the old labor frame. It asks which model adds less code while still solving the task. It does not ask whether the human coding function survives at all once models can generate, test, revise, and deploy software at machine speed. It measures the efficiency of the replacement process while pretending the central question is code cleanliness.

The Core Fallacy

The core fallacy is treating code volume as the main scarce resource after AI has made code generation abundant.

Excess code matters. Bloated systems create maintenance, security, verification, and coordination costs. But parsimony does not establish that smaller patches produce superior systems, nor that fewer syntax units correlate reliably with lower lifecycle cost. A compact patch can conceal brittle abstractions, weak tests, opaque logic, or concentrated failure modes. The benchmark itself admits this: size is not readability or design.

More importantly, the benchmark's success metric is inherited from SWE-bench: whether the task was solved. That preserves the assumption that the meaningful unit of economic value is a model completing a human-defined coding task. Under the Discontinuity Thesis, that is a lagging indicator. The decisive event is not that models write fewer units. It is that durable machine superiority erodes the need for human programmers as a mass occupational class.

Hidden Assumptions

• That “solved” tasks represent economically valuable software production rather than narrow benchmark compliance.
• That published patches are comparable despite differences in model configuration, reasoning effort, agent behavior, and task selection.
• That net syntax units are a meaningful proxy for maintainability, architectural quality, or total cost.
• That tests and pass/fail labels are sufficiently reliable even though tests are not rerun.
• That the same agent does not materially shape the result; the page itself concedes that mini-SWE-agent affects code volume.
• That reducing model waste preserves a durable human role instead of simply making automated coding cheaper and more scalable.
• That code quality remains the central bottleneck. Under AI dominance, verification, deployment, infrastructure, security, proprietary context, and control of compute and distribution become more consequential bottlenecks.

Social Function

Parsimony is a partial truth functioning as transition management and ideological anesthetic.

It correctly identifies a nuisance created by automation: models dump unnecessary complexity into codebases. That is useful engineering intelligence. But the framing turns a civilizational labor displacement problem into an optimization problem for benchmark consumers. The discussion shifts from “who remains economically necessary?” to “which model is least wasteful?” The displaced programmer is converted into an invisible maintenance externality, then the externality is converted into a chart.

The benchmark is therefore valuable to Sovereigns and emerging Servitors. It helps owners of AI capital select cheaper, cleaner automated production systems. It does not defend the bargaining power, income stream, or productive necessity of ordinary developers. It sharpens the replacement machine.

The Verdict

Parsimony is a legitimate diagnostic instrument with a narrow target and a much larger implication. It reveals that AI coding systems are not merely replacing keystrokes; they are competing on how cheaply they can manufacture maintainable software.

Its benchmark contribution is real. Its economic framing is insufficient. The winning model is not the one that writes elegant patches for human programmers. It is the one that produces acceptable systems with minimal supervision, minimal verification cost, and minimal dependence on human labor. Parsimony measures one step in that cost collapse. It does not reverse it. Under the Discontinuity Thesis, it is not a defense of the coding profession. It is a cleaner ruler for measuring the blade.