Evolving Together, Failing Apart: The Skill-Policy Co-evolution Trap in Self-Improving Agents

TEXT START: Skill-policy co-evolution, in which an agent's policy model and its skill bank improve each other during training, is a promising paradigm for self-improving agents.

The Dissection

This is an evaluation autopsy of benchmark overfitting at the policy–skill-bank boundary. Cooperative acceptance makes competence conditional on the exact bank with which the policy co-evolved. Deployment drift then exposes that peak performance as unstable.

CODA changes the target from maximum performance on a frozen delivered pair to resilience under bank evolution, semantic edits, and corruption. The reported 5–13 point worst-case decline, erasing 52–93% of the delivered bank’s advantage, means the headline capability was partly a property of the training pairing—not a durable property of the agent.

The Core Fallacy

The empirical diagnosis is not the fallacy. The fallacy is the level of analysis.

The paper treats skill-bank drift as the central threat. Under the Discontinuity Thesis, it is merely a maintenance and control defect inside a larger displacement process. The decisive question is not whether one policy survives a modified bank. It is whether AI systems can perform economically necessary cognitive work more cheaply and effectively than humans.

A competitor can retrain, fork, replace, verify, or discard a damaged skill bank. If those repairs cost less than employing humans, the displacement mechanism remains intact. CODA may improve agents by forcing robustness; it does not restore human productive necessity. The paper audits how the machine jams. It does not establish that the factory still needs workers.

Hidden Assumptions

• Benchmark gains are treated as meaningful proxies for deployment and economic value.
• Worst-case drift scenarios are assumed to represent operational reality rather than merely selected stress tests.
• The policy and skill bank remain separable, stable components rather than being replaced by more integrated architectures.
• Retraining, redundancy, verification, and automated bank repair are treated as constraints rather than competitive responses.
• A failed agent cannot simply discard its damaged components and be replaced by a better copy.
• Human auditors or maintainers remain indispensable instead of becoming temporary Servitor labor.
• Greater robustness will not accelerate adoption, scale, and labor substitution.

Social Function

Primary classification: partial truth. Secondary classifications: transition management and prestige signaling.

The paper identifies a genuine engineering weakness and supplies a useful audit protocol. It also gives the field a respectable way to discuss reliability without confronting ownership, concentration, labor displacement, or the collapse of mass productive participation. CODA can become ideological anesthetic if passing the audit is treated as evidence that the broader transition is safe or socially benign.

The Verdict

CODA is a legitimate reliability correction, not a systemic rebuttal. It demonstrates that co-evolved competence is brittle; it does not disprove AI dominance, coordination failure, or the collapse of human productive participation.

Its likely systemic effect is to increase the value of those controlling retraining, verification, skill supply chains, energy, logistics, and maintenance. Human maintainers may occupy a temporary Servitor niche, but that niche will be compressed as repair and verification become automated. The agent’s tendon is weak. The body replacing the workforce remains operational.