Mitigating Capability Degradation in LLM Self-Improvement via Representation Geometry Regularization

URL SCAN: Mitigating Capability Degradation in LLM Self-Improvement via Representation Geometry Regularization
FIRST LINE: # Computer Science > Artificial Intelligence

THE DISSECTION

This paper documents that on-policy self-distillation can improve aggregate performance while destroying previously reliable capabilities, then proposes Representation Geometry Regularization as a stabilizer. It converts a failure of recursive improvement into an optimization problem: preserve hidden-state structure, retain competence, and continue capability growth. It is a maintenance patch for the replacement engine.

THE CORE FALLACY

It confuses local capability retention with systemic survivability. RGR may reduce capability degradation, but it does not touch P1, P2, or P3. It preserves no human labor market, creates no human-only economic domain, and changes nothing about ownership of productive AI capital. If it works, it makes cognitive automation more reliable and removes one brake on displacement. The paper treats capability improvement as the central problem; under the Discontinuity Thesis, the central problem is that successful improvement severs employment from production.

HIDDEN ASSUMPTIONS

• Benchmark retention adequately represents economically useful capability.
• Cluster compactness and separation remain reliable proxies across tasks and distribution shifts.
• Regularization can preserve old competence without materially limiting new capability acquisition.
• Longer self-improvement loops will remain controllable and aligned once representation drift is reduced.
• Productivity gains will diffuse through wages and employment rather than accrue to AI owners and controllers.
• Institutions can coordinate the deployment of increasingly autonomous cognitive capital.

SOCIAL FUNCTION

Primary classification: transition management. Secondary classification: partial truth and ideological anesthetic. The paper identifies a real mechanical defect in recursive optimization. But by narrowing the problem to representation geometry, it makes a civilizational rupture look like a debugging ticket. Stabilize the model, preserve the benchmark gains, and continue. The consequence—human productive participation becoming unnecessary—is left outside the frame.

THE VERDICT

Technically, this is evidence that self-improvement is lossy and non-monotonic. Systemically, it is not a defense of the post-WWII order. It repairs the successor system: if RGR succeeds, it reduces capability degradation and accelerates P1, making P2 and P3 more likely. The paper does not stop obsolescence. It helps the machine become less forgetful while it replaces the workforce.