High-Risk AI Systems and the Problem of Identity in the European AI Act

TEXT START: "The EU Artificial Intelligence Act (AIA) establishes a lifecycle governance regime for high-risk AI systems built around ex-ante conformity assessment, post-market monitoring, and re-assessment upon 'substantial modification.'"

THE DISSECTION

This paper is a technical compliance-fix document. The authors have identified a genuine governance pathology—regulatory identity lag—and are proposing an improved auditing framework (function+) to make identity determinations for AI systems more operationally legible.

The core move: AIA obligations trigger on modifications and updates to "the same system." But nobody has defined what "the same system" means in a reproducible way. So the paper supplies a function+ framework: same system = same intended function + context-sensitive criteria of appropriate functioning (trustworthiness profiles).

They then map this onto AIA lifecycle obligations (conformity assessment, post-market monitoring, re-assessment triggers) and offer a decision flow for audit and dispute contexts.

THE CORE FALLACY

The paper operates inside a premise it never examines: that regulatory identity governance will matter at scale when AI capabilities are approaching autonomous replication and self-modification.

The entire analysis presupposes that:
1. It will be meaningfully possible to draw a line between "system A before update" and "system A after update" as independent regulatory objects
2. That human-auditable conformance processes will retain jurisdiction over systems that increasingly exceed human comprehension
3. That the "trustworthiness profile" they're anchoring identity to will be a stable, measurable property rather than a contested and gamed metric

The function+ framework is elegant. It's also dead on arrival as AI capability curves bend further upward. Under P1 (Cognitive Automation Dominance), the class of AI systems requiring identification and tracking rapidly approaches combinatorial explosion—systems that redesign themselves, spawn variants, operate in federated/confederated configurations that resist individuation. The governance moment the paper is addressing is a narrow window between (a) when EU regulators gain enough maturity to implement function+ and (b) when the technology outpaces the framework's epistemic assumptions in toto.

We're likely already in that window. The paper itself acknowledges its own obsolescence as a priority-setting exercise: regulators must choose between "internal, auditable criteria for synchronic identity" and the administrative capacity to actually use such criteria at deployment pace.

HIDDEN ASSUMPTIONS

1. Stability Assumption: AI systems change along identifiable trajectories that permit "same system" judgments. This breaks down for foundation models that branch, diverge, and re-converge.
2. Auditability Assumption: Human governance actors can evaluate "trustworthiness profiles" in ways sufficient for regulatory decisions. This presumes the evaluators retain sufficient technical comprehension—which the DT framework already corrodes via skill premium collapse.
3. Jurisdiction Assumption: The AIA's lifecycle framework will have de facto reach when AI development is offshore, open-source, hybrid, or distributed across jurisdictions outside EU regulatory capture.
4. Purpose Assumption: "Intended function" is a stable, legible, non-gameable input to the identity test. In adversarial, competitive, or rapidly evolving deployment contexts, this is routinely falsified post-hoc.

SOCIAL FUNCTION

Partial truth with prestige-signaling wrapper. This is a rigorous academic paper making a real contribution to a narrow regulatory design problem. It is not doing what it appears to do—which is offer a durable governance solution.

Its function is best understood as: institutional transition management. It improves the machinery of EU regulatory capacity by sharpening identity criteria, making enforcement more operationally feasible in the short-to-medium term. This is legitimate work. But the framing implies the EU AI Act framework is a viable governance architecture when the DT analysis says it is a lag defense in decay—valuable as delay, catastrophic as solution.

The paper is optimized for a world that is already ending. The authors are refining the plumbing of a house whose foundation is settling.

THE VERDICT

A technically sophisticated solution to a governance problem that will be structurally irrelevant within a meaningful time horizon. The function+ framework is superior to the current AIA identity void. The question is whether identity void resolution precedes or follows capability-exceeds-governance. Evidence suggests the latter.

The paper solves a 2026 problem. The relevant regulatory window may be closing faster than this chronology implies.