Agent Policy-Value Audit: Separating Transition Composition from Event Selection in Financial LLM Agents

URL SCAN: Agent Policy-Value Audit: Separating Transition Composition from Event Selection in Financial LLM Agents
FIRST LINE: # Computer Science > Artificial Intelligence

The Dissection

This paper is an evaluation correction, not a demonstration of financial intelligence. It separates two effects that a raw return metric launders together:

• composition value: gains from changing positions in action categories that happened to pay;
• selection value: gains from choosing the right individual events within those categories.

Its result is damning. The agent’s reported +15.2 bps/event decomposes into a +25.8 bps composition benchmark and −10.6 bps selection value. The apparent success came from exposure to a favorable event pool, not from identifying superior events. Matched random assignments perform comparably.

The Core Fallacy

The paper’s conceptual limit is confusing measurement purity with economic viability. Separating selection from composition can determine whether a claimed edge is real, but it does not establish that the edge is durable, scalable, or strategically important.

Even a positive selection value would be a vulnerable transient advantage under the Discontinuity Thesis. Once cognitive systems compete on the same information and decision space, selection edges are copied, compressed, and priced away. A clean alpha measurement is not a moat. It is a more accurate countdown clock.

Hidden Assumptions

• Random reassignment across eligible events is a valid counterfactual and the event pool is sufficiently exchangeable.
• Ordered action-change counts capture composition completely; the timing and size of those changes may themselves contain strategy.
• Gross bps/event adequately represents value without transaction costs, slippage, latency, capacity, drawdown, or risk adjustment.
• The 723 events across 44 consumer-facing U.S. firms generalize to financial LLM agents more broadly.
• A failure to detect selection skill reflects no meaningful skill rather than limited sample size, poor task design, or an unsuitable action taxonomy.
• Event-selection value is the economically decisive layer, when execution, coordination, risk control, and capital allocation may matter more.

Social Function

Primary classification: partial truth.

Secondary function: transition management. The paper strips a fake intelligence claim from its statistical costume and gives institutions a cleaner vocabulary for auditing automated finance. It does not challenge the larger migration of cognitive work into machines; it merely improves the accounting used during that migration.

The Verdict

This is a useful autopsy of a false trading claim. The agent did not demonstrate event-selection skill; it harvested favorable exposure while selecting events badly. The paper therefore strengthens, rather than weakens, the Discontinuity Thesis: positive deployment returns can survive after productive cognition has already failed.

The agent is not a financial sovereign on these results. It is, at best, a provisional servitor whose value depends on deployment infrastructure and market conditions. The headline return is not intelligence. It is composition beta wearing an AI badge.