$\tau$-Elicitation: Benchmarking multi-turn entity extraction in voice agents

URL SCAN: $\tau$-Elicitation: Benchmarking multi-turn entity extraction in voice agents
FIRST LINE: # Computer Science > Artificial Intelligence

The Dissection

This paper isolates the brittle last mile of voice automation: collecting exact names, addresses, identifiers, dates, and times. Its findings are damaging to voice-agent hype: text passes every task, while voice configurations achieve only 0.14–0.41 robust exact success. Verification helps, but repairs only 24–37% of verified errors. The scaffold improves reliability by charging 21–28 additional seconds per call.

The paper is turning vague claims of “human-like” voice interaction into measurable deployment debt. Its real subject is not conversation. It is whether a machine can capture economically consequential data without forcing a human to clean up the wreckage.

The Core Fallacy

The central error, under Discontinuity Thesis mechanics, is treating interface reliability as the decisive question of automation. Exact spoken capture is a modality bottleneck, not a defense of human productive necessity. It is lag: an engineering problem to be reduced through scaffolding, multimodal confirmation, text fallback, escalation, or workflow redesign.

The benchmark does not test ownership, labor substitution, or whether institutions can preserve human-only economic domains. It therefore neither proves nor refutes P1–P3. It measures how much friction remains before one narrow class of cognitive service can be automated cheaply enough.

Hidden Assumptions

• Exact spoken capture is required instead of text, keypad input, multimodal confirmation, or backend verification.
• The 21–28 second latency cost is economically prohibitive rather than an acceptable reliability premium.
• Pass$^3$ and exact success adequately measure business usefulness.
• Two hundred tasks, three environments, four voice configurations, and the tested caller realisms generalize to real deployment distributions.
• Human escalation and downstream data reconciliation are absent or irrelevant.
• Repair effort is a permanent human bottleneck rather than a target for further automation.
• Better voice-agent performance preserves productive participation instead of merely reducing the number of people needed to operate the system.

Social Function

Classification: partial truth, transition management, and technical prestige signaling.

This is not copium. The measurements expose real failures and show a concrete scaffold that buys better performance. But the paper confines the social question to capture accuracy, verification policy, and call duration. That framing makes automation more deployable while leaving ownership, displacement, and distribution outside the room. It is a maintenance manual for the transition, not an argument that the old labor order survives.

The Verdict

Useful evidence of lag defenses, not a reprieve. Voice agents remain unreliable for exact high-consequence input, and the current fix is to make calls slower and more procedural. Under DT logic, that is hospice engineering around an interface. The machine does not need perfect hearing to sever the wage circuit; it only needs a cheaper total workflow than the humans it replaces. This benchmark shows the remaining friction—and how directly it can be attacked.