Distributional Validity and Calibration of a Korean Synthetic Persona Panel for Digital and AI Service Use: A Secondary-Data Validation Against the Korea Media Panel Survey

TEXT START: Synthetic personas based on large language models (LLMs) are increasingly proposed as substitutes for human survey respondents, yet systematic validation outside English-speaking contexts remains scarce.

The Dissection

This paper is a quantitative demolition of synthetic-persona substitution claims. Errors of 15–19 percentage points, subgroup gaps up to 52.4 points, model-specific stereotypes, acquiescence bias, temporal misalignment, and framing sensitivity show that demographic labels do not create representative human populations.

Calibration halves same-period cell error, but direct real data remains substantially more accurate and the correction fails across time. Calibration is therefore local statistical repair, not recovered validity. The product is a model prior wearing demographic clothing.

The Core Fallacy

The fallacy being exposed is conditioning equals representation. Sex, age, or persona narrative cannot manufacture lived experience, current context, independent heterogeneity, or genuine human preferences. LLMs generate responses from learned priors and prompt framing.

The paper largely recognizes this. Its residual mistake is treating usefulness under extreme data scarcity as evidence of legitimacy. A synthetic panel may provide a rough prior when only about 100 real responses exist; that is utility in a data desert, not epistemic substitution.

This also has no bearing on the central Discontinuity Thesis mechanism. Replacing some survey collection with generated answers does not restore productive human participation or the wage–consumption circuit. Measurement automation is not economic inclusion.

Hidden Assumptions

• Weighted KISDI estimates are treated as a sufficiently reliable ground truth, despite being survey measurements rather than reality itself.
• MAE, correlations, and demographic-cell performance are treated as adequate proxies for decision usefulness, causal validity, and tail behavior.
• Results from eight service-use indicators and eight constructs can generalize to broader survey domains.
• Data scarcity is treated as a static condition rather than a changing environment in which AI adoption rapidly alters the underlying distribution.
• Users will distinguish a diagnostic prior from evidence rather than laundering synthetic outputs into institutional fact.
• Calibration gains will matter operationally even though the study shows they do not transfer reliably across time.
• The cost and availability of collecting a small amount of real data are treated mainly as technical variables, not as governance and accountability constraints.

Social Function

Partial truth with transition management. The study punctures the claim that synthetic respondents are drop-in replacements, while preserving a narrower institutional use case: exploratory diagnostics, stress testing, and prior formation when real data is scarce.

That niche is real, but it is verification arbitrage, not salvation. Human observations become the scarce control layer used to calibrate and audit automated systems. The synthetic panel is hospice care for data-poor institutions, not a new citizenry.

The Verdict

This paper establishes that current synthetic personas are not survey substitutes. They compress heterogeneity, import stale priors, overfit when corrected, and fail when the world moves.

Its result is terminal for the fantasy of cheap, representative artificial respondents. It does not weaken AI dominance over cognitive work; it simply identifies a boundary where automation still needs scarce human ground truth. Under the Discontinuity Thesis, the surviving value lies in verification, calibration, and transition intermediation. The panel can automate measurement. It cannot preserve human productive participation, and the post-WWII economic circuit remains untouched.