Whose Judgments Count? Representation Gaps in Crowdsourced Content Moderation Produce Unequal Protection from Perceived Toxicity

TEXT START: Content moderation is a central form of digital governance, yet people disagree over what content should be removed from shared online spaces.

The Dissection

This is an audit of moderation as a distribution system, not a neutral classifier. It shows that the demographic composition of moderator pools changes who receives protection from perceived toxicity. The central finding is structurally damning: even a nationally representative pool can underprotect Black and LGB users, while moderator pools resembling self-identified moderators on Prolific widen the disparity.

The paper’s real subject is legitimacy: who gets to define unacceptable speech, whose perceptions become platform policy, and who benefits when content is removed.

The Core Fallacy

The paper’s weakness is its residual faith in representational tuning. It treats demographic composition as the main governance lever and “equal protection” as a coherent target, but perceived toxicity is precisely what varies across groups. A representative pool does not eliminate unequal standards; it merely aggregates them with different weights. If minority users require deliberate overrepresentation to receive comparable protection, representation has already failed as a sufficient rule.

Under the Discontinuity Thesis, this is a lag-era repair mechanism. It adjusts the inputs to a human judgment pipeline while leaving ownership, objectives, deployment, and enforcement with the platform sovereign. AI can absorb and scale the judgments; changing the demographic mix does not transfer control of the system.

Hidden Assumptions

• Demographic alignment is the main cause of the observed protection gaps, rather than content selection, ideology, platform culture, or measurement artifacts.
• Comment removal is a valid proxy for protection, and non-removal is a valid proxy for exposure to harm.
• Judgments from U.S. respondents generalize across Twitter, Reddit, and 4chan despite their radically different norms and user populations.
• Counterfactual reweighting accurately predicts real moderation pools, including strategic behavior by users, platforms, and moderators.
• “Equal protection” is a single measurable standard despite conflicting group judgments and intersectional identities.
• Deliberate overrepresentation can correct disparities without generating new asymmetries or legitimacy crises.

Social Function

Classification: partial truth with a transition-management and legitimacy function.

The partial truth is real: diversity in moderation inputs is not decorative. It materially affects whose speech is treated as dangerous and whose vulnerability is recognized. But the institutional function is narrower and more convenient than the diagnosis suggests. It converts a control problem into a sampling problem. Platform operators can present moderation as fairer by adjusting the pool while retaining final authority over the objective, data, model, and enforcement system.

That is not necessarily propaganda. It is more dangerous than propaganda because it can be empirically correct while still functioning as an anesthetic: representation rituals make a sovereign governance system appear corrigible without changing who owns it.

The Verdict

A valuable partial autopsy, not a cure. The paper demonstrates that human aggregation does not naturally produce equal protection and that even representation can reproduce structural exclusion. In DT terms, it is evidence of the instability of human-only governance at scale. It identifies the asymmetry in the moderation pipeline, but leaves the platform sovereign holding the scalpel.