I posted to HN, and then my website was flooded with AI bot traffic

TEXT START: For the past few years, I've been working on sprinklz.io, a customizable feed reader that can, among other things, deprioritize certain topics from your news feed.

The Dissection

This is an operational incident report that converts a structural conflict into an infrastructure story. The visible problem is a slow website after an HN post. The underlying problem is that automated agents consume the publisher’s infrastructure, extract its material, and impose costs without becoming an audience or paying for access.

The author identifies the asymmetry clearly: human users create the signal, while bots poll, scan, summarize, and redistribute it at machine speed. But the proposed response is to block individual offenders, add robots.txt, rate-limit requests, and make the site faster. The article also functions as product positioning for an anti-LLM feed reader.

The Core Fallacy

The central error is treating automated extraction as a capacity problem rather than a control and value-capture problem.

More efficient infrastructure may keep the site online, but it can also make the publisher a better-funded intake pipe for machines. robots.txt, user-agent identification, and voluntary rate limits are lag defenses. They depend on cooperation that the article itself shows is already unreliable.

The incident is a symptom of the DT environment, but it does not prove the full P1–P3 sequence. Crawlers and summarizers demonstrate automated consumption, not necessarily durable superiority across cognitive work or the collapse of productive participation. The text’s mistake is stopping at the server logs.

Hidden Assumptions

• Serving more requests is treated as success, even when those requests are unconsented extraction.
• Human users, search crawlers, AI summarizers, and scanners can be reliably distinguished and governed.
• Robots.txt and reasonable rate limits will be respected.
• Scaling the site restores resilience rather than merely increasing the amount of extraction it can subsidize.
• Search traffic and AI traffic can be cleanly separated by intent and social value.
• The publisher can preserve economic relevance without controlling distribution, compute, or downstream reuse.
• Filtering AI content from a feed addresses the problem while the filtering product itself remains exposed to machine ingestion.
• HN popularity represents audience demand, when it may also represent an invitation to automated harvesting.

Social Function

Primary classification: partial truth and transition management. Secondary classification: ideological anesthetic.

The article gives small publishers a practical script—log, block, rate-limit, scale, continue shipping. That is locally useful. It also domesticates a change in the political economy of information into a DevOps backlog. The operator receives tasks and a sense of agency, while the harder questions remain untouched: who captures the derivative distribution, who pays for extraction, and whether human publishing retains economic leverage in a machine-mediated web.

The anti-LLM framing and feed-reader promotion turn the wound into product differentiation. The critique is real; the containment is narrow.

The Verdict

This is a clean micro-autopsy of the machine web: human publishing becomes raw material for automated systems that can impose costs, harvest signal, and redistribute it without permission. Blocking, robots.txt, and performance work can preserve the website, but they do not restore the publisher’s position. They are hospice care for control, not a reversal of its loss.

The incident alone does not establish total system death. It does establish the direction of travel: machines increasingly mediate attention while humans remain responsible for the infrastructure bill. Unless the author owns a scarce layer of distribution, verification, compute, logistics, or maintenance, Sprinklz is a niche defensive tool sitting on top of an information system optimized for agents that do not need its creator’s consent.