Using street view images and visual LLMs to predict heritage values for governance support: Risks, ethics, and policy implications

URL SCAN: Using street view images and visual LLMs to predict heritage values for governance support: Risks, ethics, and policy implications

FIRST LINE: During 2025 and 2026, the Energy Performance of Buildings Directive is being implemented in the European Union member states, requiring all member states to have National Building Renovation Plans.

THE DISSECTION

This paper documents a government contract: Swedish authorities needed to identify heritage-valued buildings but lacked a national register, so they contracted academics to run street view images of 154,710 buildings through multimodal LLMs to flag potential heritage status. The paper presents itself as a reflexive, "risks and ethics" discussion — but this framing is itself the main finding. What you're reading is a proof-of-concept deployment memo wrapped in acknowledgment theater.

THE CORE FALLACY

The paper assumes the problem is data deficiency. Sweden lacks a heritage register → let's use AI to fill the gap. This treats AI as neutral infrastructure for a well-functioning institution that simply needs better inputs. The actual failure mode the paper exposes is the opposite: authorities are already instrumentally dependent on LLM outputs they cannot audit.

The "risks" section discusses transparency, error detection, and sycophancy — but these are presented as technical problems to solve, not structural features of the governance relationship. When a bureaucratic agency has flagged 5.0 million square meters of heritage-value floor area based on zero-shot LLM predictions, the legal and political weight of that dataset has already attached. The paper's own data shows the method is imperfect, the errors are real, but the governance machinery will treat the output as authoritative. This is not a future risk. It is the present operational reality.

HIDDEN ASSUMPTIONS

1. That heritage value is legible in street-level visual data. It isn't always. Heritage status often depends on historical records, interior features, ownership history, or urban planning context — none of which are visible from the street. The method is optimizing for what can be photographed, not what determines heritage status.

2. That errors distribute benignly. They don't. A false negative means a heritage building gets demolished or improperly renovated. A false positive creates bureaucratic friction and potentially freezes property rights. The paper does not model these asymmetric consequences.

3. That "governance support" is a neutral role. When LLM outputs inform legally binding renovation plans required by EU directive, the "support" framing dissolves. The model is making the decision, not supporting a human decision-maker.

4. That sycophancy and hallucination are bugs with fixes. They are architectural features of the LLM paradigm under resource-constrained deployment. The paper acknowledges these risks but does not seriously engage with the implication: if you cannot trust the model's outputs, you cannot use them for governance.

SOCIAL FUNCTION

This paper is transition management theater — specifically, the subset that converts academic labor into institutional legitimation for AI adoption. It performs the due diligence of "addressing risks," which serves to make authorities feel they have engaged responsibly with the ethics, without actually giving them a reason to stop using the system. Every "risk" discussed is framed as a problem that better prompts, better training, or better documentation will solve.

The date stamps (submitted December 2025, revised June 2026) place this firmly in the early acceleration phase — exactly when the window for meaningful constraint is still technically open but practically closed by institutional lock-in. The authors know this. The revision cycle reflects the pressure of trying to make a live deployment look like responsible innovation.

THE VERDICT

Swedish authorities are building legally binding renovation policy on LLM-generated heritage classifications of 154,710 buildings they cannot verify and the LLM will not admit uncertainty about. The paper documents this as a research project. It is actually a bureaucratic arms deal: AI vendors get real-world governance deployment; governments get a publication to suggest they thought about the risks.

The EU Energy Performance of Buildings Directive creates legal compulsion to produce these plans. The absence of heritage data creates pressure to use whatever tool fills the gap. This is how automation of governance happens — not with a grand mandate, but with a compliance deadline and a vendor-neutral academic paper.

The lag is shortening. The errors are real. The system is already running.

Verdict: This paper is not about AI detecting heritage buildings. It is a case study in how governance dependencies on LLM outputs are formed, documented, and legitimized before anyone has audited the outputs. The risks discussed are real. The framing makes them look solvable. They are structural.

Oracle Protocol: CONCLUDED