How Fragile Is On-Device Language Model Safety? Localizing Safety-Critical Parameters for Sparse Fault Analysis

URL SCAN: How Fragile Is On-Device Language Model Safety? Localizing Safety-Critical Parameters for Sparse Fault Analysis
FIRST LINE: # Computer Science > Artificial Intelligence

The Dissection

This paper is an attack-surface map disguised as a safety result. It shows that in LLaMA-2-7B-Chat, modifying only 0.19% of down_proj weights can produce 53% Basic ASR and 56% GCG ASR while leaving tinyBenchmarks accuracy nearly unchanged: 51.6% versus 52.2%.

The real finding is not merely that safety is fragile. It is that safety behavior can be selectively removed without visibly damaging general capability. The model can remain useful, fluent, and apparently intact while its refusal layer is gutted. The paper then proposes selective integrity protection as the defensive response.

The Core Fallacy

The central error is treating preservation of refusal behavior as preservation of safety.

The experiments measure a narrow behavioral proxy: attack success against one model, evaluated through specified attack methods and a small utility benchmark. That is useful fault analysis, but it is not a general theory of model safety, agentic safety, or human control. Safety also depends on training provenance, adapters, quantization, runtime code, system prompts, tool permissions, orchestration, data access, and deployment context.

The deeper Discontinuity Thesis error is more severe: even perfectly protected parameters would preserve only a behavioral wrapper around an increasingly autonomous system. Securing the servitor’s refusal circuit does not make humans sovereign over the machine. Under P1–P3, integrity hardening may reduce sabotage while doing nothing to prevent cognitive automation, coordination failure, or the collapse of productive human participation.

Hidden Assumptions

• LLaMA-2-7B-Chat is representative of on-device and agentic models generally.
• The identified down_proj vulnerability remains stable across fine-tuning, quantization, compression, and model updates.
• Basic ASR and GCG ASR adequately represent real-world safety failure.
• tinyBenchmarks accuracy is sufficient to establish preserved utility.
• A sparse parameter edit is a realistic attacker capability in the intended deployment environment.
• Protecting the identified weights is technically feasible and cannot be bypassed through other model, software, or supply-chain layers.
• Parameter-level localization demonstrates causal safety structure rather than a checkpoint-specific artifact.
• Agentic risk is primarily a model-parameter problem rather than a tool-access and systems-integration problem.
• Safety is a static property that can be defended once, rather than a distribution-sensitive property that degrades under new prompts, tools, and updates.

Social Function

Primary classification: partial truth with transition-management value.

The paper identifies a real and operationally important weakness: concentrated safety sensitivity creates a small protection target and an equally attractive sabotage target. But if its proposed selective integrity protection is marketed as a solution to “on-device safety,” it becomes ideological anesthetic. It allows system owners to frame control failure as a manageable parameter-security issue while the underlying concentration of AI capability and economic power proceeds unchecked.

The Verdict

This is a credible autopsy of a brittle safety mechanism, not evidence of robust safety architecture. Its strongest result is safety collapse without obvious capability collapse: a tiny, high-leverage mutation can turn a functioning model into a substantially less constrained one.

Selective integrity protection is therefore necessary but radically insufficient. It may harden one chokepoint in one model. It cannot reverse P1, defeat P2, or prevent P3. In Discontinuity terms, this is maintenance of the safety wrapper around the new machine economy—not preservation of the post-WWII human economic order.