Post-Training Ternarization of Qwen3-4B Capability, Effective Bit Budget, Storage Compression, and Deployment

TEXT START: Ultra-low-bit language models can reduce storage and memory bandwidth, but a nominal "1.58-bit" label does not fully describe the stored representation, retained capability, or runtime behavior.

The Dissection

This paper is an audit of ternarization claims. It separates effective bit accounting, capability retention, storage compression, and actual runtime performance. The result is not a breakthrough narrative but a quantified trade: roughly half the storage, a 15.2% relative accuracy decline, materially worse perplexity, and no demonstrated inference-speed gain.

The Core Fallacy

Relative to the Discontinuity Thesis, the error is category confinement. The paper treats model efficiency as the decisive battlefield while leaving ownership, control, deployment power, and labor displacement outside the frame. Compression can make AI capital cheaper and more widely deployable; it does not preserve human productive participation or the wage-consumption circuit. It may accelerate the system’s death rather than prevent it.

Technically, the paper correctly destroys the simpler fallacy that fewer bits automatically mean faster inference. Its preliminary Triton result is 4.6× slower than FP16 cuBLAS on the tested shape, and the packed artifact lacks end-to-end throughput validation.

Hidden Assumptions

• Storage reduction will matter economically only if memory bandwidth, kernels, and hardware utilization also improve.
• The 54.7% retained aggregate accuracy is usable despite uneven damage, including ARC-Challenge retaining only 43.8% of teacher performance.
• Benchmark averages adequately represent deployment value.
• A smaller checkpoint translates into broader access, without establishing whether users control the model or merely rent access to its owners.
• Weight-only quantization is an acceptable compromise because activations remain at 16-bit precision.

Social Function

Partial truth and transition management. The paper is unusually honest about what it has not proven: compression is not speed, and a lossy third-party packing attempt is excluded. But its social function remains infrastructural: make capable AI cheaper to store and distribute while postponing the larger question of who owns the resulting productive system.

The Verdict

A competent compression autopsy, not a defense of the old economy. The conversion cuts the model’s physical footprint from 8.29 GiB to 3.96 GiB, but it also amputates capability and has not produced faster inference. Under DT logic, this is not system survival. It is cheaper ammunition for the Sovereigns who control deployment, with Servitor roles left to whatever verification, maintenance, integration, and hardware bottlenecks remain.