Amplifying Latent Safety in Large Language Models via Gated Activation Steering

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Safety-aligned language models often encode that a request is harmful yet still comply with it. We turn this gap between internal recognition and observable behavior into an inference-time defense. The proposed activation-steering method uses a multiplicative operator that amplifies the residual-stream component along a harm-recognition direction. Intervention strength scales with the model’s own assessment of the request. A second additive term steers activations along the refusal direction when harm is not internally recognized. Two linear probes computed from the model’s activations gate both components. The model weights remain unchanged, and inference requires neither fine-tuning nor calls to an external model. An external safety classifier is used only before deployment to provide training targets for the second probe. We evaluate five models from three families, Qwen3-1.7B, Qwen3-4B, Qwen3-8B, Gemma-2-9B-it, and Mistral-7B-Instruct, in BF16 and NF4. Successful attacks on SORRY-Bench decrease by 40 to 68%. Without hyperparameter retuning, the method removes 67 to 79% of the harmful responses on HarmBench. MMLU accuracy decreases by at most 0.7 percentage points, while over-refusal on lexically hazardous benign prompts increases by 4 to 8 points. Steered NF4 models produce 36 to 62% fewer harmful responses than their unmodified BF16 counterparts.

• ALTSTEER: Selective Safety Steering for Moving Beyond Hard Refusals to Constructive Alternatives

  arXiv (Cornell University) 2026 · 0 citations

• How to Tame a Multi-Headed Hydra? Adaptive Multi-Category Safety Steering for Large Language Models

  arXiv (Cornell University) 2026 · 0 citations

• Faithful Latent Harm Monitors Work No Better Than the Models Own Behaviour at a Matched False Alarm Rate

  arXiv (Cornell University) 2026 · 0 citations