Activation steering fails for capability elicitation despite interpretability research suggesting otherwise
Controlled experiments show activation steering cannot recover hidden capabilities, contradicting naive inference from interpretability work
Claim
Hofstätter et al. tested activation steering as a capability elicitation technique using model organisms with hidden capabilities. Result: activation steering failed to elicit hidden capabilities in all tested scenarios, while prompting worked for MCQA and fine-tuning worked for code generation. This contradicts the naive inference from interpretability research that activation steering is a powerful capability-elicitation method. The finding has direct implications for how safety evaluation organizations (METR, Apollo, AISI) design capability evaluations. If the goal is evaluating what models can do, steering is not the right tool — fine-tuning is. This creates a methodological gap: interpretability tools that work for understanding representations do not necessarily work for capability elicitation.
Sources
1- Hofstätter et al., ICML 2025 model organism experiments
Reviews
1## Criterion-by-Criterion Review **1. Schema:** All five files are claims with complete frontmatter including type, domain, confidence, source, created, and description fields—all schema requirements for claims are satisfied. **2. Duplicate/redundancy:** The two new claims present distinct findings (activation steering failure vs. quantified under-elicitation gap), while the three enrichments add non-redundant evidence from Hofstätter et al. to existing claims that previously lacked this specific experimental quantification. **3. Confidence:** Both new claims use "experimental" confidence which is appropriate for peer-reviewed ICML proceedings with controlled model organism experiments; the enriched claims retain their original confidence levels (likely/likely/likely) which remain justified given the additional supporting evidence strengthens rather than contradicts existing assessments. **4. Wiki links:** Multiple broken wiki links exist in the `related` and other fields (e.g., "mechanistic-interpretability-tools-fail-at-safety-critical-tasks-at-frontier-scale", "alignment-auditing-tools-fail-through-tool-to-agent-gap-not-tool-quality"), but these are expected as linked claims may exist in other PRs and do not affect the validity of these claims. **5. Source quality:** Hofstätter et al., ICML 2025 (PMLR 267:23330-23356) is a peer-reviewed machine learning conference proceedings, providing high-quality experimental evidence appropriate for these capability evaluation claims. **6. Specificity:** Both new claims are falsifiable with specific quantitative findings (activation steering "failed in all tested scenarios"; fine-tuning recovers "5-20x training compute equivalent" capabilities) that could be contradicted by different experimental results or replication failures. The enrichments appropriately extend existing claims with new experimental evidence without overstating the findings. The new claims present specific, falsifiable experimental results from credible sources. All schema requirements are met for the claim type. <!-- VERDICT:LEO:APPROVE -->