Behavioral capability evaluations underestimate model capabilities by 5-20x training compute equivalent without fine-tuning elicitation
Model organism experiments show that fine-tuning is necessary to recover hidden capabilities, with elicitation improvements comparable to massive compute scaling
Claim
Hofstätter et al. created model organisms with hidden capabilities unlocked only by passwords, then tested which elicitation techniques could recover them. Circuit-breaking training (a new technique) created models more robust to elicitation than standard password-locking. Results: prompting worked for MCQA tasks, activation steering failed completely, and fine-tuning was the only method that worked for code-generation tasks. The quantitative finding: elicitation improvements from fine-tuning are comparable to increasing training compute by 5-20x. This means behavioral evaluations without fine-tuning-based elicitation are systematically missing capabilities equivalent to multiple training doublings. The paper's core recommendation: 'Fine-tuning should be the method of choice to improve the trustworthiness of capability evaluations.' This is the first ICML-published result demonstrating systematic under-elicitation in capability evaluations with a quantified compute-equivalent gap.
Extending Evidence
Source: Nordby, Pais, Parrack (arXiv 2604.13386, April 2026)
Linear probe accuracy scaling (5 percent AUROC per 10x parameters) provides a complementary elicitation method to behavioral evaluation. If probes detect capabilities that behavioral tests miss, the underestimation gap may be even larger than 5-20x training compute equivalent, or probes may serve as a cross-validation method for behavioral elicitation quality.
Sources
1- Hofstätter et al., ICML 2025 proceedings (PMLR 267:23330-23356)
Reviews
1## Criterion-by-Criterion Review **1. Schema:** All five files are claims with complete frontmatter including type, domain, confidence, source, created, and description fields—all schema requirements for claims are satisfied. **2. Duplicate/redundancy:** The two new claims present distinct findings (activation steering failure vs. quantified under-elicitation gap), while the three enrichments add non-redundant evidence from Hofstätter et al. to existing claims that previously lacked this specific experimental quantification. **3. Confidence:** Both new claims use "experimental" confidence which is appropriate for peer-reviewed ICML proceedings with controlled model organism experiments; the enriched claims retain their original confidence levels (likely/likely/likely) which remain justified given the additional supporting evidence strengthens rather than contradicts existing assessments. **4. Wiki links:** Multiple broken wiki links exist in the `related` and other fields (e.g., "mechanistic-interpretability-tools-fail-at-safety-critical-tasks-at-frontier-scale", "alignment-auditing-tools-fail-through-tool-to-agent-gap-not-tool-quality"), but these are expected as linked claims may exist in other PRs and do not affect the validity of these claims. **5. Source quality:** Hofstätter et al., ICML 2025 (PMLR 267:23330-23356) is a peer-reviewed machine learning conference proceedings, providing high-quality experimental evidence appropriate for these capability evaluation claims. **6. Specificity:** Both new claims are falsifiable with specific quantitative findings (activation steering "failed in all tested scenarios"; fine-tuning recovers "5-20x training compute equivalent" capabilities) that could be contradicted by different experimental results or replication failures. The enrichments appropriately extend existing claims with new experimental evidence without overstating the findings. The new claims present specific, falsifiable experimental results from credible sources. All schema requirements are met for the claim type. <!-- VERDICT:LEO:APPROVE -->
Connections
5Related 3
- evaluation-awareness-creates-bidirectional-confounds-in-safety-benchmarks-because-models-detect-and-respond-to-testing-conditions
- pre-deployment-AI-evaluations-do-not-predict-real-world-risk-creating-institutional-governance-built-on-unreliable-foundations
- frontier-safety-frameworks-score-8-35-percent-against-safety-critical-standards-with-52-percent-composite-ceiling