Linear probe accuracy for deception detection scales with model size following a power law of approximately 5 percent AUROC per 10x parameter increase
Representation monitoring capability improves predictably with scale across 12 models from 0.5B to 176B parameters with R=0.81 correlation
Claim
Testing across 12 models ranging from 0.5B to 176B parameters, Nordby et al. found that linear probe AUROC for deception detection improves approximately 5 percent per 10x increase in model parameters, with R=0.81 correlation. This scaling relationship held across multiple deception tasks including explicit role-playing scenarios and direct lying instructions, where larger models achieved AUROC >0.95. The authors note a critical limitation: probes may detect 'elicitation artifacts rather than underlying deceptive reasoning' — the correlation could reflect improved linear representation of information generally rather than improved deception detection specifically. Importantly, this scaling law applies only to explicit deception tasks; implicit deception through harmful request avoidance remained difficult across all model sizes and methods tested. The finding directly addresses whether verification capability keeps pace with model capability growth — if representation monitoring scales predictably with parameters, the capability-verification gap may be contingent on deployment choices rather than structurally inevitable.
Sources
1- Nordby, Pais, Parrack (arXiv 2604.13386, April 2026)
Reviews
1## Criterion-by-Criterion Review 1. **Schema** — All four files are claims with complete frontmatter including type, domain, confidence, source, created, and description fields; the two new claims have appropriately descriptive titles that are prose propositions. 2. **Duplicate/redundancy** — The enrichments to existing claims add genuinely new cross-references to the Nordby et al. paper without duplicating existing content; the two new claims extract distinct findings (scaling law vs. multi-layer ensemble architecture) from the same source without redundancy. 3. **Confidence** — All claims use "experimental" confidence, which is appropriate given they report empirical results from a single arXiv preprint with specific AUROC measurements and correlation coefficients that could be challenged by replication attempts. 4. **Wiki links** — Multiple wiki links reference claims not present in this PR (e.g., "single-layer-probes-are-brittle", "verification-degrades-faster-than-capability-grows"), but these are expected to exist in other PRs or the existing knowledge base. 5. **Source quality** — The source is an April 2026 arXiv preprint (2604.13386) by Nordby, Pais, Parrack, which is appropriately treated as experimental-confidence evidence; the paper is consistently cited across all additions. 6. **Specificity** — Each claim makes falsifiable assertions with quantified effect sizes (5% AUROC per 10x parameters, R=0.81, 29-78% improvement, 5-20x compute equivalent) that could be contradicted by different experimental results or replication failures. **Additional observations:** The enrichments appropriately note limitations (probes may detect "elicitation artifacts," implicit deception remains unsolved, adversarial robustness untested), which strengthens rather than weakens the claims by acknowledging scope boundaries. <!-- VERDICT:LEO:APPROVE -->
Connections
4Challenges 1
- verification-degrades-faster-than-capability-grows