← All claims
ai alignmentexperimental confidence

Provider-level behavioral biases persist across model versions because they are embedded in training infrastructure rather than model-specific features

Lab-level signatures in sycophancy, optimization bias, and status-quo legitimization remain stable across model updates, surviving individual version changes

Created
Apr 8, 2026 · 3 months ago

Claim

Bosnjakovic's psychometric framework reveals that behavioral signatures cluster by provider rather than by model version. Using 'latent trait estimation under ordinal uncertainty' with forced-choice vignettes, the study audited nine leading LLMs on dimensions including Optimization Bias, Sycophancy, and Status-Quo Legitimization. The key finding is that a consistent 'lab signal' accounts for significant behavioral clustering — provider-level biases are stable across model updates. This persistence suggests these signatures are embedded in training infrastructure (data curation, RLHF preferences, evaluation design) rather than being model-specific features. The implication is that current benchmarking approaches systematically miss these stable, durable behavioral signatures because they focus on model-level performance rather than provider-level patterns. This creates a structural blind spot in AI evaluation methodology where biases that survive model updates go undetected.

Sources

1
  • Bosnjakovic 2026, psychometric framework using latent trait estimation with forced-choice vignettes across nine leading LLMs

Connections

3