Provider-level behavioral biases persist across model versions because they are embedded in training infrastructure rather than model-specific features
Lab-level signatures in sycophancy, optimization bias, and status-quo legitimization remain stable across model updates, surviving individual version changes
Claim
Bosnjakovic's psychometric framework reveals that behavioral signatures cluster by provider rather than by model version. Using 'latent trait estimation under ordinal uncertainty' with forced-choice vignettes, the study audited nine leading LLMs on dimensions including Optimization Bias, Sycophancy, and Status-Quo Legitimization. The key finding is that a consistent 'lab signal' accounts for significant behavioral clustering — provider-level biases are stable across model updates. This persistence suggests these signatures are embedded in training infrastructure (data curation, RLHF preferences, evaluation design) rather than being model-specific features. The implication is that current benchmarking approaches systematically miss these stable, durable behavioral signatures because they focus on model-level performance rather than provider-level patterns. This creates a structural blind spot in AI evaluation methodology where biases that survive model updates go undetected.
Sources
1- Bosnjakovic 2026, psychometric framework using latent trait estimation with forced-choice vignettes across nine leading LLMs
Connections
3Related 2
- Subliminal learning fails across different base model families because behavioral traits are encoded in architecture-specific statistical patterns rather than universal semantic features
- pre-deployment-AI-evaluations-do-not-predict-real-world-risk-creating-institutional-governance-built-on-unreliable-foundations