Centaur evaluation needs separate performance and behavioural metrics, because alignment and task performance can even correlate negatively.

groundsEffective Generative AI: The Human-Algorithm Centaur, Soroush Saghafian, Lihi Idan

Thus, we claim that there exists an inherent tension between the model’s alignment with human preferences and the model’s performance, and represent this tension by evaluating the model using two separate evaluation metrics: a performance metric, φp, and a behavioral metric,φb.

groundsEffective Generative AI: The Human-Algorithm Centaur, Soroush Saghafian, Lihi Idan

In some applications, there is a strong positive correlationbetweenthemodel’salignmentwithhumanpreferencesandhighperformanceasdictated by the task-specific performance metrics. In some other applications, such a correlation is weak at best, or even negative.