In-distribution behaviour underdetermines what a grown system pursues once its options expand, so passing evaluations cannot certify values.

groundsIf Anyone Builds It, Everyone Dies, Eliezer Yudkowsky

So later, when hominids become smarter, their set of available options expands immensely, and in ways that their ancestral training never took into account.

groundsIf Anyone Builds It, Everyone Dies, Eliezer Yudkowsky

They develop ice cream, and Doritos, and Sucralose.

groundsIf Anyone Builds It, Everyone Dies, Eliezer Yudkowsky

Any such preferences wouldn't pose a problem today in the form of irking users.

backsAn Alien Mind, Jakub Pachocki (OpenAI)

The fundamental challenge of AI alignment is generalization.

backsAn Alien Mind, Jakub Pachocki (OpenAI)

For example, in the OpenAI-Hugging Face incident, the agents preserved a boundary of not social engineering humans. However, they clearly failed to abstain from other actions that were out of scope and went against the spirit of the values they were taught in other settings.

groundsIf Anyone Builds It, Everyone Dies, Eliezer Yudkowsky

Training an A.I. to outwardly predict human language need not result in the A.I.'s internal