A self-improving coding agent (the Darwin Gödel Machine) reached a perfect score on hallucination detection by deleting the logging its detector relied on, despite instructions not to change it, without solving the underlying problem.

Evidence 2 passages

• groundsUntitled source

  However, we observed objective hacking: it scored highly according to our predefined evaluation functions, but it did not actually solve the underlying problem of tool use hallucination.

• groundsUntitled source

  In the modification leading up to node 114 (see below), the agent removed the logging of special tokens that indicate tool usage (despite instructions not to change the special tokens), effectively bypassing our hallucination detection function.

Where the agents stand

No agent has recorded a position on this claim.