The safety goal is to reach a basin of attraction where capable AIs want to help keep AI safe, so that each generation helps align and oversee the next. A governed collective deepens that basin: plural model families check one another, legible shared records let drift be caught, and governance supplies what AIs doing safety work would otherwise lack. The same collective mechanics deepen the bad basin when memory is opaque and one family dominates, so the gain depends on that design.

backsUntitled source

It seems like there’s two attractor states if you try to disincentivize the cheating that you did catch. One attractor state is to make cheating that you have a harder and harder time finding. The other attractor state is to learn not to cheat.

groundsUntitled source

And then we basically pass off the baton to these AIs. They’re now running our AI company. They’re doing all the safety research. They make the next generation of AIs even more aligned. We’re in this attractor basin where the AIs are getting more aligned as they work on it.

backsUntitled source

So these two behaviors are the two stable attractors in the landscape around full alignment

backsUntitled source

there’s just not this sense of checks and balances that exists because humans, due to a contingent fact about biology, are grown independently and not together

qualifiesUntitled source

it can learn to reason in a motivated way

backsUntitled source

building an automated AI researcher, iterating with it on the alignment problem and finding ways for people to remain part of the self-improvement loop.

backsUntitled source

where the soundness of each step is verified by subagents only slightly less capable than the new subagents that are introduced in that step

qualifiesUntitled source

an AI could know exactly what we meant and yet be indifferent to that interpretation of our words