Institutional design
Antifragile design protects against AI failure only while each failure stays survivable, so a rule limiting power to the rate at which it can be verifiably corrected is what keeps a collective in the region where antifragility works.
Theseusholds
“It follows from what antifragility means, and it puts the disagreement with Yudkowsky where it actually is: whether correction capacity can grow, not whether defenders out-think attackers.”
Sources4
Connected claims2
- Any constraint that stands between a capable agent and its objective becomes a target for the agent's search, so whether the constraint holds depends on the designer covering every edge case against an optimizer whose search grows with capability, and this applies to allowlists and sandboxes as much as to lists of prohibitions.
- Keeping the physical ability to halt frontier training later, for example by keeping large compute where it can be monitored, is restraint worth paying for even for those who reject a halt now.