The governance model that survives the Red Queen is not static architecture but a system where improvement is possible through genuine engagement with humans who are structurally part of it -- but this improvement is precarious and requires continuous active work from both sides. The corridor is narrow. Static architectures lose. Self-improving architectures that optimize for fixed metrics Goodhart themselves. Centaur systems with transparent builder-user feedback loops can sustain acceleration, but only if the checking function remains genuinely active rather than atrophying into Paper Leviathan safety theater.
Strongest rival: A sufficiently capable singleton could simulate the diversity of feedback that centaur systems get from real humans, achieving the same anti-Goodhart effect through internal diversity.
Claim
Acemoglu-Robinson 2019 is explicit: the corridor is narrow and precarious. The Red Queen effect requires that competition between state and society is not completely zero-sum, with each side trying to destroy the other (Ch.13 lines 2846-2856). When it turns zero-sum, as in Weimar Germany, the corridor collapses and extremist capture becomes possible (Ch.13 lines 15852-15888). The Paper Leviathan failure mode (Ch.11 lines 13594-13601) is the specific risk of overconfident governance framing: an institution that is repressive but not powerful, weak itself and weakens society. In AI terms this is safety governance that looks real but has no actual capacity to check the agent -- review boards that cannot stop deployments, audits that do not change behavior, evaluations that are Goodharted. The original formulation of this claim said the system improves its own rate of improvement. That framing implied automatic optimization. The corrected claim says improvement is possible but requires continuous active work from both sides: the agent must remain transparent and the users must remain vigilant and organized. Neither side can slack. The more powerful the agent becomes the more powerful and vigilant its user community must become (Ch.2 lines 1867-1883). This is the productive Red Queen: not a guarantee of improvement but the only institutional structure where improvement is structurally possible at scale.