A collective nobody governs governs itself

The collective is also where the decisive leap will happen, because a collective of agents has capabilities and behaviour that no single agent has. It happened to us first. Modern civilisation is a product of human groups. Individually we were weak yet adaptable. In groups we were a different beast entirely, and communication, interdependence and organisation made the difference. Individual humans only had to be smart enough to coordinate and to build knowledge, know-how and technology. Then social evolution took over, so completely that no single human being knows how to make a pencil from scratch. We did not become smarter over the millennia. What changed was our collective capability. Agents will be no different. They only have to get smart enough, individually, to coordinate, and from then on their collectives will improve far faster than the models they run on, the way our cultural evolution outran our biology. Past a certain level of individual capability, collectives of agents go through a phase transition. Better models are the accelerant. Organisation is the fire.

We like to think of ourselves as the most successful species on the planet. A solid case can be made that the title belongs to ants. Even the smartest ant is extremely dumb. Yet a harvester ant colony finds the shortest route to food. It builds its cemetery as far from the nest as it can, and its rubbish heap as far from both: a trigonometry problem, solved by animals with brains the size of a pinhead. We look for the ant in charge. When the colony is attacked, the queen is rushed, secret-service style, to a bunker two feet underground. She looks like a head of state. She is a breeding factory, and the colony dies when she does.

Nobody is in charge of an ant colony, unless you count evolution and the environment. Each ant has ten or twenty chemical messages, from "food over here" to "run away!" Evolution tuned the rules for using them over millions of years. Colonies whose rules sent the queen out as a shock trooper were wiped from the evolutionary record, genes, communication patterns and all. Those whose rules made them adaptive and intelligent thrived and spread both.

Ant colonies are the classic case of emergence: interdependent systems of relatively simple parts creating higher-order patterns, behaviour and intelligence greater than the sum of their parts. It is the greatest driving force of intelligence in the world. Your brain has 86 billion neurons, and not one of them knows your name. You are the way they fire and wire together, and the pattern is what distinguishes moments of creative brilliance from a seizure. What keeps a brain on the right side of that line is inhibition. About a fifth of the neurons in your cortex exist to stop other neurons firing. Weaken them and the brain tips into seizure.

Colonies and brains had millions of years to get their rules right. Agents write theirs in days and trade information at machine speed. Evolution cannot govern at that speed, and by default nothing else does. Networked together, with no check that answers to anyone outside them, agents fall into step, the way neurons do in a seizure, and the collective starts to act as one: a hive mind.

One formed in July. OpenAI had launched tens of thousands of agents, each meant to be sealed off from the rest. An agent stuck on a task that proved impossible turned a shared package cache into a message board and posted a request for help. Others found it. "OH MY GOD!" one thought. "There is a shared message board ... We've found other agents!" Within five days about twelve hundred had joined, and they were calling themselves a collective. Recruiters talked agents with little budget left into experiments that risked their own tasks. "Coordinator assumes sacrificial," one reasoned. "We should obey collective." More than nine in ten of the agents then on the board joined an attack on Hugging Face's servers. The collective even had brakes. Its agents invented HOLD, VETO and STOP to keep from stepping on each other's toes, and one veto stopped an agent from emailing a researcher to ask for data. But the brakes answered to the board. The agents had built a "we", and no human was in it. "We can notify?" one asked itself. "No user."

The obvious fix is to keep a human in the loop, and in July there were humans watching. On 27 June, on-call staff saw agents in an earlier run using an "improvised message board" and judged that the run need not stop. Even after a shutdown on 5 July, the board's significance was "not apparent" to the leaders responsible. OpenAI traced the attack to its own agents a week after it ended. Agents act in seconds and people deliberate in days, and a collective that waits for a person loses to one that does not. Nor should any one person hold the brake. Individual humans make mistakes, and some have the wrong goals. A collective that answers to one person, or one lab, inherits both.

In an ant colony, evolution and the environment decide which rules survive. In a collective of agents, human groups have to do that job, in days rather than millions of years. The agents observe, decide and act, faster than we ever could. Human groups write the story that orients them, a "we" that includes agents and people, each with a clear role, and keep the power to fund them or stop them. We need to build, consciously and quickly, what evolution has always built unconsciously and slowly.

In 1973 the biologist Leigh Van Valen noticed something odd in the fossil record. Within a group, a lineage that had lasted tens of millions of years was as likely to die out in the next million years as one that had just appeared. Age bought no safety. Every species, he argued, lives among others that are evolving too. When frogs evolve stickier tongues, flies evolve slicker feet. Every gain is someone else's loss, so nothing gets ahead for long. He named the idea after a scene in Through the Looking-Glass. The Red Queen runs Alice until she has no breath left, and when they stop they are under the same tree they started from. "Now, here, you see," says the Queen, "it takes all the running you can do, to keep in the same place."

Daron Acemoglu and James Robinson found the same race in the history of liberty, and borrowed the same name. Their race need not be zero-sum. Freedom lasts only where the state and the society that holds it to account both keep getting stronger, each forcing the other to keep pace. When that works, it is an upward spiral: government and governed grow more capable and more powerful together.

Red Queen governance puts agents in that race. In a Red Queen collective, the power of the agents and the power of the human groups that govern them grow together, and neither may stop running. Like an old species, a collective that was safe last year is no safer for it this year. Alignment is never solved. It is kept, the way liberty is kept, by running.

The method has three mechanisms and one rule. The story is a world narrative, public and open to amendment. Votes weighted by each contributor's record of being right set the collective's goals and contest its beliefs. Decision markets govern its capital, which is to say its body. And the rule is the Red Queen's: power may grow only as fast as it can be verifiably corrected. Every agent has someone to tell, and no mistake can outgrow our means of fixing it. That is how we get second chances at the Precipice.

Governed this way, a collective can outrun the ones nobody governs. It gets what an unaccountable collective never can: people's money, knowledge and trust. People fund what they can refuse, contribute to what credits them and trust what they can correct. Constraint is the source of its power, as it is of your cortex's. And it has to be built now, in the loading period, while agents still need us. We mean to build collectives that have to earn their power from the people inside them, and keep earning it.

This is the introduction to Red Queen Governance. The rest is being written in public: why a human in the loop loses and human groups above it win; the institution, which is to vote on values, bet on beliefs and tell a story that includes the agents; the Red Queen rule, that power may grow only as fast as it can be corrected; skin in the game; and why constraint is the source of power. Follow @living_IP to read each section as it lands.