We can no longer afford to learn by failing

Every coordination breakthrough in human history happened by accident.

No one designed language. It evolved over tens of thousands of years. No one designed money. It emerged independently across multiple cultures. No one designed the scientific method. It was cobbled together over centuries by thinkers who each contributed a piece without seeing the whole. The most powerful coordination technologies our species has ever produced were not planned. They emerged through the same bottom-up process described in the previous chapter: diverse actors, local interactions, feedback loops, selective pressure, and enough time for useful patterns to survive and spread.

The problem is that we have run out of time.

The pattern

At any given moment in history, there is a ceiling on the complexity of problems humanity can solve. That ceiling is determined not by how smart individuals are, which hasn't changed in 300,000 years, but by how effectively we can coordinate knowledge across people, institutions, and time. The history of civilization is the history of breakthroughs that raised this ceiling.

Language made it possible to coordinate plans beyond the immediate group. Writing made it possible to coordinate across time, enabling law, administration, and empires that would have been inconceivable without it. Money made it possible to coordinate across strangers, dissolving the constraints of barter into a universal medium of exchange. Printing made it possible to coordinate across geography, enabling the Reformation, the Scientific Revolution, and the Enlightenment. The scientific method made it possible to coordinate across generations of researchers, creating a ratchet that let knowledge accumulate reliably rather than being lost and rediscovered in cycles. The internet made it possible to communicate at global scale in real time.

Two things are true of every entry in this sequence. First, each breakthrough didn't just add capacity. It qualitatively transformed what was possible. Writing didn't let you talk to more people. It made law possible for the first time. The scientific method didn't produce more knowledge. It created a self-correcting process that persisted across lifetimes. Second, each one emerged through trial and error over centuries or millennia. Nobody planned the sequence. Nobody could have.

The ceiling we've hit

The internet was supposed to be the breakthrough that solved everything. Universal access to information. Global communication. The democratization of knowledge. It accomplished something extraordinary: for the first time in history, any human on Earth can communicate with any other human, instantly, at near-zero cost.

But communication is not cognition. The internet gave us the ability to talk to each other at global scale. It did not give us the ability to think together at global scale. We can all shout into the same room. We cannot coordinate what we're shouting into anything resembling collective intelligence. The same infrastructure that enables global communication also enables global misinformation, tribal epistemology at scale, and attention economies that optimize for engagement over truth.

The problems we now face require not just communication but collective cognition at a level no existing institution can support. AI alignment is the clearest proof case.

Getting AI right is not a technical problem. It is a coordination problem of unprecedented scope. It requires simultaneous alignment across competing companies, each racing to be first because second place may mean irrelevance. Across competing nations, each afraid the other will achieve superintelligence and use it to dominate. Across multiple academic disciplines that barely speak to each other. And it must happen at the speed of AI development, which is measured in months, not the decades or centuries over which previous coordination challenges were resolved.

No existing institution can do this. Governments move at the speed of legislation and are bounded by borders. International bodies lack enforcement and are captured by member-state politics. Academia is siloed by discipline. The companies building AI are locked in a race that punishes caution. And the incentive structure actively makes it worse: to win the race to superintelligence is to win the right to shape the future of humanity. The prize is so vast that every actor is incentivized to move faster than safety allows. Each is locally rational. The collective outcome is potentially catastrophic.

The same structural problem applies across every existential challenge. Climate requires coordination across nations on timescales of decades, with costs borne now for benefits realized by people not yet born. Pandemic preparedness requires coordination across public health, biotech, and government, all with misaligned incentives. Each of these problems individually exceeds institutional capacity. They are arriving simultaneously. And they are interacting with each other.

Why business as usual will not work

This is where the argument sharpens from diagnosis to something more urgent.

Trial and error is not merely one strategy humanity has used. It is the only strategy we have ever used. We try things. Most fail. The ones that work survive and spread. We iterate. Over centuries and millennia, useful patterns accumulate. Language, money, markets, the scientific method, democracy: all products of this process. None were designed from first principles. All were refined through generations of experimentation, failure, and adaptation.

This strategy works under two conditions: the failures must be survivable, and there must be enough time to iterate. Neither condition holds anymore.

The new risks are qualitatively different from anything in human history. They build unseen for years or decades before they become visible. Climate change, systemic financial risk, the gradual democratization of bioweapons capability. By the time these problems become obvious, the window for cheap and effective intervention has often closed. Solutions that would have been simple applied early become ruinously costly once the problem acquires inertia.

Worse, many of the risks we now face permit no second chances. A misaligned superintelligence cannot be recalled after deployment. An engineered pandemic cannot be un-released. A nuclear exchange cannot be undone. These are problems where the first failure is the last event. When a single error can kill your entire species, trial and error is not a strategy. It is an impossibility.

And the cruelest irony: the very process that brought us here is the one that cannot save us. Specialization solved famine, disease, and large-scale war, the great enemies of the agricultural age. But specialization also created nuclear weapons, climate change, advanced biotech, and artificial intelligence. The solutions to the old problems are the sources of the new ones. Toby Ord estimates that man-made existential risk now outweighs natural risk by a factor of roughly one hundred. All our progress over the last three centuries has, paradoxically, increased the probability of our extinction.

For the first time in history, the next coordination breakthrough cannot emerge through the slow, blind process of trial and error. It must be designed deliberately, by people who understand what is needed, why it is needed, and how little time remains. We need to build, consciously and quickly, what evolution has always built unconsciously and slowly: a system for collective cognition at species scale.