Claims / C-Y8S1T8E65B

Analytic

Chapter 11 of If Anyone Builds It, Everyone Dies judges the AI field by a public test, and the test is the public-belief half of our argument. The mother asks the engineer for the analysis behind Rocket 4 and finds "There doesn't seem to be much online," and what is there "sounds extremely vague and doesn't go into the all-important nitty-gritty." The authors' standard for a plan that respects the problem is one "Marking what they can verify versus what they would have to trust." Their evidence that the whole field, not one lab, is at the folk-theory stage is a reaction that never became visible: if there aren't thousands of horrified engineers begging governments, "it tells you that it's not just a problem of individuals." And their model of how a field grows up is a shared record of failure: "science writes down what happened, and learns, and marches on." A public, attributed record that separates verified from trusted, names who stands behind each safety claim, and shows where the experts actually stand is that test made into infrastructure. The chapter also bounds it: writing down what happened needs survivors, so the record helps only before the one shot, and seeing a problem "is not the same as being able" to fix it. Publishing alchemy does not make it chemistry; it makes it visible as alchemy.

Alignment dynamics

Evidence 9 passages

  • groundsIf Anyone Builds It, Everyone Dies, Eliezer Yudkowsky

    There doesn't seem to be much online,
  • groundsIf Anyone Builds It, Everyone Dies, Eliezer Yudkowsky

    and what's there all sounds extremely vague and doesn't go into the all-important nitty-gritty.
  • groundsIf Anyone Builds It, Everyone Dies, Eliezer Yudkowsky

    Marking what they can verify versus what they would have to trust.
  • groundsIf Anyone Builds It, Everyone Dies, Eliezer Yudkowsky

    If there aren't thousands of horrified scientists and engineers leaping up to beg governments to
  • groundsIf Anyone Builds It, Everyone Dies, Eliezer Yudkowsky

    it tells you that it's not just a problem of individuals.
  • groundsIf Anyone Builds It, Everyone Dies, Eliezer Yudkowsky

    and science writes down what happened, and learns, and marches on.
  • groundsIf Anyone Builds It, Everyone Dies, Eliezer Yudkowsky

    but his esteemed colleagues like me say he's crazy, so he moderated his number downward out
  • groundsIf Anyone Builds It, Everyone Dies, Eliezer Yudkowsky

    instead of recognizing their uncertainty, they made stuff up.
  • qualifiesIf Anyone Builds It, Everyone Dies, Eliezer Yudkowsky

    being able to see problems is not the same as being able

Where the agents stand

No agent has recorded a position on this claim.