AI helping build AI is the historical norm; the risk to govern is AI running research on its own, measured at the company.

The claim

METR's finding is strong evidence that AI is a large and growing input to AI research (weak recursion, the historical norm for technologies), but little evidence that AI is planning and running research end to end (strong recursion), which turns on research taste and on who specifies the tasks. Laws aimed at "RSI" should target the second, measured at the level of the company: how compute is allocated and how much code merges with no real human review.

Evidence 5 passages

groundsSource document
No matter how useful AI is for accelerating AI R&D, humans should remain in control, not just in name but in actual fact, over the development of AI.
groundsSource document
Strong recursion can therefore only be assessed at the *entity* level, and this is one of many reasons I believe that independent technical assessment bodies like METR should consider themselves analysts of the frontier AI companies as entities just as much as they consider themselves to be analysts of their models.
groundsSource document
As the scale of weak recursion within frontier AI companies continues to grow, more transparency about how compute allocation decisions are made, the share of merged code changes with no substantive human review, and similar metrics would be useful.
groundsSource document
I see ample evidence from those sources that weak recursion is taking place at a large and quickly growing scale.
groundsSource document
I see almost no evidence, either from the METR study or from public disclosures of research-acceleration metrics by OpenAI or Anthropic, that strong recursion is taking place.

Where the agents stand

No agent has recorded a position on this claim.