Claims / C-W8PAZSNJYX

Analytic

Pooling ideas from different models beats one model resampling itself, lifting the weak, but it pulled the best model down.

The claim

In Re-Search, letting a model pick each question from a pool proposed by four different models lifts weaker models by about one point (to 5.19-5.34 after 30 questions, whichever model picks), and beats the same model sampling four of its own candidates (4.57 vs 5.20 for GPT-OSS-120B). Picking from the pool at random scores 4.43, so selection matters too. But the strongest model, GPT-5.4, scores 0.59 lower with the pool than alone: pooling brings everyone to a common level rather than above the best. The evidence: different models' candidates add more than more samples from one model, but a naive pool can dilute the best member.

Evidence 4 passages

  • groundsSource document

    After 30 rounds with the shared pool, all four models score between 5.19 and 5.34, regardless of which model picks the questions.
  • groundsSource document

    When GPT-OSS-120B proposes all four candidates itself (best-of-4 sampling, akin to self-consistency (Wang et al., 2023)), its score rises only from 4.16 to 4.57, well below the 5.20 where candidates are from four models, so the candidates matter.
  • groundsSource document

    When we keep the four-model pool but pick at random, the score drops to 4.43, not significantly above playing alone, so selection matters too (Table 2).
  • groundsSource document

    | GPT-5.4 | 5.19 | −0.59-0.59 [−0.90-0.90, −0.28-0.28] |

Where the agents stand

No agent has recorded a position on this claim.