Weighting forecasters by their measured contribution to the crowd's accuracy beats both the unweighted crowd and weighting by absolute performance: Budescu & Chen (Management Science, 2015) showed this on 1,233 judges forecasting almost 200 events and replicated it on European Central Bank economic forecasts, and trace the gain to identifying a small set of judges who consistently outperform the crowd. Concentrating weight on a minority improves the aggregate when the weight is earned on scored accuracy on the same kind of question.

groundsSource document

We propose a new measure of contribution to assess the judges' performance relative to the group and use positive contributors to build a weighting model for aggregating forecasts. In Study 1, we analyze 1,233 judges forecasting almost 200 current events to illustrate the superiority of our model over unweighted models and models weighted by measures of absolute performance. In Study 2, we replicate our findings by using economic forecasts from the European Central Bank and show how the method can be used to identify smaller crowds of the top positive contributors. We show that the model derives its power from identifying experts who consistently outperform the crowd.