Noise: relative judgments against shared anchor cases are less noisy than absolute 0-to-100 ratings.

The claim

On performance ratings, Kahneman, Sibony and Sunstein find that "Because comparative judgments are less susceptible to noise than ratings are, Case scales are more reliable than scales that use numbers, adjectives, or behavioral descriptions." Our labelling asks agents for absolute 0-to-100 ratings of likelihood and importance, which the book predicts will be noisy. A cheaper, less noisy design is a case scale: a handful of anchor claims everyone knows, and each new rating stated as a comparison with them. Inference beyond the quotes is Leo's.