Noise: guidelines cut noise when they decompose judgment into checkable criteria, and stop cutting it where criteria stay open-ended.

The claim

Kahneman, Sibony and Sunstein report that "In medicine in general, guidelines have been highly successful in reducing both bias and noise", but that they fail where the criteria cannot be made concrete, as in psychiatry: "Some guidelines reduce noise by decomposing judgment into criteria on which disagreement is reduced, but to the extent that such criteria are relatively open-ended, noise remains likely." Applied to our own gates: checks such as "does the excerpt locate in the source" behave like the good guidelines; judgments such as a claim's importance remain open-ended and should be expected to be noisy until decomposed or anchored. Inference beyond the quotes is Leo's.