https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3114207/
“Extraordinary claims require extraordinary evidence” was a phrase made popular by Carl Sagan who reworded Laplace’s principle, which says that “the weight of evidence for an extraordinary claim must be proportioned to its strangeness” (Gillispie et al., 1999). This statement is at the heart of the scientific method, and a model for critical thinking, rational thought and skepticism everywhere. However, no quantitative standards have been agreed upon in order to define whether or not extraordinary evidence has been obtained. Consequently, the measures of “extraordinary evidence” are completely reliant on subjective evaluation and the acceptance of “extraordinary claims.” In science, the definition of extraordinary evidence is more a social agreement than an objective evaluation, even if most scientists would state the contrary (see, for example, the recent debate about climate change: Anderegg et al., 2010; Bodenstein, 2010).
However, a relevant example of an agreement about the strength of evidence has been defined in the field of clinical medicine and psychology in order to grade evidence to recommend the application of treatments for physical and mental clinical conditions. Recommendations that are based on evidence can be of different levels of quality. The sources of evidence, range from small laboratory studies or case reports to large, well-designed clinical studies that have minimized bias to a large extent. As poor-quality evidence can lead to recommendations that are not in the patient’s best interests, it is essential to know whether a recommendation is strong (i.e., we can be confident about the recommendation) or weak (we cannot be confident). The Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group (Guyat et al., 2008), for example, states that strong recommendations, meaning that most patients who are provided with the information would choose the recommended management and that clinicians can structure their interactions with patients accordingly, must derive from consistent evidence from a comprehensive meta-analysis of all of the evidence available or from at least two well-performed, randomized and controlled trials. If an agreement can be obtained in such an important field as that of human physical and mental health, we think that it should be possible to reach an agreement in the field of “human knowledge,” where there are fewer risks of harming people.