Authors: Rose R. Chamberlain, Matthew W. Ross, Calvin C. Long, and Miodrag Lovric (Radford University, USA)
This comprehensive article critically examines fourteen prevalent misconceptions about p-values, a cornerstone of statistical hypothesis testing. Despite their widespread use, p-values remain one of the most misunderstood concepts in scientific research. The authors offer a refined, conditional definition of the p-value, which accounts for sample size and model assumptions: "The p-value represents the probability of observing a test statistic at least as extreme as the observed value, assuming that the null hypothesis is true, and that all assumptions of the statistical model are met, for the given sample size."
The article explains how p-values are calculated and visualized in practice through simulations and histograms. A notable demonstration compares "nominal" p-values, derived under normality assumptions, with "empirical" p-values based on the actual distribution (e.g., exponential), revealing how violations of model assumptions can produce dramatically misleading results.
Among the misconceptions addressed are the beliefs that:
The authors also rebut common criticisms of p-values, including those by Jeffreys, Berger, and Sellke, arguing that such critiques often rely on problematic prior assumptions. They emphasize that p-values should not be discarded, but rather understood and applied correctly. When interpreted with consideration for model assumptions, sample size, effect size, and statistical power, p-values remain a valuable inferential tool.
The paper culminates with a call for thoughtful reform—not a demolition—of "p-value culture," echoing recent statements by the American Statistical Association. It advocates for more holistic practices that integrate confidence intervals, effect sizes, and contextual relevance.
For all fourteen misconceptions, simulation results, revised definitions, and philosophical commentary, see the full article in the International Encyclopedia of Statistical Science.