A Fundamental Challenge to Bayesian Hypothesis Testing
Same Data, Same Hypothesis, OPPOSITE Conclusions
In 1957, Dennis Lindley published "A Statistical Paradox" in Biometrika, revealing a conflict between frequentist and Bayesian inference as sample size approaches infinity.
The BFR Paradox is arguably more fundamental: it reveals a conflict of Bayesian inference with itself.
For any statistically significant result at the 0.05 level, two Bayesians using different (but equally reasonable) prior scales can reach opposite conclusions from the same data.
BF₀₁ = 0.83
BF₀₁ = 1.47
Example: z = 2.0, n = 50, p-value = 0.046 (statistically significant!)
Flip point: k* = 49.44, τ* = 0.99
The paradox is most dramatic with large samples and marginally significant results.
Effect size: d = 0.028 (tiny but "statistically significant")
Flip point: τ* = 0.09
| Prior Type | τ | BF₀₁ | Interpretation |
|---|---|---|---|
| Ultra-Skeptical | 0.05 | 0.62 | Weak for H₁ |
| JZS Default | 0.707 | 7.33 | Moderate for H₀ |
| Diffuse | 1.0 | 10.36 | Strong for H₀ |
| Very Diffuse | 2.0 | 20.72 | Strong for H₀ |
33-fold swing in the Bayes factor!
The Bayes factor in favour of H₀ for the normal model with known variance:
where \(k = n\tau^2\) and \(\tau\) is the prior standard deviation on the effect size.
For \(|z| > 1\), there exists a unique flip point \(k^{*}\) satisfying:
At this point, BF₀₁ = 1 (neutral). Below it, evidence favours H₁. Above it, evidence favours H₀.
Since every statistically significant result at α = 0.05 has \(|z| > 1.96 > 1\), every statistically significant result exhibits the paradox.
| z | z² | p-value | k* | τ* (n=50) | τ* (n=100) |
|---|---|---|---|---|---|
| 1.50 | 2.25 | 0.134 | 5.82 | 0.34 | 0.24 |
| 1.96 | 3.84 | 0.050 | 41.58 | 0.91 | 0.64 |
| 2.00 | 4.00 | 0.046 | 49.44 | 0.99 | 0.70 |
| 2.50 | 6.25 | 0.012 | 510.72 | 3.20 | 2.26 |
| 3.00 | 9.00 | 0.003 | 8093.08 | 12.72 | 9.00 |
Explore the paradox yourself! Adjust the prior scale and watch the Bayes factor flip from supporting H₁ to supporting H₀.
Complete R code for computing flip points and verifying all theoretical results.
BF01 <- function(z, k) {
# Bayes factor in favour of H0
# z: z-statistic
# k: n * tau^2 (prior variance scaled by sample size)
sqrt(1 + k) * exp(-z^2 * k / (2 * (1 + k)))
}
find_flip_point <- function(z, tol = 1e-10) {
# Returns k* such that BF01(z, k*) = 1
# Only exists for |z| > 1
if (abs(z) <= 1) return(NA)
f <- function(k) BF01(z, k) - 1
k_min <- z^2 - 1
k_upper <- k_min + 1
while (BF01(z, k_upper) < 1) {
k_upper <- k_upper * 2
}
uniroot(f, c(k_min + tol, k_upper), tol = tol)$root
}
demonstrate_paradox <- function(z = 2.0, n = 50) {
k_star <- find_flip_point(z)
tau_star <- sqrt(k_star / n)
cat(sprintf("Flip point: k* = %.2f, tau* = %.2f\n\n", k_star, tau_star))
# Priors on OPPOSITE sides of flip point
tau1 <- 0.8 # Below flip (k = 32 < 49.44)
tau2 <- 1.5 # Above flip (k = 112.5 > 49.44)
BF1 <- BF01(z, n * tau1^2)
BF2 <- BF01(z, n * tau2^2)
cat(sprintf("tau = %.1f: BF01 = %.2f --> %s\n",
tau1, BF1, ifelse(BF1 < 1, "Favours H1", "Favours H0")))
cat(sprintf("tau = %.1f: BF01 = %.2f --> %s\n",
tau2, BF2, ifelse(BF2 < 1, "Favours H1", "Favours H0")))
}
# Run:
demonstrate_paradox(z = 2.0, n = 50)
generate_table1 <- function() {
z_values <- c(1.50, 1.96, 2.00, 2.50, 3.00)
results <- data.frame(
z = z_values,
z_sq = z_values^2,
p_value = 2 * (1 - pnorm(z_values)),
k_star = sapply(z_values, find_flip_point),
tau_n50 = NA,
tau_n100 = NA
)
results$tau_n50 <- sqrt(results$k_star / 50)
results$tau_n100 <- sqrt(results$k_star / 100)
return(results)
}
# Run:
generate_table1()
If you use this work, please cite:
@article{lovric2025bfr,
author = {Lovric, Miodrag M.},
title = {The {B}ayes factor reversal paradox},
journal = {Biometrika},
year = {2025},
note = {Forthcoming}
}
🔄 Flip-Point Calculator: Check if YOUR data is vulnerable to the BFR paradox. Enter your test statistic and sample size to find the critical prior scale (τ*, r*, κ*) where Bayes factor conclusions reverse. Supports z-tests, t-tests, ANOVA, proportions, correlations, and regression.