⚠️

The Bayes Factor Reversal Paradox

A Fundamental Challenge to Bayesian Hypothesis Testing

Same Data, Same Hypothesis, OPPOSITE Conclusions

Miodrag M. Lovric
Department of Mathematics and Statistics
Radford University
[email protected]

📖 What is the BFR Paradox?

In 1957, Dennis Lindley published "A Statistical Paradox" in Biometrika, revealing a conflict between frequentist and Bayesian inference as sample size approaches infinity.

The BFR Paradox is arguably more fundamental: it reveals a conflict of Bayesian inference with itself.

🔥 The Paradox

For any statistically significant result at the 0.05 level, two Bayesians using different (but equally reasonable) prior scales can reach opposite conclusions from the same data.

Bayesian A (τ = 0.8)

BF₀₁ = 0.83

Favours H₁

Bayesian B (τ = 1.5)

BF₀₁ = 1.47

Favours H₀

Example: z = 2.0, n = 50, p-value = 0.046 (statistically significant!)
Flip point: k* = 49.44, τ* = 0.99

🌪️ The "Perfect Storm" Scenario

The paradox is most dramatic with large samples and marginally significant results.

Clinical Trial: n = 5000, z = 1.96 (p = 0.05)

Effect size: d = 0.028 (tiny but "statistically significant")

Flip point: τ* = 0.09

Prior Type τ BF₀₁ Interpretation
Ultra-Skeptical 0.05 0.62 Weak for H₁
JZS Default 0.707 7.33 Moderate for H₀
Diffuse 1.0 10.36 Strong for H₀
Very Diffuse 2.0 20.72 Strong for H₀

33-fold swing in the Bayes factor!

📐 The Mathematics

The Bayes factor in favour of H₀ for the normal model with known variance:

$$\text{BF}_{01}(z; k) = \sqrt{1+k} \cdot \exp\left\{-\frac{z^2 k}{2(1+k)}\right\}$$

where \(k = n\tau^2\) and \(\tau\) is the prior standard deviation on the effect size.

The Flip Point Theorem

For \(|z| > 1\), there exists a unique flip point \(k^{*}\) satisfying:

$$(1+k^{*})\ln(1+k^{*}) = z^2 k^{*}$$

At this point, BF₀₁ = 1 (neutral). Below it, evidence favours H₁. Above it, evidence favours H₀.

Universality Corollary

Since every statistically significant result at α = 0.05 has \(|z| > 1.96 > 1\), every statistically significant result exhibits the paradox.

📊 Table of Flip Points

z z² p-value k* τ* (n=50) τ* (n=100)
1.50 2.25 0.134 5.82 0.34 0.24
1.96 3.84 0.050 41.58 0.91 0.64
2.00 4.00 0.046 49.44 0.99 0.70
2.50 6.25 0.012 510.72 3.20 2.26
3.00 9.00 0.003 8093.08 12.72 9.00

🎮 Interactive Demonstration

Explore the paradox yourself! Adjust the prior scale and watch the Bayes factor flip from supporting H₁ to supporting H₀.

💻 R Code

Complete R code for computing flip points and verifying all theoretical results.

🐙 GitHub Repository

Bayes Factor Function

BF01.R
BF01 <- function(z, k) {
  # Bayes factor in favour of H0
  # z: z-statistic
  # k: n * tau^2 (prior variance scaled by sample size)
  sqrt(1 + k) * exp(-z^2 * k / (2 * (1 + k)))
}

Flip Point Computation

find_flip_point.R
find_flip_point <- function(z, tol = 1e-10) {
  # Returns k* such that BF01(z, k*) = 1
  # Only exists for |z| > 1
  
  if (abs(z) <= 1) return(NA)
  
  f <- function(k) BF01(z, k) - 1
  k_min <- z^2 - 1
  
  k_upper <- k_min + 1
  while (BF01(z, k_upper) < 1) {
    k_upper <- k_upper * 2
  }
  
  uniroot(f, c(k_min + tol, k_upper), tol = tol)$root
}

Demonstrate the Paradox

demonstrate_paradox.R
demonstrate_paradox <- function(z = 2.0, n = 50) {
  k_star <- find_flip_point(z)
  tau_star <- sqrt(k_star / n)
  cat(sprintf("Flip point: k* = %.2f, tau* = %.2f\n\n", k_star, tau_star))
  
  # Priors on OPPOSITE sides of flip point
  tau1 <- 0.8   # Below flip (k = 32 < 49.44)
  tau2 <- 1.5   # Above flip (k = 112.5 > 49.44)
  
  BF1 <- BF01(z, n * tau1^2)
  BF2 <- BF01(z, n * tau2^2)
  
  cat(sprintf("tau = %.1f: BF01 = %.2f --> %s\n", 
              tau1, BF1, ifelse(BF1 < 1, "Favours H1", "Favours H0")))
  cat(sprintf("tau = %.1f: BF01 = %.2f --> %s\n", 
              tau2, BF2, ifelse(BF2 < 1, "Favours H1", "Favours H0")))
}

# Run:
demonstrate_paradox(z = 2.0, n = 50)

Generate Table 1

generate_table1.R
generate_table1 <- function() {
  z_values <- c(1.50, 1.96, 2.00, 2.50, 3.00)
  
  results <- data.frame(
    z = z_values,
    z_sq = z_values^2,
    p_value = 2 * (1 - pnorm(z_values)),
    k_star = sapply(z_values, find_flip_point),
    tau_n50 = NA,
    tau_n100 = NA
  )
  
  results$tau_n50 <- sqrt(results$k_star / 50)
  results$tau_n100 <- sqrt(results$k_star / 100)
  
  return(results)
}

# Run:
generate_table1()

📜 Historical Context

1957
Lindley's Paradox — Dennis Lindley publishes "A Statistical Paradox" in Biometrika, showing frequentist-Bayesian conflict as n → ∞.
1987
Berger & Sellke — Argue that p-values should be replaced by Bayes factors for "more objective" evidence.
2016
Robert's Conjecture — Christian Robert publishes "The Expected Demise of the Bayes Factor," noting the prior scale "involves arbitrariness to a rather high degree" and calls for research on its impact.
2025
The BFR Paradox — Answers Robert's call by proving mathematically that prior scale can reverse conclusions entirely. First statistical paradox discovered through human-AI collaboration.

📚 Citation

If you use this work, please cite:

Lovric, M. M. (2025). The Bayes factor reversal paradox. Biometrika. [Forthcoming]

BibTeX

citation.bib
@article{lovric2025bfr,
  author  = {Lovric, Miodrag M.},
  title   = {The {B}ayes factor reversal paradox},
  journal = {Biometrika},
  year    = {2025},
  note    = {Forthcoming}
}

🔗 Resources

🔄 Flip-Point Calculator: Check if YOUR data is vulnerable to the BFR paradox. Enter your test statistic and sample size to find the critical prior scale (τ*, r*, κ*) where Bayes factor conclusions reverse. Supports z-tests, t-tests, ANOVA, proportions, correlations, and regression.