๐Ÿ” What is the BFR Paradox?

In 1957, Dennis Lindley published "A Statistical Paradox" in Biometrika, showing that frequentist and Bayesian methods can reach opposite conclusions as sample size approaches infinity. This became known as the Jeffreys-Lindley Paradox.

The BFR (Bayes Factor Reversal) Paradox reveals something arguably more devastating:

๐ŸŽฏ The Core Discovery

For any statistically significant result (p < 0.05), two Bayesians using different "reasonable" prior scales will reach opposite conclusions โ€” one saying "reject Hโ‚€" while the other says "accept Hโ‚€."

Unlike the Jeffreys-Lindley paradox (which requires n โ†’ โˆž), the BFR Paradox occurs with realistic sample sizes (n = 20, 50, 100) and shows that Bayesians cannot agree among themselves.

๐Ÿ“œ Robert's 2016 Conjecture โ€” Now Proven!

In 2016, Christian Robert โ€” one of the world's leading Bayesian statisticians โ€” published "The expected demise of the Bayes factor" in the Journal of Mathematical Psychology.

"this choice only proceeds from an imperative of fat tails in the prior, without in the least solving the calibration of the Cauchy scale, which has no particular reason to be set to 1. The choice thus involves arbitrariness to a rather high degree."

โ€” Christian Robert (2016, p. 34)

Robert then explicitly called for research:

"(Given the now-available modern computing tools, it would be nice to see the impact of this scale ฮณ on the numerical value of the Bayes factor.)"

โ€” Christian Robert (2016, p. 34)

โœ… Our Answer to Robert

The impact of the scale ฮณ is catastrophic: it can reverse the scientific conclusion from "reject Hโ‚€" to "accept Hโ‚€"!

๐Ÿงช Interactive Demonstration

This calculator lets you explore the BFR Paradox. You specify the test statistic z and sample size n, then adjust the prior scale ฯ„ to see how the Bayes factor changes โ€” and how it can flip from favouring Hโ‚ to favouring Hโ‚€.

๐Ÿ“˜ Key Concepts

We test Hโ‚€: ฮผ = 0 versus Hโ‚: ฮผ โ‰  0 for normal data with known variance ฯƒยฒ = 1. Under Hโ‚, we place a normal prior on ฮผ: ฮผ ~ N(0, ฯ„ยฒ).

  • z = test statistic (z = xฬ„โˆšn when ฯƒ = 1)
  • n = sample size
  • ฯ„ = prior standard deviation of ฮผ under Hโ‚
  • k = nฯ„ยฒ = key parameter combining sample size and prior variance
  • k* = the flip point โ€” the value of k where BFโ‚€โ‚ = 1 (neutral)
  • ฯ„* = โˆš(k*/n) = critical prior scale for given n
The Paradox: For any |z| > 1, a flip point exists. Choosing ฯ„ below ฯ„* gives evidence for Hโ‚; choosing ฯ„ above ฯ„* gives evidence for Hโ‚€. Same data, opposite conclusions!
โšก BFR Paradox Calculator

Load a Demonstration Scenario:

Affects ฯ„* but not k*
Determines k* and p-value
= 2ร—(1 - ฮฆ(|z|))
๐Ÿ“ Flip Point โ€” Where BFโ‚€โ‚ = 1 (Neutral)
Flip point (depends only on z)
k* = 49.44
Critical prior scale (depends on z and n)
ฯ„* = 0.99
ฯ„ < ฯ„* โ†’ BFโ‚€โ‚ < 1 โ†’ Favours Hโ‚   |   ฯ„ > ฯ„* โ†’ BFโ‚€โ‚ > 1 โ†’ Favours Hโ‚€
Prior Scale (ฯ„) โ€” adjust to see the paradox! 0.50
ฯ„ = 0.01 (concentrated) โ†‘ ฯ„* = 0.99 ฯ„ = 3.0 (diffuse)

๐Ÿ“Š Frequentist Result

p-value
0.0455
p < 0.05 โ†’ SIGNIFICANT

๐ŸŽฒ Bayesian Result

Bayes Factor
BFโ‚€โ‚ = 1.00
Neutral evidence

๐Ÿ’ฅ The Paradox in Action

Below, two Bayesians analyse the same z and n you specified above. The only difference is their choice of prior scale ฯ„ โ€” one chooses a value below ฯ„*, the other chooses a value above ฯ„*. Watch them reach opposite conclusions!

Using your values: z = 2.00, n = 50, ฯ„* = 0.99

๐Ÿ”ฌ Bayesian A

ฯ„ = 0.80 (below ฯ„*)
BFโ‚€โ‚ = 0.83
Favours Hโ‚
VS

๐ŸŒŸ Bayesian B

ฯ„ = 1.50 (above ฯ„*)
BFโ‚€โ‚ = 1.47
Favours Hโ‚€

โš ๏ธ THE BFR PARADOX

Same data. Same hypothesis. Same p-value.
Bayesian A (ฯ„ below ฯ„*): "Evidence favours Hโ‚!"
Bayesian B (ฯ„ above ฯ„*): "Evidence favours Hโ‚€!"

The conclusion depends entirely on an arbitrary prior choice.

๐Ÿ“ The Mathematics

Setup

We test Hโ‚€: ฮผ = 0 vs Hโ‚: ฮผ โ‰  0 using normal data with known variance ฯƒยฒ = 1. Under Hโ‚, we use prior ฮผ ~ N(0, ฯ„ยฒ).

The Bayes Factor

$$BF_{01}(z; \tau^2) = \sqrt{1 + n\tau^2} \cdot \exp\left\{-\frac{z^2}{2} \cdot \frac{n\tau^2}{1 + n\tau^2}\right\}$$

The Flip Point Theorem

For any |z| > 1 (which includes all p < 0.317), there exists a unique flip point ฯ„* such that:

  • For ฯ„ < ฯ„*: BFโ‚€โ‚ < 1 โ†’ Evidence for Hโ‚ (reject Hโ‚€)
  • For ฯ„ = ฯ„*: BFโ‚€โ‚ = 1 โ†’ Neutral
  • For ฯ„ > ฯ„*: BFโ‚€โ‚ > 1 โ†’ Evidence for Hโ‚€ (accept Hโ‚€)
Flip Point Equation:
$$(1 + k^*)\ln(1 + k^*) = z^2 k^* \quad \text{where } k = n\tau^2$$

The Universal Result

๐ŸŽฏ Corollary: Universality

Every statistically significant result (p < 0.05) exhibits the BFR Paradox.
Since p < 0.05 implies |z| > 1.96 > 1, a flip point always exists.

๐Ÿ“Š Comparison: P-values vs. Bayes Factors

Proponents of Bayes factors argued they are "more objective" than p-values. The BFR Paradox reveals the opposite:

Property P-value Bayes Factor
Uniquely determined by data? โœ“ Yes โœ— No (depends on ฯ„)
All analysts agree on the value? โœ“ Yes โœ— No
Can reverse with "reasonable" choices? โœ“ No โœ— Yes (BFR Paradox!)
Debate is about... Interpretation only The evidence itself!

Key insight: At least with p-values, everyone agrees on the number โ€” the debate is only about interpretation. With Bayes factors, we cannot even agree on what "the evidence" IS.

๐Ÿ“œ Historical Timeline

1957
Lindley publishes "A Statistical Paradox" in Biometrika โ€” frequentist vs. Bayesian disagreement as n โ†’ โˆž
1987
Berger & Sellke argue p-values should be replaced by Bayes factors as "objective measures of evidence"
2016
Robert publishes "The expected demise of the Bayes factor" โ€” questions the arbitrary prior scale, calls for research
2020
Lovric proves Type I/II conflicts: credible intervals vs. Bayes factors can disagree
2025
BFR Paradox โ€” First proof that Bayes factors contradict themselves: BF vs. BF disagreement with realistic sample sizes. Robert's conjecture is proven!

โš–๏ธ Comparison of Statistical Paradoxes

Feature Jeffreys-Lindley (1957) Type I/II Conflicts (2020) BFR Paradox (2025)
What varies? n โ†’ โˆž โ€” Prior variance ฯ„ยฒ
What conflicts? Frequentist vs. Bayesian CI vs. BF BF vs. BF
Sample size n โ†’ โˆž (asymptotic) Finite Realistic (n = 20โ€“100)
Nature Inter-paradigm Inter-tool Intra-tool

The BFR Paradox is arguably the most devastating because it shows that a single Bayesian tool contradicts itself based on arbitrary choices.

๐Ÿ’ก Implications

1. Robert Was Right

Christian Robert's (2016) "expected demise of the Bayes factor" is now mathematically justified. The prior scale choice is not merely arbitrary โ€” it is conclusion-determining.

2. The "Objective Bayes" Program Has Failed

For decades, statisticians sought "default" or "objective" priors. The BFR Paradox shows this goal is unattainable: ANY choice of ฯ„ is arbitrary, and different reasonable choices yield opposite conclusions.

3. Bayes Factors Cannot Replace P-values

The proposal to replace p-values with Bayes factors on grounds of "greater objectivity" is untenable. If anything, Bayes factors are less objective โ€” at least everyone agrees on the p-value!

4. Practical Consequences

  • Publication decisions: Reviewers with different priors reach opposite conclusions
  • Drug approvals: FDA decisions become prior-dependent
  • Legal evidence: Forensic conclusions depend on arbitrary choices
  • Meta-analysis: Combining Bayes factors is meaningless without standardized priors

โœ… Conclusion

Sixty-eight years after Lindley's paradox revealed frequentist-Bayesian disagreement, the BFR Paradox reveals something more fundamental: Bayesians cannot agree among themselves.

For any statistically significant result, the Bayes factor can favor either Hโ‚€ or Hโ‚ depending on an arbitrary prior variance. This confirms Robert's (2016) suspicion and provides mathematical foundation for his "expected demise of the Bayes factor."

๐ŸŽฏ The Final Verdict

The proposal to replace p-values with Bayes factors on grounds of greater objectivity is untenable.

๐Ÿ“š References

  • Berger, J. O. & Sellke, T. (1987). Testing a point null hypothesis: The irreconcilability of p values and evidence. JASA, 82, 112โ€“122.
  • Lindley, D. V. (1957). A statistical paradox. Biometrika, 44, 187โ€“192.
  • Lovric, M. M. (2020). Conflicts in Bayesian statistics between inference based on credible intervals and Bayes factors. JMASM, 18, eP3320.
  • Robert, C. P. (2016). The expected demise of the Bayes factor. Journal of Mathematical Psychology, 72, 33โ€“37.