STATISTICS for DATA SCIENCE

From probability foundations to modern inference: the mathematical backbone of data science, blending rigorous theory, computational methods, and practical wisdom for the age of big data and AI.

✨ Enhanced with breakthrough interactive design ✨

σ² Variance: Foundation of uncertainty
p < 0.05 The (in)famous significance threshold
∫ₓ Integration: Heart of probability
θ̂ Estimators that learn from data

Why Statistics for Data Science?

Statistics provides the rigorous mathematical foundation for:

  • Uncertainty quantification: confidence intervals, prediction intervals, credible regions
  • Scientific inference: hypothesis testing, effect sizes, power analysis
  • Model selection: AIC, BIC, cross-validation, regularization paths
  • Causal reasoning: randomization, identification, confounding control
The Data Science Trinity: Statistics gives us the "why" and "how confident", while computing gives us the "scale" and domain knowledge gives us the "what matters."

Course Philosophy

This course emphasizes:

  • Mathematical rigor with intuition: proofs meet visualizations
  • Computational implementation: theory paired with R/Python code
  • Real-world applications: case studies from industry and research
  • Modern extensions: from classical to contemporary (deep learning, causal inference)

Statistics isn't dead—it's more relevant than ever in the AI era! 🎓

Learning Objectives

By the end of this course, you will:

  • Master probability theory and its computational applications
  • Understand parametric and non-parametric inference
  • Build and validate statistical models confidently
  • Apply Bayesian reasoning to real problems
  • Navigate the modern landscape: regularization, resampling, causal inference
  • Communicate uncertainty effectively to stakeholders
Prerequisites: Calculus (derivatives, integrals), linear algebra (matrices, eigenvalues), basic programming (R or Python). Mathematical maturity and curiosity required!

The Statistical Workflow

1️⃣ Problem Formulation

Define estimands, hypotheses, decision frameworks. What question are we really asking?

2️⃣ Data Collection & EDA

Sampling strategies, visualizations, summary statistics. Know your data before modeling!

3️⃣ Model Building

Choose distributions, specify likelihood, incorporate prior knowledge (if Bayesian).

4️⃣ Inference & Validation

Estimate parameters, test hypotheses, check assumptions, assess fit.

5️⃣ Communication

Translate results into actionable insights. Visualize uncertainty. Tell the data story.

Probability Foundations

Axioms of Probability (Kolmogorov, 1933)

  • Non-negativity: P(A) ≥ 0 for any event A
  • Normalization: P(Ω) = 1 for sample space Ω
  • Additivity: For disjoint events, P(A ∪ B) = P(A) + P(B)
From axioms to everything: All of probability theory—random variables, expectations, limit theorems—flows from these three simple rules!

Key Concepts

Conditional Probability: P(A|B) = P(A ∩ B) / P(B)

Independence: A ⊥ B iff P(A ∩ B) = P(A)·P(B)

Bayes' Theorem: P(H|D) = P(D|H)·P(H) / P(D)

Random Variables

Discrete Random Variables

PMF (Probability Mass Function): p(x) = P(X = x)

Expectation: E[X] = Σ x·p(x)

Variance: Var(X) = E[(X - μ)²] = E[X²] - (E[X])²

Continuous Random Variables

PDF (Probability Density Function): P(a ≤ X ≤ b) = ∫ₐᵇ f(x)dx

CDF (Cumulative Distribution): F(x) = P(X ≤ x)

Fundamental theorem: For continuous X, P(X = x) = 0 for any specific x. We can only talk about probabilities over intervals!

Moment Generating Functions

M_X(t) = E[e^(tX)]

Why MGFs Matter

  • Generate moments: E[X^n] = M^(n)(0)
  • Uniqueness: MGF uniquely determines distribution
  • Independence: If X ⊥ Y, then M_(X+Y)(t) = M_X(t)·M_Y(t)
  • CLT proofs: Key tool for limit theorems

MGFs are like DNA for distributions—they encode everything! 🧬

Multivariate Probability

Joint, Marginal, Conditional

Joint PMF/PDF: p(x,y) or f(x,y)

Marginal: p_X(x) = Σ_y p(x,y) or f_X(x) = ∫ f(x,y)dy

Conditional: p(y|x) = p(x,y) / p_X(x)

Covariance & Correlation

Covariance: Cov(X,Y) = E[(X-μ_X)(Y-μ_Y)]

Correlation: ρ(X,Y) = Cov(X,Y) / (σ_X·σ_Y) ∈ [-1, 1]

Correlation ≠ Causation! But understanding their relationship (and independence) is crucial for causal inference later.

Limit Theorems

Law of Large Numbers (LLN)

As n → ∞, sample mean X̄_n converges to population mean μ

Weak LLN: Convergence in probability

Strong LLN: Almost sure convergence

Central Limit Theorem (CLT)

√n(X̄_n - μ) / σ → N(0,1) as n → ∞

This is why the normal distribution appears everywhere—it's the limiting distribution of sums!

Practical magic: CLT justifies using normal approximations for inference, even when underlying data isn't normal. Rule of thumb: n ≥ 30 often suffices.

Discrete Distributions

Bernoulli(p)

Single trial: success (1) with probability p, failure (0) with probability 1-p

E[X] = p, Var(X) = p(1-p)

Binomial(n, p)

Number of successes in n independent Bernoulli(p) trials

P(X = k) = C(n,k) p^k (1-p)^(n-k)

E[X] = np, Var(X) = np(1-p)

Poisson(λ)

Count of events in fixed interval, rate λ per interval

P(X = k) = e^(-λ) λ^k / k!

E[X] = Var(X) = λ

Poisson approximation: When n large and p small, Binomial(n,p) ≈ Poisson(np)

Continuous Distributions

Uniform(a, b)

Equal probability over [a,b]

f(x) = 1/(b-a) for x ∈ [a,b]

E[X] = (a+b)/2, Var(X) = (b-a)²/12

Exponential(λ)

Time until first event (memoryless property)

f(x) = λe^(-λx) for x ≥ 0

E[X] = 1/λ, Var(X) = 1/λ²

Normal(μ, σ²)

f(x) = (1/√(2πσ²)) exp(-(x-μ)²/(2σ²))

The bell curve! Symmetric, unimodal, defined by mean and variance.

68-95-99.7 rule: ~68% within 1σ, ~95% within 2σ, ~99.7% within 3σ

Sampling Distributions

Chi-Square (χ²) Distribution

Sum of k squared independent N(0,1) variables

Used for: variance estimation, goodness-of-fit tests

t-Distribution

Ratio: N(0,1) / √(χ²_ν/ν), heavier tails than normal

Used for: small-sample inference when σ unknown

F-Distribution

Ratio of two chi-square variables (scaled by df)

Used for: comparing variances, ANOVA, regression F-tests

Connection: As sample size grows, t → N(0,1) and F → χ²/df

Multivariate Normal

MVN(μ, Σ)

Vector X ∈ ℝ^p follows multivariate normal with mean vector μ and covariance matrix Σ

f(x) = (2π)^(-p/2) |Σ|^(-1/2) exp(-½(x-μ)ᵀΣ⁻¹(x-μ))

Key Properties

  • Linear transformations: if X ~ MVN(μ, Σ), then AX + b ~ MVN(Aμ + b, AΣAᵀ)
  • Marginals are normal: X_i ~ N(μ_i, σ_i²)
  • Conditionals are normal: X₁|X₂ ~ MVN(μ₁|₂, Σ₁|₂)
  • Zero correlation ⇔ Independence (special to MVN!)

MVN is the workhorse of multivariate statistics! 📊

Point Estimation

Method of Moments (MoM)

Match sample moments to population moments

Example: For Normal, μ̂ = X̄, σ̂² = S²

Maximum Likelihood Estimation (MLE)

Find θ that maximizes the likelihood L(θ|data) = ∏ f(x_i|θ)

In practice, maximize log-likelihood: ℓ(θ) = Σ log f(x_i|θ)

Why MLE? Under regularity conditions, MLEs are:
  • Consistent: θ̂_n → θ as n → ∞
  • Asymptotically normal: √n(θ̂ - θ) → N(0, I(θ)⁻¹)
  • Efficient: achieves Cramér-Rao lower bound

Properties of Estimators

Bias: E[θ̂] - θ (unbiased if = 0)

MSE: E[(θ̂ - θ)²] = Var(θ̂) + Bias²

Consistency: θ̂_n → θ in probability as n → ∞

Interval Estimation

Confidence Intervals

A (1-α)100% CI is an interval [L, U] such that P(θ ∈ [L, U]) = 1-α

Interpretation (tricky!):

❌ "There's a 95% chance θ is in this specific interval"

✅ "If we repeat this procedure, 95% of intervals will contain θ"

Construction Methods

Pivotal quantity: Find Q(X,θ) with known distribution

Asymptotic normal: θ̂ ± z_{α/2}·SE(θ̂)

Bootstrap: Resample to estimate sampling distribution

Trade-offs: Wider intervals → higher confidence but less precision. Balance depends on decision context!

Sufficiency & Completeness

Sufficient Statistics

T(X) is sufficient for θ if P(X|T(X), θ) doesn't depend on θ

Factorization theorem: f(x|θ) = g(T(x)|θ)·h(x)

Minimal Sufficient

The most compressed sufficient statistic—no information loss!

Rao-Blackwell Theorem

Given unbiased estimator θ̃ and sufficient T, conditioning gives:

θ̂ = E[θ̃|T]

which has lower variance than θ̃

Complete + sufficient → UMVUE (Lehmann-Scheffé): Uniformly Minimum Variance Unbiased Estimator—the best unbiased estimator!

Delta Method & Bootstrap

Delta Method

For smooth function g and asymptotically normal θ̂:

√n(g(θ̂) - g(θ)) → N(0, [g'(θ)]²·Var(θ̂))

Used for: transformations, ratios, log-odds, etc.

Bootstrap (Efron, 1979)

Algorithm:

  1. Resample data with replacement B times
  2. Compute θ̂* for each bootstrap sample
  3. Use empirical distribution of θ̂* for inference
Bootstrap confidence intervals:
  • Percentile: Use quantiles of θ̂*
  • BCa: Bias-corrected and accelerated (better coverage)

Hypothesis Testing Framework

Null vs. Alternative

H₀: Null hypothesis (status quo, no effect)

H₁: Alternative hypothesis (what we want to show)

Types of Errors

Type I (α): Reject H₀ when true (false positive)

Type II (β): Fail to reject H₀ when false (false negative)

Power: 1 - β (probability of correctly rejecting H₀)

The α = 0.05 controversy: Fisher suggested 0.05 as a convenient threshold, not a sacred rule! Consider context, costs of errors, and effect sizes.

p-values

Probability of observing data as extreme (or more) than what we got, assuming H₀ is true

NOT: P(H₀ is true | data)

NOT: The probability we made a mistake

Classical Tests

z-test (known σ)

Test statistic: z = (X̄ - μ₀) / (σ/√n) ~ N(0,1) under H₀

t-test (unknown σ)

Test statistic: t = (X̄ - μ₀) / (S/√n) ~ t_{n-1} under H₀

Two-sample: Compare means of two groups

Paired: Test differences for matched pairs

Chi-square test

Goodness of fit: Does data follow specified distribution?

Independence: Are two categorical variables independent?

Assumptions matter! Check normality, equal variances, independence before applying parametric tests. Use non-parametric alternatives when violated.

Multiple Testing

The Problem

With m tests at α = 0.05, expected false discoveries = 0.05m

Example: 100 tests → ~5 false positives even if all nulls true!

Correction Methods

Bonferroni: Use α/m for each test (very conservative)

Holm-Bonferroni: Sequential Bonferroni (less conservative)

FDR (Benjamini-Hochberg): Control false discovery rate

Modern approach: Report effect sizes, confidence intervals, and adjust for multiplicity. Pre-registration helps prevent p-hacking!

Power Analysis

The Four Quantities

  • Sample size (n): How many observations?
  • Effect size (δ): How large is the true difference?
  • Significance level (α): Type I error rate
  • Power (1-β): Probability of detecting effect

Given any three, we can solve for the fourth!

Practical Use

A priori: Determine n needed for desired power

Post hoc: Compute power achieved (but be careful—see "power pose problem")

Rule of thumb: Aim for 80% power. Small studies are risky— underpowered to detect real effects, yet susceptible to exaggerated estimates when significant!

Beyond p-values

Effect Sizes

Cohen's d: (μ₁ - μ₂) / σ_pooled

Odds ratio, risk ratio: For binary outcomes

R² or explained variance: For regression

Confidence Intervals

Provide range of plausible values—more informative than binary reject/fail to reject!

Equivalence Testing

Test if effect is within practically negligible range (TOST procedure)

ASA Statement (2016): "p-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone." Use multiple lines of evidence!

Simple Linear Regression

Model

Y_i = β₀ + β₁X_i + ε_i, where ε_i ~ N(0, σ²)

Least Squares Estimation

Minimize Σ(Y_i - β₀ - β₁X_i)²

Solutions:

β̂₁ = Cov(X,Y) / Var(X) = Σ(X_i - X̄)(Y_i - Ȳ) / Σ(X_i - X̄)²
β̂₀ = Ȳ - β̂₁X̄

Properties

  • Unbiased: E[β̂₁] = β₁, E[β̂₀] = β₀
  • Gauss-Markov: BLUE (Best Linear Unbiased Estimator)
  • Distribution: β̂₁ ~ N(β₁, σ²/Σ(X_i - X̄)²)
R² interpretation: Proportion of variance in Y explained by X. But beware—high R² doesn't imply causation or good predictions out-of-sample!

Multiple Linear Regression

Matrix Form

Y = Xβ + ε, where ε ~ MVN(0, σ²I)

Least Squares Solution

β̂ = (XᵀX)⁻¹XᵀY

Inference

Coefficient CIs: β̂_j ± t_{α/2, n-p}·SE(β̂_j)

F-test: Test if all coefficients jointly significant

Partial F-test: Test subset of coefficients

Diagnostics

  • Residual plots: Check for patterns, heteroscedasticity
  • QQ-plot: Assess normality of residuals
  • Leverage & influence: Hat values, Cook's distance
  • Multicollinearity: VIF (Variance Inflation Factor)
Regression assumptions: Linearity, independence, homoscedasticity, normality (LINE). Violations? Consider transformations or robust methods!

Model Selection

Information Criteria

AIC: -2ℓ + 2p (penalizes complexity)

BIC: -2ℓ + p·log(n) (stronger penalty)

Lower is better. Compare non-nested models!

Stepwise Selection

Forward: Start empty, add most significant

Backward: Start full, remove least significant

Caution: Inflates Type I error, unstable. Use with care!

Cross-Validation

Split data into train/test sets. Evaluate prediction error on held-out data.

k-fold CV: Average over k splits for robust estimate

Best practice: Pre-specify model or use regularization (ridge/lasso) instead of stepwise. Report out-of-sample performance!

Regularization

Ridge Regression (L2 penalty)

min ||Y - Xβ||² + λΣβ_j²

Shrinks coefficients toward zero. Good when many predictors, multicollinearity.

Lasso (L1 penalty)

min ||Y - Xβ||² + λΣ|β_j|

Performs variable selection—some β̂_j exactly zero!

Elastic Net

Combines L1 and L2: α·L1 + (1-α)·L2

Balances ridge's shrinkage with lasso's sparsity.

Choosing λ (tuning parameter)

Use cross-validation! Plot CV error vs. λ, pick minimum or "one-SE rule".

High-dimensional regime: When p > n, regularization is essential. Lasso provides sparse, interpretable solutions.

Generalized Linear Models

GLM Framework

Components:

  • Random: Y ~ exponential family (normal, binomial, Poisson, etc.)
  • Systematic: η = Xβ (linear predictor)
  • Link: g(μ) = η (connects mean to predictors)

Common GLMs

Logistic regression: Binary Y, logit link

Poisson regression: Count Y, log link

Gamma regression: Positive continuous Y, log link

Estimation & Inference

Fit via maximum likelihood (IRLS algorithm). Use Wald or likelihood ratio tests.

Interpretation: In logistic regression, exp(β_j) is the odds ratio for one-unit increase in X_j. Always report with confidence intervals!

Bayesian Philosophy

The Bayesian Paradigm

Parameters are random variables with distributions representing uncertainty!

Bayes' Theorem

P(θ|data) = P(data|θ)·P(θ) / P(data)
Posterior ∝ Likelihood × Prior

Prior Choices

  • Informative: Incorporate domain knowledge
  • Weakly informative: Gentle regularization
  • Non-informative/objective: Jeffreys, reference priors
Subjectivity debate: Critics say priors are subjective. Bayesians reply: all inference involves assumptions—at least we make ours explicit!

Conjugate Priors

Definition

If posterior is same family as prior, the prior is conjugate.

Common Pairs

Beta-Binomial: Beta(α,β) prior + Binomial → Beta posterior

Gamma-Poisson: Gamma(α,β) prior + Poisson → Gamma posterior

Normal-Normal: Normal prior + Normal data (known σ²) → Normal posterior

Example: Beta-Binomial

Prior: θ ~ Beta(α, β)

Data: X ~ Binomial(n, θ)

Posterior: θ|X ~ Beta(α + X, β + n - X)

Interpretation: α, β are "prior successes/failures." Posterior = prior + data. As n grows, data dominates!

MCMC Methods

Why MCMC?

For complex models, posterior is intractable. Solution: sample from it!

Metropolis-Hastings

  1. Propose θ* from proposal distribution q(θ*|θ_t)
  2. Compute acceptance ratio r = [P(θ*|data)·q(θ_t|θ*)] / [P(θ_t|data)·q(θ*|θ_t)]
  3. Accept θ* with probability min(r, 1)

Gibbs Sampling

Sample each parameter conditionally on others. Special case of MH with acceptance = 1.

Diagnostics

  • Trace plots: Visual check for convergence
  • R̂ (Gelman-Rubin): Compare within/between chain variance
  • Effective sample size: Account for autocorrelation
Modern tools: Stan (HMC), PyMC, JAGS make MCMC accessible. But always check convergence—garbage in, garbage out!

Bayesian Inference

Point Estimates

Posterior mean: E[θ|data]

Posterior median: 50th percentile

MAP: Mode of posterior (like MLE but with prior)

Interval Estimates

Credible interval: P(θ ∈ [L, U] | data) = 0.95

Interpretation is more intuitive than frequentist CI!

Hypothesis Testing

Bayes factor: BF = P(data|H₁) / P(data|H₀)

Quantifies evidence for H₁ vs. H₀. BF > 10 is "strong evidence."

Predictive distribution: P(Y_new|data) = ∫ P(Y_new|θ)·P(θ|data)dθ
Averages over posterior uncertainty—gold standard for forecasting!

Hierarchical Models

Motivation

Data has natural grouping (students in schools, patients in hospitals). Hierarchical models partially pool information!

Structure

Level 1: Y_ij ~ f(θ_j) (observations within group j)

Level 2: θ_j ~ g(φ) (group parameters from population)

Level 3: φ ~ h(...) (hyperpriors)

Shrinkage

Estimates for small groups "borrow strength" from population. Avoids overfitting in sparse groups!

Applications everywhere: Meta-analysis, longitudinal data, spatial models, mixed-effects regression. Hierarchical = flexible!

Bootstrap Methods

The Basic Bootstrap

  1. Resample n observations with replacement from original data
  2. Compute statistic of interest θ̂* on bootstrap sample
  3. Repeat B times (typically B = 1000+)
  4. Use empirical distribution of {θ̂*_b} for inference

What Bootstrap Estimates

Standard error: SD of {θ̂*_b}

Bias: mean(θ̂*) - θ̂

Confidence intervals: Percentile, BCa, studentized

Bootstrap magic: Works for almost any statistic—no math formulas needed! Median, correlation, ratios, complex model outputs—all fair game.

Bootstrap Variants

Parametric Bootstrap

Assume model f(·|θ̂), generate data from it. Useful when model is trusted.

Smoothed Bootstrap

Add small noise to resampled data (kernel smoothing). For continuous data.

Block Bootstrap

For time series or spatial data—resample blocks to preserve dependence structure.

Wild Bootstrap

For heteroscedastic regression—resample residuals with random signs.

When bootstrap fails: Heavy tails, extreme values, small n relative to parameter dimension. Always check diagnostics!

Permutation Tests

Idea

Under H₀ (no difference between groups), group labels are interchangeable. Permute labels, compute test statistic, repeat!

Algorithm

  1. Compute observed test statistic T_obs
  2. Randomly permute group labels B times
  3. Compute T*_b for each permutation
  4. p-value = proportion of T*_b ≥ T_obs

Advantages

  • Exact (for small n, enumerate all permutations)
  • No distributional assumptions
  • Works for any test statistic
Randomization inference: Permutation tests are the gold standard for experiments. If random assignment, permutation p-values are exact!

Cross-Validation

k-Fold CV

  1. Split data into k equal folds
  2. For each fold i: train on other k-1 folds, test on fold i
  3. Average test error over k folds

Common: k = 5 or 10. Larger k → less bias but more variance.

Leave-One-Out CV (LOOCV)

Special case: k = n. Unbiased but high variance. Computationally expensive.

Stratified CV

For classification: ensure class proportions same in each fold. Reduces variance!

Nested CV: Outer loop for model evaluation, inner loop for hyperparameter tuning. Avoids optimistic bias!

Causal Inference

Potential Outcomes Framework

Y_i(1) = outcome if unit i treated

Y_i(0) = outcome if unit i control

Causal effect: τ_i = Y_i(1) - Y_i(0)

Problem: We only observe one potential outcome!

Average Treatment Effect (ATE)

ATE = E[Y_i(1) - Y_i(0)]

Identification Strategies

Randomization: Gold standard—treatment independent of potential outcomes

Conditional independence: Given covariates X, treatment as-if random

Instrumental variables: Find Z that affects D but not Y directly

Selection bias: Treated/control differ systematically. Solutions: matching, propensity scores, regression adjustment, difference-in-differences.

High-Dimensional Statistics

The Challenge

Modern data: p >> n (more predictors than observations). Classical theory breaks!

Sparse Models

Assume only s << p predictors truly matter. Lasso, elastic net recover them.

Key Concepts

Restricted eigenvalue: Ensures lasso consistency

Oracle inequality: Lasso prediction error close to best sparse model

Debiasing: Post-lasso inference (double ML, debiased lasso)

Statistical vs. computational trade-offs: Some estimators are statistically optimal but NP-hard. Tractable alternatives (lasso) work well in practice!

Experimental Design

Randomization

Simple, blocked, stratified, cluster randomization. Ensures balance on average.

Design Types

Completely randomized: Simple lottery

Factorial: Multiple treatments, study interactions

Crossover: Each unit gets all treatments (over time)

A/B Testing (Online Experiments)

  • Large n, easy randomization
  • Sequential testing: monitor continuously, stop when significant
  • Multi-armed bandits: balance exploration/exploitation
Ethical experiments: Equipoise, informed consent, minimize harm. For some questions, experiments aren't possible—observational causal inference required.

Survival Analysis

Time-to-Event Data

Study duration until event (death, failure, churn). Key: censoring!

Survival Function

S(t) = P(T > t) = probability of surviving past time t

Hazard Function

h(t) = lim P(t ≤ T < t+Δt | T ≥ t) / Δt

Instantaneous failure rate at time t.

Models

Kaplan-Meier: Non-parametric survival curve estimator

Cox proportional hazards: h(t|X) = h₀(t)·exp(Xβ)

Applications: Medical trials, customer churn, equipment failure, loan defaults. Essential tool for time-to-event outcomes!

Spatial Statistics

Spatial Dependence

"Near things are more related than distant things" (Tobler's First Law)

Types of Spatial Data

Geostatistical: Continuous field (temperature, pollution)

Lattice: Discrete regions (counties, census tracts)

Point patterns: Locations of events (disease cases, trees)

Models

Variogram: Measure spatial correlation structure

Kriging: Optimal spatial prediction (Gaussian process)

CAR/SAR: Conditional/simultaneous autoregressive models

Ignoring spatial correlation: Standard errors too small, inflated significance. Always account for spatial structure in inference!

Reproducible Research

The Reproducibility Crisis

Many published findings don't replicate. Part of the problem: poor statistical practice!

Best Practices

  • Pre-registration: Specify hypotheses, analysis plan before seeing data
  • Open data/code: Share materials for verification
  • Version control: Git for tracking changes
  • Literate programming: R Markdown, Jupyter—code + narrative
  • Seeds: Set random seeds for replicability
The garden of forking paths: Researcher degrees of freedom can inflate false positives. Document all decisions, report all analyses!

Communicating Uncertainty

Visualizations

Error bars: Show CIs, not just point estimates

Prediction intervals: Distinguish from confidence intervals

Distribution plots: Violin, density, quantile plots reveal more than bar charts

Language

❌ "The effect is significant" → ✅ "We estimate a 15% increase (95% CI: 8-22%)"

❌ "No effect found" → ✅ "Data consistent with effects from -5% to +2%"

For non-technical audiences: Use analogies, avoid jargon. "Margin of error" more intuitive than "confidence interval" for many!

Model Validation

In-Sample vs. Out-of-Sample

Model fits training data perfectly but fails on new data → overfitting!

Validation Strategies

  • Train/test split: Simple, but variance depends on split
  • Cross-validation: More stable, use for model selection
  • External validation: Best—test on truly independent data

Metrics

Regression: RMSE, MAE, R²

Classification: Accuracy, precision/recall, AUC, calibration

Overfitting indicators: Training error << test error, or model very complex relative to n. Regularize or get more data!

Collaboration & Ethics

Working with Domain Experts

  • Understand the problem context before modeling
  • Co-design analysis plan with domain input
  • Explain methods and limitations transparently
  • Iterate based on feedback

Ethical Considerations

Fairness: Check for disparate impact across groups

Privacy: Anonymization, differential privacy

Transparency: Explain how models work, limitations

Bias: Training data bias → model bias. Audit inputs!

With great power…: Statistical models inform high-stakes decisions. Be honest about uncertainty, acknowledge limitations, advocate for responsible use.

Continuing Education

Stay Current

Statistics evolves rapidly! New methods, software, applications appear constantly.

Resources

  • Journals: JASA, Annals of Statistics, Biometrika, JRSS
  • Conferences: JSM, useR, PyData, specialized workshops
  • Online: arXiv stat, CrossValidated, blogs (Andrew Gelman, etc.)
  • Books: ESL, ISL, BDA, Causal Inference: The Mixtape

Practice

Kaggle, Tidy Tuesday, replicate papers, contribute to open-source projects!

Statistics for data science is a lifelong journey—enjoy the ride! 🚀