Permutation Tests and Bootstrapping

Never worry about a normal distribution again!

• See our free webinar "Never Worry About a Normal Distribution Again" on our Online Training Center
• This material is now fully integrated into chapters 4 to 7 of Statistical Methods in Water Resources (2020), available as a free pdf textbook.

One of the hardest things in environmental statistics has been to test differences in group means, and compute useful confidence intervals, when data are strongly skewed. Traditional t-tests and analysis of variance look for differences in means, but have low power when data are strongly skewed. Transformation to logarithms often addresses skewness, but the resulting tests on logs evaluate differences in geometric means, not means. Rank-based nonparametric tests look for differences in medians. What’s a scientist to do?

Permutation alternatives to t-tests and analysis of variance, and bootstrap confidence intervals as an alternative to t-intervals, allow inferences about the mean and confidence intervals around the mean to be made without assuming normality and without transforming variables.

What you will learn about permutation tests in Statistical Methods in Water Resources:

• What are the advantages of permutation tests over parametric tests?
• How do permutation tests work?
• Power of parametric vs nonparametric vs permutation tests
• What is bootstrapping and how does it work?
• Computing bootstrapped confidence intervals
• Permutation tests for paired data
• Permutation tests for two groups (t-test situation)
• Permutation tests for 3+ groups (ANOVA situation)


Online at: