Significance tests: Parametric & non-parametric tests

On this page

Statistical tests
Parametric vs. Non-Parametric Tests
1. Comparing two groups

Statistical tests

Statistical (significance) tests help determine whether observed differences in data are likely to be due to chance. These tests generate a test statistic, which is used to calculate a p-value. If the p-value is below a pre-set threshold (typically 0.05), the result is considered statistically significant.

Choosing the correct statistical test depends on several factors:

Type of data: categorical vs. numerical (quantitative)
Distribution of the data: normal (Gaussian) or non-normal
Number of groups being compared
Pairing of data: paired (related measurements from the same individuals) vs. unpaired (independent groups)

In practice, these calculations are usually performed using statistical software, but understanding the test selection is crucial for interpretation. Tests are often described as being parametric or non-parametric. Parametric tests are used when data are continuous, normally distributed, and measured on an interval or ratio scale (e.g., blood pressure, weight). Non-parametric tests are used for skewed data, ordinal data (e.g., Likert scales), or small sample sizes. Also used for categorical data in frequency comparisons.

Parametric vs. Non-Parametric Tests

Feature	Parametric Tests	Non-Parametric Tests
Assumptions	Requires data to be normally distributed and measured on an interval or ratio scale	No assumption of normality; suitable for skewed data or ordinal data
Basis of comparison	Compare means	Compare medians or ranks
Power	More powerful if assumptions are met	Less powerful, but more robust if assumptions are violated
Examples	Student's t-test - paired or unpaired* Pearson's product-moment coefficient - correlation	Mann-Whitney U test - unpaired data Wilcoxon signed-rank test - compares two sets of observations on a single sample chi-squared test - used to compare proportions or percentages Spearman, Kendall rank - correlation

*paired data refers to data obtained from a single group of patients, e.g. measurement before and after an intervention. Unpaired data comes from two different groups of patients, e.g. comparing response to different interventions in two groups

Common tests based on data type and study design

1. Comparing two groups

Data Type	Unpaired	Paired
Parametric	Unpaired Student’s t-test: compares means between two independent groups	Paired t-test: compares means before and after intervention in the same group
Non-parametric	Mann–Whitney U test: compares medians of two independent groups	Wilcoxon signed-rank test: compares medians of paired observations

2. Comparing more than two groups

Data Type	Unpaired	Paired
Parametric	One-way ANOVA: compares means of three or more independent groups	Repeated Measures ANOVA: compares means across multiple time points or conditions in the same group
Non-parametric	Kruskal–Wallis test: compares medians of three or more independent groups	Friedman test: compares medians across multiple paired conditions

3. Tests for categorical data
All of the tests below are non-parametric tests

Test	Use Case
Chi-squared (χ²) test	Compares proportions between two or more independent groups (e.g., stroke rates in patients on vs. off statins)
Fisher’s Exact Test	Used instead of Chi-squared when sample sizes are small (expected frequency < 5 in any cell)
McNemar’s Test	Compares proportions in paired categorical data (e.g., before and after a treatment within the same subjects)

Key Definitions:

Paired data: Repeated measures on the same subjects (e.g., pre- and post-treatment)
Unpaired data: Data from different groups of individuals
Normal distribution: Symmetrical, bell-shaped distribution; required for parametric tests
Ordinal data: Categorical data with a meaningful order but not equal spacing (e.g., MMSE scores)