35  Kruskal-Wallis Test

The Kruskal-Wallis test (William H. Kruskal & W. Allen Wallis, 1952) is a non-parametric statistical test that is used to determine if there are statistically significant differences between the medians of three or more independent groups. It is an extension of the Mann-Whitney U test and is particularly useful when the assumptions of one-way ANOVA (such as normality) cannot be met.

35.1 Assumptions

The Kruskal-Wallis test relies on the following assumptions:

  1. Independence of Samples: The groups are independent of one another.
  2. Ordinal or Continuous Data: The data within and across groups should be ordinal or continuous.
  3. Similarity of Shape: The distributions of the groups should be similar, allowing the medians to be comparable.

35.2 Hypotheses

The hypotheses for the Kruskal-Wallis test are as follows:

  • Null Hypothesis (H₀): The medians of all groups are equal.
  • Alternative Hypothesis (H₁): At least one group’s median is different from the others.

35.3 Formula

The test statistic (H) is calculated as follows: \[ H = \left(\frac{12}{n(n+1)}\right) \sum_{i=1}^k \frac{R_i^2}{n_i} - 3(n+1) \] Where:

  • \(n\) is the total number of observations.
  • \(k\) is the number of groups.
  • \(R_i\) is the sum of ranks in the \(i^{th}\) group.
  • \(n_i\) is the number of observations in the \(i^{th}\) group.

35.3.1 Calculation Steps

  1. Rank all data from all groups together; the lowest value gets rank 1, the next lowest rank 2, and so on.
  2. Calculate the sum of ranks for each group.
  3. Use the formula to calculate the H statistic.

35.3.2 Interpretation

A large value of H indicates a rejection of the null hypothesis. This value is compared against a chi-square distribution with \(k-1\) degrees of freedom. If the calculated H is greater than the critical value from the chi-square table at the desired level of significance, the null hypothesis is rejected.

35.4 Example Problem

Let’s consider an example where a researcher wants to compare the effectiveness of four different medications. The response scores from patients are as follows:

  • Medication A: 67, 75, 74, 70
  • Medication B: 70, 65, 76, 68
  • Medication C: 82, 85, 87, 83
  • Medication D: 60, 59, 61, 65

Hypotheses:

  • Null Hypothesis (H₀): The median response scores for all four medications are the same.
  • Alternative Hypothesis (H₁): At least one medication’s median response score is different from the others.

35.4.1 Kruskal-Wallis Test using Excel:

📥 Stats Basics (Excel)

35.5 Kruskal-Wallis Test using R and Python

This method allows for a robust analysis of variance when the data is not suited to traditional ANOVA, providing valuable insights in fields such as medicine, psychology, and ecological research.

Summary

Concept Description
Foundations
Kruskal-Wallis H Test A non-parametric test for differences across three or more independent groups based on ranks
Extension of Mann-Whitney Generalises Mann-Whitney to more than two groups
Non-parametric Counterpart of One-Way ANOVA Used when the assumptions of one-way ANOVA, especially normality, are not met
Assumptions
Independence of Samples Each group must contain different units, with no overlap between groups
Ordinal or Continuous Outcome The outcome must be ordinal or continuous so values from all groups can be jointly ranked
Similarity of Distribution Shape Distributions across groups should be similar in shape so the test can be read as a test of medians
Hypotheses
Null Hypothesis States that the populations have the same distribution (often summarised as equal medians)
Alternative Hypothesis States that at least one population tends to produce higher or lower values than the others
Computation
Combined Ranking All values from all groups are pooled and ranked from smallest to largest
Sum of Ranks Per Group Ranks are summed within each group to produce R_i, the rank sum for group i
H Statistic H is computed from the rank sums, sample sizes and total n using the standard Kruskal-Wallis formula
Chi-square Approximation Under H0, H follows approximately a chi-square distribution with k minus 1 degrees of freedom
Decision and Follow-up
Decision Rule Reject H0 when H exceeds the chi-square critical value or when p is below alpha
Post-hoc Comparisons Follow a significant H with pairwise tests such as Dunn's test, with adjustment for multiple comparisons
In R and Python
R via kruskal.test() Use kruskal.test(list(group1, group2, ...)) or kruskal.test(y ~ group, data) in R
Python via scipy.stats.kruskal() Use scipy.stats.kruskal(group1, group2, ...) on separate sequences in Python