A non-parametric test is a hypothesis test that does not make any assumptions about
the distribution of the samples.
The Mann-Whitney Test
The Mann-Whitney test, also known as the Wilcoxon rank sum test or the Wilcoxon-Mann-Whitney test,
tests the hypothesis that two samples were drawn from the same distribution. The test relies only on the
relative ranks of the observations in the combined sample. It does not rely on any properties of
the distributions.
The null hypothesis is that the samples were drawn from the same distribution.
Location is the dominant factor in the comparison, but if the two distributions have different shapes,
this may also affect the result. The alternative hypothesis for the two-tailed test (the default)
is that the two samples were drawn from different distributions.
For the one-tailed test, the alternative hypothesis is roughly that the population
from which the first sample was drawn has a median that is either less than (lower tailed)
or greater than (upper tailed) the median of the population from which the second sample was drawn.
The Mann-Whitney test is implemented by the
MannWhitneyTest class.
It has three constructors.
The first constructor takes no arguments. All test parameters must be provided by setting the properties of the
MannWhitneyTest object. The samples the test is to be applied to must
be specified by setting the
Sample1()()()()
and Sample1()()()()
properties.
The second constructor takes two arguments that are
NumericalVariable
objects that represent the samples the test is to be applied to.
The third constructor also takes two arguments. The first is once again a
NumericalVariable
that contains the observations from the two samples combined.
The second argument must be a
CategoricalVariable
with two levels that specifies which sample the observations in the first variable
belong to.
The distribution of the Mann-Whitney statistic U can be computed exactly.
For larger samples, an approximation
in terms of the normal distribution can be used. If the observations don't have distinct ranks
(i.e. there are ties), then the exact calculations are not available.
The exact test is used by default if there are no ties, and if the product
of the two sample sizes is less than or equal to 1600.
You can override the default behaviour by setting the
Exactness
property. Note that the 'exact' calculation gives incorrect results if ties are present.
Example
In this example, we examine whether the onset of type II diabetes is different for males and females.
The onset in our sample is as follows:
-
Males: 19 22 16 29 24
-
Females: 20 11 17 12
The code below performs the exact Mann-Whitney test on this sample:
The value of the U-statistic is 3, giving a two-tailed p-value of 0.1111. As a result,
the hypothesis that on average, type II diabetes starts around the same age in males and females
is not rejected at the 0.05 significance level.
The Kruskal-Wallis Test
The Kruskal-Wallis test is a generalization of the Mann-Whitney test to more than two samples.
It tests the hypothesis that the supplied samples were all drawn from the same distribution.
The test relies only on the
relative ranks of the observations in the combined sample. It does not rely on any properties of
the distributions.
The null hypothesis is that the samples were drawn from the same distribution.
Location is the dominant factor in the comparison, but if the distributions have different shapes,
this may also affect the result. The alternative hypothesis for the two-tailed test
is that the samples were drawn from different distributions.
The Kruskal-Wallis test is implemented by the
KruskalWallisTest class.
It has three constructors.
The first constructor takes no arguments. All test parameters must be provided by setting the properties of the
KruskalWallisTest object. The samples the test is to be applied to must
be specified through the
Samples
property.
The second constructor takes an array of
NumericalVariable
objects that represent the samples the test is to be applied to.
The third constructor also takes two arguments. The first is once again a
NumericalVariable
that contains the observations from all the samples combined.
The second argument must be a
CategoricalVariable
that specifies which sample the observations in the first variable
belong to.
Although the distribution of the Kruskal-Wallis statistic can be computed exactly,
it is impractical except for very small values. We use an approximation in terms
of the beta distribution which is more accurate than the more commonly used
approximation in terms of a Chi-square distribution.
Example
In this example, we examine whether different teaching methods have an effect on a standardized
exam score. The exam scores for students that were taught using each of the three methods
were as follows:
-
Method 1: 94, 87, 90, 74, 86, 97
-
Method 2: 82, 85, 79, 84, 61, 72, 80
-
Method 3: 89, 68, 72, 76, 69, 65
The code below performs the Kruskal-Wallis test on these samples:
The value of the Kruskal-Wallis statistic is 7.5023, giving a two-tailed p-value of 0.0235. As a result,
the hypothesis that on average, the teaching method has no effect on exam scores
is rejected at the 0.05 significance level.