A post-hoc test is a type of test that is performed following an ANOVA to determine which group means are statistically significantly different from each other.
If one of the groups in the study is considered the control group, then we should use Dunnett’s test as the post-hoc test.
This tutorial explains how to perform Dunnett’s test in R.
Example: Dunnett’s Test in R
Suppose a teacher wants to know whether or not two new studying techniques have the potential to increase exam scores for her students. To test this, she randomly splits her class of 30 students into the following three groups:
- Control Group: 10 students
- New Study technique 1: 10 students
- New Study Technique 2: 10 students
After one week of using their assigned study technique, each student takes the same exam.
We can use the following steps in R to create a dataset, visualize the group means, perform a one-way ANOVA, and lastly perform Dunnett’s test to determine which (if either) new studying technique produces different results compared to the control group.
Step 1: Create the dataset.
The following code shows how to create a dataset that contains exam scores for all 30 students:
#create data frame data rep(c("control", "new1", "new2"), each = 10), score = c(76, 77, 77, 81, 82, 82, 83, 84, 85, 89, 81, 82, 83, 83, 83, 84, 87, 90, 92, 93, 77, 78, 79, 88, 89, 90, 91, 95, 95, 98)) #view first six rows of data frame head(data) technique score 1 control 76 2 control 77 3 control 77 4 control 81 5 control 82 6 control 82
Step 2: Visualize the exam scores for each group.
The following code shows how to produce boxplots to visualize the distribution of exam scores for each group:
boxplot(score ~ technique,
data = data,
main = "Exam Scores by Studying Technique",
xlab = "Studying Technique",
ylab = "Exam Scores",
col = "steelblue",
border = "black")
Just from the boxplots we can see that the distribution of exam scores is quite different for each studying technique. Next, we’ll perform a one-way ANOVA to determine if these differences are statistically significant.
Related: How to Plot Multiple Boxplots in One Chart in R
Step 3: Perform a one-way ANOVA.
The following code shows how to perform a one-way ANOVA to test for differences among mean exam scores in each group:
#fit the one-way ANOVA model
model #view model output
summary(model)
Df Sum Sq Mean Sq F value Pr(>F)
technique 2 211.5 105.73 3.415 0.0476 *
Residuals 27 836.0 30.96
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Since the overall p-value (0.0476) is less than .05, this is an indication that each group does not have the same average exam score. Next, we will perform Dunnett’s Test to determine which studying technique produces mean exam scores that differ from the control group.
Step 4: Perform Dunnett’s Test.
To perform Dunnett’s Test in R we can use the DunnettTest() function from the DescTools library which uses the following syntax:
DunnettTest(x, g)
where:
- x: A numeric vector of data values (e.g. exam scores)
- g: A vector that specifies the group names (e.g. studying technique)
The following code shows how to use this function for our example:
#load DescTools library library(DescTools) #perform Dunnett's Test DunnettTest(x=data$score, g=data$technique) Dunnett's test for comparing several treatments with a control : 95% family-wise confidence level $control diff lwr.ci upr.ci pval new1-control 4.2 -1.6071876 10.00719 0.1787 new2-control 6.4 0.5928124 12.20719 0.0296 * --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1.' 0.1 ' ' 1
The way to interpret the output is as follows:
- The mean difference in exam scores between the new studying technique 1 and the control group is 4.2 The corresponding p-value is 0.1787.
- The mean difference in exam scores between the new studying technique 2 and the control group is 6.4 The corresponding p-value is 0.0296.
Based on the output, we can see that studying technique 2 is the only technique that produces significantly (p = .0296) different mean exam scores compared to the control group.
Additional Resources
An Introduction to the One-Way ANOVA
How to Conduct a One-Way ANOVA in R
How to Perform Tukey’s Test in R