Linear regression is used to quantify the relationship between a predictor variable and a response variable.
Whenever we perform linear regression, we want to know if there is a statistically significant relationship between the predictor variable and the response variable.
We test for significance by performing a t-test for the regression slope. We use the following null and alternative hypothesis for this t-test:
- H0: β1 = 0 (the slope is equal to zero)
- HA: β1 ≠ 0 (the slope is not equal to zero)
We then calculate the test statistic as follows:
t = b / SEb
where:
- b: coefficient estimate
- SEb: standard error of the coefficient estimate
If the p-value that corresponds to t is less than some threshold (e.g. α = .05) then we reject the null hypothesis and conclude that there is a statistically significant relationship between the predictor variable and the response variable.
The following example shows how to perform a t-test for a linear regression model in practice.
Example: Performing a t-Test for Linear Regression
Suppose a professor wants to analyze the relationship between hours studied and exam score received for 40 of his students.
He performs simple linear regression using hours studied as the predictor variable and exam score received as the response variable.
The following table shows the results of the regression model:
To determine if hours studied has a statistically significant relationship with final exam score, we can perform a t-test.
We use the following null and alternative hypothesis for this t-test:
- H0: β1 = 0 (the slope for hours studied is equal to zero)
- HA: β1 ≠ 0 (the slope for hours studied is not equal to zero)
We then calculate the test statistic as follows:
- t = b / SEb
- t = 1.117 / 1.025
- t = 1.089
The p-value that corresponds to t = 1.089 with df = n-2 = 40 – 2 = 38 is 0.283.
Note that we can also use the T Score to P Value Calculator to calculate this p-value:
Since this p-value is not less than .05, we fail to reject the null hypothesis.
This means that hours studied does not have a statistically significant relationship between final exam score.
Additional Resources
The following tutorials offer additional information about linear regression:
Introduction to Simple Linear Regression
Introduction to Multiple Linear Regression
How to Interpret Regression Coefficients
How to Interpret the F-Test of Overall Significance in Regression