Multiple Linear Regression by Hand (Step-by-Step)

Multiple linear regression is a method we can use to quantify the relationship between two or more predictor variables and a response variable.

This tutorial explains how to perform multiple linear regression by hand.

Example: Multiple Linear Regression by Hand

Suppose we have the following dataset with one response variable y and two predictor variables X₁ and X₂:

Use the following steps to fit a multiple linear regression model to this dataset.

Step 1: Calculate X₁², X₂², X₁y, X₂y and X₁X₂.

Step 2: Calculate Regression Sums.

Next, make the following regression sum calculations:

• Σx₁² = ΣX₁² – (ΣX₁)² / n = 38,767 – (555)² / 8 = 263.875
• Σx₂² = ΣX₂² – (ΣX₂)² / n = 2,823 – (145)² / 8 = 194.875
• Σx₁y = ΣX₁y – (ΣX₁Σy) / n = 101,895 – (555*1,452) / 8 = 1,162.5
• Σx₂y = ΣX₂y – (ΣX₂Σy) / n = 25,364 – (145*1,452) / 8 = -953.5
• Σx₁x₂ = ΣX₁X₂ – (ΣX₁ΣX₂) / n = 9,859 – (555*145) / 8 = -200.375

Step 3: Calculate b₀, b₁, and b₂.

The formula to calculate b₁ is: [(Σx₂²)(Σx₁y)  – (Σx₁x₂)(Σx₂y)]  / [(Σx₁²) (Σx₂²) – (Σx₁x₂)²]

Thus, b₁ = [(194.875)(1162.5)  – (-200.375)(-953.5)]  / [(263.875) (194.875) – (-200.375)²] = 3.148

The formula to calculate b₂ is: [(Σx₁²)(Σx₂y)  – (Σx₁x₂)(Σx₁y)]  / [(Σx₁²) (Σx₂²) – (Σx₁x₂)²]

Thus, b₂ = [(263.875)(-953.5)  – (-200.375)(1152.5)]  / [(263.875) (194.875) – (-200.375)²] = -1.656

The formula to calculate b₀ is: y – b₁X₁ – b₂X₂

Thus, b₀ = 181.5 – 3.148(69.375) – (-1.656)(18.125) = -6.867

Step 5: Place b₀, b₁, and b₂ in the estimated linear regression equation.

The estimated linear regression equation is: ŷ = b₀ + b₁*x₁ + b₂*x₂

In our example, it is ŷ = -6.867 + 3.148x₁ – 1.656x₂

How to Interpret a Multiple Linear Regression Equation

Here is how to interpret this estimated linear regression equation: ŷ = -6.867 + 3.148x₁ – 1.656x₂

b₀ = -6.867. When both predictor variables are equal to zero, the mean value for y is -6.867.

b₁ = 3.148. A one unit increase in x₁ is associated with a 3.148 unit increase in y, on average, assuming x₂ is held constant.

b₂ = -1.656. A one unit increase in x₂ is associated with a 1.656 unit decrease in y, on average, assuming x₁ is held constant.

Additional Resources

An Introduction to Multiple Linear Regression
How to Perform Simple Linear Regression by Hand