Two terms that students often confuse in statistics are likelihood and probability.
Here’s the difference in a nutshell:
- Probability refers to the chance that a particular outcome occurs based on the values of parameters in a model.
- Likelihood refers to how well a sample provides support for particular values of a parameter in a model.
When calculating the probability of some outcome, we assume the parameters in a model are trustworthy.
However, when we calculate likelihood we’re trying to determine if we can trust the parameters in a model based on the sample data that we’ve observed.
The following examples illustrate the difference between probability and likelihood in various scenarios.
Example 1: Likelihood vs. Probability in Coin Tosses
Suppose we have a coin that is assumed to be fair. If we flip the coin one time, the probability that it will land on heads is 0.5.
Now suppose we flip the coin 100 times and it only lands on heads 17 times. We would say that the likelihood that the coin is fair is quite low. If the coin was actually fair, we would expect it to land on heads much more often.
When calculating the probability of a coin landing on heads, we simply assume that P(heads) = 0.5 on a given toss.
However, when calculating the likelihood we’re trying to determine if the model parameter (p = 0.5) is actually correctly specified.
In the example above, a coin landing on heads only 17 out of 100 times makes us highly suspicious that the truly probability of the coin landing on heads on a given toss is actually p = 0.5.
Example 2: Likelihood vs. Probability in Spinners
Suppose we have a spinner split into thirds with three colors on it: red, green, and blue. Suppose we assume that it’s equally likely for the spinner to land on any of the three colors.
If we spin it one time, the probability that it lands on red is 1/3.
Now suppose we spin it 100 times and it lands on red 2 times, green 90 times, and blue 8 times. We would say that the likelihood that the spinner is actually equally likely to land on each color is very low.
When calculating the probability of the spinner landing on red, we simply assume that P(red) = 1/3 on a given spin.
However, when calculating the likelihood we’re trying to determine if the model parameters (P(red) = 1/3, P(green) = 1/3, P(blue) = 1/3) are actually correctly specified.
In the example above, the results of the 100 spins make us highly suspicious that each color is equally likely to occur.
Example 3: Likelihood vs. Probability in Gambling
Suppose a casino claims that the probability of winning money on a certain slot machine is 40% for each turn.
If we take one turn , the probability that we will win money is 0.40.
Now suppose we take 100 turns and we win 42 times. We would conclude that the likelihood that the probability of winning in 40% of turns seems to be fair.
When calculating the probability of winning on a given turn, we simply assume that P(winning) =0.40 on a given turn.
However, when calculating the likelihood we’re trying to determine if the model parameter P(winning) = 0.40 is actually correctly specified.
In the example above, winning 42 times out of 100 makes us believe that a probability of winning 40% of the time seems reasonable.
Additional Resources
The following tutorials provide addition information about probability:
What is a Probability Distribution Table?
What is the Law of Total Probability?
How to Find the Mean of a Probability Distribution
How to Find the Standard Deviation of a Probability Distribution