When I first learned statistics it took some time before I completely understood the concepts of Type I and Type II errors and what they were in relation to a hypothesis test. The concept is not difficult to understand once we get past the language of statistics.
First, we have to realize that almost every time we make a decision based on data there is some chance we will make an error. Unless we are “all knowing”, we have to realize descriptive statistics simply describe the data that we are working with. Usually we are working with samples of much larger populations of data. We never know with absolute certainly what the true mean of a large population is.
For a typical hypothesis decision we recall a typical null and alternate hypothesis statement for a two tailed test.
H0: μ = 0 (null hypothesis)
H1: μ ≠ 0 (alternate hypothesis)
The objective of the hypothesis test will be to make a decision about the null and alternate hypothesis statements. There are only two possible outcomes of this decision.
1) We reject the null hypothesis
2) We fail to reject the null hypothesis
The possibility of error comes in because we make this decision regardless of whether the null hypothesis is actually true or false. This gives us four possible situations that I will describe and present a table for to help with understanding the concept visually. If you can memorize this table and recreate it. You will always be able to determine what is meant by Type I and Type II errors.
1) Type I Error: This is a situation where H0 is true but our statistical test rejects it anyway. Think of this as analogous to convicting an innocent person in our judicial system. If someone is innocent but was convicted anyway the court has made a Type I error. In our case the null statement was true, but we rejected it. We have made an error. Type I error is synonymous with significance level and is often expressed as a probability value with the Greek letter α or alpha. Type I error is also known as a false positive error. An effect was not present but we claimed it was.
2) Correct Decision: Hypothesis testing yields a correct decision in two specific cases. The first is if H0 is true and we fail to reject the null hypothesis. To follow our judicial system analogy, a person is innocent and the court finds him not guilty. This is the correct decision. The second is if H0 is a false statement and we do reject the null hypothesis. In our court system a person is truly guilty and the court finds him guilty. This also is a correct decision.
3) Type II error: In this situation H0 is a false statement and we should reject it. However our hypothesis test leads us to fail to reject the null. We have made a second error known as Type II. Still keeping with the court system analysis this is a dangerous situation because a person is truly guilty and yet the jury or court finds him innocent. In statistics, there was a statistical effect present but we failed to detect it. The probability of Type II error is referred to as β or beta. Type II error is also known as a false negative error. An effect was present but we failed to detect it.
Here is a table that is often presented to show the relationships between type one and type II errors. As stated previously if you can memorize and recreate this table you will be able to succeed at identifying type I and type II errors.
H0 is True | H0 is False | |
Fail to Reject H0 | Correct Decision Confidence Level (1-α) | Type II error β |
Reject H0 | Type I error Significance Level = α | Correct Decision Power = (1-β) |
Now let’s practice on a couple of situations to help identify the correct type of error.
Problem #1 – A tire company rejected a batch of rubber from their supplier stating that it was out of specification in hardness. Later the supplier showed that the material was in spec and it was discovered an error was made in hardness analysis on the part of the tire company. What type of error did the tire company make and why?
In this case the tire company committed a type I error. There was no difference in actual hardness vs. the hardness specification for this batch. The tire company should not have rejected the material.
Problem #2 – A jet aircraft engine manufacturer has inspected a lot of 100 turbine blades for its prototype jet engine. The blades are put in production because they have passed all QC checks including a critical balance tolerance. Later an accident occurs with an engine. What type of error was made?
In this case the jet engine manufacturer made a type II error. The turbine blades were out of balance and they failed to detect the effect.
