Assume nothing has changed, then look for evidence against it
Significance level — The probability, chosen in advance, of rejecting H₀ when it is in fact true — the risk of a false alarm that you are prepared to accept.
A hypothesis test starts from a deliberately conservative position. The null hypothesis H₀ says nothing unusual is happening — the coin is fair, the mean has not moved, the new method is no better. The alternative hypothesis H₁ states what you suspect instead.
The test then asks a single question: if H₀ were true, how surprising would this data be? If the answer is "very surprising indeed", H₀ is rejected. If not, it is retained — not because it has been proved, but because there was insufficient evidence against it.
The asymmetry is deliberate and mirrors a courtroom. A verdict of not guilty does not establish innocence; it records that the evidence fell short. Hypothesis testing works the same way, which is why the conclusion is always phrased in terms of evidence rather than truth.
Never "accept H₀"
The correct phrase is "there is insufficient evidence to reject H₀". A test can fail to find evidence for many reasons, including a sample too small to detect a real effect. Writing "accept H₀" or "H₀ is true" loses the conclusion mark even when every calculation above it is perfect.
One tail or two
The alternative hypothesis decides where the evidence must lie for H₀ to be rejected, and therefore how the significance level is spent.
If H₁ specifies a direction — "the mean has increased" — then only one tail counts, and the whole of the significance level sits there. If H₁ is merely "the mean has changed", evidence in either direction counts, so the level is split between the two tails, giving 2.5% at each end for a 5% test.
This matters more than it first appears: the critical z for a 5% one-tailed test is 1.645, while for a 5% two-tailed test it is 1.960. The same data can be significant under one and not the other, so choosing correctly from the wording is worth real marks.
| Wording of H₁ | Tails | Critical z at 5% | Critical z at 1% |
|---|---|---|---|
| mean has increased | one (upper) | 1.645 | 2.326 |
| mean has decreased | one (lower) | −1.645 | −2.326 |
| mean has changed | two | ±1.960 | ±2.576 |
Drag z to 1.645 and the unshaded upper tail is 5% — the critical region for a one-tailed test. Drag to 1.960 and it is 2.5%, which is what each tail gets when 5% must be split between two.
The method, in the order the marks are awarded
Hypothesis testing is generously marked because the structure is fixed. Marks are given for each stage, so a wrong final answer built on correct stages still scores well — provided the stages are actually written down.
Two routes are available and both are accepted. The critical value route compares the test statistic against a boundary; the p-value route compares a probability against the significance level. Mixing them up — comparing a probability against a z-value — is the error to guard against.
- x̄
- the observed sample meanwhat the data gave
- μ
- the value claimed by H₀the assumption being tested
- σ/√n
- the standard errorhow much x̄ would vary by chance alone
A machine is set to fill bottles with a mean of 500 ml and known standard deviation 8 ml. A sample of 40 bottles has mean 502.6 ml. Test at the 5% level whether the mean has increased.
- H₀: μ = 500; H₁: μ > 500, where μ is the population mean fill volume.The word "increased" makes this one-tailed in the upper direction. Defining μ is part of the mark.
- One-tailed test at the 5% level, so the critical value is z = 1.645.The whole 5% sits in the upper tail because H₁ specifies a direction.
- Standard error = 8/√40 = 1.2649.The spread of the sample mean, not of a single bottle.
- z = (502.6 − 500)/1.2649 = 2.055.The observed mean is just over two standard errors above the claimed value.
- 2.055 > 1.645, so H₀ is rejected.The test statistic falls inside the critical region.
- There is sufficient evidence at the 5% level to conclude that the mean fill volume has increased above 500 ml.The conclusion must be in context and must not claim proof — "sufficient evidence" is the required register.
z = 2.06 > 1.645; reject H₀ — evidence of an increase
The six stages
- State H₀ and H₁ in symbols, with the parameter defined.
- State the significance level and whether the test is one- or two-tailed.
- Compute the test statistic, using the standard error as the denominator.
- Find the critical value, or the p-value.
- Compare, and state whether H₀ is rejected.
- Write a conclusion in the context of the question, not just "reject H₀".
The two ways a test can be wrong
A test decides on incomplete information, so it can reach the wrong verdict in two distinct ways, and they are not symmetric.
A Type I error rejects a true H₀ — a false alarm. Its probability is exactly the significance level, which is what makes that level a genuine choice: setting 1% rather than 5% means accepting a false alarm one time in a hundred rather than one in twenty.
A Type II error fails to reject a false H₀ — a missed detection. Reducing the significance level makes Type I errors rarer but Type II errors more common, so the two cannot be minimised together. Which matters more depends entirely on the consequences: a smoke alarm should tolerate false alarms, while a test that condemns a batch of expensive product should not.
| H₀ is actually true | H₀ is actually false | |
|---|---|---|
| Reject H₀ | Type I error (probability = α) | correct decision |
| Do not reject H₀ | correct decision | Type II error |
The significance level is a deliberate choice
P(Type I error) is exactly the significance level, so choosing 5% means accepting a one-in-twenty chance of a false alarm. That is a judgement about consequences, not a mathematical fact — which is why the level is stated at the start of a test rather than chosen after seeing whether the result comes out significant.