What Does a Confidence Interval Mean?
A confidence interval for a proportion is built as $\hat{p} \pm z^{*}\sqrt{\dfrac{\hat{p}(1-\hat{p})}{n}}$. Here you set the true proportion $p$ (in real life it is unknown). Each horizontal line is the interval from one sample and the vertical line is the true $p$. Intervals that contain the true proportion are green, the others red.
What to notice
- "95% confidence" is a statement about the method, not about one interval: in the long run about 95% of intervals built this way contain the true proportion. Add 1000 intervals and look at the percentage.
- A particular interval either contains the true proportion or it does not; saying "there is a 95% probability it is in this interval" is wrong.
- As $n$ grows the intervals get narrower, but the capture rate does not change. Raising the confidence level makes the intervals wider.
- Set $p$ to 0.05 and $n$ to 20: the conditions fail and the capture rate drops clearly below the promised level. This is why we check conditions.