HookThe hole in the sky a computer was told to ignore
In May 1985 three scientists at the British Antarctic Survey — Joe Farman, Brian Gardiner and Jonathan Shanklin — published a short paper in Nature announcing that the ozone layer above Antarctica had thinned so dramatically each spring that there was, in effect, a hole in the sky. The shock was not only the hole itself. It was that NASA's Nimbus-7 satellite had been measuring the same ozone for years and had seen it too — but the data-processing software had been set to treat any reading below a low threshold as a probable instrument fault and flag it aside for checking rather than report it. The values were so extreme that the computer assumed they must be errors. The most important environmental measurement of the decade was very nearly thrown out as a glitch.
That is the double-edged heart of this section: a number that looks wrong might be an error to clean away — or the most important thing in the dataset. Learning to tell the difference, and to present data honestly, is the whole job. You will read the standard diagrams for single-variable data — histograms, box plots and cumulative frequency curves, where in a histogram it is the area that means frequency; interpret scatter diagrams and regression lines for two variables and confront the fact that correlation does not imply causation; measure the centre and the spread of data, extending to the standard deviation and computing it from summary statistics; and finally identify outliers, decide what to do with them, and clean a dataset without deleting its most interesting readings.
ModelReading single-variable diagrams — area is the whole point
Three diagrams dominate single-variable data. A histogram displays grouped continuous data, and its defining rule separates the strong candidates from the weak: area represents frequency, not height. When class widths are unequal — as they usually are in exam data — you cannot plot frequency up the vertical axis, because a wide bar would exaggerate its group. Instead you plot frequency density, defined as \(\text{frequency density}=\dfrac{\text{frequency}}{\text{class width}}\), so that the area of each bar (density × width) recovers the frequency. This is also what links a histogram to a probability distribution: shrink the bars and the outline becomes a smooth curve whose area under any stretch is the proportion of data there.
A cumulative frequency curve plots the running total of frequency against the upper class boundary, producing an S-shaped graph you read medians and quartiles off directly — go up to half the total, across to the curve, down to the value. A box plot distils that into five numbers: minimum, lower quartile \(Q_1\), median \(Q_2\), upper quartile \(Q_3\), and maximum, with the box spanning the middle half of the data. Box plots are built for comparison — set two side by side and you can compare centre (medians) and spread (box widths) at a glance, which is exactly what the exam asks for.
A survey records the ages of people at an event in unequal groups. The \(20\text{–}40\) age band (a class width of \(20\) years) contains \(50\) people, while the \(40\text{–}45\) band (width \(5\)) contains \(30\). Which bar is taller on a correctly drawn histogram?
Compute frequency density for each. For \(20\text{–}40\): \(\dfrac{50}{20}=2.5\) people per year. For \(40\text{–}45\): \(\dfrac{30}{5}=6\) people per year. So although the \(20\text{–}40\) band holds more people, the \(40\text{–}45\) bar is more than twice as tall, because the people are packed into a fifth of the width. If you had wrongly plotted frequency as height, the \(50\)-bar would tower over the \(30\)-bar and completely misrepresent how concentrated the older group is. Reading it back the honest way: the \(20\text{–}40\) bar has area \(2.5\times 20=50\) and the \(40\text{–}45\) bar has area \(6\times 5=30\) — the frequencies restored.
ModelCentre and spread — extending to standard deviation
Every dataset needs a measure of its centre and a measure of its spread. For the centre you know the mean, median and mode; the mean \(\bar{x}=\dfrac{\sum x}{n}\) uses every value but is dragged by extremes, while the median resists them. For spread, the range and interquartile range are quick, but the measure that uses every value is the standard deviation — roughly, the typical distance of a value from the mean. Its square is the variance, and the working formula that the exam lives on is
\[\sigma^2=\frac{\sum x^2}{n}-\bar{x}^2=\frac{\sum x^2}{n}-\left(\frac{\sum x}{n}\right)^2,\]
read as 'the mean of the squares minus the square of the mean'. Take the square root for the standard deviation \(\sigma\). This form is powerful because it needs only three numbers — \(n\), \(\sum x\) and \(\sum x^2\) — so exams hand you those summary statistics and expect the spread without ever seeing the raw data.
The standard deviation carries the same units as the data (variance is in the square of those units, which is why we usually quote \(\sigma\), not \(\sigma^2\)), and it underpins the whole of later statistics: outlier fences at \(\bar{x}\pm 2\sigma\), the shape of the Normal distribution, and comparisons of consistency between datasets. A small \(\sigma\) means the data cluster tightly around the mean; a large \(\sigma\) means they scatter.
A dataset of \(n=10\) values has \(\sum x=50\) and \(\sum x^2=310\). Find the mean and the standard deviation.
The mean is \(\bar{x}=\dfrac{\sum x}{n}=\dfrac{50}{10}=5\). Now the variance, straight from the working formula:
\[\sigma^2=\frac{\sum x^2}{n}-\bar{x}^2=\frac{310}{10}-5^2=31-25=6.\]
The standard deviation is the square root, \(\sigma=\sqrt{6}\approx 2.449\). Notice how little you needed: not one of the ten original values, only the three summary statistics. The commonest slip is to forget the final square root and quote the variance \(6\) as the standard deviation — always ask whether your answer's units match the data. Here the typical value sits about \(2.4\) units from the mean of \(5\), which is a sensible spread for numbers of this size.
MechanismScatter, regression and the causation trap
With two variables measured on each item — height and weight, revision hours and exam score — you plot a scatter diagram and look for a relationship. Correlation describes how closely the points follow a straight line: positive when they rise together, negative when one falls as the other rises, and strong or weak depending on how tightly they hug the line. A regression line (the line of best fit) summarises a linear trend and lets you predict — but only safely within the range of the data you have. Predicting outside it, called extrapolation, assumes the trend continues into territory you never measured, and it is where confident predictions go to die.
Then comes the single most important sentence in all of statistics: correlation does not imply causation. Ice cream sales and drownings rise together every summer, but ice cream does not drown anyone — a third variable, hot weather, drives both. Such a hidden common cause is a confounding variable, and it is why a strong correlation, on its own, proves only association, never that one thing makes the other happen. When you interpret a regression line in context, describe the trend and give a number, but resist the leap to 'therefore \(x\) causes \(y\)' unless the design of the study rules out confounders — which a simple scatter plot never does.
CaseOutliers and cleaning data — without deleting the discovery
An outlier is a value far from the rest of the data, and the exam wants a rule, not a hunch. Two standard fences are used: a value is an outlier if it lies more than \(1.5\times\text{IQR}\) beyond a quartile — that is, below \(Q_1-1.5(\text{IQR})\) or above \(Q_3+1.5(\text{IQR})\) — or, using the other measure, if it lies more than two standard deviations from the mean, outside \(\bar{x}\pm 2\sigma\). The question will tell you which rule to apply; your job is to compute the fence and compare.
What you do next is data cleaning, and the ozone hole is the cautionary tale. A genuine recording error — a thermometer misread as 320°C, a weight entered as \(700\) kg for a person — should be corrected or removed, because it is not real data. But an outlier that is a true reading must stay, because it may be the most important point in the set: the low ozone values were not glitches, they were the discovery. So the disciplined process is: identify outliers by the stated rule, investigate whether each is an error or a real extreme, correct or remove only genuine mistakes, and document what you did. The same judgement governs choosing and critiquing a presentation: a box plot exposes outliers as points beyond the whiskers, a histogram shows the shape of a distribution, cumulative frequency gives medians and percentiles — pick the diagram that answers the actual question, and be ready to say why a given choice misleads.
A dataset has \(Q_1=12\) and \(Q_3=20\), and its largest reading is \(40\). Using the \(1.5\times\text{IQR}\) rule, is \(40\) an outlier — and should it be removed?
First the interquartile range: \(\text{IQR}=Q_3-Q_1=20-12=8\). Then \(1.5\times\text{IQR}=1.5\times 8=12\). The upper fence is \(Q_3+1.5(\text{IQR})=20+12=32\). Since \(40>32\), the value \(40\) lies beyond the fence and is flagged as an outlier. But flagged is not the same as deleted. The rule tells you where to look, not what to conclude: you must now investigate whether \(40\) is a data-entry error (say a \(4.0\) typed as \(40\)) or a genuine extreme value. Delete it only if you can show it is a mistake — otherwise it stays in the analysis, exactly as the Antarctic ozone readings should have. Reporting 'outlier by the \(1.5\times\text{IQR}\) rule, retained pending investigation' is the honest, full-mark answer.
VocabularyKey terms the mark scheme pays for
TrapsMisconceptions that cost marks
ExamWhat examiners want
For histograms, always work through frequency density: if a question gives you a bar's height and width, multiply to get frequency, and if it gives frequency and width, divide to plot the bar. State the rule 'area represents frequency' when asked to justify a step — it is a marked point. When comparing two box plots, make two comparisons in context: one about centre (compare medians) and one about spread (compare IQRs or ranges), each with a sentence tying it back to the data.
For standard deviation from summary statistics, quote the formula \(\sigma=\sqrt{\dfrac{\sum x^2}{n}-\bar{x}^2}\), substitute \(n\), \(\sum x\) and \(\sum x^2\) explicitly, and take the final square root — check your answer is in the data's units, not their square. On correlation questions, describe the strength and direction and then add the causation caveat proactively: if you are asked whether a link is causal, say that correlation alone cannot show it and name a plausible confounder. For outliers, compute the fence (\(1.5\times\text{IQR}\) beyond a quartile, or \(\bar{x}\pm 2\sigma\)) before declaring anything an outlier, and when asked whether to remove one, distinguish a recording error (remove) from a genuine extreme (retain and investigate) — the mark scheme reserves credit for that judgement, not just the arithmetic.