Green Belt · 15 min

What the fitted line is hiding

After this you can

read a residual plot and say whether a straight line was the right shape for the data.
identify a point that decides the fit on its own, and say what to do about it before drawing any conclusion.

Assumes you have done Two things move together. Now what?.

In 1973 the statistician Frank Anscombe published four datasets. They have the same mean x, the same mean y, the same variance, the same correlation, the same fitted line y = 3.00 + 0.500x, the same R² of 0.667, and — he arranged this too — the same residual standard error to two decimals.

One is an honest straight-line relationship. One is an exact parabola. One is a perfect line with a single bad row. One has ten readings at a single x, and one point far away that decides the slope entirely on its own.

Every number a regression produces is identical across all four. He was not making a point about statistics. He was making a point about looking.

The idea

Every statistic a regression produces is computed on the assumption that a straight line is the right shape. None of them can test that assumption — they were derived from it. So the check has to come from somewhere else.

The residual plot

A residual is what is left after subtracting the line: how far each point sits above or below it. Plot the residuals against the fitted values, and a genuine straight-line relationship leaves no pattern at all — scatter, evenly spread, centred on zero.

This works better than looking at the raw scatter, and the reason is worth understanding. On a scatter plot the eye follows the line, and a gentle curve alongside it is easy to miss. Subtracting the line removes the slope and leaves the pattern at full height. Anscombe's second set is passable as a scatter; as a residual plot it is an unmistakable arch.

Three failures show up there, and each has a name.

Curvature

The residuals form an arc — negative at both ends and positive in the middle, or the reverse. A straight line is the wrong shape: its slope overstates the effect in one part of the range and understates it in another, and predictions near the ends are wrong in a predictable direction, which is the worst kind of wrong.

This is common in operations, because most real relationships flatten out. Queues, learning curves, drying times, anything that saturates.

An influential point

One row that moves the whole line if you delete it. Cook's distance measures exactly that, and above about 1 it is worth stopping for.

A conclusion that depends on one row is a conclusion about that row. It might still be right — but it is a much weaker claim than the R² suggests, and the report should say which row it rests on.

Leverage — the one that hides

A point far out in x. It pulls the line towards itself whether or not it is right, and here is the part that catches people: its own residual is small, because the line was dragged to meet it.

The mathematics is exact. The variance of a residual is s²(1 − h), where h is the leverage. At h = 0.94 the residual is compressed to about a quarter of its usual size by construction. A high-leverage point looks like the best-behaved observation on the chart, and would look that way even if it were a transcription error.

So it cannot be found by looking for a large residual. It has to be found by measuring distance in x, which is what leverage is.

None of these is a reason to delete anything

They are reasons to go and find out what that row was. A high-leverage point is often the most informative observation in the study — it is the only one telling you what happens out there. Delete it and you may have no basis for a slope at all.

The move is: find it, explain it, and say in the report that the conclusion depends on it.

Worked example

Anscombe's four datasets, run through the tool. Eleven points each.

What every one of them reports:

	Set I	Set II	Set III	Set IV
Slope	0.5001	0.5000	0.4997	0.4999
Intercept	3.0001	3.0009	3.0025	3.0017
R²	0.6665	0.6662	0.6663	0.6667
r	0.8164	0.8162	0.8163	0.8165
Residual std error	1.2366	1.2372	1.2363	1.2357
Slope SE	0.1179	0.1180	0.1179	0.1178
t (on 9 df)	4.241	4.239	4.239	4.243
p	0.002	0.002	0.002	0.002

Nothing in that table separates them. A report containing only those rows — which is most regression reports — would describe all four identically.

What the checks report:

Set I. Curvature p = 0.487. No point above the leverage threshold of 0.364 (the highest is 0.318). No Cook's distance above 1 (the highest is 0.489). Nothing fires, because nothing is wrong. This is what an honest fit looks like.

Set II — the parabola. Curvature p < 0.001, on a t of −2219. The residual plot is an arch. Note what does not fire: no influential point, no high leverage. The data is not contaminated, it is the wrong shape, and only the curvature check can say so.

Set III — one bad row. Curvature p = 0.514: no bend. But row 3 has a Cook's distance of 1.393, well above the threshold. Ten of the eleven points lie exactly on a straight line; the eleventh is lifted off it and drags the slope. Remove it and the fit is perfect — which is the finding, not a reason to remove it.

Set IV — one point decides everything. Ten readings at x = 8 and one at x = 19. Row 8 has a leverage of exactly 1.000 against a threshold of 0.364. Its Cook's distance is reported as not defined rather than as zero, because the formula divides by (1 − h), and because "zero influence" would be the precise opposite of the truth: that point does not affect the line, it is the line. Delete it and there is no slope left to estimate at all.

The curvature check on set IV returns not testable, and says why: two distinct x values. Nothing can bend between two points. That is a more useful answer than a p-value would have been.

One more thing, worth knowing about tests generally. A runs test on the residual signs is also reported. On set II — an exact parabola, the most obvious curvature there is — it gives p = 0.067, and at the conventional threshold it would be waved through. The quadratic term gives p below 10⁻¹⁵ on the same data.

Both tests are correct. One of them is answering the question.

Your turn

The tool opens on the picking data, with the residual panel beside the scatter — never behind a toggle, for the reason this whole lesson is about.

Read the residual panel for the picking data: scatter, no pattern. That is the baseline.
Load Anscombe I and write down the slope, R² and r.
Load II, III and IV in turn, covering the charts. The headline numbers do not move. Now uncover them. The residual panel is completely different every time.
On II, read the curvature check and then the runs test underneath it. Two correct tests that disagree about whether there is anything to see.
On III, find the diamond. Note that the curvature check is clear — this is a different failure and a different check catches it.
On IV, find the square. Note the leverage of 1.000, the Cook's distance reported as not defined, and the curvature check reporting not testable rather than guessing.
On IV, delete the row 19,12.50 and read the error. There is no slope left to estimate. That is what the leverage flag was telling you, stated as plainly as it can be.

Regression & Correlation

practice

picking time = 3.2597 + 0.4297 x lines on the order

Over the range in this data, picking time rises by about 0.4297 for each one-unit increase in lines on the order (p < 0.001), and the data is consistent with a slope anywhere from 0.3918 to 0.4676. The line accounts for 96.9% of the variation in picking time, and typical points miss it by about 0.7424 in the units of picking time. A fitted line describes how these two columns moved together in this data. It is not evidence that one causes the other: the same line appears when the causation runs the other way, and when something not in the data drives both. Deciding causation needs an argument regression cannot supply — usually a change made deliberately, with everything else held still.

Slope

0.4297

R²

0.9692

Adjusted R²

0.9675

0.9845

Slope SE: 0.0180
t: 23.818 on 18 df
p: p < 0.001
95% interval on the slope: 0.3918 to 0.4676
Correlation interval: 0.960 to 0.994
Residual std error: 0.7424

Left: the data and the fitted line, drawn only across the range that was measured. Right: the same points with the line subtracted — a straight-line relationship leaves no pattern here. Squares mark points far out in lines on the order; diamonds mark points that move the line substantially if removed.

Show the data behind these charts

Scatter plot of 20 points, picking time against lines on the order, with the fitted line picking time = 3.2597 + 0.4297 x lines on the order. The line accounts for 96.9% of the variation in picking time. No point was flagged as unusually influential or far out in x. Residual plot: the vertical distance of each point from the fitted line, against the fitted value. A straight-line relationship shows no pattern here. No curvature was detected by the quadratic term test. Residuals run from -1.616 to 1.078, with a typical size of 0.742.
Row	lines on the order	picking time	Fitted	Residual	Leverage	Cook’s D
1	6	5.4	5.838	-0.438	0.186	0.049
2	9	7.7	7.127	0.573	0.138	0.055
3	11	8.5	7.987	0.513	0.111	0.034
4	14	9.4	9.276	0.124	0.081	0.001
5	17	9.9	10.565	-0.665	0.060	0.027
6	19	12.4	11.425	0.975	0.053	0.051
7	22	12.4	12.714	-0.314	0.050	0.005
8	24	12.9	13.573	-0.673	0.055	0.025
9	27	14.6	14.862	-0.262	0.070	0.005
10	29	16.8	15.722	1.078	0.086	0.108
11	31	16.9	16.581	0.319	0.107	0.012
12	34	18.5	17.871	0.629	0.147	0.072
13	7	6.7	6.268	0.432	0.169	0.041
14	12	6.8	8.416	-1.616	0.100	0.293
15	16	10	10.135	-0.135	0.066	0.001
16	21	12.9	12.284	0.616	0.050	0.019
17	26	14.5	14.433	0.067	0.064	0.000
18	30	16.8	16.152	0.648	0.096	0.045
19	33	16.7	17.441	-0.741	0.132	0.087
20	36	17.6	18.73	-1.13	0.179	0.309

Checks on the fit

ClearIs a straight line the right shape?No curvature detected (quadratic term p = 0.608). Still look at the residual panel: this tests for a bend, not for every shape.
ClearDo the residuals change sign as often as chance would?13 runs from 11 above and 9 below the line (p = 0.454). Consistent with chance — but this test is underpowered at these sample sizes and misses curvature that is plain on the plot, so it clears far more than it should.
ClearIs any point far out in lines on the order?No point exceeds a leverage of 0.200, so no single lines on the order value dominates the fit.
ClearWould removing one row change the answer?No row has a Cook’s distance above 1. The conclusion does not rest on any single observation.

Predict picking time at a given lines on the order

A fitted line describes how these two columns moved together in this data. It is not evidence that one causes the other: the same line appears when the causation runs the other way, and when something not in the data drives both. Deciding causation needs an argument regression cannot supply — usually a change made deliberately, with everything else held still.

Two numeric columns

lines on the order first, then picking time. Commas or tabs; a header row is detected.

Name of x

Name of y

Load Anscombe’s quartet

Four datasets with the same slope, intercept, R² and correlation. Load each in turn and watch the residual panel — it is the only thing that changes.

How this is calculated

The line is ordinary least squares: slope = S_xy/S_xx, intercept = ȳ − slope·x̄. Inference on the slope uses t = slope/SE on n − 2 degrees of freedom.
Leverage h_i = 1/n + (x_i − x̄)²/S_xx, flagged above 2p/n. The leverages always sum to 2, the number of parameters.
Cook’s distance measures how far the fitted line moves when a point is deleted, flagged above 1. Where leverage is exactly 1 it is reported as undefined rather than as zero — the point is fitted perfectly because it alone determines that part of the line, which is total influence, not none.
Curvature is tested by adding a quadratic term and testing its coefficient. A runs test on the residual signs is weaker: on Anscombe’s second set, an exact parabola, it gives p = 0.067 and would be waved through, while the quadratic term gives p below 10⁻¹⁵.
The correlation interval uses Fisher’s z-transformation, so it cannot run past ±1 the way a symmetric interval on r does.
Below 6 pairs the tool refuses. R² is close to 1 by construction at that size and the residuals cannot show whether a straight line was ever the right shape.

Source: Anscombe, F.J. (1973), ‘Graphs in Statistical Analysis’, The American Statistician 27(1), 17–21; Montgomery, D.C., Peck, E.A. and Vining, G.G., Introduction to Linear Regression Analysis 5e, Ch. 2 and Ch. 4; Cook, R.D. (1977), Technometrics 19(1), 15–18.

Check yourself

No hints. Wrong answers are explained, not softened.

A fitted line reports R² = 0.94 and p < 0.001 on the slope. The residual plot shows a clear arch: negative at both ends, positive in the middle. What should be concluded?

In a 20-point regression, one row has a leverage of 0.94 and a residual close to zero. How should it be read?

Four datasets share the same mean x, mean y, variance, correlation, fitted line and R² to three figures. What follows?

A regression flags one row with a Cook's distance of 2.1. What is the right next step?

Worth remembering

What does the residual plot show that R² cannot?

Whether a straight line was the right shape. Every regression statistic is derived assuming it was, so none of them can report on that assumption. Anscombe's four datasets share an R² of 0.667 and look nothing alike.

Why is a high-leverage point's residual small?

Because the line was pulled towards it. Var(e) = s²(1 − h), so at high leverage the residual is compressed by construction. The point looks well behaved even when it is wrong — leverage detects it, the residual cannot.

What does Cook's distance measure, and what should a large one prompt?

How far the fitted line moves if that point is deleted. Above about 1, stop and find out what happened on that row. It is a finding to investigate, never a licence to delete.

Why does the tool report Cook's distance as 'not defined' at leverage 1?

The formula divides by (1 − h), which is zero there. Reporting zero would state the opposite of the truth: that point does not have a small influence on the line, it IS the line — delete it and there is no slope left to estimate.

Can you do this now?

Rate yourself honestly. We compare your rating with how you actually answered — the gap is more useful than either number alone.

I can read a residual plot and say whether a straight line was the right shape for the data.
I can identify a point that decides the fit on its own, and say what to do about it before drawing any conclusion.

Your rating is recorded alongside your drill results. Neither alone marks the competency as met.

The problem