Green Belt · 15 min

Two things move together. Now what?

After this you can

fit a line to two measurements and state what its slope means in the units of the process.
explain why a fitted relationship is not evidence of cause, and name what would be.

Assumes you have done Is this difference real? And is it big enough to care about?.

A maintenance team found that machine age and scrap rate correlated at r = 0.81 across thirty machines. The oldest six were replaced at a cost of £340,000. The scrap rate did not move.

The correlation was real. The older machines were also the ones trusted with the difficult products, run on the night shift, by the newest operators. Age was a proxy for three other things, and the new machines inherited all of them.

The idea

A regression fits a line to two measurements. The arithmetic never fails: give it any two columns and it returns a slope, an intercept and an R², whether or not those two columns have anything to do with one another.

What the numbers mean

The slope is the answer to the question you actually asked: how much y changes for a one-unit change in x, in the units the process uses. It is the number that goes in a business case, and it is the only one of these that is directly actionable.

The intercept is where the line crosses x = 0. Sometimes that is a real quantity; sometimes x = 0 is nowhere near anything you measured, and the intercept is arithmetic rather than meaning. Check which before quoting it.

R² is the proportion of the variation in y that the line accounts for. It is not a measure of whether the line is right, and it is not a measure of importance. A high R² on the wrong shape is worse than a low one on the right shape, because it is more convincing.

The correlation r is R² square-rooted, with a sign. It carries no information the slope and R² do not, and it is more easily misread — "correlation" sounds like a verdict.

Two questions, asked in the same words

"What will the next one be?" — needs the prediction interval, which includes the scatter of individual points around the line.
"What is the average at this setting?" — needs the confidence interval for the mean, which is much narrower because averages vary less than individuals do.

Quoting the second when someone meant the first understates the uncertainty, often by a factor of several. Both are legitimate; only one answers the question being asked.

Where the line stops

Past the ends of the data, the line is a guess. Nothing was observed out there, so nothing in the data says the same straight relationship continues — and most real relationships eventually bend.

The interval does widen towards the edges, but not enough. It prices in the scatter of points around a straight line. It cannot price in the relationship having a different shape where you did not look, which is the actual risk.

Extrapolating is not forbidden. It is unsupported — a different kind of claim, which has to be labelled as one.

The sentence that has to be said

A fitted line is not evidence of cause. The same line appears when the causation runs the other way, and when something not in the data drives both.

A regression fits equally well in all three cases, so it cannot distinguish between them. What would: changing x deliberately, holding everything else still, and seeing y move. That is an experiment, not a regression — and it is why Improve is a separate phase from Analyse.

The correlation is still worth having. It tells you where to look. It does not tell you what to do, and the £340,000 in the Hook is the difference between the two.

Worked example

Twenty picked orders from a warehouse, recorded as the number of lines on the order and the minutes taken to pick it. The suspicion is the obvious one: bigger orders take longer. The question worth answering is how much longer, because that is what sizes a batching policy.

Step 1 — fit the line.

picking time = 3.2597 + 0.4297 × lines

Slope 0.4297 minutes per line, standard error 0.0180
t = 23.818 on 18 degrees of freedom, p < 0.001
95% interval on the slope: 0.3918 to 0.4676 minutes per line
R² 0.9692, adjusted R² 0.9675, r 0.9845
Residual standard error 0.7424 minutes

Each additional line costs about 26 seconds, and the data is consistent with anything from 23 to 28 seconds. That interval is the useful output — not "significant", but "between 23 and 28 seconds a line, and here is how many lines we pick a week".

Step 2 — the intercept, which is the trap.

3.2597 minutes at zero lines. That is a genuinely meaningful quantity in this process — the fixed cost of walking to the aisle and back regardless of order size — and it is still extrapolation, because the smallest order in the data has six lines. The tool says so. The number may well be about right; the data does not say that it is.

Step 3 — predict, and notice the gap between the two intervals.

At 20 lines, the fitted time is 11.854 minutes.

Question	Interval	Width
Where would one more order land?	10.255 to 13.453	3.198
What is the average at 20 lines?	11.503 to 12.206	0.703

Four and a half times wider. If you are staffing a shift you want the first. If you are costing a thousand orders you want the second. Reporting one and meaning the other is the error, and both are labelled here so it cannot happen silently.

Step 4 — what this does and does not license.

It licenses a batching calculation: lines per order times 26 seconds, plus the fixed cost per order. That arithmetic is exactly what the slope is for.

It does not license "reducing order size reduces picking time per line". Nothing here was changed on purpose — these are twenty orders that happened to arrive at the sizes they did.

Dataset: ds-picking-time-order-size — the same data loads in the tool below, so you can reproduce every figure here yourself.

Your turn

The tool opens on the picking data.

Confirm the slope 0.4297, the interval 0.3918 to 0.4676, and R² 0.9692.
Predict at 20 lines. Compare the prediction interval with the mean interval and note the ratio.
Predict at 0 lines. The extrapolation warning appears — the intercept is on the report, and it is still outside the data.
Predict at 60 lines. The fitted value is about 29 minutes. Read the warning carefully: it explains that the interval widens but does not widen enough, because it cannot account for the relationship having a different shape out there.
Change the names of x and y to the real ones. The equation and every sentence re-render in your units — a slope means nothing until it says minutes per line rather than per unit of x.

Regression & Correlation

practice

picking time = 3.2597 + 0.4297 x lines on the order

Over the range in this data, picking time rises by about 0.4297 for each one-unit increase in lines on the order (p < 0.001), and the data is consistent with a slope anywhere from 0.3918 to 0.4676. The line accounts for 96.9% of the variation in picking time, and typical points miss it by about 0.7424 in the units of picking time. A fitted line describes how these two columns moved together in this data. It is not evidence that one causes the other: the same line appears when the causation runs the other way, and when something not in the data drives both. Deciding causation needs an argument regression cannot supply — usually a change made deliberately, with everything else held still.

Slope

0.4297

R²

0.9692

Adjusted R²

0.9675

0.9845

Slope SE: 0.0180
t: 23.818 on 18 df
p: p < 0.001
95% interval on the slope: 0.3918 to 0.4676
Correlation interval: 0.960 to 0.994
Residual std error: 0.7424

Left: the data and the fitted line, drawn only across the range that was measured. Right: the same points with the line subtracted — a straight-line relationship leaves no pattern here. Squares mark points far out in lines on the order; diamonds mark points that move the line substantially if removed.

Show the data behind these charts

Scatter plot of 20 points, picking time against lines on the order, with the fitted line picking time = 3.2597 + 0.4297 x lines on the order. The line accounts for 96.9% of the variation in picking time. No point was flagged as unusually influential or far out in x. Residual plot: the vertical distance of each point from the fitted line, against the fitted value. A straight-line relationship shows no pattern here. No curvature was detected by the quadratic term test. Residuals run from -1.616 to 1.078, with a typical size of 0.742.
Row	lines on the order	picking time	Fitted	Residual	Leverage	Cook’s D
1	6	5.4	5.838	-0.438	0.186	0.049
2	9	7.7	7.127	0.573	0.138	0.055
3	11	8.5	7.987	0.513	0.111	0.034
4	14	9.4	9.276	0.124	0.081	0.001
5	17	9.9	10.565	-0.665	0.060	0.027
6	19	12.4	11.425	0.975	0.053	0.051
7	22	12.4	12.714	-0.314	0.050	0.005
8	24	12.9	13.573	-0.673	0.055	0.025
9	27	14.6	14.862	-0.262	0.070	0.005
10	29	16.8	15.722	1.078	0.086	0.108
11	31	16.9	16.581	0.319	0.107	0.012
12	34	18.5	17.871	0.629	0.147	0.072
13	7	6.7	6.268	0.432	0.169	0.041
14	12	6.8	8.416	-1.616	0.100	0.293
15	16	10	10.135	-0.135	0.066	0.001
16	21	12.9	12.284	0.616	0.050	0.019
17	26	14.5	14.433	0.067	0.064	0.000
18	30	16.8	16.152	0.648	0.096	0.045
19	33	16.7	17.441	-0.741	0.132	0.087
20	36	17.6	18.73	-1.13	0.179	0.309

Checks on the fit

ClearIs a straight line the right shape?No curvature detected (quadratic term p = 0.608). Still look at the residual panel: this tests for a bend, not for every shape.
ClearDo the residuals change sign as often as chance would?13 runs from 11 above and 9 below the line (p = 0.454). Consistent with chance — but this test is underpowered at these sample sizes and misses curvature that is plain on the plot, so it clears far more than it should.
ClearIs any point far out in lines on the order?No point exceeds a leverage of 0.200, so no single lines on the order value dominates the fit.
ClearWould removing one row change the answer?No row has a Cook’s distance above 1. The conclusion does not rest on any single observation.

Predict picking time at a given lines on the order

A fitted line describes how these two columns moved together in this data. It is not evidence that one causes the other: the same line appears when the causation runs the other way, and when something not in the data drives both. Deciding causation needs an argument regression cannot supply — usually a change made deliberately, with everything else held still.

Two numeric columns

lines on the order first, then picking time. Commas or tabs; a header row is detected.

Name of x

Name of y

Load Anscombe’s quartet

Four datasets with the same slope, intercept, R² and correlation. Load each in turn and watch the residual panel — it is the only thing that changes.

How this is calculated

The line is ordinary least squares: slope = S_xy/S_xx, intercept = ȳ − slope·x̄. Inference on the slope uses t = slope/SE on n − 2 degrees of freedom.
Leverage h_i = 1/n + (x_i − x̄)²/S_xx, flagged above 2p/n. The leverages always sum to 2, the number of parameters.
Cook’s distance measures how far the fitted line moves when a point is deleted, flagged above 1. Where leverage is exactly 1 it is reported as undefined rather than as zero — the point is fitted perfectly because it alone determines that part of the line, which is total influence, not none.
Curvature is tested by adding a quadratic term and testing its coefficient. A runs test on the residual signs is weaker: on Anscombe’s second set, an exact parabola, it gives p = 0.067 and would be waved through, while the quadratic term gives p below 10⁻¹⁵.
The correlation interval uses Fisher’s z-transformation, so it cannot run past ±1 the way a symmetric interval on r does.
Below 6 pairs the tool refuses. R² is close to 1 by construction at that size and the residuals cannot show whether a straight line was ever the right shape.

Source: Anscombe, F.J. (1973), ‘Graphs in Statistical Analysis’, The American Statistician 27(1), 17–21; Montgomery, D.C., Peck, E.A. and Vining, G.G., Introduction to Linear Regression Analysis 5e, Ch. 2 and Ch. 4; Cook, R.D. (1977), Technometrics 19(1), 15–18.

Check yourself

No hints. Wrong answers are explained, not softened.

Analysis of a year of records shows overtime hours and defect counts rise together, r = 0.78, p < 0.001. What does this license?

A line fitted to orders of 6 to 36 lines gives: picking time = 3.26 + 0.43 × lines. A manager asks what a 60-line order would take. What is the right answer?

A report gives a fitted value of 11.85 minutes with an interval of 11.50 to 12.21. A supervisor uses it to decide how long to allow for the next order. What is wrong?

Worth remembering

What does the slope of a fitted line tell you?

How much y changes for a one-unit change in x, in the units the process uses. It is the only regression output that is directly actionable — and it means nothing until it is stated in real units rather than 'per unit of x'.

Prediction interval or confidence interval for the mean?

Prediction interval for 'what will the next one be' — it includes the scatter of individual points. The mean interval is much narrower and answers 'what is the average at this setting'. Confusing them understates the uncertainty several times over.

What would turn a correlation into evidence of cause?

Changing x deliberately with everything else held still, and seeing y move. That is an experiment. A regression fits equally well when the causation runs the other way, or when something not in the data drives both.

Can you do this now?

Rate yourself honestly. We compare your rating with how you actually answered — the gap is more useful than either number alone.

I can fit a line to two measurements and state what its slope means in the units of the process.
I can explain why a fitted relationship is not evidence of cause, and name what would be.

Your rating is recorded alongside your drill results. Neither alone marks the competency as met.

The problem