Skip to content
Leantensify Learn

Why does adjusting after every result make things worse?

Deming's funnel, running live: choose how to adjust after each result and watch what the adjustment costs. All four rules run on the same drops, so the difference is the rule and not the luck.

The Funnel Experiment · foundations · Foundation · free, no account needed

Use this when

  • Somebody wants to act on every movement in a metric
  • This month's target is being set from last month's actual
  • You need to show a room why leaving a stable process alone is the right answer

The Funnel Experiment

sandbox

A marble drops through a funnel aimed at a target. Where it lands varies by an amount nobody controls. Choose what to do about that — and watch what it costs. All four rules run on the same drops, so any difference between them is the rule and not the luck.

Adjustment rule
Landing positions over 500 drops under rule 2. Standard deviation 1.35 against 0.98 for rule 1 — 1.39 times as wide. Worst miss 4.56 against 3.38.target
Where each marble landed, in order. The faint dashed line is rule 1 — the same drops, left alone.

Spread vs rule 1

× 1.39

Landing σ

1.352

Drop σ

0.975

Worst miss

4.56

Rule 2After each drop, shift the funnel from its current position by the opposite of the error just seen. Entirely reasonable, and it DOUBLES the variance — because you are now adding an adjustment, itself driven by noise, on top of the noise.

Over these 500 drops the spread is 39% wider than leaving the funnel alone, and the worst miss is 1.3× as far out. Nobody did anything careless. Every adjustment was a reasonable response to a real deviation.

This rule settles at 2× the variance (1.41× the standard deviation) however long you run it. Change the drop count and the ratio stays put — that is what “stationary” means.

How this is calculated

Each drop’s noise is an independent draw from a normal distribution with σ = 1, generated from the seed so the run is reproducible. Write d = landing − target. Every rule acts on d — the distance the marble landed from target — never on the noise itself, because the person adjusting can only see where it went.

  • Rule 1: d = e. Variance σ².
  • Rule 2: d = en − en−1. Variance 2σ², settled.
  • Rule 3: dn+1 = −dn + e. Sign flips each drop, amplitude grows without bound.
  • Rule 4: dn+1 = dn + e. A random walk; variance ≈ n·σ².

Source: Deming, W.E. (1994), The New Economics, Ch. 9; Deming (1986), Out of the Crisis, Ch. 11.

How this is calculated

Each drop's noise is an independent normal draw with sigma 1, generated from a seed so the run is reproducible. Writing d = landing - target, every rule acts on d rather than on the noise itself, because the person adjusting can only see where the marble went. Rule 1: d = e, variance sigma squared. Rule 2: d(n) = e(n) - e(n-1), variance 2 sigma squared and stationary. Rule 3: d(n+1) = -d(n) + e, sign flips each drop and the amplitude grows without bound. Rule 4: d(n+1) = d(n) + e, a random walk with variance approximately n sigma squared.

Source: Deming, W.E. (1994), The New Economics, Ch. 9; Deming, W.E. (1986), Out of the Crisis, Ch. 11 — the funnel and the rules for adjustment.

Learn the method