Black Belt · 15 min
Half the runs, and knowing exactly what you gave up
After this you can
- explain what aliasing is and why a fractional design estimates sums of effects rather than single effects.
- read a design's resolution and say which orders of effect it leaves confounded with each other.
- choose a fractional design for a given factor count and run budget, and state what it will not be able to tell me.
- name the follow-up experiment that would separate a confounded pair, and say why the first study cannot.
Assumes you have done Change everything at once, carefully.
The problem
A team screens seven process factors in eight runs — a design the software offered and nobody questioned. Factor C comes back with by far the largest effect, and C is expensive to change. The business case is written.
In that design, C is aliased with fifteen other effects, three of them two-factor interactions. The estimate labelled "C" is the sum of C, AE, BF, DG and twelve more. It might be C. It might be the AE interaction wearing C's name. The eight runs contain nothing that could tell the difference, and the report did not mention it.
The idea
A full factorial doubles with every factor. Five factors is 32 runs, seven is 128, and nobody is funding that to find out which three matter. The Black Belt skill is not running bigger experiments — it is knowing precisely what a smaller one gives up.
The trade is aliasing, and it is exact
A fractional factorial runs a fraction of the combinations: half, a quarter, a sixteenth. You keep the ability to estimate the same number of things — but each estimate is now the sum of two or more effects, and the design contains nothing that could separate them.
That is not a defect or an approximation. It is arithmetic, decided the moment you choose the design, and computable to the letter before a single run happens.
How the fraction is chosen
You pick generators. In a 2⁴ run as eight runs instead of sixteen, you set D = ABC — factor D's level is the product of A, B and C's. That single decision implies the defining relation I = ABCD, and the defining relation determines everything else:
Multiply any effect by the defining word and you get its alias. A × ABCD = BCD, so the estimate you call "A" is really A + BCD. AB × ABCD = CD, so "AB" is AB + CD.
Resolution, which is the number to ask for
Resolution is the length of the shortest word in the defining relation, and it tells you which orders of effect collide:
| What is aliased with what | When to use it | |
|---|---|---|
| III | Main effects with two-factor interactions | Screening many factors on a tiny budget. Cheap and treacherous. |
| IV | Main effects clear; two-factor interactions with each other | The workhorse. Main effects trustworthy, interactions ambiguous in pairs. |
| V | Main effects and two-factor interactions all clear | Nearly as good as a full factorial, for half the runs. |
Resolution III is where the Hook's team went wrong. A large "main effect" in a resolution III design may be a main effect, may be an interaction, and the data cannot say.
The trap in counting the words
A quarter fraction has two generators — and three defining words. The third is their product, and it aliases things as surely as the two you chose. A sixteenth fraction has four generators and fifteen words.
A design assessed on its generators alone always looks cleaner than it is.
What to do about a chain you care about
Nothing, within this experiment. The separation has to come from more runs:
- The complementary fraction. Run the other half of the SAME fraction. For a half fraction the two halves are the full design and every chain splits. For a quarter or smaller, combining two of the four blocks halves the fraction rather than completing it — it removes one defining word and splits each chain in two, leaving the rest confounded.
- A fold-over. Run the same design with some signs reversed. Chosen well, it de-aliases the main effects specifically, which is usually what you want after a resolution III screen.
Both are another experiment. The honest report from a fraction says which chains carry the signal and what it would cost to resolve them — not which member somebody believes in.
Worked example
Montgomery's filtration experiment again, but run as a half fraction: 8 runs instead of 16, with generator D = ABC, so the defining relation is I = ABCD and the design is resolution IV.
Step 1 — read the design before the data.
Every estimate and what it is a sum of, fixed by I = ABCD alone:
| Estimate | Really is |
|---|---|
| A | A + BCD |
| B | B + ACD |
| C | C + ABD |
| AB | AB + CD |
| AC | AC + BD |
| BC | BC + AD |
| ABC | ABC + D |
Main effects are aliased with three-factor interactions — usually negligible, which is what makes resolution IV useful. The two-factor interactions are aliased with each other, which is what makes it imperfect.
Step 2 — the eight runs, and the estimates.
Responses in the design's own order: 45, 100, 45, 65, 75, 60, 80, 96.
| Estimate | Value |
|---|---|
| A + BCD | 19.000 |
| B + ACD | 1.500 |
| C + ABD | 14.000 |
| AB + CD | −1.000 |
| AC + BD | −18.500 |
| BC + AD | 19.000 |
| ABC + D | 16.500 |
Step 3 — check them against the full experiment.
We know the full 2⁴ answers from the Green Belt lesson: A = 21.625, BCD = −2.625. Their sum is 19.000 — exactly the half fraction's estimate for A. Every row checks out the same way: AC = −18.125 and BD = −0.375 sum to −18.500; BC = 2.375 and AD = 16.625 sum to 19.000.
This is not a coincidence or an approximation. The half fraction estimates the sum, always and exactly. With only these eight runs you would have the 19.000 and no way to know it was 21.625 minus 2.625 rather than, say, 10 plus 9.
Step 4 — what the formal test says, and why it is not the answer here.
Run these eight numbers through the tool and nothing is significant. Lenth's method gives a pseudo standard error of 24.750 and a margin of 93.162 — larger than every effect in the table.
That is not a bug and it is not a null result. It is the dense-effects failure mode: Lenth builds its noise floor from the median of the effects themselves, and with seven contrasts of which five are large, the floor rises to meet them. A small unreplicated fraction cannot formally separate signal from noise when most of its contrasts are signal.
Step 5 — so read the half-normal plot, which is how this design is actually analysed.
| Rank | Estimate | |effect| |
|---|---|---|
| 1 | AB + CD | 1.000 |
| 2 | B + ACD | 1.500 |
| 3 | C + ABD | 14.000 |
| 4 | ABC + D | 16.500 |
| 5 | AC + BD | 18.500 |
| 6 | A + BCD | 19.000 |
| 7 | BC + AD | 19.000 |
Two effects sit at 1.0 and 1.5. Five sit between 14.0 and 19.0. There is nothing in between, and that gap is the finding — it is what a half-normal plot makes visible and what no formal test on seven contrasts could deliver.
Step 6 — what a report may honestly say.
Five chains carry the signal: A+BCD, C+ABD, AC+BD, BC+AD and ABC+D. Under the usual assumption that three-factor interactions are negligible, the first two read as A and C and the last as D — a defensible reading that should be labelled as an assumption, because it is one.
The pairs AC+BD and BC+AD cannot be reduced that way: both members are two-factor interactions and neither is more plausible than the other from these runs. Separating them takes the complementary eight runs. That sentence is the deliverable.
Your turn
The tool opens on this half fraction, with the responses already loaded and the alias structure showing beside them.
- Read the alias table first, ignoring the numbers. Everything in it is decided by the design — this is what choosing a design actually means, and none of it depends on a single response.
- Now read the Verdict column: every row says "Noise-sized". Lenth's margin is 93.162, larger than every effect. That is the dense-effects mode, not a null result.
- Read the half-normal plot instead. Two points near the origin, five well above the noise line, nothing in between. That gap is the analysis.
- Confirm A + BCD = 19.000 and AC + BD = −18.500 in the estimates column.
- Switch to 2^(5-1) — resolution V. The main effects (A, B, C, D) are aliased with four-factor interactions and the two-factor interactions with three-factor ones: both clear of anything low-order, for 16 runs instead of 32. The rows the tool flags in red are the high-order estimates — ABCD carries a main effect, and the three-factor estimates carry two-factor interactions. Resolution V buys clean main effects and 2fi, not clean everything.
- Switch to 2^(5-2) — the same five factors in 8 runs, resolution III. Now the main effects are the flagged rows: each is aliased with a two-factor interaction.
- Switch to 2^(6-2) and count the defining words: three, from two generators. The third is their product, and it aliases as surely as the ones you chose.
- Switch to 2^(7-4) — the Hook's design. Read the alias chain for A: fifteen entries, three of them two-factor interactions. Then decide what "the effect of A" could mean in a report.
Fractional factorial (screening design)
practice2^(4-1): 4 factors in 8 runs — one half of the 16 the full design would need. Resolution IV.
Half of a 2^4. Main effects clear of two-factor interactions; the three two-factor pairs are aliased with each other.
- Defining relation
- I = ABCD
- Resolution
- IV
- Runs saved
- 8
What each estimate is a sum of
This design never varied the aliased terms independently, so no arithmetic on its data can separate them. The chains below are fixed by the design, not by the results.
| What this estimate IS | Lowest aliased order | Value | Verdict |
|---|---|---|---|
| A + BCD | 3 | 19.000 | Noise-sized |
| B + ACD | 3 | 1.500 | Noise-sized |
| AB + CD | 2 — confounded with a two-factor interaction | -1.000 | Noise-sized |
| C + ABD | 3 | 14.000 | Noise-sized |
| AC + BD | 2 — confounded with a two-factor interaction | -18.500 | Noise-sized |
| BC + AD | 2 — confounded with a two-factor interaction | 19.000 | Noise-sized |
| ABC + D | 1 — confounded with a main effect | 16.500 | Noise-sized |
Nothing stands out from the noise floor. That is a finding about the factors chosen and the size of this fraction rather than about the process — and with few contrasts and several large effects, the floor itself rises to meet them, so a null reading here is weak evidence either way. In this 2^(4-1) design every estimate is a SUM of its alias chain, not a single effect. It ran 8 of the 16 combinations, so the aliased terms were never varied independently and cannot be separated by any arithmetic on this data. Which member of a chain is doing the work is a question for the next experiment — a fold-over, or the complementary fraction. Naming one of them in a report is a choice made on prior belief, and it should be labelled as one.
- Lenth PSE
- 24.750
- Margin of error
- 93.162
- Contrasts
- 7
This is a resolution IV design: main effects are clear of two-factor interactions, but the two-factor interactions are aliased with each other. A significant interaction estimate names a PAIR of candidates, and separating them needs more runs.
This design is unreplicated, so there is no estimate of experimental error — every degree of freedom went into effects. Significance here comes from Lenth's method, which builds a noise floor from the smaller contrasts themselves. It is a screening judgement, good at finding the effects worth a confirmation run; it is not an F test, and a borderline effect deserves the replicated follow-up rather than a verdict.
| # | A | B | C | D |
|---|---|---|---|---|
| 1 | − | − | − | − |
| 2 | + | − | − | + |
| 3 | − | + | − | + |
| 4 | + | + | − | − |
| 5 | − | − | + | + |
| 6 | + | − | + | − |
| 7 | − | + | + | − |
| 8 | + | + | + | + |
How this is calculated
- The design is built from a base 2k−p factorial in standard order, with each generated factor’s level set by its generator word. Generating the runs this way — rather than filtering the full design and keeping its order — is what makes the estimates match their labels.
- The defining relation is the GROUP generated by the generator words: every product of a subset of them, so a quarter fraction has three words and not two. Forgetting those generalised interactions is the classic way to understate a design’s aliasing.
- Resolution is the length of the shortest defining word. Resolution III aliases main effects with two-factor interactions; IV keeps main effects clear but aliases two-factor interactions with each other; V keeps both clear.
- Every estimate is the sum of its alias chain, computed as the base design’s contrast. The value shown for “A” in a 2(4−1) is A + BCD, and the table says so rather than printing “A”.
- Significance follows the full factorial rules: F tests when replicated, Lenth’s method when not — and with few contrasts and several large effects, Lenth’s floor rises to meet them, so a null reading in a small fraction is weak evidence either way.
Source: Montgomery, D.C., Design and Analysis of Experiments 8e, Ch. 8; Example 8.1 is the preset and the verification fixture. Box, Hunter & Hunter, Statistics for Experimenters 2e, Ch. 6.
Check yourself
No hints. Wrong answers are explained, not softened.
In a 2⁴⁻¹ with I = ABCD, the analysis reports A = 19.0. What has actually been estimated?
A design has defining relation I = ABCE = BCDF = ADEF. What resolution is it, and what does that mean?
Six factors, a budget of 16 runs. Which design, and what should the report warn about?
A resolution III screen finds a large estimate for the chain B + AC + DE. What is the right next step?
Worth remembering
What does a fractional factorial estimate?
The SUM of an alias chain, not a single effect. In a 2⁴⁻¹ with I = ABCD, the estimate labelled A is A + BCD — exactly, by construction, because the design never varied them independently.
How do you find a design's resolution, and what does III, IV, V mean?
The length of the SHORTEST word in the defining relation. III aliases main effects with two-factor interactions; IV keeps main effects clear but aliases two-factor interactions with each other; V keeps both clear.
How many defining words does a quarter fraction have?
Three — the two generators and their product. A sixteenth fraction has fifteen. The generalised interactions alias as surely as the ones you chose, and a design assessed on its generators alone always looks cleaner than it is.
How do you separate a confounded pair?
More runs: the complementary fraction, which splits every chain, or a fold-over, which de-aliases the main effects specifically. Replicating the original design sharpens the estimate of the sum and can never split it.
Can you do this now?
Rate yourself honestly. We compare your rating with how you actually answered — the gap is more useful than either number alone.
I can explain what aliasing is and why a fractional design estimates sums of effects rather than single effects.
I can read a design's resolution and say which orders of effect it leaves confounded with each other.
I can choose a fractional design for a given factor count and run budget, and state what it will not be able to tell me.
I can name the follow-up experiment that would separate a confounded pair, and say why the first study cannot.
Your rating is recorded alongside your drill results. Neither alone marks the competency as met.