Two people, one measurement, different answers

Before you can measure anything you have to answer two questions, and skipping either one produces numbers that look fine and cannot be compared.

1. What kind of data is it?

Four kinds, and the kind decides what arithmetic is legitimate.

Continuous — anything on a scale where halves make sense: minutes, millimetres, kilograms, pounds. Means, standard deviations and capability indices are all valid.

Count — whole things that happened: defects, tickets, complaints. Bounded below by zero, never fractional for an individual case, and usually needing a denominator before it can be compared. "42 complaints" is not a measure of anything; "42 complaints per 10,000 orders" is.

Ordinal — ordered categories with no defined distance between them: a 1-to-5 satisfaction scale, a severity grade, a triage priority. You may rank them. You may not average them. "Mean satisfaction 3.7" assumes the distance from 3 to 4 equals the distance from 4 to 5, and no survey has ever established that. Report the distribution, or the proportion at or above a stated point — "62% rated 4 or 5" is checkable in a way that 3.7 is not.

Categorical — named groups with no order at all: supplier, shift, region, failure mode. Count them and compare proportions; nothing else.

Getting this wrong does not produce an error message. It produces a perfectly valid-looking chart of a meaningless number, which is considerably worse.

2. What exactly is being measured?

An operational definition is the sentence that makes a measurement mean the same thing to two different people. Deming's point in Out of the Crisis is blunt: there is no true value of anything. There is only what a stated procedure produces — so the procedure has to be stated.

A definition is finished when it passes one test: could two people, working separately, follow this and get the same answer?

What that requires in practice:

• The characteristic, specifically. Not "responsiveness" — a concept nobody can stand next to and record — but "elapsed time from the user's first contact to the ticket being closed".
• What starts and stops the clock, if it is a duration. Including the edge case: what happens if the stop event never occurs? This is the single most-omitted line in any measurement definition, and the one that caused the ninety-minute meeting above.
• The instrument — the system, log, timestamp, gauge or form that produces the number. A definition that does not say where the number comes from cannot be repeated, and cannot be checked when it looks wrong.
• The units, always. "42" has been read as minutes by one team and hours by another.
• A rule for any judgement, with a boundary case worked through — because the boundary is exactly where two observers diverge. "Reject if visibly scratched" is not a rule. "Reject if a scratch is visible from 30 cm under standard lighting" is.

The words that most often hide a missing definition are the ones that sound like specifications: promptly, appropriate, reasonable, significant, adequate. None can be checked by a second observer, so nobody can be shown to have missed them.

The builder opens with the broken definition and checks it as you type.

1. Read what it says before changing anything. Four findings, each naming the word or the gap that triggered it.

2. Now notice which failure it did not catch: the missing start point — the 27 hours, the whole disagreement. The checker cannot flag it, because "responsiveness of the service desk" never says it is a duration. Change what to "elapsed time from first contact to closure": the abstract-concept finding clears, and a new one appears in its place — no start and stop. Naming the measurement properly is what let the checker see the gap.

   That is worth sitting with. A tool cannot tell you what you failed to say. It can only check what you did say, which is why the first field is the one that matters most.

3. Fix the rest one at a time and watch each finding clear. Start with starts when — it is the one that caused the disagreement, and it is the one nobody thinks to write.

4. Now change the data type to ordinal while the units still say hours. The tool flags the contradiction: the type is not a label, it decides which arithmetic is legitimate.

5. Try writing a definition for something in your own work. Then do the real test, which the tool cannot do for you: hand it to a colleague, have them measure five cases without asking you anything, and compare. If your numbers differ, the definition is not finished — whatever the tool says.