Skip to content
Leantensify Learn

Why do two teams report different numbers for the same thing?

Write a measurement definition precise enough that two people, working separately, get the same number — with the gaps that usually stop that happening flagged as you type.

Operational Definition Builder · measure · Foundation · free, no account needed

Use this when

  • Two reports disagree and both sides are certain their data is right
  • You are about to start collecting data and want it to mean something later
  • Somebody is averaging a 1-to-5 survey scale

Operational Definition

sandbox

One test throughout: could two people, working separately, follow this and get the same answer? Everything below is that question asked a different way.

A characteristic you could stand next to and record — not a concept.

Name the instrument: the system, log, timestamp, gauge or form.

Only for durations. The single most-omitted line in any measurement.

Include the edge case: what happens if it never occurs?

"42" has been read as minutes by one team and hours by another.

Decides which arithmetic is legitimate.

Anything measurable on a scale where halves make sense — minutes, millimetres, kilograms, pounds. Means, standard deviations and capability indices are all valid.

abstract concept"Responsiveness" names a concept, not a measurement. Two people asked to record it will record different things, because they are each measuring their own idea of it. Replace it with the specific characteristic you would actually write down — a time, a count, a dimension.

judgement without rule"happy" asks the observer to judge somebody else's state of mind. If a judgement is genuinely required, write the rule the observer should apply — with a boundary case worked through, because the boundary is where two observers diverge.

no unitsNo units. A number with no unit cannot be compared, aggregated or charted, and "42" has been read as minutes by one team and hours by another more than once.

no instrumentThe method does not say what does the measuring. Name the system, log, timestamp, gauge or form — a definition that does not say where the number comes from cannot be repeated, and cannot be checked when it looks wrong.

How this is calculated

No statistics and no score. It is a structural check, and it is pattern matching over English — English will defeat it. If you think a flag is wrong for your definition, you are probably right; the flags exist to make you look, not to grade you.

  • Words naming a concept rather than something recordable.
  • Qualifiers that sound like a specification and contain no number.
  • Words asking the observer to judge, with no rule for judging.
  • A duration with no stated start or stop.
  • A method that never names the instrument.
  • A declared data type that contradicts the units.
  • An average taken over an ordinal scale.

Source: Deming, W.E. (1986), Out of the Crisis, Ch. 9; Wheeler, D.J. & Chambers, D.S. (1992), Understanding Statistical Process Control, Ch. 2.

Saved runs can be attached to a project deliverable as evidence. Both what you entered and what the tool computed are stored, so the result can be checked again later.

How this is calculated

Structural check, no statistics and no score. Flags abstract nouns with no recordable characteristic, qualifiers that read as specifications but contain no number, words asking the observer to judge without a rule, a duration with no stated start or stop, a method that never names the instrument, a declared data type contradicted by the units, and a mean taken over an ordinal scale.

Source: Deming, W.E. (1986), Out of the Crisis, Ch. 9, on operational definitions; Wheeler, D.J. & Chambers, D.S. (1992), Understanding Statistical Process Control, Ch. 2.

Learn the method