Quality Education / The Thing Being Measured

Why the sector settles for proxies, and why each one is contested

A proxy is an honest admission that the real thing is out of reach.

Nobody chose proxies out of laziness. They were chosen because the alternative was silence. If the actual quantity of interest, the change produced in a person, cannot be observed, then something observable and plausibly correlated has to stand in for it. That substitution is legitimate and unavoidable, and it is also the origin of every argument the sector has about quality. Each proxy is defended by the people it flatters and attacked by the people it does not, and both sides are usually right about the other's proxy and wrong about their own.

What makes proxy disputes so unproductive is that they are conducted as though one contender might eventually win. It will not. Every candidate captures a genuine slice of the thing and misses the rest, and the slices do not add up to a whole no matter how many are collected. A proxy defended as merely indicative on Monday is quoted as decisive on Friday, and the drift from one to the other happens without anybody deciding to allow it. The drift is the problem, more than the proxy, and it happens fastest when the figure is repeated often enough that its origins stop being mentioned.

The useful move is to name the substitution every time. Not as a disclaimer in small print, but in the sentence where the figure appears, so that the reader is reminded what is standing in for what. A figure introduced as evidence about who completed reads differently from the same figure introduced as evidence about who learned, and the difference in framing is the difference between a defensible claim and an overreach. This costs nothing except a clause, and it is remarkable how much argument it prevents later, when somebody wants to use the figure for a purpose it was never built to serve. The habit also protects the people producing the numbers, who are usually the first to know the limits and the last to be asked.