In the mid-1990s, a computational mistake was on course to add more than $1 trillion to the U.S. government's debt.
The Bureau of Labor Statistics had been overstating the rise in the cost of living by about 1.1 percentage points a year in its consumer price index. Government statisticians weren't committing fraud or obvious blunders, just unknowingly injecting several biases into the data. Consumers had switched to cheaper goods and outlets when relative prices changed, products improved in ways the index didn't fully credit, and new products entered the index with a lag.
Because federal benefits and tax provisions are tied to the CPI, those biases flowed directly into the federal budget. A commission appointed by the Senate Finance Committee reported in 1996 that, according to Congressional Budget Office estimates, the bias would add more than $1 trillion to the national debt by 2008 if it persisted. That would have made it the fourth-largest federal program, behind only Social Security, healthcare, and defense.
The BLS made changes based on those findings, but the episode revealed something relevant again today: A measuring instrument can be statistically respectable and still be wrong.
Over the past two years, some economists and politicians have criticized the BLS for producing inaccurate inflation and jobs data. They cite large revisions to monthly jobs data, which have grown increasingly significant in recent years, as evidence. These critics are identifying a real issue, but it isn't evidence of nefarious behavior. Similar to the 1990s, the BLS's work is honest and precise, but it is less accurate than one would hope.
The BLS's estimated standard error for the one-month change in CPI is just a few hundredths -- 0.03 to 0.04 -- of a percentage point, which sounds reassuringly precise. But that number only tells us how much the inflation estimate would vary if the bureau repeatedly drew different samples of retail prices using its same methodology. It doesn't indicate whether the methodology itself is sound.
This is the distinction between precision and accuracy. A rifle that puts every shot through the same hole two feet left of the bull's-eye is extremely precise but also completely wrong. Firing more rounds will tighten the group, not move the bullets toward the target.
Similarly, more economic data from a biased instrument can only buy more confidence in the wrong answer. That affects investors, who make trading decisions based on inflation-related outlooks. Everyday Americans are also affected because the CPI feeds into Social Security's annual cost-of-living adjustments. Federal tax provisions are also indexed to CPI measures, as is the principal on Treasury inflation-protected securities.
Physicists like myself call this problem systematic error. Unlike statistical errors, which shrink as you take more readings, systematic errors come from the measuring instrument itself, such as a miscalibrated clock or an assumption built into the apparatus. It sits inside every reading, and repetition only makes you more confident in it.
In 2011, physicists working on a major experiment in Italy reported tiny, subatomic particles called neutrinos arriving about 60 nanoseconds faster than light. The finding was incredible, and the statistics back it were superb, stronger than the threshold physicists normally require to announce a discovery. The cause turned out to be a loose fiberoptic cable in the timing system. No amount of additional data would have show the error. Someone just had to recheck the wiring.
The CPI's "wiring" is its assumptions. The sampling error the BLS reports only covers whichever stores and items it happened to pick. It says nothing about what happens when a price can't be collected and the BLS must fill in the gap by assuming that price moves like prices elsewhere. Nor does it cover how the BLS decides how much of a price increase reflects a better product rather than inflation.
Those are calibration choices. If the numbers are off, they are off in the same direction month after month, and the error never registers. This is why particle physicists publish a result with two error bars, one statistical and one systematic. Markets are only being handed the first.
At one point last year, as much as 40% of the imputed prices in the BLS's commodities-and-services survey were filled in using an imprecise method that method borrows price changes from other areas or broader categories rather than from similar items in the same place. It isn't total guesswork: The BLS uses defined procedures when an individual observation is unavailable. But it is still a different kind of measurement from directly observed prices.
Most months, the bureau's imputations are just patchwork, and enough real prices come in to anchor estimates around them. But that isn't always true.
Because of an extended federal government shutdown last year, the BLS couldn't collect any survey data for October. To calculate CPI, it carried forward commodity and service prices from September and housing observations from April, which recorded rents as unchanged for the month. Months later, the BLS did what a physicist would do to size a systematic error: It reran the numbers under different, equally defensible assumptions and watched how far the answer moved.
Every alternative method produced firmer shelter inflation than the official figures. That spread is the second error bar -- the systematic error. It belongs next to the headline number on release day, not in a research paper published months later.
Luckily, making CPI more accurate doesn't require an agency-wide overhaul. The BLS already publishes its imputation statistics. It can put the relevant share prominently alongside every monthly data release. It can distinguish sampling uncertainty from uncertainty created by missing observations and methodological assumptions. And when an extraordinary event forces the agency to rely heavily on imputation, it can tell users plainly that the measurement regime had to temporarily change.
Physicists learned long ago that a result reported without its systematic error isn't really a solid result. Thirty years ago, it took a Senate commission to show Washington how far its inflation gauge had drifted. This time, the BLS can show us itself, every month.
Guest commentaries like this one are written by authors outside the Barron's newsroom. They reflect the perspective and opinions of the authors. Submit feedback and commentary pitches to ideas@barrons.com.
Nishant Sahdev is a physicist at the University of North Carolina at Chapel Hill and a contributing opinion writer at The Wall Street Journal.