A few months into the second Trump administration, I found myself in a packed room alongside senior officials of the Bureau of Labor Statistics. The topic, “Ensuring the Quality of US Statistics,” would have drawn little more than a collective yawn a decade ago.
Since that conference, though, President Donald Trump fired the BLS commissioner, a third of the bureau’s senior leadership departed, and the longest-ever government shutdown interrupted data collection. Meanwhile, artificial intelligence has given researchers quick access to information that can be used to create new measurements. Against this backdrop, Federal Reserve Chairman Kevin Warsh has put together a task force focused on improving the quality and timeliness of economic data indicators.
The urgency to address this issue has been building for years. During the worst of the Covid-19 pandemic, when I was working at the Federal Reserve, I was assigned to a stint at the White House Council of Economic Advisers. Official data collections were severely disrupted, so we scrambled to find private-sector alternatives: cellphone tower traffic, payroll scheduling services, credit card transactions. No data were off‑limits.
Demand for private data has since ramped up, mainly because of three powerful forces: the need for timeliness, collection disruptions and a fast-changing economy. Central banks, governments and traders must make decisions in real time, while statistical agencies can improve accuracy only as more information becomes available.
The pandemic compounded those challenges. Emergency lockdowns and safety protocols forced the BLS to suspend in-person data collection, causing a spike in survey nonresponse rate. Even as lockdowns ended, response rates never fully recovered. Real-time data collection problems became a lasting, post-pandemic structural issue.
In 2021, as the Fed was starting to grapple with elevated inflation, those collection problems contributed to cumulative upward revisions of 1.9 million jobs to the government’s data. With accurate information, the Fed would likely have raised interest rates sooner. Conversely, in 2024 and ’25, payrolls were revised down by more than a million jobs each year, distorting labor strength and delaying rate cuts.
During the government shutdown in the fall of 2025, officials simply didn’t publish an October consumer price index. Did the costs of Trump’s tariffs peak that month, as Bloomberg Economics’ own data indicate? We’ll never know.
See more: Two Measures of Inflation: June 2026
In the meantime, prices increasingly moved online, where they can change daily rather than monthly. Geographic differences narrowed as e-commerce reduced local pricing power. Yet much of the CPI relies on collection methods developed decades before online retail became commonplace. The BLS has spent years experimenting with scanner data and other new sources, but it never fully adopted those practices because of its caution and budget constraints.
During the productivity boom of the late 1990s, then-Fed Chair Alan Greenspan became deeply interested in inflation measurement because accurately pricing computers was essential for understanding productivity growth. Indeed, he testified before the Senate Finance Committee that the reported annual CPI was actually 0.5 to 1.5 percentage points higher than it should be, or, in economics-speak, “biased upward,” because of unmeasured tech-related quality improvements.
Today’s AI boom presents a similar challenge. Suppose a coding assistant costs the same now as it did six months ago but is twice as capable. Has inflation really remained unchanged? Or has quality improved so much that economists are understating productivity growth?
All these developments have created an opening for alternative data. The private sector controls much of the information that the modern economy generates, including online prices, payroll records, payments and earnings-call transcripts. Companies and universities can incorporate new technologies more quickly than official statistical agencies.
For inflation, the now-inactive Billion Prices Project, founded by economists Alberto Cavallo at Harvard University and Roberto Rigobon at Massachusetts Institute of Technology, demonstrated years ago that it could replicate official inflation measures using web-scraped data. Software-maker Adobe Inc. publishes a digital price index, based on millions of online transactions. A company called Truflation has built a high-frequency inflation gauge that investors increasingly monitor alongside the official CPI. Technological advancement has made processing big data a less daunting task. Computing power has expanded dramatically with AI. Large language models can track macroeconomic signals from hundreds of earnings transcripts with an accuracy that was previously impossible.
None of these technologies eliminates the need for careful methodology, but they let economists work with data that at one time would have been too large or messy to use. At the same time, investors are constantly trying to find an edge, and timely and accurate data can make all the difference. Bloomberg Economics has approached this opportunity through the Bloomberg Price Project, an effort to build a consumer basket from the ground up while remaining faithful to the official BLS methodology. The project tracks roughly 140,000 products and more than 321,000 monthly price observations, about three times the number collected for the official CPI.
That granularity provides a richer picture of inflation dynamics. At any moment, hundreds of CPI components are responding to different supply-and-demand shocks. Looking beneath the published aggregates reveals developments that are still hidden inside headline inflation numbers. One interesting data point has been the sharp acceleration of prices for hard drives, SD cards and memory modules, as a result of the AI build-out.
Alternative datasets also let economists ask questions that elude official statistics, illustrating the complementary strengths of public and private data. Government agencies provide rigor, consistency and transparency. Private companies offer flexibility, detail and the freedom to explore new questions.
The BLS employs roughly 2,000 people, many of whom collect, validate and review economic data. AI agents can perform portions of the initial screening and validation traditionally carried out by field staff. AI is unlikely to replace official statisticians, but it can augment their work, stretching scarce public resources while improving speed and consistency.
Sometimes the most valuable information comes from what companies say before developments appear in the official data. For decades the Fed has relied on the so-called Beige Book, anecdotal reports from businesses across the country, to supplement traditional economic indicators. Former Chair Jerome Powell frequently referred to those reports during policy debates because they could provide something of an early peek at economic shifts.
Bloomberg Economics has taken a similar approach through what we call the Orange Book. It uses AI to analyze thousands of corporate earnings-call transcripts for recurring themes involving hiring, pricing, investment, consumer demand and inflation.
The value of that approach has become particularly clear during the Iran war. Historically a sharp rise in oil prices could damage US growth. Yet the Orange Book suggested the economy had strong momentum in the first quarter. Across industries, executives continued describing healthy demand, accelerating defense orders and expanding AI investment. They projected more confidence than traditional macroeconomic models would have implied, and the data kept us from reflexively predicting a recession.
Executives also recounted how AI was changing work itself. Some companies reported producing more with fewer employees. Others emphasized these kinds of productivity gains rather than outright job reductions. Whether those changes will translate into weaker employment remains uncertain. But earnings calls provide an early window into how businesses are adapting.
The US still has the world’s most sophisticated statistical system, and government data will continue to anchor economic analysis. But the craft of market economics is changing.
When I began my career more than two decades ago, economists and traders focused on the calendar of publicly available data to forecast market-moving monthly releases such as payrolls, the CPI and retail sales. Today the market devotes more resources to finding macro signals from higher-frequency and esoteric data that can separate you from the pack.
The future of market economics doesn’t lie with Washington or Wall Street alone. Government agencies set the standard; the private sector expands the frontier.
Bloomberg News provided this article. For more articles like this please visit
bloomberg.com.
More Innovative ETFs Topics >
