I have made a living off prediction markets for 25 years; not as a gambler, but as a scientist and entrepreneur tapping into their uncanny ability to consolidate the informed guesses of a crowd.
Kalshi and Polymarket are handling billions of dollars of wagers during the World Cup. Yet I have watched the recent scandals around them — suspected insider trading on military operations, manipulated temperature readings and the threats to journalists — with particular dismay. Because the loudest defenders of these platforms have built their case on a false claim: Prediction markets only work because traders have real money on the line.
I know it is wrong, because more than 20 years ago, I ran an experiment disproving it.
In 2003, the Pentagon’s Defense Advanced Research Projects Agency made a visionary attempt to use prediction markets for geopolitical forecasting. However, it created a huge controversy in Congress and was quickly killed. Its opponents decried the moral hazards of letting people wager money on national security issues — even saying it could aid terrorists. Its defenders argued that forecasting required real money to serve its predictive purpose. Among the latter was the noted economist Justin Wolfers, then at Stanford.
At the time, I ran a prediction market called NewsFutures, which used play money, offered free participation and gave out very modest cash prizes to the best performers. I challenged Wolfers on an online forum of The Washington Post: Was he ready to bet that predictions on markets that used real money were more accurate than my play-money version?
He gamely accepted, waging a good bottle of California wine against a French vintage.
We teamed with computer scientist David Pennock, a pioneering researcher in prediction markets, to compare over a full American football season — more than 200 games — the predictions of NewsFutures against those of TradeSports, then the largest real-money prediction market operating in the US (it was located in Ireland to avoid US regulations). We also had both markets compete against 1,947 individual forecasters on a third-party contest platform.
From week one, both NewsFutures and TradeSports outclassed 84% of individual forecasters. Twenty weeks later, following the Super Bowl, the play-money market finished in sixth place, while its real-money rival finished eighth.
I soon received a “Stanford Edition” bottle of chardonnay, with an elegant engraving of the campus on its label.
Money did not matter at all.
That conclusion has only been reinforced since. In 2014, I launched a play-money geopolitical market, Hypermind. After nearly 1 million trades on more than 1,000 questions, Hypermind has shown calibration accuracy that the real-money platforms should envy: Events priced at an 80% probability happen very close to 80% of the time, those priced at 25% probability happen 25% of the time, and this holds at every probability level with 99.4% accuracy. (There is a tiny bit of money involved: $5,000 a year in cash prizes, distributed in proportion to everyone’s play-money profits.)
What about events that are really hard to predict? Consider the feverish political years 2016 and 2017 that shocked us with Brexit, Donald Trump's disruptive presidential nomination followed by his astounding November victory, or Emmanuel Macron's fortuitous election as president of France.
I tracked Hypermind's play-money probabilities day by day against two leading real-money markets, PredictIt in the US and Betfair in the UK. PredictIt was highly regulated with strict per-trader wager caps. Betfair was, and still is, an unrestricted real-money market comparable in liquidity and trader count to Polymarket or Kalshi today.
The play-money market, the smallest of the three by far, was the most accurate in all four cases: On average it was 16% better than PredictIt and 17% better than Betfair. Even the real-money competition was impressed: The day before the French presidential runoff, the British bookmaker Ladbrokes tweeted that Hypermind's odds had been “the best guide throughout this.”
I’m hardly alone in understanding that scale and liquidity do not drive accuracy. Design does.
Consider an alternative approach that has taken hold over the last decade: “prediction polling” has delivered amazing results. This technology was refined by the Good Judgment Project, a four-year geopolitical forecasting tournament run in the early 2010s by the Intelligence Advanced Research Projects Activity (IARPA), the research arm of the US intelligence community.
Participants were paid a few hundred dollars to take part in multiyear forecasting tournaments — they estimated the probabilities of future outcomes, updated them as events unfolded, and got scored accordingly. Sophisticated algorithms computed the collective forecasts using weighted averages based on individual track records, forecast recency, how often participants updated their entries, and other considerations.
The collective predictions of these hundreds of amateurs have rivaled those of a prediction market populated by professional intelligence analysts with access to classified information. The best among these amateurs, so-called superforecasters, outperformed the professionals by 30%.
They were not smarter; they were organized differently. Their forecasts were aggregated according to track record, they were trained in probabilities, they worked in teams and, crucially, they were rewarded for being consistently right over time rather than for a single winning bet. “If I were President Obama or John Kerry,” the New York Times columnist David Brooks wrote of the Good Judgment Project in 2013, “I'd want their predictions on my desk.”
Today, the Glimt prediction poll, run by the Swedish Defense Research Agency in collaboration with the government of Ukraine, feeds the forecasts of 20,000 global volunteers directly to Ukrainian intelligence — the first integration of public-crowd forecasting into a state’s wartime decision-making.
In a prediction poll or in a market where everyone starts with equal endowments, the only way to acquire influence is to be consistently correct over many predictions. Therefore, there is a much tighter correlation between insight and influence than in a real-money market, where the big money that moves prices is not necessarily the smart money.
Geopolitics isn’t the only high-stakes domain where this approach has delivered. A year before Covid-19 emerged, Hypermind operated a prediction poll for the Johns Hopkins Center for Health Security, with a total prize budget of a few thousand dollars. Five hundred public-health professionals recruited by the university across 88 nations joined dozens of Hypermind's own champion forecasters to predict the severity of infectious disease outbreaks in real time (e.g., number of individuals or countries impacted).
When Covid emerged from China in early 2020, the platform presciently forecasted its explosive spread. Tara Kirk Sell, the Hopkins project lead, later testified to Congress that the crowd typically called outcomes roughly three weeks ahead of time. When infections spread exponentially, three weeks is an eternity.
Kalshi and Polymarket now lead a multibillion-dollar industry. Their appeal is understandable. But the original promise of prediction markets — harnessing the wisdom of crowds to price the future — is fulfilled equally well by well-designed markets or polls with almost no money involved. These alternatives are immune to the manipulation and fraud that have led to calls in Congress for tighter oversight of prediction markets by the Commodity Futures Trading Commission.
Participants in play-money markets and tournaments respond to other incentives: They revel in the intellectual challenge, the patriotic calling, the peer recognition, or the simple pleasure of being right. For them, excellence is its own reward.
Meanwhile, the science continues to advance: AI bots have reached the accuracy of wise human crowds. “Forecasting machines can help conduct horizon scanning at scale and speed that haven't been possible until now,” notes Rafał Kierzenkowski, head of the Strategic Foresight Unit of the Organization for Economic Cooperation and Development.
Large language models that pack collective intelligence and query the open web provide yet another way to weigh the future without wagers. Well-organized crowds of LLMs can deliver evidence-backed probabilities in minutes. Well-designed tournaments can help make foresight a civic act.
The promise, without the greed.
A message from Advisor Perspectives and VettaFi: Discover something new! Click here to register for our upcoming webcasts.
Bloomberg News provided this article. For more articles like this please visit
bloomberg.com.
Read more articles by Emile Servan-Schreiber
