Google’s Bad Gemini Rollout Did the World a Favor

Google’s investors are entitled to be furious about the stunningly incompetent rollout of the company’s Gemini artificial intelligence system. For everybody else, including this grateful Google user and committed technology optimist, it was a blessing.

The laughable screw-ups in the Gemini chatbot’s image-generation — racially diverse Nazi soldiers? —offered a salutary glimpse of an Orwellian dystopia. And in so doing, they also highlighted vital questions of opacity, trust, range of application, and truth that deserve more attention as we contemplate where AI will lead.

AI is a disruptive and potentially transformative innovation — and, like all such innovations, it’s capable of delivering enormous advances in human well-being. A decade or two of AI-enhanced economic growth is just what the world needs. Even so, the exuberance over actually existing AI is premature. The concept is so exciting and the intellectual accomplishment so impressive that one can easily get swept along. Innovators, actual and potential users, and regulators all need to reflect more carefully on what’s going on — and especially on what purposes AI can usefully serve.

Part of the difficulty in grappling with AI’s full implications is the huge effort that has gone into devising AI models that express themselves like humans, presumably for marketing reasons. “Yes, I can help you with that.” Thank you, but who is this “I”? The suggestion is that AI can be understood and dealt with much as one would understand and deal with a person, except that AI is infinitely smarter and more knowledgeable. For that reason, when it comes to making decisions, it claims a measure of authority over its dimwitted users. There’s a crucial difference between AI as a tool that humans use to improve their decisions — decisions for which they remain accountable — and AI as a decision-maker in its own right.

In due course AI will likely be granted ever wider decision-making power, not just over the information (text, video and so forth) it passes to human users but also over actions. Eventually, Tesla’s “full self-driving” will actually mean full self-driving. At that point, liability for bad driving decisions will lie with Tesla. Between advisory AI and autonomous-actor AI, it’s harder to say who or what should be held accountable when systems make consequential mistakes. The courts will doubtless take this up.

Liability aside, as AI advances we’ll want to judge how good it is at making decisions. But that’s a problem too. For reasons I don’t understand, AI models aren’t said to make mistakes: They “hallucinate.” But how do we know they’re hallucinating? We know for sure when they present findings so absurd that even low-information humans know to laugh. But when AI systems make stuff up, they won’t always be so stupid. Even their designers can’t explain all such errors, and spotting them might be beyond the powers of mere mortals. We could ask an AI system, but they hallucinate.