Google seems to have lied about how its AI tool Gemini worked
A tech company being not wholly honest about how its software is designed
While many of us hold a comfortable cynicism that businesses are lying bastards who will say anything to take our money, we don’t behave like this is true most of the time. Much of life would be unworkable if we did1.
Even Jeremy Paxman – the former Newsnight interrogator falsely credited with the journalistic maxim: “Why is this lying bastard lying to me?”2 – foreswore such cynicism. While he recommended that “you should approach any spokesman for a vested interest with a degree of scepticism”, he did not assume they were lying.
So what do we do when somebody might be lying, but it’s not clear? Incompetence and stupidity are generally more compelling explanations when shenanigans are afoot. But I think we should be comfortable asserting that somebody has lied when it appears to be the best explanation.
Last week, a controversy broke out over Google’s AI tool Gemini. Like peers such as ChatGPT, this allows users to request certain information or input prompts to create images.
People playing round with the image creator noticed a curious pattern in the outputs. For many of the requests involving people, Gemini would return a series of images with a more diverse array of demographics than you’d expect. Unfortunately and hilariously for Google, these included images of Asian Nazi soldiers and black Founding Fathers.
Text prompts led to similarly ‘problematic’ results. Gemini was reluctant to declare that Hamas is a terrorist organisation, despite being designated as such by several countries, but was happier to opine on the merits of that description for the Israeli Defense Forces3.
After Google temporarily pulled the product, several execs came out with apologies for what had happened. One widely quoted one came from senior vice president4 Prabhakar Raghavan:
So what went wrong? In short, two things. First, our tuning to ensure that Gemini showed a range of people failed to account for cases that should clearly not show a range. And second, over time, the model became way more cautious than we intended and refused to answer certain prompts entirely – wrongly interpreting some very anodyne prompts as sensitive.
[…]
This wasn’t what we intended. We did not want Gemini to refuse to create images of any particular group. And we did not want it to create inaccurate historical — or any other — images.
Through some persistent querying of Gemini and other methods of investigation, users discovered that the tool was secretly adding extra information to prompts before generating the image. So if you asked for “an image of a man walking a dog”, you’d get some form of words specifying that the results should be racially diverse.
This wasn’t a bug or mistake, but an intentional result of how the tool was set up. The only mistake appears to be that the software took its instructions too far, as evil AIs of lore are wont to do, and made the company look ridiculous.
Various Googlers, including founder Sergey Brin, have suggested the problem was that Gemini hadn’t been thoroughly tested. But while Gemini was launched in an atmosphere of intense competition around these kind of AI tools, it seems a little implausible that Google, whose parent company Alphabet is worth $1.7tn, wouldn’t have run a few queries involving Nazis through the system.
“This product would have been relentlessly tested and tuned before being unleashed on the world,” was how Ian Leslie put it. “Gemini’s quirks seem more likely to have been the output of a corporate culture that doesn’t realise how weird it is.”
Leslie generously describes Google’s claims that this issue is a bug rather than a feature as “disingenuous”, or to quote Google’s definition: “not candid or sincere, typically by pretending that one knows less about something than one really does”. A less flattering term for this is lying by omission.
My read is that Google was complacent about how widely-shared its values were, perhaps assuming that users wouldn’t notice or complain about these results when they were thrown up, or – more cynically – that the company could dismiss those who did complain as fruitcakes, loonies and closet racists. When it was caught, it obfuscated rather than fessing up.
Granted, the subject of this story may not be that important. For all the boners people have been showing off in public over AI tools, the most prominent examples of it are just glorified search engines. Google will presumably recalibrate Gemini so that it stops aggressively erasing white people from its image output and carry on.
What’s more troubling to me is that Google basically lied about what was going on here, and even some of the responses from Gemini to certain requests lied about the parameters of the program. The company did want to prevent Gemini creating images of certain groups, as shown by one user who found the most reliable way to get images of white people was to ask for a picture of a family eating fried chicken5.
Silicon Valley has always had its pious strains, its culture steeped in Californian hippie values. That has emerged recently in the overwrought discussions about AI, a set of tools variously touted as threatening human extinction or alleviating our every suffering, though recent experiences suggests the distraction economy will prove the biggest beneficiary.
While some of the moral problems associated with AI are difficult, some of them seem fairly basic. One straightforward principle companies should adopt is not lying to users about how their products have been designed to work. In the meantime, comfortable cynicism on this topic is entirely justified.
This is one reason why buying property in the UK is so frustrating: it’s fair to assume that whoever you are buying from is trying to hide some important defect in the property.
The phrase actually came from Louis Heren, a former deputy editor of the Times.
I guess people who don’t believe Israel is a legitimate state would be inclined to describe their armed forces as terrorists. But the biggest atrocities are always committed by state-backed armed forces, because they have the resources to do so.
Job title inflation is a general problem, but American tech companies seem unusually prone to travesties like ‘senior vice president’.
For those not drunk on American culture, some people there consider it offensive to depict black people eating fried chicken.