Google AI Overviews hallucinated more than Ken Kesey when it first launched in 2024, fabricating facts about drizzling “glue on pizza,” among other offences.
And though the Gemini-powered technology has improved its accuracy dramatically over the last two years (unfortunately for publishers), AI Overviews still gets basic questions wrong. And that includes spelling tests.
Google’s AI tools remain abysmal at answering questions about spelling, having gone viral two years ago for responding to the question “how many r’s are in the word strawberry?” incorrectly. But it’s still bad. On Tuesday, X user Naomi Rohatyn tested the large language model’s (LLM) current ability to answer to a spelling question.
“How many e’s in the word astronomical?” they asked.
“There are exactly 2 ‘e’s in the word “astronomical” (a-s-t-r-e-n-o-m-i-c-a-e-l),” replied AI Overview.
This Tweet is currently unavailable. It might be loading or has been removed.
We tried it ourselves, getting the same answer.
Credit: Google / Mashable
It appears to work for any word with four or more syllables:
Credit: Google / Mashable
Credit: Google / Mashable
Suffice to say, people are having another gleeful round of testing the technology’s spelling weakness, sharing their findings on social media.
This Tweet is currently unavailable. It might be loading or has been removed.
This Tweet is currently unavailable. It might be loading or has been removed.
This Tweet is currently unavailable. It might be loading or has been removed.
Why can’t Google AI Overviews spell correctly?
I don’t want to be all Billy Madison spelling bee about this, but considering users are less likely to click on links when an AI summary appears in the results, surely the information provided in AI Overviews should be accurate. But it’s complicated.
AI chatbots need exact context and specifics to answer as well as they can, so surely spelling words within their training data seems easy. However, things get knotty when you ask an LLM to consider words letter-by-letter, as the model will process text in chunks rather than individual characters (it’s called tokenisation).
I thought I’d ask Google’s Gemini itself, to see what it had to say for itself. Defend your poor spelling!
“The short answer is that I don’t look at or write text the way you do,” Gemini told me. “When you write the word ‘apple,’ your brain processes five distinct letters. When I read or write text, I see the word as a single unit called a token (a numerical representation of a word or part of a word).
“Because I process words as whole blocks of meaning rather than strings of individual letters, I don’t naturally ‘spell’ out words sequentially. I know exactly what ‘apple’ means and how it relates to other words, but I don’t inherently focus on the fact that it contains two ‘p’s unless specifically asked to break it down.”
Mashable has reached out to Google for human comment.
