My All-Time Favourite AI Hallucination

Falling into the Rabbit Hole
Down the Rabbit Hole

Did you know that an English speaker has only 167 words at their disposal, unlike us Danish speakers, who can boast of having a whopping 17,241 words? No, right? Nonetheless, that was what Google’s AI, Gemini Advanced, tried to convince me of with characteristic AI assurance when I met it recently.

I asked it in Danish whether it is better to prompt in English rather than a non-English language like Danish, and whether Gemini, like ChatGPT, uses a translation layer when communicating in Danish, so we actually have (much) less memory in our chats.

Read my article: How to Manage an AI’s Limited Memory

Down the Rabbit Hole

And then suddenly, in an otherwise fairly informative conversation, we fell into the infamous rabbit hole. And then, suddenly, in an otherwise fairly informative conversation, we fell into the infamous rabbit hole. First, it claimed that English has a vocabulary of 250,000 words, while Danish has a much smaller vocabulary of 100,000 words.

One might think that that would settle the score. But no, Gemini invents a new interpretation of the linguistic metric "vocabulary density" (normally used to calculate how easy or complex a text is to read) by dividing what it believes to be the number of words in the English language by what it thinks is the number of English speakers, and just like that, English suddenly becomes a tiny and completely useless language making the Babylonian Confusion of Tongues seem like a walk in the park, assigning each English speaker 167 unique and random words that no one else can use:

“English has a population of around 1.5 billion people, while Danish has a population of around 5.8 million people. This means that there are approximately 167 words per person in English and approximately 17,241 words per person in Danish.”

You can see the full Gemini translation of its original Danish response to me in this screenshot:

Apart from being immensely funny, it’s a bit scary that we are integrating AI into every part of our IT infrastructure.

How You Can Reduce Hallucinations

So, here are 5 things you can do to reduce hallucinations when working with AI:

  1. Only prompt about subjects where you have sufficient domain knowledge, so you know how to prompt and how to evaluate the response. This is particularly important with AI automation. And if in doubt, check with reliable sources.

  2. Check if the AI has sufficient knowledge about your subject of interest, and if not, provide it with your data.

  3. Use a bigger model (which often means a paid version) as bigger models tend to hallucinate less than smaller models.

  4. Write longer, well-formed prompts with a detailed AI role description and provide relevant context so the AI knows where to look for relevant, high-quality information in its knowledge base.

  5. Prompt in English as chatbots like Gemini in the example above favour English over non-English languages such as Danish.

    Read my LinkedIn post: Why You Should Prompt in English

Share this post

Picture of Jakob Styrup Brodersen

Jakob Styrup Brodersen

I have worked with data-driven online optimization for 20 years in 5 different industries. Now, I am a freelance CRO and AI consultant: I teach and advice how to utilize the benifits of AI and I do prompt engineering.