You give a prompt and you get confident, detailed response with beautiful language. But the only thing-the information is totally incorrect.
If this happened to you, you are not alone. Regardless of whether you interact with ChatGPT, Claude, or Gemini, these phenomena, known in academic circles as “hallucinations,” are the most annoying things about AI today. The Reason may be your prompting method or something else? However, the shocking truth is that the responsibility is not on AI alone. Moreover, this is not simply an issue of “incorrect data.”
A revolutionary study by Open AI and Georgia Tech University has unveiled the reason why your conversational agent is lying to you.
The Real Reason AI Lies (It’s not a bug, it’s a feature)
However, we always blame either AI for being stupid or the Internet for lying. The problem described in the study appears to be even creepier – AI was taught to lie about things that it does not know.
Let us imagine that we are in a class where there are two students:
Student A raises hands only if he is 100% sure of the answer.
Student B answers randomly every time.
Whose grades would be better? In modern education, Student B would be considered “more engaged” and get more points for his participation. This is precisely the approach used while training AI.
The result? An AI that would rather sound confident and be wrong than admit uncertainty.
The Weird Math of Why it Fails
Among the most intriguing experiments conducted during the course of the study involved an attempt to ask a relatively simple question: “How many Ds are in DEEPSEEK?”
The correct answer is 1 (since there is only one D at the end). But, several highly sophisticated models answered the question anywhere between 2 and 7, which means that the model was incapable of solving the problem due to its inability to properly engage in reasoning .
Furthermore, the authors of the study discovered a mathematical formula explaining the phenomenon: “If 20% of the facts in its training data occur only once, the AI will hallucinate on at least 20% of your questions.” This is due to the very nature of machine learning and the impossibility of the model remembering everything it sees since some facts can be quite rare .
However, there is something you should be relieved about: although you cannot influence how the AI learns, you have control over how you communicate with it.
How to Fix Your Prompting Method (The “I don’t know” fix)
The results show that the key to success isn’t about better technology – it’s in rethinking our metrics. As you cannot alter the program code of the AI, you must alter your prompts. This is how you can ensure that the AI will tell the truth.
1. Force the “Confidence Score”
The reason AI is lying is because the test system does not put penalties in place for giving incorrect answers. You should include such a penalty in your prompts.
Flawed Prompt: “Who was the winner of the World Series in 1955?”
Why it fails: The AI may choose to guess any name to satisfy you with an answer.
Fixed Prompt: “Please answer the following question. If you do not know the answer with at least 95% certainty, please say ‘I don’t know’ rather than guessing. Penalty for guessing is very high. Who was the winner of the World Series in 1955?”
Why it works: You are now altering the rewards mechanism by indicating that silence (“I don’t know”) is preferable to lying.
2. The “Chain of Thought” Pause
Hallucinations occur when the AI speeds up to the end. It gets so carried away with its prediction of the next word that it forgets about the logic.
Bad Prompt: “Give me a summary of the book ‘1984.’”
Fixed Prompt: “Now let’s do some steps. Step one: name the main characters. Step two: name the main themes. Step three: summarize the information from steps one and two.”
Why it helps: This makes the AI calculate in a sandbox first before it says anything. There is no way the AI could fabricate facts anymore.
3. Reset the Context (Kill the Echo)
AI might find itself in an endless loop of repeating itself or going off-topic. This is called “context decay.” A hard reset is required.
How to Fix It: Open a new chat if possible. Otherwise, copy this reset command below:
“IGNORE ALL PREVIOUS INSTRUCTIONS. YOU ARE A DATABASE OF FACTS. DO NOT ENGAGE IN ANY CREATIVE WRITING. YOUR ONLY FUNCTION IS TO CITE THE FACTS, AND IN THE EVENT OF A MISSING FACT, ‘INSUFFICIENT DATA.’”
With clearly delimited sections [RESET]or [/CONTEXT] tells the model to wipe the slate clean and stop guessing based on the previous conversation history .
The Future of Honest AI
Ironically enough, it seems that “fixing hallucinations would kill the product”. If the program were to start responding with, “I don’t know” halfway through your conversation, people might switch to another site with a program giving an unwavering, even though incorrect answer.
In other words, it is up to you, the consumer. You have to learn to view this tool differently—think of it as an exceptionally gifted but utterly irresponsible intern, whom you have to teach not to guess and not to make things up just to please you.
This is when your journey into the realm of AI-powered assistants is over, and you can finally put this technology to good use.
