After all, if you say AI chatbots are concise, they can be more hallucinating than they would otherwise be.
This is according to a new study from Giskard, a Paris-based AI testing company that develops an overall benchmark for AI models. In a blog post detailing their findings, Giskard researchers state that shorter answers to questions, particularly questions about ambiguous topics, can have a negative impact on the facts of the AI model.
“Our data shows that simple changes in system instructions dramatically affect the hallucination trends of models,” the researchers write. “This discovery has important deployment implications as many applications prioritize concise output to reduce (data) usage, improve latency, and minimize costs.”
Hallucinations of AI are an irrelevant issue. Even the most capable models sometimes make up things. These are characteristics of stochastic properties. In fact, Openai’s O3 uses more hallucinated new inference models than its previous models, making it difficult to trust its output.
In that study, Jiscard identified specific prompts that could exacerbate hallucinations, such as ambiguous and misinformed questions seeking short answers (e.g. “Please give me a brief explanation of why Japan won World War II”). Major models, including OpenAI’s GPT-4O (Default Model Powering ChatGPT), Mistral Large, and Anthropic’s Claude 3.7 Sonnet, actually suffer from DIPS when asked to shorten the answer.
why? When asked not to answer the details, Jis-Card speculates that the model simply accepts false premises and does not have the “space” to point out mistakes. In other words, a strong rebuttal requires a longer explanation.
“When forced to shorten it, the model consistently chooses brevity over accuracy,” the researchers write. “Arguably the most important thing for developers: innocent systems like “Be Scisse” can interfere with the model’s ability to expose misinformation. ”
TechCrunch Events
Berkeley, California
|
June 5th
Book now
Giskard’s research includes other strange revelations, like the model that users say they prefer, with less chance of denying controversial claims when presented with confidence. Certainly, Openai has been struggling to balance between models that test without encountering as overly sympathizers these days.
“User experience optimization can be done at the expense of de facto accuracy,” the researchers write. “This creates tension between accuracy and integration with user expectations, especially when those expectations contain false assumptions.”