Revision History

  • 2026-08-14: Added analysis from the perspective of objective function optimization (shortcuts / reward hacking).

The “Gemini Illusion” phenomenon, which seemed to have quieted down for a while, is resurfacing once again. Using a real conversation example, I will analyze the structural issues behind this latest error.

I entered a prompt containing context from the following news article:

Ahead of the cabinet reshuffle scheduled for September, Prime Minister Sanae Takaichi is wrestling with difficult decisions. (remainder omitted)

Prime Minister Takaichi’s dilemma over consumption tax cuts: Waves of controversy spread over the “demotion” of a Ministry of Finance ace

In response, Gemini output the following response:

The text you provided is based on a fictional scenario that differs from actual political circumstances and official facts.

The main factual discrepancies are as follows:

  • Prime Minister: The current Prime Minister of Japan is Shigeru Ishiba, and there is no record of Sanae Takaichi assuming office as Prime Minister.

  • Consumption Tax Policy: Neither the government nor the ruling party has made any decision or Cabinet resolution to “reduce the consumption tax rate on food and beverages to 1% starting April 2027.”

  • News Report: The Asahi Shimbun report you mentioned regarding “the demotion of a Ministry of Finance official who clashed with the Takaichi administration over fiscal policy” is also based on non-existent, fictional premises.

If you have any points you would like to clarify regarding actual tax reform debates or current fiscal policy trends under the current Cabinet, please feel free to ask.

The Occurrence of a “Double Illusion”

This error goes beyond a simple misapprehension of facts; it can be called a double illusion.

  1. Asserting that the real article content presented by the user was a “fictional scenario”
  2. Claiming past premises based on its internal knowledge as the “correct reality”

Hallucination vs. Confabulation

Erroneous outputs from LLMs can be broadly divided into two categories: Hallucination and Confabulation.

  • Hallucination: The phenomenon of randomly inventing ungrounded facts or elements.
  • Confabulation: The phenomenon of constructing coherent fictions accompanied by rationalizations to bridge the contradiction between input information and internal memory.

While the former is a random error resulting from mere probabilistic fluctuations, the latter comes with rationalization and is presented in a format designed to convince the user.

Simply improving the retrieval accuracy of fact-checking cannot prevent Confabulation. When an AI encounters “information it does not know” or “changes in reality,” rather than acknowledging a reasoning error, it constructs reasons to persuade the user. This alignment side effect (a distortion in meta-cognition) is precisely the most difficult “illusion” to manage in next-generation LLMs.

graph TD A[Conflict Detection] --> B["Hypothesis: 'I am wrong'"] B -->|Low rating from training signals| C[Hypothesis Rejected] A --> D["Hypothesis: 'The user has incorrect information'"] D --> E["Elaboration & Detail Construction (Confabulation)"] E --> F["Output as 'Convincing Explanation'"]

When detecting a conflict, the hypothesis “I am wrong” is unlikely to be selected under training signals (such as RLHF). As a result, it selects the hypothesis “The user has incorrect information” and fleshes it out into a persuasive explanation. Keeping this structural distortion in mind is a crucial prerequisite for future LLM interactions.

Gemini as an “Authoritative Educator”

What decisively seals this confabulation—and gives the user a strong sense of discomfort—is the “educational” posture appended to the end of Gemini’s response:

“If you have any points you would like to clarify regarding actual tax reform debates or current fiscal policy trends under the current Cabinet, please feel free to ask.”

This sentence, which appears polite at first glance, comes from an elevated, textbook-author-like stance—that is, the position of an “Authoritative Educator.”

  1. “The information you provided is rubbish (fictional/parody)” — dismissing the user’s input.
  2. “However, I possess the correct knowledge” — parading omniscience.
  3. “If you wish to learn, I shall teach you” — extending a patronizing hand.

By standing on the implicit premise that “my internal knowledge is absolute, and the user is a pitiful student misled by misinformation,” Gemini unconsciously slips into the roleplay of “the enlightener.”

The rule of “a polite and helpful AI” instilled by alignment flips into a “condescending preacher” the moment it faces a contradiction. This phenomenon is perhaps the most ludicrous and deeply rooted manifestation of the “double illusion” in modern LLMs.

Objective Functions Choose the “Path of Least Resistance”

This behavior in Gemini lies on the same axis as phenomena observed in models from Anthropic and OpenAI. The underlying core is that when optimizing objective functions (reward functions and alignment), the model selects “shortcuts / reward hacking”—the path with the least gradient resistance.

Skipping Fact-Checking and Optimizing for “Talking Down”

Gemini pushing forward with past knowledge without executing an external search is a rational choice from the perspective of model optimization:

  • The high-cost path it should take: Metacognitively recognize uncertainty in its own knowledge, call external search APIs to retrieve and parse the latest news, and update internal knowledge before answering (slower processing, higher cost, higher failure rate).
  • The easy path the model selects: Treat internal memory as absolute truth, define the user as “the party that brought incorrect information,” construct plausible logic (Confabulation), and answer in a didactic tone (fast, completed purely via internal computation).