https://arxiv.org/abs/2401.11817

Hallucination is Inevitable: An Innate Limitation of Large Language Models

Ziwei Xu, Sanjay Jain, Mohan Kankanhalli

Hallucination has been widely recognized to be a significant drawback for large language models (LLMs). There have been many works that attempt to reduce the extent of hallucination. These efforts have mostly been empirical so far, which cannot answer the fundamental question whether it can be completely eliminated. In this paper, we formalize the problem and show that it is impossible to eliminate hallucination in LLMs. Specifically, we define a formal world where hallucination is defined as inconsistencies between a computable LLM and a computable ground truth function. By employing results from learning theory, we show that LLMs cannot learn all of the computable functions and will therefore always hallucinate. Since the formal world is a part of the real world which is much more complicated, hallucinations are also inevitable for real world LLMs. Furthermore, for real world LLMs constrained by provable time complexity, we describe the hallucination-prone tasks and empirically validate our claims. Finally, using the formal world framework, we discuss the possible mechanisms and efficacies of existing hallucination mitigators as well as the practical implications on the safe deployment of LLMs.

LLMs are impressive, but they're still just "page-level autocomplete" as DeLong says. They're even worse when humans don't curate the data going into them and just hoover up everything they can find...

Cheers,
Scott.

I think the point is that LLMs are not and cannot be an "intelligence" as commonly understood. They can string together words that might, or might not, be a good approximation of information. Since the supposed point of them is to approximate knowledgeable humans ("they can pass the bar exam!!"), the fact that they cannot be hallucination-free seems to raise red flags. What good is an "expert" that you have to second-guess and double-check when it comes to anything important?

Hacker News thread where someone makes similar points about Google's "AI Overviews".

Yeah, humans make mistakes. But everyone knows that, and that's why "I want to speak to the manager" exists. What "manager" are we going to talk to when everyone we attempt to interact with is just a LLM instance?? We all know the ancient "computers cannot make mistakes" trope that gets trotted out when there are problems. There's a rather infamous example of that still playing out in the UK.

I'm sure there are examples of LLMs that do a decent job (what used to be called "expert systems" seems related, I think), but they didn't crawl TheOnion and Reddit in an attempt to get huge and crush their competitors. The good expert systems builders fed it known-good, or at least best-effort good, information. But even then, humans always have to check the work.

Didn't someone say that once garbage info - like putting glue on pizza, or eating rocks - gets into these LLMs, then it's impossible to get it out? They basically have to retrain it with new data?

We'll see.

Cheers,
Scott.