LLMs demonstrate knowledge not intelligence.

LLMs are basically giant knowledge databases with no intelligence. They know a lot of things but they don't understand stand any of it.

Because fundamentally they can't understand things. They can only approximate understanding.

It's no different then if I asked someone to prove 2+2=4. They could tell me that yes that's correct, because they know it. But they couldn't explain it properly. Let alone explain why it also isn't always true.

Hallucinations are fundamentally the disconnect from knowledge and understanding. Everything a LLM says from its point of view is "true" it's only stating factual knowledge so far that it has it in its data set.

But with out understanding, it can't tell when that knowledge wanders from a 99% likelihood to be accurate to 60% or 20%. Because it doesn't actually understand what it's saying.

It can only state subjective not objective truth. And subjective truth is no better than a fiction with out understanding.

This is why they need a human minder to keep track of the output to correct it back to optimum output when it starts to wander.