Best responses so far in this thread. People need to stop calling these 'whatever'-intelligence engines. They're NOT INTELLIGENT. They are probabilistic text-synthesizers. Stochastic Parrots. Or one of many other terms people have begun using, any of which are much more accurate and do not incorrectly assign 'intelligence', agency or motivations to what is still only an algorithm. One with 'temperature' (randomness) parameters thrown in, to jiggle the retrieved phrasing of sentences somewhat, and so on.

The reason these things always 'hallucinate' (some are advocating we drop that term as well, and simply say the algorithms yield incorrect results) can NOT be fixed, as it's a side-effect of the fundamental way they synthesize their answers.