A Boring Dystopia

LLMs don't just regurgitate training data, it's a blend of the material used in the training material. So even if you did somehow assure that every bit of content that was fed in was in and of itself completely objectively true and factual, an LLM is still going to blend it together in ways that would no longer be true and factual.

So either it's nothing but a parrot/search engine and only regurgitates input data or it's an LLM that can do the full manipulation of the representative content and it can provide incorrect responses from purely factual and truthful training fodder.

Of course we have "real" LLM, LLM is by definition real LLM, and I actually had no problem with things like LLM or GPT, as they were technical concepts with specific meaning that didn't have to imply. But the swell of marketing meant to emphasize the more vague 'AI', or the 'AGI' (AI, but you now, we mean it) and 'reasoning' and 'chain of thought'. Having real AGI or reasoning is something that can be discussed with uncertainty, but LLMs are real, whatever they are.