Right??? I’m continually floored by how many genuinely smart people I come across who ignore this concept, which is one of the biggest reasons I just don’t trust LLMs in a general sense. Like sure, I can use them fairly effectively, but the vast majority of the people who interact with LLMs don’t use a level of caution with them that’s appropriate.

And that doesn't even touch on the huge ethical (and legal) issues around how LLM devs acquire and use training data.