Even reformatting has caused me issues. My best example is I gave it 100 citations in a non standardized format and asked for MLA. It returned 100 in MLA but randomly 10 of the books were made up. It decided to delete ten I sent at random and make them up instead of just giving me what I sent

[–] PhilipTheBucket@ponder.cat 1 points 1 week ago (1 children)

Oh... yeah, you might have a point. Beyond a certain size of repeated things, it sometimes goes haywire, I've seen that.

[–] ComradeMiao@lemmy.dbzer0.com 2 points 1 week ago (1 children)

I didn’t consider length! That’s a good point too

Another goofy example is I asked it a python question using a specific package import . I sent a big chunk of code. It answered using a package I wasn’t even importing breaking everything. It could never figure it out either lol

[–] PhilipTheBucket@ponder.cat 2 points 1 week ago

Yeah, that kind of thing requires reasoning, and it goes awry almost immediately. It's still pretty useful for generating snippets of boilerplate or finding stuff in big chunks of code, but I more or less gave up on having it actually create anything nontrivial in code.