this post was submitted on 08 Aug 2025
184 points (100.0% liked)
chapotraphouse
13980 readers
757 users here now
Banned? DM Wmill to appeal.
No anti-nautilism posts. See: Eco-fascism Primer
Slop posts go in c/slop. Don't post low-hanging fruit here.
founded 4 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I wanna see the results if you ask ChatGPT the same question a million times. What percentage of responses would actually get the correct number?
I think that heavily depends on whether it gets the initial answer right, since it will use that as context
When you're calling it through an API then you can simply choose not to pass it any context
Fair enough
It depends on the temperature. There's a variable you can play with that adds since randomness to the responses (LLMs are fully deterministic when temperature is 0). Sometimes the F1 or F2 score is used to determine correctness of many questions, but I don't have a great understanding of how that metric works and what ChatGPTs is.