this post was submitted on 01 Apr 2024
190 points (98.0% liked)
Futurology
3313 readers
28 users here now
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
There’s already more than enough training data out there. The important thing that remains is to filter it so it doesn’t also include humanity’s stupidest data.
That and make the algorithms smarter so they are resistant to hallucination and misinformation - that’s not a data problem, it’s an architecture problem.
Stupid data can be useful for training as a negative example. Image generators use negative prompts to good effect.
Well is the goal truth? Or a simulacrum of a human?
Well, it's established wisdom that the dataset size needs to scale with the number of model parameters. Quadratically, IIRC. If you don't have that much data the training basically won't work; it will overfit or just not progress.
You also have to filter out the AI generated garbage that is rapidly becoming a majority of content on the internet.