this post was submitted on 01 Apr 2024
190 points (98.0% liked)

Futurology

3314 readers
29 users here now

founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] CanadaPlus@lemmy.sdf.org 4 points 2 years ago* (last edited 2 years ago)

Well, it's established wisdom that the dataset size needs to scale with the number of model parameters. Quadratically, IIRC. If you don't have that much data the training basically won't work; it will overfit or just not progress.