this post was submitted on 03 Mar 2025
562 points (96.8% liked)
Technology
64937 readers
5580 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Current games have a limit. Current models have a limit. New games could scale until people don't see a quality improvement. New models can scale until people don't see a quality improvement.
I'm supposed to be able to take a model architecture from today, scale it up 100x and get an improvement. I can't make the settings in Crysis 100x higher than they can go.
Games always have a limit, AI is supposed to get better with scale. Which part do you not understand?
You can make Crysis run at higher fps. You can add polygons. (remember ati clown feet?) You can add details to textures. https://research.nvidia.com/publication/2016-06_infinite-resolution-textures
But really the "game" is the model. Throwing more hardware at the same model is like throwing more hardware at the same game.
Which part of diminished returns not offering as much profit did you not understand?
Current models give MS an extra 30% revenue. If they spend billions on a new model will customer pay even more? How much would you pay more for a marginally better AI?
No, it's not! AI models are supposed to scale. When you throw more hardware at them, they are supposed to develop new abilities. A game doesn't get a new level because you're increasing the resolution.
At this point, you either have a fundamental misunderstanding of AI models, or you're trolling.
I don't think you understand how it works at all.
Data is collected. Training is done on the data. Training is done on the trained data (deep seek). You now how a model. That model is a static software program. It requires 700 GB of ram to run (deep seek). Throwing more hardware at the model does nothing but give you a quicker response.
If everyone pays you to use your model, you have no reason to develop a new one. Like Skyrim.
My god.
There are many parameters that you set before training a new model, one of which (simplified) is the size of the model, or (roughly) the number of neurons. There isn't any natural lower or upper bound for the size, instead you choose it based on the hardware you want to run the model on.
Now the promise from OpenAI (from their many papers, and press releases, and ...) was that we'll be able to reach AGI by scaling. Part of the reason why Microsoft invested so much money into OpenAI was their promise of far greater capabilities for the models, given enough hardware. Microsoft wanted to build a moat.
Now, through DeepSeek, you can scale even further with that hardware. If Microsoft really thought OpenAI could reach ChatGPT 5, 6 or whatever through scaling, they'd keep the GPUs for themselves to widen their moat.
But they're not doing that, instead they're scaling back their investments, even though more advanced models will most likely still use more hardware on average. Don't forget that there are many players in this field that keep bushing the bounds. If ChatGPT 4.5 is any indication, they'll have to scale up massively to keep any advantage compared to the market. But they're not doing that.
"training a new model"
Is equivalent to "make a new game" with better graphics.
I've already explained that analogy several times.
If people pay you for the existing model you have no reason to immediately train a better one.