Futurology

1801 readers

43 users here now

founded 1 year ago

MODERATORS

As Big Tech's AI efforts plateau, in the past 24 hours two open-source AI's have caught up with them. (techcrunch.com)

submitted 4 months ago by Lugh to c/futurology

6 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] Lugh 2 points 4 months ago (1 children)

There is certainly progress to be made with multi-modality, but I wonder if they've already exhausted scaling LLMs based on data.

[–] hendrik@palaver.p3x.de 3 points 4 months ago* (last edited 4 months ago)

That is the big question this year. Scientists certainly need to find some approach, or it's going to be that way. I'm pretty sure they already scraped most of the internet, took most books and infringed on every copyright imaginable. I think OpenAI knows a bit more than we do, since they've probably been working on the next generation for some time already. But it's also discussed amongst scientists:

We're going to find out. I think it's interesting that we have people claiming it'll plateau out soon and won't get much more intelligent. (And I've tried all the AI tools and I don't think they're very intelligent as of now. At least when faced with any of my real-life tasks.) And other people claiming in a few years it's going to be more intelligent than the most intelligent human and in 10 years we look like ants from its perspective.
I think I lean more towards being cautious. I think it's going to be challenging to make progress. And the amount of human generated text available is finite. Maybe we'd need to find a fundamentally different approach to train models. One that works with less many trillion tokens to train a model.