Futurology

1855 readers

94 users here now

founded 1 year ago

MODERATORS

273

Evidence is growing that LLMs will never be the route to AGI. They are consuming exponentially increasing energy, to deliver only linear improvements in performance. (arxiv.org)

submitted 8 months ago by Lugh to c/futurology

67 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] CanadaPlus@lemmy.sdf.org 1 points 8 months ago (1 children)

So does having more parts make something a mystery, like the second paragraph, or not a mystery like the first?

I was a skeptic back in the day too, but they've already far exceeded what an algorithm I could write from memory seems like it should be able to do.

[–] conciselyverbose@sh.itjust.works 1 points 8 months ago (1 children)

A combination of unique, varied parts is a complex algorithm.

A bunch of the same part repeated is a complex model.

Model complexity is not in any way similar to algorithmic complexity. They're only described using the same word because language is abstract.

[–] CanadaPlus@lemmy.sdf.org 1 points 8 months ago* (last edited 8 months ago)

So I guess it comes down to a neurology question. How much algorithmic complexity have we found in the brain?

As far as I'm aware, we've found a few islands of neurons that work together in an obvious way, to track location on a grid for example, and hormone cycles that form a nice negative feedback loop, to keep you at an acceptable blood-sugar level for example. Most of it is still a mystery glob of neurons and other cells, albeit with a fixed pattern of layers and folds.

If we had measured massive algorithmic complexity in the brain, I'd agree with you. As it is, though, it seems unclear how much of the structure we see is conventional algorithm, and how much is the equivalent of an ANN architecture, that ultimately does the same job as no structure but learns more efficiently, or even is just a biological spandrel.