LLMs in their current form are dangerous to society, but not existential threats. To see the danger of human extinction, multiply some probabilities together:

At some point, AIs will likely become super-intelligent, meaning they are smarter than the entire human race put together. Let's say there's a 50% chance of this happening in the next 30 years. Something this smart would indeed be capable of doing something very clever and killing all humans (if it wanted to); there are various theories about how it might go about doing that, and if this is the part that sounds outlandish to you then I can elaborate. Needless to say, if something is extraordinarily smarter than you, it can figure out a way to kill you that you didn't think of, even if they're sandboxed in a high-security prison. (Mind you, it will probably be exposed like ChatGPT to the general public, not very likely to be sandboxed if you ask me.)

Okay but surely nobody would build an AI that would want to kill us all right? This is the "alignment problem" -- we currently don't know how to make sure an AI has the same "goals" that its creators want it to have. There's that meme -- an AI tasked with optimizing a silverware production process might end up turning the whole universe and everyone in it into spoons. Because almost nobody is actually taking this problem seriously, I think the first superintelligent AI has an 80% chance to be unaligned.

Would an unaligned AI want to kill us? It might be unaligned but conveniently still value human life. Let's say be generous and say it's a 50% chance that the unaligned AI works out in our favour. (It's probably less likely.)

So that's a 20% chance in the next 30 years that someone will create an AI which is clever enough to kill us all, and (by unhappy accident) wants to kill us all. You can put your own numbers on all of these things and get your own probability. If you put very low numbers (like 0.1%) on any of these steps, you should ask yourself if in 2010 you thought it was likely that AI would be where it is today.

Edit: yes I know it sounds absurd and like fantasy, but a lot of real things sound absurd at first. One of the most pervasive arguments against global warming is that it sounds absurd. So if you're going to disagree with this, please at least have a better reason than "it sounds absurd."

[–] qbduubdp@hexbear.net 8 points 3 weeks ago (9 children)

Percentages and probability don't work like that. You can't just make up percentages for no reason to confidently proclaim a probability of an outcome in the future.

AI killing us all discourse is silly and is promoted by the people who want more funding to align AI with their goals to create the modern version of the steam engine to automate away white collar work. Even if it was a likely event, it's an appeal to an unknowable, far away event, which distracts from the very real impacts it already is having today.

One of the most pervasive arguments against global warming is that it sounds absurd

I don't know? Seemed pretty straight forward to me when I was as young as eight.

[–] jsomae@lemmy.ml 1 points 3 weeks ago (8 children)

Probabilities do work like that, if you believe in Bayesian probability anyway. The more we learn about the situation more accurate a probability we can get. The Drake equation works exactly in this way.

It may be true that the discourse is promoted by people who want more funding for AI, but that does not invalidate the point. I was on board with this concern since around 2016 or so, long before LLMs, and I don't have a vested interest in AI. And there is overlap between commonplace concerns about AI and existential ones -- for instance, a moratorium would advance both goals. Frankly, if people see AI as an existential threat, that should be a great boon for other anti-AI parties, no?

Global warming made sense to me when I was 8 too, but it's a common talking point among conservatives that it's ludicrous to suggest that humans could have an impact on something as large as the planet as a whole.

[–] qbduubdp@hexbear.net 2 points 3 weeks ago (1 children)

Another user already touched the Bayesian point, so I'm not going to follow that rabbit.

I was on board with this concern since around 2016 or so, long before LLMs, and I don't have a vested interest in AI.

Ok? AI will become Skynet is such a popular idea that has permeated society since before I was even born, and I'm guessing before you were, or at least was something you were exposed to in your early years. It's frankly not an original thought you came up with in 2016, but rather something you and everyone else has inherited from popular media. Saying we need to slow research on AI to align it with "human values" still allows for this idea that we can control AI to not kill us. Moreover, it allows for the idea that only large companies can align the AI to human values, and the "human values" they are currently aligning it with have nothing to do with saving humanity. Instead, the human values are to reinforce dominant classes in society, accelerate climate change through forcing scale as the only path forward (at least until deepseek dropped), and spark mass layoffs as white collar work is automated away.

We're not going to create a paper clip machine that kills us all because it wants to simply make paper clips. We're going to make a sophisticated bullshit generator whose primary role is to replace labor. Hopefully, I don't need to spell out what this means in a capitalist society which is currently free falling into fascism. We're reaching a point where LLMs have slightly preferable error rates at scale than human workers, and that's the real danger here.

a moratorium would advance both goals. Frankly, if people see AI as an existential threat, that should be a great boon for other anti-AI parties, no?

I'm all for a Butlerian jihad, mount up. I'm not going to join you for a Yudkowskian Jihad, though.

In my view, the danger remains that if the only concern being talked about is AI will kill us all in some fantastical war or apocalyptic scenario, it creates a "hero" (i.e. Sam Altman or some other ghoul) who alone can fix it. The apocalypse argument is not currently pushing anyone towards any moratorium on AI development, but rather just creating a subfield of "alignment" which is more concerned with making sure LLMs don't say mean things, follow the narrative, and don't suggest people use irons to smooth out the wrinkles in their balls.

Global warming made sense to me when I was 8 too, but it's a common talking point among conservatives that it's ludicrous to suggest that humans could have an impact on something as large as the planet as a whole.

This part is tangential, but it actually helps as an allegory to this issue. Exxon new in the late 70s the effects their production would have, that climate change was due to our use of fossil fuels. Rather than act accordingly and pivot away, they protected their profits and muddied the waters by bringing these talking points to media and conservative outlets. Conservatives didn't organically think this is ridiculous, they were told it was absurd by media empires, and they ate it up and spread it.

I get the feeling you are here in good faith, so if you want to read more about the very real, current, actually happening dangers of AI, I would point you to Atlas of AI, Resisting AI, and the work of Bender and Gebru.

[–] jsomae@lemmy.ml 1 points 3 weeks ago* (last edited 3 weeks ago) (1 children)

Those are good points, I'll take a look at the resources you suggested. I think my counter-argument to you right now can basiclaly be summed up as: I do agree that the danger of AI you are talking about is serious and is the more current and pressing concern, but that doesn't really invalidate the X-risk factor of AI. I am not saying that X-risk is the only risk, and your point warning about a "hero" (which I agree with!) also doesn't invalidate the concern. I mean, if it turns out that only a heroic space agency can save us from that asteroid, does that mean the threat from the asteroid isn't real?

[–] qbduubdp@hexbear.net 1 points 3 weeks ago (1 children)

Following the asteroid analogy, I view it as this: If there's a 20% chance that an asteroid could hit us in 2050, does that supplant the threat of climate change today?

I'm not trying to say that AI systems won't kill us all, just that they are using to directly harm entire populations right now and the appeal to a future danger is being used to minimize that discussion.

Another thing to consider: If an AI system does kill us all, it will still be a human or organization that gave it the ability to do so, whether that be through training practices, or plugging it in to weapons systems. Placing the blame on the AI itself absolves any person or organization of the responsibility, which is in line with how AI is used today (i.e. the promise of algorithmic 'neutrality'). Put another way, do the bombs kill us all in a nuclear armageddon or do the people who pressed the button? Does the gun kill me, or does the person pulling the trigger?

[–] jsomae@lemmy.ml 1 points 3 weeks ago* (last edited 3 weeks ago)

On each of your paragraphs:

I think we completely agree -- there can be both a 20% threat of extinction and also the threat of climate change
no, I don't agree with this; that's like saying the threat of the asteroid is used to supplant the threat of climate change. The X-risk threat of AI does not invalidate the other threats of AI, and I disagree with anyone who thinks it does. I have not seen anyone use the X-risk threat of AI to invalidate the other threats of AI, and I implore you to not let such an argument sway you for or against either of those threats, which are both real (!).
I do not blame the gun, I blame the manufacturer. I am calling for more oversight over AI companies and for people who research AI to take this threat more seriously. If an AI apocalypse happens, it will of course be the fault of the idiotic AI development companies who did not take this threat seriously because they were blinded by profits. What did I say that made you think I was blaming the AI itself?

load more comments (6 replies)

load more comments (18 replies)