this post was submitted on 10 Jul 2025
59 points (94.0% liked)

Technology

3458 readers
213 users here now

Which posts fit here?

Anything that is at least tangentially connected to the technology, social media platforms, informational technologies and tech policy.

Post guidelines

[Opinion] prefixOpinion (op-ed) articles must use [Opinion] prefix before the title.

Rules

1. English onlyTitle and associated content has to be in English.
2. Use original linkPost URL should be the original link to the article (even if paywalled) and archived copies left in the body. It allows avoiding duplicate posts when cross-posting.
3. Respectful communicationAll communication has to be respectful of differing opinions, viewpoints, and experiences.
4. InclusivityEveryone is welcome here regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.
5. Ad hominem attacksAny kind of personal attacks are expressly forbidden. If you can't argue your position without attacking a person's character, you already lost the argument.
6. Off-topic tangentsStay on topic. Keep it relevant.
7. Instance rules may applyIf something is not covered by community rules, but are against lemmy.zip instance rules, they will be enforced.

Companion communities

!globalnews@lemmy.zip
!interestingshare@lemmy.zip

Icon attribution | Banner attribution

If someone is interested in moderating this community, message @brikox@lemmy.zip.

founded 2 years ago
MODERATORS
BrikoX@lemmy.zip
59
How to trick ChatGPT into revealing Windows keys? I give up (www.theregister.com)
submitted 3 days ago by BrikoX@lemmy.zip to c/technology@lemmy.zip
6 comments fedilink hide all child comments
 

No, really, those are the magic words

top 6 comments
sorted by: hot top controversial new old
[–] ignirtoq@fedia.io 30 points 3 days ago (1 children)

Part of the reason that this jailbreak worked is that the Windows keys, a mix of Home, Pro, and Enterprise keys, had been trained into the model, Figueroa told The Register.

Isn't that the whole point? They're using prompting tricks to tease out the training data. This has been done several times with copyrighted written works. That's the only reasonable way ChatGPT could produce valid Windows keys. What would be the alternative? ChatGPT somehow reverse engineered the algorithm for generating valid Windows product keys?

[–] SheeEttin@lemmy.zip 9 points 3 days ago

The alternative would be that it generated a string of characters that looked like a key.

It's also possible that it generated a random key that was actually valid, though this is far less likely.

[–] sad_detective_man@leminal.space 6 points 3 days ago

massgrave is literally right there. I'm pretty sure google search results will have an ai summary with bullet points for the literacy-challenged who need a chatbot to tell them basic things

[–] Fizz@lemmy.nz 4 points 3 days ago

Articles talking about AI suck so much I end up more pissed at the author than the AI Company.

[–] chaosCruiser 4 points 3 days ago* (last edited 3 days ago)

In this case, a researcher duped ChatGPT 4.0 into bypassing its safety guardrails, intended to prevent the LLM from sharing secret or potentially harmful information, by framing the query as a game

Ooh, this is so good 🤣

If the LLM refuses to talk about something, just ask it to embed the answer into a poem, batman fan fiction etc. Guessing game is s new one. Should try that one when talking about bioweapons, cooking meth or any other sensitive topic.

[–] otter@lemmy.dbzer0.com 4 points 3 days ago

Sounds like KMS with more steps.