Technology

1178 readers

48 users here now

A tech news sub for communists

founded 3 years ago

MODERATORS

muad_dibber@lemmygrad.ml

DeepSeek R1's recipe to replicate o1 and the future of reasoning LMs (www.interconnects.ai)

submitted 7 months ago* (last edited 7 months ago) by yogthos@lemmygrad.ml to c/technology@lemmygrad.ml

3 comments fedilink hide all child comments

R1 utilizes a training method called direct reinforcement learning which is a form of unsupervised learning that forgoes the need for labelled data or explicit solutions. Instead, the model explores various approaches and generates multiple potential answers that are grouped and evaluated using a reward score. This score acts as a fitness function, allowing for learning and adjusting strategies over time. R1 progressively improves its problem-solving abilities by reinforcing successful approaches. This is a similar process to how humans learn to solve problems through trial and error.

you are viewing a single comment's thread
view the rest of the comments

[–] yogthos@lemmygrad.ml 3 points 7 months ago

The key advantage here is that you can see how it arrives at a solution, this is key for being able to guarantee correctness. The core problem with LLMs is that they can't explain how they landed on a particular solution. When the steps are explicitly spelled out, you can review the steps and ask it to fix a specific one. Another possibility going forward could be to allow for directed learning the same way we teach humans. If it gives you the steps you could explain the mistake in a specific step and have it learn to avoid making it going forward.