this post was submitted on 22 Jan 2025
13 points (100.0% liked)

Technology

1147 readers
48 users here now

A tech news sub for communists

founded 2 years ago
MODERATORS
muad_dibber@lemmygrad.ml
13
DeepSeek R1's recipe to replicate o1 and the future of reasoning LMs (www.interconnects.ai)
submitted 5 months ago* (last edited 5 months ago) by yogthos@lemmygrad.ml to c/technology@lemmygrad.ml
3 comments fedilink hide all child comments
 

R1 utilizes a training method called direct reinforcement learning which is a form of unsupervised learning that forgoes the need for labelled data or explicit solutions. Instead, the model explores various approaches and generates multiple potential answers that are grouped and evaluated using a reward score. This score acts as a fitness function, allowing for learning and adjusting strategies over time. R1 progressively improves its problem-solving abilities by reinforcing successful approaches. This is a similar process to how humans learn to solve problems through trial and error.

you are viewing a single comment's thread
view the rest of the comments
[โ€“] ksynwa@lemmygrad.ml 5 points 5 months ago (2 children)

Tangential but I understand how these reasoning based systems are supposed to work. I saw some sample output from R1 and it looks like it generates a thought process for answering the prompt before actually answering it. I can see how it would make it more likely to answer more logically but the thinking part was like 2 to 3 times longer than the actual answer.

I am assuming OpenAI, Anthropic etc. are doing something similar. As a concept I see nothing wrong with it but since these services charge per token wouldn't this process balloon the context size? It would make querying them much more expensive.

[โ€“] KrasnaiaZvezda@lemmygrad.ml 5 points 5 months ago

It is more expensive although Deepseek's model is quite cheaper than the others making this much less of a factor. Aditionally these "reasoning models" aren't necessecerilly better for every task though, so for many things a normal, and cheaper model, might be prefered still.

load more comments (1 replies)