Technology

1253 readers

5 users here now

A tech news sub for communists

founded 3 years ago

MODERATORS

muad_dibber@lemmygrad.ml

How To Run Deepseek R1 671b Fully Locally On a $2000 EPYC Server (digitalspaceport.com)

submitted 1 month ago* (last edited 1 month ago) by yogthos@lemmygrad.ml to c/technology@lemmygrad.ml

21 comments fedilink hide all child comments

A note that this setup runs a 671B model in Q4 quantization at 3-4 TPS, running a Q8 would need something beefier. To run a 671B model in the original Q8 at 6-8 TPS you'd need a dual socket EPYC server motherboard with 768GB of RAM.

you are viewing a single comment's thread
view the rest of the comments

[–] bennieandthez@lemmygrad.ml 10 points 1 month ago (1 children)

Brb gotta start my FundMe campaign for one of these servers lol

[–] FuckBigTech347@lemmygrad.ml 3 points 1 month ago

DW. in like two years from now, companies will start throwing out similar machines. Just keep an eye on second-hand markets and dumpsters.