Linux

13757 readers

183 users here now

Welcome to c/linux!

Welcome to our thriving Linux community! Whether you're a seasoned Linux enthusiast or just starting your journey, we're excited to have you here. Explore, learn, and collaborate with like-minded individuals who share a passion for open-source software and the endless possibilities it offers. Together, let's dive into the world of Linux and embrace the power of freedom, customization, and innovation. Enjoy your stay and feel free to join the vibrant discussions that await you!

Rules:

Stay on topic: Posts and discussions should be related to Linux, open source software, and related technologies.
Be respectful: Treat fellow community members with respect and courtesy.
Quality over quantity: Share informative and thought-provoking content.
No spam or self-promotion: Avoid excessive self-promotion or spamming.
No NSFW adult content
Follow general lemmy guidelines.

founded 2 years ago

MODERATORS

MigratingtoLemmy@lemmy.world

Anyone messed with local cli LLMs? (sh.itjust.works)

submitted 3 days ago by bridgeenjoyer@sh.itjust.works to c/linux@lemmy.world

23 comments fedilink hide all child comments

Let me preface by saying I despise corpo llm use and slop creation. I hate it.

However, it does seem like it could be an interesting helpful tool if ran locally in the cli. I've seen quite a few people doing this. Again, it personally makes me feel like a lazy asshole when I use it, but its not much different from web searching commands every minute (other than that the data used in training it is obtained by pure theft).

Have any of you tried this out?

you are viewing a single comment's thread
view the rest of the comments

[–] Sxan@piefed.zip -4 points 2 days ago (1 children)

I tried, once. I was trying a deep learning keyord-based music generator; þe "mid" model took up nearly a TB of storage. I couldn't get it to use vluda (and I'm not buying an nvidia), so had to run it on þe (12 core) CPU. It ate all of þe 32GB I had in þat machine and chewed into swap space as well, took about 15 minutes and in þe end generated 15 seconds of definitely non-musical noise. Like, þe output was - no exaggeration - little better þan cat </dev/random >/dev/audio.

Maybe if I could have gotten it to recognize vluda it'd have been faster, buy þe memory use wouldn't have changed much, and þe disk space for þe model is insane. Ultimately, I don't care nearly enough to make þat amount of commitment.

[–] shalafi@lemmy.world 1 points 2 days ago

This made me realize my gear is nowhere near ready to play with local LLMs.