this post was submitted on 28 Oct 2025
41 points (87.3% liked)

Linux

13757 readers
104 users here now

Welcome to c/linux!

Welcome to our thriving Linux community! Whether you're a seasoned Linux enthusiast or just starting your journey, we're excited to have you here. Explore, learn, and collaborate with like-minded individuals who share a passion for open-source software and the endless possibilities it offers. Together, let's dive into the world of Linux and embrace the power of freedom, customization, and innovation. Enjoy your stay and feel free to join the vibrant discussions that await you!

Rules:

  1. Stay on topic: Posts and discussions should be related to Linux, open source software, and related technologies.

  2. Be respectful: Treat fellow community members with respect and courtesy.

  3. Quality over quantity: Share informative and thought-provoking content.

  4. No spam or self-promotion: Avoid excessive self-promotion or spamming.

  5. No NSFW adult content

  6. Follow general lemmy guidelines.

founded 2 years ago
MODERATORS
MigratingtoLemmy@lemmy.world
41
Anyone messed with local cli LLMs? (sh.itjust.works)
submitted 3 days ago by bridgeenjoyer@sh.itjust.works to c/linux@lemmy.world
23 comments fedilink hide all child comments
 

Let me preface by saying I despise corpo llm use and slop creation. I hate it.

However, it does seem like it could be an interesting helpful tool if ran locally in the cli. I've seen quite a few people doing this. Again, it personally makes me feel like a lazy asshole when I use it, but its not much different from web searching commands every minute (other than that the data used in training it is obtained by pure theft).

Have any of you tried this out?

you are viewing a single comment's thread
view the rest of the comments
[–] tal@lemmy.today 3 points 3 days ago* (last edited 3 days ago) (1 children)

If by "CLI"", you just mean "terminal", I've used ellama in emacs as a frontend to ollama and llama.cpp. Emacs, can run on a terminal, and that's how I use it.

If you specifically want "CLI", I'm sure that there are CLI clients out there. Be almost zero functionality, though.

Usually a local LLM server, what does the actual computation, is a faceless daemon, has clients talk to it over HTTP.

EDIT: llama-cli can run on the commandline for a single command and does the computation itself. It'll probably have a lot of overhead, though, if you're running a bunch of queries in a row

the time to load a model is significant.

[–] shalafi@lemmy.world 2 points 1 day ago (1 children)

What's the difference in a command line interface and a terminal?

[–] tal@lemmy.today 2 points 1 day ago

If you're being rigorous, a "CLI" app is a program that one interacts with entirely from a shell command line. One types the command and any options in (normally) a single line in bash or similar. One hits enter, the program runs, and then terminates.

On a Linux system, a common example would be ls.

Some terminal programs, often those that use the curses/ncurses library, are run, but then one can also interact with them in other ways. This broader class of programs is often called something like "terminal-based" "console-based", or "text-based`, and called "TUI" programs. One might press keys to interact with them while they run, but it wouldn't necessarily be at a command line. They might have menu-based interfaces, or use various other interfaces.

On a Linux system, some common examples might be nano, mc, nmtui or top.

nmtui and nmcli are actually a good example of the split. nmcli is a client for Network Manager that takes some parameters, runs, prints some output, and terminates. nmtui runs in a terminal as well, but one uses it theough a series of menus.