this post was submitted on 12 Dec 2024
45 points (81.7% liked)

Selfhosted

40708 readers
476 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS
 

Hello! πŸ˜€
I want to share my thoughts on docker and maybe discuss about it!
Since some months I started my homelab and as any good "homelabing guy" I absolutely loved using docker. Simple to deploy and everything. Sadly these days my mind is changing... I recently switch to lxc containers to make easier backup and the xperience is pretty great, the only downside is that not every software is available natively outside of docker πŸ™ƒ
But I switch to have more control too as docker can be difficult to set up some stuff that the devs don't really planned to.
So here's my thoughts and slowly I'm going to leave docker for more old-school way of hosting services. Don't get me wrong docker is awesome in some use cases, the main are that is really portable and simple to deploy no hundreds dependencies, etc. And by this I think I really found how docker could be useful, not for every single homelabing setup, and it's not my case.

Maybe I'm doing something wrong but I let you talk about it in the comments, thx.

top 50 comments
sorted by: hot top controversial new old
[–] beerclue@lemmy.world 31 points 1 week ago

I'm actually doing the opposite :)

I've been using vms, lxc containers and docker for years. In the last 3 years or so, I've slowly moved to just docker containers. I still have a few vms, of course, but they only run docker :)

Containers are a breeze to update, there is no dependency hell, no separate vms for each app...

More recently, I've been trying out kubernetes. Mostly to learn and experiment, since I use it at work.

[–] PhilipTheBucket@ponder.cat 30 points 1 week ago (2 children)

It's hard for me to tell if I'm just set in my ways according to the way I used to do it, but I feel exactly the same.

I think Docker started as "we're doing things at massive scale, and we need to have a way to spin up new installations automatically and reliably." That was good.

It's now become "if I automate the installation of my software, it doesn't matter that the whole thing is a teetering mess of dependencies and scripted hacks, because it'll all be hidden inside the container, and also people with no real understanding can just push the button and deploy it."

I forced myself to learn how to use Docker for installing a few things, found it incredibly hard to do anything of consequence to the software inside the container, and for my use case it added extra complexity for no reason, and I mostly abandoned it.

[–] Croquette@sh.itjust.works 10 points 1 week ago (1 children)

I hate how docker made it so that a lot of projects only have docker as the official way to install the software.

This is my tinfoil opinion, but to me, docker seems to enable the "phone-ification" ( for a lack of better term) of softwares. The upside is that it is more accessible to spin services on a home server. The downside is that we are losing the knowledge of how the different parts of the software work together.

I really like the Turnkey Linux projects. It's like the best of both worlds. You deploy a container and a script setups the container for you, but after that, you have the full control over the software like when you install the binaries

[–] Strit@lemmy.linuxuserspace.show 11 points 1 week ago (2 children)

I hate how docker made it so that a lot of projects only have docker as the official way to install the software.

Just so we are clear on this. This is not dockers fault. The projects chose Docker as a distribution method, most likely because it's as widespread and known as it is. It's simply just to reach more users without spreading too thin.

[–] Croquette@sh.itjust.works 4 points 1 week ago

You are right and I should have been more precise.

I understand why docker was created and became popular because it abstracts a lot of the setup and make deployment a lot easier.

load more comments (1 replies)
load more comments (1 replies)
[–] mesamunefire@lemmy.world 10 points 1 week ago* (last edited 1 week ago) (2 children)

Honestly after using docker and containerization for more than a decade, my home setups are just yunohost or baremetal (a small pi) with some periodic backups. I care more about my own time now than my home setup and I want things to just be stable. Its been good for a couple of years now, without anything other than some quick updates. You dont have to deal with infa changes with updates, you dont have to deal with slowdowns, everything works pretty well.

At work its different Docker, Kubernetes, etc... are awesome because they can deal gracefully with dependencies, multiple deploys per day, large infa. But ill be the first to admit that takes a bit more manpower and monitoring systems that are much better than a small home setup.

[–] WeAreAllOne@lemm.ee 5 points 1 week ago (1 children)

I tend to also agree with your opinion,but lately Yunohost have quite few broken apps, they're not very fast on updates and also not many active developers. Hats off to them though because they're doing the best they can !

[–] mesamunefire@lemmy.world 4 points 1 week ago (1 children)

I have to agree, the community seems to come and go. Some apps have daily updates and some have been updated only once. If I were to start a new server, I would probably still pick yunohost, but remove some of the older apps as one offs. The lemmy one for example is stuck on a VERY old version. However the GotoSocial app is updated every time there is an update in the main repo.

Still super good support for something that is free and open source. Stable too :) but sometimes stability means old.

[–] foremanguy92_@lemmy.ml 4 points 1 week ago (1 children)

Didn't really tried YunoHost it's basically a simple selfhostable cloud server?

[–] mesamunefire@lemmy.world 3 points 1 week ago (1 children)

Basically. It's just Ubuntu server with some really good niceties.

load more comments (1 replies)
[–] foremanguy92_@lemmy.ml 3 points 1 week ago (4 children)

yeah I think that at the end even if it seems a bit "retro" the "normal install" with periodic backups/updates on default vm (or even lxc containers) are the best to use, the most stable and configurable

load more comments (4 replies)
[–] ikidd@lemmy.world 9 points 1 week ago (1 children)

Are you using docker-compose and local bind mounts? I'd not, you're making backing up uch harder than it needs to be. Its certainly easier than backing up LXCs and a whole lot easier to restore.

load more comments (1 replies)
[–] CameronDev@programming.dev 9 points 1 week ago (4 children)

Are you using docker compose scripts? Backup should be easy, you have your compose scripts to configure the containers, then the scripts can easily be commited somewhere or backed up.

Data should be volume mounted into the container, and then the host disk can be backed up.

The only app that I've had to fight docker on is Seafile, and even that works quite well now.

load more comments (4 replies)
[–] InnerScientist@lemmy.world 7 points 1 week ago (1 children)

I use podman using home-manager configs, I could run the services natively but currently I have a user for each service that runs the podman containers. This way each service is securely isolated from each other and the rest of the system. Maybe if/when NixOS supports good selinux rules I'll switch back to running it native.

[–] agile_squirrel@lemmy.ml 2 points 1 week ago (1 children)

This sounds great! I'd love to see your config. I'm not using home manager, but have 1 non root user for all podman containers. 1 user per service seems like a great setup.

[–] InnerScientist@lemmy.world 2 points 1 week ago* (last edited 1 week ago)

Yeah it works great and is very secure but every time I create a new service it's a lot of copy paste boilerplate, maybe I'll put most of that into a nix function at some point but until then here's an example n8n config, as loaded from the main nixos file.

I wrote this last night for testing purposes and just added comments, the config works but n8n uses sqlite and probably needs some other stuff that I hadn't had a chance to use yet so keep that in mind.
Podman support in home-manager is also really new and doesn't support pods (multiple containers, one loopback) and some other stuff yet, most of it can be compensated with the extraarguments but before this existed I used pure file definitions to write quadlet/systemd configs which was even more boilerplate but also mostly copypasta.

Gaze into the boilerplate

{ config, pkgs, lib, ... }:

{
    users.users.n8n = {
        # calculate sub{u,g}id using uid
        subUidRanges = [{
            startUid = 100000+65536*( config.users.users.n8n.uid - 999);
            count = 65536;
        }];
        subGidRanges = [{
            startGid = 100000+65536*( config.users.users.n8n.uid - 999);
            count = 65536;
        }];
        isNormalUser = true;
        linger = true; # start user services on system start, fist time start after `nixos-switch` still has to be done manually for some reason though
        openssh.authorizedKeys.keys = config.users.users.root.openssh.authorizedKeys.keys; # allows the ssh keys that can login as root to login as this user too
    };
    home-manager.users.n8n = { pkgs, ... }:
    let
        dir = config.users.users.n8n.home;
        data-dir = "${dir}/${config.users.users.n8n.name}-data"; # defines the path "/home/n8n/n8n-data" using evaluated home paths, could probably remove a lot of redundant n8n definitions....
    in
    {
        home.stateVersion = "24.11";
        systemd.user.tmpfiles.rules =
        let
            folders = [
                "${data-dir}"
                #"${data-dir}/data-volume-name-one" 
            ];
            formated_folders = map (folder: "d ${folder} - - - -") folders; # a function that takes a path string and formats it for systemd tmpfiles such that they get created as folders
        in formated_folders;

        services.podman = {
            enable = true;
            containers = {
                n8n-app = { # define a container, service name is "podman-n8n-app.service" in case you need to make multiple containers depend and run after each other
                    image = "docker.n8n.io/n8nio/n8n";
                    ports = [
                        "${config.local.users.users.n8n.listenIp}:${toString config.local.users.users.n8n.listenPort}:5678" # I'm using a self defined option to keep track of all ports and uids in a seperate file, these values just map to "127.0.0.1:30023:5678", a caddy does a reverse proxy there with the same option as the port.
                    ];
                    volumes = [
                        "${data-dir}:/home/node/.n8n" # the folder we created above
                    ];
                    userNS = "keep-id:uid=1000,gid=1000"; # n8n stores files as non-root inside the container so they end up as some high uid outside and the user which runs these containers can't read it because of that. This maps the user 1000 inside the container to the uid of the user that's running podman. Takes a lot of time to generate the podman image for a first run though so make sure systemd doesn't time out
                    environment = {
                        # MYHORSE = "amazing";
                    };
                    # there's also an environmentfile option for secret management, which works with sops if you set the owner of the secret/secret template
                    extraPodmanArgs = [
                        "--pull=newer" # always pull newer images when starting, I could make this declaritive but I haven't found a good way to automagically update the container hashes in my nix config at the push of a button.
                    ];
                 # few more options exist that I didn't need here
                };
            };
        };
    };
}

[–] markc@lemmy.world 7 points 1 week ago (1 children)

Docker is a convoluted mess of overlays and truly weird network settings. I found that I have no interest in application containers and would much prefer to set up multiple services in a system container (or VM) as if it was a bare-metal server. I deploy a small Proxmox cluster with Proxmox Backup Server in a CT on each node and often use scripts from https://community-scripts.github.io/ProxmoxVE/. Everything is automatically backed up (and remote sync'd twice) with a deduplication factor of 10. A Dockerless Homelab FTW!

load more comments (1 replies)
[–] Sibbo@sopuli.xyz 6 points 1 week ago (3 children)

I can recommend NixOS. It's quite simple if your wanted application is part of NixOS already. Otherwise it requires quite some knowledge to get it to work anyways.

[–] hendrik@palaver.p3x.de 15 points 1 week ago* (last edited 1 week ago) (1 children)

Yeah, It's either 4 lines and you got some service running... Or you need to learn a functional language, fight the software project and make it behave on an immutable filesystem and google 2 pages of boilerplate code to package it... I rarely had anything in-between. πŸ˜†

[–] InnerScientist@lemmy.world 7 points 1 week ago

Hey now, you can also spend 20 pages of documentation and 10 pages of blogs/forums/github^1^ and you can implement a whole nix module such that you only need to write a further 3 lines to activate the service.

1 Your brain can have a little source code, as a threat.

load more comments (2 replies)
[–] gaylord_fartmaster@lemmy.world 4 points 1 week ago (1 children)

Just run docker in an LXC. That's what I do when I have to.

load more comments (1 replies)
[–] Auli@lemmy.ca 4 points 1 week ago (1 children)

And I've done the exact opposite moves everything off of lxc to docker containers. So much easier and nicer less machines to maintain.

[–] foremanguy92_@lemmy.ml 1 points 6 days ago

Less "machines" but you need to maintain docker containers at the end

[–] huskypenguin@sh.itjust.works 3 points 1 week ago (9 children)

I love docker, and backups are a breeze if you're using ZFS or BTRFS with volume sending. That is the bummer about docker, it relies on you to back it up instead of having its native backup system.

load more comments (9 replies)
[–] SpazOut@lemmy.world 3 points 1 week ago (1 children)

For me the power of docker is its inherent immutability. I want to be able to move a service around without having to manual tinker, install packages and change permissions etc. It’s repeatable and reliable. However, to get to the point of understanding enough about it to do this reliably can be a huge investment of time. As a daily user of docker (and k8s) I would use it everyday over a VM. I’ve lost count of the number of VMs I’ve setup following installation guidelines, and missed a single step - so machines that should be identical aren’t. I do however understand the frustration with it when you first start, but IMO stick with it as the benefits are huge.

[–] foremanguy92_@lemmy.ml 1 points 6 days ago (1 children)

Yeah docker is great for this and it's really a pleasure to deploy apps so quickly but the problems comes later, if you want to really customize the service to you, you can't instead of doing your own image...

[–] SpazOut@lemmy.world 2 points 6 days ago

In most cases you can get away with over mounting configuration files within the container. In extreme cases you can build your own image - but the steps for that are just the changes you would have applied manually on a VM. At least that image is repeatable and you can bring it up somewhere else without having to manually apply all those changes in a panic.

[–] Neptr@lemmy.blahaj.zone 2 points 1 week ago* (last edited 1 week ago)

Docker is good when combined with gVisor runtime for better isolation.

What is gVisor?gVisor is an application kernel, written in memory safe Golang, that emulates most system calls and massively reduces the attack surface of the kernel. This is important since the host and guest share the same kernel, and Docker runs rootful. Root inside a Docker container is the same as root on the host, as long as a sandbox escape is used. This could arise if a container image requires unsafe permissions like Docker socket access. gVisor protects against privilege escalation by only using root at the start and never handing root over to the guest.

Sydbox OCI runtime is also cool and faster than gVisor (both are quick)

[–] 2xsaiko@discuss.tchncs.de 2 points 1 week ago (2 children)

Yeah, when I got started I initially put everything in Docker because that's what I was recommended to do, but after a couple years I moved everything out again because of the increased complexity, especially in terms of the networking, and that you now have to deal with the way Docker does things, and I'm not getting anything out of it that would make up for that.

When I moved it out back then I was running Gentoo on my servers, by now it's NixOS because of the declarative service configuration, which shines especially in a server environment. If you want easy service setup, like people usually say they like about Docker, I think it's definitely worth a try. It can be as simple as "services.foo.enable = true".

(To be fair NixOS has complexity too, but most of it is in learning how the configuration language which builds your operating system works, and not in the actual system itself, which is mostly standard except for the store. A NixOS service module generates a normal systemd service + potentially other files in the file system.)

[–] foremanguy92_@lemmy.ml 3 points 1 week ago

nixos definitely gives a try

I ditched nix and install software only through portage. If needed, i make my own ebuilds.

This has two advantages:

  • it removes all the messy software: i am not going to install something if I can't make the ebuild becayse the development was a mess , like everything TS/node
  • i can install, rollback, reinstall, upgrad and provision (configuration) everything using portage
  • i am getting to know gentoo and portage in great details, making the use of my desktop and laptop much much easier
load more comments
view more: next β€Ί