this post was submitted on 04 Nov 2023
79 points (92.5% liked)

Selfhosted

40347 readers
361 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
ruud@lemmy.world
Loki@lemmy.world
CannaVet@lemmy.world
devve@lemmy.world
HybridSarcasm@lemmy.world
HybridSarcasm@lemmy.hybridsarcasm.xyz
79
What I just learned about restoring BorgBackups (github.com)
submitted 1 year ago by Carunga@feddit.de to c/selfhosted@lemmy.world
20 comments fedilink hide all child comments
 

Maybe I state the obvious here but anyway:

Tonight I found a super comfi way to browse and restore data from Borg repositories. In vorta you can add any existing borg repository and mount the containing backups in a very functional GUI. Until now I somehow assumed you could only deal with backups created in vorta which is not true.

For context let me explain why I find that so useful: I am using a wonderful Borg / Borgmatic container from the borgmatic collective. It works super reliable. But for restoring files you have to mount the backup in the container and deal with the files inside the container. This can be difficult i.e. when looking for a specific file in a folder with a lot of files (like a foto collection where you accidentally deleted one file).

If any Borg users are out there: I would be interested what your workflows look like.

all 21 comments
sorted by: hot top controversial new old
[–] witten@lemmy.world 22 points 1 year ago* (last edited 1 year ago) (2 children)

borgmatic dev here. First of all, if Vorta is working well for you to recover files, then by all means use Vorta! Right tool for the job and all. Having said that, a couple of thoughts on using borgmatic in Docker and recovering files:

borgmatic has a search feature that makes finding a particular file in an archive or across archives pretty easy. So that might be step one in restoring an accidentally deleted file.

Once you've found the file and archive to restore, you can either use borgmatic extract or borgmatic mount. With extract, you copy one or more files out of a backup archives. The challenge though is that with borgmatic in a container, by default there's not an easy way to copy those files into their original locations. However I think the "fix" is to mount your source volumes as read-write instead of (the documented) read-only. That way you can easily copy extracted files back to where they belong.

As for borgmatic mount, you've got a similar challenge and fix. You can presumably mount backup archives (or a whole repository) within the container, but then you need to copy your recovered files out of that mount into their original source volumes. So that probably also means those volumes need to be mounted read-write.

Let me know if you have any questions!

[–] Carunga@feddit.de 2 points 1 year ago (1 children)

Thanks for your answer and taking the time! Borgmatic search I did not know. That is an amazing tool. You are right about the mounting. My way of dealing with that is a NFS share I mount RW so I can restore to that and than copy whereever. This might it be ideal for very large restores though. Initially I thought I could borgmount to the NFS share and then access the filesystem via NFS. But this does nof work I suppose as Borg only lives inside the container. Generally I do like having Borg and Borgmatic containerized as almost everything else I selfhost but it adds complexity restoring. Anyways great project, it is just so powerful and in many ways elegant. Really enjoy using it!

[–] witten@lemmy.world 1 points 1 year ago

Glad to hear it's (mostly) working out for you! I know you came here looking for best practices with restores, but if you end up coming up with anything yourself, feel free to comment on that Docker borgmatic ticket with requests or ideas. I use the container myself on some systems for the same reasons you do, and I also wouldn't mind smoother restores!

[–] shadowbert@kbin.social 1 points 1 year ago (2 children)

Is there a way to have an ssh remote without borg installed on the target?

[–] witten@lemmy.world 3 points 1 year ago (1 children)

Nope! Borg always requires Borg on the remote side. It's Borg's biggest strength and weakness versus competing backup systems IMO. Strength, because it can do pretty smart stuff with its own code running on both sides. Weakness, because it means it doesn't work natively with cloud object storage like S3. It's a tradeoff like anything else.

[–] shadowbert@kbin.social 1 points 1 year ago* (last edited 1 year ago) (1 children)

Makes sense - compromises like you say.

So in the context of unraid, do I just run the borgmatic container on both ends? Or should there be a specific 'server' one?

EDIT: I found borgserver. I'm pretty sure that's the correct one to use.

[–] witten@lemmy.world 2 points 1 year ago (1 children)

I'm not super familiar with Unraid, but yeah, the borgserver image sounds like it'd work for this.. You don't need borgmatic on the server side unless you want it there to make running Borg commands easier.

[–] shadowbert@kbin.social 1 points 1 year ago* (last edited 1 year ago) (1 children)

Is it normal for borg to nag for my password so many times?

80a8f8fc1129:/# borgmatic list
local: Listing archives
Enter passphrase for key /mnt/borg-repository:
Enter passphrase for key /mnt/borg-repository:
80a8f8fc1129-2023-11-08T14:54:25.535734 Wed, 2023-11-08 14:54:28 [5a6245718dcb7e7dfec023cf0a62f568a9714b0f27d7a422e97b44870e9ecfbd]
80a8f8fc1129-2023-11-08T14:55:06.056089 Wed, 2023-11-08 14:55:08 [bce2ae9ff5b58212993281c938eb6812fe706bb1b62b80e40a32d38179f3b86c]
80a8f8fc1129-2023-11-08T14:55:24.704310 Wed, 2023-11-08 14:55:28 [7793ab62dac221d3b90434c19e0c44f917c9caedbc6250b71c86c78327348539]
80a8f8fc1129-2023-11-08T15:01:25.862051 Wed, 2023-11-08 15:01:28 [b1c83dc0ccd3d04ddfcb7d39358e8978f93a705b792105ef9c0389faf6d6e387]
80a8f8fc1129-2023-11-08T15:02:25.774879 Wed, 2023-11-08 15:02:28 [fe74cdf86dceea5a21d5305e3314733adf31d5cfe3d63e934381cec2176e59c2]
80a8f8fc1129-2023-11-08T15:02:31.879678 Wed, 2023-11-08 15:02:38 [9a2189207aa475ac866a4f7aa62d6d3c83ae53bd008dd4240de2b4277554620e]
local: Listing archives
Enter passphrase for key /mnt/borg-repository:
Enter passphrase for key /mnt/borg-repository:
80a8f8fc1129-2023-11-08T14:54:25.535734 Wed, 2023-11-08 14:54:28 [5a6245718dcb7e7dfec023cf0a62f568a9714b0f27d7a422e97b44870e9ecfbd]
80a8f8fc1129-2023-11-08T14:55:06.056089 Wed, 2023-11-08 14:55:08 [bce2ae9ff5b58212993281c938eb6812fe706bb1b62b80e40a32d38179f3b86c]
80a8f8fc1129-2023-11-08T14:55:24.704310 Wed, 2023-11-08 14:55:28 [7793ab62dac221d3b90434c19e0c44f917c9caedbc6250b71c86c78327348539]
80a8f8fc1129-2023-11-08T15:01:25.862051 Wed, 2023-11-08 15:01:28 [b1c83dc0ccd3d04ddfcb7d39358e8978f93a705b792105ef9c0389faf6d6e387]
80a8f8fc1129-2023-11-08T15:02:25.774879 Wed, 2023-11-08 15:02:28 [fe74cdf86dceea5a21d5305e3314733adf31d5cfe3d63e934381cec2176e59c2]
80a8f8fc1129-2023-11-08T15:02:31.879678 Wed, 2023-11-08 15:02:38 [9a2189207aa475ac866a4f7aa62d6d3c83ae53bd008dd4240de2b4277554620e]
80a8f8fc1129:/#

Context: I have 2 config files, as I would prefer to have each app I want backed up in a separate config if reasonable. I don't have a remote setup yet - it's just using a local repository.

[–] witten@lemmy.world 2 points 1 year ago (1 children)

Separate configs is totally reasonable. It just sounds like you haven't configured your Borg passphrase with borgmatic.. Otherwise it wouldn't prompt for your passphrase at all.

[–] shadowbert@kbin.social 2 points 1 year ago* (last edited 1 year ago) (1 children)

I missed that part in the docs - thanks. Now it's working way better. Thanks heaps - I'm going to trial this alongside my duplicati for a bit (as I've heard a few too many horror stories about duplicati...)

[–] witten@lemmy.world 2 points 1 year ago

I hope one (or both!) of them end up working out for you.

[–] raldone01@lemmy.world 2 points 1 year ago

You could always sftp mount the repo if you cant install borg but have sftp support. Probably won't be as efficient though.

[–] bloopernova@programming.dev 14 points 1 year ago (1 children)

I used Borg to backup a bunch of dev servers a few years ago, about 5TB from several dozen hosts, over several years. It worked flawlessly, and its dedupe is downright magical.

The restore workflow, where you just mount a snapshot and copy what you need, is fantastic. Very straightforward and reliable.

[–] Carunga@feddit.de 1 points 1 year ago

Agreed, the dedupe feels like magic!

[–] narc0tic_bird@lemm.ee 13 points 1 year ago (1 children)

Nice frontend, but honestly the Borg CLI and borgmatic config is so easy and straightforward to use, you shouldn't need any GUI.

May I ask why you are running Borg inside a container? You could just run it directly on the host and that would've solved your problem it seems.

[–] Carunga@feddit.de 4 points 1 year ago (1 children)

Borg is running on a headless server. Everything is dockerized, so I did the same with Borg. Advantages are that the setup is easy to setup, backup the config and move it to a different server. At first I did not realize that the mount of the backup only exists in the container and that this is making things a little harder.

[–] shadowbert@kbin.social 1 points 1 year ago (1 children)

Could you mount a blank directory into docker, then use borg to mount the backups into that?

[–] Carunga@feddit.de 1 points 1 year ago

This was the first idea. I cannot explain very well why this does not work. But I think the issue is that the borg mount magic lives inside the container so the filesystem cannot be seen from the host. You can mount an empty directory and copy the files you want to access from the host into it. Problem with that is that you are stuck with the tooling provided by the container.

[–] ebits21@lemmy.ca 9 points 1 year ago* (last edited 1 year ago) (1 children)

Yeah it’s just a frontend for Borg (which is great).

I just mount and browse in Nautilus. I have a mount script aliased to open-backup.

[–] Carunga@feddit.de 2 points 1 year ago* (last edited 1 year ago)

That does make sense. Could work for me as well. I was just not aware I can mount a repository from a remote host that was created by a different Borg instance on another server and just browse the files like they were local on my notebook.