A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
I put encrypted backups (borg or restic) on a storage box from Hetzner. One local copy on a different drive and one remote. Keep your encryption passwords safe though, otherwise they aren't worth much.
Oh, and I plan to report status of the cron jobs that run these backup scripts via MQTT and display backup status in Home Assistant. But haven't started that yet. So far I dump the logs and view them occasionally.
I'm a bit lazy so it's simple but it works: for the important stuff i have a borg endpoint on onedrive.
I'd recommend using a job monitoring service that will alerts you if it doesn't get a check in. It's very useful if your backups fail silently or hang mid-job for some reason.
I use https://healthchecks.io/ since it has a free tier and it works well.
Or setting proper mail support on everything so you get an email with full output whenever something fails. Ubuntu's postfix doc is really good.
I want to start by saying I am not suggesting you use any of the products these companies offer, but I'm linking to the standard strategy - 3-2-1.
https://www.backblaze.com/blog/the-3-2-1-backup-strategy/
https://www.acronis.com/en/blog/posts/backup-rule/
https://www.techtarget.com/searchdatabackup/definition/3-2-1-Backup-Strategy
For me, I have two boxes for NAS. One is the prod, one is the backup of anything I can't replace (or can't replace easily). I have another at the home of a member of my family, which gets a weekly diff. I also backup an encrypted set to cloud storage I got some time ago. So I actually have 4 sets of data (1 prod + 3 backups), two off-site locations. The media portion is treated differently today - it used to be tape, DVD backups, whatever, but today I consider different devices and cloud storage to fit that bill. In which case I have an abundance of forms of storage media
Mine goes a slight bit past what's needed for 3-2-1 which is appropriate for me. I consider 3-2-1 the minimum for any data considered critical or irreplaceable.
For me, that includes home movies, family photos, financial records, etc. It does not include my rips of my DVD collection. It does include config files and backups of services I run though.
The right backup strategy depends on your own concern about data. If I lost the photos/videos of my kids, I'd be devastated. If I lost the rips of VHS tapes my dad recorded, I'd be devastated.
If I lost the iso for a random esoteric piece of hardware that has its drivers, I'd be disappointed but its not a big deal.
Prioritize your data. Absolutely critical, important, preferred to keep, annoying but replaceable, and who cares I'll just download it again if I have to.
Once you know how much you need to store for each of those, add a bit to plan ahead, and see what backup strategy fits as you move down the priority list, and go from there.
I couldn't sleep at night if I didn't have my data backed up in 6 different places. I spent way too many years as a sysadmin to deal with 2 backups.
ZFS mirrors on my Proxmox server with multiple nodes replicating to each other. Replications of those datasets to zfs.rent. Proxmox backup server taking hourly snapshots and doing it to multiple drives. Rotating USB drives on that PBS server. Backups of the data for each VM and each docker container stack via rsync. Borg backup. Multiple Nextcloud clients with each having their file syncs held locally, then rsynced to a secondary drive.
I could probably come up with a couple more that I've forgotten I have running. I got burned once and it made me mad.
I have a file server for copy 1, an external drive with incremental backups for copy 2, and copy 3 is a physically unplugged copy in a firesafe lockbox that I update manually. I don’t use any cloud providers to back up anything.
I do local backups to another drive, and online encrypted backups to cloud storage (I use backblaze B2)
I see lots of solutions here, but some explanation of the basics are missing for someone starting out... this is not meant to sound preachy...
RAID is not a backup. It's just better hardware fault tolerance. Delete does the same thing on RAID as it does one 1 drive.
Everyone syncs / copies / duplicates files somewhere, but you need a way of finding the previous backup in case something was deleted. This can be done with various ways / tech, but the point is - have some history not just 1 copy. Many pointers to 3-2-1 in here, but that also doesn't mean 3 copies of just today's data...
Backups are nothing without Restores. Test the backups. Various ways, but do it. Often.
And consider what you're backing up and why.. ie just your data? (Ie photos), or all the config files, databases, operating systems, etc to do a full restore on new metal. If the latter, I recommend keeping your data separate from the OS / config files, etc.
Source: decades of tech disasters 😁
I copy my data to a nas at home. That copies once a day to an off-site nas. Once in a while I connect an external disk to the nas and copy all and disconnect it.
Perfect or not, compliant with backup procedures or not ..... it works for me and i'm happy having with an air gap backup.
The only concern I see here is the external drive. My experience has been that powered off drives fail more often than constantly-on drives. So my external drives are always powered on, I just run a replication script to them on a schedule.
But you do have good coverage, so that's a small risk.
I know and I agree. I still want an air-gap backup. Because ......it's air gapped
Your garage is not remote. Remote means somewhere it won't be affected in case of theft, fire, flood, earthquake, hurricane, that sort of thing.
I would like to see a thief try and take my server rack down a flight of stairs
You should assume that experienced burglars will have no trouble taking a rack apart piece by piece and that inexperienced ones will one happy to try, if only to send it flying down the stairs into destruction.
I mean, of course, they might decide it’s just not worth the trouble, but don’t count on it.
I mailed my buddy a Raspberry pi with a large hard drive attached and rsyc to it.
Basically:
Small proxmox node (Zimablade) that basically does only operate a Proxmox Backup Server for local clients and fast backup.
Offsite ZFS send to a VPS I operate for that purpose. As well as Proxmox Backup Server for VMs,etc. Basically meant as a fast recovery option. (Layer7)
Offsite S3 storage backup to a different provider from above. Meant for a medium term backup. (Hetzner and IONOS)
Portable HD: I have two different portable HDs. One is hooked up to the Backup server, the other one is in a lock box in my banks safe. The "connected one" does a weekly backup (and is switched off in between). Once in a while (around 6 to 12 weeks, with 12 weeks being the hard maximum) I take the active one to the bank and both drives switch places. That provides a full backup. (WD My Book and Seagate Expansion - the differrnt manufacturers are intentional)
Last line of defence: The real real important things (photos of life events-weddings,etc.- important documents,Password DBs) etc.) get burned on a M-disc Archive blue ray. They are also in the bank safe and at a secure third location. They are more meant for "shit hit the fan and I might not be there anymore,but maybe the kids want these". Additionally they provide a defence against encryption viruses - write once reas many (WORM) has it's advantages here.
This is another thing to consider: Have detailed descriptions for others how to retrieve your data in case something happens. I operate a private wiki (on an external server) that also gets saved into the M-Discs that has step by step instructions, as they might need to be followed by someone not that tech adept. (Like my In-Laws in case both my wife and I perish.), have notes in my password DB (Vaultwarden, which has a digital heritage/emergency access function and is also exported), in the vault, and a note in my will notifying people about this.
Edit: And: Test your recovery. Almost every data loss I have witnessed in the last years was a recovery problem. Missing encryption keys, data structure issues, etc. I have seen them all. Personally I try to recover a random file (as in: A script tells me which one) twice a year from every method and try a full recovery of each method at least once six month after introduction. Thst being said: It's nice to have encrypted backups,but that doesn't help if you can't find the keys/the software does no longer exist,etc. Currently a LOT of my clients have the same problem: They use Tandberg RDX for backup, including WORM. Now, Tandberg has gone bust and it's not that unlikely that yhey won't be able get another RDX drive in 5 or 10 years. Or 20. Which is the legal requirement for some official files here. Well,fuck. They needed to get additional drives asap when the bankruptcy became official.
Friends have used ancient LTOs and now face the same issues - LTOs are not downwards compatible. (That's why I use "common" technology. It's extremely likely that I will be able to find a spare BD drive in 20 years,etc.)
Have detailed descriptions for others how to retrieve your data in case something happens.
Lots of great advice here but this stands out as a really good bit that a lot of people (including myself) need to consider.
Weekend proooojeeeeeect! 🎵
Thanks a lot. I expanded it a little bit.
Nextcloud (later Opencloud) and Immich as primary data sources, backup to:
This worked and works well for me.
With backups two is one and one is none, so you are very much in a right track. Personally I have my stuff running on proxmox VMs with a proxmox backup server (VM as well) storing backups to Hetzner Storagebox. I'm planning to set up a another host in garage to have "local" backups too, as mine is detached as well the risk of both going up in flames in event of fire is pretty low. However, a voltage spike due to lightning on the grid or something else might blow up both hosts so that's a threat model to be aware of. Also if your connection to garage is over copper it can cause other problems, fibre or wireless is highly recommended.
With backups it's largely about the bandwidth available. I personally have enough so uploading to cloud is not an issue, but backing up a terabyte of data over 10Mbps connection might not work out at all.
For more info search for 3-2-1 strategy, that should give you plenty of ideas what you need to think about and what are industry best practises about making sure backups are in order.
I have three backups. One is my laptop where all the backups initially start. Then that gets copied to a plugin USB SSD. Then another copy goes to my server which has another USB SSD. That means I don't have an off site backup.
I don't have a place to host an off site backup and I'm not comfortable or interested in using cloud services. Instead I just decided that if it all goes up in flames. So be it.
It's just data and backups are just nice a convenience. I'll be upset but there's more important things in life to worry about.
I've always lived a life of minimalism and to me stuff is stuff. None of it mattered before I was born and none of it will matter after I die. That happiest and most free feeling I ever experienced was when I spent years travelling with only a 34 litre backpack and that's kind of been my baseline for happiness ever since.
My friend who lives a thousand miles away swaps hard drives with me that are backups of critical stuff. He keeps my data, I keep his. As others have said your garage is a start but you really want some sort of geographically separate backup.
That’s a really interesting idea. That makes me think it would even be better in the sense that the data would be protected rather than risk a cloud service going out of business or changed their storage location. Not that that is a likely scenario but still.
Just for the sake of conversation, I recently did some crude math on this. I have few friends around who are well capable of running a backup server for me (hardware maintenance and stuff is always needed anyways) and at first it seemed like a good plan. Just get a 4TB SSD/NVME and throw that on a Raspberry Pi (or something small to keep electricity consumption low and setup silent), set up encryption, connect that to my network with wireguard or some other VPN and let it do it's thing.
But I'd need to purchase everything as setting up a remote location with old hardware is just asking for trouble. The drive alone is 300€ (give or take) and the rest is easily another 100€. Currently my storagebox costs ~10€/month for 5TB. Even if I scored a fantastic black week offer and got everything for -50% discount that hardware with multiple single point of failures would cost nearly 2 years worth of cloud backups. And I'd still owe at least few beers to the friend for the trouble.
Your mileage may obviously vary, there's a million different scenarios, but for me with my current setup it just makes sense to pick couple cloud providers and let them store my bits instead of getting more hardware to maintain and upgrade.
For my buddy and I our critical data doesn't change often so once or twice a year when we get together we swap drives again. Simple spinning discs for us. No need for hardware or anything to keep them running. They just sit on a shelf just in case something happens we can hand it back to pull the data back onto a running server
I actually still burn 5-8 BluRay disks about twice a year with my most important data (Photos, memories etc.) which I store at my parents house as true immutable offsite backups. Furthermore I mirror my TrueNAS Backup Server every night at my brother’s place through a VPN connection .
I remember being told cdr aren’t reliable for long term storage. Are Blu-ray better? Looks like there’s something called M Disc too…
That’s why I burn new ones every 6 months. There’s always some nee stuff to add anyways and it’s easier to just replace the whole backup then.
3-2-1 never let me down since more than two decades. I just bought an external nextcloud host. And everyone of these 3 backups was destroyed in the past and I was glad to have them in another backup. Fire at the external server host. Fire at home and a HDD failure. But not at the same time.
Through borg, I have the Option to go back to any point in time with the backups. I will probably never need this, hence why it happens in this step, not on the rsync job to the NAS.
Things like movies and tv shows are not backed up, they are replaceable. All in all, about 2tb of documents, pictures, and VM state is backed up to Hetzner, out of the 16tb on the NAS.
Pick and choose your battles.
For stuff like movies I simply use replication as my backup.
Since I share media with fruends/family, I act as the central repository and replicate to them on a schedule (Mom on Monday, Friend 1 on Tuesday, etc), so I have a few days to catch an error. It's not perfect but I check those replication logs weekly.
I also have 2 local replicas of media, so I'm pretty safe.
I use remote encrypted backup and it's been good so far. Hetzner Storage box, around $11/month for 5TB, pretty reasonable. If you want a lot more storage you can pay somewhat less per TB.
I follow the good 9pd 3 2 2 rule. Three backups, in two locations, one is remote.
First copy on an external hard drive that gets mounted only for that, then unmounted.
Second copy still at home, on a disk connected to an OpenWRT access point in the patio.
Third copy on my VPS, so remote.
Each night restic take care of all that.
Once had problems with an internal drive so each device I run uses an external SSD/HDD. Anything important that has an "export data" or backup option such as Paperless I'll export/backup & put that into Nextcloud. Nextcloud files are synced between multiple desktops, one of which then gets automatically backed up to a separate drive each week.
For all my other self host stuff I since deployed Kopia to perform nightly local backups of each thing I self host. Once per month a Kopia backup for each software gets moved to a separate drive.
On top of that, things I deem as particularly important get encrypted in Cryptomator & uploaded off site.
No doubt there's probably better/easier ways but thats my current workflow.
I have Proxmox Backup Server backing up to an external drive nightly, and then about every 2 or 3 weeks also backup to a cold storage which I store offsite. (this is bad practice I know but I have enough redundancies in place of personal data that I'm ok with it).
For critical info like my personal data I have a sync-thing that is syncing to 3 devices, so for personal info I have roughly 4 copies(across different devices) + the PBS + potentially dated offsite.