I am technologically skilled to some extent, but not nearly enough to justify the financial investment I’m about to make In a home server without checking that I’m not way off the mark. Reddit tech answers are usually useless, all jerking each other off about being the most vague and useless. So I’m here, just humbly asking if my plan will work before I waste a ton of money if it doesn’t.
So I want to run immich, jellyfin, vaultwarden, and a couple other small programs (including backup automation) on a home server. The photos and videos are very important and I want off of large corporate cloud services for all the reasons you can imagine. There are about 2TB of important photos and data right now to backup.
So my plan: a mini pc with a low-level i3 or i5 chip but about 256gb SSD and 8gb ram. I will buy a UPS, a hub/NAS for 2 4TB HDDs, and a separate hub for backups elsewhere on another 4TB HDD. I already have a pc with an ok graphics card (only like 8 years old, but it was above average then) and good processor. My plan is to run the server with Linux, docker, and portainer on the mini-pc, and do the most intensive work (things like local Machine Learning on the photo library in immich) on the gaming pc. To save on energy, the mini pc is the only thing on most of the time, and I will turn on the old gaming pc to do the hard tasks.
I am hoping a friend will allow me to (somehow? I don’t know how yet) keep a hub+HDD backup at their place which is updated weekly automatically whenever their computer turns on.
Is this all achievable? Am I missing something small or huge? Any tech people here know something that I’ll likely miss on my first attempt?
Very good information! I had no idea that 3 was the minimum for drives for RAID error correction. I thought a RAID6 configuration with 2 drives was already doing error correction?
You can’t have a RAID6 with 2 drives. RAID6 uses 2 drives just for parity.
You can have a RAID1 (mirroring) with 2 drives.
Yeah I definitely confused those. I do not have money for higher RAID than 1 lol. Also don’t need it, I think. But I would like to be as little dependent on 1 HDD as possible, because I care about the data. So does it make sense to do RAID 1 so that 1 can break and the other still works? Then I still have back while redundancy is lost?
It’s fine if you are willing to use half of your disk space on redundancy.
ZFS can do that.
Bever heard of ZFS, dont really understand it. Does a software achieve this for my backups?
It’s a filesystem with support for software RAIDs among other things. There is also Linux device mapper.
A 3-way mirror is basically software level RAID 1 but you can lose 2 discs without losing data. This is the same redundancy level as RAID 6 / RAIDZ2, but it’s less storage efficient. On the plus side, it’s very low overhead, simple to think about, and will be faster at reading / writing.
So let’s say you’re doing a 4 TB disc buy and can choose between mirror and erasure coding, same level of redundancy at somewhat optimal setups for this scale.
For 3 way mirror: 3 discs, you have 4 TB useable storage and should really try to keep it under 80% utilization so more like 3.2 TB. Cost is 3 x drive price. You can add one disc at a time and restripe if using a software RAID that supports it or get 3 more discs to double storage. You can use ZFS or btrfs for this, maybe some others.
For erasure coding: 6 discs, you have 16 TB useable storage, 12.8 TB if staying under 80% utilization. Cost is 6 x drive price. This is harder to expand but it’s sometimes possible, usually with a lengthy redistribution process. You basically need to use ZFS for this. You pay more for power due to the number of discs always spinning.
IMO you should figure out a reasonable guess at how much storage you need and then double it, then compare prices for the drives you’d need to get that with either level.
Is a 3 way mirror possible with one at a distance/on a different network or processor? Or is there some way I could make the 3rd of the mirror at a distance at a friend’s house?
It is possible but performance is so bad and setup so complex that your definitely don’t want to do it.
Ok so reliable saving of my data in a safe way is, instead of 3-2-1, actually 4-2-1 if I want mirroring for corruption check… damn
Kinda yeah the redundancy is less of a backup and more of a resiliency for your main data repository. 3-2-1 could be 1 off-site backup and a local backup within that NAS, e.g., so only 1 off-site drive needed.
You need 3 copies, technically. If 1 out if 3 disagrees with the other 2 then the 1 gets overwritten. It doesn’t have to be disc level, you can do this on one drive (I actually do this sometimes to get error correction on system drives), but if you’re setting up redundant discs anyways you want to play to the strengths of those systems by getting e.g. a 3 way disc mirror.
Technically some systems can get away with two copies and some careful hashing tricks but best not to think about that for this use case.
So this means 3 drives for mirroring in my system and then still 1 at a different location to make sure it’s safe from fires if I want to avoid clouds… damn
Yeah basically you want to plan for what happens if a drive fails. With a 3-way mirror you have 2 redundant drives so when you need to replace a drive it rapidly copies from those 2 and gets you back up and running. Drive health is correlated so the most likely time for another drive to go down is during this rebuild. If a drive does go down during a rebuild, you’re still okay so long as that last drive makes it. 2 redundant is plenty to make rebuilding a 3-way mirror basically guaranteed.
Keep in mind this is just keeping you running and isn’t a backup, but I think that + error correction makes it worth it for any nas with data you care about.
Damn I care about the data but with 4tb hdds thats like 1000 bucks instead of 700… which I already can barely afford
Is that for 1 or 2 extra HDDs?
I usually target refurbished drives on ebay from a reliable seller. They’re used and have a lower lifetime but are usually about half price and that’s good enchanting for me.
2 extras. Europe is fucking expensive and the used market is shit. It’s impossible to find used hardware thats not at like 85% new price
Oh dang that sucks. I stocked up on a great deal a while back to avoid needing to buy anything for hopefully like 15 years. Not bragging, more like if you have the option to be patient you can sometimes find good deals. Though with a likely fuel crunch it might be good to try something soon.
I do believe you can get error correcting with a btrfs 2 way mirror, you just lose that redundancy for a rebuild. Your backups are more important than the nas redundancy, so that could be a good trade-off. Plus you can add a drive later if you find a good deal and it’ll be fairly easy to turn it into 3-way. I think zfs might be able to do this as well but I’m less certain there. I use ZFS for bulk storage because the erasure coding modes offer great density but I use dup and raid modes with btrfs for system / fast drives and it’s rock solid. I do recommend staying away from erasure coding on btrfs though (raid 6 etc), it’s unstable.