I am planning on creating a home server with either 2 (RAID1) or 3 (RAID5) HDDs as bulk storage and 1 SSD as bcache.
The question is, what file system should I use for the HDDs? I am thinking of ext4 or xfs, as I heard btrfs is not recommended for my use case for some reason.
Do you all have some advice to give on what file system to use, as well as some other tips?
Personally I would go for ZFS with the SSD as a L2ARC. But among the options you listed I would do BTRFS RAID1 if you’re only gonna use two HDDs, and mdadm RAID5 with BTRFS on top if using three.
L2ARC will kill a SSD faster than normal wear would.
It will yeah, although with modern SSDs it really isn’t a big problem. I’ve used an Samsung 840 EVO as L2ARC for 8 years now.
What are the advantages of this over mdadm raid and bcache?
Btrfs still has some issues, but it’s not like it’s dangerous or anything.
Xfs is going to give more flexibility for managing volumes, better performance than ext4 across multiple disks, and more fault protection.
Ext4 doesn’t really have any benefits in this race but being stable I suppose. An argument could be made it might be slightly faster under LUKS.
Zfs is more complex, but a bit more flexible than XFS, has CoW, snapshots, built in encryption and dynamic storage allocation.
Many suggest zfs, I want to spend a word on ext4 instead. Solid, reliable, well proven. Does the job and works pretty well.
Been on ext4 on RAID1 for decades, since it got stable. Never had an issue, except when I borked it by my mistake.
It has maybe less features than zfs, but doesn’t need external kernel patches or complex tools, and again its solid, well proven and very stable
Edit: ext4 on top of Linux software raid (mdadm)
I would absolutely recommend a file system with snapshot capabilities for a home server. One of btrfs mirror, dm-raid (raid5) with btrfs, or zfs would work. The practical differences would be negligible at this scale and you can just pick whatever you fancy.
The BTRFS thing is cutting the power or losing the disks in the middle of a write which corrupts your data. If you don’t think that will be a problem then BTRFS is fine. I recommend ZFS personally, but it sounds like you want to use mdadm instead so basically anything will work.
If you might need to shrink your filesystem later then avoid XFS. EXT4 is relatively featureless but ol’ reliable. ZFS is good for long term data integrity and protection. BTRFS is similar to ZFS. BcacheFS is new but like a swirl of EXT4 and BTRFS. Just pick the one with the features you want.
Powerloss might happen as I don’t have a ups.
And when it comes to mdadm, it just happens to be the first and only redundancy tool I know. I am however open to learn and try new things.
ZFS seems interesting, but: I read that ZFS would require quite a lot of RAM, and I was going for 32 GBs only, would it be enough?
ZFS doesn’t require a lot of RAM, but it will use more RAM if it’s available. 32G would be plenty for a home setup. I think my home file server has 24 or 32G of RAM and ZFS. If it’s important data then stick to what you know; there’s nothing wrong with mdadm.
ZFS doesn’t require lots of RAM, more RAM just improves the caching (ARC) it can do. You can set ZFS to use all unused RAM as ARC, so it doesn’t interfere with other services running on the same PC. I ran ZFS with lots of VMs on an old office PC with 16GB RAM and it was still able to max out a 10gig nic.
ZFS for it all and maybe btrfs if you are ok with its limitations