How can I install Fedora Workstation with a different btrfs hash algorithm?

I’ve got a new system and I want to use the xxhash checksum algorithm instead of crc32c for my / and /home filesystems. How can I do that at install time?

Related question: How do I see what hash algorithm a given btrfs filesystem is using?:

1 Like

FWIW, on my X1 Carbon Gen 8 (i7-10510U), hash-speedtest (from btrfs-utils, but not built for some reason so I had to compile it separately), xxhash seems to be faster:

    NULL-NOP: cycles:    186680650, c/i      186
 NULL-MEMCPY: cycles:    293741428, c/i      293
      CRC32C: cycles:    967826572, c/i      967
      XXHASH: cycles:    712516754, c/i      712
      SHA256: cycles:  50492224828, c/i    50492
     BLAKE2b: cycles:  12540176332, c/i    12540

and it’s 64-bit instead of 32, so it really seems like a clear win. You can see that the two 256-bit algorithms are substantially slower and therefore probably not desired in the general case (but may be worth it for a backup drive, for example).

As I understand it, it is _not _ possible to change this for already-existing filesystems, so I’d like to start with it for my new system.

I have to say I have no clue about those checksum algorithms, but it sounds like Fedora should default to the xxxhash - too late for F34, but maybe F35? Or are there any disadvantages compared to crc32c?

cc @chrismurphy

It seems the only option is to do the disk partitioning and mkfs.btrfs in command line, and then add btrfs subvol mount points in Anaconda.

Thought process that went into it:
https://kdave.github.io/selecting-hash-for-btrfs/

Since they are new only since kernel 5.5, and it’s not being backported, the downside is any Btrfs that uses something other than crc32c is not mountable on kernel 5.4 and older.

Another small cost is that it’s 8 bytes per 4KiB data block, twice that of crc32c, which will double the size of the csum tree. And also means there’s some small IO cost as well.

I’m not sure how to calculate the collision risk of crc32c, or assess at what point we should be concerned about it. I think any file system you think might become “large” i.e. 8T or bigger, you probably should consider using xxhash. I’m not aware of any dedup tools that take advantage of the csum tree yet, mainly because it’s crc32c based up until recently, but the idea is they could use xxhash to identify candidates much more cheaply than the current out of band method that has to compute hashes on extents to find candidate extents for dedup.

So I think it’s a good idea to consider switching. I’m not sure it’s urgent. And I think we should coordinate with upstream. These things are probably best assessed with a change proposal for Fedora 35.

2 Likes

This actually does happen to be on an 8TB drive. I agree that changing the default would be best for the future if ever — just wondering the best way to do it as a non-default for my case now.

The work around right now is to create the file system outside the installer with the mkfs options you want, and then installer will let you use that file system without reformatting. Btrfs is exempt from the installer reformat requirement, although it does require a new subvolume be created for / mountpoint. Although to use an existing Btrfs in this manner means using Custom or Advanced-Custom UI.

For a file system of that size I suggest:

mkfs.btrfs --csum xxhash -L fedorapool -R free-space-tree -O no-holes /dev/sdXY

You can switch over to free space tree (a.k.a. space cache v2) and no-holes later. Free space tree gets progressively more expensive to create when the file system isn’t empty, depending on how full it is. It’s the inverse of the extent tree, so the more full the file system the bigger the extent tree thus the more expense to compute a free space tree.

1 Like

Unfortunately, this doesn’t work, as Anaconda forces the root filesystem to be formatted at install time and won’t proceed without allowing it to reformat. It knows that the “base” btrfs filesystem needs to be formatted, not the subvolume, but therefore forces that to be formatted by Anaconda.

It does not sound right.

Anaconda will insist on a new subvol be created for the rootfs. But it will happily use any existing base btrfs filesystem.

Yeah, I see in retrospect what I did wrong. I should have just created the base filesystem and used anaconda to create the subvolumes. That didn’t occur to me. Oh well, I’ll do it next time I need to reinstall :slight_smile:

2 Likes