convert: support xfs->btrfs conversion in-place.

[arcivanov]

I have both server, storage and desktop machines running on XFS (v4 and v5) (Fedora/CentOS).
Since Fedora looks to be switching to btrfs by default and FB has been running their servers on btrfs for years, I'm starting to drift towards converting to btrfs for everything (NVMe SSD laptops, NASes, SATA SSD servers).

XFS->Btrfs in-place conversion is not yet available. I would be interested in implementing this feature.

1. Any objections: conceptual, technical, spiritual?
2. Anyone has already done research and determined it to be impossible to convert xfs to btrfs in place so I don't have to waste my time?

[adam900710]

Not sure if Fedora is decided to go to btrfs by default.

I have no reason to do any objections. It looks completely sane to do the convert. (Although I'm pretty sure the XFS community would be very sarcastic).

For the technical possibility, as long as there is enough free continuous space inside any AG of xfs, btrfs-convert should be able to handle it pretty well.

The convert will firstly scan the source fs, finding out any used space (thus can figure out the unused space), then fill btrfs metadata into the unused space, crafting a btrfs file points to all used space (as the image), and copy the 0~1M and other superblock ranges to other unused space.
Then call it a day.

So as long as the XFS extent allocator isn't doing strange behavior (aka, fragment the free space intentionally), then it should be pretty much the same as EXT*.

For code level implementation, we only need several read-only call backs from the source fs in user space:

• open_fs()
• read_used_space()
• copy_inodes()
• close_fs()
• check_state()

So it shouldn't be that complex.

[arcivanov]

Not sure if Fedora is decided to go to btrfs by default.

It's not decided yet per se, but it looks like things are going that way: https://fedoraproject.org/wiki/Changes/BtrfsByDefault

Thanks for your comments.

[kdave]

Please note that converting the filesystems should be used in cases where mkfs and copy is not possible. The converted filesystem inherits layout of the original one and that can lead to some pathological cases like too small data/metadata chunks where even repeated balances can't make that more compact.

Any objections: conceptual, technical, spiritual?

Conceptual/technical no, the btrfs-convert codebase has been cleaned up to allow more source filesystems, additional cleanups or enhancements of the support code is fine. Adding more filesystems requires more testing and the testing matrix source features ⊗ btrfs features becomes quite large. This has worked reasonably well for ext4 and probably will for xfs as long as the library code covers all the details.

Spritual, maybe some people will not like to see it implemented :)

Anyone has already done research and determined it to be impossible to convert xfs to btrfs in place so I don't have to waste my time?

It should be possible, but I'm not aware of any prototype code to tell for sure.

[arcivanov]

Fedora approved default to btrfs: https://pagure.io/fesco/issue/2429

[arcivanov]

@kdave based on your comments, irrespective of this functionality, a question:

[[gpt][xfs].....] becomes [[gpt][xfs][btrfs]] with mkfs.btrfs + rsync followed by [[gpt]....[btrfs]] when xfs is dropped.

The question then is, obviously, can btrfs grow to fill the empty space before it or at least is there a path at all with gpt manipulation? I understand this can be partially accomplished by a subvolume, but assuming that's undesirable would there be a way?

[kdave]

Growing "to the left" is not implemented, but it is possible without moving all the data. https://github.com/kdave/drafts/blob/master/btrfs/chunk-tree-mover.txt are my notes on that. Moving the entire btrfs partition to another one works, as long as the relative offsets from the partitions are preserved.

[braindevices]

any progress on this feature?

[arcivanov]

Sorry, I got distracted by paid work. I'll get back to it hopefully soon.

[kdave]

I had a quick look what libxfs provides, it's not the same API like e2fsprogs has so that we can enumerate space info, inodes, extent info and be done. There are more low-level functions so the logical structures we want need to be assembled first.

This would be probably a longer process to get it working so if you're interested we can do the feature as experimental (ie. not built by default) and work on it incrementally.

[arcivanov]

That would be wonderful :)
When I started digging into it a lot of functionality with space info, inodes and extents etc and can extracted directly from some of the tools, like xfs_spaceman: https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git/tree/spaceman/freesp.c#n331

[kdave]

Ok, so start with whatever you have. The file naming in convert/ should be self explanatory and you can add the other bits into same places like eg. ext2 already does. Probably implementing the empty callbacks (returning errors) should be ok as a base. I'll add the conditions for the experimental mode afterwards, you don't need to bother with that.