-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Advanced partitioning customizations (COMPOSER-2355) #926
base: main
Are you sure you want to change the base?
Commits on Sep 19, 2024
-
blueprint: partitioning customization
- New structure that's more powerful than the existing FilesystemCustomization. - The general structure starts at 3 top-level objects: - Plain: customizations for plain disk partitioning, meaning partitions with traditional filesystem payloads (xfs, ext4, etc). - LVM: customizations for creating LVM layouts. Supports one or more VolumeGroups. Each VolumeGroup implies the creation of a partition with the appropriate type and payload. - Btrfs: customizations for creating btrfs layouts. Supports one or more volumes and subvolumes for each volume. Each btrfs volume implies the creation of a partition with the appropriate type and payload. - Partitioning.Filesystems is an array of the old FilesystemCustomization struct. - The new FilesystemCustomization struct supports a label and Type. - The LVCustomization (logical volume) embeds the FilesystemCustomization and adds a name for the LV. Unmarshal tests have been updated with new fields.
Configuration menu - View commit details
-
Copy full SHA for f39037a - Browse repository at this point
Copy the full SHA f39037aView commit details -
disk: NewCustomPartitionTable() function
New function that creates a partition table from scratch based on the new partitioning customizations.
Configuration menu - View commit details
-
Copy full SHA for 805ece4 - Browse repository at this point
Copy the full SHA 805ece4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 68fb75a - Browse repository at this point
Copy the full SHA 68fb75aView commit details -
disk: more testing for NewCustomPartitionTable()
Add one test for each general layout: plain, lvm, and btrfs.
Configuration menu - View commit details
-
Copy full SHA for 548e871 - Browse repository at this point
Copy the full SHA 548e871View commit details -
disk: move handling of the /boot partition
Move the handling of the /boot partition before the plain partition handling so we don't have to find where to insert it (after BIOS boot and ESP, which are going to be added in a later commit, but before plain partitions). Instead, we build the partition list sequentially as needed. Update tests.
Configuration menu - View commit details
-
Copy full SHA for ae2d48b - Browse repository at this point
Copy the full SHA ae2d48bView commit details -
pkg/{distro,platform}: move BootMode to platform
Move the BootMode enum from pkg/distro to pkg/platform. We want to import BootMode to the disk package, which would create an import loop with distro.
Configuration menu - View commit details
-
Copy full SHA for e0ba877 - Browse repository at this point
Copy the full SHA e0ba877View commit details -
disk: add bios and esp partitions based on boot mode
Add BIOS boot and ESP partitions based on boot mode. The boot mode is passed as an argument to the NewCustomPartitionTable() function. Update tests.
Configuration menu - View commit details
-
Copy full SHA for 8b5653d - Browse repository at this point
Copy the full SHA 8b5653dView commit details -
disk: add root partition / lv / subvol if missing
At the end of the partition table's creation, add a root partition if one wasn't created during the processing of the customizations.
Configuration menu - View commit details
-
Copy full SHA for 80ce76d - Browse repository at this point
Copy the full SHA 80ce76dView commit details -
disk: create root partition for empty customizations
When the customizations argument is nil, initialise it with an empty struct and continue processing the function normally so that the root partition is created as well.
Configuration menu - View commit details
-
Copy full SHA for 766911f - Browse repository at this point
Copy the full SHA 766911fView commit details -
blueprint: include minsize in partitioning customization
Add minsize to the top level of the partitioning customizations. In osbuild-composer, this value comes from a command line option through distro.ImageOptions. The plan is to have that be a blueprint customization that osbuild-composer will resolve when it's defined in multiple places and pass it to images via the customization.
Configuration menu - View commit details
-
Copy full SHA for 7e9ca23 - Browse repository at this point
Copy the full SHA 7e9ca23View commit details -
disk: add defaultType to NewCustomPartitionTable()
Add a new argument, defaultType, to the NewCustomPartitionTable() function. The purpose of the type is to determine which filesystem will be used for root and boot when they are added automatically. It will also be used to set the filesystem for any mountpoint that has no type defined. The default type should be determined by the distro definition.
Configuration menu - View commit details
-
Copy full SHA for f88e6e2 - Browse repository at this point
Copy the full SHA f88e6e2View commit details -
disk: use defaultType when missing fs type customization
When a custom partition or LogicalVolume does not specify a type, use the defaultType.
Configuration menu - View commit details
-
Copy full SHA for 397efb0 - Browse repository at this point
Copy the full SHA 397efb0View commit details -
disk: define FSType enum for all supported types
Add a new enum, FSType, for referring to all supported filesystem types. Use it as the default type argument to the NewCustomPartitionTable() function.
Configuration menu - View commit details
-
Copy full SHA for 9b75803 - Browse repository at this point
Copy the full SHA 9b75803View commit details -
disk: relayout the partition table in NewCustomPartitionTable()
Run a relayout() on the partition table before returning it from NewCustomPartitionTable() to set the start position and size for each partition as well as the total partition table size. Update tests. Partition sizes in tests changed to MiB (from plain bytes) so that the values make more sense when headers are added and values are rounded to the grain size (1 MiB).
Configuration menu - View commit details
-
Copy full SHA for 4a689db - Browse repository at this point
Copy the full SHA 4a689dbView commit details -
disk: CustomPartitionTableOptions struct
Move the options for the NewCustomPartitionTable() function to a struct. More will be added later and handling them all as one argument will make things simpler.
Configuration menu - View commit details
-
Copy full SHA for 51c44a5 - Browse repository at this point
Copy the full SHA 51c44a5View commit details -
disk: add PartitionTableType to CustomPartitionTableOptions
Add an option for the partition table type to the CustomPartitionTableOptions. The value must be one of "gpt", "dos", or "". An empty string (or nil options) defaults to "gpt". The tests have been updated to explicitly specify "dos". The relayout() method, which is affected by this, checks whether `pt.Type == "gpt"`, which means it was defaulting to "dos" when deciding to add a footer.
Configuration menu - View commit details
-
Copy full SHA for 986b64c - Browse repository at this point
Copy the full SHA 986b64cView commit details -
disk: add "gpt", "dos", and "" types to tests
Add examples of each valid value for the PartitionTableType (including unset) to the tests. More tests will be added later to make sure all combinations are covered.
Configuration menu - View commit details
-
Copy full SHA for d7b9dd4 - Browse repository at this point
Copy the full SHA d7b9dd4View commit details -
disk: generate UUIDs in NewCustomPartitionTable()
Generate all (missing) UUIDs at the end of NewCustomPartitionTable() before returning the final PT. Update tests: - Move the random number generator initialisation inside the test loop so that each test uses its own generator. This makes tests independent of the order they're run in. - Add generated UUIDs to the expected result.
Configuration menu - View commit details
-
Copy full SHA for 24b83ca - Browse repository at this point
Copy the full SHA 24b83caView commit details
Commits on Sep 20, 2024
-
disk: add RequiredSizes to CustomPartitionTableOptions
Introduce the concept of required sizes from the existing NewPartitionTable() function into the NewCustomPartitionTable(). RequiredSizes is necessary for setting a "sane default" (minimum) size for certain directories. Unlike the old partition table creation function, there is no default value for the custom partition table generator. Instead, image definitions will need to set their own defaults. Without this mechanism, automatically created root filesystems can end up being too small to fit the base OS packages or to have a usable system in the long run. It's also needed when automatically creating root partitions or LVM Logical Volumes. Without a required minimum, the root partition will only be the size of the leftover space on the image (the base image size minus the sum of the rest of the partitions, which can sometimes end up being 0) or LVM VolumeGroup. Add some required sizes to tests.
Configuration menu - View commit details
-
Copy full SHA for d844a8a - Browse repository at this point
Copy the full SHA d844a8aView commit details -
blueprint: remove GetFilesystemsMinSize()
This is leftover from the repo split. It is implemented and used in osbuild-composer.
Configuration menu - View commit details
-
Copy full SHA for 4e7a7ee - Browse repository at this point
Copy the full SHA 4e7a7eeView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2a0bc1b - Browse repository at this point
Copy the full SHA 2a0bc1bView commit details -
blueprint: add CheckPartitioningPolicy()
This is the equivalent to the CheckMountpointsPolicy() function for the new PartitioningCustomization. It collects all the mountpoints from Plain, LVM, and Btrfs then compares them to the policy.
Configuration menu - View commit details
-
Copy full SHA for 9a5f8dd - Browse repository at this point
Copy the full SHA 9a5f8ddView commit details -
distro/fedora: pass all customizations to getPartitionTable()
Preparing the function to use the new PartitioningCustomization when set.
Configuration menu - View commit details
-
Copy full SHA for 92f089b - Browse repository at this point
Copy the full SHA 92f089bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5da5eae - Browse repository at this point
Copy the full SHA 5da5eaeView commit details -
distro/fedora: add minimum directory sizes to image types
For all image types that don't specify a requiredPartitionSizes, add the defaults that are used in the NewPartitionTable() so that NewCustomPartitionTable() has similar behaviour. The only image type that explicitly doesn't specify minimum sizes is the iot-raw-image, which has a pre-existing empty map.
Configuration menu - View commit details
-
Copy full SHA for bcfb63a - Browse repository at this point
Copy the full SHA bcfb63aView commit details -
distro/fedora: check partitioning customizations
Add checks for Partitioning customizations in Fedora's checkOptions() function. The function now returns an error if: - Partitioning is defined at the same time as Filesystem - Partitioning is defined for ostree image types (commit and container). - The mountpoints in Partitioning violate the mountpoint policies for the image type.
Configuration menu - View commit details
-
Copy full SHA for d50bf94 - Browse repository at this point
Copy the full SHA d50bf94View commit details -
blueprint: validate partitioning customizations
Add a Validate() method to PartitioningCustomization that returns an error if: 1. Btrfs and LVM are both defined. 2. Any mountpoint is invalid (empty string, relative path, or not "clean"). 3. Any mountpoint is defined more than once in the whole structure. 4. There is more than one LVM Volume Group or more than one btrfs volume. 5. Any LVM Logical Volume name is used more than once in the same VolumeGroup. 6. Any Btrfs subvolume has an empty name. 7. Any Btrfs subvolume name is used more than once in the same Btrfs volume. This function validates the customization itself in the absence of any distro or image-type specific policies. We may relax rule 3 in the future, at least in images if not in osbuild-composer, but for now, let's disallow it while keeping the customization structure flexible. val
Configuration menu - View commit details
-
Copy full SHA for 84d94e0 - Browse repository at this point
Copy the full SHA 84d94e0View commit details -
disk: refactor LVM Logical Volume name generation
The VolumeGroup.CreateLogicalVolume() function assumed that the given name was a mountpoint and would automatically escape it, append 'lv', and append numbers in case of collisions. This made it impossible to use the helper function for LV creation with desired names. Refactor the function and modify all the calls so that now it uses the provided name without validating, escaping, or checking for collisions, and generates a name from the payload's mountpoint when one is not provided. The CreateMountpoint() function's functionality is not affected.
Configuration menu - View commit details
-
Copy full SHA for d9fe555 - Browse repository at this point
Copy the full SHA d9fe555View commit details -
disk: use vg.CreateLogicalVolume() in NewCustomPartitionTable()
Use the CreateLogicalVolume() helper when creating LVs for the custom partition table. This changes size calculations in tests because each LV is resized to be an integer number of extents, i.e. sizes are rounded up to 4 MiB multiples.
Configuration menu - View commit details
-
Copy full SHA for 541f95f - Browse repository at this point
Copy the full SHA 541f95fView commit details -
disk: generate volume group names when unspecified
When a volume group's name is not specified in the customizations, the custom partition table creator assigned the names vg00, vg01, etc. A new test is added with multiple volume groups to test this. The scenario of the test is not supported by customizations, the blueprint package does not allow multiple volume groups, but the function can handle it so it's good to have it covered.
Configuration menu - View commit details
-
Copy full SHA for c7c3486 - Browse repository at this point
Copy the full SHA c7c3486View commit details -
test: add partitioning build configs
One for each variant: - Plain - Plain + btrfs - Plain + lvm
Configuration menu - View commit details
-
Copy full SHA for 837ce9b - Browse repository at this point
Copy the full SHA 837ce9bView commit details -
blueprint: add extra validation for mountpoints and fstypes
Added extra validation for: - invalid filesystems for specific mountpoints (e.g. xfs on /boot/efi) - /boot or /boot/efi on lvm or btrfs - btrfs as a filesystem on plain Added tests to cover all new validation errors. Added an extra happy test for /boot and /boot/efi with automatic fs type selection (unset, empty strings).
Configuration menu - View commit details
-
Copy full SHA for 9451486 - Browse repository at this point
Copy the full SHA 9451486View commit details -
disk: private resolver for CustomPartitionTableOptions.DefaultFSType
A small function for resolving the filesystem type for a mountpoint with the global default. Will return an error if both are unset.
Configuration menu - View commit details
-
Copy full SHA for 01b0ab7 - Browse repository at this point
Copy the full SHA 01b0ab7View commit details -
disk: split NewCustomPartitionTable() into smaller functions
Split the large NewCustomPartitionTable() function into smaller ones for readability and nicer handling of error conditions (no need to beak on a loop label).
Configuration menu - View commit details
-
Copy full SHA for 5bf3416 - Browse repository at this point
Copy the full SHA 5bf3416View commit details -
disk: more custom partition table tests
- Legacy boot mode. - UEFI boot mode. - Auto root creation on btrfs.
Configuration menu - View commit details
-
Copy full SHA for f6fd14a - Browse repository at this point
Copy the full SHA f6fd14aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0266934 - Browse repository at this point
Copy the full SHA 0266934View commit details -
disk: reword invalid pt type error message
Change the error message for invalid partition table types to match the other error messages from the NewCustomPartitionTable() function. Add a test.
Configuration menu - View commit details
-
Copy full SHA for 6b1f037 - Browse repository at this point
Copy the full SHA 6b1f037View commit details