Add in changes to support MPS settings

[yeazelm]

Issue number:

Closes # #4673

Description of changes:
Update the shared defaults configuration files and services to accomodate the nvidia-mps-control-daemon. Add a migration for the new settings.

Testing done:
cargo make completes. For full testing of the feature, look at bottlerocket-os/bottlerocket-core-kit#789

Migration testing passed.

From the console when downgrading:

         Starting Write network status...
[  OK  ] Finished Prepare Containerd Directory (/var/lib/containerd).
[    5.718341] migrator[1537]: 22:11:30 [INFO] Running migration 'migrate_v1.54.0_kubelet-device-plugins-mps-prefix-settings.lz4'
[  OK  ] Finished Prepare Kubelet Directory (/var/lib/kubelet).
[  OK  ] Finished Prepare Opt Directory (/opt).
[  OK  ] Finished Prepare Var Directory (/var).
[  OK  ] Finished Write network status.
[  OK  ] Reached target Network is Online.
         Mounting CNI Plugin Directory (/opt/cni)...
[    5.977593] migrator[1537]: 22:11:31 [INFO] Running migration 'migrate_v1.54.0_kubelet-device-plugins-mps-settings.lz4'
[    6.070127] migrator[1537]: 22:11:31 [INFO] Removing the weak settings and metadata.
         Mounting CSI Helper Directory (/opt/csi)...

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

[piyush-jena]

You might have to add "configuration-files.nvidia-mps-control-daemon-exec-start-conf",.

[piyush-jena]

I was wondering if you have to add a migration for "settings.kubelet-device-plugins.nvidia.device-sharing-strategy" as well because the allowed value can also be mps now.

fn run() -> Result<()> {
    migrate(RestrictListsMigration(vec![ListRestriction {
        setting: "settings.kubelet-device-plugins.nvidia.device-sharing-strategy",
        allowed_vals: &["time-slicing"],
    }]))
}

[yeazelm]

I added a custom function for this since we don't have a helper for Enum variants.

[piyush-jena]

Check the setting - KubernetesCPUManagerPolicyOption (https://github.com/bottlerocket-os/bottlerocket-settings-sdk/pull/104/changes) in settings-sdk and this migration - https://github.com/bottlerocket-os/bottlerocket/blob/develop/sources/settings-migrations/v1.51.0/kubernetes-beta-cpu-manager-policy-options/src/main.rs

[piyush-jena]

It seems like the custom function would still allow "mps" upon downgrade.

[yeazelm]

I saw this after re-writing my code, this is a nice find! My implementation is a bit more targeted and specific to this one use case but that example would have worked as well. would have worked as well.

[yeazelm]

^ Updated with core and kernel kit releases, Bottlerocket SDK update, and comments for migrations.

[yeazelm]

^ Updated the code to break into two migrations. The first migration's datastore would be lost when the second one ran. This keeps the two separate and ensures both complete.

[KCSesh]

Just a note, if we wanted to do 2 sets of migrations in 1, this works as well:

use migration_helpers::{migrate, Migration, MigrationData, Result};
use std::process;

const MPS_PREFIX: &str = "settings.kubelet-device-plugins.nvidia.mps";
const DEVICE_SHARING_STRATEGY_SETTING: &str =
    "settings.kubelet-device-plugins.nvidia.device-sharing-strategy";

pub struct MpsMigration;

impl Migration for MpsMigration {
    fn forward(&mut self, input: MigrationData) -> Result<MigrationData> {
        println!("MpsMigration has no work to do on upgrade.");
        Ok(input)
    }

    fn backward(&mut self, mut input: MigrationData) -> Result<MigrationData> {
        // Remove all settings with the MPS prefix
        let to_remove: Vec<_> = input.data.keys()
            .filter(|k| k.starts_with(MPS_PREFIX))
            .cloned()
            .collect();
        for key in to_remove {
            if let Some(data) = input.data.remove(&key) {
                println!("Removed {key}, which was set to '{data}'");
            }
        }

        // Change device-sharing-strategy from "mps" to "none"
        if let Some(data) = input.data.get_mut(DEVICE_SHARING_STRATEGY_SETTING) {
            if let serde_json::Value::String(s) = data {
                if s == "mps" {
                    *data = serde_json::Value::String("none".to_string());
                    println!("Changed device-sharing-strategy from 'mps' to 'none' on downgrade.");
                }
            }
        }
        Ok(input)
    }
}

fn run() -> Result<()> {
    migrate(MpsMigration)
}

fn main() {
    if let Err(e) = run() {
        eprintln!("{e}");
        process::exit(1);
    }
}

But I think I like the 2 separate migrations since they are different trees anyways.