Better support for x-kubernetes properties in schemas

[clux]

There is a slew of x-kubernetes extension properties that is usable inside CRD schemas to improve the validation and merge behaviour of structures of members.

From Kubernetes 1.25 this is being leaned into even more heavily with x-kubernetes-validations rules on CRD schemas:

  openAPIV3Schema:
    type: object
    properties:
      spec:
        type: object
        x-kubernetes-validations:
          - rule: "self.minReplicas <= self.replicas"
            message: "replicas should be greater than or equal to minReplicas."
          - rule: "self.replicas <= self.maxReplicas"
            message: "replicas should be smaller than or equal to maxReplicas."

This type of validation makes it possible to write controllers with much richer type of native validation without having to lean on the clunkier admission controllers / admission controller frameworks to manage extraneous policy rules - and has a lot of potential use. There's a KubeConNA'23 talk called Declarative Everything that explores this and the direction of the feature from sig apimachinery.

Current Usage

Such properties can be specified via schemars attributes to override the schema generation for a type as per

kube/examples/crd_derive_schema.rs

Lines 84 to 102 in 3a4f724

	// Listable field with specified 'set' merge strategy
	#[serde(default)]
	#[schemars(schema_with = "set_listable_schema")]
	set_listable: Vec<u32>,
	}
	// https://kubernetes.io/docs/reference/using-api/server-side-apply/#merge-strategy
	fn set_listable_schema(_: &mut schemars::gen::SchemaGenerator) -> schemars::schema::Schema {
	serde_json::from_value(serde_json::json!({
	"type": "array",
	"items": {
	"format": "u32",
	"minium": 0,
	"type": "integer"
	},
	"x-kubernetes-list-type": "set"
	}))
	.unwrap()
	}

This is not ideal because you then have to manage all the other schema properties for that type yourself.

Ideas

Schemars Extensions

To properly support adding properties like this, it would be interesting to see if schemas which has support for a large amount of attributes already, could be enhanced with a kubernetes attribute feature set to allow stuff like e.g.:

struct Spec {
    #[schemars(x-kubernetes-validations(
        rule = "self.minReplicas <= self.replicas",
        message = "replicas should be greater than or equal to minReplicas."
    ))]
    spec: usize,
}

Have asked upstream about what they think about this in GREsau/schemars#258

CEL Crate?

Maybe it's better to start some of this from a CEL POV because maybe we want to be able to run validations we put into CEL rules locally. In this case a crate that allows us to do something like

struct Spec {
    #[cel_validation(
        rule = "self.minReplicas <= self.replicas",
        message = "replicas should be greater than or equal to minReplicas."
    )]
    spec: usize,
}

in either case we probably still need something in schemars so that the rules are forwarded into the schema, but it means we will have our own properties (like serde/validator) that schemars can pickup on later.

And we likely would need a new crate or majorly extend the cel-parser / cel-interpreter crate (both of which live in cel-rust).

[benkeil]

This would be a great feature. Looking for how to enforce immutability

[azat]

FWIW here is a better example, that allows to preserve existing schema - https://github.com/s3rius/kuo/blob/4c83b4033093e84035cefb0f5a22c46c7b9d471c/src/crds/managed_user.rs#L70-L84

[clux]

That example is really useful @azat, thank you.

We could maybe create a higher level version of that that allows generating schema modifying functions like that, if we can curry the "rule" and "message" as initial arguments, and return the mutating fn from this.