[Feature] KPT KCL SDK

[Peefy]

Background

KCL, which is specifically used for configuration writing and policy verification in cloud-native and Kubernetes scenarios. However, it is obviously not enough to use only one language for Kubernetes configuration management until we find KPT. We design KCL plug-ins and SDKs for KPT to write KPT functions. One of the core issues to be addressed by KCL is to enhance configuration tools and provide declarative configuration verification and editing capabilities, serving to a certain extent as a glue layer for Configuration as Data (CaC) and Configuration as Data (CaD). Besides, KCL is a Python-like domain language with enhancement schema models and multiple override strategies for transforming configuration data without out-of-band access to the host filesystem and networking.

For example, the following image shows how KCL can be applied to cloud-native configuration and policy scenarios:

kpt is a package-centric toolchain that enables a WYSIWYG configuration authoring, automation, and delivery experience, which simplifies managing Kubernetes platforms and KRM-driven infrastructure (e.g., Config Connector, Crossplane) at scale by manipulating declarative Configuration as Data for automate Kubernetes configuration editing including transforming and validating.

The core concepts of KCL are Config, Schema, Lambda, and Rule, they correspond almost one-to-one to concepts associated with KPT, such as KRM, Package, Function, Validator, etc.

• Config

• Schema

• Lambda

• Rule

• KPT Concepts

In KCL, lambda is designed to be "pure" (for example, it prohibits modifying features such as closure variables), which means that executing the same function multiple times will yield the same results, which naturally conforms to some design specifications of the KPT Function.

It should be added that KCL itself can be compiled into WASM and provides the kclvm-go SDK, which is well integrated into the KPT Function.

We have designed Helm KCL Plugin, Helm is a great Kubernetes package manager, but there are still many needs that cannot be well met (Off-the-shelf packages are rarely deployed without any customization). So we have reason and feasibility to verify the integration of KCL and KPT tools, and KPT has considered the extension mechanism that users use to combine DSL and CaD they like from the beginning of the design.

Design

KPT/KRM KCL Function

A KPT/KRM KCL function contains as follows

where:

• input items: The input list of KRM resources to operate on.
• output items: The output list obtained from adding, removing, or modifying items in the input.
• functionConfig: An optional meta resource containing the arguments to this invocation of the function.
• results: An optional meta resource emitted by the function for observability and debugging purposes.

We use golang to wrap the KPT KCL SDK.

Workflow

We embed KCL into the KPT Go SDK through kclvm-go, and call functions written by KCL through Go code. KCL functions are maintained by users in the KCLRun configuration KRM file or ConfigMap.

Function Config

# kcl-fn-config.yaml
apiVersion: fn.kpt.dev/v1alpha1
kind: KCLRun
metadata:
  name: set-annotation
# EDIT THE SOURCE! 
source: |
   # One line KCL transformer.
   [item | {metadata.annotations: {"managed-by" = "kcl-kpt"}} for item in option("resource_list").items]

Besides, we can use a ConfigMap to store the KCL source. To use a ConfigMap as the functionConfig, the KCL script source must be specified in the data.source field. Additional parameters can be specified in the data field.

Here's an example:

apiVersion: v1
kind: ConfigMap
metadata:
  name: set-replicas
data:
  replicas: "5"
  source: |
    resources = option("resource_list")
    setReplicas = lambda items, replicas {
       [item | {
          spec.replicas = replicas
       } if item.kind == "Deployment" and item.apiVersion == "apps/v1" else {} for item in items]
    }
    setReplicas(resources.items or [], resources.functionConfig.data.replicas)

In the example above, the script accesses the replicas parameters using option("resource_list").functionConfig.data.replicas.
To use a KCLRun as the functionConfig, the KCL source must be specified in the source field. Additional parameters can be specified in the params field. The params field supports any complex data structure as long as it can be represented in YAML.

apiVersion: fn.kpt.dev/v1alpha1
kind: KCLRun
metadata:
  name: conditionally-add-annotations
params:
  toMatch:
    config.kubernetes.io/local-config: "true"
  toAdd:
    configmanagement.gke.io/managed: disabled
source: |
  resource = option("resource_list")
  items = resource.items
  params = resource.functionConfig.params
  toMatch = params.toMatch
  toAdd = params.toAdd
  [item | {
     # If all annotations are matched, patch more annotations
     if all key, value in toMatch {
        item.metadata.annotations[key] == value
     }:
         metadata.annotations: {**params.toAdd}
  } for item in items]

More user guide and examples are here

KCL Go KPT Function Catalog Implementation

• https://github.com/KusionStack/kpt-kcl-sdk

Reference

• KPT: https://github.com/GoogleContainerTools/kpt
• KPT KCL SDK Issue: [Feature Request] Support functions written by KCL and its SDKs for KPT  kptdev/kpt#3671
• KRM Function: https://github.com/kubernetes-sigs/kustomize/blob/master/cmd/config/docs/api-conventions/functions-spec.md#krm-functions-specification
• kpt-functions-catalog: https://github.com/GoogleContainerTools/kpt-functions-catalog
• A KPT go function catalog: https://github.com/GoogleContainerTools/kpt-functions-catalog/tree/master/functions/go/set-namespace
• Kustomize: https://github.com/kubernetes-sigs/kustomize
• Build Custom Kustomize Plugin With Containerized KRM function: https://thirumurthi.hashnode.dev/build-custom-kustomize-plugin-with-containerized-krm-function
• Helm: https://github.com/helm/helm
• Helm Schema Generation Plugin: https://github.com/holgerjh/helm-schema
• Helm Diff Plugin: https://github.com/databus23/helm-diff
• Helm S3 Plugin: https://github.com/hypnoglow/helm-s3
• Validating Helm Chart Values with JSON Schemas: https://www.arthurkoziel.com/validate-helm-chart-values-with-json-schemas/
• Score Helm Chart: https://github.com/score-spec/score-helm-charts
• Helmfile: https://github.com/helmfile/helmfile
• Crossplane: https://github.com/crossplane/crossplane
• keptn: https://github.com/keptn/keptn
• Kubernetes Resource Model (KRM): https://github.com/kubernetes/design-proposals-archive/blob/main/architecture/resource-management.md
• ArgoCD: https://github.com/argoproj/argo-cd
• Terragrunt: https://terragrunt.gruntwork.io/
• Grafana Helm charts: https://github.com/grafana/helm-charts
• Grafana Jsonnet libs: https://github.com/grafana/jsonnet-libs
• Grafana Tanka based on Jsonnet: https://github.com/grafana/tanka
• Grafana Tanka Helm integration: https://tanka.dev/helm#helm-support
• Prometheus Kubernetes Manifests based on Jsonnet: https://github.com/prometheus-operator/kube-prometheus
• https://www.reddit.com/r/kubernetes/comments/118bmwu/helm_or_kustomize_for_my_situation/
• https://www.reddit.com/r/kubernetes/comments/119jolm/separate_repository_for_helm_charts_vs_integrated/
• https://www.reddit.com/r/kubernetes/comments/xx0e6h/why_we_use_cue_and_not_helm/
• Why we use cue not helm: https://cloudplane.org/blog/why-cue
• External Secret: https://external-secrets.io/v0.7.2/
• Kubescape: https://github.com/kubescape/kubescape
• https://github.com/GoogleContainerTools/kpt-functions-catalog/blob/master/functions/go/starlark/starlark/config.go