Skip to content

[Feature] KPT KCL SDK #434

New issue
New issue
Closed
Closed
[Feature] KPT KCL SDK#434
Assignees
Peefy
Labels
featintegration
@Peefy

Description

@Peefy
Peefy
opened on Mar 6, 2023

Background

KCL, which is specifically used for configuration writing and policy verification in cloud-native and Kubernetes scenarios. However, it is obviously not enough to use only one language for Kubernetes configuration management until we find KPT. We design KCL plug-ins and SDKs for KPT to write KPT functions. One of the core issues to be addressed by KCL is to enhance configuration tools and provide declarative configuration verification and editing capabilities, serving to a certain extent as a glue layer for Configuration as Data (CaC) and Configuration as Data (CaD). Besides, KCL is a Python-like domain language with enhancement schema models and multiple override strategies for transforming configuration data without out-of-band access to the host filesystem and networking.

For example, the following image shows how KCL can be applied to cloud-native configuration and policy scenarios:

image

kpt is a package-centric toolchain that enables a WYSIWYG configuration authoring, automation, and delivery experience, which simplifies managing Kubernetes platforms and KRM-driven infrastructure (e.g., Config Connector, Crossplane) at scale by manipulating declarative Configuration as Data for automate Kubernetes configuration editing including transforming and validating.

The core concepts of KCL are Config, Schema, Lambda, and Rule, they correspond almost one-to-one to concepts associated with KPT, such as KRM, Package, Function, Validator, etc.

  • Config

image

  • Schema

image

  • Lambda

image

  • Rule

image

  • KPT Concepts

image

In KCL, lambda is designed to be "pure" (for example, it prohibits modifying features such as closure variables), which means that executing the same function multiple times will yield the same results, which naturally conforms to some design specifications of the KPT Function.

It should be added that KCL itself can be compiled into WASM and provides the kclvm-go SDK, which is well integrated into the KPT Function.

We have designed Helm KCL Plugin, Helm is a great Kubernetes package manager, but there are still many needs that cannot be well met (Off-the-shelf packages are rarely deployed without any customization). So we have reason and feasibility to verify the integration of KCL and KPT tools, and KPT has considered the extension mechanism that users use to combine DSL and CaD they like from the beginning of the design.

Design

KPT/KRM KCL Function

A KPT/KRM KCL function contains as follows

image

where:

  • input items: The input list of KRM resources to operate on.
  • output items: The output list obtained from adding, removing, or modifying items in the input.
  • functionConfig: An optional meta resource containing the arguments to this invocation of the function.
  • results: An optional meta resource emitted by the function for observability and debugging purposes.

We use golang to wrap the KPT KCL SDK.

Workflow

We embed KCL into the KPT Go SDK through kclvm-go, and call functions written by KCL through Go code. KCL functions are maintained by users in the KCLRun configuration KRM file or ConfigMap.

image

Function Config

# kcl-fn-config.yaml
apiVersion: fn.kpt.dev/v1alpha1
kind: KCLRun
metadata:
  name: set-annotation
# EDIT THE SOURCE! 
source: |
   # One line KCL transformer.
   [item | {metadata.annotations: {"managed-by" = "kcl-kpt"}} for item in option("resource_list").items]

Besides, we can use a ConfigMap to store the KCL source. To use a ConfigMap as the functionConfig, the KCL script source must be specified in the data.source field. Additional parameters can be specified in the data field.

Here's an example:

apiVersion: v1
kind: ConfigMap
metadata:
  name: set-replicas
data:
  replicas: "5"
  source: |
    resources = option("resource_list")
    setReplicas = lambda items, replicas {
       [item | {
          spec.replicas = replicas
       } if item.kind == "Deployment" and item.apiVersion == "apps/v1" else {} for item in items]
    }
    setReplicas(resources.items or [], resources.functionConfig.data.replicas)

In the example above, the script accesses the replicas parameters using option("resource_list").functionConfig.data.replicas.
To use a KCLRun as the functionConfig, the KCL source must be specified in the source field. Additional parameters can be specified in the params field. The params field supports any complex data structure as long as it can be represented in YAML.

apiVersion: fn.kpt.dev/v1alpha1
kind: KCLRun
metadata:
  name: conditionally-add-annotations
params:
  toMatch:
    config.kubernetes.io/local-config: "true"
  toAdd:
    configmanagement.gke.io/managed: disabled
source: |
  resource = option("resource_list")
  items = resource.items
  params = resource.functionConfig.params
  toMatch = params.toMatch
  toAdd = params.toAdd
  [item | {
     # If all annotations are matched, patch more annotations
     if all key, value in toMatch {
        item.metadata.annotations[key] == value
     }:
         metadata.annotations: {**params.toAdd}
  } for item in items]

More user guide and examples are here

KCL Go KPT Function Catalog Implementation

Reference

Metadata

Metadata

Assignees

Labels

featintegration

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions