Minor changes to KEP 5339

Signed-off-by: michaelawyu <chenyu1@microsoft.com>

Author: michaelawyu

keps/sig-multicluster/5339-clusterprofile-plugin-credentials/README.md

@@ -224,7 +224,8 @@ controller, applications or consumers without requiring changes. It also cannot
 is not sensitive.
 
 The definition is as follows:
-```
+
+```golang
 type CredentialProviders struct {
   // +listType=map
   // +listMapKey=name
@@ -234,10 +235,17 @@ type CredentialProviders struct {
 // CredentialsTypes defines the type of credentials that are accepted by the cluster. For example, GCP credentials (tokens that are understood by GCP's IAM) are designated by the string `google`.
 type CredentialsType string
 
+// AdditionalInformation is a piece of additional information with a nickname.
+type AdditionalInformation struct {
+  Name string
+  Data runtime.RawExtension
+}
+
 // CredentialsConfig gives more details on data that is necessary to reach out the cluster for this kind of Credentials
 type CredentialsConfig struct {
   Name string // name of the provider type
-  Cluster *Cluster // Configuration to reach the cluster (endpoints, proxy, etc) // See following section for details.
+  Cluster *Cluster // Configuration to reach the cluster (endpoints, proxy, etc) // See the sections below for details.
+  AdditionalInfo []AdditionalInformation // AdditionalInfo holds additional information that might be of use to the credential provider. See the sections below for details.
 }
 ```
 
@@ -290,6 +298,75 @@ In this structure, not all fields would apply, such as:
 
 * `CertificateAuthority`, which points to a file (and a ClusterProfile doesn't have a filesystem)
 
+##### About the `Extensions` field
+
+The `Cluster` struct defined in `client-go` ([link](https://github.com/kubernetes/client-go/blob/d32752779319f587c42ff9edbc6ed533575f2136/tools/clientcmd/api/types.go#L69))
+features a field, `extensions`, for holding additional information about the cluster, with each extension associated with a name.
+[KEP-541](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/541-external-credential-providers/README.md) further reserves a name,
+`client.authentication.k8s.io/exec`, for per-cluster additional information for authentication exec plugins.
+Per KEP-541's explanation, if the `client.authentication.k8s.io/exec` extension has been set in the `Cluster` struct, the data shall be
+parsed and populated into the `Config` field of the `ExecConfig` struct (also defined in the `client-go` package, see [link](https://github.com/kubernetes/client-go/blob/d32752779319f587c42ff9edbc6ed533575f2136/tools/clientcmd/api/types.go#L209)).
+If the `ExecConfig` struct has its `ProvideClusterInfo` flag set to true, the `client-go` package, upon invocation of an exec plugin, will build a
+`Cluster` object (from the `client.authentication.k8s.io` API group, see [link](https://pkg.go.dev/k8s.io/client-go/pkg/apis/clientauthentication#Cluster)),
+which includes the parsed extension data (`Config` field in the `ExecConfig` struct), and save it to an environment variable, `KUBERNETES_EXEC_INFO`.
+
+For workflow defined in this KEP, however, as described earlier in this document, it is up to the community-provided library to build a 
+`rest.Config` and return it to the caller based on some user-supplied exec plugin information, which already features an `ExecConfig` struct
+(with the path to the exec plugin, arguments, environment variables, etc.) before it reads a `ClusterProfile` object; if the `ClusterProfile` object
+has extensions set, it might be of conflict with the `ExecConfig.Config` field that the community libraries sees (typically, as KEP-541 dictates, data
+in `ExecConfig.Config` should be sourced from `Cluster.Extensions`). Aside from the conflicts, there are other complications as well:
+
+* without proper hints, it would not be clear how the data from the `client.authentication.k8s.io/exec` extension entry should be parsed; to use the data,
+the credential provider code must know a concrete object type that implements `runtime.Object` so that it could deserialize the data, in the form of
+`runtime.RawExtension`, to the object type; or choose to use `runtime.Unknown` as the object type (with its type meta information possibly missing).
+* whether the data from the `extensions` field can be seen by the exec plugin is ultimately decided by the `ProvideClusterInfo` flag in the `ExecConfig`, which
+is solely controlled by the credential provider code. If the flag is unset, any extension set on the cluster profile object will not used at all. Note also 
+that the default value for `ProvideClusterInfo` is set to false.
+* there are also cases where setting the `ProvideClusterInfo` flag to true is not desirable or practicable, as it might involve writing a very large piece of CA data
+as an environment variable.
+* there are exec plugins that do not read the `KUBERNETES_EXEC_INFO` environment variable, or only read partial data from it (e.g., API versions only).
+
+For the reasons explicated above, it is suggested that, when setting the `Cluster` struct in a `ClusterProfile` object, the `extensions` field should be handled
+by the community-provided credential provider library as follows:
+
+* If the user-supplied `ExecConfig` struct has a non-nil `Config` field, or the `ProvideClusterInfo` flag is set to false, the library shall ignore
+any `extensions` data set in `ClusterProfile` objects;
+* Otherwise, the library checks for the reserved name, `client.authentication.k8s.io/exec`, in the `extensions` field from the `ClusterProfile` object
+in use, and source the `ExecConfig.Config` field accordingly by saving the the `extensions` data as a `runtime.Unknown` object. 
+
+Considering the complications with the usage of `extensions` field in a multi-cluster environment for authentication, users might need a more direct way
+to provide cluster-specific information to the exec plugin to complete the authentication workflow; the information might include client IDs,
+tenant IDs, and/or audiences. Such information is very difficult to handle on the controller side as it either requires engineers to hard-code
+the values for each cluster, or implement some form dynamic discovery mechanism. To facilitate such use cases, this KEP adds an `AdditionalInfo` field
+to the `CredentialsConfig` struct in the Cluster Profile API, as specified in the next section, so that cluster-specific information, if applicable,
+can be discovered via the Cluster Profile API. It can also be used to allow other forms of extensions.
+
+#### `AdditionalInfo`
+
+The `AdditionalInfo` field in the `CredentialsConfig` struct, as added by this KEP to the Cluster Profile API, holds additional information that might help
+the credential provider code complete the authentication workflow. Each additional information entry is uniquely identified by a name, and features
+a piece of arbitrary data; it is up to the credential provider code the process and apply the data.
+
+Furthermore, for authentication with exec plugins, this KEP reserves the following two names and dictates how the additional information should be
+formatted and used under the two names:
+
+* `multicluster.x-k8s.io/clusterprofiles/auth/exec/additional-args`
+
+  This extension type supplies additional arguments that should be added when calling an exec plugin; if the credential provider code features its own set of
+  arguments to use locally, the additional arguments in this extension should be appended. This might be useful in cases where the authentication workflow
+  requires cluster-specific information, such as audiences, IDs, etc. Secrets or any form of sensitive data should not be stored in this extension.
+
+  The additional arguments should be stored as a string array in the YAML format.
+
+* `multicluster.x-k8s.io/clusterprofiles/auth/exec/additional-envs`
+
+  This extension type supplies additional environment variables that should be unioned to other environment variables when calling an exec plugin; if the
+  credential provider code features its own set of environment variables to add locally, the additional variables in this extension should be appended.
+  This might also be useful in cases where the authentication workflow requires cluster-specific information, such as audiences, IDs, etc.
+  Secrets or any form of sensitive data should not be kept in this extension.
+
+  The additional environment variables should be stored as a mapping between strings in the YAML format.
+
 
 #### ClusterProfile Example
 
@@ -312,9 +389,9 @@ status:
    - name: location
      value: us-central1
   credentialProviders:
-  - name: google
-    cluster:
-      server: https://connectgateway.googleapis.com/v1/projects/123456789/locations/us-central1/gkeMemberships/my-cluster-1
+    google:
+      cluster:
+        server: https://connectgateway.googleapis.com/v1/projects/123456789/locations/us-central1/gkeMemberships/my-cluster-1  
 ```