implement the last subresources

[clux]

Subresources have a lot of special case logic, that is not easily derivable from k8s_openapi. So far we have implemented (see subresources.rs):

• pods/eviction (impl in add eviction subresource - for #127 #393)
• pods/log (already implemented, used as example herein)
• pods/attach (hard - impl in Add WebSocket connection support and implement /attach and /exec #360)
• pods/portforward (hard How should we implement WebSocket connection? #229, sketch in Add port forward #446)
• pods/exec (hard How should we implement WebSocket connection? #229 - impl in Add WebSocket connection support and implement /attach and /exec #360)
• {service,node,pod}/proxy - see Proxy Subresource #1161
• pods/binding (see https://docs.rs/k8s-openapi/latest/src/k8s_openapi/v1_26/api/core/v1/binding.rs.html#79-100 )
• ephemeral containers - Implement ephemeral containers subresource #1153
• certificatesigningrequest/approval (easy - follow from Request: support arbitrary subresources #487 and see docs) - Add Api fns for arbitrary subresources and approval subresource for CertificateSigningRequest #773
• TokenRequest - for client util (same as csr approach, easy) - done in Add create_subresource method to Api and create_token_request method to Api<ServiceAccount> #989
• arbitrary subresources - probably port Request work from Request: support arbitrary subresources #487 (comment) to API - Add Api fns for arbitrary subresources and approval subresource for CertificateSigningRequest #773

The general outline for the http based subresources:

We need a definition of what they are in subresources. The way to upgrade would be:

1. Implement subresource queries straight on Resource without restrictions (for now):

impl Resource {
    /// Get a pod logs
    pub fn log(&self, name: &str, lp: &LogParams) -> Result<http::Request<Vec<u8>>> {
        ...
        // Check https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.17/ for urls
    }
}

2. Make a marker trait for the subresource:

pub trait Loggable {}

impl<K> Api<K>
where
    K: Clone + DeserializeOwned + Meta + Loggable,
{
    pub async fn log(&self, name: &str, lp: &LogParams) -> Result<String> {
        let req = self.api.log(name, lp)?;
        Ok(self.client.request_text(req).await?)
    }

    pub async fn log_stream(&self, name: &str, lp: &LogParams) -> Result<impl Stream<Item = Result<Bytes>>> {
        let req = self.api.log(name, lp)?;
        Ok(self.client.request_text_stream(req).await?)
    }
}

3. Designate what resources implement this subresource:

impl Loggable for k8s_openapi::api::core::v1::Pod {}

Write one test case for one resource in examples (e.g. examples/log_openapi.rs).

EDIT (0.46.0):

Now this is not as difficult for the specialized requests requiring websockets (ws feature).
This was discussed at length in #229, and finally implemented in #360.
Implementors of the last subresources ought to look at that PR for help.

[clux]

0.46.0 introduces the ws feature using async-tungstenite to pave the way for the last subresources listed above. Have updated the issue slightly. Implementations of the last ws-based subresources can look at #360 for an excellent PR that covers (among the websocket handling itself) examples for implementing pods/attach and pods/exec.

[kazk]

I'll do pods/portforward. kube should probably just provide AsyncRead + AsyncWrite for each port and not bind to local ports. Users should be able to do that if they want to.

I have a rough sketch that can do something similar to Python client's example:

// Portforward nginx container
let mut pf = pods.portforward("example", &[80]).await?; // Portforwarder
let mut ports = pf.ports().unwrap(); // Vec<Port>
let mut port = ports[0].stream().unwrap(); // impl AsyncRead + AsyncWrite + Unpin
port.write_all(b"GET / HTTP/1.1\r\nHost: 127.0.0.1\r\nConnection: close\r\nAccept: */*\r\n\r\n")
    .await?;
let mut rstream = tokio_util::io::ReaderStream::new(port);
if let Some(res) = rstream.next().await {
    match res {
        Ok(bytes) => {
            let response = std::str::from_utf8(&bytes[..]).unwrap();
            println!("{}", response);
            assert!(response.contains("Welcome to nginx!"));
        }
        Err(err) => eprintln!("{:?}", err),
    }
}

I'll open a draft PR later (it's too messy at the moment to show...) to discuss the design.

That should make kube a gold client except for "Proto encoding".

Gold Requirements

• Support exec, attach, port-forward calls (these are not normally supported out of the box from swagger-codegen)
• Proto encoding

I'm not sure what that means. Do we need to support Protobufs?

[clux]

Yeah, that sounds sensible. Listening sounds like an application concern. In an example we can maybe use TcpListener or something and direct it to the writable stream to show how to do the kubectl port-forward equivalent.

protobufs

Ugh. Ah, yeah, I am not sure that's really possible atm. The go api has protobuf codegen hints (see api/types.go) like:

// +optional
// +patchMergeKey=name
// +patchStrategy=merge
EphemeralContainers []EphemeralContainer `json:"ephemeralContainers,omitempty" patchStrategy:"merge" patchMergeKey:"name" protobuf:"bytes,34,rep,name=ephemeralContainers"`

whereas the (huge) swagger codegen has the equivalent json for that part of the PodSpec:

        "ephemeralContainers": {
          "description": "...",
          "items": {
            "$ref": "#/definitions/io.k8s.api.core.v1.EphemeralContainer"
          },
          "type": "array",
          "x-kubernetes-patch-merge-key": "name",
          "x-kubernetes-patch-strategy": "merge"
        },

think the 34 there would be required, but haven't looked at this much. Arnavion might be able to say for sure whether it's possible, I can ping him on k8s-openapi.

[kazk]

The official JavaScript client is Gold and just have something like this (only deserializing):

export class ProtoClient {
    public readonly 'config': KubeConfig;
    public async get(msgType: any, requestPath: string): Promise<any> {
        // ...
        const req = http.request(options);
        const result = await new Promise<any>((resolve, reject) => {
            let data = '';
            req.on('data', (chunk) => {
                data = data + chunk;
            });
            req.on('end', () => {
                const obj = msgType.deserializeBinary(data);
                resolve(obj);
            });
            req.on('error', (err) => {
                reject(err);
            });
        });
        req.end();
        return result;
    }
}

https://github.com/kubernetes-client/javascript/blob/master/src/proto-client.ts

(BTW, it can't portforward more than one port. https://github.com/kubernetes-client/javascript/blob/a8d4f1c7bea2b27403d380e22e492b6680f80b31/src/portforward.ts#L19)

I guess the requirement for Gold is not well-defined? Python one seems more capable, but that's Silver.

---

When I searched "Kubernetes Proto Encoding", https://kubernetes.io/docs/reference/using-api/api-concepts/#protobuf-encoding came up.

[clux]

Yeah, I think the client grading document is a thing they they wrote and applied to the clients they generated themselves. It technically doesn't apply to us (because we don't have our client in that place they say it should be in), but it's a nice benchmark otherwise. I have opened an issue to continue protobuf discussions in #371 at any rate.

[chroche]

Naive question about pods/eviction, this feature doesn't seem to get much love at all. Is there a possible workaround / a direct way to create ak8s_openapi Eviction? Is there a reason why there seems to be no interest in this feaure?

[clux]

It's probably just that no one has asked for it yet. It actually looks very simple. I can have a go at adding it.

[clux]

Eviction handled in #393

[jmintb]

Can I have a go at implementing ephemeral container support?

[clux]

@jmintb Go for it!