Proposal to swap deprecated request for node-fetch

[OliverMKing]

Proposal to swap deprecated request for node-fetch

Request is fully deprecated requiring us to switch libraries (see #414 for more information).

@brendanburns outlined some of the clear options forward. This change should ideally be a long-term solution leaving us with the option of either modifying the node-typescript open-api-generator to use a different request library or migrate to one of the other generator options.

Modifying an outdated node-typescript library does not seem like the right path forward. It would require us to still use a new library which would likely change the api and still be a breaking change. We might as well migrate to a newer version of openapi-generator while we do that and acquire the advantages of a more up-to-date version.

Choosing a new library

@d-cs posted a nice summary of the different choices. We narrowed it down to Axios or Fetch. The other libraries didn't match how widely adopted and focused these libraries are.

Fetch is simply the standard JavaScript as a whole is moving to, which brings more longterm support aligning us better with our goal of migrating to a long-term solution. node-fetch has significantly more weekly downloads than axios. The package size of node-fetch is also noticeably smaller than axios.

Additionally, future improvements could likely be done to allow us to also support the browser #165 since Fetch is native to the browser.

Node-fetch

Fetch is a native browser API that is not implemented in Node.js by default. Node-fetch is the natural choice for the Node implementation because it is the most widely adopted.

The openapi-generator uses the browser implementation of Fetch so we must substitute node-fetch in manually.

Implementation steps

@davidgamero and I will work to implement the following changes.

Other repositories

• Update kubernetes-client/gen's typescript-fetch files to let us pass in the typescriptThreePlus config option ^{1 2}
• Update openapi-generator's typescript-fetch flavor to mark parameters as optional if all parameters are optional ³

Kubernetes-client repository

• Increment OPENAPI_GENERATOR_COMMIT to be version 5.3.0
• npm install node-fetch to install node-fetch
• Switch generate-client script to use typescript-fetch
• Import and inject node-fetch in src/api.ts with the following

import fetch from 'node-fetch';

// inject node-fetch
if (!globalThis.fetch) {
    // @ts-ignore
    globalThis.fetch = fetch;
    globalThis.Headers = Headers;
    globalThis.Request = Request;
    globalThis.Response = Response;
}

• Generate api with npm run generate
• Match src/gen/api.ts to new generated layout (it changes slightly)
• Fix errors in /src folder (due to new api)
• Fix errors in test (due to new api)
• Fix examples (due to new api)
• Document breaking changes for users
• Release new version

[brendandburns]

This approach is going to introduce breaking changes for consumers of this client library (there were 151k downloads last week alone).

We need to have a more careful plan for how we roll out this change.

I would like to support existing users for at least a few versions (perhaps 3?) while we make this transition.

Perhaps the right way to do this is to introduce a new major version to signify the new generator?

e.g.

0.17.x == old generator, Kubernetes 1.23 API
0.18.x == old generator, Kubernetes 1.24 API
0.19.x == old generator, Kubernetes 1.25 API
Support for old generator stops after 1.25

1.0.x == new generator, Kubernetes 1.23 API
1.1.x == new generator, Kubernetes 1.24 API
1.2.x == new generator, Kubernetes 1.25 API
Support for subsequent kubernetes versions continues with the new generator.

This will require introducing a new branch for the 1.x series of releases, that's probably going to be necessary anyway since we don't want to make this change in a single PR.

Please note that this is going to be a big change that will touch nearly the entire library, I think

[OliverMKing]

That phasing plan makes sense to me. It's in-line with the npm semantic versioning.

It might make sense to deprecate the 0.x releases with a warning telling them that support for the old client api will no longer be supported after a certain version and encouraging them to upgrade. That should alert existing users of the changes.

I'm prepared to take on the responsibility associated with this upgrade.

[dominykas]

Is using both generators and fetch/request in parallel not an option during the transition? I haven't thought through all implications or the workload of doing that, just thinking out loud, but it would be nice if you could do something like makeApiClient(CoreV1Api, { fetch: true }) for the next 3 releases or smth, and only do the breaking change then?

[davidgamero]

I agree it would be nice if we could keep the user experience that cohesive, but unfortunately these changes seem too broad and breaking for that to be doable. Given the extent of the proposed changes, it could introduce significant overhead and complexity to have two major versions bundled in a single release.

I've taken a shot at incrementing OPENAPI_GENERATOR_COMMIT as proposed, and it appears that several response types have changed, an example is the removals of the .body properties ex:

(await api.listNamespacedPod(namespace)).body -> (await api.listNamespacedPod({ namespace }))

Another predictable area of major changes is the error interfaces since we are changing the network library, and the proposal includes incrementing typescript by a major version.

This may be a config option that I've inadvertently changed, but IMO it does come across cleaner.

This could be a concern if we had { fetch: true } as an option, since the option wouldn't just change functionality under the hood, but also several parts of the api interfaces.

I'll keep checking it out. There could be an easier way tweaking the generator options that I'm not familiar with.

[brendandburns]

One other option would be to package the old generated code and the new generated code into different directories and effectively have:

import api_old from @kubernetes/node-client

or

import api_new from @kubernetes/node-client

e.g. keep everything in one place from a release perspective, but introduce a fork in the imports that you can use.

[davidgamero]

That could definitely work if we introduce a named export for api_old. I'm not sure how we could do that without potentially breaking existing default import compatibility unless we nest the api_new inside it.

Another possibility would be to have

import api_old from @kubernetes/node-client

or

import api_new from @kubernetes/node-client/fetch

to retain existing import compatibility for import k8s from @kubernetes/node-client

1. Current Top Level Exports

// ✅  `import { CoreV1Api } from @kubernetes/node-client`
// ✅ `import k8s from @kubernetes/node-client`
@kubernetes/node-client
├── CoreV1Api
...
└── KubeConfig

2. Named Level Exports

// ✅ `import { api_new } from @kubernetes/node-client`
// ✅ `import { api_old } from @kubernetes/node-client`
// ❌ `import k8s from @kubernetes/node-client` <- no longer works
@kubernetes/node-client
├── api_old
│    ├─ CoreV1Api
│    ...
│    └─ KubeConfig
│
└── api_new
     ├─ CoreV1Api
     ...
     └─ KubeConfig

3. Messy Nested Export

// ✅ `import { api_new } from @kubernetes/node-client`
// ✅ `import { api_old } from @kubernetes/node-client`
// ✅ `import k8s from @kubernetes/node-client` <- works still, but gross way to do it
@kubernetes/node-client
├── CoreV1Api
...
├── KubeConfig
└── api_new
     ├─ CoreV1Api
     ...
     └─ KubeConfig

[brendandburns]

I think there are different kinds of breaking changes. I feel like changing an import statement is a much lower cost breaking change vs changing the entire API surface area. So I would be ok to require

import { api_new as api } from @kubernetes/node-client

or

import { api_old as api } from @kubernetes/node-client

And I think that is feasible...

[OliverMKing]

I don't think putting both the new api and old api in the same package / directory makes sense to me.

The fact that it's a breaking change between them means that someone can't just swap out API's when making the change.

// switching from this
import { api_new as api } from @kubernetes/node-client
// to this would still require code changes in lots of places where the api is used
import { api_old as api } from @kubernetes/node-client

Someone migrating to the new api would have to change every single node-client import statement in their repo which would be time consuming and tedious. It's simpler from a user-standpoint to simply upgrade their version then fix any errors in the code.

By having both the old and new api in the same package we also wouldn't be able to take advantage of a npm deprecation warning which would help warn users of the upcoming drop of support for the old api.

Users wouldn't be upgraded to the breaking api without doing it manually (or changing their package.json to allow for major release updates) due to npm semantic versioning.

From a user-standpoint I don't think it's easier to change the import and I believe one of our goals should be gently pushing consumers to migrate when they get the chance (or at least be aware of upcoming drops in support). From our standpoint it's more difficult to work with a directory layout that has two apis rather than just different branches. Storing the new and old api in the in the same package would make more sense to me if there was some reason why a user would want to use both the old and new api but I can't think of one.

[davidgamero]

The major version suggestion seems be the cleanest way.

Putting the changes in a new major version would make upgrading a clear decision without interfering with existing code.

That way we don't mix code from two generators in a single package.

[ivanstanev]

Completely supporting the major version upgrade. Our team doesn't mind spending some time adjusting code to the new API, as long as we can stop using deprecated or regularly vulnerable packages! 😄

[brendandburns]

Ok, sounds like the consensus is for a major version upgrade. I have created a release-0.x branch, which we will use for the releases of the old generated client for the next ~9 months.

We can start sending PRs to the master branch now to start making these changes.

[brendandburns]

We should also probably turn this discussion into a markdown document that describes our approach and links off of the main README.md

[OliverMKing]

Quick update: @davidgamero and I have been working on this for the last two weeks. We're having to make some changes to open-api generator to support the way we need to do auth. @davidgamero has been working on identifying the exact improvements necessary.