Add support for kubernetes version 1.30.0

[MichaelKatsoulis]

Proposed commit message

Add support to 1.30 Kubernetes version:

• Update documentation
• Change K8s cluster version to 1.30
• Update controller manager, scheduler and proxy metrics files.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

• Relates to #3068

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @MichaelKatsoulis? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[elasticmachine]

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2024-04-25T13:57:30.775+0000

• Duration: 40 min 8 sec

Test stats 🧪

Test	Results
Failed	0
Passed	4583
Skipped	904
Total	5487

Steps errors

Expand to view the steps failures

Setup kind

• Took 0 min 13 sec . View more details here
• Description: .ci/scripts/kind-setup.sh

Integration tests

• Took 1 min 27 sec . View more details here
• Description: MODULE=kubernetes make -C metricbeat integration-tests

Error signal

• Took 0 min 0 sec . View more details here
• Description: Error 'hudson.AbortException: script returned exit code 2'

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[MichaelKatsoulis]

Note:

Jenkins tests fail because it cannot create a kubernetes cluster with 1.30.0 version, as there is not such an image yet for kind in https://hub.docker.com/r/kindest/node/tags.
We should wait until the image gets released.

[MichaelKatsoulis]

/test

[MichaelKatsoulis]

The PR is now ready for review!
@fearful-symmetry or @faec could you also review it as part of data plane team?

[gizas]

Can we update the list based on the https://kubernetes.io/releases/?

[MichaelKatsoulis]

As kind is used to spin up the kubernetes clusters, we can only use the available kindest/node as found in https://hub.docker.com/r/kindest/node/tags?page=&page_size=&ordering=&name=.

The latest for 1.27 available is v1.27.13
For 1.28 the latest available is v1.28.9
For 1.29 the latest available is v1.29.4.

Do you suggest to use those or keep the ones we already have?

[gizas]

The latter the better then. So I would say to update them

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[tetianakravchenko]

in those files is defined K8S_VERSION:
https://github.com/elastic/beats/blob/main/.buildkite/deploy/kubernetes/deploy-k8s-pipeline.yml
https://github.com/elastic/beats/blob/main/.buildkite/metricbeat/pipeline.yml
https://github.com/elastic/beats/blob/main/.buildkite/filebeat/filebeat-pipeline.yml
https://github.com/elastic/beats/blob/main/Jenkinsfile
should it be also updated?

[tetianakravchenko]

Please update documentation https://github.com/elastic/beats/blob/main/metricbeat/module/kubernetes/_meta/docs.asciidoc#compatibility

[elasticmachine]

Pinging @elastic/obs-ds-hosted-services (Team:obs-ds-hosted-services)

[MichaelKatsoulis]

in those files is defined K8S_VERSION: https://github.com/elastic/beats/blob/main/.buildkite/deploy/kubernetes/deploy-k8s-pipeline.yml https://github.com/elastic/beats/blob/main/.buildkite/metricbeat/pipeline.yml https://github.com/elastic/beats/blob/main/.buildkite/filebeat/filebeat-pipeline.yml https://github.com/elastic/beats/blob/main/Jenkinsfile should it be also updated?

These tests are new. @oakrizan should we update those files, testing with the new kubernetes version as part of this PR ?

[MichaelKatsoulis]

This PR scope is to update and test the support of kubernetes version 1.30.0 for kubernetes metricbeat module . The tests and files of #39210 (review) are not maintained by elastic/obs-ds-hosted-services team. So I will leave them out of this PR.