Skip to content

Commit

Permalink
Require py39, k8s 1.24, use golang 1.21, update traefik from v2.9 to …
Browse files Browse the repository at this point in the history
…v2.10 (#739)

* Use golang 1.21 to compile dask-gateway-proxy

* Require Python 3.9

* helm chart: require k8s 1.24+

* helm chart: upgrade traefik from v2.9 to v2.10

* ci: update tested versions

* helm chart: avoid test failures on old chart versions schema
  • Loading branch information
consideRatio authored Sep 26, 2023
1 parent 1068863 commit cfc19bd
Show file tree
Hide file tree
Showing 19 changed files with 2,519 additions and 140 deletions.
29 changes: 16 additions & 13 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -56,14 +56,12 @@ jobs:
# we test integration with. We can test against only one version and
# that would be fine.
#
- python-version: "3.8"
go-version: "1.18"
- python-version: "3.9"
go-version: "1.18"
go-version: "1.20"
- python-version: "3.10"
go-version: "1.19"
go-version: "1.21"
- python-version: "3.11"
go-version: "1.19"
go-version: "1.21"

steps:
- uses: actions/checkout@v3
Expand Down Expand Up @@ -131,7 +129,7 @@ jobs:
go test ./internal/logging
kubernetes-tests:
name: "Test k8s backend and Helm chart"
name: Test k8s backend & chart
runs-on: ubuntu-22.04
timeout-minutes: 15

Expand All @@ -146,14 +144,16 @@ jobs:
# k3s-channel: https://update.k3s.io/v1-release/channels
#
include:
- k3s-channel: v1.20
upgrade-from: "0.9.0"
- k3s-channel: v1.24
dask-namespace: default
- k3s-channel: stable
- k3s-channel: v1.25
dask-namespace: separate-namespace
- k3s-channel: v1.26
dask-namespace: default
- k3s-channel: stable
dask-namespace: scheduler
- k3s-channel: latest
upgrade-from: "2022.4.0"
- k3s-channel: v1.27
dask-namespace: default
- k3s-channel: v1.28
dask-namespace: default

steps:
Expand Down Expand Up @@ -199,7 +199,7 @@ jobs:
- if: matrix.dask-namespace != 'default'
run: |
kubectl create namespace ${{ matrix.dask-namespace }} || true
kubectl create namespace ${{ matrix.dask-namespace }} || true
- name: helm install previous version ${{ matrix.upgrade-from }}
if: matrix.upgrade-from != ''
Expand Down Expand Up @@ -232,6 +232,9 @@ jobs:
- name: Manual upgrade of CRDs
if: matrix.upgrade-from != ''
run: |
# A charts CRDs are installed once by default, but they aren't
# upgraded subsequently. Due to this, we manually upgrade the CRDs
# here.
kubectl apply --server-side --force-conflicts -f resources/helm/dask-gateway/crds/daskclusters.yaml
kubectl apply --server-side --force-conflicts -f resources/helm/dask-gateway/crds/traefik.yaml
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ repos:
hooks:
- id: pyupgrade
args:
- --py38-plus
- --py39-plus

# Autoformat: Python code
- repo: https://github.com/PyCQA/isort
Expand Down
8 changes: 4 additions & 4 deletions dask-gateway-server/dask-gateway-proxy/go.mod
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
module github.com/dask/dask-gateway/dask-gateway-proxy

go 1.18
go 1.21

require github.com/stretchr/testify v1.7.1
require github.com/stretchr/testify v1.8.4

require (
github.com/davecgh/go-spew v1.1.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
gopkg.in/yaml.v3 v3.0.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
14 changes: 6 additions & 8 deletions dask-gateway-server/dask-gateway-proxy/go.sum
Original file line number Diff line number Diff line change
@@ -1,12 +1,10 @@
github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.7.1 h1:5TQK59W5E3v0r2duFAb7P95B6hEeOyEnHRa8MjYSMTY=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0 h1:hjy8E9ON/egN1tAYqKb61G10WtihqetD4sz2H+8nIeA=
gopkg.in/yaml.v3 v3.0.0/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -989,7 +989,7 @@ async def create_ingressroute_if_not_exists(self, cluster, sched_pod):
)
try:
await self.custom_client.create_namespaced_custom_object(
"traefik.containo.us", "v1alpha1", namespace, "ingressroutes", route
"traefik.io", "v1alpha1", namespace, "ingressroutes", route
)
except ApiException as exc:
if exc.status != 409:
Expand All @@ -1013,7 +1013,7 @@ async def create_ingressroutetcp_if_not_exists(self, cluster, sched_pod):
self.log.info("Creating scheduler TCP route for cluster %s.%s", namespace, name)
try:
await self.custom_client.create_namespaced_custom_object(
"traefik.containo.us", "v1alpha1", namespace, "ingressroutetcps", route
"traefik.io", "v1alpha1", namespace, "ingressroutetcps", route
)
except ApiException as exc:
if exc.status != 409:
Expand Down Expand Up @@ -1244,7 +1244,7 @@ def make_service(self, cluster_name):
def make_ingressroute(self, cluster_name, namespace):
route = f"{self.proxy_prefix}/clusters/{namespace}.{cluster_name}/"
return {
"apiVersion": "traefik.containo.us/v1alpha1",
"apiVersion": "traefik.io/v1alpha1",
"kind": "IngressRoute",
"metadata": {
"labels": self.get_labels(cluster_name, "dask-scheduler"),
Expand Down Expand Up @@ -1272,7 +1272,7 @@ def make_ingressroute(self, cluster_name, namespace):

def make_ingressroutetcp(self, cluster_name, namespace):
return {
"apiVersion": "traefik.containo.us/v1alpha1",
"apiVersion": "traefik.io/v1alpha1",
"kind": "IngressRouteTCP",
"metadata": {
"labels": self.get_labels(cluster_name, "dask-scheduler"),
Expand Down
3 changes: 1 addition & 2 deletions dask-gateway-server/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -236,7 +236,6 @@ def run(self):
"Topic :: System :: Distributed Computing",
"Topic :: System :: Systems Administration",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
Expand All @@ -258,7 +257,7 @@ def run(self):
package_data=package_data,
install_requires=install_requires,
extras_require=extras_require,
python_requires=">=3.8",
python_requires=">=3.9",
entry_points={
"console_scripts": [
"dask-gateway-server = dask_gateway_server.app:main",
Expand Down
3 changes: 1 addition & 2 deletions dask-gateway/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@
"Topic :: Scientific/Engineering",
"Topic :: System :: Distributed Computing",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
Expand All @@ -52,6 +51,6 @@
package_data={"dask_gateway": ["*.yaml"]},
install_requires=install_requires,
extras_require=extras_require,
python_requires=">=3.8",
python_requires=">=3.9",
zip_safe=False,
)
7 changes: 7 additions & 0 deletions docs/source/changelog.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# Changelog

## Unreleased

### Breaking changes

- The Python packages `dask-gateway` and `dask-gateway-server` now require Python 3.9+
- The Helm chart now requires k8s 1.24+

## 2023.1.1

([full changelog](https://github.com/dask/dask-gateway/compare/2023.1.0...2023.1.1))
Expand Down
1 change: 0 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ profile = "black"
[tool.black]
line-length = 88
target_version = [
"py38",
"py39",
"py310",
"py311",
Expand Down
2 changes: 1 addition & 1 deletion resources/helm/dask-gateway/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ home: https://gateway.dask.org/
sources:
- https://github.com/dask/dask-gateway/
icon: https://avatars3.githubusercontent.com/u/17131925?v=3&s=200
kubeVersion: ">=1.20.0-0"
kubeVersion: ">=1.24.0-0"
Loading

0 comments on commit cfc19bd

Please sign in to comment.