Document the version of Dask associated with images for helm charts?

[TomAugspurger]

I'm deploying Dask gateway and Jupyterhub independently (with helm). Right now, I'm just using the default images in dask-gateway, and I want to sync up the version of Dask in my jupyterhub deployment so that the version of Dask on my client matches the scheduler and workers.

Right now, figuring that out involves going to https://github.com/dask/dask-gateway/blob/master/resources/helm/images/dask-gateway/Dockerfile, and inspecting that Dockerfile. It'd be nice if that was easier to find, possibly at https://dask.org/dask-gateway-helm-repo/ or on a list at https://gateway.dask.org/install-kube.html.

[consideRatio]

@minrk made the following PR to z2jh that help us manage dependencies: jupyterhub/zero-to-jupyterhub-k8s#1562

Min added a script that let us manage requirements.in that isn't pinned or very loosely pinned, and then the script will freeze dependencies from within an associated Dockerfile, which is then extracted to requirements.txt. The script can also check if the frozen requirements.txt is up to date or not, which can be part of a travis cronjob or similar to provide info if we should refreeze etc.

[rsignell]

After reading this I'm still not sure how to tell when the latest version of dask no longer work s with the latest version of dask gateway.

For example, right now it seems dask==2023.12.0 works just fine with dask-gateway=2023.9.0, so I'm tempted not to pin dask and just use the latest. But at some point that approach could fail, right?

[consideRatio]

Dask-gateway the client is very robust for use against dask-gateway-server's api-versions etc. I'd say don't worry about the version of dask-gateway in relation to other dask software.

[consideRatio]

I can expand later, but i'm on parental leave and little time on mobile - but it would be relevant to clarify how little coupling dask-gateway itself has with the software running in the clusters spawned by dask-gateway-server on requests from dask-gateway clients.

It will matter to have dask software alignment between scheduler, workers, and the environment where jobs are submitted from i think - that coupling is far more important.