Remove worker-template from dask-kubernetes config

[jhamman]

I'd like to propose dask-kubernetes deprecate the current usage of worker-template and worker-template-path from the config file. I suggest this for a few reasons:

• Specification of the full cli executable sequence is implemented as a template in other cluster managers. Typically (e.g. dask-jobqueue, dask-yarn), keys like cores, processes, and memory are provided to the Cluster constructor.
• It is easy to end up with templates that have misaligned specifications (i.e. a user may edit dask's memory-limit without editing the resource requirements.

Here's an example of the default dask-kubernetes setup:

dask-kubernetes/dask_kubernetes/kubernetes.yaml

Lines 11 to 38 in e17accc

	worker-template-path: null
	worker-template: {}
	# kind: Pod
	# metadata:
	# labels:
	# foo: bar
	# baz: quux
	# spec:
	# restartPolicy: Never
	# containers:
	# - image: daskdev/dask:latest
	# args:
	# - dask-worker
	# - --nthreads
	# - '2'
	# - --no-bokeh
	# - --memory-limit
	# - 6GB
	# - --death-timeout
	# - '60'
	# resources:
	# limits:
	# cpu: "1.75"
	# memory: 6G
	# requests:
	# cpu: "1.75"
	# memory: 6G

Ideally, we can switch to something more concise:

kubernetes:
  worker:
    cores: 2
    processes: 1
    memory: 6G
    image: daskdev/dask:latest

Where the template and pod spec can be created on the fly.

[mrocklin]

From my perspective the challenge here is that it's hard to wrap up all of the options in a comprehensive way. Someone will come by with a more custom configuation that they'd like to use.

I think it'd be fine to have an alternate approach to construct a pod spec template, (this already exists in a utility function I think) but I think that there is a lot of value to being able to pass through a template.

[jacobtomlinson]

I agree. I think it's important to be able to pass a full spec through for maximum flexibility and customization.

However I'm not opposed to also having a more convenient way to create specs like the one Joe proposes. This could even be the default in the config, but should be overridden by a full pod spec.

[jhamman]

Okay, I think I agree with you both now. As @mrocklin says, there is already a utility function to do this so we just need to pass config options through.

[jacobtomlinson]

The classic KubeCluster was removed in #890. All users will need to migrate to the Dask Operator. Closing.