[autoscaler] make 0 default min/max workers for head node

[sasha-s]

Why are these changes needed?

Related issue number

Checks

• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[DmitriGekhtman]

It's customary to add a tag to the title to aid in release tracking -- added "[autoscaler]".

[DmitriGekhtman]

typo in function name

[sasha-s]

thanks

[DmitriGekhtman]

We want max_workers for the head node type defaulted to 0 (and to global max_workers for the rest of the node types).

I think the autoscaler infers min_workers internally 0 if min_workers is missing for a node type.

[sasha-s]

Changed it to 0 form the head node, setting min_workers to 0 as default.

[DmitriGekhtman]

Other things that need to be changed:

There's a test in test_cli.py that explicitly checks for the max_workers warning message that's been deleted -- that test and the file it uses should be deleted.

The documentation on node type max workers should be updated
https://github.com/ray-project/ray/blob/55680a1f9ed3d8d1d4762039221b9fba4645383b/doc/source/cluster/config.rst

The helm chart template

ray/deploy/charts/ray/templates/raycluster.yaml

Lines 22 to 23 in 55680a1

	minWorkers: {{ $val.minWorkers }}
	maxWorkers: {{ $val.maxWorkers }}

needs to be updated to ignore min_workers and max_workers if they're not specified in values.yaml

ray/deploy/charts/ray/values.yaml

Lines 16 to 17 in 55680a1

	minWorkers: 0
	maxWorkers: 0

.

There's another example that needs to be modified here

ray/deploy/components/example_cluster.yaml

Lines 33 to 35 in 55680a1

	minWorkers: 0
	# Maximum number of Ray workers of this Pod type. Takes precedence over minWorkers.
	maxWorkers: 0

[sasha-s]

Other things that need to be changed:

There's a test in test_cli.py that explicitly checks for the max_workers warning message that's been deleted -- that test and the file it uses should be deleted.

The documentation on node type max workers should be updated
https://github.com/ray-project/ray/blob/55680a1f9ed3d8d1d4762039221b9fba4645383b/doc/source/cluster/config.rst

The helm chart template

ray/deploy/charts/ray/templates/raycluster.yaml

Lines 22 to 23 in 55680a1

	minWorkers: {{ $val.minWorkers }}
	maxWorkers: {{ $val.maxWorkers }}

needs to be updated to ignore min_workers and max_workers if they're not specified in values.yaml

ray/deploy/charts/ray/values.yaml

Lines 16 to 17 in 55680a1

	minWorkers: 0
	maxWorkers: 0

.
There's another example that needs to be modified here

ray/deploy/components/example_cluster.yaml

Lines 33 to 35 in 55680a1

	minWorkers: 0
	# Maximum number of Ray workers of this Pod type. Takes precedence over minWorkers.
	maxWorkers: 0

thanks, I missed those

[DmitriGekhtman]

This spot should mention the default behavior for the head node type max workers
https://github.com/ray-project/ray/blob/55680a1f9ed3d8d1d4762039221b9fba4645383b/doc/source/cluster/config.rst#available_node_typesnode_type_namenode_typemax_workers

[sasha-s]

This spot should mention the default behavior for the head node type max workers
https://github.com/ray-project/ray/blob/55680a1f9ed3d8d1d4762039221b9fba4645383b/doc/source/cluster/config.rst#available_node_typesnode_type_namenode_typemax_workers

done

[DmitriGekhtman]

lgtm -- I'm just going to run some quick manual checks that k8s stuff works

[DmitriGekhtman]

The somewhat unorthodox convention we use for Ray github is to add reviewers as assignees (makes searching PRs to be reviewed easier for reviewers)

[DmitriGekhtman]

LGTM -- would you mind doing a manual check by launching a cluster on AWS with these changes checked out locally?

[DmitriGekhtman]

Looks like there's a Mac build failure. Could you rebase or merge master to retry that?

[DmitriGekhtman]

eh, sry, realized there's one more issue with the helm chart.

As is, there's an incompatibility with the default rayproject/ray:latest operator image, since the max/min-workers-filling logic runs in the operator container.

To avoid the incompatibility, could you modify this part to default to 0 if minWorkers/maxWorkers is not present? There's Helm syntax for defaults which simplifies this.

[sasha-s]

eh, sry, realized there's one more issue with the helm chart.

As is, there's an incompatibility with the default rayproject/ray:latest operator image, since the max/min-workers-filling logic runs in the operator container.

To avoid the incompatibility, could you modify this part to default to 0 if minWorkers/maxWorkers is not present? There's Helm syntax for defaults which simplifies this.

eh, sry, realized there's one more issue with the helm chart.

As is, there's an incompatibility with the default rayproject/ray:latest operator image, since the max/min-workers-filling logic runs in the operator container.

To avoid the incompatibility, could you modify this part to default to 0 if minWorkers/maxWorkers is not present? There's Helm syntax for defaults which simplifies this.
I guess it is not feasible to remove the defaults from rayproject/ray:latest.
Changes min/max to default 0.

[DmitriGekhtman]

looks good now!
Windows failures are unrelated.