Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[autoscaler] Kubernetes autoscaler backend #5492

Merged
merged 48 commits into from
Oct 3, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
86d9c6f
Add Kubernetes NodeProvider to autoscaler
edoakes Aug 12, 2019
be1b96b
Split off SSHCommandRunner
edoakes Aug 19, 2019
ffe00a6
Add KubernetesCommandRunner
edoakes Aug 20, 2019
0915df9
Cleanup
edoakes Aug 22, 2019
4558a76
More config options
edoakes Aug 22, 2019
15d1856
Check if auth present
edoakes Aug 22, 2019
8ed1ed9
More auth checks
edoakes Aug 22, 2019
8ec5f23
Better output
edoakes Aug 22, 2019
3c454c2
Always bootstrap config
edoakes Aug 22, 2019
56e3bf0
All working
edoakes Aug 23, 2019
4379355
Add k8s-rsync comment
edoakes Aug 23, 2019
44327bb
Clean up manual k8s examples
edoakes Aug 23, 2019
2d6fcc3
Fix up submit.yaml
edoakes Aug 23, 2019
9c98da7
Automatically configure permissisons
edoakes Aug 24, 2019
a7265ad
Fix get_node_provider arg
edoakes Aug 24, 2019
3174aa1
Fix permissions
edoakes Aug 24, 2019
aa13c22
Fill in empty auth
edoakes Aug 26, 2019
158876f
Merge branch 'master' into k8s
edoakes Aug 26, 2019
6ff3c2a
Remove ray-cluster from this PR
edoakes Aug 26, 2019
c218965
No hard dep on kubernetes library
edoakes Aug 26, 2019
f904982
Move permissions into autoscaler config
edoakes Aug 26, 2019
2452e1b
lint
edoakes Aug 26, 2019
a4e16d6
Fix indentation
edoakes Aug 26, 2019
e13f4a1
namespace validation
edoakes Aug 28, 2019
0fc7a0f
Use cluster name tag
edoakes Aug 28, 2019
9dd310f
Remove kubernetes from setup.py
edoakes Aug 28, 2019
da0ad01
Comment in example configs
edoakes Aug 28, 2019
34b6592
Same default autoscaling config as aws
edoakes Aug 28, 2019
8a0bb24
Add Kubernetes quickstart
edoakes Aug 28, 2019
6df676c
lint
edoakes Aug 28, 2019
15d3573
Revert changes to submit.yaml (other PR)
edoakes Aug 28, 2019
c610f34
Install kubernetes in travis
edoakes Aug 29, 2019
0cc97d6
address comments
edoakes Aug 29, 2019
e1c6fa2
Improve autoscaling doc
edoakes Aug 29, 2019
3bb44d6
Merge remote-tracking branch 'upstream/master' into k8s
edoakes Aug 29, 2019
2e449ea
kubectl command in setup
edoakes Sep 3, 2019
8fa6126
Force use_internal_ips
edoakes Sep 3, 2019
6559635
Merge remote-tracking branch 'upstream/master' into k8s
edoakes Sep 3, 2019
3fdb850
comments
edoakes Sep 3, 2019
c01d471
backend env in docs
edoakes Sep 4, 2019
670d819
Merge remote-tracking branch 'upstream/master' into k8s
edoakes Sep 4, 2019
86d5cc4
Change namespace config
edoakes Sep 4, 2019
1b80017
comments
edoakes Sep 18, 2019
16adb26
Merge remote-tracking branch 'upstream/master' into k8s
edoakes Sep 18, 2019
e88ec5f
comments
edoakes Sep 18, 2019
fee41e4
Merge branch 'master' into k8s
edoakes Sep 20, 2019
734320a
Merge branch 'master' into k8s
edoakes Oct 2, 2019
b29d172
Fix yaml test
edoakes Oct 3, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
comments
  • Loading branch information
edoakes committed Sep 18, 2019
commit 1b800179c2b7c76176cab060d2c53c9996b37d25
2 changes: 1 addition & 1 deletion doc/source/autoscaling.rst
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ Test that it works by running the following commands from your local machine:
$ kubectl -n ray get pods

# Get a remote screen on the head node.
$ ray attach ray/python/ray/autoscaler/gcp/example-full.yaml
$ ray attach ray/python/ray/autoscaler/kubernetes/example-full.yaml
$ # Try running a Ray program with 'ray.init(address="localhost:6379")'.

# Tear down the cluster
Expand Down
2 changes: 1 addition & 1 deletion python/ray/autoscaler/autoscaler.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@
"head_ip": (str, OPTIONAL), # local cluster head node
"worker_ips": (list, OPTIONAL), # local cluster worker nodes
"use_internal_ips": (bool, OPTIONAL), # don't require public ips
"namespace": (dict, OPTIONAL), # k8s namespace, if using k8s
"namespace": (str, OPTIONAL), # k8s namespace, if using k8s

# k8s autoscaler permissions, if using k8s
"autoscaler_service_account": (dict, OPTIONAL),
Expand Down
3 changes: 2 additions & 1 deletion python/ray/autoscaler/kubernetes/node_provider.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ class KubernetesNodeProvider(NodeProvider):
def __init__(self, provider_config, cluster_name):
NodeProvider.__init__(self, provider_config, cluster_name)
self.cluster_name = cluster_name
self.namespace = provider_config["namespace"]["metadata"]["name"]
self.namespace = provider_config["namespace"]

def non_terminated_nodes(self, tag_filters):
# Match pods that are in the 'Pending' or 'Running' phase.
Expand All @@ -34,6 +34,7 @@ def non_terminated_nodes(self, tag_filters):
"status.phase!=Failed",
"status.phase!=Unknown",
"status.phase!=Succeeded",
"status.phase!=Terminating",
])

tag_filters[TAG_RAY_CLUSTER_NAME] = self.cluster_name
Expand Down