-
Notifications
You must be signed in to change notification settings - Fork 6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[autoscaler] GCP node provider #2061
Merged
richardliaw
merged 73 commits into
ray-project:master
from
hartikainen:feature/gcp-node-provider
May 31, 2018
Merged
Changes from 1 commit
Commits
Show all changes
73 commits
Select commit
Hold shift + click to select a range
a896450
Google Cloud Platform scaffolding
hartikainen a3b44df
Add minimal gcp config example
hartikainen 281c7b6
Add googleapiclient discoveries, update gcp.config constants
hartikainen 8cd3287
Rename and update gcp.config key pair name function
hartikainen 8615abb
Implement gcp.config._configure_project
hartikainen f0539d3
Fix the create project get project flow
hartikainen ba8cdbf
Implement gcp.config._configure_iam_role
hartikainen 012c5d8
Implement service account iam binding
hartikainen f88449b
Implement gcp.config._configure_key_pair
hartikainen 08f53a4
Implement rsa key pair generation
hartikainen d17e244
Implement gcp.config._configure_subnet
hartikainen 3e67a60
Save work-in-progress gcp.config._configure_firewall_rules.
hartikainen d05df31
Remove unnecessary firewall configuration
hartikainen b499ea0
Update example-minimal.yaml configuration
hartikainen 5deb6ba
Add new wait_for_compute_operation, rename old wait_for_operation
hartikainen 352a4ff
Temporarily rename autoscaler tags due to gcp incompatibility
hartikainen 16a8605
Implement initial gcp.node_provider.nodes
hartikainen 944468f
Implement initial gcp.node_provider.create_node
hartikainen 9a5c8a3
Implement initial gcp.node_provider._node and node status functions
hartikainen 6c78b40
Implement initial gcp.node_provider.terminate_node
hartikainen bd09fbc
Implement node tagging and ip getter methods for nodes
hartikainen 52def7b
Temporarily rename tags due to gcp incompatibility
hartikainen ad301bb
Tiny tweaks for autoscaler.updater
hartikainen 9a1d052
Remove unused config from gcp node_provider
hartikainen 423f791
Add new example-full example to gcp, update load_gcp_example_config
hartikainen 5d90308
Implement label filtering for gcp.node_provider.nodes
hartikainen f5634d3
Revert unnecessary change in ssh command
hartikainen 7e1ea09
Revert "Temporarily rename tags due to gcp incompatibility"
hartikainen d71d4e4
Revert "Temporarily rename autoscaler tags due to gcp incompatibility"
hartikainen 9cf4840
Refactor autoscaler tagging to support multiple tag specs
hartikainen 7037922
Remove missing cryptography imports
hartikainen d9bea64
Update quote function import
hartikainen bd07ff1
Fix threading issue in gcp.config with the compute discovery object
hartikainen e6559fd
Add gcs support for log_sync
hartikainen de2980b
Fix the labels/tags naming discrepancy
hartikainen a221c0d
Add expanduser to file_mounts hashing
hartikainen 9c3899c
Fix gcp.node_provider.internal_ip
hartikainen c3341b0
Add uuid to node name
hartikainen 5fa034c
Remove 'set -i' from updater ssh command
hartikainen c152faa
Update ssh key creation in autoscaler.gcp.config
hartikainen 5948f0f
Fix wait_for_compute_zone_operation's threading issue
hartikainen f40d7ef
Address pr feedback from @ericl
hartikainen 44a3458
Expand local file mount paths in NodeUpdater
hartikainen 646dc81
Add ssh_user name to key names
hartikainen 23a0066
Update updater ssh to attempt 'set -i' and fall back if that fails
hartikainen e50dd00
Update gcp/example-full.yaml
hartikainen 85c1e4b
Fix wait crm operation in gcp.config
hartikainen 5517743
Update gcp/example-minimal.yaml to match aws/example-minimal.yaml
hartikainen 35d946e
Fix gcp/example-full.yaml comment indentation
hartikainen dd8fc5f
Add gcp/example-full.yaml to setup files
hartikainen 693de75
Update example-full.yaml command
hartikainen 183453e
Revert "Refactor autoscaler tagging to support multiple tag specs"
hartikainen 46250d3
Update tag spec to only use characters [0-9a-z_-]
hartikainen 7a84bbd
Change the tag values to conform gcp spec
hartikainen 3e7f91e
Add project_id in the ssh key name
hartikainen b81ab5a
Replace '_' with '-' in autoscaler tag names
hartikainen 9f87340
Revert "Update updater ssh to attempt 'set -i' and fall back if that …
hartikainen a791241
Revert "Remove 'set -i' from updater ssh command"
hartikainen 1489971
Add fallback to `set -i` in force_interactive command
hartikainen ae5a586
Update autoscaler tests to match current implementation
hartikainen 8e37ab4
Update GCPNodeProvider.create_node to include hash in instance name
hartikainen 79c0c19
Add support for creating multiple instance on one create_node call
hartikainen 34a403a
Clean TODOs
hartikainen ccbe2aa
Update styles
hartikainen 5d78ef5
Remove unnecessary comment. Fix indentation.
hartikainen d8f66b3
Merge branch 'master' into feature/gcp-node-provider
hartikainen 6856117
Yapfify files that fail flake8 test
hartikainen 41e90ed
Yapfify more files
hartikainen a59f81c
Update project_id handling in gcp node provider
hartikainen feeb3a8
Merge branch 'master' into hartikainen-feature/gcp-node-provider
richardliaw b6744e4
temporary yapf mod
richardliaw dd6b5ab
Revert "temporary yapf mod"
hartikainen 940c1b1
Fix autoscaler/updater.py lint error, remove unused variable
hartikainen File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Implement initial gcp.node_provider.nodes
* Still missing filter support
- Loading branch information
commit 16a860516a61f64c4fad8627a19b2c1bc734e161
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume this is guaranteed to be unique per node?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep. In fact, I had to change the create_node arguments to pass unique name to the nodes because gcp requires the names to be unique.