Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Serve] Revisiting ProxyState to fix draining sequence #41722

Merged
merged 29 commits into from
Jan 22, 2024
Merged
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
39c1447
Merge `start_new_ready_check` into `is_ready`
alexeykudinkin Dec 7, 2023
0231ccf
Cleaned up `try_update_status`;
alexeykudinkin Dec 7, 2023
0ebfe1b
Streamlined `is_healthy`, `is_ready` checks in `ProxyActorWrapper`
alexeykudinkin Dec 7, 2023
b6b3660
Tidying up
alexeykudinkin Dec 7, 2023
feffe80
Simplified `is_healthy`, `is_ready` checks even more
alexeykudinkin Dec 7, 2023
2144898
Fixed `is_drained` check to properly return whether proxy actor is pr…
alexeykudinkin Dec 7, 2023
d04d010
Streamlined proxy's state reconciliation flow
alexeykudinkin Dec 7, 2023
ab1cc40
Simplified readiness, health, drain checks;
alexeykudinkin Dec 8, 2023
5ba6068
Make sure exceptions are properly handled in proxy state reconciliation
alexeykudinkin Dec 8, 2023
d9ef34b
Cleaned up docs
alexeykudinkin Dec 8, 2023
e7c5e1e
Added test for ActorProxyWrapper
alexeykudinkin Dec 8, 2023
759d904
Fixed typos;
alexeykudinkin Dec 8, 2023
953343a
Fixed tests
alexeykudinkin Dec 8, 2023
73d881e
Fixed more tests
alexeykudinkin Dec 8, 2023
9e05832
Fixed some more tests
alexeykudinkin Dec 8, 2023
8d0cdcd
`lint`
alexeykudinkin Dec 8, 2023
136df1c
Fixing typo
alexeykudinkin Dec 8, 2023
189cad4
After rebase clean-up
alexeykudinkin Jan 10, 2024
c8dcb57
Tidying up
alexeykudinkin Jan 10, 2024
06b1755
Make proxy health-check configurable
alexeykudinkin Jan 11, 2024
f4283df
Unify `is_drained` API to require timeout be explicitly provided
alexeykudinkin Jan 11, 2024
b787188
Missing log statement
alexeykudinkin Jan 11, 2024
adfd2f3
Tidying up
alexeykudinkin Jan 18, 2024
eff6f23
Cleaned up exception handling
alexeykudinkin Jan 18, 2024
959feba
`_try_cancel_future` -> `_try_set_exception`
alexeykudinkin Jan 18, 2024
c5d2034
Make timeout_s optional
alexeykudinkin Jan 18, 2024
7002e11
Fixing typo
alexeykudinkin Jan 18, 2024
cc99b4a
Fixing another typo
alexeykudinkin Jan 18, 2024
8a1e954
Fixing tests
alexeykudinkin Jan 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Cleaned up docs
Signed-off-by: Alexey Kudinkin <ak@anyscale.com>
  • Loading branch information
alexeykudinkin committed Jan 20, 2024
commit d9ef34b8cd6122868ee585c27c1787a8c132a6a9
27 changes: 9 additions & 18 deletions python/ray/serve/_private/proxy_state.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,44 +43,35 @@ def actor_id(self) -> str:
def is_ready(self, timeout_s: float) -> Optional[bool]:
"""Return the payload from proxy ready check when ready.

TODO update

Since actual readiness check is asynchronous, this method could return
either of the following statuses:

- FINISHED_SUCCEED: Readiness check finished successfully
- FINISHED_FAILED: Readiness check finished with failure (either timing out or failing)
- PENDING: Readiness check is still pending
- None: Readiness check is pending
- True: Readiness check completed successfully (proxy is ready)
- False: Readiness check completed with failure (either timing out or failing)
"""
raise NotImplementedError

@abstractmethod
def is_healthy(self, timeout_s: float) -> Optional[bool]:
"""Return whether the proxy actor is healthy

TODO update

Since actual health-check is asynchronous, this method could return
either of the following statuses:

- FINISHED_SUCCEED: Health-check finished successfully
- FINISHED_FAILED: Health-check finished with failure (either timing out or failing)
- PENDING: Health-check is still pending
- None: Health-check is pending
- True: Health-check completed successfully (proxy is healthy)
- False: Health-check completed with failure (either timing out or failing)
"""
raise NotImplementedError

@abstractmethod
def is_drained(self) -> Optional[bool]:
"""Return whether the proxy actor is drained

TODO update

Since actual check whether proxy is drained is asynchronous, this method could return
either of the following statuses:

- FINISHED_SUCCEED: Drain-check finished successfully
- FINISHED_FAILED: Drain-check finished with failure
- PENDING: Drain-check is still pending
- None: Drain-check is pending
- True: Drain-check completed, node *is drained*
- False: Drain-check completed, node is *NOT* drained
"""
raise NotImplementedError

Expand Down