[Serve] Revisited Serve agent to avoid rejecting Serve configs in case HTTP options diverge #38739
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
GeneDer
approved these changes
Aug 22, 2023
…ptions diverge Signed-off-by: Alexey Kudinkin <ak@anyscale.com>
…ema`; Fixed `get_serve_instance_details` to return all HTTP options Signed-off-by: Alexey Kudinkin <ak@anyscale.com>
…deployed Signed-off-by: Alexey Kudinkin <ak@anyscale.com>
alexeykudinkin
force-pushed
the
ak/rsrv-http-opt-upd-fix
branch
from
26c16fe to
c14448a
Compare
alexeykudinkin
changed the title
zcin
approved these changes
Aug 23, 2023
arvind-chandra
pushed a commit
to lmco/ray
that referenced
this pull request
Aug 31, 2023
…e HTTP options diverge (ray-project#38739) This change revisits Serve agent handling of updated HTTP options when Serve application is already running: - Previously: agent would reject PUT /api/serve/applications/ request, in case provided HTTP options are divergent from the ones Serve application was started with - Now: agent would log as warning all options that diverged, and then proceed to ignore them, however still accepting the rest of the config This change is necessary to avoid complexity of external handling of HTTP option changes: we configure Serve application programmatically, and that includes some of its HTTP options that might change dynamically (request, keep-alive timeouts, etc). We want to make sure that as these programmatic changes make their way into the updated Serve configs, they don't block other Serve application changes (like deploying application changes itself) and are just silently ignored. Signed-off-by: e428265 <arvind.chandramouli@lmco.com>
vymao
pushed a commit
to vymao/ray
that referenced
this pull request
Oct 11, 2023
…e HTTP options diverge (ray-project#38739) This change revisits Serve agent handling of updated HTTP options when Serve application is already running: - Previously: agent would reject PUT /api/serve/applications/ request, in case provided HTTP options are divergent from the ones Serve application was started with - Now: agent would log as warning all options that diverged, and then proceed to ignore them, however still accepting the rest of the config This change is necessary to avoid complexity of external handling of HTTP option changes: we configure Serve application programmatically, and that includes some of its HTTP options that might change dynamically (request, keep-alive timeouts, etc). We want to make sure that as these programmatic changes make their way into the updated Serve configs, they don't block other Serve application changes (like deploying application changes itself) and are just silently ignored. Signed-off-by: Victor <vctr.y.m@example.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
This change revisits Serve agent handling of updated HTTP options when Serve application is already running:
This change is necessary to avoid complexity of external handling of HTTP option changes: we configure Serve application programmatically, and that includes some of its HTTP options that might change dynamically (request, keep-alive timeouts, etc).
We want to make sure that as these programmatic changes make their way into the updated Serve configs, they don't block other Serve application changes (like deploying application changes itself) and are just silently ignored.
Related issue number
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/under thecorresponding
.rstfile.