Skip to content

Conversation

axreldable
Copy link
Contributor

@axreldable axreldable commented Oct 10, 2025

Why are these changes needed?

  1. Fix bug with 'proxy_location' set for 'serve run' CLI command

serve run CLI command ignores proxy_location from config and uses default value EveryNode.

Steps to reproduce:

  • have a script:
# hello_world.py
from ray.serve import deployment

@deployment
async def hello_world():
    return "Hello, world!"

hello_world_app = hello_world.bind()

Execute:

ray stop
ray start --head
serve build -o config.yaml hello_world:hello_world_app
  • change proxy_location in the config.yaml: EveryNode -> Disabled
serve run config.yaml
curl -s -X GET "http://localhost:8265/api/serve/applications/" | jq -r '.proxy_location'

Output:

Before change:
EveryNode - but Disabled expected
After change:
Disabled
  1. Fix discrepancy for 'proxy_location' in the Python API 'start' method

serve.start function in Python API sets different http_options.location depending on if http_options is provided.

Steps to reproduce:

  • have a script:
# discrepancy.py
import time

from ray import serve
from ray.serve.context import _get_global_client

if __name__ == '__main__':
    serve.start()
    client = _get_global_client()
    print(f"Empty http_options: `{client.http_config.location}`")

    serve.shutdown()
    time.sleep(5)

    serve.start(http_options={"host": "0.0.0.0"})
    client = _get_global_client()
    print(f"Non empty http_options: `{client.http_config.location}`")

Execute:

ray stop
ray start --head
python -m discrepancy

Output:

Before change:
Empty http_options: `EveryNode`
Non empty http_options: `HeadOnly`
After change:
Empty http_options: `EveryNode`
Non empty http_options: `EveryNode`

It changes current behavior in the following ways:

  1. serve run CLI command respects proxy_location parameter from config instead of using the hardcoded EveryNode.
  2. serve.start function in Python API stops using the default HeadOnly in case of empty proxy_location and provided http_options dictionary without location specified.

Related issue number

Aims to simplify changes in the PR: #56507

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run pre-commit jobs to lint the changes in this PR. (pre-commit setup)
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

Signed-off-by: axreldable <aleksei.starikov.ax@gmail.com>
Signed-off-by: axreldable <aleksei.starikov.ax@gmail.com>
…' method

Signed-off-by: axreldable <aleksei.starikov.ax@gmail.com>
@axreldable axreldable force-pushed the 56163_http_options_5 branch 2 times, most recently from 3de646d to 3cdf062 Compare October 10, 2025 13:13
Signed-off-by: axreldable <aleksei.starikov.ax@gmail.com>
@axreldable axreldable force-pushed the 56163_http_options_5 branch from 3cdf062 to b4c6107 Compare October 10, 2025 16:02
_system_config={"metrics_report_interval_ms": 1000, "task_retry_delay_ms": 50},
)
serve.start(
proxy_location=ProxyLocation.HeadOnly,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the current value which is used implicitly.

@axreldable axreldable changed the title 56163 http options 5 [serve] Fix bug with 'proxy_location' set for 'serve run' CLI command + discrepancy fix in Python API 'serve.start' function Oct 10, 2025
@axreldable axreldable marked this pull request as ready for review October 10, 2025 19:28
@axreldable axreldable requested a review from a team as a code owner October 10, 2025 19:28
elif isinstance(http_options, dict):
http_options = HTTPOptions(**http_options)
http_options.location = ProxyLocation._to_deployment_mode(proxy_location)
return http_options
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Proxy Location Conversion Error

The _prepare_http_options function is missing the string-to-ProxyLocation enum conversion for proxy_location. This logic was present in the original start() function but removed, causing runtime errors when proxy_location is passed as a string to ProxyLocation._to_deployment_mode.

Fix in Cursor Fix in Web

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This transformation was redundant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-contribution Contributed by the community serve Ray Serve Related Issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant