-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Serve] Java Serve improvement #42
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks like a great step forward. A few high-level comments:
- As I indicated below, we are moving away from the "InputNode" and "DAGDriver" concepts (they're unnecessarily complicated). For Java, let's prioritize matching the
.bind()
API for multiple deployments and using the handles that are substituted in the constructor. - The recommended way for users to deploy in production these days is via config file (e.g., embedded in a KubeRay CR). Can we include in the proposal adding support to deploy Java this way as well?
- [P1] As part of this plan, it would be great to have a path forward to unify the handle implementations (e.g., shared C++ core).
@liuyang-my looks great now, just need to add the config section! |
Signed-off-by: chuhan.ly <chuhan.ly@antgroup.com>
Signed-off-by: chuhan.ly <chuhan.ly@antgroup.com>
Signed-off-by: chuhan.ly <chuhan.ly@antgroup.com>
Signed-off-by: chuhan.ly <chuhan.ly@antgroup.com>
…mentResponse demo. 3. Adjusting the way Python deploys deployment for other languages. Signed-off-by: chuhan.ly <chuhan.ly@antgroup.com>
Signed-off-by: chuhan.ly <chuhan.ly@antgroup.com>
eb2a07a
to
c22d959
Compare
Signed-off-by: chuhan.ly <chuhan.ly@antgroup.com>
grpc_servicer_functions: [] | ||
|
||
applications: | ||
- name: app |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that this can take args:
parameter as well (will be passed in the same way that it is via command line)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great, the example of adding args in the config file has been supplemented.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great! Can't wait to see this implemented.
Signed-off-by: chuhan.ly <chuhan.ly@antgroup.com>
This is about Serve Java Improvement. The goal of this PR is to make the Java API consistent with Python. The design of the user API is discussed in this proposal: ray-project/enhancements#42. This PR only covers the Java programming API part, including Deployment.bind, Serve.run, DeploymentHandle, and so on, ensuring that these APIs can be used properly. The alignment of some internal core logic, documentation enhancements, and support for the config file will be submitted in subsequent PRs. Signed-off-by: chuhan.ly <chuhan.ly@antgroup.com> Co-authored-by: chuhan.ly <chuhan.ly@antgroup.com>
No description provided.