Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix instance type to CPU for optional python model #4230

Merged
merged 1 commit into from
Apr 16, 2022
Merged

Conversation

CoderHam
Copy link
Contributor

  • Fixed jetson failure since GPU instances are not supported

Fixes L0_backend_python failure

- Fixed jetson failure since GPU instances are not supported
@CoderHam CoderHam merged commit fb923ee into main Apr 16, 2022
@GuanLuo
Copy link
Contributor

GuanLuo commented Apr 19, 2022

Why need to set CPU instance explicitly? @Tabrizian wouldn't Python backend create CPU instance if there is no supported GPU?

@CoderHam
Copy link
Contributor Author

The GPU support for the python backend refers to supporting the GPU tensors as I/O. This means that a model can run with CPU instance and still use GPU with python backend as long as the I/O for that model is via the CPU.

@Tabrizian
Copy link
Member

Using GPU/CPU instances doesn't change the behavior of Python backend at all. The instance settings will be passed to the model's initialize function and it is up to the user on how they want to handle this. The error is returned from the common backend utilities: https://github.com/triton-inference-server/core/blob/b338d65a97a9cfef9f68910f516556599f583057/src/backend_model_instance.cc#L220 Do we default to using the KIND_GPU if the user doesn't specify anything?

CoderHam pushed a commit that referenced this pull request Apr 26, 2022
- Fixed jetson failure since GPU instances are not supported
CoderHam pushed a commit that referenced this pull request Apr 26, 2022
- Fixed jetson failure since GPU instances are not supported
@CoderHam CoderHam deleted the hemantj-jp-python branch April 27, 2022 23:17
mc-nv pushed a commit that referenced this pull request Apr 30, 2022
- Fixed jetson failure since GPU instances are not supported
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants