test: Add L0_backend_onnxruntime test for enabling bfloat16 dtype in ONNXRuntime backend#8660
Conversation
yinggeh
changed the title
yinggeh
changed the title
There was a problem hiding this comment.
Pull request overview
This pull request adds comprehensive test coverage for bfloat16 (BF16) data type support in the ONNXRuntime backend. The tests verify that the backend can properly handle BF16 tensors for inference operations through both HTTP and gRPC protocols.
Changes:
- Added a new test script that validates bfloat16 support in the ONNXRuntime backend
- Created a test model configuration and ONNX model for bfloat16 addition operations
- Implemented comprehensive Python test cases covering multiple edge cases for BF16 arithmetic
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| qa/L0_backend_onnxruntime/test.sh | Main test script that starts the Triton server and executes bfloat16 tests for both HTTP and gRPC protocols |
| qa/L0_backend_onnxruntime/bfloat16_test.py | Python test implementation with comprehensive BF16 test cases including zeros, negatives, large/small values, cancellation, and identical inputs |
| qa/L0_backend_onnxruntime/models/add_bf16/config.pbtxt | Model configuration defining BF16 input/output tensors for a simple addition operation |
| qa/L0_backend_onnxruntime/models/add_bf16/1/model.onnx | Binary ONNX model file implementing BF16 addition |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
… yinggeh/tgh-26-onnx-backend-does-not-support-bfloat16-inputs
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…nputs' of github.com:triton-inference-server/server into yinggeh/tgh-26-onnx-backend-does-not-support-bfloat16-inputs
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
yinggeh
deleted the
yinggeh/tgh-26-onnx-backend-does-not-support-bfloat16-inputs
branch
What does the PR do?
Add
L0_backend_onnxruntimetest for enabling bfloat16 dtype in ONNXRuntime backendChecklist
<commit_type>: <Title>Commit Type:
Check the conventional commit type
box here and add the label to the github PR.
Related PRs:
triton-inference-server/onnxruntime_backend#335
Where should the reviewer start?
Test plan:
Caveats:
Background
Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)