Open
Description
Version
23.01
Which installation method(s) does this occur on?
No response
Describe the bug.
Configuring the TritonInferenceStage to use the default gRPC port will trigger this warning when using the C++ implementation of the stage:
W20221216 12:40:36.421149 996827 triton_inference.cpp:329] Failed to connect to Triton at 'localhost:8001'. Default gRPC port of (8001) was detected but C++ InferenceClientStage uses HTTP protocol. Retrying with default HTTP port (8000)
At this point if the stage is able to reconnect to the HTTP port, the pipeline continues as normal.
The problem is that the Python impl of the stage only uses gRPC, while the C++ impl only uses HTTP. If you have a pipeline that can run in either Python or C++ mode, there is no way to avoid this warning and still have a pipeline that can run in both modes.
This also only works when Triton is running on the default ports.
We could do one of:
- Remove the warning, and only warn if the second connection attempt fails
- Explicitly configure the stage with both the gRPC & HTTP ports, allowing the stage to chose it's own protocol, and remove the fall-back behavior.
- Support gRPC in our C++ impl
Minimum reproducible example
Run the `examples/developer_guide/2_1_real_world_phishing` developer guide example as documented.
morpheus --log_level=debug --plugin examples/developer_guide/2_1_real_world_phishing/recipient_features_stage.py run pipeline-nlp --labels_file=data/labels_phishing.txt --model_seq_length=128 from-file --filename=examples/data/email_with_addresses.jsonlines recipient-features deserialize preprocess --vocab_hash_file=data/bert-base-uncased-hash.txt --truncation=true --do_lower_case=true --add_special_tokens=false inf-triton --model_name=phishing-bert-onnx --server_url=localhost:8001 --force_convert_inputs=true monitor --description="Inference Rate" --smoothing=0.001 --unit=inf filter --threshold=0.9 serialize to-file --filename=/tmp/detections.jsonlines --overwriteRelevant log output
W20221216 12:40:36.421149 996827 triton_inference.cpp:329] Failed to connect to Triton at 'localhost:8001'. Default gRPC port of (8001) was detected but C++ InferenceClientStage uses HTTP protocol. Retrying with default HTTP port (8000)
### Full env printout
_No response_
### Other/Misc.
_No response_
### Code of Conduct
- [X] I agree to follow Morpheus' Code of Conduct
- [X] I have searched the [open bugs](https://github.com/nv-morpheus/Morpheus/issues?q=is%3Aopen+is%3Aissue+label%3Abug) and have found no duplicates for this bug report
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Metadata
Assignees
Labels
Type
Projects
Status
Todo