Populate handler exception message

[hgong-snap]

🚀 The feature

Populate exception message

Motivation, pitch

Currently, if the handler raise some random error, the client will receives only InternalServerException without error/exception details

<_InactiveRpcError of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Prediction failed
org.pytorch.serve.http.InternalServerException"
	debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B::1%5D:17100 {created_time:"2022-09-26T15:40:29.722145-07:00", grpc_status:14, grpc_message:"Prediction failed\norg.pytorch.serve.http.InternalServerException"}"
>

what I do now is basically add a wrapper everywhere:

from ts.utils.util import PredictionException

# in handler
try:
  self.do_something_and_raise_exception()  # this will raise some random Exception
except Exception as e:
  raise PredictionException(str(e), 503)

This way exception message from original exception will be populated in GRPC:

<_InactiveRpcError of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see https://cloud.google.com/docs/authentication/getting-started
org.pytorch.serve.http.InternalServerException"

Is this expected? is it possible to have this kinda wrapper everywhere?

Alternatives

No response

Additional context

No response

[msaroufim]

Yeah for now the suggestion would be to manually add try blocks around parts of your code that could potentially raise an exception. I think the ask is valid though that whatever exception is thrown by a handler it should get caught by the frontend with an error code and message by default but just need to think about how to do this.

[lxning]

@hgong-snap Here is the map b/w grpc code and http code defined in Torchserve. I expect the error message from handler is still available for client according to the code.

[msaroufim]

@hgong-snap did the above answer your question?

[hgong-snap]

thanks for the code pointer. but that seems not the case... seems error detail can be dropped... with that said, error passed to .withDescription(error) seems already a truncated error. (example is in above description)

[msaroufim]

Is the error truncated only on the client end? I'm assuming you see the error correctly populated in logs logs/model_log.log?