stub.withWaitForReady()'s DEADLINE_EXECEEDED lacks connection failure information

[mohit-0109]

try {
  stub.withWaitForReady().withDeadline().hello();
} catch(StatusRuntimeException e) {
  // always receiving DEADLINE_EXCEEDED as status code if server is unavailable or client certs are invalid
}

There is no way to distinguish the underlying cause of failure , say if the failure is due to SSLHanshake or server genuinely not being available.

As an alternative, without using withWaitForReady() we are receiving UNAVAILABLE code in StatusRuntimeException which wraps the underlying cause say for example the SSLHandshake exception. Looking at the UNAVAILABLE code how can we safely take a call if we to reattempt connection, as we do not want to reattempt the connection in SSLException scenarios?

[ejona86]

Drive-by comment: What needs to happen here is we enhance DelayedStream/PendingStream to remember the last pick failure and then add it during appendTimeoutInsight() if there is no real stream. We've been aware of this shortcoming, but most people aren't using waitForReady(). Then you'd get better error information without doing anything in particular (as long as DEADLINE_EXCEEDED still triggers). #6569 might be related.

[ejona86]

I don't think we'd provide any guarantees that you can programatically distinguish between the server can't be contacted and invalid client certs. In particular, I believe the exact error you get for invalid client certs even without waitForReady can vary depending on lots of details.

But for this we can make it so a human could know which case was being hit.

[shivaspeaks]

PendingStream to remember the last pick failure

By this you mean to remember shutdownStatus of PickerState? To do so, we should define a data member in DelayedStream.java and update this inside the PendingStream as it has access to PickerState.

And then adding it during appendTimeoutInsight() if there is no real stream in DelayedStream!

[ejona86]

No, not shutdownStatus. That is only used when the channel itself is shut down. The "last pick failure" would be in the value returned from pickSubchannel().