SocketUtils considers a port available if a ConnectException is thrown during connect attempt

[JasonTypesCodes]

This might help out with #3670

Currently, SocketUtils considers a port available if an attempt to connect to it fails. This isn't always the case. For example, a timeout might indicate that something is using the port but did not respond to the connection in time. I'm sure there are other situations where an exception might mean that the port is not available.

I'm not convinced that what I've proposed is actually the best approach here, but I think it is more correct than what we have now. I believe that a port that isn't in use will always throw ajava.net.ConnectException with a message of "Connection refused".

I was thinking that it may be more conclusive to try to listen on the port, or maybe listen and send a short message through to determine if it is actually available, but I suspect that will be a lot slower than what we have and I'm also concerned that the port may not be cleaned up in time for the caller to use it for something else.

[graemerocher]

Can you elaborate where you see this issue manifest?

[JasonTypesCodes]

I've not really seen it manifest. I was experimenting with an issue @jameskleeh opened regarding an intermittent test failure (#3670 ). I ran the test multiple times and never saw it fail, but I looked into reasons that it might think that a port is available when it actually isn't. I asked around and heard from folks that getting back a port that isn't actually available happens in a number of contexts not just in the test referenced here.

[JasonTypesCodes]

Looking here: https://docs.oracle.com/javase/8/docs/api/java/net/Socket.html#connect-java.net.SocketAddress-int-

It seems that there are reasons that a connect would throw an exception and the port is still not available. Specifically, SocketTimeoutException and IllegalBlockingModeException might actually mean that the port is not available

[ilopmar]

We've seen that in a few client projects when running the tests in CI environments like Google Cloud and AWS. Not so much locally although that happens from time to time.