Broken pipe Exception from Jersey layer while closing response in ServerRuntime$Responder.writeResponse() not handled or re-thrown #5783
Description
Exception from Jersey layer while closing response is not handled and implications can very depending on the implementation of ResponseWriter.
For Eg. Helidon is using a custom implementation of the container response writer(JAXRSResponseWriter) which is using a CountdownLatch that is initialized with the value of 1 and decremented to 0 on commit and failure methods. If these methods do not get called then await will hang.
we saw this behavior of await hanging when Broken pipe Exception occurred while closing the response in Jersey code. More details available in helidon-io/helidon#9442
In Jersey, the below code segment in ServerRuntime.writeResponse() method does not handle or re-throw the exception while closing the response. It simply logs the messages and ignores the exception.
ServerRuntime.writeResponse()
if (close) {
try {
response.close();
} catch (final Exception e) {
LOGGER.log(Level.SEVERE, LocalizationMessages.ERROR_CLOSING_COMMIT_OUTPUT_STREAM(), e);
}
}
For successful cases, the implementation of the close calls the “getResponseWriter().commit()” which would have decrement the countDownLatch to 0 and make sure that await() would not block.
ContainerResponse.close()
public void close() {
if (!closed) {
closed = true;
messageContext.close();
requestContext.getResponseWriter().commit();
requestContext.setWorkers(null);
}
}
However, for the error cases, failure() method never gets called and causes the await() to block in the case of Helidon use case. The broken pipe exception is thrown from "messageContext.close()".
stack below:
java.io.UncheckedIOException: java.net.SocketException: Broken pipe
at io.helidon.common.buffers.GrowingBufferData.writeTo(GrowingBufferData.java:69)
at io.helidon.common.socket.PlainSocket.write(PlainSocket.java:136)
at io.helidon.common.socket.SocketWriter.writeNow(SocketWriter.java:81)
at io.helidon.common.socket.SocketWriterDirect.write(SocketWriterDirect.java:43)
at io.helidon.webserver.http1.Http1ServerResponse$BlockingOutputStream.write(Http1ServerResponse.java:631)
at io.helidon.webserver.http1.Http1ServerResponse$BlockingOutputStream.write(Http1ServerResponse.java:505)
at java.base/java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:125)
at java.base/java.io.BufferedOutputStream.implFlush(BufferedOutputStream.java:252)
at java.base/java.io.BufferedOutputStream.flush(BufferedOutputStream.java:240)
at java.base/java.io.FilterOutputStream.close(FilterOutputStream.java:184)
at io.helidon.webserver.http1.Http1ServerResponse$ClosingBufferedOutputStream.close(Http1ServerResponse.java:801)
at org.glassfish.jersey.message.internal.CommittingOutputStream.close(CommittingOutputStream.java:251)
at org.glassfish.jersey.message.internal.OutboundMessageContext.close(OutboundMessageContext.java:568)
at org.glassfish.jersey.server.ContainerResponse.close(ContainerResponse.java:403)
at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:763)
at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:398)
at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:388)
at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277)
-----
-----
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:266)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:253)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:696)
at io.helidon.microprofile.server.JaxRsService.doHandle(JaxRsService.java:234)
at io.helidon.microprofile.server.JaxRsService.lambda$handle$2(JaxRsService.java:185)
at io.helidon.common.context.Contexts.runInContext(Contexts.java:117)
at io.helidon.microprofile.server.JaxRsService.handle(JaxRsService.java:185)
at io.helidon.webserver.http.HttpRoutingImpl$RoutingExecutor.doRoute(HttpRoutingImpl.java:169)
we tried out a prototype fix where we modified the below code in Jersey to invoke “request.getResponseWriter().failure(e) “ in the exception block as below.
ServerRuntime.writeResponse() with exception handling
if (close) {
try {
response.close();
} catch (final Exception e) {
LOGGER.log(Level.WARNING, LocalizationMessages.ERROR_CLOSING_COMMIT_OUTPUT_STREAM(), e);
try {
request.getResponseWriter().failure(e);
} catch (Exception ex) {
LOGGER.log(Level.WARNING, LocalizationMessages.ERROR_CLOSING_COMMIT_OUTPUT_STREAM(), e);
}
}
}
with this change the call to failure decrements the latch and does not cause await to hang in the Helidon implementation of the container response writer.
Would like to check the feasibility of this exception handling change implemented in Jersey.