Error: socket hang up

[trusktr]

I'm getting errors that look like this happening every hour or two, causing systemd to restart my nodeProxyServer.js script:

Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: throw arguments[1]; // Unhandled 'error' event
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: ^
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: Error: socket hang up
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at createHangUpError (http.js:1360:15)
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at ServerResponse.OutgoingMessage._writeRaw (http.js:507:26)
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at ServerResponse.OutgoingMessage._send (http.js:476:15)
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at ServerResponse.OutgoingMessage.write (http.js:740:18)
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at ClientRequest.proxyError (/srv/nodeProxyServer/node_modules/http-proxy/lib/node-http-proxy/http-proxy.js:211:13)
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at ClientRequest.g (events.js:192:14)
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at ClientRequest.EventEmitter.emit (events.js:96:17)
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at Socket.socketCloseListener (http.js:1411:9)
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at Socket.EventEmitter.emit (events.js:126:20)
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at Socket._destroy.destroyed (net.js:358:10)
Nov 10 16:49:17 fountainsunique nodeProxyServer[27651]: Proxy server started on port 80.
Nov 10 16:51:17 fountainsunique nodeProxyServer[27651]: res.end error: socket hang up

May it be related to node-http-proxy, or something else?

[yawnt]

can you paste me an issue that demonstrates the problem? also are you using caronte or the master branch?

[jondubois]

I also encounter the issue.

I'm using sample code from the README:

var http = require('http'),
    httpProxy = require('http-proxy');
//
// Create your proxy server and set the target in the options.
//
httpProxy.createProxyServer({target:'http://localhost:9000'}).listen(8000);

//
// Create your target server
//
http.createServer(function (req, res) {
  res.writeHead(200, { 'Content-Type': 'text/plain' });
  res.write('request successfully proxied!' + '\n' + JSON.stringify(req.headers, true, 2));
  res.end();
}).listen(9000);

One way to reproduce is to refresh the page many times really quickly (hold down refresh keys). The 'socket hang up' error kills the process if not handled correctly.

EDIT The issue appears to be caused by this line: https://github.com/nodejitsu/node-http-proxy/blob/master/lib/http-proxy/passes/web-incoming.js#L127 - It looks like Node.js always throws an error when you call req.abort().
I think ideally Node.js shouldn't raise an error there because the abort() in this case is fully intentional - Errors should only be raised when an unexpected condition occurs.

[jcrugzz]

@jondubois You should listen on the error event for the proxy so that you can handle the errors and return proper responses when hitting this case. It is most likely triggered because there are not anymore sockets to be used and there is no socket pooling by default.

[seglo]

I noticed that when I revert back to the 0.10.x version I don't encounter this problem. It looks like 1.x.x was a fairly significant rewrite, so it's tough to say what may have changed that could be causing this issue. Perhaps someone more familiar with codebase could comment?

[jcrugzz]

@seglo in 0.10.x there was a proxyError event that was emit when something happened with the conenction so there could be a response sent to the client. This is now an error event. For example usage, see here. The error event should not go unhandled, its the default behavior of the EventEmitter to throw when it has no listeners.

[seglo]

@jcrugzz I see, OK I'll start handling that. Are you saying that with 0.10.x a proxyError event would be raised, but is internally handled?

[jcrugzz]

@seglo in 0.10.x a proxyError event would be raised but it wouldn't crash the server if it wasn't handled. You just wouldn't properly respond to clients who disconnect if you didn't handle the event

[jondubois]

I feel that this error isn't very useful at all for the following reasons:

• It happens predictably every time a connection is intentionally aborted by the client (can be quite often)
• There is no way to 'properly respond to clients' since the connection is now closed so when this error happens, there is no way to inform the client anyways (all we can do is log it).

Also using an agent for socket pooling doesn't fix the issue.

In my particular project (which makes use of http-proxy internally), for a while, I was just logging the error (not doing anything else) but it seemed to cause confusion among some developers who thought that it was an actual error that they needed to handle.

Unlike other Node.js errors, this error appears to be completely harmless - It doesn't cause any side effects if you ignore it (I've tested this for a while) - No memory leaks. I feel that exceptions are not exceptions if they happen predictably during a common use case of the system (in this case, aborting the connection). I think it's more of a Node.js issue than an http-proxy issue - I was able to reproduce this issue in Node by creating a client request and then aborting the connection (it would throw an error whenever I called req.abort()).

[seglo]

@jcrugzz Thanks for the example. I've made the appropriate updates to my project.

[jcrugzz]

@jondubois my explanation was how the error event in general should be handled. In node the semantics in various libraries are never consistent. Socket hangups don't only happen with that particular edge case you ran into. Do you have a gist of that consistent reproducible case? Im curious how its hitting the code paths in node core.