agentkeepalive detecting ECONNRESET

[fengmk2]

from Aryeh Schechter email:

Hello,

We recently started using agentkeepalive to pool our connections to aws and have been experiencing ECONNRESETs. We are using version 2.0.2 which has the fix for issue #23, but we are still getting these errors.

Why are you not listening for 'error' on the socket and handling it similar to 'close'? This would allow for the ability to catch the ECONNRESETs and retry the request without the hassle of the requiring library to handle 'uncaughtException' and trying to figure out where it came from.

Thanks,
Aryeh Schechter
www.glide.me

[fengmk2]

But because you are not listening on 'error' the process will crash before 'close' will get emitted. I added a listener locally to lib/_http_agent.js on line 233 on 'error' and have caught it once and not crashed. Since it only happens sporadicly, I don't know how to reproduce it.

The stack trace isn't very helpful:

write ECONNRESET

   at exports._errnoException (util.js:746:11)

   at WriteWrap.afterWrite (net.js:775:14)

[fengmk2]

agentkeepalive don't handle socket error event.

[LarsEckart]

Hello,

I have the same use case, using agentkeepalive together with aws/dynamodb.
node10, about 6500 requests per minute per server and all 4 servers/applications ran into this error and didn't continue working (but also didnt crash completely)

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: read ECONNRESET
    at errnoException (net.js:905:11)
    at TCP.onread (net.js:559:19)

is what I got from logs.

[christhorwarth]

I was able to get around this by making the following changes:

In _http_agent.js around line 260, add this listener to the socket:

s.on('error', function(err) {
self.emit('error', err);
});

Then in agent.js, I catch the error and log it (around line 70):

self.on('error', function(err) {
log && log.error('Socket err', err);
});

[impomezia]

Workaround without change the source code:

var createSocket = Agent.prototype.createSocket;
Agent.prototype.createSocket = function(req, options) {
  var socket = createSocket.call(this, req, options);
  socket.on('error', function(err) {
    this.closeSocketCount++;
  }.bind(this));

  return socket;
};

ECONNRESET can easily reproduce: just shutdown a server after successful client request.
ECONNREFUSED just try connect to closed port.

[fengmk2]

I will add a default error listen to avoid Unhandled 'error' event throw.

[vvo]

Just to make it clear, you added a default error handler but errors will still bubble up at some point to the request object for example?

I am not sure to understand when does this unhandled error was triggered. In which cases?

[vvo]

cc @fengmk2

[fengmk2]

the request object is not handle by this module.
This agent only handle socket.

[vvo]

Ok sorry, so to be more precise, when was this unhandled error triggered?

What were the possible unhandled errors, only ECONNRESET? Or was it possible to also get ETIMEDOUT or ENOTFOUND?

I am asking because we got a support request on one of our libs using agentkeepalive and wondered if this was linked.

I talked about request error event because in the end this is where we catch error when using agentkeepalive (because you forward them right?)

[fengmk2]

The error event is emitted by the socket and agentkeepalive won't block it.

I think request object need to handle the error if the request object is an eventemitter.

[vvo]

I am trying to find a more detailed explanation of this issue, were you able to easily reproduce the error?

Reading 24afa4a I cannot see a test case that is reproducing this behavior.

To sump up: can you explain how from a user POV this error would happen.

Using agentkeepalive for example and trying to make a request to an unknown hostname, I did get the error event (in 2.0.2). So I am not sure to understand the issue.