if server is unavail connection hangs over other connection

[mitechie]

I'm working on error catching a postgres condition when the server is not available. The first time, everything works fine. The error is passed in and I can check for it and I'm set.

The second pass in hangs the at the connection line and I never get to the callback to get the error.

    console.log('about to connect');
    pg.connect(DBSTRING, function(err, client) {
        if (err) {
            console.log("ERROR DB " + err.message);
            stream.write(err.message);
        } else {
            // do query and such
        }
    });

Is the bit of code I'm working with. The about to connect is dumped on every request in, but the callback only seems to occur every other pass into here. Am I missing something in the error handling? Is there something I can check?

Thanks

[brianc]

Would you be able to create a gist or test case which I can execute to reproduce the issue?

[mitechie]

Sure thing, this is a gist with the two files cut down as much as I can get. Basically there's a http app that calls the postgres app. I've left in the debugging message I was using and the top of the file walks through where I'm getting the issue.

Let me know if I can help in any way.

https://gist.github.com/993332

Thanks

Rick

[brianc]

Thank Rick. I hate to be a pain but do you think you could reproduce the issue in a more isolated way? Like just a script that maybe calls pg.connect a few times with the failing connections? Also, what type of connection error is being returned when you do receive the error?

Remove any http server or socket things if possible. I'll work with what you've given to try to reproduce later this evening, but the more variables we can remove from the equation the easier it will be to identify the problem within node-postgres.

[mitechie]

OK, this is a smaller gist of just the pgconnect. When I test it, I only get the error message every other call to pgconnect:

https://gist.github.com/993332

Sample output:

running
interval set

calling
    in check
    connect
    ERROR DB ECONNREFUSED, Connection refused
calling
    in check
calling
    in check
    connect
    ERROR DB ECONNREFUSED, Connection refused
calling
    in check
calling
    in check
    connect
    ERROR DB ECONNREFUSED, Connection refused

[brianc]

Perfect! Will fix this after work today! <3

[brianc]

I figured out this problem & have written a failing test for the scenario, unfortunately it's a bit tricky. I've been on holiday and so haven't had much time to work on it...I'll push the failing test out tonight & go over what's going on.

[mitechie]

Thanks for the update. Fortunately it's not killer since the db is up and I'm just testing node.js for a use case right now.

[tmuellerleile]

Hi, is there any progress on this issue? As far as I can see the problem can be traced back to the following lines in Connection.connect

  if(this.stream.readyState === 'closed'){
    this.stream.connect(port, host);
  }
  else if(this.stream.readyState == 'open') {
    this.emit('connect');
  }

As the underlying stream is in state opening after the first connection attempt, no action is actually being taken here on the second one. Repeating the connection attempt only leads to adding another client to the connection pool (until all its free slots are exhausted).

My first attempt at a solution is adding a line in pg.connect:

var onError = function(error) {
  client.removeListener('connect', onReady);
  callback(error);
  client = new Client(config); // <- this one
  pool.checkIn(client);
}

This seems to work fine here, however I am not convinced that this is actually the best solution as it leaves the original stream object (and underlying fd) in its half-open state.

[freewil]

@brianc Any progress with this? I'm running into a similar scenario where I want to be able to handle disconnects/connection failures from the server and try to reconnect on a given interval.

[hampsterx]

bump this is kind of a huge deal breaker for me right now!! Restart postgres and my nodes apps are hung.. :(

[brianc]

Do you have a way to reproduce this? A failing test? It's been a long while since I've taken a look at this. If you can gist some code to reproduce the issue I can absolutely fix it within 1-2 days.