lazy getting connections from pool

[calibr]

Hello, sorry for my English, it's my first pool request.
In some projects there are required to use One mysql connection per HTTP request, but in some cases such requests don't need a mysql connection, for example if response is cached or auth failed, etc.

• When use connections pool, need to extra call Pool.getConnection to obtain connection, it's complicates code.
• If request connection from pool for every http connection - size of pool grows up very fast in projects with large amount of http requests.

I write class that proxy calls to mysql connection, and get it from pool only if user really need it.
It works like a Connection object( but for pooled connections ) when it lazily connects on user request.

Simple example:

var pool = mysql.createPool({...});
var conn = pool.getLazyConnection();
conn.query("SELECT fields FROM data", function(){
  // do something
  conn.release();
});

More complex example for express:

var pool = mysql.createPool({...});

app.use(function( req, res, next ){  
  req.mysql = pool.getLazyConnection();  
  res.on("finish", function _onfinish(){
    res.removeListener("finish", _onfinish);  
    // if connection not obtained from pool, the "release" call do nothing
    req.mysql.release();
  });  
  next();  
});

app.get("/:field", function(req, res){  
  // condition which defines use mysql or not
  if( req.params.field == 1 ){
    req.mysql.query("SELECT field FROM data", function( err, data ){
      res.send("QUERY");
    });
  }
  else{
    res.send("OK");
  }  
});

This approach to get connections is similar to JAVA class LazyConnectionDataSourceProxy

[sidorares]

It's already possible with pool - just use pool.query() as lazyConnection.query() in your example

[calibr]

But pool.query() immediately release a connection after query, it makes impossible to use transactions

[dougwilson]

I'm not so sure if this is really needed in this library. I think this is more of a question about your server architecture. For example, you do not need to get lazy connections for your example if you implemented your routes like the following:

var pool = mysql.createPool({...});

function getMySQLConnection( req, res, next ){  
  pool.getConnection(function( err, conn ){
    if (err) return next(err);
    req.mysql = conn; 
    res.on("finish", function _onfinish(){
      res.removeListener("finish", _onfinish);  
      req.mysql.release();
    });  
  });
}

app.get("/1", getMySQLConnection, function(req, res){  
  req.mysql.query("SELECT field FROM data", function( err, data ){
    res.send("QUERY");
  });
});
app.get("/:field", function(req, res){  
  res.send("OK");
});

As you can see, if you moved your middleware to add the req.mysql property to be only directly on the routes the MySQL connection is actually used instead of unconditionally on all requests, you can then only acquire the connection when you need it.

Above, your conditional in /:field was if field was "1", so I split that into two routes such that when it was "1" you would acquire the connection and run a query, but for the rest you won't.

[calibr]

Thank you for response, I think that lazy pooling make code little bit clean, but maybe it's issue for another library.. If node-mysql lib doesn't need this approach please close this pull request.

[dougwilson]

We are still considering it. My main point was that express already has a built-in way to do what you want, you just have to use express with that in mind. This "lazy connection" implementation just adds an enormous amount of code for something express already solves.

[calibr]

in some cases express didn't solve this problem, if I put additional conditions like in this example:

app.get("/", getMysqlConnection, function( req, res ){  
  if( !req.body.email || !req.body.login || !/\d+/.test(req.body.num) ){
    // mysql connection not need here, but it's already instantiated
    return res.send("something wrong...");
  }  
  req.mysql.query("...");  
});

or when use cache system:

app.get("/", getMysqlConnection, function( req, res ){  
  var cache = Cache.get("...");
  if(cache){
    // cache used, but mysql connection instantiated
    return res.send(cache);
  }
  req.mysql.query("...");  
});

Yes, this issue can be fixed by adding more middleware functions.
But I think that my approach do it more simpler, by adding some abstraction level over connection.

Another design improvement in my opinion:

• when use var conn = mysql.getConnection() you can immediately run conn.query()
• when use var conn = pool.getLazyConnection() you can immediately run conn.query()

I think it's looks good, because uses the same ways to work with connections.

pool.getLazyConnection() may be replaced with pool.getConnection() without callback argument.