Connection Pool

[Vidhya-RajaMani]

Is there any key to control the DB- connection pool size?

[ColinH]

Not at the moment, sorry. What would you expect to happen when another connection is requested from a pool that is at capacity?

[d-frey]

The question is also not easy to answer, as the pool might contain a number of connection, but there are more connections that are currently in use by the application and will return to the pool. Those two sets of connections might play a role when deciding whether to open another (new) connection or not.

Usually, new connections will only be opened when there are no available connections waiting in the pool. Therefore, the number of connections currently in use by the application is crucial to implement some limitation properly. There are more things to consider: How many connections should we try to open simultaneously? In a multi-threaded environment, multiple threads might run into a "empty" pool and will then try to open connections. Should this also be limited?

Besides the limit on the number of connections, you might also want to limit the rate with which you open connections. And if you are rate-limited, do you throw an exception or do you block the thread until a (new) connection becomes available? Also, to avoid spamming the network (and possibly a log-file), you might want to limit the initial connection to be a single, rate-limited connection attempt. After all, if you can not reach the DB-server, going faster or opening multiple connections in parallel will not help and only makes it harder to properly debug the issue.

Besides a maximum you also might want to have a minimum number of connections, e.g. you proactively open new connections and put them in the pool to reduce the lag of the threads that need connections. And when you close connections due to timeouts, you don't want to close too many at the same time to avoid starving the pool.

Note that I have some experience with all of the above points. In a different code base I had to deal with all of the above issues, and more. So yeah, long story short: It's difficult. If I find more time, I'll have a look at how to add the necessary callbacks/virtual functions, etc. so you could implement some of these requirements.

[d-frey]

I added some code to support tracking the number of attached connections to a pool as well as methods to query the pools size. You can find an example in src/lib/pq/connection_pool.cpp that shows how to create a pool with a limited size. Feedback welcome.