Why does Transactor.fromDataSource take an ExecutionContext and a Blocker?

[changlinli]

It seems like I should only need one of those.

That is if we're worried about doing stuff on the right blocking thread pool, then we use Blocker or we could not use Blocker and just use ExecutionContext as the blocking pool.

Looking at the implementation it looks like getting the DB connection is done on the ExecutionContext, closing the connection is done on Blocker. Is there any reason for this?

If so, perhaps it should be documented in the Scaladoc.

[tpolecat]

Here is the explanation. https://tpolecat.github.io/doobie/docs/14-Managing-Connections.html#about-threading

You need distinct pools because you never want a thread holding a connection to be blocked by a thread waiting on a connection, otherwise you can deadlock.

[tpolecat]

I'll take a PR if you'd like to improve the book text or Scaladoc.

[changlinli]

Ah I totally forgot about that page; sure I think a link to it in some of the Scaladocs might make sense. I'll probably whip up a PR for that in a bit.

On the other hand, something still feels off.

The main thing I guess that feels weird to me is that there's a "mixing" of different APIs going on. On the one hand ContextShift explicitly advertises itself as a pure equivalent to ExecutionContext and I think Blocker is one as well, so it feels weird to see all of them in the same API signature.

On the other hand, isn't this a more specific case of the general principle that when you have multiple threads with blocking operations they can deadlock if you have a bounded number of threads and data dependencies among them? I thought this was the main reason to have a separate, unbounded threadpool for blocking operations altogether. As long as your blocking threadpool is unbounded, couldn't you have both thread connections and waiting on a connection both be in the same threadpool?

[tpolecat]

The problem is that you can have an unbounded number of requests waiting waiting for connections, so you can run out of threads. The bounded connect pool throttles that down, but as a result you need a second pool for progress-making threads to avoid deadlock.

[tpolecat]

I agree it's weird to require all three things. That's just the state of the cats-effect API at this point.

[Gonzih]

Is this api now gonna change with release cats-effect 3?