[dst] Improve tail-latency for operations of transactions using wait queues

[robertsami]

Jira Link: DB-3158

The biggest contributor to higher tail latency is caused by the following case of starvation -- in case there is a high-degree of contention, waiting transactions may get starved by incoming operations which contend for the same latch. We currently have no mechanism to prevent this, which can lead to high tail-latency in some workloads.

Less critically, our process for determining which waiters can be resumed and subsequently resuming them could be improved in a couple ways:

1. We currently iterate over each of the blocker's waiters and separately acquire a write lock on a mutex to remove the waiter from waiter_status_ before resuming the waiter. We need not re-acquire this write lock for every waiter and can simply acquire it once
2. We currently resume waiters in the order they arrived, and in serial on a single thread. It might be better to understand which of the waiters will conflict with each other, and then either:
   a. Resolve the first-in waiter and all non-conflicting other waiters in parallel
   b. Resolve the largest set of non-conflicting waiters in parallel, then the second largest, etc

[rthallamko3]

@robertsami , Can we close this or were you planning additional followups?

[robertsami]

the main fix for fairness has landed in f69dc2a

regarding the remaining minor points in the description:

1. is no longer relevant
2. is captured here and not as critical: [DocDB] Avoid re-running conflict resolution for waiters which are still blocked #16389