Fix DB to prefer batches with preferred batch size for default max_queue_d…

[tanmayv25]

…elay

[deadeyegoodwin]

@tanmayv25 oops, didn't mean to approve yet. I had one requested change...

[GuanLuo]

if pending_batch_delay_ns_ == 0, should the batcher NOT return 0 to wait for preferred batch or max batch?

[tanmayv25]

But the question then becomes how long should it wait for forming the preferred batch size. These changes only let batcher prefer a preferred batch size if one can be formed. Otherwise it will issue the request with pending batches.

[deadeyegoodwin]

When delay is 0, the batcher should form the largest possible preferred batch, or if can't form any preferred batch then just the largest possible batch, and execute it immediately.

Tanmay, we should have a test with no delay where there are multiple preferred sizes and make sure that the largest preferred size is used, or when no preferred can be formed that the largest possible batch is formed. We already have these kind of tests but I guess they all use delay.

[GuanLuo]

I see.

[tanmayv25]

I am adding the tests...