kafka: bias fetch planner to prefer reading partitions with data in SI cache

[jcsp]

When a client issues a fetch RPC that touches many partitions, the kafka layer makes a decision about which partitions to include in the result. Currently this decision does not take account of which partitions may have fetch results returned cheaply from the SI cache, vs. which may require an expensive segment hydration.

Fetches should advance somewhat uniformly across all the partitions they are interested in, but within some bounds we may bias this: for example, as long as no partition is more than 1 segment size ahead of the others, we can continue to return results from the partitions we happen to have data handy for.

Without this change, if the segment_size * partition_count > cache_size, then even one consumer reading a topic may experience very bad thrashing of segments in/out of the cache.

This change will make no difference in the case of 1 consumer per partition, but in the case of 1 consumer reading many partitions under cache space pressure, it will be far more efficient.

JIRA Link: CORE-1041

[twmb]

I don't think I'm reading this well enough, and I have a question of how this impacts the client side. Clients are supposed to rotate through the partitions they fetch to ensure no partition starves. First I fetch [1, 2, 3], then I fetch [2, 3, 1], then I fetch [3, 1, 2], and so on. Does the above impact starvation?

[jcsp]

I don't think I'm reading this well enough, and I have a question of how this impacts the client side. Clients are supposed to rotate through the partitions they fetch to ensure no partition starves. First I fetch [1, 2, 3], then I fetch [2, 3, 1], then I fetch [3, 1, 2], and so on. Does the above impact starvation?

Interesting question. I guess we'd be taking this out of the client's hands to some extent: with some threshold, we would be ignoring the client's order of preference in order to serve data from cache as much as possible.

The hand-wavy argument for why it's okay for partitions to be more out-of-step than usual is: if someone's reading data from S3, they're probably doing a full-topic replay rather than trying to keep up with a particular "latest" point.