Dedup Index Cache requests

[yeya24]

Is your proposal related to a problem?

It is very common to have duplicate Index Cache requests on a single Store Gateway. For example, when opening a Grafana dashboard, multiple panels might request the same matcher, causing getting/setting the same posting/series multiple times.

Especially for those postings that has very high cardinality, downloading those data can consume a large amount of bandwidth. For example, we have posting with size over > 100 MB. When we have 10 duplicated requests we overfetch 1GB of data.

Describe the solution you'd like

Introduce https://pkg.go.dev/golang.org/x/sync/singleflight to index cache clients. For example, memcached client, redis client and even the inmemory cache client.

Describe alternatives you've considered

Implement GroupCache which provides the same SingleFlight capability. However, as a user I still want to use existing cache clients but have SingleFlight support.

[yeya24]

cc @GiedriusS does this make sense to you? If it makes sense then I can start working on it.