Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Index Cache: Change fetchMulti interface to return slice rather than map #7604

Open
yeya24 opened this issue Aug 6, 2024 · 3 comments
Open

Comments

@yeya24
Copy link
Contributor

yeya24 commented Aug 6, 2024

Is your proposal related to a problem?

IndexCache interface has two FetchMulti methods FetchMultiPostings and FetchMultiSeries.

For example, FetchMultiPostings looks like below:

FetchMultiPostings(ctx context.Context, blockID ulid.ULID, keys []labels.Label, tenant string) (hits map[labels.Label][]byte, misses []labels.Label)

The hits map is kind of unnecessary. It can be replaced by a slice [][]byte where the index maps to the input keys. This should save some memory when fetching data from caches.

Describe the solution you'd like

Change FetchMultiPostings to this method below. We can even change misses to []int and see if it is something easy to do.

FetchMultiPostings(ctx context.Context, blockID ulid.ULID, keys []labels.Label, tenant string) (hits [][]byte, misses []labels.Label)
@Zyyeric
Copy link

Zyyeric commented Aug 16, 2024

To convert 'Label' to an int probably requires some forms of hash function, but it will always compromise to the possibility (despite very low) of collision which would result a missing cache hit.

Would this be acceptable? or there is a better implementation you have in mind?

@yeya24
Copy link
Contributor Author

yeya24 commented Aug 17, 2024

@Zyyeric The return result slice can use the index as of keys []labels.Label by maintaining the same order. But the downside is that we have to allocate the slice even though nothing is cached.

@Zyyeric
Copy link

Zyyeric commented Aug 18, 2024

@yeya24 Ahh I see, this sounds valid. Despite the allocating for miss hits for the slice, its memory overhead shall still be lower than that of a map. This shall not be hard to do. I'll give it a shot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants