Support for Batch Get and Missing Keys Detection (Granular Caching Use Case)

[AlirezaEiji191379]

Use Case: Granular Caching for Entity Lists

Hi, and thanks for this amazing library!

I have a use case involving multiple microservices communicating with each other and caching shared entities.

Scenario:

Imagine I have a microservice that requests entities 1 and 2 together and caches them. Later, that microservice requests entities 2 and 3 and also caches them.

The problem is that entity 2 is now cached twice, separately, and when entity 2 gets updated, I need to invalidate it in both places — which is error-prone and inefficient.

To solve this, I want to cache each entity separately using a per-entity cache key (e.g., entity:{id}). So when I need a list of entities, I can check which ones are already in the cache and which ones are missing, then fetch only the missing ones from the source.

This requires an efficient batch get/check mechanism from the cache.

My question:

Does FusionCache support (or plan to support) a way to:

• Provide a list of keys
• Retrieve the cached values for those keys (if present)
• Identify which keys are missing from the cache

This would allow me to implement a smart caching strategy where only uncached entities are fetched, and already cached ones are reused.

What would be the best strategy to implement this pattern with FusionCache?

Thanks in advance!

[jodydonetti]

Hi @AlirezaEiji191379 , eheh you just touched a very interesting but also complex topic 😅

I'd like to give you a thorough answer since there are multiple considerations to be made, but I need some time do it, so please be patient.

Meanwhile, one thing about this passage:

The problem is that entity 2 is now cached twice, separately, and when entity 2 gets updated, I need to invalidate it in both places — which is error-prone and inefficient.

Have you already taken a look at Tagging? I think you can find an interesting approach there...

Anyway I'll come back to you as soon as I can!

[AlirezaEiji191379]

Thanks a lot for your response — I'm glad you found the topic interesting!

Yes, the double caching and invalidation issue is something I’ve been struggling with. I haven’t explored Tagging deeply yet, but it definitely sounds like a promising approach. I’ll take a closer look.

In the meantime, I’d be happy to help with any discussions or development around this feature if needed.

One approach I was considering for handling this within FusionCache was to cache each entity individually. This would effectively solve the invalidation issues and provide more granular control.

However, the downside is that each time we load data from the source (e.g., a downstream microservice like service B), we’d need to scan through the keys to determine which entities are already in the cache and which aren’t — resulting in O(n) operations. For the missing ones, we would then need to make requests to the source.

The main concern here is that for each entity, we’re hitting both L1 and L2 caches individually, which adds overhead.
What I had in mind was to experiment with using something like an in-memory SQLite database as an alternative L1 cache. This would allow querying a batch of keys efficiently and returning only what’s available. With a feature flag, this could be swappable with the default IMemoryCache.

This way, we could even support lightweight querying over the internal cache store and simplify how we handle entity breakdowns.
Would love to hear your thoughts on this approach!

[ZUOXIANGE]

in-memory LiteDB is also a good choice for L1 cache

[Xor-el]

I was thinking — a more flexible and backend-agnostic approach (if technically possible) would be to support key pattern searches, where the storage layer returns a list of values that match a given wildcard key pattern.

For example, if the cache backend is Redis, this could be implemented using a KEYS or SCAN command to retrieve all matching keys, and then fetch their corresponding values in bulk.

or maybe this could be achieved by retrieving the values via their associated tags?

both approaches would be super awesome 💯

something like this for Redis if we are implementing it from the scratch

        System.Net.EndPoint[] endpoints = _connectionMultiplexer.GetEndPoints();

        foreach (System.Net.EndPoint endpoint in endpoints)
        {
            IServer server = _connectionMultiplexer.GetServer(endpoint);

            if (server is null)
            {
                continue;
            }

            for (int dbNumber = 0; dbNumber < server.DatabaseCount; dbNumber++)
            {
                IDatabase db = _connectionMultiplexer.GetDatabase(dbNumber);

                if (db is null)
                {
                    continue;
                }

                // Scan the database to find all keys patterns
                await foreach (RedisKey key in server.KeysAsync(database: dbNumber, pattern: $"fusion*"))
                {
                    var json = await db.StringGetAsync(key);
                    // fusionCache does it's magic
                }
            }
        }

what do you think @jodydonetti ?

[jodydonetti]

Hi @Xor-el

I was thinking — a more flexible and backend-agnostic approach (if technically possible) would be to support key pattern searches, where the storage layer returns a list of values that match a given wildcard key pattern.

FusionCache is designed on the shared IDistributedCache abstraction to be able to automatically work on any implementation out there (see here for a partial but almost comprehensive list), therefore the only features available are the ones defined there.

For example I've been able to pull off Tagging even though IDistributedCache doesn't support it because of the particular design I came up with, but those limitations still apply.

Hope this helps, let me know.

[Xor-el]

Hello @jodydonetti this makes sense, thanks for the clarification.

[esd-thg]

I found this looking for the same as OP

• Provide a list of keys
• Retrieve the cached values for those keys (if present)
• Identify which keys are missing from the cache

I've tinkered a bit and found that fetching each key individually from the cache is acceptable performance - it actually seems like the only way :)

However fetching each key from the factory (a database) individually is incredibly slow. This is somewhat expected.

@jodydonetti Would it be possible to batch the factory calls? Perhaps something like

await cache.GetOrCreateAsync(
    ["key1", "key2", ...],
    async (missedKeys, cancel) => // call db with missedKeys
);

The factory would have to return results in a way that each value can be mapped to a cache key.