Cache support

[vicb]

Cache support.

This is a big chunk of work that should be split into multiple sub-tasks when we start the work.

References:

• https://opennext.js.org/aws/inner_workings/architecture
• https://opennext.js.org/aws/inner_workings/caching
• https://opennext.js.org/aws/inner_workings/cache_interception
• https://opennext.js.org/aws/inner_workings/components/revalidation
• https://github.com/opennextjs/opennextjs-aws/tree/main/packages/open-next/src/cache

AWS uses multiple components for the cache:

• a S3 storage for the content
• dynamoDB for the tags
• a queue to serialize & dedup access

This task is about providing a cache (incremental + tag) on the cloudflare infrastructure.

We have started to explore multiple possibles solutions using:

• KV
• Queues
• Durable Object

TODO:

Support for Cache Invalidation by Tag (#91)

--

Second thing we should tackle with this issue is errors when running i.e. the middleware example on the experimental branch with browser cache enabled. fixed in opennextjs/opennextjs-aws#683

--

Edit the main branch has basic cache support (kv + assets).
It's missing revalidation

[conico974]

If you plan on reusing the aws cache handler as it is i can give some advice.
The 3 component we have can easily be overrided

S3 storage

For the S3 storage ( the incremental cache and the fetch cache ) it should be the easiest and the obvious choice seems to be KV.
Here is the aws implementation for context https://github.com/opennextjs/opennextjs-aws/blob/main/packages/open-next/src/cache/incremental/s3.ts.
You'll just need to provide something that implement the IncrementalCache interface
The cache also needs to be populated at build time

Queue

The role of the queue is for ISR revalidation, when a page is marked as stale, the queue component will be called.
For this one there is an easy and a more complex but better implementation.

For the easy one, you don't need any other component you just do exactly the same thing as in the run locally example for the dev/queue.ts component : https://opennext.js.org/aws/contribute/local_run
This solution has a drawback, it doesn't handle deduplication. Multiple user reaching a stale page would trigger multiple revalidation

For the more complex solution, i'm not sure, but it seems that durable object could be a way to go
The message that you get include a MessageDeduplicationId, it is computed in a way that there is a finite number of id (defined by this env variable MAX_REVALIDATE_CONCURRENCY, default is 10) and every route will always output the same id.
This could allow to spawn MAX_REVALIDATE_CONCURRENCY durable object that would then need to handle deduplication and trigger the revalidation in the same way as the easy one.
Not sure if a cloudflare queue could be used this way.

Tag cache (DynamoDB)

It is only used for revalidateTag or revalidatePath

Really not sure about this one, it is heavily geared toward DynamoDB to take full advantage of it.
If used as it is now the only component that i could see working for this would be D1.
For reference here is the default aws implementation : https://github.com/opennextjs/opennextjs-aws/blob/main/packages/open-next/src/cache/tag/dynamodb.ts
It also needs to be prepopulated at build time

I also have plan to offer an alternative way to query for the tag cache (that is closer to the next implementation in the default handler), but that means having to query for a bunch of tags for every get request

These 3 are also used by the experimental cache interceptor that avoid having to reach the server to serve ISR/SSG route.

[vicb]

Thank you so much for the super valuable info on those github issues/tasks.
❤️

[IgorMinar]

I suspect that the easiest way to go about this is to use KV for storage and serving. In terms of latency, it's on par with the Workers Cache API, but unlike the Cache API, KV is globally distributed.

Queue can be used to dedupe revalidation requests so that we don't overwhelm the DB.

DO is likely not a great fit here as it could get overwhelmed by the thundering herd when the cache is empty.

One more thing to figure out is the tag based invalidation. We might need some kind of tag to paths mapping to support that.

There is one more thing that I don't see mentioned here at all. This issue is about serving cache, but Next also has a separate concept of a data cache, which is used to cache data requests. @dario-piotrowicz and @james-elicx spent built out a solution for that in next-on-pages: https://github.com/cloudflare/next-on-pages/blob/main/packages/next-on-pages/docs/caching.md Is this cache already supported or do we need to create a separate tracking issue for it?

[vicb]

but Next also has a separate concept of a data cache, which is used to cache data requests

The Next fetch cache shares the same infra - there are different value types in a common store.

[ha1fstack]

Can the worker cache api be also supported too? KV is great but I would use worker cache if I don't need global distribution since it costs less.

[IgorMinar]

yes, ideally the implementation is replaceable.

we could even follow the precedent in next-on-pages and provide workers cache implementation by default, and offer kv+queue as a more scalable alternative as an opt-in.

[conico974]

About the cache api, there might be some issues with fallback:false types of route as they need to be present at runtime. (If not next will throw an exception and there is no easy way to bypass this behaviour)
Another thing about using the cache api is that (unless i'm mistaken and there is a way to preload the local cache everywhere) page will have to be rerendered on every cache miss (even for SSG route)

We might need some kind of tag to paths mapping to support that.

OpenNext already provides that, but for it to work properly, the tag cache needs to be initialized at build time.
There is a function that can do that automatically if properly setup

[james-elicx]

About the cache api, there might be some issues with fallback:false types of route as they need to be present at runtime. (If not next will throw an exception and there is no easy way to bypass this behaviour) Another thing about using the cache api is that (unless i'm mistaken and there is a way to preload the local cache everywhere) page will have to be rerendered on every cache miss (even for SSG route)

The seed files don't necessarily have to be put into the cache, they could be stored as part of the deployment instead and be served from there internally - similar to what is currently done in the main branch (it was removed from experimental branch). Then it would be a bit more friendly for cache stores that aren't as easy to seed.

Unsure how that would work with what the aws adaptor has as I haven't really looked at it that much since Victor is leading that.

[conico974]

The seed files don't necessarily have to be put into the cache, they could be stored as part of the deployment instead and be served from there internally - similar to what is currently done in the main branch (it was removed from experimental branch)

So if the file are not there in the cache, it relies on the seed files ?
Wouldn't this cause issue with ISR or On Demand revalidation then ? If a file has been revalidated in a region and requested later from another one, then you'll get the old seed file (or a different revalidated version)

I guess what would make more sense would be to separate the routing/middleware in a different worker and rely on the cache control that is set by next server for the cache api (effectively shielding most of the request with the cache api)
And with this why not use R2 instead, it's a lot cheaper than KV on read and would provide what's needed for ISR and On Demand revalidation.

[james-elicx]

The seed files don't necessarily have to be put into the cache, they could be stored as part of the deployment instead and be served from there internally - similar to what is currently done in the main branch (it was removed from experimental branch)

So if the file are not there in the cache, it relies on the seed files ? Wouldn't this cause issue with ISR or On Demand revalidation then ? If a file has been revalidated in a region and requested later from another one, then you'll get the old seed file (or a different revalidated version)

It tries to serve from the cache, but if there is no file in the cache, it would serve the seed file from the deployment (which would be correct if a page hasn't been revalidated before). Works in the same way, just falls back to a different location if you're not able to seed the cache.