oxygen-full-page-cache: uncacheable

[dvnrsn]

I have a number of pages that don't seem to be utilizing full page cache. The client cache is working in the immediate browser, but upon inspection in different browser or incognito, the page is not readily available as I would have expected. It seems there are additional requests sent to workers (rerunning loader) which is undesired.

How can I remedy this oxygen-full-page-cache: uncacheable response header. I have checked the list here and think I've met all conditions?

Full-page cache works by intercepting incoming requests and checking if a cached version of the requested content is available

Response (edited for brevity)

HTTP/3 200 
date: Tue, 10 Sep 2024 22:28:58 GMT
content-type: application/json; charset=utf-8

cache-control: public, max-age=2592000, stale-while-revalidate=86400
vary: Accept-Language. Accept-Encoding
oxygen-cache-control: public, max-age=2592000, stale-while-revalidate=86400
oxygen-full-page-cache: uncacheable
powered-by: Shopify, Oxygen, Hydrogen
x-remix-response: yes

server: cloudflare
content-encoding: br

another example (both GET)

:status: 200
Alt-Svc: h3=":443"; ma=86400
Cache-Control: public, max-age=2592000, stale-while-revalidate=86400
cf-ray: ...
Content-Encoding: br
Content-Type: application/json; charset=utf-8
Date: Tue, 10 Sep 2024 22:36:29 GMT
nel: {"success_fraction":0.01,"report_to":"cf-nel","max_age":604800}
oxygen-cache-control: public, max-age=2592000, stale-while-revalidate=86400
oxygen-full-page-cache: uncacheable
powered-by: Shopify, Oxygen, Hydrogen
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=wef"}],"group":"cf-nel","max_age":604800}
Server: cloudflare
Vary: Accept-Language. Accept-Encoding
x-remix-response: yes
x-request-id: ...
x-shopid: ...

• The request has method GET.
• The response has an Oxygen-Cache-Control header
• The response has a public directive specified in the Oxygen-Cache-Control header.
• The response has a 2XX or 3XX status code.
• The response doesn't have a Set-Cookie header.
• The response has a Vary header
• The response doesn't a Vary: * header

[jgodson]

Hey Devin,

We have been meaning to see if we can make this less confusing/improve this, but we haven't been able to get to that yet.

Cache reads are not free, so we've made some decisions here on the Oxygen side as a tradeoff to ensure that when you are not using full page cache, we don't add unnecessary latency. Let me describe how full page cache works for context:

First note that response cache as well as the ephemeral cache I mention below is PoP (point of presence/datacenter) local.

• The first time a request hits Oxygen and it renders your Hydrogen app that returns a response eligible for full page cache, we store that in ephemeral state and the response cache state is uncacheable (this is something that I think is confusing and I'd like to see if we can fix). This is a signal to Oxygen that your app "opted in" to full page cache.
• The second request to Oxygen for your app will return the response as well as actually store it in the response cache. The response cache state here is miss.
• The third request to Oxygen for your app will then return the cached response and the response cache state will be hit.

The above hit should continue for all requests to that datacenter that are eligible to use the cached response, until you start getting stale and then we can revalidate in the background (assuming you've supplied state-while-revalidate). Once revalidation is done, response cache state should return it to hit. That said however, you may potentially see the pattern restart if the ephemeral state is cleared or if the pages that return full page cacheable responses are low traffic (or low traffic in general).

Hopefully the above helps explain the behaviour you are seeing.

[jgodson]

Answering your questions based on the current state of things, that may change based on us looking into improvements in this area:

Where is the threshold between low and sufficient traffic, and what constitutes "sufficient" traffic?

There is no magic number here. As long as there is enough traffic to keep hitting a url that is opted into full page cache every 2 minutes, that should be sufficient. That said, this is per worker at the moment. It's entirely possible that there are multiple workers serving traffic and it takes some time before all of them are "aware" of the use of full page cache. This is the reason you can end up with a hit and then an uncacheable on the next response.

I will say that most likely if your page that uses Oxygen full page cache is visited often, that you will see better hit rates with production traffic than in development.

Also a note on Vary, if you do not change responses based on Accept-Language, then you can omit Accept-Language entirely, which should help. I am looking into whether Accept-Encoding is actually necessary to add as well. If you are using another mechanism to set language besides a url (ie: a session cookie and can return different content for the same url) then you should not use Oxygen's full page cache as currently you can only Vary by header values at the moment.

What type of traffic is being measured?

Any page that returns a response that is eligible for full page cache. If 1 page out of 4.5k is using it, that is fine as long as that page gets visits.

What level of traffic would be sufficient to maintain longer cache durations?

Again, I can't say exactly what this number would be.

Is the traffic measured per page or across all pages within a specific datacenter?

All pages within a datacenter

Lastly, is the caching behavior for sufficient traffic regulated by the max-age and stale-while-revalidate headers? Are there any production-specific limitations that we should be aware of in advance?

Those have no implications on what "sufficient traffic" would be.

In the case of multiple datacenters, does each datacenter independently determine when to activate full-page caching?

Currently, yes. As mentioned though, split more granularly into each worker running in that datacenter. It is definitely possible to get an unexpected mixture of responses because of this.

The custom cache implementation is a reasonable approach if Oxygen's full page cache doesn't fit what you need. You have much more granular control that way for what is served. It is still datacenter specific (because the Cache API is), however doesn't have the same opt in behaviour that I mentioned.

Are there any potential pitfalls or best practices we should follow when using this approach?

Similar to using Oxygen's full page cache, ensure that the cached response is definitely applicable to those requests you are going to serve it for. ie: taking the buyer's selected language into account.

Oxygen full page cache also supports stale while revalidate, the Cache API does not do that for you automatically. See here for some details on how you could do that if you are not using Hydrogen's built in options.

One oddity we’ve noticed is that this custom cache seems to last significantly longer than the full-page cache—sometimes
several hours—even without traffic. It also appears to persist even when the worker is inactive. Could you explain why this occurs?

I would suspect this is entirely due to the opt-in behaviour I mentioned. I would expect that after leaving it for more than 2 minutes, you would get a uncacheable, then immediately start getting hit's again once it was opted in again. Actual time the response is available in cache should adhere to what was set in the Oxygen-Cache-Control response header.

Tiered Caching to distribute the cache across all workers simultaneously.

Tiered Caching works with the assets mentioned here. Tiered Caching unfortunately does not work with the Cache API in workers.

Cloudflare KV for more granular caching control.

We do not support Cloudflare KV currently on Oxygen.

Hopefully that answers your questions. I'll be sure to update this thread with any details of improvements that we make in this area.

[usulpro]

Thanks a lot for the answers! Things are much clearer now. It’d be really great if you could share any cache updates here too.

[usulpro]

I’ve encountered another issue with full-page caching, which seems to come from the Hydrogen boilerplate implementation itself.

At the server.js level, a user's shopping cart is added to the application context. As a result, the cached response includes this data, and it gets distributed to all users who access the cached page. The challenge here is that cart data is added to literally every page, as it's part of the global context for rendering.

To reproduce the issue, I created a new hydrogen app from scratch, linked it to our store, and applied caching headers to the category pages. Other pages don’t use caching, so I can easily compare behaviors. Next In two incognito browser sessions, I added a product to the cart in one, triggered caching for the page, and then get the cached version of the same page in another browser. The cart data from the first session appeared in the second one, which is obviously a major security concern.

I’ve recorded a video to demonstrate the issue:

full-page-cache-with-card.mp4

You can also try it yourself here: https://quick-start-b6f02faf55b07f837ce5.o2.myshopify.dev/collections/collezione-letto (cache enabled page).
Repo: https://github.com/usulpro/hydrogen-quickstart.

Could you clarify how caching should be implemented in this scenario? Adding Oxygen-Cache-Control headers alone clearly isn't sufficient, given the issue. Is this behavior a bug, or is it expected that further development is required to enable caching safely?

If modifications are needed, what other aspects should be considered? I assume the same issue could occur with user sessions. Do you have any best practices or examples on how to safely use caching without exposing personalized data?

Ideally, I believe cart data and any personalized information should be handled client-side, excluding such data during server-side rendering. Does this approach align with your recommendations, or am I missing something here?

[jgodson]

Yes, exactly, the TLDR is that you can't use full page cache with a server side cart or if you have any personalization rendered server side. You would need to move to a client side implementation instead.

There was an example started here of using full page cache, but it didn't make it in. As you can see in the issue there, it was going to be part of the starter template. However I believe the Hydrogen team has been holding off on that due to the current issues.

#2214

[jgodson]

For anyone following along here, we shipped a small change today to improve the uncacheable behaviour when a response was actually cacheable.

Instead of previously where you would get uncacheable -> miss -> hit you should now get miss -> hit.

And previously after 2 minutes of no full page cache eligible responses returned from the hostname (or if the ephemeral state gets cleared, which as I mentioned can be random) you would get uncacheable -> hit, you now should get miss -> hit.

I think these small changes at least make things less confusing, which not necessarily improving the hit rates yet.