-
Notifications
You must be signed in to change notification settings - Fork 11.6k
[10.x] More scalable Redis cache tags #45690
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Other than the missing command, this looks good 👍🏻
Regarding the tagKey
/resetTag
- if those are to be considered a BC, maybe it's best to get rid of those functions altogether, although this will likely only be possible in 11.x as it needs some changes in tags for other drivers too.
* @param string $name | ||
* @return string | ||
*/ | ||
public function tagKey($name) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tagKey
used to return just "tag:$name:key" whose value pointed to the name of tag reference key, while tagId
returned the reference key itself (laravel:6307213384953462982890:standard_ref
). Technically, if somebody used those for whatever reason, it will be a breaking change for them.
Same goes for resetTag
- if somebody calls that and expects the tag key to be present in cache afterwards (which is what used to happen), this is also technically a breaking change.
Both of these aren't major at all and I doubt many people used these at all, but it's worth mentioning.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Couldn't see a change in the tests for this kind of intermediate entries being stored properly either. Unless I missed it.
I'm not able to figure out why my When I run the code locally on Redis 6.2.1 and use Predis, nothing is left in the Redis database after running that code (which is the correct behavior). However, on phpredis on GitHub actions I'm getting this failure. The test also passes locally when forced to only use Predis. @oprypkhantc I added the |
From what I can tell, it seems the GT flag is being totally ignored on |
Created |
Switched to using /cc @oprypkhantc |
|
Nice one! Thanks. Follow up question: do we need the same thing for other drivers supporting tags? Were they using the same kind of initial logic? |
Yes, all of them leave stale tags behind, unfortunately. Thankfully there's now proper implementation for Redis - which seems to be the most scalable solution (now that AWS has MemoryDB) either way. |
@clemblanco in Memcached this wasn't as much of a concern because Memcached doesn't grow indefinitely. Once you reach your configured memory size it will start pruning entries based on last access time. Redis has no such mechanism (afaik) so it just grew indefinitely. |
Command committed here: 16e9f0e |
This PR implements @oprypkhantc's solution for more performant / scalable cache tags when using Redis. Namely, it solves the problem of orphan reference keys in the cache which can accumulate to take up significant space and can't be purged efficiently.
This solution stores the list of keys for a given tag in a sorted set where the expiration time is the score. Then, references may be easily removed using that score + a scheduled command (scheduled command not included in this PR at the moment).
Have removed existing Redis cache tag tests which were heavily mocked and will be adding integration tests for this feature.