Skip to content

Conversation

taylorotwell
Copy link
Member

This PR implements @oprypkhantc's solution for more performant / scalable cache tags when using Redis. Namely, it solves the problem of orphan reference keys in the cache which can accumulate to take up significant space and can't be purged efficiently.

This solution stores the list of keys for a given tag in a sorted set where the expiration time is the score. Then, references may be easily removed using that score + a scheduled command (scheduled command not included in this PR at the moment).

Have removed existing Redis cache tag tests which were heavily mocked and will be adding integration tests for this feature.

Copy link
Contributor

@oprypkhantc oprypkhantc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than the missing command, this looks good 👍🏻

Regarding the tagKey/resetTag - if those are to be considered a BC, maybe it's best to get rid of those functions altogether, although this will likely only be possible in 11.x as it needs some changes in tags for other drivers too.

* @param string $name
* @return string
*/
public function tagKey($name)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tagKey used to return just "tag:$name:key" whose value pointed to the name of tag reference key, while tagId returned the reference key itself (laravel:6307213384953462982890:standard_ref). Technically, if somebody used those for whatever reason, it will be a breaking change for them.

Same goes for resetTag - if somebody calls that and expects the tag key to be present in cache afterwards (which is what used to happen), this is also technically a breaking change.

Both of these aren't major at all and I doubt many people used these at all, but it's worth mentioning.

Copy link
Contributor

@clemblanco clemblanco Jan 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't see a change in the tests for this kind of intermediate entries being stored properly either. Unless I missed it.

@taylorotwell
Copy link
Member Author

taylorotwell commented Jan 17, 2023

I'm not able to figure out why my testIncrementedTagEntriesProperlyTurnStale fails with phpredis / GitHub actions.

When I run the code locally on Redis 6.2.1 and use Predis, nothing is left in the Redis database after running that code (which is the correct behavior). However, on phpredis on GitHub actions I'm getting this failure. The test also passes locally when forced to only use Predis.

@oprypkhantc I added the GT stuff to increment and decrement to try to correct a behavior in your implementation where if a value was incremented it automatically essentially became a "forever" cached item in the sorted set with a score of -1, even if it was put originally with an expiration timestamp.

@taylorotwell
Copy link
Member Author

From what I can tell, it seems the GT flag is being totally ignored on phpredis, even on Redis 7 and the latest phpredis extension (5.3.7).

@taylorotwell
Copy link
Member Author

Created phpredis issue here: phpredis/phpredis#2319

@taylorotwell
Copy link
Member Author

Switched to using NX flag in the mean time which does seem to be respected and still gives me the behavior I want from increment and decrement I believe.

/cc @oprypkhantc

@oprypkhantc
Copy link
Contributor

NX should do the job just fine I think. We don't use cache increments/decrements with tags, so there are possibly other bugs with them too I haven't noticed, although I can't think of any.

@taylorotwell taylorotwell merged commit 129a0d7 into 10.x Jan 18, 2023
@taylorotwell taylorotwell deleted the redis-cache-tags branch January 18, 2023 14:56
@clemblanco
Copy link
Contributor

Nice one! Thanks.

Follow up question: do we need the same thing for other drivers supporting tags? Were they using the same kind of initial logic?

@oprypkhantc
Copy link
Contributor

Yes, all of them leave stale tags behind, unfortunately. Thankfully there's now proper implementation for Redis - which seems to be the most scalable solution (now that AWS has MemoryDB) either way.

@taylorotwell
Copy link
Member Author

@clemblanco in Memcached this wasn't as much of a concern because Memcached doesn't grow indefinitely. Once you reach your configured memory size it will start pruning entries based on last access time. Redis has no such mechanism (afaik) so it just grew indefinitely.

@taylorotwell
Copy link
Member Author

Command committed here: 16e9f0e

@GrahamCampbell GrahamCampbell changed the title More scalable Redis cache tags [10.x] More scalable Redis cache tags Jan 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants