You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
While adding some integration tests for our Akka.Cluster.Sharding persistence cleanup tool, I ran into a small problem on this PR: petabridge/Akka.Cluster.Sharding.RepairTool#7 - Akka.Persistence.Redis' IReadJournal does not support ICurrentPersistenceIds().
I know we removed it as part of #126 because we thought it'd be too expensive to implement in clustered redis scenarios, but I wonder if that's true.
Describe the solution you'd like
The "all events" and "events by tag" queries are out of the question - they're expensive because they can't effectively be cleaned up in a clustered environment without some kind of inter-node index and we're not database architects so we're not going to build one. Users can use a relational database for that.
But Akka.Persistence requires us to keep track of all used PersistentIds even after the entities have had all of their data purged - we're required to keep those records in perpetuity and therefore cleanup is not an issue.
So, I suppose we can try implementing the ICurrentPersistentIds and IPersistentIds queries in one of two ways:
Scatter-gather to all of the nodes in the cluster each time we need to run one of those queries and just use the built-in key indicies we have now. That's O(n) where n = entity count.
Create a special table that only really needs to get replicated to single entity and have it contain a hashset that has all of the entity ids and we can use a Redis channel to receive notifications when that entity is updated. I don't really like this idea because it's single point of failure.
Describe alternatives you've considered
We could also go on not doing this, but since we're taking a dependency on ICurrentPersistentIds for helping Cluster.Sharding users cleanup I'd like to do our best to support it.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
While adding some integration tests for our Akka.Cluster.Sharding persistence cleanup tool, I ran into a small problem on this PR: petabridge/Akka.Cluster.Sharding.RepairTool#7 - Akka.Persistence.Redis'
IReadJournal
does not supportICurrentPersistenceIds()
.I know we removed it as part of #126 because we thought it'd be too expensive to implement in clustered redis scenarios, but I wonder if that's true.
Describe the solution you'd like
The "all events" and "events by tag" queries are out of the question - they're expensive because they can't effectively be cleaned up in a clustered environment without some kind of inter-node index and we're not database architects so we're not going to build one. Users can use a relational database for that.
But Akka.Persistence requires us to keep track of all used PersistentIds even after the entities have had all of their data purged - we're required to keep those records in perpetuity and therefore cleanup is not an issue.
So, I suppose we can try implementing the
ICurrentPersistentId
s andIPersistentId
s queries in one of two ways:Describe alternatives you've considered
We could also go on not doing this, but since we're taking a dependency on
ICurrentPersistentIds
for helping Cluster.Sharding users cleanup I'd like to do our best to support it.The text was updated successfully, but these errors were encountered: