State Cleanup Ideas #956
Replies: 4 comments 9 replies
-
I'm in favour of this general workstream, so I'm glad to see some ideas about how we can improve this -- the size of the current state tree is an invisible tax that saps energy from the Filecoin ecosystem (node operator time in downloading a snapshot, barrier to new folks joining the network, processing time of anyone performing analytics over the entire state, developer and protocol evolution time when we perform state migrations). Having said that, I'm not especially inclined towards the ideas listed here. I don't really want to add more work into cron (even deleting state), and Forced Cleanups, while agreeable, are ultimately just more protocol logic that I'm disinclined to add. What I would personally be more favourable towards is one-off cleanups that occur at network upgrades. This keeps the protocol simpler (no new potentially buggy code), doesn't add anything into cron (which would only affect things moving forward anyway), and gives the network much more control over the cleanup: we can all observe and agree exactly what would get cleaned up at the migration time. I don't want to derail this discussion, so if there's support for that idea I'm happy to write up an alternative proposal. |
Beta Was this translation helpful? Give feedback.
-
much needed discussion. literally years ago we discussed somewhere to allow active miner termination, delete the actor, under certain conditions in exchange for unlocking the vested rewards immediately. would give an extra ~6 month less state compared to waiting for all funds to release from reward vesting.
i don't like this approach at all. its literally a punishment we try to sell as an incentive here.
this should not be the beginning of the discussion on how to design incentives to "force" positive behavior. most of the screws you wanna turn can easily be turned in the opposite direction to have folks benefit from doing this. we as a community agreed that those states are needed and being kept in the way they are. asking a specific group to come up for the clean up cost by threatening a punishment should be the last resort. between cutting the 180 days vesting time down to 0 and in the end i agree with @arajasek that this is best solved via network upgrades. it's community debt, not debt of specific actors on chain. |
Beta Was this translation helpful? Give feedback.
-
I think we have the opportunity to implement this by allowing the miner owner to terminate a "zombie" miner and delete all related state data. FIP-0077 proposes adding an opportunity cost for creating new miners. By integrating this idea, we could treat the fee for new miner creation as collateral, which the network would retain until the miner owner decides to terminate the miner actor. This approach could simplify both the implementation of adding costs and the removal of miner states, as they would become more independent. For the cleanup of other forced sectors or claims, I would prefer handling this during a network upgrade, as suggested by @arajasek . |
Beta Was this translation helpful? Give feedback.
-
If we require running compact partitions to release IP after expiration this would solve 95% of future state issues. And it would serve the extra benefit of removing more work from cron. If we ever need/get to safe cron and move handle proving deadline out of cron we could just do compact partitions inline. There is the additional problem of 30-40G of dead sector infos belonging to SPs who may be long gone from the network. There's a lot of things we could do here but the simplest thing is probably just a migration to clean these up. |
Beta Was this translation helpful? Give feedback.
-
See #771 for gas improvements, this discussion focuses on cleaning up dead state.
Delete miner state when unenrolling from cron
We unenroll from cron when the locked funds and pledge all go to zero, indicating that there are no live sectors and no remaining vesting funds:
https://github.com/filecoin-project/builtin-actors/blob/8ca9c6c667fd069705ca186a7e4fd2346d6b4cd1/actors/miner/src/lib.rs#L4400-L4410
When that happens, we have an opportunity to remove miner state.
Ideally we'd just send all funds to the beneficiary and delete the entire actor, but some users may object to that. Instead, we can just delete all the sector infos, deadline information, and market information:
info
(miner info).fee_debt
(could still be non-zero).allocated_sectors
(we still don't want to reuse sector IDs)proving_period_start
provider_sectors
table. The deal states will get cleaned up automatically over time.This will still leave some dead state, but nothing that won't be cleaned up automatically (unless I'm missing something).
Forced Sector Cleanup
The above only handles "dead" miner actors. If we want to cleanup sectors from live miner actors, we likely need to force it.
At the moment, any SP can run
CompactPartitions
to cleanup/compact their partitions, deleting dead sector infos. Unfortunately, SPs rarely do this as the cost of keeping around dead state is pretty low.One option is to have forced garbage collection by:
The idea is to provide a strong incentive to "fix" the problem quickly, without causing an emergency.
Forced Claim Cleanup
Forcing the cleanup of expired claims would require a little bit of extra state, but it's possible. Basically, we'd:
Given that the term size is 5 years, we can keep keep track of this with a 60 element ring-buffer (maybe a bit longer to take term starts into account?).
In terms of incentives, increasing gas fees as "cleanup debt" builds seems like a decent approach. We could also have a hard cutoff, but I'm not a huge fan of those.
Beta Was this translation helpful? Give feedback.
All reactions