Experiment with removing zero lamport accounts from non-empty storages

[carllin]

Problem

As a refresher, we add keys to the candidate set on:

1. Dead slots, we add the storage entry to the candidate set
2. Shrinking slots, we add the storage entry to the candidate set
3. New roots, we add the storage entry to the candidate set
4. Bank freeze we add the dirty pubkeys to the candidate set

So a zero lamport key will never be cleaned if there ever is ever a scenario where:

1. We add a zero lamport key to the candidate set
2. The zero lamport key is not removed via clean
3. We then never add the zero lamport key back to the candidate set

I think this can happen if:

1. We add the zero lamport key to the candidate set via one of the four options above
2. We don't clean the zero lamport key, even though the ref count is 1, due to this check: https://github.com/solana-labs/solana/blob/master/runtime/src/accounts_db.rs#L1594-L1597
3. The slot for the zero lamport key is never shrunk again because it never goes under 80% capacity (Maybe the zero lamport account data is large or there are a lot of zero lamport keys in this slot that can't be cleaned for the same reason as 2) above, even if rent overwrites all the remaining non-zero pubkeys.

Summary of Changes

Remove zero lamport keys if ref_count is 1, even if the storage entry is not empty

Fixes #

[lijunwangs]

Why account_infos.len(). would be = 1?
I thought the issue could be the pub key for some 0 lamports are not returned at all in construct_candidate_clean_keys and hence we will not catch them either.

[sakridge]

We don't clean the zero lamport key, even though the ref count is 1, due to this check: https://github.com/solana-labs/solana/blob/master/runtime/src/accounts_db.rs#L1594-L1597

Remember that we already decremented the ref counts here. So if it was 1 originally, it should be 0 here.

[mergify]

Sorry but I didn't understand the command. Please consult the commands documentation 📚.

Hey, I reacted but my real name is @Mergifyio

[carllin]

This section where we depth-first search through all the affected pubkeys and their containing stores was introduced to fix this issue: #9431, by this PR: #9447. I don't quite recall the context here, but it seems unnecessary

[codecov]

Codecov Report

Merging #19420 (74957a8) into v1.7 (8ab358c) will increase coverage by 0.0%.
The diff coverage is 100.0%.

@@           Coverage Diff           @@
##             v1.7   #19420   +/-   ##
=======================================
  Coverage    82.3%    82.3%           
=======================================
  Files         436      436           
  Lines      123572   123549   -23     
=======================================
- Hits       101746   101728   -18     
+ Misses      21826    21821    -5     

[carllin]

Closed in favor of #19570 which will merge to master, as 1.7 has no urgency to be fixed currently