Why

We can reduce memory, storage use, etc. by combining non-atree domain registers into 1 atree payload per account. We can further reduce execution state size by using atree register inlining feature added in Atree v0.8.0.

This improvement can be deployed via HCU and implements first on-the-fly migration to avoid causing further downtime.

The full impact on memory reduction (TBD) will be realized after all the accounts eventually migrate all their non-atree domain registers to a single combined atree payload per account.

NOTE: Memory reduction is only 1 benefit from reducing payload count.

How will we measure success ?

After all the non-atree domain registers are migrated to 1 atree map per account we expect:

• reduce execution state size by 10-12% (~40GB as of Sept. 2024 mainnet state)
• Less network bandwidth needed for chunk data packs (TBD)
• reduce payload count by 20-30% (by combining payloads)
• reduce mtrie nodes by 20-30% (by eliminating mtrie nodes)
• reduce EN memory use by TBD (usually better than state size reduction)

Requires HCU.
If we use on-the-fly migration, then the full impact won't be realized until all accounts are migrated.

Requires "cleanup" migration for accounts that were not migrated on-the-fly.

Estimate

~3dev/weeks to get ready for deployment.

DACI

Details

New domains were added in Cadence 1.0 and domain register count increased to 150 million (was 80 million on Sept. 4 pre-spork). Each 8-byte domain register also requires ~2 mtrie nodes (~2x96 byte overhead).

Nearly 25% of total payloads are 8-byte domain registers.

These 8-byte "domain registers" refers to 8-byte account domain payloads. They are not atree payloads.

As of Sept 10, 2024, mainnet state had:

• total payload count: 610 million
• atree payload count: 324,800,993
• domain register (aka payload) count: 150,127,970
• etc.

Suggestion

Currently, all 150+ million account domain registers are not using atree.

I think we should replace non-atree domain registers with 1 atree register (atree map) per account. Currently, there are 35 million accounts.

Also, we can use atree inlining to inline small domain data. So instead of reducing payload count from 150 million to 35 million, we can reduce payload count by 174 million.

DRAFT Estimates

Based on preliminary estimates using Sept. 17, 2024 mainnet state, I think we can:

• eliminate mtrie nodes: -425 million (-28.5%)
• reduce payload count: -174 million (-28.5%)
• reduce register reads (inlined domain data)
• reduce state size by ~40GB (roughly 10-12% of state size)
• reduce peak RAM use by TBD (usually better than state size)
• provide single point of entry for account data

We can reduce payload count by 28% (instead of being limited to 24%) due to atree inlining.

Misc

This issue replaces my older issue #2867.

EDIT: add DRAFT estimates and mention why we can reduce 174 million payload count if there are only 150 million domain registers.

Effort Estimation

2dev/weeks to wrap-up testing and open PR for review. Will need @turbolent to review.