Increase incremental root threshold by catching up in pieces

[Rjected]

Describe the feature

Currently the incremental root stage has a low threshold, because it accumulates lots of data in memory and cannot process large ranges at once. When the number of blocks we need to catch up exceeds this threshold, we do a clean merkle root calculation, which is often slower than if we had run the incremental stage.

The incremental stage branch is here

reth/crates/stages/stages/src/stages/merkle.rs

Lines 253 to 266 in 5c64d59

	} else {
	debug!(target: "sync::stages::merkle::exec", current = ?current_block_number, target = ?to_block, "Updating trie");
	let (root, updates) =
	StateRoot::incremental_root_with_updates(provider.tx_ref(), range)
	.map_err(|e| {
	error!(target: "sync::stages::merkle", %e, ?current_block_number, ?to_block, "Incremental state root failed! {INVALID_STATE_ROOT_ERROR_MESSAGE}");
	StageError::Fatal(Box::new(e))
	})?;
	provider.write_trie_updates(&updates)?;
	let total_hashed_entries = (provider.count_entries::<tables::HashedAccounts>()? +
	provider.count_entries::<tables::HashedStorages>()?)
	as u64;

We could instead do this in a loop with a fixed range each time, until we catch up. This way we could avoid doing the clean merkle stage

Additional context

No response

[Ayushdubey86]

can i look into this?

[github-actions]

This issue is stale because it has been open for 21 days with no activity.