building index during file merge fails to find not yet built file

[awskii]

so file lookup has been done within dirty files and i am not sure what should be the current logic of the merge

WARN[04-24|02:33:36.221] [agg] lookupDirtyFileByItsRange: file not found stepFrom=192 stepTo=200 files=0-128;128-192;192-196;196-198;198-199;199-200; filesCount=6
WARN[04-24|02:33:36.221] [snapshots] state merge failed err=build storage idx: loadIntoTable : stopped accounts(val=192-200, hist=192-200, idx=192-200), storage(val=192-200, hist=192-200, idx=192-200), code(val=1
92-200, hist=192-200, idx=192-200), commitment(val=192-200), receipt(val=192-200, hist=192-200, idx=192-200), idx0=LogAddrIdx192-200, idx1=LogTopicIdx192-200, idx2=TracesFromIdx192-200, idx3=TracesToIdx192-200
INFO[04-24|02:33:36.874] total                                    took=57h47m20.902559587s

Within files, only domain having merged range 192-200 is accounts:

-rw-rw-r-- 1 erigon  17M Apr 23 03:07 /erigon-data/arb-sep/snapshots/domain/v1-accounts.0-128.bt
-rw-rw-r-- 1 erigon 842M Apr 23 03:07 /erigon-data/arb-sep/snapshots/domain/v1-accounts.0-128.kv
-rw-rw-r-- 1 erigon  18M Apr 23 03:07 /erigon-data/arb-sep/snapshots/domain/v1-accounts.0-128.kvei
-rw-rw-r-- 1 erigon 829K Apr 24 00:16 /erigon-data/arb-sep/snapshots/domain/v1-accounts.128-192.bt
-rw-rw-r-- 1 erigon  30M Apr 24 00:16 /erigon-data/arb-sep/snapshots/domain/v1-accounts.128-192.kv
-rw-rw-r-- 1 erigon 964K Apr 24 00:16 /erigon-data/arb-sep/snapshots/domain/v1-accounts.128-192.kvei
-rw-rw-r-- 1 erigon  76K Apr 24 01:18 /erigon-data/arb-sep/snapshots/domain/v1-accounts.192-196.bt
-rw-rw-r-- 1 erigon 2.8M Apr 24 01:18 /erigon-data/arb-sep/snapshots/domain/v1-accounts.192-196.kv
-rw-rw-r-- 1 erigon  88K Apr 24 01:18 /erigon-data/arb-sep/snapshots/domain/v1-accounts.192-196.kvei
-rw-rw-r-- 1 erigon 116K Apr 24 02:31 /erigon-data/arb-sep/snapshots/domain/v1-accounts.192-200.bt
-rw-rw-r-- 1 erigon 4.2M Apr 24 02:31 /erigon-data/arb-sep/snapshots/domain/v1-accounts.192-200.kv
-rw-rw-r-- 1 erigon 135K Apr 24 02:31 /erigon-data/arb-sep/snapshots/domain/v1-accounts.192-200.kvei
-rw-rw-r-- 1 erigon  56K Apr 24 01:56 /erigon-data/arb-sep/snapshots/domain/v1-accounts.196-198.bt
-rw-rw-r-- 1 erigon 2.0M Apr 24 01:56 /erigon-data/arb-sep/snapshots/domain/v1-accounts.196-198.kv
-rw-rw-r-- 1 erigon  65K Apr 24 01:56 /erigon-data/arb-sep/snapshots/domain/v1-accounts.196-198.kvei
-rw-rw-r-- 1 erigon  35K Apr 24 02:12 /erigon-data/arb-sep/snapshots/domain/v1-accounts.198-199.bt
-rw-rw-r-- 1 erigon 1.2M Apr 24 02:12 /erigon-data/arb-sep/snapshots/domain/v1-accounts.198-199.kv
-rw-rw-r-- 1 erigon  40K Apr 24 02:12 /erigon-data/arb-sep/snapshots/domain/v1-accounts.198-199.kvei
-rw-rw-r-- 1 erigon  36K Apr 24 02:28 /erigon-data/arb-sep/snapshots/domain/v1-accounts.199-200.bt
-rw-rw-r-- 1 erigon 1.3M Apr 24 02:28 /erigon-data/arb-sep/snapshots/domain/v1-accounts.199-200.kv
-rw-rw-r-- 1 erigon  41K Apr 24 02:28 /erigon-data/arb-sep/snapshots/domain/v1-accounts.199-200.kvei
-rw-rw-r-- 1 erigon  16M Apr 23 04:08 /erigon-data/arb-sep/snapshots/domain/v1-code.0-128.bt
-rw-rw-r-- 1 erigon 1.4G Apr 23 04:08 /erigon-data/arb-sep/snapshots/domain/v1-code.0-128.kv
-rw-rw-r-- 1 erigon  16M Apr 23 04:08 /erigon-data/arb-sep/snapshots/domain/v1-code.0-128.kvei
-rw-rw-r-- 1 erigon 197K Apr 24 00:47 /erigon-data/arb-sep/snapshots/domain/v1-code.128-192.bt
-rw-rw-r-- 1 erigon 326M Apr 24 00:47 /erigon-data/arb-sep/snapshots/domain/v1-code.128-192.kv
-rw-rw-r-- 1 erigon 130K Apr 24 00:47 /erigon-data/arb-sep/snapshots/domain/v1-code.128-192.kvei
-rw-rw-r-- 1 erigon  13K Apr 24 01:20 /erigon-data/arb-sep/snapshots/domain/v1-code.192-196.bt
-rw-rw-r-- 1 erigon  30M Apr 24 01:20 /erigon-data/arb-sep/snapshots/domain/v1-code.192-196.kv
-rw-rw-r-- 1 erigon 8.1K Apr 24 01:20 /erigon-data/arb-sep/snapshots/domain/v1-code.192-196.kvei
-rw-rw-r-- 1 erigon 6.3K Apr 24 01:57 /erigon-data/arb-sep/snapshots/domain/v1-code.196-198.bt
-rw-rw-r-- 1 erigon  21M Apr 24 01:57 /erigon-data/arb-sep/snapshots/domain/v1-code.196-198.kv
-rw-rw-r-- 1 erigon 3.9K Apr 24 01:57 /erigon-data/arb-sep/snapshots/domain/v1-code.196-198.kvei
-rw-rw-r-- 1 erigon  692 Apr 24 02:12 /erigon-data/arb-sep/snapshots/domain/v1-code.198-199.bt
-rw-rw-r-- 1 erigon 1.1M Apr 24 02:12 /erigon-data/arb-sep/snapshots/domain/v1-code.198-199.kv
-rw-rw-r-- 1 erigon  516 Apr 24 02:12 /erigon-data/arb-sep/snapshots/domain/v1-code.198-199.kvei
-rw-rw-r-- 1 erigon 2.3K Apr 24 02:28 /erigon-data/arb-sep/snapshots/domain/v1-code.199-200.bt
-rw-rw-r-- 1 erigon 6.2M Apr 24 02:28 /erigon-data/arb-sep/snapshots/domain/v1-code.199-200.kv
-rw-rw-r-- 1 erigon 1.5K Apr 24 02:28 /erigon-data/arb-sep/snapshots/domain/v1-code.199-200.kvei
-rw-rw-r-- 1 erigon  11G Apr 23 04:18 /erigon-data/arb-sep/snapshots/domain/v1-commitment.0-128.kv
-rw-rw-r-- 1 erigon 431M Apr 23 04:22 /erigon-data/arb-sep/snapshots/domain/v1-commitment.0-128.kvi
-rw-rw-r-- 1 erigon 2.8G Apr 24 00:48 /erigon-data/arb-sep/snapshots/domain/v1-commitment.128-192.kv
-rw-rw-r-- 1 erigon  54M Apr 24 00:49 /erigon-data/arb-sep/snapshots/domain/v1-commitment.128-192.kvi
-rw-rw-r-- 1 erigon 718M Apr 24 01:20 /erigon-data/arb-sep/snapshots/domain/v1-commitment.192-196.kv
-rw-rw-r-- 1 erigon 9.3M Apr 24 01:20 /erigon-data/arb-sep/snapshots/domain/v1-commitment.192-196.kvi
-rw-rw-r-- 1 erigon 478M Apr 24 01:57 /erigon-data/arb-sep/snapshots/domain/v1-commitment.196-198.kv
-rw-rw-r-- 1 erigon 5.8M Apr 24 01:57 /erigon-data/arb-sep/snapshots/domain/v1-commitment.196-198.kvi
-rw-rw-r-- 1 erigon 270M Apr 24 02:12 /erigon-data/arb-sep/snapshots/domain/v1-commitment.198-199.kv
-rw-rw-r-- 1 erigon 3.2M Apr 24 02:12 /erigon-data/arb-sep/snapshots/domain/v1-commitment.198-199.kvi
-rw-rw-r-- 1 erigon 300M Apr 24 02:28 /erigon-data/arb-sep/snapshots/domain/v1-commitment.199-200.kv
-rw-rw-r-- 1 erigon 3.6M Apr 24 02:28 /erigon-data/arb-sep/snapshots/domain/v1-commitment.199-200.kvi
-rw-rw-r-- 1 erigon   83 Apr 23 04:28 /erigon-data/arb-sep/snapshots/domain/v1-receipt.0-128.bt
-rw-rw-r-- 1 erigon   53 Apr 23 04:28 /erigon-data/arb-sep/snapshots/domain/v1-receipt.0-128.kv
-rw-rw-r-- 1 erigon  116 Apr 23 04:28 /erigon-data/arb-sep/snapshots/domain/v1-receipt.0-128.kvei
-rw-rw-r-- 1 erigon   83 Apr 24 00:55 /erigon-data/arb-sep/snapshots/domain/v1-receipt.128-192.bt
-rw-rw-r-- 1 erigon   48 Apr 24 00:55 /erigon-data/arb-sep/snapshots/domain/v1-receipt.128-192.kv
-rw-rw-r-- 1 erigon  116 Apr 24 00:55 /erigon-data/arb-sep/snapshots/domain/v1-receipt.128-192.kvei
-rw-rw-r-- 1 erigon   83 Apr 24 01:21 /erigon-data/arb-sep/snapshots/domain/v1-receipt.192-196.bt
-rw-rw-r-- 1 erigon   48 Apr 24 01:21 /erigon-data/arb-sep/snapshots/domain/v1-receipt.192-196.kv
-rw-rw-r-- 1 erigon  116 Apr 24 01:21 /erigon-data/arb-sep/snapshots/domain/v1-receipt.192-196.kvei
-rw-rw-r-- 1 erigon   83 Apr 24 01:57 /erigon-data/arb-sep/snapshots/domain/v1-receipt.196-198.bt
-rw-rw-r-- 1 erigon   44 Apr 24 01:57 /erigon-data/arb-sep/snapshots/domain/v1-receipt.196-198.kv
-rw-rw-r-- 1 erigon  116 Apr 24 01:57 /erigon-data/arb-sep/snapshots/domain/v1-receipt.196-198.kvei
-rw-rw-r-- 1 erigon   83 Apr 24 02:12 /erigon-data/arb-sep/snapshots/domain/v1-receipt.198-199.bt
-rw-rw-r-- 1 erigon   48 Apr 24 02:12 /erigon-data/arb-sep/snapshots/domain/v1-receipt.198-199.kv
-rw-rw-r-- 1 erigon  116 Apr 24 02:12 /erigon-data/arb-sep/snapshots/domain/v1-receipt.198-199.kvei
-rw-rw-r-- 1 erigon   83 Apr 24 02:28 /erigon-data/arb-sep/snapshots/domain/v1-receipt.199-200.bt
-rw-rw-r-- 1 erigon   48 Apr 24 02:28 /erigon-data/arb-sep/snapshots/domain/v1-receipt.199-200.kv
-rw-rw-r-- 1 erigon  116 Apr 24 02:28 /erigon-data/arb-sep/snapshots/domain/v1-receipt.199-200.kvei
-rw-rw-r-- 1 erigon 195M Apr 23 03:56 /erigon-data/arb-sep/snapshots/domain/v1-storage.0-128.bt
-rw-rw-r-- 1 erigon 7.6G Apr 23 03:55 /erigon-data/arb-sep/snapshots/domain/v1-storage.0-128.kv
-rw-rw-r-- 1 erigon 195M Apr 23 03:56 /erigon-data/arb-sep/snapshots/domain/v1-storage.0-128.kvei
-rw-rw-r-- 1 erigon  24M Apr 24 00:47 /erigon-data/arb-sep/snapshots/domain/v1-storage.128-192.bt
-rw-rw-r-- 1 erigon 982M Apr 24 00:46 /erigon-data/arb-sep/snapshots/domain/v1-storage.128-192.kv
-rw-rw-r-- 1 erigon  24M Apr 24 00:47 /erigon-data/arb-sep/snapshots/domain/v1-storage.128-192.kvei
-rw-rw-r-- 1 erigon 1.8M Apr 24 01:20 /erigon-data/arb-sep/snapshots/domain/v1-storage.192-196.bt
-rw-rw-r-- 1 erigon  87M Apr 24 01:20 /erigon-data/arb-sep/snapshots/domain/v1-storage.192-196.kv
-rw-rw-r-- 1 erigon 1.7M Apr 24 01:20 /erigon-data/arb-sep/snapshots/domain/v1-storage.192-196.kvei
-rw-rw-r-- 1 erigon 892K Apr 24 01:57 /erigon-data/arb-sep/snapshots/domain/v1-storage.196-198.bt
-rw-rw-r-- 1 erigon  45M Apr 24 01:57 /erigon-data/arb-sep/snapshots/domain/v1-storage.196-198.kv
-rw-rw-r-- 1 erigon 849K Apr 24 01:57 /erigon-data/arb-sep/snapshots/domain/v1-storage.196-198.kvei
-rw-rw-r-- 1 erigon 389K Apr 24 02:12 /erigon-data/arb-sep/snapshots/domain/v1-storage.198-199.bt
-rw-rw-r-- 1 erigon  20M Apr 24 02:12 /erigon-data/arb-sep/snapshots/domain/v1-storage.198-199.kv
-rw-rw-r-- 1 erigon 371K Apr 24 02:12 /erigon-data/arb-sep/snapshots/domain/v1-storage.198-199.kvei
-rw-rw-r-- 1 erigon 502K Apr 24 02:28 /erigon-data/arb-sep/snapshots/domain/v1-storage.199-200.bt
-rw-rw-r-- 1 erigon  25M Apr 24 02:28 /erigon-data/arb-sep/snapshots/domain/v1-storage.199-200.kv
-rw-rw-r-- 1 erigon 478K Apr 24 02:28 /erigon-data/arb-sep/snapshots/domain/v1-storage.199-200.kvei

which makes me think that some race has been occured

this happened after merging main of 7612f95

[VBulikov]

VBulikov added Imp1 as Importance