4Tb assert

[AskAlexSharov]

Hi. Looks like at 4Tb threshold mdbx getting next assert:
Assertion failed: ((pl)[1]) < limit (mdbx: mdbx_pnl_check: 6368)

[erthink]

Could you provide the coredump, or at least a stack backtrace?

[AskAlexSharov]

coredump will come tomorrow

[flywukong]

Hi, I test it on bsc-erigon and get a error when the mdbx.dat. file reach 4T , the errors looks like blow

then I notice that you have fix the issue and merge the commit into devel branch , so I have updated the go package into devel branch , and then recompile erigon and restart it (continue syncing), but I still get errors below .

you can get log here ,https://transfer.toolsfdg.net/ySKaN/nohup.out, I wonder if the issue is completely repaired

[erthink]

@flywukong, this issue not fixed for now, but I made some changes to dig it.

1. The line number for this assertion is differ for current code. Please use current the devel or issue-260 branch for your test(s).
2. As I noted abote the coredump or at least a stack backtrace is recuired, if problem still.
3. The link you provided to the log is inaccessible.

[flywukong]

@erhink thanks for your reply , I seems have already changed to package to devel by run command like" go get github.com/erthink/libmdbx@devel", the go.mod file changed and the id is the latest commit id in devel branch.

but the log shows that it is not devel branch ? I will try to use go replace to update package intead of this way.
you can see log here
log.zip

[AskAlexSharov]

@flywukong please don’t be confused - erigon and mdbx are not related projects. mdbx is C language project and has no go.mod (can’t “go get” it).

There are several steps to get another version of mdbx into erigon (if you need another version of erigon - better ask about it in erigon’s channel/repo).

There is same name branch in erigon “issue-260” - with right version of mdbx. What need to do now - run it on existing db and get core dump. Such core dump can be attached here.

[flywukong]

@AskAlexSharov thanks for your advice, so I think ignore the level of mdbx-go , if this part have not been fixed , the erigon can not work well . But I was testing on bsc branch of erigon , so I wonder I can not just use “issue-260” of erigon， may be I can merge the change of this branch to fix the problem

[AskAlexSharov]

@AskAlexSharov thanks for your advice, so I think ignore the level of mdbx-go , if this part have not been fixed , the erigon can not work well . But I was testing on bsc branch of erigon , so I wonder I can not just use “issue-260” of erigon， may be I can merge the change of this branch to fix the problem

It’s not fix yet, it’s debug branch to get coredump - which will help us understand root cause and fix.

[flywukong]

@AskAlexSharov thanks , I am not sure if this commit have solved the problem , 1813bf9 , it is merged into devel branch , I think may be we can update mdbx-go code to downloaded and called this branch for testing . If problem is solved in test, we just need to wait for this commit to be merged into master of libmdbx. Syncing 4T data from scratch get coredump would take too much time , this way may be faster

[erthink]

@flywukong, AFAIK erigon uses DB with default 4K pages.
If so, then the mentioned commit is relevant to the issues, but does not fix it.
Because with 4K page size, such arithmetic overflow will happen when the size is 8T, not 4T.
For overflow at 4T the 2K sized page should be used.

So the output of mdbx_chk -vv for your 4T DB will help to understand situation, and if we see the 2K sized pages then we could assume that issue is fixed.

[flywukong]

@erthink thanks , the pagesize in our DB is also 4K

by the way, our bug doesn't seem to be triggered by 4T. When it reachded 4T, it was an error indicating that the mapsize was not enough（the mapsize we configured is 4T）. When I tried to adjust the mapsize, I kept restarting erigon, and then triggered the core

[AskAlexSharov]

@flywukong on which Erigon's branch? if on "bsc" or "devel" - try "issue-260" branch. If you still see crush - please send us coredump. thank.