Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core/rawdb: Enforce readonly in freezer instantiation #24119

Merged
merged 16 commits into from
Jan 18, 2022

Conversation

s1na
Copy link
Contributor

@s1na s1na commented Dec 16, 2021

When opening the freezer in readonly mode (e.g. db inspect command) we still perform repair, possibly modifying the underlying files. This PR changes that and only raises an error instead when freezer is in an invalid state.

Copy link
Contributor

@holiman holiman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you also should take care of the case where someone tries to actually write to the freezer. This can be done in several ways -- all write happens via the batch, afaik, so you could centralize the check there. Perhaps make the singular f.writeBatch be nil if readonly, or set the batch to non-nil but "refusing to write".

@holiman
Copy link
Contributor

holiman commented Dec 17, 2021

Hm, my comment was a bit misplaced. So the freezer already checks the readonly, and the individual table doesn't have any public method to append items (any more).
However, we have the freezerTableBatch, which has an Append method where we could have a corresponding check.

@holiman
Copy link
Contributor

holiman commented Jan 10, 2022

However, we have the freezerTableBatch, which has an Append method where we could have a corresponding check.

This is not implemented, and I added a test to check how it would behave. What happens is that the AppendRaw goes ahead, but at some point, when the commi is called (either explicitly or due to the append causing a new table to be opened), it will try to append to the index file, and find that the file descriptor is bad (probably wrong mode).

I think that's an ok scenario, The caller can write to a batch, but it will eventually fail, and there should be no side-effects, afaict. So I'm good.

Copy link
Contributor

@holiman holiman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

var length uint64
var name string
// Hack to get length of any table
for k, table := range f.tables {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can merge these two loops into single one, to have less code.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other nit, please rename k to something a bit better, lest go with kind without a better idea.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can merge these two loops into single one, to have less code.

I understand that would be shorted in terms of lines, but I find it more readable to have them logically separate. If you don't have a strong feeling about it I'd like to keep it as is.

core/rawdb/freezer.go Outdated Show resolved Hide resolved
@s1na
Copy link
Contributor Author

s1na commented Jan 11, 2022

So I found out db inspect still truncates an inconsistent db. That's because the freezer is opened twice, and the first time it's opened in write-mode via makeConfigNode->setEthConfig->MakeChainDatabase.

@s1na
Copy link
Contributor Author

s1na commented Jan 11, 2022

After fixing that issue the logs for db inspect on an inconsistent db look like this:

❯ ./build/bin/geth --dev --datadir dev-datadir-legacy db inspect
INFO [01-11|19:57:55.086] Maximum peer count                       ETH=50 LES=0 total=50
INFO [01-11|19:57:55.087] Set global gas cap                       cap=50,000,000
INFO [01-11|19:57:55.151] Using developer account                  address=0xe0AD84EE1bC06DCb400235A3006997e262A3106F
INFO [01-11|19:57:55.151] Allocated cache and file handles         database=~/data/geth/chaindata cache=512.00MiB handles=5120 readonly=true
WARN [01-11|19:57:55.158] Truncating dangling indexes              database=~/data/geth/chaindata/ancient table=receipts indexed=1.19MiB stored=1.19MiB
Fatal: Could not open database: truncate ~/data/geth/chaindata/ancient/receipts.cidx: invalid argument

Not super clean IMHO. I'm tilting slightly towards adding back the extra checks.

@holiman holiman added this to the 1.10.16 milestone Jan 17, 2022
Copy link
Member

@karalabe karalabe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM

@karalabe karalabe merged commit 4aab440 into ethereum:master Jan 18, 2022
sidhujag pushed a commit to syscoin/go-ethereum that referenced this pull request Jan 18, 2022
* freezer: add readonly flag to table

* freezer: enforce readonly in table repair

* freezer: enforce readonly in newFreezer

* minor fix

* minor

* core/rawdb: test that writing during readonly fails

* rm unused log

* check readonly on batch append

* minor

* Revert "check readonly on batch append"

This reverts commit 2ddb5ec.

* review fixes

* minor test refactor

* attempt at fixing windows issue

* add comment re windows sync issue

* k->kind

* open readonly db for genesis check

Co-authored-by: Martin Holst Swende <martin@swende.se>
@s1na s1na deleted the freezer/readonly branch January 20, 2022 11:32
JacekGlen pushed a commit to JacekGlen/go-ethereum that referenced this pull request May 26, 2022
* freezer: add readonly flag to table

* freezer: enforce readonly in table repair

* freezer: enforce readonly in newFreezer

* minor fix

* minor

* core/rawdb: test that writing during readonly fails

* rm unused log

* check readonly on batch append

* minor

* Revert "check readonly on batch append"

This reverts commit 2ddb5ec.

* review fixes

* minor test refactor

* attempt at fixing windows issue

* add comment re windows sync issue

* k->kind

* open readonly db for genesis check

Co-authored-by: Martin Holst Swende <martin@swende.se>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants