-
Notifications
You must be signed in to change notification settings - Fork 523
Allow all trackers committing into DB #3014
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow all trackers committing into DB #3014
Conversation
1f6e0d4 to
4ceb0c7
Compare
Codecov Report
@@ Coverage Diff @@
## master #3014 +/- ##
==========================================
+ Coverage 43.65% 43.66% +0.01%
==========================================
Files 391 391
Lines 86792 86860 +68
==========================================
+ Hits 37890 37930 +40
- Misses 42873 42895 +22
- Partials 6029 6035 +6
Continue to review full report at Codecov.
|
d294984 to
963e521
Compare
963e521 to
6341fd2
Compare
d3d06ff to
e21a220
Compare
8d3e969 to
1574db9
Compare
ledger/ledger.go
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
these two methods are used for testing only. I think it would be cleaner to contain these inside the trackerRegistry.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the primary reason of having these is account updates initialization:
// flush the account data
// TODO: figure out how to move it the upper level
au.ledger.scheduleCommit(blk.Round())
// wait for the writing to complete.
au.ledger.waitAccountsWriting()
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm ok leaving it here for now - but it's not the right place for that. It looks to me as a "hint" that it doesn't really belong here. This part would "roll-forward" the database to be at the correct round ( i.e. x -> latest-320 ).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, it is part of account updates initialization, I did not figure out a better way to accessing upper level from an tracker.
tsachiherman
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that the changes here are going to the right direction, but I'd suggest the following:
in trackerRegistry.commitRound, before calling prepareCommit on all the children, create an object called roundCommitContext.
have the roundCommitContext contains the offset, dbRound, lookback, plus all the temporary staging data we need while committing the current round.
Then, on each of the ledgerTracker(s), implement the following methods:
prepareCommit(*roundCommitContext)
commitRound(ctx, tx, *roundCommitContext)
postCommitRound(ctx)
and call all of them ( in all the cases ).
This pattern would allow you to use the go-interface more, and would allow us to break up the trackers even further ( i.e. the shared data can be used across the trackers, so we can have the merle trie hashes be in the same tracker as the catchpoint file generation.. )
* TODO: fix initializeCaches - it needs committing on init * TODO: fix tests
* Most likely this needs to be changed * Went easiest way to run and fix leftovers with existing tests
4843595 to
fc649c2
Compare
fc649c2 to
4459a02
Compare
a23acb8 to
3ccf5cd
Compare
tsachiherman
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good.
## Summary Historically only `account updates` tracker was committing into DB. The PR changes this: 1. `dbRound` management and `commitSyncer` business moved to trackerRegistry. 2. `account updates` is used as a driver for committing tasks scheduling - by given `commttedRound` and `dbRound` it figures out offset in detlas for saving to the disk. This is basically old `au.committedUpTo` method`. 3. `commitRound` replaced by `prepareCommit` that returns commit and post commit callbacks 4. trackerRegistry has own `commitRound` that calls trackers' `prepareCommit` and commit/post commit procs if any. 5. `account updates` still have a cached dbRound to be used in Lookup methods. ## Test Plan It is refactoring, use existing tests
Summary
Historically only
account updatestracker was committing into DB. The PR changes this:dbRoundmanagement andcommitSyncerbusiness moved to trackerRegistry.account updatesis used as a driver for committing tasks scheduling - by givencommttedRoundanddbRoundit figures out offset in detlas for saving to the disk. This is basically oldau.committedUpTomethod`.commitRoundreplaced byprepareCommitthat returns commit and post commit callbackscommitRoundthat calls trackers'prepareCommitand commit/post commit procs if any.account updatesstill have a cached dbRound to be used in Lookup methods.Test Plan
It is refactoring, use existing tests