fix(core-snapshots): remove bogus skipRoundRows #2973
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
skipRoundRows was only set to something else than 0 when appending to an existent snapshot and then it was set to the number of rows in the rounds table in the appended-to snapshot. Then it would translate to an OFFSET in the SQL query: SELECT round, balance, public_key FROM rounds WHERE round BETWEEN 9804 AND 9902 ORDER BY round, balance DESC, public_key OFFSET 500004 but the proper rows to be selected are already defined by the BETWEEN clause. So, the usage of OFFSET is wrong and leads to bogus results - when appending to a snapshot the result would be a snapshot with missing rows from the rounds table, and subsequently when that snapshot is restored - gaps (missing rows) in the rounds table in the database.
vasild
requested review from
air1one,
faustbrian,
kristjank and
spkjp
as code owners
Codecov Report
@@ Coverage Diff @@
## develop #2973 +/- ##
===========================================
+ Coverage 68.22% 68.23% +<.01%
===========================================
Files 419 419
Lines 10012 10011 -1
Branches 508 508
===========================================
Hits 6831 6831
+ Misses 3141 3140 -1
Partials 40 40
Continue to review full report at Codecov.
|
When appending to a snapshot, in 50 of 51 cases it will happen that the last round from the appended-to snapshot is the same as the first round in the being-appended snapshot. Dont store that twice in the snapshot because then it cannot be restored due to duplicate key error from the database.
spkjp
approved these changes
Sep 26, 2019
ghost
deleted the
vasild
added a commit
that referenced
this pull request
Sep 27, 2019
…ts-nonce * ArkEcosystem/core/develop: refactor(core-transaction-pool): don't accept expired v1 transactions (#2948) fix(core-snapshots): remove bogus skipRoundRows (#2973) feat(core-api): endpoints for locks/businesses/bridgechains (#2940) fix(core-blockchain): round deletion during rollback (#2970) fix: range selection in pool's getTransactions() (#2952) feat: expose `isValidPeer` via ajv format rule (#2960) chore: remove trailing whitespace (#2971) fix(core-transactions): update wallet nonce when applying v1 transaction (#2959) fix(core-blockchain): do not reset `noBlockCounter` when `downloadBlocks` succeeds (#2968) refactor(pool): remove unnecessary check from validateTransactions() (#2951) ci: temporarily disable pull_request.synchronize event (#2966) refactor: strengthen a nonce check in performGenericWalletChecks() (#2949) ci: setup github action workflow for e2e tests (#2964) chore: drop node 11 support (#2965) test(e2e): No need to run as root the chmod command when modifying own's files (#2958) refactor(crypto): change maximum recipients of multipayment via milestone (#2961) ci: split functional job into 1 job per type (#2963) refactor(core-p2p): make peer reply errors less verbose (#2962) fix(core-state): index recipient wallets during bootstrap (#2947) Tell git to ignore vim's temporary files (#2957) ci: force exit tests if there are hanging promises ci: use different test coverage directories for each type (#2956)
ghost
mentioned this pull request
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
skipRoundRows was only set to something else than 0 when appending to an
existent snapshot and then it was set to the number of rows in the
rounds table in the appended-to snapshot. Then it would translate to an
OFFSET in the SQL query:
SELECT round, balance, public_key
FROM rounds
WHERE round BETWEEN 9804 AND 9902
ORDER BY round, balance DESC, public_key
OFFSET 500004
but the proper rows to be selected are already defined by the BETWEEN
clause.
So, the usage of OFFSET is wrong and leads to bogus results - when
appending to a snapshot the result would be a snapshot with missing rows
from the rounds table, and subsequently when that snapshot is restored -
gaps (missing rows) in the rounds table in the database.
Summary
Checklist