-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(dgraph): ludicrous mode mutation error #5701
Conversation
…el/lud-mode-tests
This reverts commit 0502962.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand everything but based on https://discuss.dgraph.io/t/ludicrous-mode-fails-queries-on-21-million-dataset/7320 and the explanation @harshil-goel gave me, this change looks good to me.
Reviewed 1 of 5 files at r2.
Reviewable status: 1 of 5 files reviewed, 4 unresolved discussions (waiting on @harshil-goel, @manishrjain, and @vvbalaji-dgraph)
systest/21million/test-21million.sh, line 61 at r2 (raw file):
--mode normal = run dgraph in normal mode none = run dgraph in normal mode ludicrous = run dgraph in ludicrous mode
Indentation.
worker/draft.go, line 646 at r2 (raw file):
totalSize += int64(psz) if x.WorkerConfig.LudicrousMode && proposal.Mutations != nil && proposal.Mutations.StartTs == 0 {
If we're not assigning startTs
to mutations, can they still have a non-zero startTs? If not, I'd convert this proposal.Mutations.StartTs == 0
into an assert inside the if
worker/draft.go, line 1164 at r2 (raw file):
span.Annotate(nil, "Proposal found in CommittedEntries") } if x.WorkerConfig.LudicrousMode && len(proposal.Mutations.GetEdges()) > 0 {
I believe this change is not related to this PR.
worker/task.go, line 2185 at r2 (raw file):
for itr.Seek(countKey); itr.Valid(); itr.Next() { item := itr.Item() key := append(make([]byte, 0), item.Key()...)
I believe this change is not related to this PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 1 of 5 files reviewed, 4 unresolved discussions (waiting on @jarifibrahim, @manishrjain, and @vvbalaji-dgraph)
systest/21million/test-21million.sh, line 61 at r2 (raw file):
Previously, jarifibrahim (Ibrahim Jarif) wrote…
Indentation.
Done.
worker/draft.go, line 646 at r2 (raw file):
Previously, jarifibrahim (Ibrahim Jarif) wrote…
If we're not assigning
startTs
to mutations, can they still have a non-zero startTs? If not, I'd convert thisproposal.Mutations.StartTs == 0
into an assert inside theif
ALTER Operations already have a timestamp. do end up coming with startTs. The same check == 0 is applied when we assign a startTs to a mutation.
worker/draft.go, line 1164 at r2 (raw file):
Previously, jarifibrahim (Ibrahim Jarif) wrote…
I believe this change is not related to this PR.
I needed the change to run the tests. I will get merge those other PRs first.
worker/task.go, line 2185 at r2 (raw file):
Previously, jarifibrahim (Ibrahim Jarif) wrote…
I believe this change is not related to this PR.
I needed the change to run the tests. I will get merge those other PRs first.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 4 of 5 files at r2, 1 of 1 files at r3.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @jarifibrahim and @vvbalaji-dgraph)
worker/draft.go, line 646 at r2 (raw file):
Previously, harshil-goel (Harshil Goel) wrote…
ALTER Operations already have a timestamp. do end up coming with startTs. The same check == 0 is applied when we assign a startTs to a mutation.
100 chars. Seems like a fine thing to do.
Fixes out of order commit in ludicrous mode. (cherry picked from commit 2a8c17f)
Fixes out of order commit in ludicrous mode. (cherry picked from commit 2a8c17f)
Fixes out of order commit in ludicrous mode.
Fixes out of order commit in ludicrous mode.
https://discuss.dgraph.io/t/ludicrous-mode-fails-queries-on-21-million-dataset/7320/2
Time taken to live load 21 million dataset in ludicrous mode went down from 5:11 min on average to 3:01 min on average.
This change is
Docs Preview: