Merge branch `refactor-v0.7.0-new` #962

bsbds · 2024-08-23T10:19:40Z

Merge branch refactor-v0.7.0-new into master.

Please briefly answer these questions:

what problem are you trying to solve? (or if there's no problem, what's the motivation for this change?)
what changes does this pull request make?
are there any non-obvious implications of these changes? (does it break compatibility with previous versions, etc)

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com> chore: add reminder of a revision issue Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

refactor: remove cmd worker and mpmc channel from task manager Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com> refactor: cmd trait Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com> WIP: fix rebase cmd worker refactor: change command execute from async to sync Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

refactor: curp server refactor: update auth wrapper with the new stream api refactor: log persistent refactor: add new tasks to task manager WIP: command execution chore: clippy task manager chore: clippy client tests mock client chore: client retry chore: clippy curp error pattern match chore: clippy client tests Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com> refactor: implement ResponseSender and ResponseReceiver chore: clippy response receiver Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com> chore: remove log persistent task from task manager Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com> fix: return immediately when receiving an error on propose Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com> refactor: command execution logic Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com> chore: fix clippy Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com> fix: use propose id for read only cmds Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

Request::set_timeout only works on the server side, if the server is not ready, the client may wait indefinitely Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

…tonic doesn't support it Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com> fix: test ce Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

This check is not necessary, the urls are allowed to be empty before publish Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

Because client id may change during retry, the propose id generation must be called for each retry Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

It seems sync wait on a event listener will potentialy cause a deadlock, we will address this in the future. Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

codecov · 2024-08-23T11:42:19Z

Codecov Report

Attention: Patch coverage is 80.09934% with 601 lines in your changes missing coverage. Please review.

Project coverage is 75.19%. Comparing base (e35b35a) to head (5843ee5).
Report is 194 commits behind head on master.

Files	Patch %	Lines
crates/xline/src/storage/kv_store.rs	72.61%	107 Missing and 42 partials ⚠️
crates/curp/src/server/raw_curp/mod.rs	85.61%	28 Missing and 12 partials ⚠️
crates/curp/src/server/curp_node.rs	85.09%	29 Missing and 9 partials ⚠️
crates/curp/src/server/storage/wal/storage.rs	76.58%	19 Missing and 18 partials ⚠️
crates/curp/src/client/tests.rs	75.20%	22 Missing and 8 partials ⚠️
crates/curp/src/server/cmd_worker/mod.rs	84.35%	20 Missing and 8 partials ⚠️
crates/xline/src/server/command.rs	88.42%	17 Missing and 5 partials ⚠️
crates/curp/src/client/unary.rs	84.29%	14 Missing and 5 partials ⚠️
crates/curp/src/response.rs	76.54%	10 Missing and 9 partials ⚠️
crates/curp/src/server/storage/wal/mock/mod.rs	43.75%	17 Missing and 1 partial ⚠️
... and 33 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #962      +/-   ##
==========================================
- Coverage   75.55%   75.19%   -0.37%     
==========================================
  Files         180      189       +9     
  Lines       26938    28376    +1438     
  Branches    26938    28376    +1438     
==========================================
+ Hits        20353    21337     +984     
- Misses       5366     5750     +384     
- Partials     1219     1289      +70

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

Phoenix500526

LGTM

mergify · 2024-08-26T04:37:18Z

@bsbds Convert your pr to draft since CI failed

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

Phoenix500526

LGTM

bsbds added 30 commits July 25, 2024 10:07

refactor: xline store

f9368cf

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

refactor: after sync command

8a38a5b

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com> chore: add reminder of a revision issue Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

refactor: revision fallback

4ab054d

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

refactor: execute in after sync

9b78057

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

refactor: add CE to CurpNode

d5ff7d8

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

chore(madsim): remove CE from curp server creation

11ea107

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

feat(wal): add mock implementation of WAL

93db8b6

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

chore(wal): move wal mod.rs to storage.rs

73e7f48

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

refactor(wal): add proxy type for WALStorage

2c574d8

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

refactor: curp storage api

e28817c

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

fix: recover wal in tests

b134b2d

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

chore: remove unecessary trait bound for WAL

3157abf

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

chore: add a comment to the recover behavior of the curp db

fe71e78

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

chore: resolve comments

c6b9be4

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

chore: fix clippy

60c3b2f

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

fix: add client side timeout to the rpc connect

b816b09

Request::set_timeout only works on the server side, if the server is not ready, the client may wait indefinitely Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

chore: change tonic::Result<T> to Result<T, tonic::Status> as madsim …

d9f3865

…tonic doesn't support it Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

chore(madsim): update madsim curp client api

890966c

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

chore: clippy raw curp

e2ea316

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

chore: clippy cmd worker

ba5dbab

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

chore: resolve comments

0e8ebb9

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

chore: move locking into remove_from_sp_ucp

e6a77bd

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

chore: correct comment location

b7c110e

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

chore: remove outdated todos

54726a7

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

fix: only remove from conflict pools in after sync stage

27e1789

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

refactor(xline, curp): switch after sync to sync

d5842da

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

refactor(xline, curp): after sync error returning mechanism

3cd7a50

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com> fix: test ce Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

chore: fix clippy

1b611dd

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

bsbds added 5 commits August 23, 2024 15:54

chore: use join_all to concurrently build clients in benchmark

c6d7d9b

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

fix: remove check_members

c4f1dcb

This check is not necessary, the urls are allowed to be empty before publish Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

fix: generate propose id inside client retry closure

313b819

Because client id may change during retry, the propose id generation must be called for each retry Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

refactor: use synchronous compaction in sync_compaction

2ba7ae1

It seems sync wait on a event listener will potentialy cause a deadlock, we will address this in the future. Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

chore: update cargo hakari

47a8900

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

bsbds force-pushed the merge-refactor-v0.7 branch from 60bd0d4 to 97f6232 Compare August 23, 2024 10:24

Merge branch 'fix-tests' into merge-refactor-v0.7

1157cbd

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

bsbds force-pushed the merge-refactor-v0.7 branch from 97f6232 to 1157cbd Compare August 23, 2024 11:29

chore: remove unused index from lease_store

7d0bc8f

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

bsbds marked this pull request as ready for review August 26, 2024 03:37

mergify bot requested a review from a team August 26, 2024 03:38

Phoenix500526 previously approved these changes Aug 26, 2024

View reviewed changes

mergify bot requested a review from a team August 26, 2024 04:12

bsbds added 2 commits August 26, 2024 12:25

Merge branch 'fix-tests' into merge-refactor-v0.7

cad274f

Merge remote-tracking branch 'upstream/master' into merge-refactor-v0.7

a31b2f2

bsbds dismissed Phoenix500526’s stale review via a31b2f2 August 26, 2024 04:29

mergify bot marked this pull request as draft August 26, 2024 04:37

mergify bot added the CI:fail CI has failed label Aug 26, 2024

bsbds force-pushed the merge-refactor-v0.7 branch from 74467e2 to 0079e9c Compare August 26, 2024 07:03

fix: potential panic in shutdown listener

6f909f5

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

bsbds force-pushed the merge-refactor-v0.7 branch from 0079e9c to 6fa7400 Compare August 26, 2024 07:07

fix: only return shutdown error on cluster shutdown

5843ee5

Signed-off-by: bsbds <69835502+bsbds@users.noreply.github.com>

bsbds force-pushed the merge-refactor-v0.7 branch from 6fa7400 to 5843ee5 Compare August 26, 2024 07:35

bsbds marked this pull request as ready for review August 26, 2024 08:24

mergify bot removed the CI:fail CI has failed label Aug 26, 2024

Phoenix500526 approved these changes Aug 26, 2024

View reviewed changes

mergify bot requested a review from a team August 26, 2024 08:31

Phoenix500526 merged commit d19df09 into xline-kv:master Aug 26, 2024
15 of 16 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge branch `refactor-v0.7.0-new` #962

Merge branch `refactor-v0.7.0-new` #962

bsbds commented Aug 23, 2024 •

edited

Loading

codecov bot commented Aug 23, 2024 •

edited

Loading

Phoenix500526 left a comment

mergify bot commented Aug 26, 2024

Phoenix500526 left a comment

Merge branch refactor-v0.7.0-new #962

Merge branch refactor-v0.7.0-new #962

Conversation

bsbds commented Aug 23, 2024 • edited Loading

codecov bot commented Aug 23, 2024 • edited Loading

Codecov Report

Phoenix500526 left a comment

Choose a reason for hiding this comment

mergify bot commented Aug 26, 2024

Phoenix500526 left a comment

Choose a reason for hiding this comment

Merge branch `refactor-v0.7.0-new` #962

Merge branch `refactor-v0.7.0-new` #962

bsbds commented Aug 23, 2024 •

edited

Loading

codecov bot commented Aug 23, 2024 •

edited

Loading