Skip to content

Enable Fully Concurrent Snapshot Operations #56911

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
147 commits
Select commit Hold shift + click to select a range
346f306
Fully Concurrent Snapshots
original-brownbear Jun 15, 2020
0fffc10
cleanup todo
original-brownbear Jun 15, 2020
26e4bce
docs
original-brownbear Jun 15, 2020
7076a0f
cleanup
original-brownbear Jun 15, 2020
7cd1c4e
doc
original-brownbear Jun 15, 2020
a9b894c
reduce noise
original-brownbear Jun 15, 2020
98516c6
less noise
original-brownbear Jun 15, 2020
37d439f
fix test
original-brownbear Jun 15, 2020
8d10c4c
cover primary fail-over
original-brownbear Jun 15, 2020
ca110ec
drier tests
original-brownbear Jun 15, 2020
75a4aad
fix test
original-brownbear Jun 15, 2020
37e7994
update comment
original-brownbear Jun 15, 2020
8a5eed5
drier
original-brownbear Jun 15, 2020
457c443
make things nicer looking
original-brownbear Jun 15, 2020
66c6428
drop pointless short-circuit
original-brownbear Jun 15, 2020
72dbbcf
doc
original-brownbear Jun 15, 2020
fc267dc
moar test
original-brownbear Jun 15, 2020
fa5176f
another corner case fixed
original-brownbear Jun 15, 2020
c976fce
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 15, 2020
6cb5989
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 15, 2020
075a0b7
more docs
original-brownbear Jun 15, 2020
e27ec41
bck
original-brownbear Jun 15, 2020
7117707
save some more repo data loading
original-brownbear Jun 16, 2020
cd142a9
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 16, 2020
34f9345
bck
original-brownbear Jun 16, 2020
1cdab75
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 16, 2020
de9a3f7
moar corner cases
original-brownbear Jun 16, 2020
166e639
cleanup tests more
original-brownbear Jun 16, 2020
cadad70
cleanups
original-brownbear Jun 16, 2020
0e726a0
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 16, 2020
05d8718
be smarter about failover handling
original-brownbear Jun 16, 2020
cf39da6
fewer CS updates
original-brownbear Jun 16, 2020
7762343
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 16, 2020
e90de9d
nicer
original-brownbear Jun 16, 2020
e69d0b7
bck
original-brownbear Jun 16, 2020
2abeda2
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 16, 2020
c44bdb3
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 17, 2020
5cc68d4
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 17, 2020
72d12f7
reproducer
original-brownbear Jun 17, 2020
ee88275
fix waiting+relocation failure propagation
original-brownbear Jun 17, 2020
62559a5
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 17, 2020
3188225
docs fixes
original-brownbear Jun 17, 2020
d28bddb
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 17, 2020
7f961f5
drop dead code
original-brownbear Jun 17, 2020
a7aa0bf
stop double wrapping list
original-brownbear Jun 17, 2020
4ba1f61
rename
original-brownbear Jun 17, 2020
c0ff7dd
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 17, 2020
a9d7a20
further cleanups
original-brownbear Jun 18, 2020
2f98b63
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 18, 2020
38090b7
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 19, 2020
e766028
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 19, 2020
962a761
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 19, 2020
8200eba
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 19, 2020
a3c5196
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 20, 2020
10b504b
much cuter
original-brownbear Jun 20, 2020
ea4f9ed
fix
original-brownbear Jun 21, 2020
6ab07d9
moar docs, cleaner logic
original-brownbear Jun 21, 2020
01a3de8
tweaks
original-brownbear Jun 21, 2020
685d545
fix
original-brownbear Jun 21, 2020
5a72c28
test cleanups
original-brownbear Jun 22, 2020
337643a
shorter
original-brownbear Jun 22, 2020
40abfdd
much more readable tests
original-brownbear Jun 22, 2020
ccfa36e
much much stricter tests and fixes
original-brownbear Jun 22, 2020
dc9efc2
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 23, 2020
8bf6f04
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 23, 2020
b566d48
cleaner
original-brownbear Jun 23, 2020
58981c2
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 23, 2020
6d31169
even moar docs
original-brownbear Jun 23, 2020
f207d15
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 23, 2020
cd21994
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 23, 2020
e3acf79
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 23, 2020
1581aa8
test for and fix more listener leaks
original-brownbear Jun 23, 2020
808d827
better doc + naming
original-brownbear Jun 23, 2020
e19d0c6
optimize
original-brownbear Jun 23, 2020
fa822de
writable
original-brownbear Jun 23, 2020
fa1006b
faster
original-brownbear Jun 23, 2020
a118af0
different exceptions
original-brownbear Jun 23, 2020
6d221d1
renaming
original-brownbear Jun 23, 2020
13b3d4d
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 23, 2020
fc11ce1
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 24, 2020
5d3d447
simpler listener
original-brownbear Jun 24, 2020
d89ead6
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 24, 2020
a6d47f2
shorter
original-brownbear Jun 24, 2020
7627a30
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 24, 2020
c951e3b
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 25, 2020
cedcd88
resolve some conflicts
original-brownbear Jun 25, 2020
aa02752
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 26, 2020
5bf38b7
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 26, 2020
3eef5af
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 26, 2020
cc88a1c
changes
original-brownbear Jun 26, 2020
72c4019
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 26, 2020
095023c
fix
original-brownbear Jun 26, 2020
1009297
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 27, 2020
291e5f5
fix corner case
original-brownbear Jun 27, 2020
737cd48
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 29, 2020
86d0071
rename
original-brownbear Jun 29, 2020
d09f68e
add assertion
original-brownbear Jun 29, 2020
3e7662f
add assertion
original-brownbear Jun 29, 2020
974bd62
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 29, 2020
e388a6e
add queued state for shard snapshots
original-brownbear Jun 29, 2020
647e2ed
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 29, 2020
4b63b2f
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 29, 2020
f84c9b7
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 29, 2020
fba7536
50% fix
original-brownbear Jun 29, 2020
878fa07
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 30, 2020
5b40416
fixes
original-brownbear Jun 30, 2020
cefd323
fix
original-brownbear Jun 30, 2020
94d4608
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 30, 2020
d7674a6
moar coverage
original-brownbear Jun 30, 2020
dcfc8fd
moar coverage
original-brownbear Jun 30, 2020
d8a6b25
docs
original-brownbear Jun 30, 2020
6d97b42
even moar coverage
original-brownbear Jun 30, 2020
f74e4d1
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 30, 2020
217137d
fix
original-brownbear Jun 30, 2020
68aaa8e
cover another tricky corner case
original-brownbear Jun 30, 2020
cc36ffd
and another corner case
original-brownbear Jun 30, 2020
e2bfadd
almost full coverage
original-brownbear Jun 30, 2020
0afdbb5
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jun 30, 2020
b8c06c9
another failure handler tested
original-brownbear Jun 30, 2020
2f44a8f
fix bugs
original-brownbear Jun 30, 2020
b6cdba2
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jul 1, 2020
9c65059
docs and simpler
original-brownbear Jul 1, 2020
355432b
cover and fix very edgy edge case
original-brownbear Jul 1, 2020
8cf751a
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jul 2, 2020
773f905
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jul 2, 2020
de169f0
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jul 2, 2020
945ef0c
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jul 2, 2020
81fdf77
fix conflict
original-brownbear Jul 2, 2020
95a1dc5
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jul 3, 2020
5e79b5c
CR: small fixes
original-brownbear Jul 3, 2020
d2e97de
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jul 3, 2020
4c0fcaf
CR: add setting to limit concurrency
original-brownbear Jul 3, 2020
1f75a52
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jul 6, 2020
bfff04e
merge in master
original-brownbear Jul 6, 2020
528cc81
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jul 7, 2020
08177e1
merge in master
original-brownbear Jul 7, 2020
bb9cd43
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jul 7, 2020
883d220
Merge branch 'master' of github.com:elastic/elasticsearch into allow-…
original-brownbear Jul 7, 2020
906f51b
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jul 7, 2020
a7bb2d2
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jul 8, 2020
cb9af07
add test for concurrent ops and parallel ops
original-brownbear Jul 8, 2020
ca50bf4
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jul 9, 2020
f5a44d4
be smarter with shard generations to always allow for concurrent stuff
original-brownbear Jul 9, 2020
e9b0311
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jul 9, 2020
897e6d1
CR: limit 1k
original-brownbear Jul 9, 2020
382962b
Merge remote-tracking branch 'elastic/master' into allow-multiple-sna…
original-brownbear Jul 10, 2020
9e1a0db
review comments
original-brownbear Jul 10, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,7 @@ public void finalizeSnapshot(ShardGenerations shardGenerations, long repositoryS

@Override
public void deleteSnapshots(Collection<SnapshotId> snapshotIds, long repositoryStateId, Version repositoryMetaVersion,
ActionListener<Void> listener) {
ActionListener<RepositoryData> listener) {
if (SnapshotsService.useShardGenerations(repositoryMetaVersion) == false) {
listener = delayedListener(listener);
}
Expand Down

Large diffs are not rendered by default.

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,7 @@ private void buildResponse(SnapshotsInProgress snapshotsInProgress, SnapshotsSta
break;
case INIT:
case WAITING:
case QUEUED:
stage = SnapshotIndexShardStage.STARTED;
break;
case SUCCESS:
Expand Down
Loading