Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor shard modes logic #1559

Merged
merged 14 commits into from
Jul 21, 2022
Merged

Refactor shard modes logic #1559

merged 14 commits into from
Jul 21, 2022

Conversation

fyrchik
Copy link
Contributor

@fyrchik fyrchik commented Jun 29, 2022

No description provided.

@codecov
Copy link

codecov bot commented Jun 29, 2022 •
edited
Loading

Codecov Report

Merging #1559 (1788fe4) into master (2455b72) will decrease coverage by 0.04%.
The diff coverage is 54.44%.

❗ Current head 1788fe4 differs from pull request most recent head 41ebb4e. Consider uploading reports for the commit 41ebb4e to get more accurate results

@@            Coverage Diff             @@
##           master    #1559      +/-   ##
==========================================
- Coverage   33.13%   33.08%   -0.05%     
==========================================
  Files         329      330       +1     
  Lines       22468    22558      +90     
==========================================
+ Hits         7444     7463      +19     
- Misses      14421    14482      +61     
- Partials      603      613      +10
Impacted Files Coverage Δ
...kg/local_object_storage/blobovnicza/blobovnicza.go 82.85% <0.00%> (ø)
pkg/local_object_storage/blobstor/blobstor.go 82.53% <ø> (ø)
pkg/local_object_storage/blobstor/mode.go 0.00% <0.00%> (ø)
pkg/local_object_storage/engine/container.go 0.00% <0.00%> (ø)
pkg/local_object_storage/metabase/db.go 72.18% <ø> (-1.33%) ⬇️
pkg/local_object_storage/metabase/mode.go 0.00% <0.00%> (ø)
pkg/local_object_storage/pilorama/interface.go 100.00% <ø> (ø)
pkg/local_object_storage/shard/container.go 0.00% <0.00%> (ø)
pkg/local_object_storage/shard/exists.go 0.00% <0.00%> (ø)
pkg/local_object_storage/shard/id.go 0.00% <0.00%> (ø)
... and 62 more

realloc
realloc reviewed Jul 18, 2022
View reviewed changes
cmd/neofs-cli/modules/control/shards_set_mode.go Outdated
shardModeDegraded = "degraded"
shardModeReadOnly = "read-only"
shardModeReadWrite = "read-write"
shardModeDegraded = "degraded"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's rename this state to degraded-read-write for the sake of consistency.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fyrchik

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

carpawell
carpawell reviewed Jul 19, 2022
View reviewed changes
Copy link
Member

@carpawell carpawell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you check all the CreateIfNotExists in the metabase's "read" operation?

pkg/local_object_storage/blobstor/mode.go Show resolved Hide resolved
pkg/local_object_storage/writecache/mode.go Show resolved Hide resolved
pkg/local_object_storage/shard/mode/mode.go Show resolved Hide resolved
pkg/local_object_storage/shard/exists.go Show resolved Hide resolved
pkg/local_object_storage/engine/delete.go Outdated Show resolved Hide resolved
pkg/local_object_storage/engine/delete.go Show resolved Hide resolved
pkg/local_object_storage/blobstor/blobovnicza.go Outdated Show resolved Hide resolved
pkg/local_object_storage/writecache/mode.go Outdated Show resolved Hide resolved
pkg/local_object_storage/writecache/mode.go Show resolved Hide resolved
cmd/neofs-cli/modules/control/shards_set_mode.go Outdated
shardModeDegraded = "degraded"
shardModeReadOnly = "read-only"
shardModeReadWrite = "read-write"
shardModeDegraded = "degraded"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fyrchik

@fyrchik
Copy link
Contributor Author

fyrchik commented Jul 21, 2022

Did you check all the CreateIfNotExists in the metabase's "read" operation?

Could you elaborate a bit?

carpawell
carpawell previously approved these changes Jul 21, 2022
View reviewed changes
pkg/local_object_storage/shard/mode/mode.go Show resolved Hide resolved
@carpawell
Copy link
Member

Could you elaborate a bit?

@fyrchik, I remember that we could create some buckets even in "read" operations via CreateBucketIfNotExists. Guess it is not smth that we wanna do if we try to save the disk and do not write any data to it. But Im not sure, did not check.

@fyrchik
Copy link
Contributor Author

fyrchik commented Jul 21, 2022

@carpawell bbolt returns an error if CreateBucket is used in a transaction that is not writeable.

fyrchik and others added 14 commits July 21, 2022 15:10
@fyrchik
[nspcc-dev#1559] metabase: Remove public functions
5f6edad 
Reduce public interface of this package. Later each result will contain
an additional status, so it makes more sense to use the same functions
and result processing everywhere.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
@fyrchik
[nspcc-dev#1559] metabase: Use Set prefix for parameter setting
2b632ab 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
@fyrchik
[nspcc-dev#1559] shard: Remove public functions
42ca1c0 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
@fyrchik
[nspcc-dev#1559] shard: Use Set prefix for parameter setting
b57e448 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
@fyrchik
[nspcc-dev#1559] local_object_storage: Move shard.Mode to a separat…
5de09a3 
…e package

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
@fyrchik
[nspcc-dev#1559] shard: Do not consult metabase in a degraded mode
1ec2c94 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
@fyrchik
[nspcc-dev#1559] engine: Do not count logical errors as storage ones
37df468 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
@fyrchik
[nspcc-dev#1549] engine: Disable shard on blobovnicza init failure
107ea21 
There is a need to support working w/o shard if it has problems with
blobovnicza tree.

Make `BlobStor.Init` to return new `ErrInitBlobovniczas` error. Remove
shard from storage engine's shard set if it returned this error from
`Init` call. So if some of the shards (but not all) return this error,
the node will be able to continue working without them.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
@fyrchik
[nspcc-dev#1559] local_object_storage: Provide readOnly flag to Open
ba0ac94 
We should be able to reopen storage in readonly in runtime.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
@fyrchik
[nspcc-dev#1559] local_object_storage: Allow to set mode for all comp…
21fc153 
…onents

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
@fyrchik
[nspcc-dev#1559] local_object_storage: Move shard to the `DegradedRea…
db106dd 
…dOnly` mode

`Degraded` mode can be set by the administrator if needed.
Modifying operations in this mode can lead node into an inconsistent state
because metabase checks such as lock checking are not performed.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
@fyrchik
[nspcc-dev#1549] shard: Turn to read-only mode on metabase failure
b970206 
If metabase can't be opened in the default mode, try opening shard
first in `ReadOnly` mode and then in `DegradedReadOnly`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
@fyrchik
[nspcc-dev#1559] shard: Change Degraded mode string representation
d0313a5 
It is a flag, but is a `degraded-read-write` mode for a user.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
@fyrchik
[nspcc-dev#1559] local_object_storage: Fix tests and some data races
41ebb4e 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
@fyrchik fyrchik merged commit 1691364 into nspcc-dev:master Jul 21, 2022
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
@fyrchik @aprasolova
[nspcc-dev#1559] metabase: Remove public functions
d9d1c2b 
Reduce public interface of this package. Later each result will contain
an additional status, so it makes more sense to use the same functions
and result processing everywhere.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
@fyrchik @aprasolova
[nspcc-dev#1559] metabase: Use Set prefix for parameter setting
c9f4029 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
@fyrchik @aprasolova
[nspcc-dev#1559] shard: Remove public functions
706ae4f 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
@fyrchik @aprasolova
[nspcc-dev#1559] shard: Use Set prefix for parameter setting
f395de3 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
@fyrchik @aprasolova
[nspcc-dev#1559] local_object_storage: Move shard.Mode to a separat…
35d3418 
…e package

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
@fyrchik @aprasolova
[nspcc-dev#1559] shard: Do not consult metabase in a degraded mode
d559438 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
@fyrchik @aprasolova
[nspcc-dev#1559] engine: Do not count logical errors as storage ones
c9185c7 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
@fyrchik @aprasolova
[nspcc-dev#1559] local_object_storage: Provide readOnly flag to Open
a025512 
We should be able to reopen storage in readonly in runtime.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
@fyrchik @aprasolova
[nspcc-dev#1559] local_object_storage: Allow to set mode for all comp…
24febd1 
…onents

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
@fyrchik @aprasolova
[nspcc-dev#1559] local_object_storage: Move shard to the `DegradedRea…
36c1b1f 
…dOnly` mode

`Degraded` mode can be set by the administrator if needed.
Modifying operations in this mode can lead node into an inconsistent state
because metabase checks such as lock checking are not performed.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
@fyrchik @aprasolova
[nspcc-dev#1559] shard: Change Degraded mode string representation
e76f902 
It is a flag, but is a `degraded-read-write` mode for a user.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
@fyrchik @aprasolova
[nspcc-dev#1559] local_object_storage: Fix tests and some data races
1ab1249 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@carpawell carpawell carpawell approved these changes

@realloc realloc Awaiting requested review from realloc

Assignees

@fyrchik fyrchik

Labels
neofs-storage Storage node application issues
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@fyrchik @carpawell @realloc