Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/v2 split placement to non-container node #2792

Merged
merged 3 commits into from
Apr 1, 2024

Conversation

carpawell
Copy link
Member

No description provided.

@carpawell carpawell added the bug Something isn't working label Mar 29, 2024
@carpawell carpawell self-assigned this Mar 29, 2024
@carpawell
Copy link
Member Author

Run is here (was not finished at the comment creation time but should be ok).

Why PUT == object not found: #2744.

Copy link

codecov bot commented Mar 29, 2024

Codecov Report

Attention: Patch coverage is 0% with 37 lines in your changes are missing coverage. Please review.

Project coverage is 22.02%. Comparing base (0d6d585) to head (695cad1).

Files Patch % Lines
pkg/services/object/put/distributed.go 0.00% 28 Missing ⚠️
pkg/services/object_manager/placement/traverser.go 0.00% 8 Missing ⚠️
pkg/services/object/put/streamer.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2792      +/-   ##
==========================================
- Coverage   22.03%   22.02%   -0.01%     
==========================================
  Files         789      789              
  Lines       47245    47265      +20     
==========================================
  Hits        10410    10410              
- Misses      35941    35961      +20     
  Partials      894      894              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@@ -20,7 +20,8 @@ type preparedObjectTarget interface {
}

type distributedTarget struct {
traversal traversal
traversalParams traversal
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its not params, its a traversal state, so new name is unclear

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i was not sure i can understand what traversal is while there is traverser, so let it be traversalState then

@@ -156,6 +156,27 @@ func (x Node) PublicKey() []byte {
return x.key
}

// KeyInContainer passes nodes' public keys from the calculated placement vectors to the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iterator method (action) is named as property. I suggest LookupKeyInContainer

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can call it as KeyInContainer(isNotLocalKey()), so some IterateContainerKeys() can be even more appropriate.

Copy link
Member Author

@carpawell carpawell Apr 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i do not like iteration where i do not need iteration (the same way here), i just wanted bool but there was no key in the distributedTarget, no idea why isLocalKey is useful but did not want to change this part in a little tests fix PR so let it be IterateContainerKeys

@@ -243,7 +249,11 @@ loop:
}

// perform additional container broadcast if needed
if t.traversal.submitPrimaryPlacementFinish() {
if t.traversalParams.submitPrimaryPlacementFinish() {
// reset traversal progress
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

afaik u dont reset processed nodes, so its not a progress reset

Copy link
Member Author

@carpawell carpawell Apr 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

traverser has SubmitSuccess method. it breaks additional broadcasting if do not reset this state

// for the link validation and may decline it, does not matter what this node
// thinks about it
if t.obj.Type() != objectSDK.TypeLink || t.traverser.KeyInContainer(t.isLocalKey) {
if t.objMeta, err = t.fmt.ValidateContent(t.obj); err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think some syntactic check can still be made here, although it doesn't seem to be critical.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you mean to try to parse the link object but not validate? or what "syntactic check"?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, parse, but don't go fetching/checking things.

@@ -156,6 +156,27 @@ func (x Node) PublicKey() []byte {
return x.key
}

// KeyInContainer passes nodes' public keys from the calculated placement vectors to the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can call it as KeyInContainer(isNotLocalKey()), so some IterateContainerKeys() can be even more appropriate.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
@carpawell carpawell force-pushed the fix/v2-split-placement-to-non-container-node branch from 419bbff to 59e2f7e Compare April 1, 2024 19:24
This may be impossible to validate depending on container privacy settings.
Also, if a node does not have to store a link object, its validation is
literally useless work: it will repeat all the work one more time on any
container inclusion and corresponding link object replication.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
1.22.0 says it should be this way.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
@carpawell carpawell force-pushed the fix/v2-split-placement-to-non-container-node branch from 59e2f7e to 695cad1 Compare April 1, 2024 19:31
@roman-khimov roman-khimov merged commit 97a2dc7 into master Apr 1, 2024
12 of 16 checks passed
@roman-khimov roman-khimov deleted the fix/v2-split-placement-to-non-container-node branch April 1, 2024 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants