Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Do not skip shards that are busy #2871

Open
carpawell opened this issue Jun 13, 2024 · 0 comments
Open

Do not skip shards that are busy #2871

carpawell opened this issue Jun 13, 2024 · 0 comments
Labels
bug Something isn't working I4 No visible changes S3 Minimally significant U4 Nothing urgent
Milestone
v0.44.0

Comments

@carpawell
Copy link
Member

Expected Behavior

Prioritized shards should be blocked until they are free or network timeout is reached. Do not break HRW rules just cause some shard is less loaded.

Current Behavior

Every shard has a number of parallel write operations (ants.Pool), if this number is reached, the next shard is tried, object is placed to a "wrong" (in HRW terms) shard.

Possible Solution

Make every pool in every shard blocking.

Context

On the other hand, this (WithNonblocking) looks not good to me and can be changed. It may be the reason of objects-on-wrong-shards effects and also may lead to an error when all shards are busy (i would prefer to wait for them and fail with deadline). @roman-khimov, @cthulhu-rider

Originally posted by @carpawell in #2868 (comment)
@roman-khimov roman-khimov added bug Something isn't working U4 Nothing urgent S3 Minimally significant I4 No visible changes labels Jun 14, 2024
@roman-khimov roman-khimov added this to the v0.44.0 milestone Aug 5, 2024
@carpawell carpawell mentioned this issue Oct 4, 2024
Open
carpawell added a commit that referenced this issue Oct 7, 2024
@carpawell
node/engine: do not skip objects if shards are busy
0f8b19e 
If every shard's pool is overloaded with routines, choose the best one and try
to PUT an object to it 30 seconds. Closes #2871.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
carpawell added a commit that referenced this issue Oct 7, 2024
@carpawell
node/engine: do not skip objects if shards are busy
03f1f87 
If every shard's pool is overloaded with routines, choose the best one and try
to PUT an object to it 30 seconds. Closes #2871.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
carpawell added a commit that referenced this issue Oct 16, 2024
@carpawell
node/engine: do not skip objects if shards are busy
13b546f 
If every shard's pool is overloaded with routines, choose the best one and try
to PUT an object to it 30 seconds. Closes #2871.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
carpawell added a commit that referenced this issue Oct 16, 2024
@carpawell
node/engine: do not skip objects if shards are busy
12246c2 
If every shard's pool is overloaded with routines, choose the best one and try
to PUT an object to it 30 seconds. Closes #2871.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working I4 No visible changes S3 Minimally significant U4 Nothing urgent
Projects
None yet
Milestone
v0.44.0
Development

No branches or pull requests
2 participants
@roman-khimov @carpawell