DoBatch preference to 4xx if error #4783
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
alvinlin123
approved these changes
Jul 13, 2022
rohang98
approved these changes
Jul 19, 2022
|
LGTM! |
bboreham
reviewed
Jul 20, 2022
Comment on lines
+49
to
+53
| if i.failed5xx.Load() > i.failed4xx.Load() { | ||
| return i.err5xx.Load() | ||
| } | ||
|
|
||
| return i.err4xx.Load() |
There was a problem hiding this comment.
How about fetching the two values into local variables, so you don't make 4x atomic calls?
There was a problem hiding this comment.
Bryan, i am not sure i get your idea. They are 4 different variables. 2 for amount of erros and 2 for the actual error. Current we can only return the last known error, the change is basically to hold one error for each family type.
Signed-off-by: Daniel Blando <ddeluigg@amazon.com>
Signed-off-by: Daniel Blando <ddeluigg@amazon.com>
alanprot
approved these changes
Jul 26, 2022
alexqyle
pushed a commit
to alexqyle/cortex
that referenced
this pull request
Aug 2, 2022
* DoBatch preference to 4xx if error Signed-off-by: Daniel Blando <ddeluigg@amazon.com> * Fix comment Signed-off-by: Daniel Blando <ddeluigg@amazon.com> Signed-off-by: Alex Le <leqiyue@amazon.com>
alanprot
added a commit
that referenced
this pull request
Sep 3, 2022
* Introduced lock file to shuffle sharding grouper Signed-off-by: Alex Le <leqiyue@amazon.com> * let redis cache logs log with context (#4785) * let redis cache logs log with context Signed-off-by: Mengmeng Yang <mengmengyang616@gmail.com> * fix import Signed-off-by: Mengmeng Yang <mengmengyang616@gmail.com> Signed-off-by: Alex Le <leqiyue@amazon.com> * DoBatch preference to 4xx if error (#4783) * DoBatch preference to 4xx if error Signed-off-by: Daniel Blando <ddeluigg@amazon.com> * Fix comment Signed-off-by: Daniel Blando <ddeluigg@amazon.com> Signed-off-by: Alex Le <leqiyue@amazon.com> * Updated CHANGELOG and ordered imports Signed-off-by: Alex Le <leqiyue@amazon.com> * Fixed lint and removed groupCallLimit Signed-off-by: Alex Le <leqiyue@amazon.com> * Changed lock file to json format and make sure planner would not pick up group that is locked by other compactor Signed-off-by: Alex Le <leqiyue@amazon.com> * Fix updateCachedShippedBlocks - new thanos (#4806) Signed-off-by: Alan Protasio <approtas@amazon.com> Signed-off-by: Alex Le <leqiyue@amazon.com> * Join memberlist on starting with no retry (#4804) Signed-off-by: Daniel Blando <ddeluigg@amazon.com> * Fix alertmanager log message (#4801) Signed-off-by: Xiaochao Dong (@damnever) <the.xcdong@gmail.com> Signed-off-by: Alex Le <leqiyue@amazon.com> * Grafana Cloud uses Mimir now, so remove Grafana Cloud as hosted service in documents (#4809) * Grafana Cloud uses Mimir, for of Cortex, now Signed-off-by: Alvin Lin <alvinlin123@gmail.com> * Improve doc Signed-off-by: Alvin Lin <alvinlin@amazon.com> Signed-off-by: Alex Le <leqiyue@amazon.com> * Created block_locker to handle all block lock file operations. Added block lock metrics. Signed-off-by: Alex Le <leqiyue@amazon.com> * Moved lock file heart beat into planner and refined planner logic to make sure blocks are locked by current compactor Signed-off-by: Alex Le <leqiyue@amazon.com> * Updated documents Signed-off-by: Alex Le <leqiyue@amazon.com> * Return concurrency number of group. Use ticker for lock file heart beat Signed-off-by: Alex Le <leqiyue@amazon.com> * Renamed lock file to be visit marker file Signed-off-by: Alex Le <leqiyue@amazon.com> * Fixed unit test Signed-off-by: Alex Le <leqiyue@amazon.com> * Make sure visited block can be picked by compactor visited it Signed-off-by: Alex Le <leqiyue@amazon.com> Signed-off-by: Alex Le <leqiyue@amazon.com> Signed-off-by: Mengmeng Yang <mengmengyang616@gmail.com> Signed-off-by: Daniel Blando <ddeluigg@amazon.com> Signed-off-by: Alan Protasio <approtas@amazon.com> Signed-off-by: Xiaochao Dong (@damnever) <the.xcdong@gmail.com> Signed-off-by: Alvin Lin <alvinlin@amazon.com> Signed-off-by: Alex Le <emoc1989@gmail.com> Co-authored-by: Mengmeng Yang <mengmengyang616@gmail.com> Co-authored-by: Daniel Blando <ddeluigg@amazon.com> Co-authored-by: Alan Protasio <approtas@amazon.com> Co-authored-by: Xiaochao Dong <the.xcdong@gmail.com> Co-authored-by: Alvin Lin <alvinlin@amazon.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Daniel Blando ddeluigg@amazon.com
What this PR does:
After #4388, we started returning the error which most failed. This CR improves the logic to also prioritize 4xx if it was the same amount of 5xx errors. The logic being that in a case of 4xx and 5xx, we are predicting the customer was close to their limits and 4xx is more relevant than 5xx. The change also creates more consistency in responses.
If we had a 3 quorum
Before:
2xx, 5xx, 4xx -> returns 4xx
2xx, 4xx, 5xx -> returns 5xx
5xx, 4xx, 5xx -> returns 5xx
4xx, 5xx, 4xx -> returns 4xx
After change:
2xx, 5xx, 4xx -> returns 4xx
2xx, 4xx, 5xx -> returns 4xx
5xx, 4xx, 5xx -> returns 5xx
4xx, 5xx, 4xx -> returns 4xx
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]