ci: Fix L0_batch related flaky tests #7999
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rmccorm4
reviewed
Feb 11, 2025
Comment on lines
-1945
to
+1948
| # and the remaining requests will form the second batch. | ||
| # and the remaining requests will either form the second batch | ||
| # or more batches depending on their arrival time. |
There was a problem hiding this comment.
Can you explain why the originally intended behavior is not consistent and needs to be relaxed here?
rmccorm4
reviewed
Feb 11, 2025
| @@ -1895,7 +1897,7 @@ def test_preferred_batch_only_use_no_preferred_size(self): | |||
| def test_max_queue_delay_only_non_default(self): | |||
| # Send 12 requests with batch size 1. The max_queue_delay is set | |||
| # to non-zero. Depending upon the timing of the requests arrival | |||
| # there can be either 1 or 2 model executions. | |||
| # there can be either 1 or multiple model executions. | |||
There was a problem hiding this comment.
Can you explain why the originally intended behavior is not consistent and needs to be relaxed here? (different for zero and non-zero queue delay scenarios?)
rmccorm4
reviewed
Feb 11, 2025
| @@ -619,13 +619,15 @@ def test_multi_batch_not_preferred_different_shape(self): | |||
| }, | |||
| ) | |||
| ) | |||
| # Add some delay to ensure the first two requests arrive before the third | |||
| time.sleep(2) | |||
There was a problem hiding this comment.
Should this 2 seconds be some function of the _max_queue_delay_ms instead?
… yinggeh-DLIS-7826-fix-L0-batcher
yinggeh
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does the PR do?
Set looser time requirement and execution counts to L0_batcher* tests. The actual response time and execution counts are highly dependent to the request arrival time.
Checklist
<commit_type>: <Title>Commit Type:
Check the conventional commit type
box here and add the label to the github PR.
Related PRs:
Where should the reviewer start?
Test plan:
L0_batcher--base
L0_batcher_shm--base
L0_batcher_cudashm--base
23748413
Caveats:
Background
Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)