Skip to content

Replace busy wait in async loop with a Semaphore #80

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 27, 2025

Conversation

sjmonson
Copy link
Collaborator

Use a semaphore to concurrency limit the scheduling of new tasks rather than a busy wait. Additionally reduce async sleep at the end of the loop iteration as that is sufficient to cause preemption. As a side effect #70 is fixed by this PR since the offending code has been reworked.

Closes #70

Reduce sleep at end of run loop to 0
@sjmonson sjmonson force-pushed the fix/remove_busy_wait branch from 61eaec7 to 2636b6c Compare February 27, 2025 19:13
@sjmonson sjmonson requested a review from markurtz February 27, 2025 19:31
Copy link
Member

@markurtz markurtz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great!

@markurtz markurtz merged commit 7f611a5 into neuralmagic:main Feb 27, 2025
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

GUIDELLM__MAX_CONCURRENCY is off by 1
2 participants