Worker Pool #103
Open
Worker Pool #103
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Updates the trigger function to use a callback function as the result handler, rather than the function response. This has been done to not spawn a new go routine for every job, and instead utilize a consistent worker pool. This significantly reduces the number of go routines in high-throughput scenarios, improving performance and reducing resource consumption. Jobs are routed to a set of worker loops based on a simple hash of the job name. By default, 64 workers are created, but this can be adjusted. Signed-off-by: joshvanl <me@joshvanl.dev>
nelson-parente
approved these changes
Dec 2, 2025
Contributor
nelson-parente
left a comment
nelson-parente
left a comment
There was a problem hiding this comment.
overall looks good to me though I'm not familiar with this code 🙃
JoshVanL
added a commit
to JoshVanL/dapr
that referenced
this pull request
Dec 2, 2025
Workers is the number of workers that handle job events. The higher the number the more go routines will be spawned, each working over a partition of the total job space. The higher the number, the higher the number of jobs which can be concurrently delivered to runtimes. Increasing this number increases the number of go routines for this instance. This number should be tuned to the bottleneck of job execution, i.e., CPU, I/O, memory, etc. Defaults to 128 workers. `--workers=N` flag for the scheduler binary. `dapr_scheduler.workers=N` in the Helm chart. Branched from dapr#9205 Uses diagridio/go-etcd-cron#103 Signed-off-by: joshvanl <me@joshvanl.dev>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Updates the trigger function to use a callback function as the result handler, rather than the function response. This has been done to not spawn a new go routine for every job, and instead utilize a consistent worker pool. This significantly reduces the number of go routines in high-throughput scenarios, improving performance and reducing resource consumption.
Jobs are routed to a set of worker loops based on a simple hash of the job name. By default, 64 workers are created, but this can be adjusted.