This repository was archived by the owner on Jan 23, 2023. It is now read-only.
Use # of processors + 1 available to OS scheduler on Linux when building #100
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`getconf _NPROCESSORS_ONLN` provides the number of cores available to the OS scheduler, while `nproc` provides the number of cores available to `nproc` itself. We want to use the former value since make spawns new processes instead of running compilation tasks in the same process. (FWIW, the two values are usually the same, but `getconf` gets us the value that we actually want.) Additionally, `getconf` is available on Mac OSX while `nproc` is not. I also fixed the printout of the `make` command that's actually executed when we build.
|
It turns out that N + 1 processors is slightly faster, 2-3%, than N processors. I'm going to update this to use N+1 processors. |
N+1 processors appears to perform 2-3% faster than using just N processors. I've done an A/B test both locally and on a cloud VM. Fixes for this and the previous commit were reported by Sedar Dilek <sedar.dilek@gmail.com>. Thanks!
xyziemba
changed the title
|
LGTM |
Closed
|
LGTM |
jkotas
added a commit
that referenced
this pull request
Feb 6, 2015
Use # of processors + 1 available to OS scheduler on Linux when building
|
Ingenious solution! |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
getconf _NPROCESSORS_ONLNprovides the number of cores available tothe OS scheduler, while
nprocprovides the number of cores availableto
nprocitself. We want to use the former value since make spawnsnew processes instead of running compilation tasks in the same process.
(FWIW, the two values are usually the same, but
getconfgets us thevalue that we actually want.)
Additionally,
getconfis available on Mac OSX whilenprocis not.I also fixed the printout of the
makecommand that's actuallyexecuted when we build.
Also, thank you to Sedat Dilek for reporting this via email.