Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

README: explain parallel build [no ci] #7618

Merged

Conversation

JohannesGaessler
Copy link
Collaborator

Fixes #7518 .

This PR adds instructions for how to speed up compilation with make and cmake. I am not aware of a portable method for specifying multiple parallel jobs for cmake so I explained how to pass arguments to the internally used build tool instead.

@mofosyne mofosyne added documentation Improvements or additions to documentation Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix merge ready indicates that this may be ready to merge soon and is just holding out in case of objections labels May 29, 2024
README.md Outdated Show resolved Hide resolved
@mofosyne
Copy link
Collaborator

mofosyne commented May 29, 2024

Oh btw have you considered recommending the use of nproc instead?

make -j $(nproc)

nproc

Print the number of processing units (normally CPUs) available. More information: https://www.gnu.org/software/coreutils/nproc.

  • Display the number of available processing units: nproc
  • Display the number of installed processing units, including any inactive ones: nproc --all
  • If possible, subtract a given number of units from the returned value: nproc --ignore {{count}}

@JohannesGaessler
Copy link
Collaborator Author

I don't think we should overcomplicate things. The main point of this change is to make users aware that a parallel build is possible at all, and for those users I would just keep the whole thing as simple as possible.

@simonlui
Copy link

I am not aware of a portable method for specifying multiple parallel jobs for cmake so I explained how to pass arguments to the internally used build tool instead.

This has been an option since CMake 3.12 though. https://cmake.org/cmake/help/latest/manual/cmake.1.html#build-tool-mode

-j [], --parallel []

New in version 3.12.

The maximum number of concurrent processes to use when building. If is omitted the native build tool's default number is used.

The CMAKE_BUILD_PARALLEL_LEVEL environment variable, if set, specifies a default parallel level when this option is not given.

Some native build tools always build in parallel. The use of value of 1 can be used to limit to a single job.

@JohannesGaessler JohannesGaessler merged commit 972b555 into ggerganov:master May 30, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation merge ready indicates that this may be ready to merge soon and is just holding out in case of objections Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[docs] make: set the number of jobs to boost compilation speeds for less experienced users
4 participants