ninja: error: loading 'build.ninja': No such file or directory

[alwa-nordic]

If a pristine build fails during CMake configure, it leaves a build folder that breaks incremental building. The telltale log message is ninja: error: loading 'build.ninja': No such file or directory. This problem does not appear in vanilla non-Zephyr CMake.

To Reproduce
Steps to reproduce the behavior:

• Start with the hello world sample.
• Make sure there is no build folder yet.
• Add some error to CMakeLists.txt. E.g. append this_is_not_defined().
• Run west build -b native_posix. (Selected board is irrelevant.)
• Observe CMake error.
• Fix error in CMakeLists.txt.
• Run west build for incremental build.
• Observe bug ninja: error: loading 'build.ninja': No such file or directory.

The workaround is to delete the build folder.

Impact
User confusion, major annoyance.

[junaidaslamRS]

Looks like this is not fixed yet. I did a fresh clone of the mainline today, and it is still there...

[github-actions]

This issue has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this issue will automatically be closed in 14 days. Note, that you can always re-open a closed issue at any time.

[theob-pro]

I am experiencing this issue too, is there some plan to investigate it?

[github-actions]

This issue has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this issue will automatically be closed in 14 days. Note, that you can always re-open a closed issue at any time.

[tejlmand]

This is not a west build or Zephyr issue.

Incremental builds are done using cmake --build <build-folder> and same behavior is seen both with CMake and ninja when trying to do incremental builds.

For example:

$ cmake -GNinja -DBOARD=nrf52840dk/nrf52840 -Bbuild samples/hello_world
Loading Zephyr default modules (Zephyr repository).
-- Application: /projects/github/ncs/zephyr/samples/hello_world
...
CMake Error at CMakeLists.txt:7 (foobar):
  Unknown CMake command "foobar".

As described in the issue, then fixing the issue and trying to do an incremental build results in:

$ cmake --build build
ninja: error: loading 'build.ninja': No such file or directory

same with ninja:

$ ninja -Cbuild
ninja: Entering directory `build'
ninja: error: loading 'build.ninja': No such file or directory

west can be to forced to re-run CMake in the above case, by using -c / --cmake.

$ west build
ninja: error: loading 'build.ninja': No such file or directory
FATAL ERROR: command exited with status 1: /usr/bin/cmake --build /projects/github/ncs/zephyr/build
$ west build -c
-- west build: generating a build system
Loading Zephyr default modules (Zephyr base (cached)).
-- Application: /projects/github/ncs/zephyr/samples/hello_world
...
[135/135] Linking C executable zephyr/zephyr.elf
Memory region         Used Size  Region Size  %age Used
           FLASH:       18584 B         1 MB      1.77%
             RAM:        5696 B       256 KB      2.17%
        IDT_LIST:          0 GB        32 KB      0.00%
Generating files from /projects/github/ncs/zephyr/build/zephyr/zephyr.elf for board: nrf52840dk

[alwa-nordic]

I did not say it was a west issue. I added the "area: Build System" label. @thedjnK changed it to "area: West". This is a DX issue that should be addressed, so I'll reopen this.

[tejlmand]

@alwa-nordic you also wrote:

This problem does not appear in vanilla non-Zephyr CMake.

west build is just a wrapper around CMake for convenience.
And as I showed, the exact same errors happens if you try to invoke cmake --build or ninja on a build folder where the build.ninja is missing.

When you refer to This problem does not appear in vanilla non-Zephyr CMake. then I interprete that as you are doing a manual CMake run, and the equivalent to that, when using west build, is to run west build -c as I wrote in earlier reply.

So the solution to your issue is to use west build -c for your incremental build.
Therefore I don't consider this a bug, and thus nothing needs to be fixed.

The workaround is to delete the build folder.

Use west build -c to ensure cmake is invoked.

[tejlmand]

and just to make it clear, from west help build:

$ west help build
usage: west build ....

Convenience wrapper for building Zephyr applications.

...
cmake and build tool:
  -c, --cmake           force a cmake run
...

[mmahadevan108]

@alwa-nordic , can this issue be closed?

[jori-nordic]

I still think it's a west build failure.
If it was really a "convenient wrapper" it would:

• if there's a build.ninja just run ninja build
• if there is no build.ninja, do a cmake run before

Doesn't seem like rocket science to me, and would solve this UX issue.

[tejlmand]

if there's a build.ninja just run ninja build

because it's a wrapper then west build technically doesn't know that ninja is to be used.
You can run west build ... -- -G'Unix Makefiles' and CMake will generate Makefiles instead of ninja files.
Or you can use another generator of your choice, of course depending on your OS.

This is exactly why west doesn't call ninja (or any other build tool) but calls cmake --build because then it's CMake's responsibility to call the correct build tool.

So what you're basically asking is that CMake should re-run itself when there is a CMake cache but no build file, but such request is better placed at CMake, not west.

[tejlmand]

Does n't seem like rocket science to me, and would solve this UX issue.

you're welcome to open a proposal, just remember that such change must continue to work with other generators besides ninja, and not suddenly start re-running CMake because ninja files are missing when one of the Makefile based generators is used: https://cmake.org/cmake/help/latest/manual/cmake-generators.7.html#cmake-generators

And remember to maintain whatever changes you introduce and post additional fixes in case a given generator is not working as expected.

Personally I consider this to be a design decision in how cmake --build works and thus not a west build issue.

But although not being a bug, then you can still consider it an enhancement.
Several changes are made to Zephyr and tooling on things that are not bugs 😉

[jori-nordic]

CMake cache but no build file, but such request is better placed at CMake

I was not aware of that, and the comments before hinted that west was calling ninja itself.

[tejlmand]

and the comments before hinted that west was calling ninja itself.

I actually tried to avoid such misunderstanding in my first comment here: #67927 (comment)

This is not a west build or Zephyr issue.

Incremental builds are done using cmake --build <build-folder> ...

[alwa-nordic]

Ok, so from a purely DX-perspective, the issue is thus: It appears west build is a unitary command that will normally perform both the configure and build steps as needed. But this mental picture fails in the described case.

There is an argument that in CMake, the underlying system, the configure and build tasks are entirely separate. Counterargument: Since these are not separate steps in West, this should be hidden as an implementation detail if possible. Especially since the resulting error is not good (it's not instructive for new users).

How does the "configure when needed" work today?: Internally, there two separate triggers for starting the configure: West does it when there is no build folder yet. And the generated Makefile has deps on the CMake files and reruns the configure step when those change.

How does it fail?: A failure during the configure step will leave a build folder, but not a Makefile. West only checks for the presence of the build folder to determine if the Makefile can take the responsibility of "configuring when needed". But with no Makefile, the make program does not take this responsibility.

Solutions:

• Make West by default always perform the configure step. This is the safest option. It can be made very fast. The configure step is also incremental in CMake, in the sense that it should use the CMake cache to speed up the reruns. I'm for this.

  Modern CMake has "workflows" that can be defined in CMakePresets.txt, and in that system it seems intended to just run the configure step every time.

• Make West's initial configure test more strict. I looked over the variables in a failed CMakeCache.txt, but I can't see anything that plainly records the success or failure, but I noticed there is a file CMakeFiles/TargetDirectories.txt that only written in the case of success. This file exists both with Ninja and GNU Make as generator. I don't know what this file is though and documentation is hard to find. Could we use the presence of this file as the check?

[alwa-nordic]

Solution:

• West leaves a marker file in the build directory upon the first successful configure (looking at exit code of CMake) and uses the presence of that file to determine the handover.