[RFC] Discussion of "cmake" vs "make" variables, aka "build environment" vs "work environment" setup

[pfalcon]

There was some discussion of this issue between @SebastianBoe and me in #5140 and #5142 , and as those tickets are particular cases, and the discussion is far from complete, I'd like to create this ticket for wider and longer discussion.

So, idea I'd like to convey is that original Kbuild based systems consisted (at least) of the following, pretty distinct parts:

1. Build system per se, which compiled sources into binaries.
2. "Work with"/test/debug system, e.g. "make run", "make gdbserver", etc.

Some aspects of p.2 were controlled with make variables, e.g. one could say "make run QEMU_PTY=1" to run just this qemu invocation via pseudo-tty.

During CMake conversion, it seems that (almost?) all variables which controlled p.2 behavior were converted into CMake variables, which of course change workflow relying on those vars adversely, making there usage rather inconvenient.

The obvious solution is to convert them back into make variables, like they were originally, and how people expected to use them.

Two following counter-arguments were raised/can be imagined to be raised:

a) Due to @SebastianBoe: "But we want IDE integration..."
b) Was used on a different occasion by @mbolivar, so I guess could be raised here too: "But CMake can generate for Ninja and other build systems..."

My response to both is the same: Folks, when you want to use IDE, when you want to use Ninja, you want to use p.1 of the above, the "build system" part of Zephyr's control environment. p.2 is outside of your control, sorry. It innately and natively targeted at unix shell, make, and other command-line, specific tools. If you want to build p.2 for another tools, you'll likely need to work out a solution for that particular tool (a specific IDE, a specific make alternative). It might be that you could reuse some parts of unix command line p.2, but even you can't be sure for now, quoting @SebastianBoe #5142 (comment) "I can't be more specific than that because we haven't yet figured out how this will work, that will come after CMake has stabilized and Windows support has improved."

In the meantime, command-line and make using people just want their productive workflow, which was disrupted by CMake migration, back. And my argument is that converting as much as possible p.2-related vars from CMake to make is a reasonable and viable approach.

This is of course just RFC, representing my partial and biased opinion. I'd like to invite all interested parties to share their outlook, so we all understand where we're heading, and in via which ways.

Thanks.

[carlescufi]

@pfalcon I think that moving those variables to actual "runtime" (p2) with Make/Ninja should not preclude the option of later setting those via IDE, meaning that, unless I'm missing something I don't see why we cannot move those back to p2 for some reason that escapes me now. I agree that options like QEMU_PTY belong in Make and not CMake, since they have nothing to do with the actual building of the target but rather with the way the target is run?

[SebastianBoe]

I suspect that instead of a migration of variables from CMake to Make, we can do both in many cases and keep everyone happy.

[pfalcon]

@carlescufi

should not preclude the option of later setting those via IDE

That's somewhat ambiguous. My understanding is that folks like @SebastianBoe, @mbolivar treat CMake as an abstraction layer with its own closed-circuit semantics which can generate a spec for an abstract, arbitrary build-worker system.

Of course, that's not going to work if we add rules for a particular build-worker system, namely, make, because it would contain things like qemu -serial $(if $(FOO),bar,baz), because only make can interpret that $(if $(FOO),bar,baz).

My arguments:

1. That would be done only to "p.2" run/test/work part of the system.
2. p.2 part would be largely specific to any particular environment anyway, and variables for one "backend" won't make much sense for other. For example, how a GUI IDE user can appreciate the meaning of QEMU_PTY var?

[pfalcon]

I suspect that instead of a migration of variables from CMake to Make, we can do both in many cases and keep everyone happy.

I did in #5154 , but that's extra work, extra testing, extra maintenance. And for what? In most cases, those opts aren't useful outside of command line unix make env (IMHO).

[carlescufi]

@pfalcon

Of course, that's not going to work if we add rules for a particular build-worker system, namely, make, because it would contain things like qemu -serial $(if $(FOO),bar,baz), because only make can interpret that $(if $(FOO),bar,baz).

Maybe I'm missing something here, but I believe one can generate code in CMake, using the CMake language, that then gets translated to the particular build system chosen. So @SebastianBoe, instead of writing qemu -serial $(if $(FOO),bar,baz), can't we write something like if(FOO in baz) qemu --serial in CMake scripting language?
Or maybe use cmake -E env qemu --serial for these sort of things instead of Make, and then you make it cross-build system?

[pfalcon]

Maybe I'm missing something here, but I believe one can generate code in CMake, using the CMake language, that then gets translated to the particular build system chosen.

Maybe I'm missing something too. Per my understanding, CMake-based logic is static, in the sense that it happens once at cmake invocation. While we're talking about decision-making logic at each make's invocation. I didn't hear about CMake language transplier to JavaScript make/shell language, but maybe I'm missing its advanceness indeed.

[SebastianBoe]

I did in #5154 , but that's extra work, extra testing, extra maintenance. And for what? In most cases, those opts aren't useful outside of command line unix make env (IMHO).

The benefit of, e.g. the technique of having flash_jlink, flash_openocd, targets instead of
a single target flash, which accepts make variables is that IDEs can know of the existance
of these targets and present them to the user.

As I said, we haven't started working on IDE support, so I don't have all the answers,
but out of curiosity, I tried it out with CLion, and indeed, CLion presented every
custom target in a list of possible things that could "Run"; ram_report, flash, run, xconfig, etc.

So you can rest assured that it is not for nothing that we keep CMake aware of how a project
can be configured.

[pfalcon]

So you can rest assured that it is not for nothing that we keep CMake aware of how a project
can be configured.

Sure, I'm well aware that I have limited insight into this stuff, that's why I'm posted this RFC, to have a wider discussion not directly tied to a particular value.

having flash_jlink, flash_openocd, targets instead of a single target flash

So, trying to translate that directly to "run" target, would "run_pty" work? It would, though "run" was an abstraction over original "qemu" to allow running across various emulators, but running over PTY is a QEMU feature, though other emul's could support that too, why not? But with QEMU_PIPE that wouldn't work right away, because that var specifies a pipe path (not just a present-means-bool value).

Overall, it's clear that it's more complex structure than "build" vs "non-build" parts, some non-build parts (user-facing ones, like flashing) could take further evolution (instead of just bringing back to Kbuild), while more obscure like QEMU_PTY might be just brought back to older semantics. And some are "hard", like V=1.

[nashif]

IMHO we are going back to how kbuild in zephyr was overloaded with things a build system is not supposed to be doing in the first place. Trying to make the build environment to act as a test-runner supporting different complex scenarios with different options, sometimes added adhoc does not scale and will make the build system very difficult to deal with, just like we had with kbuild.
We do have most of the data we need in different places and available in the build directory in form of configs, cache etc. so could actually have dedicated test scripts and runners consume this data and run on a built kernel in a more efficient and manageable way with as many options as possible. This is valid for all the qemu related scenarios and complex networking setups but not limited to qemu and simulator invocation, the same can be applied on flashing, console handling and others.

So lets not try and force cmake/make to do something they were not designed to do and try to step back and figure out a more sustainable and scalable solution.

(Hint: I have a POC already)