Proposal: remove per-module targets

[mlugg]

#18160 was responsible for expanding Zig modules to include lots of state, such as link libraries, C source files, and optimization mode. One such property which can now be set on a per-module basis is the target.

Clearly, targets with (for instance) different object formats cannot be combined into a single output executable; it is also unclear whether there is a sane way to combine, for instance, different CPU architectures in one executable. The real stated use case in #18160 for this feature is to allow modules to be compiled with different CPU feature sets.

Presumably, the idea here is that a piece of source code can be compiled N times with N different CPU feature sets, allowing a single binary to run correctly and optimally across a variety of CPU models. However, the problem with this is that Zig does not allow a single file to exist in multiple modules. As such, this use case immediately breaks down: you cannot compile the same code N times for different CPU feature sets using modules alone.

Meanwhile, modules having different targets introduces other complexities. Clearly, this is difficult to support in the compiler; indeed, it doesn't really work today at all. Additionally, it introduces the problem that the standard library either cannot be shared, or needs to be compiled for the "least feature set" of all targets. Aside from code bloat, non-sharing of the standard library leads to the same usability issues as the ability to have one file in multiple modules would; namely, that std.ArrayListUnmanaged(u8) from module A is not necessarily the same type as std.ArrayListUnmanaged(u8) from module B. This leads to another issue with modules having different targets: it is unclear how these modules are actually allowed to interact. Can datastructures be exchanged between them?

Given that this feature fails to solve the problem it sets out to solve, and introduces many more issues and complexities along the way, I propose to remove this feature. The target should become a global option; for the build system, it should be set on std.Build.Step.Compile rather than on std.Build.Module.

Notes

• The use case of having one binary which supports multiple CPU feature sets can be satisfied by compiling your application's hot path as an object file N separate times, and having your main application link all of those object files, calling the hot path over a well-defined API boundary.
• This proposal does not suggest removing per-module optimization modes. While this feature is still not solved -- for instance, it is unclear what optimization mode the standard library should have -- it has clear utility, unlike per-module targets, and is strictly less problematic.

[aqrit]

compile/link object file N separate times

Note for dynamic dispatch, if anybody else is wondering how this might work:
I assume this requires custom name mangling via @export and std.fmt.comptimePrint().

[GalaxyShard]

Clearly, targets with (for instance) different object formats cannot be combined into a single output executable; it is also unclear whether there is a sane way to combine, for instance, different CPU architectures in one executable. The real stated use case in #18160 for this feature is to allow modules to be compiled with different CPU feature sets.

What about executables with a mix of arm and thumb? I think this could be important for embedded development, or in my case, homebrew for the Nintendo DS. I haven't actually tried any code that does mix arm and thumb, so I'm not sure what the current behavior is, but intuitively I would think to just use different targets in different modules to switch between the two architectures.

[andrewrk]

Real world use case: https://jonot.me/posts/zig-gba/#thumb-codearm-code

[alexrp]

Real world use case: https://jonot.me/posts/zig-gba/#thumb-codearm-code

Can this use case be covered by just having the arm_interrupt calling convention force the function to be in Arm mode? Alternatively, a builtin function?

Per-module target for this feels like overkill.

[andrewrk]

That's a good suggestion for this use case.

Another use case is #1018

[mlugg]

Thumb/ARM is essentially the only valid use case I've really seen, and it feels like something that should be handled outside of this system. As suggested by Alex, a builtin (@setInstructionSet?) to set it on a per-function basis is more flexible (well, you can do all the same stuff, but you can do it without having to create a module boundary), and doesn't bring all of the problems described in this proposal, because it keeps the feature restricted in scope.

Another use case is #1018

I think I covered this in the original issue:

Presumably, the idea here is that a piece of source code can be compiled N times with N different CPU feature sets, allowing a single binary to run correctly and optimally across a variety of CPU models. However, the problem with this is that Zig does not allow a single file to exist in multiple modules. As such, this use case immediately breaks down: you cannot compile the same code N times for different CPU feature sets using modules alone.

I'm up for having a language-level way to do this rather than requiring multiple CUs, but trying to achieve it with per-module targets feels like at best a local maximum.

[gkgoat1]

#18160 was responsible for expanding Zig modules to include lots of state, such as link libraries, C source files, and optimization mode. One such property which can now be set on a per-module basis is the target.

Clearly, targets with (for instance) different object formats cannot be combined into a single output executable; it is also unclear whether there is a sane way to combine, for instance, different CPU architectures in one executable. The real stated use case in #18160 for this feature is to allow modules to be compiled with different CPU feature sets.

A compiler from WASM to c/zig would be a sane way to combine WASM with other architectures in one executable.

Presumably, the idea here is that a piece of source code can be compiled N times with N different CPU feature sets, allowing a single binary to run correctly and optimally across a variety of CPU models. However, the problem with this is that Zig does not allow a single file to exist in multiple modules. As such, this use case immediately breaks down: you cannot compile the same code N times for different CPU feature sets using modules alone.

Meanwhile, modules having different targets introduces other complexities. Clearly, this is difficult to support in the compiler; indeed, it doesn't really work today at all. Additionally, it introduces the problem that the standard library either cannot be shared, or needs to be compiled for the "least feature set" of all targets. Aside from code bloat, non-sharing of the standard library leads to the same usability issues as the ability to have one file in multiple modules would; namely, that std.ArrayListUnmanaged(u8) from module A is not necessarily the same type as std.ArrayListUnmanaged(u8) from module B. This leads to another issue with modules having different targets: it is unclear how these modules are actually allowed to interact. Can datastructures be exchanged between them?

Given that this feature fails to solve the problem it sets out to solve, and introduces many more issues and complexities along the way, I propose to remove this feature. The target should become a global option; for the build system, it should be set on std.Build.Step.Compile rather than on std.Build.Module.

I, however, agree with setting targets in std.Build.Step.Compile; it would permit WASM and host code to coexist within one module. However, setting targets within both would allow third-party modules to compile to WASM within a bigger app, with the caller module's obligation to compile it to/embed it within code for its target, and provide a host environment for said WASM.

Notes

• The use case of having one binary which supports multiple CPU feature sets can be satisfied by compiling your application's hot path as an object file N separate times, and having your main application link all of those object files, calling the hot path over a well-defined API boundary.
• This proposal does not suggest removing per-module optimization modes. While this feature is still not solved -- for instance, it is unclear what optimization mode the standard library should have -- it has clear utility, unlike per-module targets, and is strictly less problematic.