Structured inputs for Rust (and more languages in the future, probably)

[dcbaker]

The fixed issue has most of the background on this, so I'm not going to repeat it, go look at #8725 for more info.

The goal of this series is to make it possible to use generated files with languages that do all of their compilation in one go (Rust, Cython, Zig), and rely on the layout of those files on disk for imports (Rust, Cython, Zig). For those languages we don't generate a single .o for each source file, the compilers and/or languages simply don't make that possible. Instead, we generate a singlle compilation target for all of those sources.

To that end this series introduces a new concept, the "structured input". This input is a dictionary mapping an arbitrary "root" to a directory tree (at the moment the target's private directory is used as the root), like so:

executable(
  'exe',
  {
    '': 'main.rs',
    'foo' : 'mod.rs',
    'bar': generated_mod.rs,
  }
)

Meson will ensure that at the root of the tree is main.rs, then a root/foo/mod.rs exists, as does root/bar/generated_mod.rs, it will do this before compiling mian.rs If necessary it will copy files (at build time) into the build tree. It does this at build time to avoid reconfiguring on file changes, and to ensure that the tree is always accurate. This is unnecessary if there are no generated sources, and simply passing executable('exe', 'main.rs') has not been changed.

Fixes #8725

[lgtm-com]

This pull request introduces 2 alerts when merging 9a0dd4c into be015a3 - view on LGTM.com

new alerts:

• 1 for Unused local variable
• 1 for Unused import

[jpakkane]

executable(
  'exe',
  {
    '': 'main.rs',
    'foo' : 'mod.rs',
    'bar': generated_mod.rs,
  }
)

This is unacceptable as a syntax. It is completely unreadable with the magic dictionaries and all.

[lgtm-com]

This pull request introduces 2 alerts when merging f396f76 into be015a3 - view on LGTM.com

new alerts:

• 1 for Unused local variable
• 1 for Unused import

[dcbaker]

Then do you have another idea on what the syntax should be, @jpakkane?

[dcbaker]

@jpakkane, how would you feel about a new function instead?:

sources = structured_source(
  foo_gen_pyx, 'foo_static.pyx',
  sub : bar_gen_pyx, 'bar_static.pyx',
  ...
)

target = py.module('foo', sources)

[jpakkane]

Sure, something like that makes a lot more sense as it is its own isolated thing.

[dcbaker]

This is going to need enough rework that I'm just giong to open a new PR for this.