Allow configure_file, custom_target and generator output file to subdirectories under builddir

[jasonszang]

I've encountered two cases that would require configurating / generating file into subdirectories.

1. Project configuration header that would be included from other headers in the same project

Say I have a library project named foo and the project layout is like this:

include/
    foo/
        bar/
            foobar/
                foobar.h # includes config.h
            bar.h        # includes config.h
        foo.h            # includes config.h
        config.h.in # configured by meson
src/
    foo/
        foo.cpp
...

We would want our client code use #include <foo/config.h> to include our config.h since we don't install config.h directly to includedir. But for our own code this would be impossible since generated config.h is directly under <builddir>, not <builddir/foo>.

Of course, for foo.cpp the work-around is easy: just #include <config.h>. But it becomes complicated we need to include config.h from bar.h and foobar.h.

In fact, there is no way to correctly include config.h that both work in project directory and after installation. If we want bar.h and foobar.h to be correct after installation we must #include <foo/config.h> or use relative path in them. Including <foo/config.h> won't compile, and although relative path happens to work for bar.h, it won't for foobar.h. Neither will it work if we reside in a subdir. Using #include <config.h> allows us to compile, but bar.h and foobar.h breaks up after installation.

The correctly way to configure and include config.h should be just generate it under builddir/foo and #include <foo/config.h> everywhere, just as if it were in the position of config.h.in. Currently meson does not allow this, and forces a flat include directory, or hacking into builddir.

2. Protobuf generation

This one is simpler. If we have some protos like

protos/
    foo/
        foo.proto
    bar.proto

in which bar.proto imports foo.proto.
it would be nice if we could use one generator for both protos and and get under build directory something like this:
For the compiled protos to work the directory structure of generated sources must mirror the protos, because bar.pb.h will #include "foo/foo.pb.h". The generated sources must look like this:

protos/
    foo/
        foo.pb.h
        foo.pb.cc
    bar.pb.h
    bar.pb.cc

Althouth this would also require something like @dirname@ substitution if we are to pass generated sources to build targets properly, but without supporting generating source trees we cannot use protos with subdirectories. This also prevents wrapping of existing projects that use protos this way, which is quite a common practice.

PS: I suggest putting every generated file from configure_file() and custom_target() (including ones in subdirs) into one directory tree named something like <builddir>/sources@gen, add it to include directory, and allow outputing to subdirectories under it. This should avoid potential name clashes and give developers control over generated folder layout.

[rom1v]

I think I have a similar case: https://stackoverflow.com/questions/46729488/how-to-compile-aidl-files-using-meson

I need these source files:

• src/com/rom1v/example/Service.aidl
• src/another/package/Other.aidl

to be compiled under:

• builddir/gen/com/rom1v/example/Service.java
• builddir/gen/another/package/Other.java

From the java compiler point of view, this is an error for a java file not be in a directory matching the package name.

[liugang]

#2548 is possible way to fix this type of issue.

if one project call configure_file() with install_dir, install_headers(), and so on. copy these header files to a include_intermediates_dir, and all compile target include this intermediate directory.

[jasonszang]

@nirbheek
Would you please be kind enough to explain to me the considerations behind the design choice of Output must not contain a path segment. in source generation targets like custom_target and why are generating code into subdirectories of build dir considered harmful?
I am trying to figure out a way to properly compile protobuf proto files with custom_target (by properly I mean get meson to figure out the dependency relations correctly), which I would like to use in a small production project which for some reason does not fit in our in-house build system. I acturally found a way that work with proto files that has no subdirectories (all protos must live in a single folder), but proto subtrees like the one I wrote at the top can not be built by meson. Then I commented out these two lines in build.py

    if '/' in i:
        raise InvalidArguments('Output must not contain a path segment.')

And it worked surprisingly well. But I'm afraid by doing this I may be breaking something else inside meson. Could you please give me some explanation about this limitation and let me know your thought about its removal (or replacement with something like must not contain .. component if escaping is the problem)? With this limitation proto trees with subdirectories seems never could be properly built by meson.

[rib]

Just to mention that I recently (#3023) also found it a little awkward that configure_file()'s output filename couldn't include a subdirectory while I wanted to generate a .java file based on some build time config options. The javac compiler and Meson's jar() require source code to be laid out according to the namespace of the Java package.

In my case it's possible to work around the limitation by creating a meson.build under src/java/com/impossible/glimpse/meson.build where I use configure_file() which results in a file like <builddir>/src/java/com/impossible/glimpse/<output file>

I wonder if the subdirectory were limited to matching the subdirectory of the input filename that might appease some of the general concern around having random control over the build directory layout. This would still allow meson control to change the top level structure of the build directory but it would have to preserve the subdirectories requested for an output file.

E.g. if I want to generate a GlimpseConfig.java from GlimpseConfig.java.in in the package com.impossible.glimpse with a file under src/java/com/impossible/glimpse/GlimpseConfig.java.in then ideally I could do:

configure_file(input: 'com/impossible/glimpse/GlimpseConfig.java.in',
        output: 'com/impossible/glimpse/GlimpseConfig.java',
        configuration: conf_data)

And in the build directory meson can create:
<build>/any/meson/private/top/level/com/impossible/glimpse/GlimpseConfig.java

The top level layout doesn't really matter (at least in my case) so long as the java compiler + ninja backends knows what it is.

So it might not imply spaghetti access to arbitrary build directories, just some control to build a sub-heirachy within the build directory.

[NickeZ]

Also ran into this problem with custom_target and doxygen. The call to doxygen will produce the html files needed and the latex sources needed. Doxygen also produces a Makefile that helps you run pdflatex the right amount of times and in the right order.

I would like to write something like this:

cdata = configuration_data()
cdata.set('TOP_SRCDIR', meson.source_root())
cdata.set('TOP_BUILDDIR', meson.build_root())
cdata.set('VERSION', meson.project_version())

doxyfile = configure_file(
    input: 'Doxyfile.in',
    output: 'Doxyfile',
    configuration: cdata,
    install: false
)

html_target = custom_target(
    'htmldocs',
    build_by_default: false,
    input: doxyfile,
    output: ['html/index.html', 'latex/Makefile'],
    command: [doxygen, doxyfile],
)

pdf_target = custom_target(
    'pdfdocs',
    build_by_default: false,
    input: 'latex/Makefile',
    output: 'latex/refman.pdf',
    command: [make, '-C', '@OUTDIR@/latex'],
)

docs/meson.build:15:0: ERROR: Output must not contain a path segment.

[nussbrot]

I also ran into this problem and currently don't know how to work around it.
I'm evaluating meson and am trying to figure out if i can use it to build a FPGA project.
One of the problems is IP-generation.

This usually works like this: You generate something like a IP-variation file, which is some kind of XML format, good for versioning and then you run a IP-generator on this file and get a bunch of output files.

Example output files for generating the IP cms_sys_pll:

cms_sys_pll.cmp
cms_sys_pll.csv
cms_sys_pll.html
cms_sys_pll.spd
cms_sys_pll.xml
cms_sys_pll_generation.rpt
simulation\aldec\rivierapro_setup.tcl
simulation\cadence\cds.lib
simulation\cadence\cds_libs\pll_0.cds.lib
simulation\cadence\hdl.var
simulation\cadence\ncsim_setup.sh
simulation\cms_sys_pll.sip
simulation\cms_sys_pll.v
simulation\mentor\msim_setup.tcl
simulation\submodules\cms_sys_pll_pll_0.vo
simulation\synopsys\vcs\vcs_setup.sh
simulation\synopsys\vcsmx\synopsys_sim.setup
simulation\synopsys\vcsmx\vcsmx_setup.sh
synthesis\cms_sys_pll.debuginfo
synthesis\cms_sys_pll.qip
synthesis\cms_sys_pll.vhd
synthesis\submodules\cms_sys_pll_pll_0.qip
synthesis\submodules\cms_sys_pll_pll_0.v

So i tried to put this into a cutom target like so:

qsys = find_program('qsys-generate')
python = import('python').find_installation('python')

files = run_command(
  python, 'cat.py', files('ip/cms_sys_pll.outputs'),
).stdout().strip().split('\n')

custom_target('cms_sys_pll',
  build_by_default: true,
  output: files,
  input: 'ip/cms_sys_pll/cms_sys_pll.qsys',
  command: [
      qsys,
      '--output-directory=@OUTDIR@/cms_sys_pll',
      '--synthesis=VHDL',
      '--simulation=VERILOG',
      '@INPUT@']
)

And of course i ran into the error:

Output 'simulation\\aldec\\rivierapro_setup.tcl' must not contain a path segment.

Do you have any suggestions to work around this issue?

[jasonszang]

Well personally I decided to give up meson and went back to the good old CMake because of this one. And I find modern style CMake comfortable enough I stopped looking for another build system generator.
And unfortunately I am not aware of any universal work-around beside removing this limitation from meson.

[nussbrot]

Hi @jasonszang, thanks for your answer. I see this issue was first posted by you. If you don't mind i would like to ask you if you have experience with CMake and let's say "custom toolchains"?
But it is really off-topic and this is probably not the right place to ask you that.

[jpakkane]

The question here is does the file really need to be in a cm_sys_pll subdirectory (some tools are nasty in this way). If no, then write it to @OUTDIR@. If yes, then you need to have the custom target declaration in the cm_sys_pll subdirectory (and also have just --outdir=@OUTDIR@. In Meson all outputs go to the build directory that corresponds to the source directory they are declared in. This simplifies debugging build problems, since you always know that files in build directory X must come from the corresponding source dir X.

[nussbrot]

Yeah, you're right, that is a mistake in my build file. I can give the tool any directory, so --outdir=@OUTDIR@ works fine. But i can not control the subfolders like synthesis\... or simulation\... that are created in @OUTDIR@.
So i can not add the resulting files as outputs since stuff like output: 'synthesis\cms_sys_pll.qip' does not work.

[ptsneves]

@jasonszang I have the same frustration. One way to work around it is that you create a sub meson.buil in the subdir and do the remaining thing there. Then subdir the newly created meson.build. Not optimal but i think it works

[valpackett]

gir-to-d also generates subdirectories, like

somelib/c/types.d
somelib/c/functions.d
somelib/Obj1.d
somelib/Thingy.d

Why is this ridiculous limitation still present? >_<

UPD: seems like there are workarounds in gir-to-d for this

[zasdfgbnm]

For this specific case you can create a resources subdirectory and have the custom target to do that in that dir's meson.build file. You can either first recurse to the runtime dir and do a files there or just directly specify the source files as '../runtime/a.cu`.

This workaround is not perfect, but sounds acceptable. Thanks for the suggestion!

[gabeblack]

There's been several occasions where I've had to wrap my custom_target with a script that copies the generated output in to the custom_target's working directory so that it meet's the custom_target's requirement of the output field not containing a path or subdirectory.

I've been porting the building of source rpms to building with meson. As you might know, source rpms have a structure of rpmbuild_dir/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}. The final installation of building the source goes in the BUILDROOT folder often with a structure matching what one would see in their filesystem (libs go in BUILDROOT/<pkg>/usr/lib64 and headers in BUILDROOT/<pkg>/usr/include).

However with meson's custom_target, I'm forced to copy/flatten the generated directory structure's include files and libraries in to the directory where my meson.build's custom_target lives because of the 'no subdirectory' requirement.

I guess this just reeks of a code smell -- but I'm getting convinced that it isn't in our meson.build files, but rather meson's design in this area. I like the restrictions that meson does in other areas as I think it helps you re-think your design and structure to make sure you do things correctly and cleanly. However, in respect of having to port third-party code generators and now rpmbuild... I think this restriction is actually a code-smell in meson instead. If we could just simply specify our outputs in the outputs attribute of custom_target, we wouldn't have to add wrapper scripts to code generators (to copy the outputs in the flat directory structure). We wouldn't have to have copies laying of libraries, sources, and header files at all! The code smell is having to do the copies...

I like how meson tries to prevent you from having to 'hack' the build system to get it to work, but I think having to 'hack' rpmbuild and other tools to flatten out all their outputs to the calling custom_target directory is just as bad.

I admit I may not be aware of a meson feature that allows me to do what I'm doing, so I'd love to hear suggestions of meson features that would let me work around having to copy around the output of building source rpms to be able to have those headers and libraries used as part of other build targets.

As you can see from other comments in this thread, the restriction doesn't just apply to rpmbuild, but there are many other code generators and other external build tools that create outputs in a nice, clean subdirectory structure ready for user consumption. Meson should not be requiring that clean structure to be flattened or that directory structure to be pre-existing (so you can put meson.build files in them) in order to be consumed.

[eli-schwartz]

As you can see from other comments in this thread, the restriction doesn't just apply to rpmbuild, but there are many other code generators and other external build tools that create outputs in a nice, clean subdirectory structure ready for user consumption.

My interest here is for code generators and external build tools that require structures as part of their design intrinsics, such as generated headers which must match the #include <...> structure (solvable with a specific variant of generator() that doesn't apply to non-C-headers). And more generally things like rust or python require specific output structures in order to be usable, even at compile time.

Producing a declare_dependency() which is interchangeable with the installed version is also important, mainly for external wraps where you cannot just arbitrarily re-order public headers into the common include/* & src/ pattern because you don't own that structure.

However I'm not certain I understand the case for rpmbuild. For rpmbuild, you should be invoking meson install --destdir=BUILDROOT/<pkg>/ for example using these rpmbuild macros:

meson/data/macros.meson

Lines 1 to 45 in 5e76e2a

	%__meson %{_bindir}/meson
	%__meson_wrap_mode nodownload
	%__meson_auto_features enabled
	%meson \
	%set_build_flags \
	%{shrink:%{__meson} setup \
	--buildtype=plain \
	--prefix=%{_prefix} \
	--libdir=%{_libdir} \
	--libexecdir=%{_libexecdir} \
	--bindir=%{_bindir} \
	--sbindir=%{_sbindir} \
	--includedir=%{_includedir} \
	--datadir=%{_datadir} \
	--mandir=%{_mandir} \
	--infodir=%{_infodir} \
	--localedir=%{_datadir}/locale \
	--sysconfdir=%{_sysconfdir} \
	--localstatedir=%{_localstatedir} \
	--sharedstatedir=%{_sharedstatedir} \
	--wrap-mode=%{__meson_wrap_mode} \
	--auto-features=%{__meson_auto_features} \
	%{_vpath_srcdir} %{_vpath_builddir} \
	%{nil}}
	%meson_build \
	%{shrink:%{__meson} compile \
	-C %{_vpath_builddir} \
	-j %{_smp_build_ncpus} \
	--verbose \
	%{nil}}
	%meson_install \
	%{shrink:DESTDIR=%{buildroot} %{__meson} install \
	-C %{_vpath_builddir} \
	--no-rebuild \
	%{nil}}
	%meson_test \
	%{shrink:%{__meson} test \
	-C %{_vpath_builddir} \
	--num-processes %{_smp_build_ncpus} \
	--print-errorlogs \
	%{nil}}

[gabeblack]

However I'm not certain I understand the case for rpmbuild. For rpmbuild, you should be invoking meson install --destdir=BUILDROOT// for example using these rpmbuild macros:

I was unclear, I apologize. I was giving an example of using rpmbuild to build the source code of certain packages, not to use it to generate an rpm. By defining _topdir you can use it to compile a source rpm and stage and install it in a temporary directory. You can then point other projects to those generated includes/libraries -- all within a folder that is temporary (i.e. without installing the rpm in the host environment (often requiring sudo privileges)).

It was probably a bad example, but I'm finding more and more third-party tools that stage built libraries and headers in a directory structure. AFAIK, meson doesn't allow you to consume it without copying the files to either pre-existing folders, or to copy the files to the custom_target folder.

Am I mistaken? Do you know if there any plans to allow output to allow subdirectories?

[eli-schwartz]

It was probably a bad example, but I'm finding more and more third-party tools that stage built libraries and headers in a directory structure. AFAIK, meson doesn't allow you to consume it without copying the files to either pre-existing folders, or to copy the files to the custom_target folder.

Meson does provide import('pkgconfig').generate(mylib) which generates pkg-config files for installation e.g. in installable rpms. This function also automatically creates "-uninstalled.pc" files which can be used by adding builddir/meson-uninstalled/ to your $PKG_CONFIG_PATH -- pkg-config will pick "uninstalled" projects up first, and emit compile/link flags for the built libraries in the builddir and the headers in the source/build dir. This usually works pretty well.

Am I mistaken? Do you know if there any plans to allow output to allow subdirectories?

I would like to allow it for the reasons, and using the approach, which I outlined in #2320 (comment) and which @jpakkane tweaked in #2320 (comment)

I would like to know from @jpakkane if that is an acceptable approach and worth my investing time to implement it.

[gabeblack]

Personally, I like what you have outlined in #2320 (comment) much better. It is more intuitive. People on this thread are already naturally looking to add paths to the output argument. IMO, if the API is natural where you do it without looking up docs, then it is a good API.

Having another argument like is_opaque_directory is not very intuitive and adding other accessor methods on top of that I think would require much time for us consumers looking at documentation trying to figure out what that all means.

That is my opinion though, and I respect your decisions as you are the ones maintaining and know better the side-effects of all this.

I wouldn't mind putting in some skin in the game and helping implement the addition.

[intractabilis]

Here is another scenario where I think I need paths in the custom_target's output argument. I am seeking a way to generate C++ bindings for GTK on the fly. The tool, cppgir, generates hundreds of files for a single library. For example, for GTK4, I would want C++ bindings in the build tree's gi directory:

cppgir --output gi --class Gtk-4.0

This command will generate 1536 files. I need to compile the following generated files out of these 1536:

• gi/cairo/cairo.cpp
• gi/freetype2/freetype2.cpp
• gi/gdk/gdk.cpp
• gi/gdkpixbuf/gdkpixbuf.cpp
• gi/gio/gio.cpp
• gi/glib/glib.cpp
• gi/gmodule/gmodule.cpp
• gi/gobject/gobject.cpp
• gi/graphene/graphene.cpp
• gi/gsk/gsk.cpp
• gi/gtk/gtk.cpp
• gi/harfbuzz/harfbuzz.cpp
• gi/pango/pango.cpp
• gi/pangocairo/pangocairo.cpp

How do I pass these files as sources to a library function? They don't exist during the setup stage, so library fails if I specify direct paths to these files.

[slavaandrejev]

Here is my workaround for this issue that generates GTK C++ bindings. It isn't lovely. It involves copying the generated source files into one directory, generating a dummy header file, and using directories as an output of custom_target.

[denizzzka]

I ran into this problem by unusual way:

Since cmake builds broken for my case (#13390) I tried to call cmake as custom_target

Build results is placed into ./build_dir_name/lib/useful_file.a by cmake scripts, and, thus, it isn't accessible by Meson

[intractabilis]

If not for this, I had a chance to convince a huge company to switch to Meson.

[apache-hb]

I've found this to be a pain point as well. Could this functionality not be part of the fs module?

A function that accepts a dict(any) and uses it to create a directory hierarchy and copy files into the correct places, meson could still manage the private build dir for the target to prevent it clashing with top-level folder names.

fs = import('fs')

fs.copytree('bundle', {
    'config.ini': configure_file('config.ini.in', configuration : { 'name': 'example' }),
    'assets': {
        'icon': compressed_icon,
        'redist': {
            'd3d12': {
                'D3D12Core.dll': files('data/redist/d3d12/D3D12Core.dll'),
                'D3D12Core.pdb': files('data/redist/d3d12/D3D12Core.pdb'),
                'D3D12SDKLayers.dll': files('data/redist/d3d12/D3D12SDKLayers.dll'),
                'd3d12SDKLayers.pdb': files('data/redist/d3d12/D3D12SDKLayers.pdb'),
            }
        }
    }
})

I originally tried using install_data to achieve this but running a full install every build doesn't scale well once assets get past a few GB in size.

[intractabilis]

Maybe Muon or Boson can implement this? They are both clones of Meson. Has anyone tried to approach them about this feature?

[tristan957]

What is boson?

[intractabilis]

What is boson?

It's a Meson clone written in C: https://sr.ht/~bl4ckb0ne/boson/

[lessismordaunt]

Still can't figure this out. Thought I could get away with using the stamp approach described above, but then when you try to declare a dependency on a .stamp file for an executable, the generated Makefile just ignores the dependency completely, I guess because it's not a recognised filetype. Why on Earth is there no way of declaring a "non-source" dependency for an executable??? There's extra_files but the description suggests that's just for IDE compat.

This restriction really just feels arbitrary at this point. 😢

[tristan957]

Still can't figure this out. Thought I could get away with using the stamp approach described above, but then when you try to declare a dependency on a .stamp file for an executable, the generated Makefile just ignores the dependency completely, I guess because it's not a recognised filetype. Why on Earth is there no way of declaring a "non-source" dependency for an executable??? There's extra_files but the description suggests that's just for IDE compat.

This restriction really just feels arbitrary at this point. 😢

https://mesonbuild.com/Reference-manual_functions.html#custom_target_depend_files

[germandiagogomez]

Shower thought: perhaps we could allow directories in the output filename, but only as long as that output directory segment

1. is never descended into using `subdir()`

2. does not contain a `../` component, it's a pure child of the current subdir

@eli-schwartz

I need this and there is no reasonable workaround with the current state of things. I tried to use generator to generate the output in a subdir. I need to generate like 15 files in 5 or 6 different subdirectories, like this:

some/proj/path/subdir1/
             f1.h.in
             f2.h.in
some/proj/path/subdir/
            f3.h.in

With generator I tried something like this:

# I should know the subdir where to generate the output file to when invoking the generator,
# but generating the output with the subdir depends on the file path itself, which cannot
# be done generically in the generator itself.
headers_generator = generator(my_compiler, arguments: [..., '@BUILD_DIR@'],
                                                     output: '@BASENAME@.h',
                                                     preserve_from_path: meson.current_source_dir())

# This will output in the wrong place.
my_headers = headers_generator.process([meson.current_source_dir() / 'subdir1/f1.h.in')

With a custom target it also works wrong, the output cannot be added subdirs of any kind.

Moving down in subdirs also will not work, since those subdir generated files are consumed into the parent dir, so the build dir is the one from the parent, it seems. Not sure if it is an internal detail.

I think that not being able to control the output path at least in the way proposed by you is by far the thing that more headaches gave me in Meson when using it for over 6 years (do not remember when I started to use it exactly).

I really think @jpakkane this should be fixed and it is a recurrent problem

[lessismordaunt]

Still can't figure this out. Thought I could get away with using the stamp approach described above, but then when you try to declare a dependency on a .stamp file for an executable, the generated Makefile just ignores the dependency completely, I guess because it's not a recognised filetype. Why on Earth is there no way of declaring a "non-source" dependency for an executable??? There's extra_files but the description suggests that's just for IDE compat.
This restriction really just feels arbitrary at this point. 😢

https://mesonbuild.com/Reference-manual_functions.html#custom_target_depend_files

Already looked at this - that's not what I'm looking for. Firstly, the documentation states that this allows adding dependencies that are not listed in the command keyword argument.
First, does that imply that dependencies are extracted from the command argument? That sounds incredibly fragile and unexpected.
Second, this seems to be a way of adding dependencies to a custom target, not an executable. I want to form a dependency between an executable and a custom target.

As I already mentioned, trying to add this dependency via a .stamp file source dependency is just ignored.

[Jehan]

@exposedcranium

Still can't figure this out. Thought I could get away with using the stamp approach described above, but then when you try to declare a dependency on a .stamp file for an executable, the generated Makefile just ignores the dependency completely, I guess because it's not a recognised filetype. Why on Earth is there no way of declaring a "non-source" dependency for an executable??? There's extra_files but the description suggests that's just for IDE compat.

Ahah I feel your pain because we went through the same thing. The solution we went with (in GIMP) is that it's not just a stamp file. It's a stamp header file.

Basically have a custom_target do whatever it needs to do (in our case, it generates a bunch of source code), then at the end, it creates a stamp header file. In our code, this header file is just an empty file with a comment containing the date so that the file actually changes.

Then any other target actually depending on it will actually wait or be rebuilt when appropriate.

This restriction really just feels arbitrary at this point. 😢

I agree. In the end, we usually eventually always find some weird workaround for every use case (we had quite a few headaches in GIMP before having meson finally cater to us), but it does always feel like jumping through hoops instead of having some more generic target-making and dependency rules.

meson is still great for a lot of things and we don't regret moving to it, but it for sure could be a lot better if some parts evolved (like this restriction indeed, or again #8123). 😛

@tristan957

https://mesonbuild.com/Reference-manual_functions.html#custom_target_depend_files

The various options to add file dependencies don't work because every rule out there has just a limited set of acceptable types. In particular here, @exposedcranium is not asking to make a @custom_target, they are talking about an executable, which likely means they wanted to use executable() or library(). These don't have depend_files option. So you have to go with tricks like making a fake (empty) header file as a source instead. This is this other issue where meson doesn't have a generic dependency rule (#8123). Everytime your build is a bit different than the standard usage, you often don't have an available option for your case, then you must request to have a new option (or to modify one) to support this special-case. It often means adding supporting a given type as a dependency. For instance, look at the custom_target rule you mention: it has input, depend_files, depends… So many options for just basically the same thing (one target depending on another, who cares what type they are?).

[virtuald]

I've been exploring porting my program to meson, and I'm also hitting this. In a lot of ways, any generator that is generating an arbitrary directory structure is roughly equivalent to tar -xf @INPUT@ -C @OUTDIR@... which not surprisingly, meson is going to be really unhappy with as it currently stands.

I think to make this easy to use, we have at least the following constraints/features needed:

• Should not need to specify all of the output files in the custom_target declaration, just the output directory
• Use individual generated files as inputs to other meson things (compiler, generator, whatever)
• Need to be able to mark the entire generated directory for installation

To meet those constaints, maybe it could look something like this (or overload custom_target).

myfiles = complex_generator(
   'name',
    input: 'foo.tar.gz',
    output: '@OUTDIR@',
    command: [tar, '-xf', '@INPUT@', '-C', '@OUTDIR@'],
)

foo_lib = library('foo', [myfiles / 'path' / 'to' / 'foo.cpp'])

The idea is that when you do this more complex generator thing, that the object it returns has an overloaded / operator that outputs the path to whatever the generated directory is -- and we also disallow referencing it from meson.current_build_dir(). This seems better than trying to determine which generated created some arbitrary path. This would also let meson cleanly infer what targets depend on each other without needing to do crazy path things.

I think the private output directory should not be the current build path (to avoid the subdir problems), but something artificial but recognizable. So maybe if the rule is in 'some/directory/path', meson would use _generated_some_directory_path/name.

It feels like ninja supports this sort of dynamic target using dyndep, but it's not quite right.

[gabeblack]

@virtuald -- you can already have output be a directory (as long as it isn't more than one level deep (e.g., NOT 'outdir/files', but just 'outdir'').

[virtuald]

@virtuald -- you can already have output be a directory (as long as it isn't more than one level deep (e.g., NOT 'outdir/files', but just 'outdir'').

You are correct, my example did not show multiple levels. I updated my proposed syntax to show multiple levels.

[madushan1000]

I came across this issue while trying to build some parts of aosp with meosn. Specifically tools like aconfig and aidl generates sources into arbitrary paths in the builddir. I was able to get it more or less working with java by removing this check.

diff --git a/mesonbuild/interpreter/type_checking.py b/mesonbuild/interpreter/type_checking.py
index ed34be950..b6c89fa72 100644
--- a/mesonbuild/interpreter/type_checking.py
+++ b/mesonbuild/interpreter/type_checking.py
@@ -327,8 +327,6 @@ def _output_validator(outputs: T.List[str]) -> T.Optional[str]:
             return 'Output must not be empty.'
         elif i.strip() == '':
             return 'Output must not consist only of whitespace.'
-        elif has_path_sep(i):
-            return f'Output {i!r} must not contain a path segment.'
         elif '@INPUT' in i:
             return f'output {i!r} contains "@INPUT", which is invalid. Did you mean "@PLAINNAME@" or "@BASENAME@?'