The functionality introduced in #5216 by @carterbox promises to be very useful on several feedstocks that need to "slice and dice" the result of a build into different outputs, so I started testing this for llvmdev in conda-forge/llvmdev-feedstock#283. This runs into a couple of different problems though.

files: does not take into account files already existing in host:

In llvmdev, we want to version the binaries (and some libraries), and create symlinks for the unversioned ones (that point to the versioned ones). These should go into different outputs. As such, llvm-tools-18 should contain the versioned binaries (except llvm-config), and llvm-tools should contain the symlinks. The following configuration expresses that intent.

  - name: llvm-tools-{{ major_ver }}
    files:
      include:
        - bin/*-{{ major_ver }}     # [unix]
      exclude:
        # belongs into llvmdev
        - bin/llvm-config*          # [unix]
      [...]

  - name: llvm-tools
    files:
      include:
        # opt-viewer tool is in share
        - bin/*                     # [unix]
        - share/*                   # [unix]
        - Library/bin/*.exe         # [win]
        - Library/share/*           # [win]
      exclude:
        # belongs into llvmdev
        - bin/llvm-config*          # [unix]
        - Library/bin/llvm-config*  # [win]
    requirements:
      host:
        - {{ pin_subpackage("llvm-tools-" ~ major_ver, exact=True) }}   # [not win]

However, what ends up happening is that llvm-tools repackages the output of llvm-tools-18, despite llvm-tools-18 being a host-dependence of llvm-tools (which should therefore be part of the baseline snapshot before installation that's the basis for determining what got installed on top).

This is a common setup, i.e. it's often desirable to slice off lib/libfoo.so and lib/libbar.so into separate outputs, and everything else in lib should go into yet another output. Similarly here, for the llvmdev output, we want to pick up everything that hasn't already been installed into other outputs. That too doesn't seem to work (i.e. bin/llvm-config ends up missing), using the seemingly obvious

  - name: llvmdev
    files:
      include:
        # everything not already in other outputs
        - "*"

Exclusion rule causes glob error

One output has:

  - name: libllvm{{ major_ver }}
    files:
      include:
        - lib/lib*                  # [unix]
      exclude:
        # separate output, see below
        - lib/libLLVM-C.*           # [unix]
        - Library/**/libLLVM-C.*    # [win]

on osx-arm, the exclusion causes a glob error:

ERROR: Glob $PREFIX/lib/libLLVM-C.* did not match in root_dir $PREFIX

It's worth noting that this file doesn't currently get installed, but if it ever does in the future, it should be excluded (hence the rule). Excluding files that aren't present needs to not cause errors.

Also, somehow the osx-arm case (cross-compiled from x64) manages to trigger something for the wrong architecture:

  File "/Users/runner/miniforge3/lib/python3.10/site-packages/conda_build/post.py", line 1635, in post_process_shared_lib
    mk_relative_osx(path, host_prefix, m, files=files, rpaths=rpaths)
  File "/Users/runner/miniforge3/lib/python3.10/site-packages/conda_build/os_utils/macho.py", line 221, in otool
    lines = check_output([otool, "-l", path], stderr=STDOUT).decode("utf-8")
  File "/Users/runner/miniforge3/lib/python3.10/subprocess.py", line 421, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/Users/runner/miniforge3/lib/python3.10/subprocess.py", line 503, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/Users/runner/miniforge3/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/Users/runner/miniforge3/lib/python3.10/subprocess.py", line 1863, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 86] Bad CPU type in executable: '$PREFIX/bin/llvm-otool'

That might be a corner case though, as we're packaging the binary that perhaps gets called by conda-build itself (if a symlink otool -> llvm-otool exists). However, this aspect didn't fail before switching to the files:-based approach.