incompatible_disallow_unsound_directory_outputs

[tjgq]

This flag, which I'm planning to flip for Bazel 7, forbids an action from creating a directory where a file is expected (which is unsound, as Bazel will not accurately track the directory contents). Such an action will produce the following error message:

ERROR: /Users/tjgq/example/BUILD:1:8: output 'out' of //:target is a directory; dependency checking of directories is unsound

The only native rule for which this is likely to be a problem is genrule through its outs attribute. For Starlark rules, the ctx.actions.declare_file, attr.output and attr.output_list APIs, as well as the deprecated outputs parameter to the rule constructor, all declare output files. An output directory must instead be declared through the ctx.actions.declare_directory API, which offending rules will have to migrate to.

[allsey87]

@tjgq what is the workaround for the use case outlined in #23015?

[tjgq]

@allsey87 Use the path to a file as the $(location ...) argument, then manipulate it in your cc_binary to obtain the parent directory.

[allsey87]

That would work for a cc_binary that you can modify or patch, but it doesn't solve the problem in general. Consider the following situation using the Meson build system via rules_foreign_cc. Here I need to set the environment variable PYTHONPATH to the directory containing Cython:

cython.BUILD

exports_files(["."])

numpy.BUILD

meson(
    name = "numpy",
    build_data = [
        "@cython//:.",
    ],
    env = {
        "PYTHONPATH": "$(execpath @cython//:.)",
    },
)

I have admitted most fields for brevity, but the above code currently works but results in the "dependency checking of directories is unsound" warning. Is there an alternative solution to this problem?

[fmeum]

What happens if you add two labels to build_data, one for all files and one for a single file, and then reference the single file via location with /.. appended as needed?

[tjgq]

What @fmeum said is a valid workaround; another is to set the --host_jvm_args=-DBAZEL_TRACK_SOURCE_DIRECTORIES=1 startup option, which would make source directories incrementally correct and disable the warning.

But more importantly: I've just realized that your example is a source directory, not an output directory, so it's unaffected by this flag flip, which only affects the latter.

[allsey87]

@fmeum do you mean something like:

cython.BUILD

# this file always exists
exports_files(["BUILD.bazel"])

numpy.BUILD

meson(
    name = "numpy",
    build_data = [
        "@cython//:BUILD.bazel",
    ],
    env = {
        "PYTHONPATH": "$(execpath @cython//:BUILD.bazel)/.."
    },
)

This would assign PYTHONPATH to something like:

PYTHONPATH=/home/developer/.cache/bazel/_bazel_developer/25e07d78077dfe1eca932359d50e41ef/sandbox/processwrapper-sandbox/10/execroot/_main/external/cython/BUILD.bazel/..

Which is not a valid path under Linux and results in the Python module not being found. I also tried placing the ../ inside build data as follows:

build_data = [
      "@cython//:BUILD.bazel/..",
],

However, this gives errors such as: invalid label '@cython//:BUILD.bazel/..' in element 0 of attribute 'build_data' in 'meson' rule: invalid target name 'BUILD.bazel/..': target names may not contain up-level references '..'

Perhaps I have misunderstood your workaround/suggestion?

[allsey87]

@tjgq setting --host_jvm_args=-DBAZEL_TRACK_SOURCE_DIRECTORIES=1 works but gives somewhere between 100 and 1000 warnings that state Directory artifact crosses package boundary into package rooted at bazel-builder-bazel/external/bazel_tools/X/Y/Z. Note the double space after "artifact".

Sample of these warnings:

WARNING: Directory artifact  crosses package boundary into package rooted at bazel-builder-bazel/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/r8
WARNING: Directory artifact  crosses package boundary into package rooted at bazel-builder-bazel/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/proto
WARNING: Directory artifact  crosses package boundary into package rooted at bazel-builder-bazel/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/idlclass
WARNING: Directory artifact  crosses package boundary into package rooted at bazel-builder-bazel/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/dexer
WARNING: Directory artifact  crosses package boundary into package rooted at bazel-builder-bazel/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/ziputils
WARNING: Directory artifact  crosses package boundary into package rooted at bazel-builder-bazel/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/desugar
WARNING: Directory artifact  crosses package boundary into package rooted at bazel-builder-bazel/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/incrementaldeployment
WARNING: Directory artifact  crosses package boundary into package rooted at bazel-builder-bazel/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/desugar/scan

I think the blank artifact name is the same item being referenced in the input '' warning below, when --host_jvm_args=-DBAZEL_TRACK_SOURCE_DIRECTORIES=0

WARNING: /home/developer/.cache/bazel/_bazel_developer/25e07d78077dfe1eca932359d50e41ef/external/numpy/BUILD.bazel:33:6: input '' of @@numpy//:numpy is a directory; dependency checking of directories is unsound

[fmeum]

You are right, this only works if the tool consuming the variable cleans the path before interpreting it (although that's arguably not correct).

An alternative would be to extract CYTHON = Label("@cython//:BUILD.bazel") into a Starlark constant in a .bzl file and then use CYTHON.workspace_name or CYTHON.workspace_name + "/" + CYTHON.package in BUILD files to construct the path. This only works for source files though.