Allow inheritance of .bazelrc snippets/flags across projects

[armandomontanez]

Description of the feature request:

One of the side-effects of incompatible Bazel flags in the Bazel ecosystem is that some projects require certain incompatible flags to be set in order for the project to build at all. As middleware, managing these flags can be particularly tricky since you may require different incompatible flags depending on the version of your dependencies, and you also may prescribe your own required flags. These are forked in downstream users projects, and they may accidentally diverge in ways that cause subtle or confusing breakages as updates are rolled out. Because there's no way to import .bazelrc snippets from external dependencies, there's virtually no solution for inheritance at all and you just have to manually manage your own project's .bazelrc and hope for the best.

Perhaps the right way to handle this is to integrate it into bzlmod, as that mechanism strives to address similar problems that existed in WORKSPACE files. Admittedly, this would likely be a massive undertaking to design and implement.

Which category does this issue belong to?

External Dependency

What underlying problem are you trying to solve with this feature?

Inconsistencies in .bazelrc requirements across dependent projects.

Which operating system are you running Bazel on?

No response

What is the output of bazel info release?

No response

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

[armandomontanez]

As a concrete example, --incompatible_default_to_explicit_init_py is required for Pigweed to function correctly. Rolling out the change that required that involved updating various downstream projects to include that flag as well. The next time we have a similar required flag (which is immediately on the horizon), we'll need to do this piecemeal again, which is an unfortunate maintenance burden that we can't just fix once via a nice little import @pigweed//:required_bazelrc or similar.

[Wyverald]

cc @gregestren who's working on related issues.

[fmeum]

Rolling out the change that required that involved updating various downstream projects to include that flag as well.

Could you explain what you mean by that? The .bazelrc files of external repos aren't used by Bazel, so flipping the flag in Pigweed's own .bazelrc should be sufficient.

[armandomontanez]

The .bazelrc files of external repos aren't used by Bazel, so flipping the flag in Pigweed's own .bazelrc should be sufficient.

Right, it's sufficient for Pigweed to build, but then making said incompatible change with the associated flag then breaks downstream projects, which is the entire problem. When I roll downstream users, I have to make sure that when I update their reference to Pigweed I also update their .bazelrc with new required flags. If I could instead have a list of "flags required to successfully build pigweed," then whenever I roll Pigweed in downstream projects the'll implicitly pick up the new list.

Here's a step-by-step example:

1. @pigweed//:libfoo has a known issue
2. @pigweed//:libfoo is fixed, but requires setting --incompatible_default_to_explicit_init_py Bazel flag in Pigweed's .bazelrc.
3. Someone tries to build downstream project @myproj against the new version of Pigweed, and it breaks.
4. Downstream project @myproj updates their .bazelrc to include --incompatible_default_to_explicit_init_py as part of the update to @pigweed, and their build succeeds again.

This required manual intervention isn't just inconvenient; sometimes the breakages are extremely subtle and don't manifest until long after the update. We end up with a lot of fragmentation across various .bazelrc files since there's no way to to scalably and maintainably centralize these critical required flags as they evolve over time.

[fmeum]

Including a .bazelrc snippet for projects using pigweed could have the reverse problem though: By rolling forward pigweed, the downstream project could break if it's incompatible with the new flag.

Could you add a config_setting matching the required flag value and then add a select on it with a descriptive no_match_error to your core targets? That way users would see the error if the flag values mismatch, but would still need to take explicit action (which avoids spooky action at a distance).

[gregestren]

I appreciate the basic issue.

I also don't know what the right resolution is.

Knowing if a flag is safe has to be a combined assessment of Pigweed and the downstream project, right? Each for their own reasons.

If we could more easily model these requirements (which we can certainly think about), what's the right way for Bazel to communicate to the user how the cumulative constraints resolve? And how automatic do we want flag flips to be, without users explicitly reviewing / setting them?

[armandomontanez]

The risks of magically having flags change under your feet is both a scary prospect, and in some cases what you really want to happen for specific dependencies. But you're absolutely right; only the specific downstream project can correctly make the call on when this should happen! Here's two ideas that might spark more ideas:

Make flag inheritance opt-in, and tracked in MODULE.bazel.lock

By default, you wouldn't magically inherit flags. You could, however, explicitly opt-in to flag inheritance for a specific dependency. Doing that causes additional metadata added to your MODULE.bazel.lock to track inherited flags. Whenever you roll a dependency forwards and gain/lose flags, those changes would be reflected locally in your MODULE.bazel.lock.

Emit warnings when flags required by dependencies aren't satisfied

I think @fmeum touched on a great idea here with config_setting, thought it's maybe a little too rigid as it leaves downstream projects no choice whatsoever. Perhaps the right answer is to just have Bazel emit warnings when flag requirements of a dependency are not met, and have ways to silence said warnings if the difference is intentional. Since warnings are often ignored, though, this is only particularly helpful for people who pay attention to them. You could create additional suppression flags and config_settings today to track if a flag is both unset and suppressed, but you still have to inject that somewhere in a dependency path that users will hit it.

Actual requirements

If we boil this down to basic requirements, in my head this should address the root problem:

• If I add or update a dependency, and they require changes in specific flags to operate correctly, I should be able to know this through the action of updating my dependencies.
• If I believe a downstream project has erroneously required a specific flag, I should have a way to explicitly override that requirement.
• [bonus] I don't have to manually duplicate every required flag into my project's .bazelrc.