Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loading language specific rules removed from Bazel #23043

Open
5 tasks
comius opened this issue Jul 19, 2024 · 2 comments
Open
5 tasks

Loading language specific rules removed from Bazel #23043

comius opened this issue Jul 19, 2024 · 2 comments
Assignees
Labels
incompatible-change Incompatible/breaking change P2 We'll consider working on this in future. (Assignee optional) team-Rules-API API for writing rules/aspects: providers, runfiles, actions, artifacts

Comments

@comius
Copy link
Contributor

comius commented Jul 19, 2024

Motivation

Language specific rules (Protos, Java, C++, Android) are being rewritten to Staralark and moved from Bazel into their rules repositories (protobuf, rules_java, rules_cc, rules_android, apple_support).

Because of the move the rules need to be loaded from their repositories. This requires adding load statements to all bzl and BUILD files that are using those rules.

Description

The flag --incompatible_autoload_externally accepts a list of rules or symbols (previously part of Bazel) that are automatically loaded from external repositories. For configuration see https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/packages/AutoloadSymbols.java.

Prefixing a symbol with '+', loads the symbol from rules_repository, but keeps the native implementation available in rules_repositories (redirects work). Symbol without a prefix is loaded and the native implementation is not available. Symbol prefixed with a '-' is not loaded and the native implementation is not available (same as if the symbol was deleted from Bazel).

Incompatible Flag

--incompatible_autoload_externally

Migration Guide

Bazel <=7:
TODO

Bazel 8:
The flag introduced incompatibilities because the rules are automatically loaded. For this to work we need to depend on the version of the rules_repostiories that implement the rules.

Required versions:
protobuf - 28.3
rules_java - 8.5.1
rules_cc - 0.0.17
rules_android - TODO
apple_support - TODO

With bzlmod the right version is selected automatically. Using WORKSPACE files, this needs to be done manually.

Bazel 9:
Load statements for all of the rules need to be added to BUILD and bzl files. To test the behaviour flag is set to an empty value, that is --incompatible_autoload_externally=

Cycles:
TODO

In which Bazel LTS version will this incompatible change be enabled?

Bazel 9

Additional Context

In Bazel 8, the flag will be set to the list of rules that are removed ==> rules are loaded automatically
In Bazel 9, the flag will be set to an empty list ==> rules are not loaded automatically anymore, load statements must be used

TODO List

With incompatible flag (highly recommend for incompatible changes having large impact):

  • Implement the incompatible change at Bazel HEAD and guard it behind a flag.
  • Backport the incompatible flag to the latest LTS release.
  • Test the incompatible change with downstream projects and inform broken projects.
  • Flip the incompatible flag at Bazel HEAD.
  • Delete the incompatible flag and the old behavior at Bazel HEAD.
@comius comius added untriaged incompatible-change Incompatible/breaking change labels Jul 19, 2024
@comius comius added team-Rules-API API for writing rules/aspects: providers, runfiles, actions, artifacts P2 We'll consider working on this in future. (Assignee optional) and removed untriaged incompatible-change Incompatible/breaking change labels Jul 19, 2024
@comius comius assigned comius and unassigned sgowroji, iancha1992 and satyanandak Jul 19, 2024
@meteorcloudy meteorcloudy added the incompatible-change Incompatible/breaking change label Jul 22, 2024
@tpasternak
Copy link

Would it mean they would be just loaded, or downloaded from BCR?

@tpasternak
Copy link

Just a few questions more:

  1. Will it be possible to import locally defined symbols?
  2. There are some rulesets mentioned above, I guess rules_python is missing

copybara-service bot pushed a commit that referenced this issue Aug 23, 2024
Issue: #22928
Incompatible flag issue: #23043

The flag supports Bazel users in migrating the rules from being embedded in Bazel to external repositories. Listing a symbol or a rule in the flag automatically adds a load to the respective repository. Listing it with a '+' keeps the symbol available in the rules_repository. Listing a symbol with "-" forcefully removes it from Bazel.

Cycles are prevented with a list of repositories where autoloads should not be used.

The new environments are injected into BzlCompile, BzlLoad and Package function.

StarlarkBuiltinsFunction with autoloads true key is used to load the external symbols.

Closes #23016.

Downstream test: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/4003

PiperOrigin-RevId: 666710259
Change-Id: Idaa9b6b13c9a474f700e69e22b6162ed59b7fab0
comius added a commit to comius/bazel that referenced this issue Oct 9, 2024
Issue: bazelbuild#22928
Incompatible flag issue: bazelbuild#23043

The flag supports Bazel users in migrating the rules from being embedded in Bazel to external repositories. Listing a symbol or a rule in the flag automatically adds a load to the respective repository. Listing it with a '+' keeps the symbol available in the rules_repository. Listing a symbol with "-" forcefully removes it from Bazel.

Cycles are prevented with a list of repositories where autoloads should not be used.

The new environments are injected into BzlCompile, BzlLoad and Package function.

StarlarkBuiltinsFunction with autoloads true key is used to load the external symbols.

Closes bazelbuild#23016.

Downstream test: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/4003

PiperOrigin-RevId: 666710259
Change-Id: Idaa9b6b13c9a474f700e69e22b6162ed59b7fab0
comius added a commit to comius/bazel that referenced this issue Oct 9, 2024
Issue: bazelbuild#22928
Incompatible flag issue: bazelbuild#23043

The flag supports Bazel users in migrating the rules from being embedded in Bazel to external repositories. Listing a symbol or a rule in the flag automatically adds a load to the respective repository. Listing it with a '+' keeps the symbol available in the rules_repository. Listing a symbol with "-" forcefully removes it from Bazel.

Cycles are prevented with a list of repositories where autoloads should not be used.

The new environments are injected into BzlCompile, BzlLoad and Package function.

StarlarkBuiltinsFunction with autoloads true key is used to load the external symbols.

Closes bazelbuild#23016.

Downstream test: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/4003

PiperOrigin-RevId: 666710259
Change-Id: Idaa9b6b13c9a474f700e69e22b6162ed59b7fab0
comius added a commit to comius/bazel that referenced this issue Oct 9, 2024
Issue: bazelbuild#22928
Incompatible flag issue: bazelbuild#23043

The flag supports Bazel users in migrating the rules from being embedded in Bazel to external repositories. Listing a symbol or a rule in the flag automatically adds a load to the respective repository. Listing it with a '+' keeps the symbol available in the rules_repository. Listing a symbol with "-" forcefully removes it from Bazel.

Cycles are prevented with a list of repositories where autoloads should not be used.

The new environments are injected into BzlCompile, BzlLoad and Package function.

StarlarkBuiltinsFunction with autoloads true key is used to load the external symbols.

Closes bazelbuild#23016.

Downstream test: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/4003

PiperOrigin-RevId: 666710259
Change-Id: Idaa9b6b13c9a474f700e69e22b6162ed59b7fab0
comius added a commit to comius/bazel that referenced this issue Oct 9, 2024
Issue: bazelbuild#22928
Incompatible flag issue: bazelbuild#23043

The flag supports Bazel users in migrating the rules from being embedded in Bazel to external repositories. Listing a symbol or a rule in the flag automatically adds a load to the respective repository. Listing it with a '+' keeps the symbol available in the rules_repository. Listing a symbol with "-" forcefully removes it from Bazel.

Cycles are prevented with a list of repositories where autoloads should not be used.

The new environments are injected into BzlCompile, BzlLoad and Package function.

StarlarkBuiltinsFunction with autoloads true key is used to load the external symbols.

Closes bazelbuild#23016.

Downstream test: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/4003

PiperOrigin-RevId: 666710259
Change-Id: Idaa9b6b13c9a474f700e69e22b6162ed59b7fab0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
incompatible-change Incompatible/breaking change P2 We'll consider working on this in future. (Assignee optional) team-Rules-API API for writing rules/aspects: providers, runfiles, actions, artifacts
Projects
None yet
Development

No branches or pull requests

6 participants