Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update java_tools v13.9 / rules_java 7.12.0 #23649

Closed
wants to merge 27 commits into from
Closed

Update java_tools v13.9 / rules_java 7.12.0 #23649

wants to merge 27 commits into from

Conversation

iancha1992
Copy link
Member

@github-actions github-actions bot added the awaiting-review PR is awaiting review from an assigned reviewer label Sep 18, 2024
@iancha1992 iancha1992 mentioned this pull request Sep 18, 2024
Closed
@iancha1992
Copy link
Member Author

I am trying to update the lockfile, but I get this error:

(base) heec-macbookpro:bazel heec$ bazel run //src/test/tools/bzlmod:update_default_lock_file
ERROR: For repository 'bazel_skylib', the root module requires module version bazel_skylib@1.6.1, but got bazel_skylib@1.7.1 in the resolved dependency graph.
ERROR: For repository 'io_bazel_skydoc', the root module requires module version stardoc@0.5.6, but got stardoc@0.7.0 in the resolved dependency graph.
ERROR: For repository 'rules_cc', the root module requires module version rules_cc@0.0.9, but got rules_cc@0.0.10 in the resolved dependency graph.
ERROR: For repository 'rules_proto', the root module requires module version rules_proto@6.0.0, but got rules_proto@6.0.2 in the resolved dependency graph.
ERROR: Error computing the main repository mapping: Direct dependency check failed.

Please advise @hvadehra Thanks!

@iancha1992 iancha1992 removed the awaiting-review PR is awaiting review from an assigned reviewer label Sep 18, 2024
@hvadehra hvadehra mentioned this pull request Sep 18, 2024
Closed
@hvadehra
Copy link
Member

Attempting to update them in #23654, lets see how it goes.

@iancha1992 iancha1992 marked this pull request as draft September 18, 2024 21:24
copybara-service bot pushed a commit that referenced this pull request Sep 20, 2024
@hvadehra @copybara-github
Update Bazel dependencies
b88b53a 
 - bazel_skylib: 1.6.1 -> 1.7.1
 - stardoc: 0.5.6 -> 0.7.1
 - rules_cc: 0.0.9 -> 0.0.10
 - rules_proto: 6.0.0 -> 6.0.2

Also fixes version of rules_jvm_external to 6.0 for the patch override

Split off from #23649

Closes #23654.

PiperOrigin-RevId: 676762672
Change-Id: Iae347bac2d4ea4da5f03b20419c8f15367455203
@hvadehra
Copy link
Member

@iancha1992 #23654 is merged, please try after rebasing this on HEAD.

@iancha1992 iancha1992 force-pushed the java_tools-v13.9 branch 2 times, most recently from 43c7d12 to d665734 Compare September 20, 2024 18:10
iancha1992 pushed a commit that referenced this pull request Sep 20, 2024
@hvadehra @iancha1992
Update Bazel dependencies
e331a43 
 - bazel_skylib: 1.6.1 -> 1.7.1
 - stardoc: 0.5.6 -> 0.7.1
 - rules_cc: 0.0.9 -> 0.0.10
 - rules_proto: 6.0.0 -> 6.0.2

Also fixes version of rules_jvm_external to 6.0 for the patch override

Split off from #23649

Closes #23654.

PiperOrigin-RevId: 676762672
Change-Id: Iae347bac2d4ea4da5f03b20419c8f15367455203
@iancha1992 iancha1992 closed this Sep 20, 2024
@iancha1992 iancha1992 reopened this Sep 20, 2024
@iancha1992
Copy link
Member Author

@hvadehra I've rebased and got this error while updating the lockfile

(base) heec-macbookpro:bazel heec$ bazel run //src/test/tools/bzlmod:update_default_lock_file
ERROR: /private/var/tmp/_bazel_heec/1ec8945322d13a649f0866417f1bb8c5/external/rules_java+/java/common/proguard_spec_info.bzl:16:20: ProguardSpecProvider is experimental and thus unavailable with the current flags. It may be enabled by setting --experimental_google_legacy_api
ERROR: /Users/heec/Desktop/bazel/src/test/tools/bzlmod/BUILD:11:10: While resolving toolchains for target //src/test/tools/bzlmod:update_default_lock_file (bf2c9ef): invalid registered execution platform '//:default_host_platform': error loading package '': at /Users/heec/Desktop/bazel/tools/distributions/distribution_rules.bzl:15:6: at /private/var/tmp/_bazel_heec/1ec8945322d13a649f0866417f1bb8c5/external/rules_java+/java/defs.bzl:17:6: at /private/var/tmp/_bazel_heec/1ec8945322d13a649f0866417f1bb8c5/external/rules_java+/java/java_import.bzl:16:6: at /private/var/tmp/_bazel_heec/1ec8945322d13a649f0866417f1bb8c5/external/rules_java+/java/bazel/rules/bazel_java_import.bzl:22:6: at /private/var/tmp/_bazel_heec/1ec8945322d13a649f0866417f1bb8c5/external/rules_java+/java/common/rules/impl/bazel_java_import_impl.bzl:25:6: at /private/var/tmp/_bazel_heec/1ec8945322d13a649f0866417f1bb8c5/external/rules_java+/java/common/rules/impl/proguard_validation.bzl:20:6: compilation of module 'java/common/proguard_spec_info.bzl' failed
ERROR: Analysis of target '//src/test/tools/bzlmod:update_default_lock_file' failed; build aborted
INFO: Elapsed time: 0.231s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully
ERROR: Build failed. Not running target
FAILED: 
    Fetching module extension internal_deps in @@rules_python+//python/private/bzlmod:internal_deps.bzl; starting

copybara-service bot pushed a commit to bazelbuild/rules_java that referenced this pull request Sep 23, 2024
@hvadehra
Partial rollback of commit 70970af.
e4d1f09 
*** Reason for rollback ***

The rules aren't ready to be used with any Bazel version other than HEAD. Unblocks bazelbuild/bazel#23649

*** Original change description ***

Switch `rules_java` rule macro redirects from `native.*` to those defined locally

Using the rules requires a bazel version that contains all the changes in bazelbuild/bazel#23591

PiperOrigin-RevId: 677675846
Change-Id: Ibfd5d59e1dbfa04b0f0d5117493662d06f40f557
@hvadehra
Copy link
Member

@iancha1992 Should be fixed by bazelbuild/rules_java@e4d1f09. You'll need to update your rules_java branch (https://github.com/bazelbuild/rules_java/tree/java_v13.9) to HEAD.

@iancha1992
Copy link
Member Author

iancha1992 commented Sep 24, 2024 •
edited
Loading

@hvadehra I am experiencing another failure. Could you please take a look?
I've updated the rules_java branch by rebasing it like you suggested and tried to update the lockfile in this branch (please see below). Thanks!

(base) heec-macbookpro:bazel heec$ bazel run //src/test/tools/bzlmod:update_default_lock_file
ERROR: /private/var/tmp/_bazel_heec/1ec8945322d13a649f0866417f1bb8c5/external/rules_java+/java/common/proguard_spec_info.bzl:16:20: ProguardSpecProvider is experimental and thus unavailable with the current flags. It may be enabled by setting --experimental_google_legacy_api
ERROR: /Users/heec/Desktop/bazel/src/test/tools/bzlmod/BUILD:11:10: While resolving toolchains for target //src/test/tools/bzlmod:update_default_lock_file (bf2c9ef): invalid registered execution platform '//:default_host_platform': error loading package '': at /Users/heec/Desktop/bazel/tools/distributions/distribution_rules.bzl:15:6: at /private/var/tmp/_bazel_heec/1ec8945322d13a649f0866417f1bb8c5/external/rules_java+/java/defs.bzl:17:6: at /private/var/tmp/_bazel_heec/1ec8945322d13a649f0866417f1bb8c5/external/rules_java+/java/java_import.bzl:16:6: at /private/var/tmp/_bazel_heec/1ec8945322d13a649f0866417f1bb8c5/external/rules_java+/java/bazel/rules/bazel_java_import.bzl:22:6: at /private/var/tmp/_bazel_heec/1ec8945322d13a649f0866417f1bb8c5/external/rules_java+/java/common/rules/impl/bazel_java_import_impl.bzl:25:6: at /private/var/tmp/_bazel_heec/1ec8945322d13a649f0866417f1bb8c5/external/rules_java+/java/common/rules/impl/proguard_validation.bzl:20:6: compilation of module 'java/common/proguard_spec_info.bzl' failed
ERROR: Analysis of target '//src/test/tools/bzlmod:update_default_lock_file' failed; build aborted
INFO: Elapsed time: 0.201s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully
ERROR: Build failed. Not running target

hvadehra
hvadehra reviewed Sep 24, 2024
View reviewed changes
MODULE.bazel Outdated Show resolved Hide resolved
Googler and others added 8 commits September 24, 2024 11:00
@iancha1992
Support adding validation during package loading
5448f4d 
PiperOrigin-RevId: 676919221
Change-Id: Ia5c1eddfd493d9efac63c45baa14818daf7d8ecf
@justinhorvitz @iancha1992
Precompute StarlarkDefinedConfigTransition hashcodes.
e70af32 
PiperOrigin-RevId: 676921410
Change-Id: Ie12eb5ecc28c4eeaa15b684847a7fe3ec8c75e3b
@fmeum @iancha1992
Fix upb build with Clang 16
72b0abc 
With Xcode 16, `upb` fails its own `-Werror` check due to using Clang extensions.

Closes #23667.

PiperOrigin-RevId: 676957876
Change-Id: I8e038b5f879077fefdb5811aff6ae5f56fb4f762
@justinhorvitz @iancha1992
Use a constant for the _builtins RepositoryName.
4c8147e 
Saves overhead of interning and validation.

PiperOrigin-RevId: 677198027
Change-Id: I3d8cd0bab0a7f5f5758d1b17c3f481b187eeeb18
@cushon @iancha1992
Update one version allowlist flag
dd03061 
PiperOrigin-RevId: 677623384
Change-Id: Ic5ddcb5dd3b0671fb0319b49690d29a81e0f44ba
@lberki @iancha1992
Add tests for dormant dependencies.
722e458 
Both for "bazel query" and "bazel cquery".

RELNOTES: None.
PiperOrigin-RevId: 677687076
Change-Id: I6da9fa9dcd5f4a2de2100f24546d6bcaf9954a94
@lberki @iancha1992
Forbid dependency resolution rules from being parents.
50c9f0c 
Otherwise, one could have a dormant attribute on a regular rule by inheriting from a dependency resolution rule.

RELNOTES: None.
PiperOrigin-RevId: 677688900
Change-Id: Idab7268e125381008a0120033e8be9d93cfbf41a
@pzembrod @iancha1992
Remove unused code from ObjcCompilationContext in preparation to star…
cbb05fe 
…larkification.

PiperOrigin-RevId: 677690028
Change-Id: I8aee192551f22ada82f22d9219f97b1fb20e65ae
tjgq and others added 17 commits September 24, 2024 11:00
@tjgq @iancha1992
Rename RemoteCache to CombinedCache, reflecting its true purpose.
1cbf790 
PiperOrigin-RevId: 677697291
Change-Id: I1f201c4abdccd79fd80901c7f0ee68c728fc20cb
@fmeum @iancha1992
Fix naming inconsistency in spawn.proto
bb98b05 
Closes #23706.

PiperOrigin-RevId: 677697462
Change-Id: I3e7d6d856881c5208df86011f165389db744abf0
@tjgq @iancha1992
Automated rollback of commit 9acc146.
28248fa 
*** Reason for rollback ***

Pivoting to a different approach.

*** Original change description ***

Add a starting point for the disk cache garbage collector.

This is not yet a fully functional implementation. For now, the garbage collector (when enabled by the --experimental_disk_cache_max_size flag) merely checks whether the index database is missing or empty and, if so, recreates it from the filesystem contents.

PiperOrigin-RevId: 677739324
Change-Id: Ic80419c01cf35fce034350b60ea6a8ebb6ae605b
@benjaminp @iancha1992
Make DigestUtils.CacheKey a record.
c391419 
save a bunch of boilerplate

Closes #23695.

PiperOrigin-RevId: 677739586
Change-Id: Iaa3b515ff1bcba965cb4ff5467058f258eb4e706
@coeuvre @iancha1992
Correctly propagate errors and abort the build for certain remote err…
3f887fa 
…ors.

Fixes #23473.

Closes #23656.

PiperOrigin-RevId: 677788155
Change-Id: I974c8412fa3be775ec645e4001bf736e8ba93932
@ted-xie @iancha1992
Delete dexer:{dexer,DexBuilder} and related files
2e6a1d9 
PiperOrigin-RevId: 677802443
Change-Id: I8583514fa9791c1655abea28e933b4abd44f305c
@hvadehra @iancha1992
LSC(tp): Add load()s for the builtin java rules
7d14f54 
BEGIN_PUBLIC

Load the builtin Bazel java rules from @rules_java

END_PUBLIC

PiperOrigin-RevId: 677821021
Change-Id: Id06be6a2dd11d9ef23ae0b70860864f47290c2c1
@cushon @iancha1992
Internal change
526610f 
PiperOrigin-RevId: 677821571
Change-Id: Ic06383559272c0eb3a01b2045155e642b852858f
@cushon @iancha1992
Rename java_toolchain.oneversion_whitelist to oneversion_allowlist
2c68da6 
PiperOrigin-RevId: 677839283
Change-Id: I60599ec17e31f038485e79bb68f67d67431eb9d7
@tjgq @iancha1992
Implement core garbage collection logic for the disk cache.
b683102 
The garbage collection policy is defined by a maximum target size and a maximum age of individual cache entries, both of which may be simultaneously provided. I/O operations are parallelized to improve performance for large caches or slow filesystems.

PiperOrigin-RevId: 677860078
Change-Id: Ib342ad5e80ef4ef4af237aae243a300d13caaa06
@katre @iancha1992
Convert ToolchainType from AutoValue to record.
2feebcb 
PiperOrigin-RevId: 677907326
Change-Id: I987aa2d34afb756ac9a0373df787626316e3a6dc
@cushon @iancha1992
Add -Djava.lang.Thread.allowVirtualThreads=true as a startup JVM flag
45f27cc 
PiperOrigin-RevId: 677985968
Change-Id: I622cb57ff11d2f867ac75400113e19da25ad305f
@stagnation @iancha1992
python: Add provides = [PyInfo] to py_library/py_binary/py_test
f971aac 
This is required for aspects to propagate along python libraries (and binaries) with `required_providers` set.
Otherwise the aspect terminates on the top-level target. This follows from/was inspired by #19609 where the `CcInfo` was added to `cc_binary` targets.

The `cc_library` does provide `CcInfo` so an aspect can successfully propagate the `cc_library` tree.
Also related: #17214

<details>
<summary>Reproduction example to show the problem with py targets</summary>

```
#!/bin/sh

set -eu

dir=${1:-$(mktemp -d)}

# "$dir"/BUILD.bazel {{{
cat > "$dir"/BUILD.bazel <<'EOF'
# stack a high dependency tree
py_binary(
    name = "base_py",
    srcs = ["base.py"],
    main = "base.py",
)

py_library(
    name = "stack_0_py",
    srcs = ["extra.py"],
    deps = ["//:base_py"],
)

[
    py_library(
        name = "stack_{}_py".format(index),
        srcs = ["extra.py"],
        deps = [":stack_{}_py".format(index - 1)],
    )
    for index in range(1, 2)
]
cc_library(
    name = "base_cc",
    srcs = ["base.c"],
)

cc_library(
    name = "stack_0_cc",
    srcs = ["extra.c"],
    deps = ["//:base_cc"],
)

[
    cc_library(
        name = "stack_{}_cc".format(index),
        srcs = ["extra.c"],
        deps = [":stack_{}_cc".format(index - 1)],
    )
    for index in range(1, 2)
]
EOF
# }}}

# "$dir"/file_count.bzl {{{
cat > "$dir"/file_count.bzl <<'EOF'
FileCountInfo = provider(
    'count',
    fields = {
        'count' : 'number of files'
    }
)

def _file_count_aspect_impl(_, ctx):
    name = ctx.rule.attr.name
    count = 0
    # Make sure the rule has a srcs attribute.
    if hasattr(ctx.rule.attr, 'srcs'):
        # Iterate through the sources counting files
        for src in ctx.rule.attr.srcs:
            for _ in src.files.to_list():
                count = count + 1
    # Get the counts from our dependencies.
    for dep in ctx.rule.attr.deps:
        if FileCountInfo in dep:
            count = count + dep[FileCountInfo].count

    print(name, count)
    return [FileCountInfo(count = count)]

ok = aspect(
    implementation = _file_count_aspect_impl,
    attr_aspects = ['deps'],
)

required = aspect(
    implementation = _file_count_aspect_impl,
    required_providers = [
        [PyInfo],
        [CcInfo],
    ],
    attr_aspects = ['deps'],
)
EOF
# }}}

touch "$dir"/WORKSPACE
touch "$dir"/base.py
touch "$dir"/extra.py
touch "$dir"/base.c
touch "$dir"/extra.c

echo Reproducing in "$dir"
cd "$dir" || exit 1

bazelisk --version

echo "[Python] First print the succesfull traversal with transient target, and accumulating count"
(
    set -x
    bazelisk build \
        --ui_event_filters=-info --noshow_progress --noshow_loading_progress \
        --show_result=0 \
        --aspects=//:file_count.bzl%ok //:stack_1_py
    set +x
)
echo "[Python] Now add 'required_providers', and the transitive dependencies disappear."
(
    set -x
    bazelisk build \
        --ui_event_filters=-info --noshow_progress --noshow_loading_progress \
        --show_result=0 \
        --aspects=//:file_count.bzl%required //:stack_1_py
    set +x
)

echo "[CC] Both works for cc_library"
(
    set -x
    bazelisk build \
        --ui_event_filters=-info --noshow_progress --noshow_loading_progress \
        --show_result=0 \
        --aspects=//:file_count.bzl%ok //:stack_1_cc
    bazelisk build \
        --ui_event_filters=-info --noshow_progress --noshow_loading_progress \
        --show_result=0 \
        --aspects=//:file_count.bzl%required //:stack_1_cc
    set +x
)
```

Output:
```
$ env USE_BAZEL_VERSION=7.0.0rc5 sh reproduction-aspect-required-provider.
sh aspect-required-provider/
Reproducing in aspect-required-provider/
bazel 7.0.0rc5
[Python] First print the succesfull traversal with transient target, and accumulating count
+ bazelisk build --ui_event_filters=-info --noshow_progress --noshow_loading_progress --aspects=//:file_count.bzl%ok //:stack_1_py
Starting local Bazel server and connecting to it...
DEBUG: /home/nils/task/reproductions/aspect-required-provider/file_count.bzl:22:10: base_py 1
DEBUG: /home/nils/task/reproductions/aspect-required-provider/file_count.bzl:22:10: stack_0_py 2
DEBUG: /home/nils/task/reproductions/aspect-required-provider/file_count.bzl:22:10: stack_1_py 3
+ set +x
[Python] Now add 'required_providers', and the transitive dependencies disappear.
+ bazelisk build --ui_event_filters=-info --noshow_progress --noshow_loading_progress --aspects=//:file_count.bzl%required //:stack_1_py
DEBUG: /home/nils/task/reproductions/aspect-required-provider/file_count.bzl:22:10: stack_1_py 1
+ set +x
[CC] Both works for cc_library
+ bazelisk build --ui_event_filters=-info --noshow_progress --noshow_loading_progress --aspects=//:file_count.bzl%ok //:stack_1_cc
DEBUG: /home/nils/task/reproductions/aspect-required-provider/file_count.bzl:22:10: base_cc 1
DEBUG: /home/nils/task/reproductions/aspect-required-provider/file_count.bzl:22:10: stack_0_cc 2
DEBUG: /home/nils/task/reproductions/aspect-required-provider/file_count.bzl:22:10: stack_1_cc 3
+ bazelisk build --ui_event_filters=-info --noshow_progress --noshow_loading_progress --aspects=//:file_count.bzl%required //:stack_1_cc
DEBUG: /home/nils/task/reproductions/aspect-required-provider/file_count.bzl:22:10: base_cc 1
DEBUG: /home/nils/task/reproductions/aspect-required-provider/file_count.bzl:22:10: stack_0_cc 2
DEBUG: /home/nils/task/reproductions/aspect-required-provider/file_count.bzl:22:10: stack_1_cc 3
```
</details>

Closes #20436.

PiperOrigin-RevId: 677997144
Change-Id: Id2f75db10447b334a9d5ec9872e15f490ae44927
@justinhorvitz @iancha1992
Create the fileset mapping lazily in ActionInputMetadataProvider.
ca4722f 
It may never be needed, for example when there is an action cache hit or an action file system is in use.

PiperOrigin-RevId: 678030778
Change-Id: I9db28d5eb4637be3d628dc190f2b39e7a81dce77
@iancha1992
Clarify the documentation of filegroup.
c68b5e4 
PiperOrigin-RevId: 678110302
Change-Id: I2caa3ad37e0cb2fd1d300bfdace76c886fc1c405
@iancha1992
Update MODULE.bazel and workspace_deps.bzl again for fixes for rules_…
0310d2d 
…java
@iancha1992
Update lockfile
0694b23
@iancha1992 iancha1992 closed this Sep 24, 2024
@iancha1992 iancha1992 deleted the java_tools-v13.9 branch September 24, 2024 18:35
@iancha1992
Copy link
Member Author

Duplicate: #23738

copybara-service bot pushed a commit to bazelbuild/rules_java that referenced this pull request Oct 2, 2024
@hvadehra
Automated rollback of commit e4d1f09.
3cdce96 
*** Reason for rollback ***

Rollforward of https://github.com/rules_java/bazel/commit/70970af9590a49b310312b2d01e85ac4fb72fff7, Bazel 7.3.2 is released and the rules are usable

*** Original change description ***

Partial rollback of commit 70970af.

*** Reason for rollback ***

The rules aren't ready to be used with any Bazel version other than HEAD. Unblocks bazelbuild/bazel#23649

*** Original change description ***

Switch `rules_java` rule macro redirects from `native.*` to those defined locally

Using the rules requires a bazel version that contains all the changes in bazelbuild/bazel#23591

PiperOrigin-RevId: 681344593
Change-Id: Idea53aea2a0569ff6e345edd6a1d1b44722117bf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hvadehra hvadehra hvadehra left review comments

@lberki lberki Awaiting requested review from lberki lberki will be requested when the pull request is marked ready for review lberki is a code owner

@Wyverald Wyverald Awaiting requested review from Wyverald Wyverald will be requested when the pull request is marked ready for review Wyverald is a code owner

@meteorcloudy meteorcloudy Awaiting requested review from meteorcloudy meteorcloudy will be requested when the pull request is marked ready for review meteorcloudy is a code owner

@ahumesky ahumesky Awaiting requested review from ahumesky ahumesky will be requested when the pull request is marked ready for review ahumesky is a code owner

@ted-xie ted-xie Awaiting requested review from ted-xie ted-xie will be requested when the pull request is marked ready for review ted-xie is a code owner

@kevin1e100 kevin1e100 Awaiting requested review from kevin1e100 kevin1e100 will be requested when the pull request is marked ready for review kevin1e100 is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.