Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correctly match regex with tree artifact #16949

Closed
wants to merge 4 commits into from

Conversation

coeuvre
Copy link
Member

@coeuvre coeuvre commented Dec 7, 2022

It doesn't work when setting --experimental_remote_download_regex to match files inside tree artifact after e01e7f5. This change fixes that.

Fixes #16922.

@coeuvre coeuvre requested a review from a team as a code owner December 7, 2022 17:00
@coeuvre coeuvre requested a review from tjgq December 7, 2022 17:00
break;
if (output.isTreeArtifact()) {
var children = metadataHandler.getTreeArtifactChildren((SpecialArtifact) output);
if (outputMatchesPattern(output)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the pattern has a $, would this incorrectly match tree artifact paths?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have a concrete example?

If the pattern matches tree root, all files inside the tree are downloaded. (See newly added test cases)

Copy link
Contributor

@brentleyjones brentleyjones Dec 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So that seems like a bug to me, since if we weren't using a tree artifact we wouldn't download everything under a/path with the pattern a/path$ (for example, if we had an output of a/path/and/more/lib.a).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check is guarded by output.isTreeArtifact(). File output a/path/and/more/lib.a won't match pattern a/path$ so it won't be downloaded. Added another test case to illustrate that.

Copy link
Contributor

@brentleyjones brentleyjones Dec 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but I'm saying that the behavior shouldn't differ between if it's a tree artifact or not. We shouldn't match on the tree artifact itself if it has children, if using $ in the pattern.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(If I change the lib.a example from a normal output to a tree artifact output, the download behavior of any patterns shouldn't change.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Good catch! Updated the code.

@copybara-service copybara-service bot closed this in 86dee6d Dec 8, 2022
coeuvre added a commit to coeuvre/bazel that referenced this pull request Dec 8, 2022
It doesn't work when setting `--experimental_remote_download_regex` to match files inside tree artifact after bazelbuild@e01e7f5. This change fixes that.

Fixes bazelbuild#16922.

Closes bazelbuild#16949.

PiperOrigin-RevId: 493838296
Change-Id: I6eceffbffce30949173d10120a9120c6c608a983
meteorcloudy pushed a commit that referenced this pull request Dec 8, 2022
It doesn't work when setting `--experimental_remote_download_regex` to match files inside tree artifact after e01e7f5. This change fixes that.

Fixes #16922.

Closes #16949.

PiperOrigin-RevId: 493838296
Change-Id: I6eceffbffce30949173d10120a9120c6c608a983
@coeuvre coeuvre deleted the fix-16922 branch December 9, 2022 14:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting-review PR is awaiting review from an assigned reviewer team-Remote-Exec Issues and PRs for the Execution (Remote) team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

--experimental_remote_download_regex doesn't work with a cache in 6.0.0rc4
3 participants