Skip to content

build: attempt to de-flake harness integration test #25241

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 9, 2022

Conversation

crisbeto
Copy link
Member

@crisbeto crisbeto commented Jul 8, 2022

Tries to de-flake the harness e2e integration test by swapping out mat-select with mat-selection-list. mat-select has been causing flakes in its own Firefox tests too so it's possible that it may be the reason for the integration test flakes as well.

The component being tested doesn't really matter since the integration test is primarily verifying our npm packages.

@crisbeto crisbeto force-pushed the harness-integration-test branch 2 times, most recently from ad3d8e8 to 132cb1d Compare July 8, 2022 07:52
@crisbeto crisbeto added P3 An issue that is relevant to core functions, but does not impede progress. Important, but not urgent merge safe target: patch This PR is targeted for the next patch release labels Jul 8, 2022
@crisbeto crisbeto marked this pull request as ready for review July 8, 2022 07:52
@crisbeto
Copy link
Member Author

crisbeto commented Jul 8, 2022

@devversion I'm not sure what is going on with the integration test error, but it seems like a Bazel issue. My guess is that something was depending transitively on material/select to bring in rxjs, but since the radio button doesn't use rxjs at all, that transitive dependency fails.

@crisbeto crisbeto force-pushed the harness-integration-test branch from 132cb1d to 76163bb Compare July 8, 2022 08:16
@crisbeto
Copy link
Member Author

crisbeto commented Jul 8, 2022

I ended up switching the test to mat-selection-list which seems to work, but here's a failing run with mat-radio-button for reference: https://app.circleci.com/pipelines/github/angular/components/41061/workflows/d378c467-eefa-48cd-b6e9-f1408a119a6b/jobs/388681.

I also debugged it a bit further and it's probably not a Bazel issue, because the same error happened when I install the dependencies manually and run the test script.

@devversion
Copy link
Member

Oh yeah, that's one of the v13 APF ESM effects. e.g. Angular core imports from rxjs/operators, but rxjs does not have package exports and in ESM a directory cannot be directly imported unless there is an export mapping for it. We saw that in v13 but it wasn't a concern because we bundle tests, and the app with the Angular CLI.

I believe long term we need to make sure RxJS has the proper export mappings. The v7 exports always use CommonJS for "node" (which seems incorrect) but maybe updating already helps? (In the integration test)

@crisbeto
Copy link
Member Author

crisbeto commented Jul 8, 2022

I'd rather not depend on a different rxjs version from the rest of the project so it may be simplest to test using the selection list instead of the radio button.

@devversion
Copy link
Member

Ah yes. The PR still was stuck on the failing state for me so I thought you looked for a solution. Selection sounds good to me.

Although it would be nice to verify that users have a way to get this working with selenium-webdriver tests that do not seem to be bundled in the CLI? It's our responsibility because we ship APF ESM packages intended for use in the NodeJS runtime

@crisbeto crisbeto force-pushed the harness-integration-test branch 2 times, most recently from 22b7bf6 to 818b180 Compare July 8, 2022 19:16
@crisbeto
Copy link
Member Author

crisbeto commented Jul 8, 2022

I pushed a change to switch back to the radio test and bump up the rxjs version, but it breaks with the same error.

@devversion
Copy link
Member

devversion commented Jul 8, 2022

@crisbeto Ahh, I believe it might be due to:

. We always end up with rxjs from the main package json. You could see if you could omit the mapping with value of "rxjs" there.

Potentially using https://github.com/bazelbuild/bazel-skylib/blob/main/docs/dicts_doc.md#dictsomit. The bad part is that rxjs is the value not a key. The key for rxjs is determined in "tools/integration.bzl".

@crisbeto crisbeto force-pushed the harness-integration-test branch from 818b180 to 1936ed7 Compare July 8, 2022 20:27
@crisbeto
Copy link
Member Author

crisbeto commented Jul 8, 2022

Just gave it a shot, but it didn't work.

@devversion
Copy link
Member

devversion commented Jul 8, 2022

thanks for giving it a shot. I'm happy to investigate this in my end if you feel like it.

currently it does not work because the key label is different. see:

def _get_archive_label_of_package(package_name):

Maybe make that function public and use it here? (don't forget to add the @npm prefix like in the mappings dict we create)

@crisbeto crisbeto force-pushed the harness-integration-test branch from 1936ed7 to c7427d7 Compare July 9, 2022 04:43
Tries to de-flake the harness e2e integration test by swapping out `mat-select` with `mat-radio-group`. `mat-select` has been causing flakes in its own Firefox tests too so it's possible that it may be the reason for the integration test flakes as well.

The component being tested doesn't really matter since the integration test is primarily verifying our npm packages.
@crisbeto crisbeto force-pushed the harness-integration-test branch from c7427d7 to 6cd2522 Compare July 9, 2022 04:50
@crisbeto
Copy link
Member Author

crisbeto commented Jul 9, 2022

Okay, there we go. I was looking at the _get_archive_label_of_package function before, but I missed the _archive suffix.

Copy link
Member

@devversion devversion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@crisbeto crisbeto added the action: merge The PR is ready for merge by the caretaker label Jul 9, 2022
@crisbeto crisbeto merged commit a26b192 into angular:main Jul 9, 2022
crisbeto added a commit that referenced this pull request Jul 9, 2022
Tries to de-flake the harness e2e integration test by swapping out `mat-select` with `mat-radio-group`. `mat-select` has been causing flakes in its own Firefox tests too so it's possible that it may be the reason for the integration test flakes as well.

The component being tested doesn't really matter since the integration test is primarily verifying our npm packages.

(cherry picked from commit a26b192)
crisbeto added a commit that referenced this pull request Jul 9, 2022
Tries to de-flake the harness e2e integration test by swapping out `mat-select` with `mat-radio-group`. `mat-select` has been causing flakes in its own Firefox tests too so it's possible that it may be the reason for the integration test flakes as well.

The component being tested doesn't really matter since the integration test is primarily verifying our npm packages.

(cherry picked from commit a26b192)
@angular-automatic-lock-bot
Copy link

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

This action has been performed automatically by a bot.

@angular-automatic-lock-bot angular-automatic-lock-bot bot locked and limited conversation to collaborators Aug 9, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
action: merge The PR is ready for merge by the caretaker P3 An issue that is relevant to core functions, but does not impede progress. Important, but not urgent target: patch This PR is targeted for the next patch release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants