Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

E2E job parallelization by user role and functionality #9243

Closed
yonasberhe23 opened this issue Jun 29, 2023 · 6 comments · Fixed by #9887
Closed

E2E job parallelization by user role and functionality #9243

yonasberhe23 opened this issue Jun 29, 2023 · 6 comments · Fixed by #9887
Assignees
Labels
area/e2e area/test Test (e2e and unit) kind/tech-debt Technical debt QA/None
Milestone

Comments

@yonasberhe23
Copy link
Contributor

yonasberhe23 commented Jun 29, 2023

Currently, admin and standard user tests run asynchronously but we want them to run in parallel. To handle this, lets pull the Rancher setup spec out of the admin test run and put it in its own step, that way the setup step would precede both the admin and standard user test steps and they can run in parallel.

For more details see conversation here.

NOTE: standard_user creation has been added in the Rancher setup test.

@richard-cox
Copy link
Member

This should be done alongside #9064

@richard-cox richard-cox added this to the v2.7.next3 milestone Jun 30, 2023
@gaktive gaktive modified the milestones: v2.7.next3, v2.8.0 Aug 3, 2023
@nwmac nwmac modified the milestones: v2.8.0, v2.8.next1 Aug 15, 2023
@cnotv
Copy link
Contributor

cnotv commented Oct 10, 2023

I think that if the only issue here is to initialize the Rancher instance, we may rather create a function for that.

In the current state it's necessary to build and run an image of Rancher for both to have 2 separated environments.

@cnotv cnotv self-assigned this Oct 10, 2023
@zube zube bot removed the [zube]: To Triage label Oct 10, 2023
@cnotv
Copy link
Contributor

cnotv commented Oct 10, 2023

I can see that the issue here is not about the setup at all, we have also to create the user as admin first. I was not aware of the update where we create the user in the setup.

@cnotv
Copy link
Contributor

cnotv commented Oct 10, 2023

This should be done alongside #9064

That's more about run tests in parallels, not jobs in parallel, so are 2 different cases.

@cnotv cnotv changed the title E2E test setup: Put Rancher setup spec in its own step E2E group user and admin job parallelization Oct 10, 2023
@cnotv
Copy link
Contributor

cnotv commented Oct 17, 2023

After talking to @richard-cox and @nwmac we decided to try more job splitting by path/tag, so we can create categories or isolate long jobs which cause issues.

@cnotv cnotv changed the title E2E group user and admin job parallelization E2E job parallelization by user role and area Oct 17, 2023
@cnotv cnotv changed the title E2E job parallelization by user role and area E2E job parallelization by user role and functionality Oct 17, 2023
@cnotv
Copy link
Contributor

cnotv commented Nov 17, 2023

@yonasberhe23 @richard-cox Are we sure the tests are idempotent? I have run 5 times in parallel and the same tests keep failing.

Screenshot 2023-11-17 at 09 19 47

@zube zube bot removed the [zube]: Done label Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/e2e area/test Test (e2e and unit) kind/tech-debt Technical debt QA/None
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants