Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SYCL][DevOps] Fuse OCL CPU testing into GEN9 Linux task #9546

Merged
merged 2 commits into from
May 23, 2023
Merged

[SYCL][DevOps] Fuse OCL CPU testing into GEN9 Linux task #9546

merged 2 commits into from
May 23, 2023

Conversation

aelovikov-intel
Copy link
Contributor

@aelovikov-intel aelovikov-intel commented May 21, 2023 •
edited
Loading

It would increase its duration by several minutes while eliminating 20+ minutes task completely.

@aelovikov-intel aelovikov-intel temporarily deployed to aws May 21, 2023 22:34 — with GitHub Actions Inactive
@aelovikov-intel aelovikov-intel temporarily deployed to aws May 21, 2023 23:09 — with GitHub Actions Inactive
@aelovikov-intel aelovikov-intel temporarily deployed to aws May 22, 2023 04:19 — with GitHub Actions Inactive
@aelovikov-intel aelovikov-intel temporarily deployed to aws May 22, 2023 04:55 — with GitHub Actions Inactive
@aelovikov-intel aelovikov-intel changed the title [SYCL][DevOps] WIP [SYCL][DevOps] Fuse OCL CPU testing into GEN9 Linux task May 22, 2023
@aelovikov-intel aelovikov-intel marked this pull request as ready for review May 22, 2023 17:09
@aelovikov-intel aelovikov-intel requested review from a team as code owners May 22, 2023 17:09
@aelovikov-intel
[SYCL][DevOps] Fuse OCL CPU testing into GEN9 Linux task
a8cdcbf 
It would increase its duration by several minutes while eliminating 20+
minutes task completely.
bader
bader reviewed May 22, 2023
View reviewed changes
devops/test_configs.json Outdated Show resolved Hide resolved
sycl/test-e2e/Config/allowlist.cpp Show resolved Hide resolved
@aelovikov-intel aelovikov-intel mentioned this pull request May 22, 2023
Open
@aelovikov-intel aelovikov-intel temporarily deployed to aws May 22, 2023 18:53 — with GitHub Actions Inactive
@bader
Copy link
Contributor

bader commented May 22, 2023

It would increase its duration by several minutes while eliminating 20+ minutes task completely.

"several minutes" - could you be more specific, please?

I think it's worth noting that in your pre-commit the task took < 12 minutes (https://github.com/intel/llvm/actions/runs/5048597334/jobs/9057922308).
Another observation: separate workflow is usually executed on runners with AMD GPU, whereas "merged workflow" can only be executed on runners with Intel GPU (which are already the main CI system bottleneck).

@aelovikov-intel
Copy link
Contributor Author

"several minutes" - could you be more specific, please?

33 vs 30 minutes

I think it's worth noting that in your pre-commit the task took < 12 minutes

That was helpful. Turns out that we have x86-cpu property set on both AMD/Intel GEN9 runners. When dispatched onto a GEN9 runner it takes 20 minutes. And that happens when CI is already slow, and we start putting even more tasks from AMD onto GEN9 runners.

As such, it would be even more helpful than I originally thought for the times with peak loads.

Another observation: separate workflow is usually executed on runners with AMD GPU, whereas "merged workflow" can only be executed on runners with Intel GPU (which are already the main CI system bottleneck).

I think that by decreasing the load on AMD runners we will get more heavy build jobs dispatched onto them, freeing our GEN9 runners just for this fused task.

@aelovikov-intel aelovikov-intel temporarily deployed to aws May 22, 2023 19:52 — with GitHub Actions Inactive
bader
bader approved these changes May 22, 2023
View reviewed changes
Copy link
Contributor

@bader bader left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI scripts changes look okay to me.

@aelovikov-intel aelovikov-intel merged commit 18b745c into intel:sycl May 23, 2023
@aelovikov-intel aelovikov-intel deleted the ci-fuse-cpu branch May 23, 2023 17:32
@aelovikov-intel aelovikov-intel mentioned this pull request Jun 26, 2023
Merged
aelovikov-intel added a commit that referenced this pull request Jun 27, 2023
@aelovikov-intel
[CI] Fix GPU reset (#10085)
2ddb42c 
#9546 and similar accidentally broke
code that was responsible to reset Intel GPU before running End-to-End
tests. Restore functionality + add reset for the pre-commit task running
End-to-End tests with a nightly build.
Chenyang-L pushed a commit that referenced this pull request Jul 11, 2023
@aelovikov-intel @Chenyang-L
[CI] Fix GPU reset (#10085)
85d0f95 
#9546 and similar accidentally broke
code that was responsible to reset Intel GPU before running End-to-End
tests. Restore functionality + add reset for the pre-commit task running
End-to-End tests with a nightly build.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bader bader bader approved these changes

@bso-intel bso-intel bso-intel approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@aelovikov-intel @bader @bso-intel