Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable SBOM strace for all Linux platforms #1065

Merged
merged 7 commits into from
Jul 5, 2024

Conversation

andrew-m-leonard
Copy link
Contributor

@andrew-m-leonard andrew-m-leonard commented Jul 1, 2024

Fixes #1064

Enable SBOM strace for all Linux platforms (except riscv64 which fails in emulation environment)

Note:

  1. temurin aarch64 builds are 160 core, which exhausts memory when using strace, so are throttled to 16 cores.
  2. temurin arm32 builds run on 160 core hosts, which exhausts resources when using strace causing very very slow build (8+hours), so are throttled to 4 cores (builds in under 1 hour !).

Also, fix "cleanWorkspaceBuildOutputAfterBuild" so it cleans straceOutput, since it is about 3Gb in size!

Signed-off-by: Andrew Leonard <anleonar@redhat.com>
Copy link

github-actions bot commented Jul 1, 2024

Thank you for creating a pull request!

Please check out the information below if you have not made a pull request here before (or if you need a reminder how things work).

Code Quality and Contributing Guidelines

If you have not done so already, please familiarise yourself with our Contributing Guidelines and Code Of Conduct, even if you have contributed before.

Tests

Github actions will run a set of jobs against your PR that will lint and unit test your changes. Keep an eye out for the results from these on the latest commit you submitted. For more information, please see our testing documentation.

In order to run the advanced pipeline tests (executing a set of mock pipelines), it requires an admin to post run tests on this PR.
If you are not an admin, please ask for one's attention in #infrastructure on Slack or ping one here.
To run full set of tests, use "run tests"; a subset of tests on specific jdk version, use "run tests quick 11,21"

Signed-off-by: Andrew Leonard <anleonar@redhat.com>
@github-actions github-actions bot added the aarch label Jul 2, 2024
@andrew-m-leonard
Copy link
Contributor Author

run tests

@github-actions github-actions bot added the arm label Jul 2, 2024
@andrew-m-leonard
Copy link
Contributor Author

jdk8u mac fails for un-related issue:

12:50:15  xcrun: error: unable to load libxcrun (dlopen(/Applications/Xcode-11.7.app/Contents/Developer/usr/lib/libxcrun.dylib, 0x0005): tried: '/Applications/Xcode-11.7.app/Contents/Developer/usr/lib/libxcrun.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need '')), '/System/Volumes/Preboot/Cryptexes/OS/Applications/Xcode-11.7.app/Contents/Developer/usr/lib/libxcrun.dylib' (no such file), '/Applications/Xcode-11.7.app/Contents/Developer/usr/lib/libxcrun.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need ''))).

Signed-off-by: Andrew Leonard <anleonar@redhat.com>
@andrew-m-leonard
Copy link
Contributor Author

run tests

@andrew-m-leonard andrew-m-leonard requested a review from sxa July 2, 2024 15:52
Copy link
Member

@sxa sxa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok for this to go in for now, but noting that this will have a significant impact on the build times by reducing the jobs, and also this is hard coding to values which won't be great if we run on a machine with smaller numbers of cores.

@karianna
Copy link
Contributor

karianna commented Jul 3, 2024

@andrew-m-leonard holding off merging since the jdk 11 on Linux went over 8 hours, was this new feature the cause? not sure :-)

@eclipse-temurin-bot
Copy link
Collaborator

 PR TESTER RESULT 

❎ Some pipelines failed or the job was aborted! ❎
See the pipeline-build-check below for more information...

@andrew-m-leonard
Copy link
Contributor Author

@andrew-m-leonard holding off merging since the jdk 11 on Linux went over 8 hours, was this new feature the cause? not sure :-)

thanks @karianna yes, so that's interesting, i've just anaylsed why jdk11u x64Linux took so long, and I think it is because it is using ccache, which results in thousands of these effectively unknown package C source being used:

18:27:41  /home/jenkins/.ccache/a/8/fd306b20290e3d6fb387593c6b7b5a-140229.o.93ac9f76d35a.84120.3WdDjJ

Need to think about that one

@andrew-m-leonard andrew-m-leonard marked this pull request as draft July 3, 2024 08:01
Signed-off-by: Andrew Leonard <anleonar@redhat.com>
@andrew-m-leonard
Copy link
Contributor Author

run tests

@eclipse-temurin-bot
Copy link
Collaborator

 PR TESTER RESULT 

❎ Some pipelines failed or the job was aborted! ❎
See the pipeline-build-check below for more information...

Signed-off-by: Andrew Leonard <anleonar@redhat.com>
@andrew-m-leonard
Copy link
Contributor Author

run tests

@eclipse-temurin-bot
Copy link
Collaborator

 PR TESTER RESULT 

✅ All pipelines passed! ✅

@andrew-m-leonard
Copy link
Contributor Author

run tests

@eclipse-temurin-bot
Copy link
Collaborator

 PR TESTER RESULT 

❎ Some pipelines failed or the job was aborted! ❎
See the pipeline-build-check below for more information...

@andrew-m-leonard
Copy link
Contributor Author

run tests

@eclipse-temurin-bot
Copy link
Collaborator

 PR TESTER RESULT 

✅ All pipelines passed! ✅

@andrew-m-leonard andrew-m-leonard marked this pull request as ready for review July 5, 2024 12:36
@andrew-m-leonard andrew-m-leonard merged commit 259c9a6 into adoptium:master Jul 5, 2024
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable Linux SBOM strace for all Linux platforms and all versions
4 participants