Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Test coverage information is not completely carried over to codecov #4801

Closed
nibix opened this issue Oct 9, 2024 · 5 comments · Fixed by #4802
Closed

[CI] Test coverage information is not completely carried over to codecov #4801

nibix opened this issue Oct 9, 2024 · 5 comments · Fixed by #4802
Labels
bug Something isn't working untriaged Require the attention of the repository maintainers and may need to be prioritized

Comments

@nibix
Copy link
Collaborator

nibix commented Oct 9, 2024

On the PR #4380 I am noticing that the test coverage information that is displayed by codecov does not match the test coverage I see in my local dev environment. On first sight, it seems that the coverage from the integrationTest directory is missing.

For example:

Coverage reported by codecov for the class ActionPrivileges:

Screenshot from 2024-10-09 12-20-22

Coverage reported by Intellij just after running ActionPriviliegesTest (in integrationTest) :

Screenshot from 2024-10-09 12-22-39

While IntelliJ shows full coverage, codecov shows completely missing coverage.

My current suspicion is that the coverage information from integrationTest is not evaluated by codecov. But that's speculation for now, I could not find any clues so far where the issue might come from.

@nibix nibix added bug Something isn't working untriaged Require the attention of the repository maintainers and may need to be prioritized labels Oct 9, 2024
@nibix
Copy link
Collaborator Author

nibix commented Oct 9, 2024

I just noticed that the ci.yml script uses actions/upload-artifact@v4 to upload the reports from test while it uses alehechka/upload-tartifact@v2 to upload the reports from integrationTest 🤔

@nibix
Copy link
Collaborator Author

nibix commented Oct 9, 2024

Output of the Display structure of downloaded files step of the report-coverage job. The integration test data should have the name pattern integration-${{ matrix.platform }}-JDK${{ matrix.jdk }}-reports. This is missing.

.:
ubuntu-latest-JDK21-ciSecurityIntegrationTest-reports
ubuntu-latest-JDK21-citest-reports
ubuntu-latest-JDK21-crossClusterTest-reports
ubuntu-latest-JDK21-dlicDlsflsTest-reports
ubuntu-latest-JDK21-dlicRestApiTest-reports
ubuntu-latest-JDK21-indicesTest-reports
ubuntu-latest-JDK21-opensslCITest-reports
ubuntu-latest-JDK21-sslTest-reports
windows-latest-JDK21-ciSecurityIntegrationTest-reports
windows-latest-JDK21-citest-reports
windows-latest-JDK21-crossClusterTest-reports
windows-latest-JDK21-dlicDlsflsTest-reports
windows-latest-JDK21-dlicRestApiTest-reports
windows-latest-JDK21-indicesTest-reports
windows-latest-JDK21-opensslCITest-reports
windows-latest-JDK21-sslTest-reports

@nibix
Copy link
Collaborator Author

nibix commented Oct 9, 2024

I am now trying to see what happens if I use artifact instead of tartifact: #4802

@nibix
Copy link
Collaborator Author

nibix commented Oct 9, 2024

The tartifact action was introduced in #3537. However, the explicit untar which was also introduced there is now missing from the ci.yml.

@nibix
Copy link
Collaborator Author

nibix commented Oct 9, 2024

Structure of artifacts in #4802:

integration-ubuntu-latest-JDK21-reports
integration-windows-latest-JDK21-reports
ubuntu-latest-JDK21-ciSecurityIntegrationTest-reports
ubuntu-latest-JDK21-citest-reports
ubuntu-latest-JDK21-crossClusterTest-reports
ubuntu-latest-JDK21-dlicDlsflsTest-reports
ubuntu-latest-JDK21-dlicRestApiTest-reports
ubuntu-latest-JDK21-indicesTest-reports
ubuntu-latest-JDK21-opensslCITest-reports
ubuntu-latest-JDK21-sslTest-reports
windows-latest-JDK21-ciSecurityIntegrationTest-reports
windows-latest-JDK21-citest-reports
windows-latest-JDK21-crossClusterTest-reports
windows-latest-JDK21-dlicDlsflsTest-reports
windows-latest-JDK21-dlicRestApiTest-reports
windows-latest-JDK21-indicesTest-reports
windows-latest-JDK21-opensslCITest-reports
windows-latest-JDK21-sslTest-reports

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working untriaged Require the attention of the repository maintainers and may need to be prioritized
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant