Upload test report

[albertteoh]

Which problem is this PR solving?

• Relates to Publish test report #2859
• Continues from Publish test report #5030

Description of the changes

• Enables a test report summary in the PR comment and check summary.

How was this change tested?

• Confirm upload job is running successfully in my fork: https://github.com/albertteoh/jaeger/actions/runs/7316648725.
• Can only know if it's working once the PR is created.

Checklist

• I have read https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
• I have signed all commits

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (5c6348e) 95.62% compared to head (36a32c9) 95.63%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5035      +/-   ##
==========================================
+ Coverage   95.62%   95.63%   +0.01%     
==========================================
  Files         314      314              
  Lines       18294    18294              
==========================================
+ Hits        17493    17496       +3     
+ Misses        642      640       -2     
+ Partials      159      158       -1     

Flag	Coverage Δ
cassandra-3.x	25.60% <ø> (ø)
cassandra-4.x	25.60% <ø> (ø)
elasticsearch-5.x	19.86% <ø> (-0.02%)	⬇️
elasticsearch-6.x	19.88% <ø> (ø)
elasticsearch-7.x	20.00% <ø> (ø)
elasticsearch-8.x	20.10% <ø> (ø)
grpc-badger	19.49% <ø> (ø)
kafka	14.10% <ø> (ø)
opensearch-1.x	20.01% <ø> (+0.01%)	⬆️
opensearch-2.x	20.01% <ø> (+0.01%)	⬆️
unittests	93.32% <ø> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[albertteoh]

Set to draft to test a few things.

[yurishkuro]

do we know what's different about it? It's preferable to use official actions like https://github.com/actions/download-artifact

[albertteoh]

Not specifically, but I did spend a good number of hours trying out actions/download-artifact without any luck. Oddly, it wasn't able to discover the uploaded files, but there may have been a silly mistake on my part 🤷🏼 Happy to take suggestions!

[albertteoh]

This file needs to be merged into main first before we can see the results of the test report summary in PRs, which is why we can't see it in this PR.

[yurishkuro]

Why is it necessary to separate this into a different workflow? Doing so requires all the juggling with artifact files. Can we not do all it at once directly in the unit test workflow, where we already call publish reports action?

[albertteoh]

Why is it necessary to separate this into a different workflow? Doing so requires all the juggling with artifact files. Can we not do all it at once directly in the unit test workflow, where we already call publish reports action?

It's a valid question, because one would think the action to execute the unit test and generate the results artifact is done as part of this repo's CI pipeline, so the relevant files should be easily accessible.

In fact, we're trying to do that now (publish test results as a comment to this PR) because, I agree, it's a lot simpler than juggling artifact uploads and downloads; but it's failing with this error message (from example run):

2024-01-01 12:55:11 +0000 - publish - INFO - This action is running on a pull_request event for a fork repository. Pull request comments and check runs cannot be created, so disabling these features. To fully run the action on fork repository pull requests, see https://github.com/EnricoMi/publish-unit-test-result-action/blob/v2.12.0/README.md#support-fork-repositories-and-dependabot-branches

I don't fully understand why, but from the log message, it almost sounds like a security measure to prevent PRs from another fork from writing to a PR owned by the "parent" repo, which seems to make sense.

Can you see any other alternative?

[yurishkuro]

The output of this reporting action keeps rising my blood pressure... (not your fault, Albert)

https://github.com/jaegertracing/jaeger/actions/runs/7387766706

What is "1 🔥"? When I look at JSON output there is clearly a test that failed:

{"Time":"2024-01-02T16:00:59.335298108Z","Action":"fail","Package":"github.com/jaegertracing/jaeger/plugin/storage/es","Elapsed":0.635}

[albertteoh]

What is "1 🔥"? When I look at JSON output there is clearly a test that failed

It's documented here, albeit a little vaguely that it's an "erroneous" test, as opposed to a "failed" test.

Based on your example "erroneous" test and an example "failed" test my interpretation is:

• "erroneous": a test failure caused by a panic or exit. In your example, it's triggered by a goleak check which invokes syscall.Exit.
• "failed": a test failure caused by a failed assertion.

It is a little confusing though that, in both cases, the "Action" is "failed".

[yurishkuro]

@albertteoh one of the tests failed in Go Tip run: https://github.com/jaegertracing/jaeger/actions/runs/7428429111

Is there a way to see detailed test report? The one in the summary doesn't seem to have any links.

[albertteoh]

Is there a way to see detailed test report? The one in the summary doesn't seem to have any links.

Yes (I see you've already approved, thanks!): #5082