Use tools/print_test_stats.py in GHA #57647

samestep · 2021-05-05T17:01:00Z

Judging from #57584, it seems like the test-reports artifact was originally intended to be downloaded to $PWD/test-reports instead of just directly into $PWD. To minimize confusion, this PR changes it to download into test/test-reports, which should match where the files came from in the test step anyway.

TODOs:

change the extract path for test-reports
install Python dependencies
call tools/print_test_stats.py
use deep clone to allow git commands
correctly set CIRCLE_* environment variables
set Scribe credentials
set AWS credentials

Test plan:

CI.

facebook-github-bot · 2021-05-05T17:01:06Z

💊 CI failures summary and remediations

As of commit b701767 (more details on the Dr. CI page):

3/3 failures introduced in this PR

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pytorch_xla_linux_bionic_py3_6_clang9_build (1/1)

Step: "Build" (full log | diagnosis details | 🔁 rerun)

May 10 17:12:14 Failed to generate ATEN binding...b/jenkins/workspace/xla/scripts/generate_code.sh']

May 10 17:12:12 + CXX=clang++-9
May 10 17:12:12 + sed -i '/bazel build/ a --remote_http_cache=http://localhost:7777 \\' build_torch_xla_libs.sh
May 10 17:12:12 + python setup.py install
May 10 17:12:14 Traceback (most recent call last):
May 10 17:12:14   File "/var/lib/jenkins/workspace/xla/scripts/../../tools/codegen/gen_backend_stubs.py", line 6, in <module>
May 10 17:12:14     from tools.codegen.gen import FileManager, get_grouped_native_functions, LineLoader, parse_native_yaml
May 10 17:12:14 ModuleNotFoundError: No module named 'tools'
May 10 17:12:14 Building torch_xla version: 1.9
May 10 17:12:14 XLA Commit ID: 32fe90840490e6d89673febe306c3639b1ad18a7
May 10 17:12:14 PyTorch Commit ID: 34e1ed2fd9284507d12aeffd831044116293b632
May 10 17:12:14 Failed to generate ATEN bindings: ['/var/lib/jenkins/workspace/xla/scripts/generate_code.sh']
May 10 17:12:14 =================== sccache compilation log ===================
May 10 17:12:14 + cleanup
May 10 17:12:14 + retcode=1
May 10 17:12:14 + set +x
May 10 17:12:14 =========== If your build fails, please take a look at the log above for possible reasons ===========
May 10 17:12:14 Compile requests                    4962
May 10 17:12:14 Compile requests executed           4639
May 10 17:12:14 Cache hits                          4115
May 10 17:12:14 Cache hits (C/C++)                  4115
May 10 17:12:14 Cache misses                         505

2 failures not recognized by patterns:

Job	Step	Action
^{pytorch_linux_bionic_py3_8_gcc9_coverage_test2}	^{Run tests}	🔁 rerun
^{pytorch_linux_xenial_cuda11_1_cudnn8_py3_gcc7_test2}	^{Run tests}	🔁 rerun

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

.github/templates/linux_ci_workflow.yml.in

codecov · 2021-05-06T01:39:59Z

Codecov Report

Merging #57647 (946b600) into master (cbfce37) will increase coverage by 0.01%.
The diff coverage is n/a.

❗ Current head 946b600 differs from pull request most recent head b701767. Consider uploading reports for the commit b701767 to get more accurate results

@@            Coverage Diff             @@
##           master   #57647      +/-   ##
==========================================
+ Coverage   76.83%   76.84%   +0.01%     
==========================================
  Files        1986     1980       -6     
  Lines      197402   196877     -525     
==========================================
- Hits       151668   151285     -383     
+ Misses      45734    45592     -142

facebook-github-bot · 2021-05-10T16:06:24Z

@samestep has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

janeyx99 · 2021-05-10T17:14:40Z

.github/workflows/pytorch-linux-xenial-py3.6-gcc5.4.yml

+          CIRCLE_PR_NUMBER: ${{ github.event.pull_request.number }}
+          CIRCLE_SHA1: ${{ github.event.pull_request.head.sha || github.sha }}
+          CIRCLE_TAG: ${{ steps.parse-ref.outputs.tag }}
+          CIRCLE_WORKFLOW_ID: ${{ github.run_id }} # dunno if this corresponds


What do we use this for?

dunno; @walterddr do you happen to know?

.github/templates/linux_ci_workflow.yml.in

facebook-github-bot · 2021-05-10T19:31:13Z

@samestep merged this pull request in e1cbc43.

Summary: Judging from pytorch#57584, it seems like the test-reports artifact was originally intended to be downloaded to `$PWD/test-reports` instead of just directly into `$PWD`. To minimize confusion, this PR changes it to download into `test/test-reports`, which should match where the files came from in the `test` step anyway. TODOs: - [x] change the extract path for test-reports - [x] install Python dependencies - [x] call `tools/print_test_stats.py` - [x] use deep clone to allow `git` commands - [x] correctly set `CIRCLE_*` environment variables - [x] set Scribe credentials - [x] set AWS credentials Pull Request resolved: pytorch#57647 Test Plan: CI. Reviewed By: seemethere Differential Revision: D28325833 Pulled By: samestep fbshipit-source-id: cc322bad76747f59b764a1a0a863153bb26095e7

Try using print_test_stats in render_test_results

65c70ad

facebook-github-bot added the cla signed label May 5, 2021

walterddr reviewed May 5, 2021

View reviewed changes

.github/templates/linux_ci_workflow.yml.in Outdated Show resolved Hide resolved

Try setting PYTHONPATH

acfd026

samestep mentioned this pull request May 5, 2021

Migrate build/test jobs from CircleCI to GitHub Actions #57686

Closed

71 tasks

samestep added 11 commits May 6, 2021 08:58

Merge branch 'master' into print-test-stats-gha

fb09fc3

Download artifact into test/test-reports

0bc5394

Update the template and other workflow too

41fd1ba

Fix the parts that run on PRs

340862c

Merge branch 'master' into print-test-stats-gha

1260b73

Regenerate from template

62dad7e

Merge branch 'viable/strict' into print-test-stats-gha

639179c

Fix typo

0cdbd09

Fix branch and tag by using Python instead of Bash

946b600

Merge branch 'viable/strict' into print-test-stats-gha

0f4f592

Set Scribe and AWS credentials

b701767

samestep marked this pull request as ready for review May 10, 2021 16:06

samestep requested a review from a team May 10, 2021 16:06

janeyx99 reviewed May 10, 2021

View reviewed changes

samestep requested a review from a team May 10, 2021 18:06

seemethere reviewed May 10, 2021

View reviewed changes

.github/templates/linux_ci_workflow.yml.in Show resolved Hide resolved

samestep requested a review from a team May 10, 2021 18:17

seemethere approved these changes May 10, 2021

View reviewed changes

facebook-github-bot closed this in e1cbc43 May 10, 2021

facebook-github-bot added the Merged label May 10, 2021

samestep deleted the print-test-stats-gha branch May 10, 2021 19:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use tools/print_test_stats.py in GHA #57647

Use tools/print_test_stats.py in GHA #57647

samestep commented May 5, 2021 •

edited

Loading

facebook-github-bot commented May 5, 2021 •

edited

Loading

codecov bot commented May 6, 2021 •

edited

Loading

facebook-github-bot commented May 10, 2021

janeyx99 May 10, 2021

samestep May 10, 2021

facebook-github-bot commented May 10, 2021

Use tools/print_test_stats.py in GHA #57647

Use tools/print_test_stats.py in GHA #57647

Conversation

samestep commented May 5, 2021 • edited Loading

facebook-github-bot commented May 5, 2021 • edited Loading

💊 CI failures summary and remediations

🕵️ 1 new failure recognized by patterns

pytorch_xla_linux_bionic_py3_6_clang9_build (1/1)

2 failures not recognized by patterns:

codecov bot commented May 6, 2021 • edited Loading

Codecov Report

facebook-github-bot commented May 10, 2021

janeyx99 May 10, 2021

Choose a reason for hiding this comment

samestep May 10, 2021

Choose a reason for hiding this comment

facebook-github-bot commented May 10, 2021

samestep commented May 5, 2021 •

edited

Loading

facebook-github-bot commented May 5, 2021 •

edited

Loading

codecov bot commented May 6, 2021 •

edited

Loading