Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix duplicated regtests #716

Merged
merged 3 commits into from
May 25, 2022
Merged

fix duplicated regtests #716

merged 3 commits into from
May 25, 2022

Conversation

aliabdolali
Copy link
Contributor

@aliabdolali aliabdolali commented May 20, 2022 •
edited
Loading

Pull Request Summary

Instead of running 4 of the same tests, update the input for MPI to match serial for PR3_UNO_MPI, PR3_UNO_MPI_SCRIP and PR3_UNO_MPI for mww3_test_02 tests

Description

Updates matrix.base

Issue(s) addressed

fixes #707

Commit Message

Update duplicative regression tests (2)

Check list

Testing

  • How were these changes tested? matrix

  • Are the changes covered by regression tests? yes

  • Have the matrix regression tests been run (if yes, please note HPC and compiler)? intel ncep

  • Please indicate the expected changes in the regression test output, (Note the list of known non-identical tests.)
    The updated tests are different:

  • Please provide the summary output of matrix.comp (matrix.Diff.txt, matrixCompFull.txt and matrixCompSummary.txt):

@aliabdolali aliabdolali added the bug Something isn't working label May 20, 2022
@JessicaMeixner-NOAA
Copy link
Collaborator

JessicaMeixner-NOAA commented May 20, 2022 •
edited
Loading

@aliabdolali please see the branch mentioned in #707 (comment) there are a few more places where duplicative regression tests exist beyond the lists @kgerheiser mentioned and you fixed here.

@aliabdolali
Merge pull request #1 from JessicaMeixner-NOAA/bug/regtests
ceeae6f 
update matrix.base to remove more duplicitive regression tests
@aliabdolali
Copy link
Contributor Author

@JessicaMeixner-NOAA I merged your branch to this one. Tests are submitted. Thanks for providing them.

@aliabdolali
Copy link
Contributor Author

aliabdolali commented May 24, 2022 •
edited
Loading

The tests ran with changes in the following cases (all expected):

**********************************************************************
********************* non-identical cases ****************************
**********************************************************************
mww3_test_02/./work_PR3_UQ_MPI_b_c                     (55 files differ)
mww3_test_02/./work_PR2_UQ_MPI_c                     (55 files differ)
mww3_test_02/./work_PR2_UQ_MPI_b                     (55 files differ)
mww3_test_02/./work_PR3_UNO_MPI_c                     (55 files differ)
mww3_test_02/./work_PR3_UQ_MPI_c_c                     (2 files differ)
mww3_test_02/./work_PR3_UQ_MPI_d_c                     (5 files differ)
mww3_test_02/./work_PR2_UQ_MPI_d                     (5 files differ)
mww3_test_02/./work_PR3_UQ_MPI_d                     (5 files differ)
mww3_test_02/./work_PR3_UNO_MPI_d                     (5 files differ)
mww3_test_02/./work_PR3_UQ_MPI_b                     (55 files differ)
mww3_test_02/./work_PR3_UNO_MPI_d_c                     (5 files differ)
mww3_test_02/./work_PR3_UNO_MPI_b_c                     (55 files differ)
mww3_test_02/./work_PR3_UNO_MPI_c_c                     (2 files differ)
mww3_test_02/./work_PR3_UQ_MPI_c                     (55 files differ)
mww3_test_02/./work_PR3_UNO_MPI_b                     (55 files differ)

and preknown non-identical cases:

mww3_test_03/./work_PR2_UQ_MPI_d2                     (12 files differ)
mww3_test_03/./work_PR2_UNO_MPI_d2                     (12 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2_c                     (15 files differ)
mww3_test_03/./work_PR1_MPI_d2                     (18 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2_c                     (12 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2                     (15 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2                     (15 files differ)
ww3_tp2.10/./work_MPI_OMPH                     (5 files differ)
ww3_tp2.16/./work_MPI_OMPH                     (4 files differ)
ww3_ufs1.3/./work_a                     (1 files differ)

matrixCompFull.txt
matrixCompSummary.txt
matrixDiff.txt

@aliabdolali
Copy link
Contributor Author

The tests ran with changes in the following cases (all expected):

**********************************************************************
********************* non-identical cases ****************************
**********************************************************************
mww3_test_02/./work_PR3_UQ_MPI_b_c                     (55 files differ)
mww3_test_02/./work_PR2_UQ_MPI_c                     (55 files differ)
mww3_test_02/./work_PR2_UQ_MPI_b                     (55 files differ)
mww3_test_02/./work_PR3_UNO_MPI_c                     (55 files differ)
mww3_test_02/./work_PR3_UQ_MPI_c_c                     (2 files differ)
mww3_test_02/./work_PR3_UQ_MPI_d_c                     (5 files differ)
mww3_test_02/./work_PR2_UQ_MPI_d                     (5 files differ)
mww3_test_02/./work_PR3_UQ_MPI_d                     (5 files differ)
mww3_test_02/./work_PR3_UNO_MPI_d                     (5 files differ)
mww3_test_02/./work_PR3_UQ_MPI_b                     (55 files differ)
mww3_test_02/./work_PR3_UNO_MPI_d_c                     (5 files differ)
mww3_test_02/./work_PR3_UNO_MPI_b_c                     (55 files differ)
mww3_test_02/./work_PR3_UNO_MPI_c_c                     (2 files differ)
mww3_test_02/./work_PR3_UQ_MPI_c                     (55 files differ)
mww3_test_02/./work_PR3_UNO_MPI_b                     (55 files differ)

and preknown non-identical cases:

mww3_test_03/./work_PR2_UQ_MPI_d2                     (12 files differ)
mww3_test_03/./work_PR2_UNO_MPI_d2                     (12 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2_c                     (15 files differ)
mww3_test_03/./work_PR1_MPI_d2                     (18 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2_c                     (12 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2                     (15 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2                     (15 files differ)
ww3_tp2.10/./work_MPI_OMPH                     (5 files differ)
ww3_tp2.16/./work_MPI_OMPH                     (4 files differ)
ww3_ufs1.3/./work_a                     (1 files differ)

@JessicaMeixner-NOAA
Copy link
Collaborator

@aliabdolali were the files to big to post the full txt files from the regression tests?

@aliabdolali
Copy link
Contributor Author

@aliabdolali were the files to big to post the full txt files from the regression tests?

oops, I forgot it, just updated it.

JessicaMeixner-NOAA
JessicaMeixner-NOAA approved these changes May 24, 2022
View reviewed changes
Copy link
Collaborator

@JessicaMeixner-NOAA JessicaMeixner-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aliabdolali Have we confirmed if any of these new tests have reproducibility issues? Either way, the PR looks good and should be merged.

@aliabdolali
Copy link
Contributor Author

@aliabdolali Have we confirmed if any of these new tests have reproducibility issues? Either way, the PR looks good and should be merged.

Yes, they are reproducible.

@aliabdolali aliabdolali merged commit acc7afc into NOAA-EMC:develop May 25, 2022
@ukmo-ccbunney ukmo-ccbunney mentioned this pull request May 26, 2022
Merged
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JessicaMeixner-NOAA JessicaMeixner-NOAA Awaiting requested review from JessicaMeixner-NOAA

Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Regtest run multiple times with the same input
2 participants
@aliabdolali @JessicaMeixner-NOAA