Add support for forecast-only low resolution #1648

JessicaMeixner-NOAA · 2023-05-24T23:16:16Z

Description

This PR ensures that we can run C96 5deg and C48 5 deg for 2021032312.

ICs were staged on hera from @sanatcumar and @guillaumevernieres so that this can be run "out of the box", but will have to be transferred to other machines if desired.

This PR requires two fix file updates:
#1647
#1645
that must be completed before this is merged.

Thank you to: @sanatcumar @barlage @HelinWei-NOAA Bing Fu @jiandewang @guillaumevernieres @aerorahul @NeilBarton-NOAA and others who helped provide input, fix files, directions, and other help!

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How Has This Been Tested?

Hera C96 5deg S2S forecast only 2021032312 see: /scratch1/NCEPDEV/climate/Jessica.Meixner/lowresgw/newtest/c96o500
NOTE The default for C96 is 1 deg therefore you must change your config.base to have the correct ocean resolution
Hera C48 5deg S2S forecast only 2021032312 see: /scratch1/NCEPDEV/climate/Jessica.Meixner/lowresgw/newtest/c48o500
Hera C384 .25deg S2S forecast only 20130101 see: /scratch1/NCEPDEV/climate/Jessica.Meixner/lowresgw/newtest/c384o025

Checklist

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
My changes need updates to the documentation. I have made corresponding changes to the documentation
My changes generate no new warnings
New and existing tests pass with my changes
Any dependent changes have been merged and published

WalterKolczynski-NOAA

Looks good so far.

workflow/rocoto/workflow_tasks.py

--- fix from Jiande Wang

JessicaMeixner-NOAA · 2023-05-25T19:44:19Z

There's been an update removing a "to do" item in this PR from @jiandewang who removed the 025 resolution dependency in the ocean post. The only thing that changes is meta-data. The three tests with this new output can be found here:
/scratch1/NCEPDEV/climate/Jessica.Meixner/lowresgw/oceanupdate/co for the various resolutions.

WalterKolczynski-NOAA · 2023-05-26T07:10:24Z

New cpl fix version 20230526 created for this update. Orog fix changes was just additions, so existing version (20220805) is still used.

JessicaMeixner-NOAA · 2023-05-26T12:32:54Z

I have updated the version for cpl and am currently working to test a fresh clone for out of the box runs on hera. Note the ICs are currently only staged on hera, so if these tests are desired for other machines we need to have @jiandewang move them to WCOSS2 and others to move to orion into the IC staged area.

Please let me know what else I can do for this PR. Note, I also just finished adding a low res wave capability to this, which should get a new PR for that soon as well.

JessicaMeixner-NOAA · 2023-05-26T19:02:05Z

Latest test will be here: /scratch1/NCEPDEV/climate/Jessica.Meixner/lowresgw/usefixfromofficial/c48o500

WalterKolczynski-NOAA

Approved pending CI testing

emcbot · 2023-05-30T06:52:41Z

Automated global-workflow Testing Results:

Machine: Orion
Start: Tue May 30 00:40:24 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Tue May 30 00:42:05 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Tue May 30 01:52:31 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Tue May 30 01:52:38 CDT 2023 for experiment C96_atm3DVar

emcbot · 2023-05-30T08:21:11Z

Automated global-workflow Testing Results:

Machine: Orion
Start: Tue May 30 00:40:24 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Tue May 30 00:42:05 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Tue May 30 01:52:31 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Tue May 30 01:52:38 CDT 2023 for experiment C96_atm3DVar
Experiment C96_atm3DVar Terminated: *FAILED*
Experiment C96_atm3DVar Terminated with 1 tasks failed at Tue May 30 03:21:07 CDT 2023
Error logs:
/work2/noaa/stmp/GFS_CI_ROOT/PR/1648/RUNTESTS/C96_atm3DVar/COMROT/C96_atm3DVar/logs/2021122100/gdasvrfy.log

JessicaMeixner-NOAA · 2023-05-30T12:45:55Z

Automated global-workflow Testing Results:

Machine: Orion
Start: Tue May 30 00:40:24 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Tue May 30 00:42:05 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Tue May 30 01:52:31 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Tue May 30 01:52:38 CDT 2023 for experiment C96_atm3DVar
Experiment C96_atm3DVar Terminated: *FAILED*
Experiment C96_atm3DVar Terminated with 1 tasks failed at Tue May 30 03:21:07 CDT 2023
Error logs:
/work2/noaa/stmp/GFS_CI_ROOT/PR/1648/RUNTESTS/C96_atm3DVar/COMROT/C96_atm3DVar/logs/2021122100/gdasvrfy.log

The error is:

+ radmon_verf_angle.sh[215](2021122100): gzip radmon_angle.tar
+ radmon_verf_angle.sh[216](2021122100): mv radmon_angle.tar.gz /work/noaa/global/mterry/monitor/radmon/stats/C96_atm3DVar/gdas.20211221/00/.
mv: cannot move ‘radmon_angle.tar.gz’ to a subdirectory of itself, ‘/work/noaa/global/mterry/monitor/radmon/stats/C96_atm3DVar/gdas.20211221/00/./radmon_angle.tar.gz’
+ radmon_verf_angle.sh[1](2021122100): postamble radmon_verf_angle.sh 1685434259 1

However I'm not sure how any of the changes I made changed something here? I'll sync in the develop to this branch in case that's the issue?

WalterKolczynski-NOAA · 2023-05-30T14:22:02Z

Automated global-workflow Testing Results:

Machine: Orion
Start: Tue May 30 00:40:24 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Tue May 30 00:42:05 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Tue May 30 01:52:31 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Tue May 30 01:52:38 CDT 2023 for experiment C96_atm3DVar
Experiment C96_atm3DVar Terminated: *FAILED*
Experiment C96_atm3DVar Terminated with 1 tasks failed at Tue May 30 03:21:07 CDT 2023
Error logs:
/work2/noaa/stmp/GFS_CI_ROOT/PR/1648/RUNTESTS/C96_atm3DVar/COMROT/C96_atm3DVar/logs/2021122100/gdasvrfy.log

The error is:

+ radmon_verf_angle.sh[215](2021122100): gzip radmon_angle.tar
+ radmon_verf_angle.sh[216](2021122100): mv radmon_angle.tar.gz /work/noaa/global/mterry/monitor/radmon/stats/C96_atm3DVar/gdas.20211221/00/.
mv: cannot move ‘radmon_angle.tar.gz’ to a subdirectory of itself, ‘/work/noaa/global/mterry/monitor/radmon/stats/C96_atm3DVar/gdas.20211221/00/./radmon_angle.tar.gz’
+ radmon_verf_angle.sh[1](2021122100): postamble radmon_verf_angle.sh 1685434259 1

However I'm not sure how any of the changes I made changed something here? I'll sync in the develop to this branch in case that's the issue?

@EdwardSafford-NOAA would you please take a look at this failure? It seems unrelated to @JessicaMeixner-NOAA's PR, but CI tests passed before this. Might just be a fluke this PR hit it, but it needs to be fixed.

WalterKolczynski-NOAA · 2023-05-30T15:04:22Z

Oops, hit the wrong button.

@EdwardSafford-NOAA @JessicaMeixner-NOAA I'm going to rerun the CI test, so I copied the rotdir and radmon directories to /work2/noaa/stmp/wkolczyn/PR1648 to continue examination.

EdwardSafford-NOAA · 2023-05-30T15:43:55Z

@WalterKolczynski-NOAA I agree this error is likely unrelated to changes in this PR.

emcbot · 2023-05-30T16:20:46Z

Automated global-workflow Testing Results:

Machine: Orion
Start: Tue May 30 10:09:00 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Tue May 30 10:10:42 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Tue May 30 11:20:37 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Tue May 30 11:20:44 CDT 2023 for experiment C96_atm3DVar

emcbot · 2023-05-30T18:42:13Z

Automated global-workflow Testing Results:

Machine: Orion
Start: Tue May 30 10:09:00 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Tue May 30 10:10:42 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Tue May 30 11:20:37 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Tue May 30 11:20:44 CDT 2023 for experiment C96_atm3DVar
Experiment C96_atm3DVar completed: *SUCCESS*
Experiment C96_atm3DVar Completed at Tue May 30 13:42:12 CDT 2023
with 73 successfully completed jobs

emcbot · 2023-05-30T18:49:06Z

Automated global-workflow Testing Results:

Machine: Orion
Start: Tue May 30 10:09:00 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Tue May 30 10:10:42 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Tue May 30 11:20:37 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Tue May 30 11:20:44 CDT 2023 for experiment C96_atm3DVar
Experiment C96_atm3DVar completed: *SUCCESS*
Experiment C96_atm3DVar Completed at Tue May 30 13:42:12 CDT 2023
with 73 successfully completed jobs

JessicaMeixner-NOAA · 2023-05-30T18:50:58Z

@WalterKolczynski-NOAA @EdwardSafford-NOAA the CI seems to be working, but I'm not sure what changed other than merging in the develop, was that the issue?

WalterKolczynski-NOAA · 2023-05-30T18:57:04Z

@WalterKolczynski-NOAA @EdwardSafford-NOAA the CI seems to be working, but I'm not sure what changed other than merging in the develop, was that the issue?

Maybe, which is kinda why I reran it. Though I don't know why that should've mattered and it still seems like a bug, even if it went away here.

JessicaMeixner-NOAA · 2023-05-30T18:58:22Z

@WalterKolczynski-NOAA anything I need to do on my end?

WalterKolczynski-NOAA · 2023-05-30T19:09:55Z

@JessicaMeixner-NOAA Not now, just waiting for the CI test to complete and go green.

Edit: Oh wait, it did. Shellcheck complaints are trivial and in-line with the surrounding code. Merging.

This adds an option for 2 or 5 deg global wave models, which will provide a light weight version to assist in engineering development. -- This PR requires PR #1648 to be merged as this builds on it as well as issue #1652 which adds the fix files needed for this PR to be addressed first. The resources were chosen to stay under 1 or 2 node and be faster than the atm component. - Fixes #1333

JessicaMeixner-NOAA added 2 commits May 23, 2023 20:16

updates to run low res C96mx500 or C48mx500

d49e56b

Merge remote-tracking branch 'EMC/develop' into trylowres

f4230bc

JessicaMeixner-NOAA requested a review from aerorahul May 24, 2023 23:16

JessicaMeixner-NOAA changed the title ~~Trylowres~~ Low resolution forecast only May 24, 2023

This was referenced May 24, 2023

C96.mx500 data for orog fix files #1647

Closed

New set of cpl fix files #1645

Closed

WalterKolczynski-NOAA requested changes May 25, 2023

View reviewed changes

workflow/rocoto/workflow_tasks.py Outdated Show resolved Hide resolved

WalterKolczynski-NOAA changed the title ~~Low resolution forecast only~~ Add support for forecast-only low resolution May 25, 2023

JessicaMeixner-NOAA added 2 commits May 25, 2023 12:24

delete old code

a5ff0f0

update to make ocnpost remove resolution dependency

0983d12

--- fix from Jiande Wang

WalterKolczynski-NOAA self-requested a review May 26, 2023 07:02

updated cpl version number

ecdce7d

JessicaMeixner-NOAA marked this pull request as ready for review May 26, 2023 12:29

JessicaMeixner-NOAA mentioned this pull request May 26, 2023

Add support for waves at low resolution #1653

Merged

13 tasks

WalterKolczynski-NOAA added the CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion label May 30, 2023

WalterKolczynski-NOAA approved these changes May 30, 2023

View reviewed changes

emcbot added CI-Orion-Failed **Bot use only** CI testing on Orion for this PR has failed and removed CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress labels May 30, 2023

Merge branch 'NOAA-EMC:develop' into trylowres

a7471b7

WalterKolczynski-NOAA closed this May 30, 2023

WalterKolczynski-NOAA reopened this May 30, 2023

WalterKolczynski-NOAA added CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion and removed CI-Orion-Failed **Bot use only** CI testing on Orion for this PR has failed labels May 30, 2023

emcbot added CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion and removed CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion labels May 30, 2023

emcbot added CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress and removed CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion labels May 30, 2023

emcbot added CI-Orion-Passed **Bot use only** CI testing on Orion for this PR has completed successfully and removed CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress labels May 30, 2023

WalterKolczynski-NOAA merged commit d6ae3d0 into NOAA-EMC:develop May 30, 2023

JessicaMeixner-NOAA deleted the trylowres branch June 1, 2023 17:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for forecast-only low resolution #1648

Add support for forecast-only low resolution #1648

JessicaMeixner-NOAA commented May 24, 2023 •

edited

Loading

WalterKolczynski-NOAA left a comment

JessicaMeixner-NOAA commented May 25, 2023

WalterKolczynski-NOAA commented May 26, 2023 •

edited

Loading

JessicaMeixner-NOAA commented May 26, 2023

JessicaMeixner-NOAA commented May 26, 2023

WalterKolczynski-NOAA left a comment

emcbot commented May 30, 2023

emcbot commented May 30, 2023

JessicaMeixner-NOAA commented May 30, 2023

WalterKolczynski-NOAA commented May 30, 2023

WalterKolczynski-NOAA commented May 30, 2023

EdwardSafford-NOAA commented May 30, 2023

emcbot commented May 30, 2023

emcbot commented May 30, 2023

emcbot commented May 30, 2023

JessicaMeixner-NOAA commented May 30, 2023

WalterKolczynski-NOAA commented May 30, 2023

JessicaMeixner-NOAA commented May 30, 2023

WalterKolczynski-NOAA commented May 30, 2023 •

edited

Loading

Add support for forecast-only low resolution #1648

Add support for forecast-only low resolution #1648

Conversation

JessicaMeixner-NOAA commented May 24, 2023 • edited Loading

WalterKolczynski-NOAA left a comment

Choose a reason for hiding this comment

JessicaMeixner-NOAA commented May 25, 2023

WalterKolczynski-NOAA commented May 26, 2023 • edited Loading

JessicaMeixner-NOAA commented May 26, 2023

JessicaMeixner-NOAA commented May 26, 2023

WalterKolczynski-NOAA left a comment

Choose a reason for hiding this comment

emcbot commented May 30, 2023

emcbot commented May 30, 2023

JessicaMeixner-NOAA commented May 30, 2023

WalterKolczynski-NOAA commented May 30, 2023

WalterKolczynski-NOAA commented May 30, 2023

EdwardSafford-NOAA commented May 30, 2023

emcbot commented May 30, 2023

emcbot commented May 30, 2023

emcbot commented May 30, 2023

JessicaMeixner-NOAA commented May 30, 2023

WalterKolczynski-NOAA commented May 30, 2023

JessicaMeixner-NOAA commented May 30, 2023

WalterKolczynski-NOAA commented May 30, 2023 • edited Loading

JessicaMeixner-NOAA commented May 24, 2023 •

edited

Loading

WalterKolczynski-NOAA commented May 26, 2023 •

edited

Loading

WalterKolczynski-NOAA commented May 30, 2023 •

edited

Loading