Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[develop]: Remove shell workflow. #764

Conversation

danielabdi-noaa
Copy link
Collaborator

@danielabdi-noaa danielabdi-noaa commented May 11, 2022

DESCRIPTION OF CHANGES:

This PR removes the shell workflow. Almost all of the duplicate files under ush/ are removed, choosing either the shell or python version. Two files still need both versions since it turned out to be difficult to use the python versions in job scripts without regional_workflow environment activated there. The PR will make updates much easier because there is only one workflow generation path to worry about.

TESTS CONDUCTED:

I have run tests successfully on HERA, ORION and JET. The set of tests are taken from a recent run using the shell workflow generation path done in PR #744 slighltly modified to use the new "custom_ESG/GFDLgrid" names.

Out of 56 tests

  • community_ensemble_008mems
  • community_ensemble_2mems
  • deactivate_tasks
  • grid_CONUS_25km_GFDLgrid_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_CONUS_3km_GFDLgrid_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_AK_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_AK_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2
  • grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp
  • grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp_regional
  • grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2
  • grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_RAP_suite_HRRR
  • grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_2017_gfdlmp
  • grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v15p2
  • grid_RRFS_CONUS_25km_ics_NAM_lbcs_NAM_suite_HRRR
  • grid_RRFS_CONUS_25km_ics_NAM_lbcs_NAM_suite_RRFS_v1beta
  • grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2
  • grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15_thompson_mynn_lam3km
  • grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_CONUScompact_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_CONUScompact_13km_ics_HRRR_lbcs_RAP_suite_HRRR
  • grid_RRFS_CONUScompact_13km_ics_HRRR_lbcs_RAP_suite_RRFS_v1alpha
  • grid_RRFS_CONUScompact_13km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta
  • grid_RRFS_CONUScompact_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_HRRR_suite_HRRR
  • grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_HRRR_suite_RRFS_v1beta
  • grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_HRRR
  • grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1alpha
  • grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta
  • grid_RRFS_CONUScompact_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_CONUScompact_3km_ics_HRRR_lbcs_RAP_suite_GFS_v15p2
  • grid_RRFS_CONUScompact_3km_ics_HRRR_lbcs_RAP_suite_HRRR
  • grid_RRFS_CONUScompact_3km_ics_HRRR_lbcs_RAP_suite_RRFS_v1alpha
  • grid_RRFS_CONUScompact_3km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta
  • grid_RRFS_NA_13km_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1beta
  • grid_RRFS_SUBCONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_SUBCONUS_3km_ics_HRRR_lbcs_RAP_suite_GFS_v15p2
  • grid_SUBCONUS_Ind_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_SUBCONUS_Ind_3km_ics_HRRR_lbcs_RAP_suite_HRRR
  • grid_SUBCONUS_Ind_3km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta
  • inline_post
  • MET_ensemble_verification
  • MET_verification
  • nco_ensemble
  • nco_grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_HRRR
  • custom_ESGgrid
  • custom_GFDLgrid
  • custom_GFDLgrid__GFDLgrid_USE_GFDLgrid_RES_IN_FILENAMES_eq_FALSE
  • custom_GFDLgrid__GFDLgrid_USE_GFDLgrid_RES_IN_FILENAMES_eq_TRUE
  • pregen_grid_orog_sfc_climo
  • specify_DOT_OR_USCORE
  • specify_DT_ATMOS_LAYOUT_XY_BLOCKSIZE
  • specify_RESTART_INTERVAL
  • specify_template_filenames

these are the failures where the reason is mentioned in brackets

HERA
0 failures

ORION
1 failure

nco_grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_HRRR (run_fcst segfault)

JET
only 5 failures which is a lot of improvement since last time I tried.

MET_ensemble_verification (run_gridstatvx_)
MET_verification (run_gridstatvx)
grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_HRRR_suite_HRRR (make_lbcs)
nco_ensemble (run_fcst)
pregen_grid_orog_sfc_climo (run_fcst)

DEPENDENCIES:

None

DOCUMENTATION:

None

ISSUE (optional):

None

CONTRIBUTORS (optional):

@christinaholtNOAA @venitahagerty

@JeffBeck-NOAA
Copy link
Collaborator

JeffBeck-NOAA commented May 22, 2022

WE2E using the config.community.sh file succeeded on Hera.

@JeffBeck-NOAA JeffBeck-NOAA added the Tested on Hera Tested successfully on Hera machine label May 22, 2022
@danielabdi-noaa danielabdi-noaa force-pushed the feature/remove_shell_workflow branch 3 times, most recently from d43cfe2 to 36e42b7 Compare May 23, 2022 16:52
@danielabdi-noaa danielabdi-noaa force-pushed the feature/remove_shell_workflow branch from 36e42b7 to 8659e07 Compare May 23, 2022 17:45
@danielabdi-noaa danielabdi-noaa added the Tested on Jet Successfully tested on Jet machine label Jun 16, 2022
@danielabdi-noaa danielabdi-noaa force-pushed the feature/remove_shell_workflow branch 2 times, most recently from 7fc93ac to cd22251 Compare June 27, 2022 19:30
@danielabdi-noaa danielabdi-noaa force-pushed the feature/remove_shell_workflow branch from cd22251 to 9f88f60 Compare July 14, 2022 20:25
@JeffBeck-NOAA JeffBeck-NOAA added the Tested on Cheyenne Successfully tested on NCAR Cheyenne machine label Jul 15, 2022
Copy link
Contributor

@christinaholtNOAA christinaholtNOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@danielabdi-noaa danielabdi-noaa merged commit a1885f4 into ufs-community:develop Jul 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Tested on Cheyenne Successfully tested on NCAR Cheyenne machine Tested on Hera Tested successfully on Hera machine Tested on Jet Successfully tested on Jet machine
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants