JJOBs for archive and verification jobs #723
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- remove envvar module load from module-setup scripts - replace module purge with module reset on WCOSS2 Refs: NOAA-EMC#399
…re/ecflow-ops-ecf2
…oss2 Build updates based on NCO feedback and new WAFS tag
- Move enkf out of gdas and rename it to enkfgdas.
Include all ecflow definition files job name
Include all ecflow scripts name and job/log name
- Move "model=gfs" to the top on each job except all jobs under obsproc.
obsproc will no longer be part of GFS. Therefore leave it without change for testing purpose.
- Remove the source of model_ver from each ecflow script except all jobs under obsproc.
obsproc will no longer be part of GFS. Therefore leave it without change for testing purpose.
Update analysis ecflow script to use 128 for wcoss2 Remove extra CDATE
Update ecflow package for wcoss2 GFS transition
- update ROTDIR setting in NCO mode base config to use compath.py in its definition - this change supports the removal of a RUN_ENVIR=nco if-block in the JJOB scripts that set ROTDIR Refs: NOAA-EMC#399
- remove a RUN_ENVIR=nco if-block in JJOB scripts that set ROTDIR via COMROOT - ROTDIR is now set from the configuration level using compath.py - also update COMIN[COMOUT]wave paths in JGLOBAL_FORECAST to use compath.py for defaults Refs: NOAA-EMC#399
- update GLDAS tag to gldas_gfsv16_release.v1.24.0 - update WAFS to gfs_wafs.v6.2.6 Refs: NOAA-EMC#399
- remove "-o" after compath.py in COMIN definitions
- add "${envir}" and move closing ")" forward in line
Refs: NOAA-EMC#399
…back Incorporate ecflow feedback from NCO - part 1
- update WAFS tag in sorc/checkout.sh and release notes Refs: NOAA-EMC#399
…NOAA/global-workflow into feature/ops-wcoss2 * 'feature/ops-wcoss2' of https://github.com/KateFriedman-NOAA/global-workflow: revert ecflow include files to NCO versions. Will adapt as necessary for proper use
…eedback Add PBS debug directive.
- update COMIN paths in GEMPAK JJOB scripts for COMINukmet, COMINecmwf, and COMINnam to add the respective systems to the end of the path definition Refs: NOAA-EMC#399
…oss2 Updates to ROTDIR/COMIN definitions related to compath.py and new GLDAS/WAFS tags
…e and intel. Ignore swp files
…nv-intel, craype and intel
…nv-intel, craype and intel
…nv-intel, craype and intel
…load Load compiler env. and modules in the ecf scripts.
…oss2 Fold GFSv16.1.6 updates into GFSv16.2.0 package
- new GLDAS tag fixes default path for CPCGAUGE to point to group account space for GDA on WCOSS2 instead of personal account Refs: NOAA-EMC#399, NOAA-EMC#662
…oss2 Update GLDAS tag to gldas_gfsv16_release.v.2.0.0
* upstream/feature/ops-wcoss2: Update GLDAS tag to gldas_gfsv16_release.v.2.0.0 Update DMPDIR and BASE_GIT paths for WCOSS2 Update Externals.cfg with GFSv16.2.0 component versions Update release notes for current ops version Move all PBS place settings to separate line Remove commented out lines from transfer lists Update WAFS tag to gfs_wafs.v6.2.8 GFS v16.1.6: Update release notes and comment in config.anal Update npe_node_fcst_gfs in config.resources.emc.dyn Updates to support wcoss2.ver Fold in transfer parm list updates from NCO Move transfer lists into new transfer folder Update wave job resources with NCO feedback Update EMC tag name in v16.2.0 release notes Updates to run.ver and create wcoss2.ver Script updates from NCO GFS v16.1.6: GSI update to add commercial GPSRO in DO-4 Move excl setting into resource line in ecf scripts Update gfs_forecast job resources Update several versions in run.ver Add OMP_PLACES=cores for fcst block in WCOSS2.env Update compilation flags for gaussian_sfcanl build Conflicts: versions/wcoss2.ver
Sync merge feature/ops-wcoss2 into dev_v16 (EMC-v16.2.0.4 tag state)
- update the build scripts for fv3nc2nemsio, regrid_nemsio, and tropcy_NEMS to set corrected LIBS and NETCDF_LDFLAGS flags Refs: NOAA-EMC#665
- update the LIBS setting in enkf_chgres_recenter_nc.fd/makefile - replace COMP value of "ftn" with "$(FC)" to accept overrides for different compilers on different platforms - FC will be set via myFC at upper level Refs: NOAA-EMC#665
- no longer needed, now called libpng Refs: NOAA-EMC#665
- Add "/gempak" subfolder to several HPCUKMET, grid, and grid3 path variables used in gempak ush scripts for ukmet data - Boi Vuong and Wei Wei tested on WCOSS2 to confirm GDAS ukmet data match in terms of size between WCOSS1 and WCOSS2 Refs: NOAA-EMC#399
…oss2 Add needed gempak subfolder to gempak ush scripts
- add obsproc_run_ver and prepobs_run_ver to wcoss2.ver for use when pointing to package installs during prep job Refs: NOAA-EMC#399
- add modulefile load from prepobs install - update JJOB name to JOBSPROC_GLOBAL_PREP and path to adjusted install location variable $HOMEobsproc Refs: NOAA-EMC#399
- change HOMEobsproc_prep to HOMEobsproc - remove HOMEobsproc_network and HOMEobsproc_global; replace with HOMEprepobs - point HOMEobsproc to group account BASE_GIT obsproc install using new obsproc_run_ver variable set in $target.ver files - point HOMEprepobs to group account BASE_GIT prepobs install using new prepobs_run_ver variable set in $target.ver files Refs: NOAA-EMC#399
- remove HOMEobsproc_prep, HOMEobsproc_network, and HOMEobsproc_global - add HOMEobsproc and point it to NCO ops install location Refs: NOAA-EMC#399
…oss2 Update prep job functionality to use new WCOSS2-port obsproc/prepobs packages
Sync merge feature/ops-wcoss2 into dev_v16 (gempak/prep)
- remove now-unneeded symlinks in link_fv3gfs.sh for nemsio_chgdate.fd, nemsio_get.fd, and nemsio_read.fd Refs: NOAA-EMC#665
- only checkout the GFS WAFS package when building the package for operations Refs: NOAA-EMC#665
- change MODULEPATH to /apps/contrib/NCEP/libs/hpc-stack-gfsv16/modulefiles/stack in modulefiles - remove libjpeg, bufr_dump, and util_shared from module_base.orion.lua; won't be available outside of production or aren't needed on Orion Refs: NOAA-EMC#665
- add new obsproc/prepobs version variables - update hpc_ver to 1.2.0 - add override versions for modules that differ on Orion compared to WCOSS2: gempak_ver, python_ver, wrf_io_ver, esmf_ver, nco_ver Refs: NOAA-EMC#665
- change MODULEPATH to /scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack-gfsv16/modulefiles/stack - remove png module load in modulefile.storm_reloc_v6.0.0.hera.lua; now libpng - change python to anaconda module in module_base.hera.lua - remove unavailable or unneeded module loads in module_base.hera.lua: perl, libjpeg, bufr_dump, util_shared - add anaconda_ver=anaconda3-5.3.1 to hera.ver Refs: NOAA-EMC#665
- add new obsproc/prepobs run versions - update hpc_ver to 1.2.0 - add needed overrides for module versions that differ on Hera compared to WCOSS2: cmake_ver, esmf_ver, nco_ver, gempak_ver, wrf_io_ver Refs: NOAA-EMC#665
- update GSI tag to gfsda.v16.2.0.1 - update UPP tag to upp_v8.1.1 Refs: NOAA-EMC#665
- update ufs-weather-model checkout to GFS.v16.2.1 tag - update UFS_UTILS checkout to ops-gfsv16.2.1 tag - add "--init" with GSI submodule command to correct libsrc checkout Refs: NOAA-EMC#665
- set WW3=Y on WCOSS2 and WW3=N on Hera/Orion - will not be supporting waves on Hera/Orion for this version of GFS.v16.2 due to need to update WW3 code - will remove machine check for v16.3 upgrade and return to WW3=Y Refs: NOAA-EMC#665
dev_v16 build and versions updates to support Hera/Orion
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
ecflow PR is for issue #720
In this PR it will contain:
Archive JJobs changes:
ecf/scripts/workflow_manager/cycled/enkfgdas/.gitignore
ecf/scripts/workflow_manager/cycled/enkfgdas/jenkfgdas_emc_earc_master.ecf
ecf/scripts/workflow_manager/cycled/gdas/jgdas_emc_arch.ecf
ecf/scripts/workflow_manager/cycled/gfs/jgfs_emc_arch.ecf
jobs/JENKFGDAS_ARCHIVE
jobs/JGLOBAL_ARCHIVE
Verification JJobs changes:
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Checklist