Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update global-workflow scripts for runnng the NG-GODAS application #995

Closed
wants to merge 23 commits into from
Closed

Update global-workflow scripts for runnng the NG-GODAS application #995

Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
3bae39c
updated the following files for NG-GODAS application:
binli2337 Jun 6, 2022
f94140d
Add parm/config/config.defaults.datm for the NG-GODAS application.
binli2337 Jun 6, 2022
6a0c233
Merge branch 'NOAA-EMC:develop' into add_NG_GODAS
binli2337 Jun 6, 2022
188e6ed
Update parm/config/config.coupled_ic and sorc/link_workflow.sh.
binli2337 Jun 6, 2022
923a3d6
Merge branch 'add_NG_GODAS' of https://github.com/binli2337/global-wo…
binli2337 Jun 6, 2022
f2acb0d
Update parm/config/config.coupled_ic and ush/rocoto/setup_workflow_fc…
binli2337 Jun 7, 2022
ebd4797
Merge branch 'NOAA-EMC:develop' into add_NG_GODAS
binli2337 Jun 7, 2022
872a6cf
Update the following four files:
binli2337 Jun 9, 2022
9bdd721
Add ATMTYPE and remove DO_ATM.
binli2337 Jun 10, 2022
ae635eb
Merge branch 'NOAA-EMC:develop' into add_NG_GODAS
binli2337 Jun 10, 2022
6cec5ce
Update setup_workflow_fcstonly.py to use atm_type.
binli2337 Jun 13, 2022
a673e3d
Merge branch 'develop' into add_NG_GODAS
binli2337 Jun 13, 2022
74cbb9e
* update parm/config/config.base.emc.dyn
binli2337 Jun 15, 2022
58cdd27
Update setup_workflow_fcstonly.py.
binli2337 Jun 15, 2022
3046d16
Merge branch 'NOAA-EMC:develop' into add_NG_GODAS
binli2337 Jun 15, 2022
ed0ebb3
add a gefs test
binli2337 Jun 26, 2022
2217ed7
update config.resources
binli2337 Jun 27, 2022
a58d837
Merge branch 'develop' into add_gefs
binli2337 Aug 23, 2022
fa0f72b
Merge branch 'NOAA-EMC:develop' into update_gefs
binli2337 Aug 23, 2022
98aba56
Update workflow_tasks.py
binli2337 Aug 23, 2022
ffe8e4e
Merge branch 'NOAA-EMC:develop' into update_gefs
binli2337 Aug 24, 2022
9c9d8a7
Merge branch 'NOAA-EMC:develop' into update_gefs
binli2337 Aug 24, 2022
41adf6b
Update arch.sh, config.base.emc.dyn, and setup_expt.py
binli2337 Aug 30, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 25 additions & 13 deletions jobs/rocoto/arch.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,17 +59,23 @@ PDY_MOS=$(echo $CDATE_MOS | cut -c1-8)
# Archive online for verification and diagnostics
###############################################################

COMIN=${COMINatmos:-"$ROTDIR/$CDUMP.$PDY/$cyc/atmos"}
if [ $ATMTYPE = "MODEL" ]; then
COMIN=${COMINatmos:-"$ROTDIR/$CDUMP.$PDY/$cyc/atmos"}
else
COMIN=${COMINdatm:-"$ROTDIR/$CDUMP.$PDY/$cyc/datm"}
fi
binli2337 marked this conversation as resolved.
Show resolved Hide resolved
cd $COMIN

source "${HOMEgfs}/ush/file_utils.sh"

[[ ! -d $ARCDIR ]] && mkdir -p $ARCDIR
nb_copy ${APREFIX}gsistat $ARCDIR/gsistat.${CDUMP}.${CDATE}
nb_copy ${APREFIX}pgrb2.1p00.anl $ARCDIR/pgbanl.${CDUMP}.${CDATE}.grib2
if [ $ATMTYPE = "MODEL" ]; then
[[ ! -d $ARCDIR ]] && mkdir -p $ARCDIR
nb_copy ${APREFIX}gsistat $ARCDIR/gsistat.${CDUMP}.${CDATE}
nb_copy ${APREFIX}pgrb2.1p00.anl $ARCDIR/pgbanl.${CDUMP}.${CDATE}.grib2
fi

# Archive 1 degree forecast GRIB2 files for verification
if [ $CDUMP = "gfs" ]; then
if [ $CDUMP = "gfs" -a $ATMTYPE = "MODEL" ]; then
fhmax=$FHMAX_GFS
fhr=0
while [ $fhr -le $fhmax ]; do
Expand All @@ -79,7 +85,7 @@ if [ $CDUMP = "gfs" ]; then
fhr=$((10#$fhr + 10#$FHOUT_GFS ))
done
fi
if [ $CDUMP = "gdas" ]; then
if [ $CDUMP = "gdas" -a $ATMTYPE = "MODEL" ]; then
flist="000 003 006 009"
for fhr in $flist; do
fname=${APREFIX}pgrb2.1p00.f${fhr}
Expand All @@ -100,7 +106,7 @@ if [ $CDUMP = "gdas" -a -s gdas.t${cyc}z.cyclone.trackatcfunix ]; then
cat gdasp.t${cyc}z.cyclone.trackatcfunix | sed s:AVNO:${PLSOT4}:g > ${ARCDIR}/atcfunixp.${CDUMP}.$CDATE
binli2337 marked this conversation as resolved.
Show resolved Hide resolved
fi

if [ $CDUMP = "gfs" ]; then
if [ $CDUMP = "gfs" -a $ATMTYPE = "MODEL" ]; then
nb_copy storms.gfso.atcf_gen.$CDATE ${ARCDIR}/.
nb_copy storms.gfso.atcf_gen.altg.$CDATE ${ARCDIR}/.
nb_copy trak.gfso.atcfunix.$CDATE ${ARCDIR}/.
Expand Down Expand Up @@ -188,13 +194,19 @@ fi
cd $ROTDIR

if [ $CDUMP = "gfs" ]; then
if [ $ATMTYPE = "MODEL" ]; then
binli2337 marked this conversation as resolved.
Show resolved Hide resolved
targrp_list="gfsa gfsb"
else
targrp_list=""
fi

targrp_list="gfsa gfsb"
if [ $ATMTYPE = "MODEL" ]; then

if [ ${ARCH_GAUSSIAN:-"NO"} = "YES" ]; then
targrp_list="$targrp_list gfs_flux gfs_${format}b gfs_pgrb2b"
if [ $MODE = "cycled" ]; then
targrp_list="$targrp_list gfs_${format}a"
if [ ${ARCH_GAUSSIAN:-"NO"} = "YES" ]; then
targrp_list="$targrp_list gfs_flux gfs_${format}b gfs_pgrb2b"
if [ $MODE = "cycled" ]; then
targrp_list="$targrp_list gfs_${format}a"
fi
fi
fi

Expand Down Expand Up @@ -376,7 +388,7 @@ done
# gaussian files to prevent the files from being removed by automatic
# scrubber present on some machines.

if [ $CDUMP = "gfs" ]; then
if [ $CDUMP = "gfs" -a $ATMTYPE = "MODEL" ]; then
fhmax=$((FHMAX_FITS+36))
RDATE=$($NDATE -$fhmax $CDATE)
rPDY=$(echo $RDATE | cut -c1-8)
Expand Down
43 changes: 28 additions & 15 deletions jobs/rocoto/coupled_ic.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,25 +38,36 @@ status=$?

# Create ICSDIR if needed
[[ ! -d $ICSDIR/$CDATE ]] && mkdir -p $ICSDIR/$CDATE
[[ ! -d $ICSDIR/$CDATE/atmos ]] && mkdir -p $ICSDIR/$CDATE/atmos
if [ $ATMTYPE = "MODEL" ]; then
[[ ! -d $ICSDIR/$CDATE/atmos ]] && mkdir -p $ICSDIR/$CDATE/atmos
fi
if [ $ATMTYPE = "DATA" ]; then
[[ ! -d $ICSDIR/$CDATE/datm ]] && mkdir -p $ICSDIR/$CDATE/datm
fi
[[ ! -d $ICSDIR/$CDATE/ocn ]] && mkdir -p $ICSDIR/$CDATE/ocn
[[ ! -d $ICSDIR/$CDATE/ice ]] && mkdir -p $ICSDIR/$CDATE/ice

if [ $ICERES = '025' ]; then
ICERESdec="0.25"
fi
if [ $ICERES = '050' ]; then
ICERESdec="0.50"
ICERESdec="0.50"
fi

# Setup ATM initial condition files
cp -r $BASE_CPLIC/$CPL_ATMIC/$CDATE/$CDUMP/* $ICSDIR/$CDATE/atmos/
rc=$?
if [[ $rc -ne 0 ]] ; then
echo "FATAL: Unable to copy $BASE_CPLIC/$CPL_ATMIC/$CDATE/$CDUMP/* to $ICSDIR/$CDATE/atmos/ (Error code $rc)"
if [ $ATMTYPE = "MODEL" ]; then
# Setup ATM initial condition files
cp -r $BASE_CPLIC/$CPL_ATMIC/$CDATE/$CDUMP/* $ICSDIR/$CDATE/atmos/
rc=$?
if [[ $rc -ne 0 ]] ; then
echo "FATAL: Unable to copy $BASE_CPLIC/$CPL_ATMIC/$CDATE/$CDUMP/* to $ICSDIR/$CDATE/atmos/ (Error code $rc)"
fi
((err+=$rc))
fi
err=$((err + rc))

if [ $ATMTYPE = "DATA" ]; then
# Setup DATM forcing files
cp -r $BASE_CPLIC/${CPL_DATM}/${datm_src}/$CDATE/${datm_src}.*.nc $ICSDIR/$CDATE/datm/
fi

# Setup Ocean IC files
cp -r $BASE_CPLIC/$CPL_OCNIC/$CDATE/ocn/$OCNRES/MOM*.nc $ICSDIR/$CDATE/ocn/
Expand Down Expand Up @@ -87,13 +98,15 @@ if [ $DO_WAVE = "YES" ]; then
done
fi

# Stage the FV3 initial conditions to ROTDIR
export OUTDIR="$ICSDIR/$CDATE/atmos/$CASE/INPUT"
COMOUT="$ROTDIR/$CDUMP.$PDY/$cyc/atmos"
[[ ! -d $COMOUT ]] && mkdir -p $COMOUT
cd $COMOUT || exit 99
rm -rf INPUT
$NLN $OUTDIR .
if [ $ATMTYPE = "MODEL" ]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move this block up with the other model atmosphere one so they can be combined in a single block.

# Stage the FV3 initial conditions to ROTDIR
export OUTDIR="$ICSDIR/$CDATE/atmos/$CASE/INPUT"
COMOUT="$ROTDIR/$CDUMP.$PDY/$cyc/atmos"
[[ ! -d $COMOUT ]] && mkdir -p $COMOUT
cd $COMOUT || exit 99
rm -rf INPUT
$NLN $OUTDIR .
fi

#Stage the WW3 initial conditions to ROTDIR
if [ $DO_WAVE = "YES" ]; then
Expand Down
8 changes: 7 additions & 1 deletion parm/config/config.arch
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,13 @@ export ARCH_GAUSSIAN_FHMAX=${ARCH_GAUSSIAN_FHMAX:-36}
export ARCH_GAUSSIAN_FHINC=${ARCH_GAUSSIAN_FHINC:-6}

#--online archive of nemsio files for fit2obs verification
export FITSARC="YES"

if [[ $ATMTYPE = "MODEL" ]]; then
export FITSARC="YES"
else
export FITSARC="NO"
fi

export FHMAX_FITS=132
[[ "$FHMAX_FITS" -gt "$FHMAX_GFS" ]] && export FHMAX_FITS=$FHMAX_GFS

Expand Down
25 changes: 20 additions & 5 deletions parm/config/config.base.emc.dyn
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ export QUEUE_SERVICE="@QUEUE_SERVICE@"
export PARTITION_BATCH="@PARTITION_BATCH@"

# Project to use in mass store:
HPSS_PROJECT=emc-global
binli2337 marked this conversation as resolved.
Show resolved Hide resolved
HPSS_PROJECT=emc-marine
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't change this default.


# Directories relative to installation areas:
export HOMEgfs=@HOMEgfs@
Expand All @@ -31,6 +31,7 @@ export SCRgfs=$HOMEgfs/scripts

export FIXcice=$HOMEgfs/fix/fix_cice
export FIXmom=$HOMEgfs/fix/fix_mom6
export FIXdatm=$HOMEgfs/fix/fix_datm
export FIXreg2grb2=$HOMEgfs/fix/fix_reg2grb2

########################################################################
Expand Down Expand Up @@ -85,7 +86,8 @@ export NCP="/bin/cp -p"
export NMV="/bin/mv"
export NLN="/bin/ln -sf"
export VERBOSE="YES"
export KEEPDATA="NO"
export KEEPDATA="YES"
binli2337 marked this conversation as resolved.
Show resolved Hide resolved

export CHGRP_RSTPROD="@CHGRP_RSTPROD@"
export CHGRP_CMD="@CHGRP_CMD@"
export NEMSIOGET="$HOMEgfs/exec/nemsio_get"
Expand Down Expand Up @@ -132,6 +134,8 @@ export COMINice=${ROTDIR}/${CDUMP}.${PDY}/${cyc}/ice
export COMOUTice=${ROTDIR}/${CDUMP}.${PDY}/${cyc}/ice
export COMINaero=${ROTDIR}/${CDUMP}.${PDY}/${cyc}/chem
export COMOUTaero=${ROTDIR}/${CDUMP}.${PDY}/${cyc}/chem
export COMINdatm=${ROTDIR}/${CDUMP}.${PDY}/${cyc}/datm
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of using a different directory, can't we just use atmos to hold a data atmosphere? I don't see the need for the variable proliferation.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my understanding, this datm directory will contain restart files in the future. The NG-GODAS will be run 6 hours before the coupled model is run.

export COMOUTdatm=${ROTDIR}/${CDUMP}.${PDY}/${cyc}/datm

export jlogfile="${EXPDIR}/logs/jlogfile"
export ERRSCRIPT=${ERRSCRIPT:-'eval [[ $err = 0 ]]'}
Expand Down Expand Up @@ -163,6 +167,12 @@ export WAVE_CDUMP="" # When to include wave suite: gdas, gfs, or both
export DOBNDPNT_WAVE="NO"
export cplwav2atm=".false."

export ATMTYPE="MODEL"
export DO_POST="YES"

# Verification options
export DO_METP="YES" # Run METPLUS jobs - set METPLUS settings in config.metp

case "${APP}" in
ATM)
export confignamevarfornems="atm"
Expand All @@ -181,6 +191,14 @@ case "${APP}" in
export DO_ATM="NO"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we still using this setting, or has it been replace by checking against ATMTYPE? I think we only need one of them.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DO_ATM is used in workflow/applications.py.

Copy link
Contributor

@WalterKolczynski-NOAA WalterKolczynski-NOAA Aug 30, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then it should probably be changed. There should only be one setting for DATM, either DO_ATM or ATMTYPE.

export DO_OCN="YES"
export DO_ICE="YES"
export REALTIME="NO"
export DO_COUPLED="YES"
export ATMTYPE="DATA"
export datm_src="gefs"
export DO_POST="NO"
export DO_WAVE="NO"
export DO_METP="YES"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You set DO_METP to YES here, then immediately change it to NO in defaults.datm. It should probably be just here for consistency with the other APPs. Same for DO_VRFY.

source $EXPDIR/config.defaults.datm
;;
S2S*)
export DO_COUPLED="YES"
Expand Down Expand Up @@ -373,9 +391,6 @@ export INCVARS_EFOLD="5"
export netcdf_diag=".true."
export binary_diag=".false."

# Verification options
export DO_METP="YES" # Run METPLUS jobs - set METPLUS settings in config.metp

# Archiving options
export HPSSARCH="@HPSSARCH@" # save data to HPSS archive
export LOCALARCH="@LOCALARCH@" # save data to local archive
Expand Down
30 changes: 30 additions & 0 deletions parm/config/config.defaults.datm
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
#!/bin/bash

# Empty variables must include a space otherwise they will be overwritten

FHMAX_GFS_00=24
FHMAX_GFS_06=24
FHMAX_GFS_12=24
FHMAX_GFS_18=24
FHMAX_GFS=24
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You shouldn't need to set this. FHMAX_GFS is set to one of the four previous settings based on the cycle.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For ocean DA, FHMAX_GFS should probably be 6. Here it is set to 24 for testing. FHMAX_GFS_00, FHMAX_GFS_06, FHMAX_GFS_12, and FHMAX_GFS_18 are also set in config.defaults.s2sw for the S2SW case.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are missing my point. FHMAX_GFS is set to one of FHMAX_GFS_00, FHMAX_GFS_06, etc. based on the cycle. It does not need to be set separately here as long as this is called before that part of config.base.

FHOUT_GFS=6
NFHOUT=6
NFHOUT_HF=3
OCNTIM=3600
DELTIM=900
ICETIM=$DELTIM
CPL_SLOW=$OCNTIM
ktherm=1
tfrz_option='linear_salt'
MOM6_USE_LI2016='False'
export DO_METP="NO"
export DO_VRFY="NO"
RUN="data"
flux_convergence=0.0
flux_iteration=2
flux_scheme=0
# gefs forcing:
atm_datamode=${atm_datamode:-"GEFS"}
ATM_NX_GLB=${ATM_NX_GLB:-"1536"}
ATM_NY_GLB=${ATM_NY_GLB:-"768"}
confignamevarfornems="datm_cdeps"
3 changes: 3 additions & 0 deletions parm/config/config.fcst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,9 @@ fi
if [ $DO_OCN = "YES" ]; then
export cplflx=".true."
fi
if [ $ATMTYPE = "DATA" ]; then
export cpldatm=".true."
fi
if [ $DO_WAVE = "YES" ]; then
export cplwav=".true."
fi
Expand Down
Loading