Skip to content

Update marine DA j-jobs to new format #1149

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Dec 12, 2022
Merged

Update marine DA j-jobs to new format #1149

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b67ec89
make ocean analysis j-jobs consistent
aerorahul Dec 6, 2022
c2fb077
fix SC2155
aerorahul Dec 6, 2022
72f2848
fix SC2086 in the canonical form
aerorahul Dec 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 26 additions & 3 deletions jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_POST
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,37 @@
export STRICT="NO"
source "${HOMEgfs}/ush/preamble.sh"

##############################################
# make temp directory
##############################################
export DATA=${DATA:-${DATAROOT}/ocnanal_${cyc}}
mkdir -p "${DATA}"
cd "${DATA}" || (echo "${DATA} does not exist. ABORT!"; exit 1)


##############################################
# Run setpdy and initialize PDY variables
##############################################
export cycle="t${cyc}z"
setpdy.sh
. ./PDY


##############################################
# Determine Job Output Name on System
##############################################
export pid=${pid:-$$}
export pgmout="OUTPUT.${pid}"
export pgmerr=errfile


#############################
# Source relevant config files
#############################
export EXPDIR=${EXPDIR:-${HOMEgfs}/parm/config}
configs="base ocnanalpost"
config_path=${EXPDIR:-${PACKAGEROOT}/gfs.${gfs_ver}/parm/config}
for config in ${configs}; do
. "${config_path}"/config."${config}"
. "${EXPDIR}/config.${config}"
status=$?
[[ "${status}" -ne 0 ]] && exit "${status}"
done
Expand All @@ -18,6 +41,6 @@ done
# Remove the Temporary working directory
##########################################
cd "${DATAROOT}" || exit 1
[[ ${KEEPDATA} = "NO" ]] && rm -rf "${DATA}"
[[ "${KEEPDATA}" = "NO" ]] && rm -rf "${DATA}"

exit 0
56 changes: 27 additions & 29 deletions jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,34 +2,11 @@
export STRICT="NO"
source "${HOMEgfs}/ush/preamble.sh"

#############################
# Source relevant config files
#############################
export EXPDIR=${EXPDIR:-${HOMEgfs}/parm/config}
configs="base ocnanal ocnanalprep"
config_path=${EXPDIR:-${PACKAGEROOT}/gfs.${gfs_ver}/parm/config}
for config in ${configs}; do
. "${config_path}"/config."${config}"
status=$?
[[ ${status} -ne 0 ]] && exit "${status}"
done


##########################################
# Source machine runtime environment
##########################################
. "${HOMEgfs}"/env/"${machine}".env ocnanalprep
status=$?
[[ ${status} -ne 0 ]] && exit "${status}"


##############################################
# Obtain unique process id (pid) and make temp directory
# make temp directory
##############################################
export pid=${pid:-$$}
export outid=${outid:-"LL${job}"}

export DATA=${DATA:-${DATAROOT}/ocnanal_${CDATE}}
export DATA=${DATA:-${DATAROOT}/ocnanal_${cyc}}
Copy link
Contributor Author

@aerorahul aerorahul Dec 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guillaumevernieres @CoryMartin-NOAA
Take note here in the use of ${cyc} instead of ${CDATE}.

At this stage, we only have $cyc in ops coming from ecFlow.
In our development we provide $PDY and $CDATE via the Rocoto XML.

Please take special note of this.

Copy link
Contributor

@CoryMartin-NOAA CoryMartin-NOAA Dec 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aerorahul where in ops does PDY come in then? the ex-script? I thought the setpdy was run in the j-job?

Copy link
Contributor Author

@aerorahul aerorahul Dec 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

setpdy.sh sets PDY based on a file in the ops space.
If PDY is previously set (as is the case in development), setpdy.sh uses that info (with the cycle).

Copy link
Contributor

@CoryMartin-NOAA CoryMartin-NOAA Dec 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah so I guess my point is that we can assume we have ${PDY}${cyc} at this point, right? Not that it really matters for this ${DATA} directory.

Copy link
Contributor Author

@aerorahul aerorahul Dec 12, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In ops PDY is set by the setpdy.sh script.
setpdy.sh needs to be executed while in the DATA directory as it creates a (pesky) file ncepdate.

As you can see, in this (and other) j-job, DATA directory name contains PDY which is undefined in ops at this point in the script.

rm -rf "${DATA}" # Ensure starting with a clean DATA
mkdir -p "${DATA}"
cd "${DATA}" || (echo "${DATA} does not exist. ABORT!"; exit 1)

Expand All @@ -45,24 +22,45 @@ setpdy.sh
##############################################
# Determine Job Output Name on System
##############################################
export pid=${pid:-$$}
export pgmout="OUTPUT.${pid}"
export pgmerr=errfile


#############################
# Source relevant config files
#############################
export EXPDIR=${EXPDIR:-${HOMEgfs}/parm/config}
configs="base ocnanal ocnanalprep"
for config in ${configs}; do
. "${EXPDIR}/config.${config}"
status=$?
[[ ${status} -ne 0 ]] && exit "${status}"
done


##########################################
# Source machine runtime environment
##########################################
. "${HOMEgfs}/env/${machine}.env ocnanalprep"
status=$?
[[ ${status} -ne 0 ]] && exit "${status}"


##############################################
# Set variables used in the script
##############################################
export CDATE=${CDATE:-${PDY}${cyc}}
export CDUMP=${CDUMP:-${RUN:-"gfs"}}
export COMPONENT=${COMPONENT:-ocean}
export COMPONENT="ocean"

##############################################
# Begin JOB SPECIFIC work
##############################################

GDATE=$(date +%Y%m%d%H -d "${CDATE:0:8} ${CDATE:8:2} - ${assim_freq} hours")
GDATE=$(date +%Y%m%d%H -d "${PDY} ${cyc} - ${assim_freq} hours")
export GDATE
gPDY=${GDATE:0:8}
export gPDY=${GDATE:0:8}
export gcyc=${GDATE:8:2}
export GDUMP=${GDUMP:-"gdas"}

Expand Down
53 changes: 25 additions & 28 deletions jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,34 +2,11 @@
export STRICT="NO"
source "${HOMEgfs}/ush/preamble.sh"

#############################
# Source relevant config files
#############################
export EXPDIR=${EXPDIR:-${HOMEgfs}/parm/config}
configs="base ocnanal ocnanalrun"
config_path=${EXPDIR:-${PACKAGEROOT}/gfs.${gfs_ver}/parm/config}
for config in ${configs}; do
. "${config_path}"/config."${config}"
status=$?
[[ ${status} -ne 0 ]] && exit "${status}"
done


##########################################
# Source machine runtime environment
##########################################
. "${HOMEgfs}"/env/"${machine}".env ocnanalrun
status=$?
[[ ${status} -ne 0 ]] && exit "${status}"


##############################################
# Obtain unique process id (pid) and make temp directory
# make temp directory
##############################################
export pid=${pid:-$$}
export outid=${outid:-"LL${job}"}

export DATA=${DATA:-${DATAROOT}/ocnanal_${CDATE}}
export DATA=${DATA:-${DATAROOT}/ocnanal_${cyc}}
mkdir -p "${DATA}"
cd "${DATA}" || (echo "${DATA} does not exist. ABORT!"; exit 1)

Expand All @@ -45,16 +22,37 @@ setpdy.sh
##############################################
# Determine Job Output Name on System
##############################################
export pid=${pid:-$$}
export pgmout="OUTPUT.${pid}"
export pgmerr=errfile


#############################
# Source relevant config files
#############################
export EXPDIR=${EXPDIR:-${HOMEgfs}/parm/config}
configs="base ocnanal ocnanalrun"
for config in ${configs}; do
. "${EXPDIR}/config.${config}"
status=$?
[[ ${status} -ne 0 ]] && exit "${status}"
done


##########################################
# Source machine runtime environment
##########################################
. "${HOMEgfs}/env/${machine}.env ocnanalrun"
status=$?
[[ ${status} -ne 0 ]] && exit "${status}"


##############################################
# Set variables used in the script
##############################################

export CDUMP=${CDUMP:-${RUN:-"gfs"}}
export COMPONENT=${COMPONENT:-ocean}
export COMPONENT="ocean"

##############################################
# Begin JOB SPECIFIC work
Expand Down Expand Up @@ -82,9 +80,8 @@ if [[ -e "${pgmout}" ]] ; then
fi

##########################################
# Remove the Temporary working directory
# Do not remove the Temporary working directory (do this in POST)
##########################################
cd "${DATAROOT}" || exit 1
[[ ${KEEPDATA} = "NO" ]] && rm -rf "${DATA}"

exit 0