Optimize DA clone & build

[RussTreadon-NOAA]

Description
This issue is opened to document changes to optimize the checkout and build for DA components. This development is done in g-w branch feature/clone.

Requirements
This issue adds flexibility to the g-w clone and build via the following changes:

• add option to select DA repositories to clone
• modify build so that it only builds cloned DA components.

The default checkout behavior is to not build any DA components.

Acceptance Criteria (Definition of Done)
The work in this issue is complete when

• default behavior is as outlined above
• added options yield the desired behavior
• PR is opened, reviewed, approved, and merged

Dependencies
This issue has no dependencies on other issues or PRs.

[RussTreadon-NOAA]

Create feature/clone as copy of develop at ffcd5bb .

[RussTreadon-NOAA]

Confirm that the default behavior of sorc/checkout.sh is to only clone the GSI repo (ie, populate gsi_enkf.fd for GSI-based DA). By default the modified checkout.sh does not clone the GDASApp repo (JEDI-based DA).

[RussTreadon-NOAA]

Test the following on Orion:

1. checkout.sh with no options (default behavior)
2. checkout.sh -g for GSI-based DA
3. checkout.sh -j for JEDI-based DA

Confirm the following:

1. checkout.sh clones the GSI repo into gsi_enkf.fd for GSI-based DA. The GDASApp repo is not cloned. This is the intended default behavior.
2. checkout.sh -g clones the GSI repo into gsi_enkf.fd for GSI-based DA. The GDASApp repo is not cloned. This is the intended behavior for the -g option of checkout.sh
3. checkout.sh -j clones the GDASApp repo into gdas.cd for JEDI-based DA. The GSI repo is not cloned. This is the intended behavior for the -j- option of checkout.sh

Confirm that build_all.sh correctly builds the apps in the cloned repos:

• For both checkout.sh and checkout.sh -g, GSI-based DA executables gsi.x and enkf.x are built. No JEDI-based DA apps are built because gdas.cd does not exist in the sorc/ directory.
• For checkout.sh -j, JEDI-based DA apps are built. No GSI-based DA apps are built because gsi_enkf.fd does not exist in the sorc/ directory.

[RussTreadon-NOAA]

@WalterKolczynski-NOAA , @aerorahul , @KateFriedman-NOAA - question for the g-w team: What default behavior do we want for the DA sections of checkout.sh?

Line 154 of checkout.sh currently defaults variable checkout_gsi to YES while line 158 defaults variable checkout_jedi to NO.

if [[ "${checkout_gsi:-YES}" == "YES" ]]; then
  checkout "gsi_enkf.fd" "https://github.com/NOAA-EMC/GSI.git" "67f5ab4"; errs=$((errs + $?))
fi

if [[ "${checkout_jedi:-NO}" == "YES" ]]; then
  checkout "gdas.cd" "https://github.com/NOAA-EMC/GDASApp.git" "5952c9d"; errs=$((errs + $?))
fi

Thus, by default, checkout.sh clones the GSI repo and skips the GDASApp repo. We want to keep the GDASApp clone off by default, at least for the time being. What about the GSI clone?

Do we want to change the line 154 default to NO so that by default checkout.sh does NOT clone the GSI repo?

[RussTreadon-NOAA]

Execute the following on Orion in a clone of feature/clone with noted results:

• cd sorc
• ./checkout.sh - gsi, gsi_monitor, gsi_utils, gldas, and GDASApp not cloned
• './build_all.sh` - skip build of gsi & enkf, gsi utilities, GDASApp, gsi monitor, and gldas
• ./link_workflow.sh emc orion - no dead links created

[RussTreadon-NOAA]

Execute the following on Orion in a clone of feature/clone with the following results:

• ./checkout -g - gsi, gsi_utils, gsi_monitor, and gldas cloned
• ./build_all.sh - skip build of GDASApp
• ./link_workflow.sh emc orion - the following dead links found in sorc/gsi_utils.fd

sorc/gsi_utils.fd/src/global_angupdate/kinds.F90
sorc/gsi_utils.fd/src/global_angupdate/read_diag.f90
sorc/gsi_utils.fd/src/Radiance_Utilities/diag2grads/kinds.F90
sorc/gsi_utils.fd/src/Radiance_Utilities/diag2grads/read_diag.f90

The above dead links are actually committed in the GSI-utils repo. An issue should be opened in GSI-utils to remove these dead links in this repo.

Also find sorc/ufs_model.fd/HYCOM-interface/HYCOM/ALT_CODE/README.src.dimensions to be a dead link. This link points at `../../README.src/README.src.dimensions. This file does not exist. An issue should be opened in the repo from which the HYCOM-interface resides.

I think I misunderstood a comment form @aerorahul in a related email thread indicating that we could specify CHECKOUT_GSI=YES or CHECKOUT_GDAS=YES on the command line prior to checkout.sh to toggle between the DA modes.

checkout.sh currently uses the -g option to internally set CHECKOUT_GSI=YES and CHECKOUT_GDAS=NO. The -u option internally sets CHECKOUT_GSI=NO and CHECKOUT_GDAS=YES.

Do we want to allow for the external setting of CHECKOUT_GSI and CHECKOUT_GDAS, thereby bypassing the -g, -u options?

[RussTreadon-NOAA]

Execute the following on Orion in a clone of feature/clone with the following results:

• `./checkout -u - GDASApp, gsi_utils, gsi_monitor, and gldas cloned
• ./build_all.sh - skip build of gsi and enkf
• ./link_workflow.sh emc orion - the same dead links mentioned in the previous comment also found in this test. Additionally dead links found in gdas.cd/build directories for crtm based ctests. I believe these links are populated when executing the first ctest, get_crtm_coeffs. Four dead saber links were also found in gdas.cd/build

sorc/gdas.cd/build/bin/saber_tar_ref.sh
sorc/gdas.cd/build/bin/saber_tar_data.sh
sorc/gdas.cd/build/bin/saber_set_ref.sh
sorc/gdas.cd/build/bin/saber_plot/bump_lct_cor.py

I do not know if source files for these saber links are created when running saber ctests.