Generalize getges_driver COM lookback #19
Conversation
The upcoming update to GFS v17 will substantially change the COM paths
for the GFS package. This requires updates to the lookback capability of
getges_driver where it checks previous cycles. Rather than add additional
logical branches to handle new COM patterns, the process is generalized by
instead doing a string substitution to replace the current day and cycle
with those for the prior cycle. This method is backwards compatible and
should work for any path as long as it contains `${PDY}/${cyc}`.
Additional changes will likely be needed for GFSv17 to run prepobs stand-
alone (as in ops) to match the new `$COMINgdas` and `$COMINgfs`, but
these changes work when running prepobs as part of global workflow in
development.
Also, I have not otherwise modified the existing style of the scripts,
redefining `$COMINgdas` and `$COMINgfs`. However, this will need to be
changed before GFSv17 to stop overwriting these variables to meet NCO
requirements (see NOAA-EMC/global-workflow#293).
Refs: NOAA-EMC#17
Refs: NOAA-EMC/global-workflow#761
There was a problem hiding this comment.
Looks good and I like that it's backwards compatible. Can likely get this update into whatever comes next for prepobs in ops. Need to test some more and discuss with @ilianagenkova and @ShelleyMelchior-NOAA but this works for current GFSv17 COM reorg development.
|
@KateFriedman-NOAA @WalterKolczynski-NOAA We will test prepobs's feature/GFSv17_com_reorg and let you know how it goes. |
COMINgdas_orig isn't a substitute for COMINgdas_root, it is something different. Previously, the script had to know the full structure of the GFS output so it could back up the tree to COMIN*_root, then extend it again with new dates. Now, it just looks for the date and cycle in the path and substitutes them without altering the rest of the path; it doesn't care how many additional directory levels are below the substitution point. Also, as noted in the PR, the *_orig variables are only used because the originals are currently being overwritten, but my understanding is that NCO does not like that and a different variable should be used when changing the cycle. At that point, the *_orig variables won't be needed anymore. |
|
@WalterKolczynski-NOAA , thanks for the explanation! |
|
Sorry, @WalterKolczynski-NOAA , where do you define COMINgdas and COMINgfs and what do they point to, b/f using them in: |
It should be the full path to the directory that holds the desired files. For GFS v16, that would be: For GFS v17, the provisional plan is: See the global-workflow PR noted in this PR for more information about the new GFS COM structure and how we have centralized the definition for all these paths for GFS v17 (the files prepobs look for are in the path defined by |
Add Jet support to GFSv17 COM reorg branch
|
(2) Won't work with any existing data, as that is only the planned path for v17. To test with existing data; use |
The upcoming update to GFS v17 will substantially change the COM paths for the GFS package. This requires updates to the lookback capability of getges_driver where it checks previous cycles. Rather than add additional logical branches to handle new COM patterns, the process is generalized by instead doing a string substitution to replace the current day and cycle with those for the prior cycle. This method is backwards compatible and should work for any path as long as it contains
${PDY}/${cyc}.Additional changes will likely be needed for GFSv17 to run prepobs stand- alone (as in ops) to match the new
$COMINgdasand$COMINgfs, but these changes work when running prepobs as part of global workflow in development.Also, I have not otherwise modified the existing style of the scripts, redefining
$COMINgdasand$COMINgfs. However, this will need to be changed before GFSv17 to stop overwriting these variables to meet NCO requirements (see NOAA-EMC/global-workflow#293).Refs: #17
Refs: NOAA-EMC/global-workflow#761