Skip to content

Conversation

RussTreadon-NOAA
Copy link
Contributor

@RussTreadon-NOAA RussTreadon-NOAA commented Aug 14, 2024

Description

This PR adds JEDI ATM lgetkf observer and solver jobs to global-workflow. This is approach is akin GSI-based eobs and eupd. Splitting the single JEDI ATM lgetkf job into separate observer and solver jobs improves memory and computational efficiency.

Resolves #2415

Dependency: GDASApp PR #1252

DO NOT MERGE THIS PR into global-workflow deveop until

  • GDASApp PR #1252 is merged into GDASApp develop
  • the gdas.cd hash in this PR is updated
  • JEDI format ATMS radiance bias correction files are staged in ICSDIR_ROOT for C96C48_ufs_hybatmDA CI (g-w issue #2836)

Type of change

  • New feature (adds functionality)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? YES

How has this been tested?

The following tests have been conducted

  • run GDASApp ctests inside g-w. All tests Passed
  • run g-w CI for C96C48_ufs_hybatmDA, C48mx500_3DVarAOWCDA, and C96_atmaerosnowDA on WCOSS2
  • run cycled UFSDA-based 3dEnVar test over the period 2024050212 to 2024050300

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code

RussTreadon-NOAA and others added 30 commits March 20, 2024 18:28
@emcbot emcbot added CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress and removed CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Aug 27, 2024
@emcbot emcbot added CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Aug 27, 2024
@emcbot
Copy link

emcbot commented Aug 27, 2024

CI Passed on Hera in Build# 1
Built and ran in directory /scratch1/NCEPDEV/global/CI/2833


Experiment C48mx500_3DVarAOWCDA_067331be Completed 2 Cycles: *SUCCESS* at Tue Aug 27 17:43:14 UTC 2024
Experiment C48_ATM_067331be Completed 1 Cycles: *SUCCESS* at Tue Aug 27 17:43:14 UTC 2024
Experiment C48_S2SWA_gefs_067331be Completed 1 Cycles: *SUCCESS* at Tue Aug 27 19:14:52 UTC 2024
Experiment C48_S2SW_067331be Completed 1 Cycles: *SUCCESS* at Tue Aug 27 20:16:26 UTC 2024
Experiment C96_atm3DVar_067331be Completed 3 Cycles: *SUCCESS* at Tue Aug 27 20:22:32 UTC 2024
Experiment C96C48_hybatmDA_067331be Completed 3 Cycles: *SUCCESS* at Tue Aug 27 20:22:33 UTC 2024
Experiment C96C48_ufs_hybatmDA_067331be Completed 2 Cycles: *SUCCESS* at Tue Aug 27 20:22:34 UTC 2024
Experiment C96C48_hybatmaerosnowDA_067331be Completed 3 Cycles: *SUCCESS* at Tue Aug 27 20:34:50 UTC 2024

@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules CI-Wcoss2-Ready **CM use only** PR is ready for CI testing on WCOSS labels Aug 27, 2024
@emcbot emcbot added CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS and removed CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules CI-Wcoss2-Ready **CM use only** PR is ready for CI testing on WCOSS labels Aug 27, 2024
@emcbot
Copy link

emcbot commented Aug 27, 2024

CI Update on Wcoss2 at 08/27/24 08:45:11 PM
============================================
Cloning and Building global-workflow PR: 2833
with PID: 216598 on host: clogin03

@emcbot emcbot added CI-Hercules-Running **Bot use only** CI testing on Hercules for this PR is in-progress CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress and removed CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS labels Aug 27, 2024
@emcbot
Copy link

emcbot commented Aug 27, 2024

Automated global-workflow Testing Results:

Machine: Wcoss2
Start: Tue Aug 27 20:55:28 UTC 2024 on clogin03
---------------------------------------------------
Build: Completed at 08/27/24 09:41:01 PM
Case setup: Completed for experiment C48_ATM_067331be
Case setup: Skipped for experiment C48mx500_3DVarAOWCDA_067331be
Case setup: Skipped for experiment C48_S2SWA_gefs_067331be
Case setup: Completed for experiment C48_S2SW_067331be
Case setup: Completed for experiment C96_atm3DVar_extended_067331be
Case setup: Skipped for experiment C96_atm3DVar_067331be
Case setup: Completed for experiment C96C48_hybatmaerosnowDA_067331be
Case setup: Completed for experiment C96C48_hybatmDA_067331be
Case setup: Completed for experiment C96C48_ufs_hybatmDA_067331be

@emcbot emcbot added CI-Hercules-Passed **Bot use only** CI testing on Hercules for this PR has completed successfully and removed CI-Hercules-Running **Bot use only** CI testing on Hercules for this PR is in-progress labels Aug 28, 2024
@emcbot
Copy link

emcbot commented Aug 28, 2024

CI Passed on Hercules in Build# 2
Built and ran in directory /work2/noaa/stmp/CI/HERCULES/2833


Experiment C48_ATM_067331be Completed 1 Cycles: *SUCCESS* at Tue Aug 27 18:28:17 CDT 2024
Experiment C96C48_hybatmDA_067331be Completed 3 Cycles: *SUCCESS* at Tue Aug 27 19:29:07 CDT 2024
Experiment C96_atm3DVar_067331be Completed 3 Cycles: *SUCCESS* at Tue Aug 27 19:34:53 CDT 2024
Experiment C48_S2SWA_gefs_067331be Completed 1 Cycles: *SUCCESS* at Tue Aug 27 20:36:09 CDT 2024
Experiment C48_S2SW_067331be Completed 1 Cycles: *SUCCESS* at Tue Aug 27 20:41:31 CDT 2024

@emcbot emcbot added CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully and removed CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress labels Aug 28, 2024
@emcbot
Copy link

emcbot commented Aug 28, 2024

All CI Test Cases Passed on Wcoss2:

Experiment C48_ATM_067331be *** SUCCESS *** at 08/27/24 11:07:25 PM
Experiment C48_S2SW_067331be *** SUCCESS *** at 08/27/24 11:14:20 PM
Experiment C96C48_hybatmDA_067331be *** SUCCESS *** at 08/27/24 11:56:34 PM
Experiment C96C48_hybatmaerosnowDA_067331be *** SUCCESS *** at 08/28/24 12:42:26 AM
Experiment C96C48_ufs_hybatmDA_067331be *** SUCCESS *** at 08/28/24 01:42:21 AM
Experiment C96_atm3DVar_extended_067331be *** SUCCESS *** at 08/28/24 09:14:34 AM

@WalterKolczynski-NOAA WalterKolczynski-NOAA merged commit 9ad7d3e into NOAA-EMC:develop Aug 28, 2024
10 of 11 checks passed
@RussTreadon-NOAA
Copy link
Contributor Author

Thank you @WalterKolczynski-NOAA !

@RussTreadon-NOAA RussTreadon-NOAA deleted the feature/ensda_obs branch August 28, 2024 16:24
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this pull request Aug 30, 2024
* origin/develop:
  Add 3 and 9 hr increment files to IC staging (NOAA-EMC#2876)
  Add diffusion/diag B for aerosol DA and some other needed changes (NOAA-EMC#2738)
  Correct ocean `MOM.res_#` stage copy (NOAA-EMC#2868)
  Support coupling on AWS (NOAA-EMC#2859)
  Add JEDI ATM lgetkf observer and solver jobs (NOAA-EMC#2833)
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this pull request Sep 9, 2024
* origin/develop:
  Create JEDI class (NOAA-EMC#2805)
  Restructure the bufr sounding job    (NOAA-EMC#2853)
  Add an archive task to GEFS system to archive files locally (NOAA-EMC#2816)
  Reenable Orion Cycling Support (NOAA-EMC#2877)
  Eliminate race conditions and remove DATAROOT last in cleanup (NOAA-EMC#2893)
  Update aerosol climatology to 2013-2024 mean (NOAA-EMC#2888)
  Add ability to run CI test C96_atm3DVar.yaml to Gaea-C5 (NOAA-EMC#2885)
  Support global-workflow GEFS C48 on Google Cloud (NOAA-EMC#2861)
  Add 3 and 9 hr increment files to IC staging (NOAA-EMC#2876)
  Add diffusion/diag B for aerosol DA and some other needed changes (NOAA-EMC#2738)
  Correct ocean `MOM.res_#` stage copy (NOAA-EMC#2868)
  Support coupling on AWS (NOAA-EMC#2859)
  Add JEDI ATM lgetkf observer and solver jobs (NOAA-EMC#2833)
  Fix gdas build on Gaea and add Gaea to available CI list (NOAA-EMC#2857)
  Support ATM forecast only on Google (NOAA-EMC#2832)
  Add GEFS C48 support on AWS (NOAA-EMC#2818)
  Update omega calculation (NOAA-EMC#2751)
  Add snow DA update and recentering for the EnKF forecasts (NOAA-EMC#2690)
  support ATM forecast only on Azure (NOAA-EMC#2827)
  Convert staging job to python and yaml (NOAA-EMC#2651)
  Fixed test on UNAVAILBLE in python Rocoto check (NOAA-EMC#2842)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully CI-Hercules-Passed **Bot use only** CI testing on Hercules for this PR has completed successfully CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add JEDI ATM equivalent to GSI eobs
6 participants