Atmospheric Nudging with NorESM3 spectral element scheme

[MichaelSchulzMETNO]

Nudging winds in NorESM3 would be useful to do shorter runs, eg in a PPEs.

The shift to the dynamical core with spectral elements is suspected to require a different solution then in NorESM 2.3 . Is that true? What needs to be done?

Any experience and suggestions on how to develop this functionality and test this?
@IngoBethke @DirkOlivie @Ovewh @gold2718

[gold2718]

An example of (the new CAM) nudging configured for both FV and SE can be found in:
<NorESM root>/components/cam/bld/namelist_files/use_cases/hist_trop_strat_nudged_cam6.xml

Is this helpful?

[IngoBethke]

Nudging functionality for SE should be in place already since CESM2.1. When I find the time, I will try to test it.

[MichaelSchulzMETNO]

thanks @IngoBethke, would be great to have an example simulation with beta01 or beta02 documented in
https://github.com/NorESMhub/noresm3_dev_simulations/issues

[IngoBethke]

I will give it a try and will report back in a week or so. There are two use cases to test: (1) nudging against output produced by the model itself and (2) nudging against reanalysis e.g. ERA5. The first use case is easy to test as it does not require any interpolation of the nudging input data. The second case is a bit more work. I will test (1) first and then (2).

[Ovewh]

Hi @IngoBethke what is the status? I would be happy with an example of just nudging against model output. I'm in the process of design a perturbed parameter ensemble (PPE) targeting aerosol processes. I would soon like to do some initial one at the time sensitivity tests of the parameters I'm planning to perturbe in the PPE.

[IngoBethke]

Hi @Ovewh @MichaelSchulzMETNO

U,V nudging in NorESM3 works out of the box, as shown in the verification plot below (I produced some hourly output at a single location).

I performed three NF2000 simulations with noresm3_0_beta02:

NF2000_ne30pg3_ne30pg3_mtn14_default_0001-01-01: reference that produces the nudging data, cold start in 0001-01-01

NF2000_ne30pg3_ne30pg3_mtn14_nudgeUV_0001-01-10: nudged run, cold start in 0001-01-10 (i.e. ten days later than the reference)

NF2000_ne30pg3_ne30pg3_mtn14_default_0001-01-10: free run without nudging, cold start in 0001-01-10 (i.e. ten days later than the reference)

The output and content from case directories can be accessed at https://ns9039k.web.sigma2.no/datalake/ingo/noresm3_nudging/
or locally on NIRD at /nird/datalake/NS9039K/www/ingo/noresm3_nudging

On betzy, the case directories can be accessed at /cluster/projects/nn9039k/people/ingo/noresm3_0_beta02/cases
(let me know if you need access to nn9039k, ns9039k)

Important notes:

Some useful info on the nudging implementation is found at
https://ncar.github.io/CAM/doc/build/html/users_guide/physics-modifications-via-the-namelist.html#nudging
https://noresm-docs.readthedocs.io/en/noresm2.2/Nudging.html

The nudging functionality is always compiled and available. The nudging implementation lives in components/cam/src/physics/cam/nudging.F90 (the old metdata implementation that only works for the FV core is still available components/cam/src/dynamics/fv/metdata.F90).

To use the nudging functionality, the only file that needs to be modified in the case directory is user_nl_cam. An example from my test is https://ns9039k.web.sigma2.no/datalake/ingo/noresm3_nudging/cases/NF2000_ne30pg3_ne30pg3_mtn14_nudgeUV_0001-01-10/user_nl_cam

The nudging implementation "guesses" the file names of the nudging input based on Nudge_Path and Nudge_File_Template specification in the CAM's namelist (there is no file list like in the metdata implementation). Different from the metdata implementation, the time information is given via the file names and there has to be one file per record. In-file metadata is not used, the only thing the matters is that the variable names are correct (i.e. set to "U", "V", "T", "Q", "PS").

If the model does not find a nudging file then it will simple continue without applying nudging. When testing, I recommend to check the info in the atmospheric log file if the file is found and nudging is applied. In addition, you can write extra nudging diagnostics to the diagnostic output (like the "target" U that I plotted as stippled lines).

The effect of the namelist parameters Nudge_Beg_Day, Nudge_Beg_Month, Nudge_Beg_Year, Nudge_End_Day, Nudge_End_Month, Nudge_End_Year is not exactly what I anticipated. If the model date is outside the specified period then the first/last nudging data in that period is applied (i.e. the nudged fields are kept constant). Somewhat questionable functionality if you ask me. Maybe best not using these parameters.

Happy to elaborate if you have any questions.

I will have a look at the ERA5 nudging input preparation when I find the time.

Plotting script at https://ns9039k.web.sigma2.no/datalake/ingo/noresm3_nudging/noresm3_nudging_test.py

[MichaelSchulzMETNO]

wonderful - thanks @IngoBethke

[Ovewh]

I'm struggling to make progress on my own to generate ERA5 forcing data on the spectral element grid. While I have manage to interpolate a single field of ERA5 to the ne30pg3. Using the ESMF_RegridWeightGen to create the weighting file and then loop over all the weights and them multiplying with the ERA data. The looping becomes prohibitively slow at least in python.

This is what i did:

• 1. Add longitude and latitude bounds to the ERA5 fields to make them CFGRID compliant
• 2. I used a SCRIP formated grid description of the ne30pg3 grid
• 3. Generate weights with ESMF_RegridWeightGen: ESMF_RegridWeightGen -s era5_CFGRID.nc -d ne30pg3_scrip_170611.nc -w era5_to_se_ne30pg3_weights.nc --ignore_unmapped
• 4. Then loop over all the weights:

model_nudged_raw = xr.open_dataset('NF2000_ne30pg3_ne30pg3_mtn14_nudgeUV_0001-01-10.cam.h1i.0001-01-10-43200.nc')
model_nudged = model_nudged_raw[['U','V']].isel(lev=0)
# Creating empty dataset of the same shape as the tested model generated nudging data
out_field = xr.zeros_like(model_nudged).isel(time=0)
era5_one_level = era5_raw.isel(time=0,level=0)

S = weights['S']
row = weights['row']-1
col = weights['col']-1

nlon0=len(era5_one_level['longitude'])
for i in range(0,len(S)) :
     out_field['U'][row[i]] = out_field['U'][row[i]] + S[i] * era5_one_level['u'][col[i] // nlon0,  col[i] % nlon0]
     out_field['V'][row[i]] = out_field['V'][row[i]] + S[i] * era5_one_level['v'][col[i] // nlon0,  col[i] % nlon0]

However this is very inefficient and I would presume there be more optimized approaches somewhere? ESMF have ESMF_regrid CLI tool, which according to the documentation should do both calculation of weights and multiplication, but I'm not able to get that working. Something about the grid definition files that not quite right.

There is also a set of old ncl scripts nudging, However i have limited experience with ncl and not sure'll be able to make that work.

Example of one of the interpolated ERA5 windfield at one level, that I manged to produce.

[mvertens]

@Ovewh - I think you need to do this in parallel and I think writing a routine in fortran that leverages ESMF and PIO to do this would solve the problem.
I recently wrote several modules in the BLOM NUOPC cap to map WOA multi-level initial data to the BLOM grid. This does not map the vertical levels - only the horizontal. If you checkout noresm3_0_beta03a - you can see these routines in the
$SRCROOT/components/blom/drivers/nuopc

• ocn_map_woa.F90 (driver)
• ocn_pio_share.F90
• mod_io_input.F90 (maps to the blom grid)
• mod_io_output.F90 (outputs the horizontal mapped data)
  This uses ESMF and PIO.
  You should theoretically be able to leverage the code in mod_io_input.F90 and mod_io_output.F90 with your input data - but you will need to build a stand-alone executable to do this. Here is an example of this that @jedwards4b (the author of PIO) pointed me to.
  https://github.com/NCAR/ParallelIO/blob/main/examples/basic/testpio.F90
  I am happy to help with this. Can we meet tomorrow and go over this?