MOM6-CICE6-WW3 coupling notes

[aekiss]

This is just a place to share the details we've learned about MOM6-CICE6-WW3 coupling with NUOPC.

[aekiss]

@DeniseWorthen's comments on CICE6-WW3 coupling, 2 Sep 2022

I can answer for where the working branches are on the EMC side.

Both ESCOMP (Mariana) and EMC (me) collaborate on the CICE6-NUOPC cap. The forks of the two institutions are not always in sync, and that is the case here. At EMC, I am required to stay up-to-date with the Consortium/main and so I am often ahead of what ESCOMP maintains.

I have merged the wave-ice coupling changes into our fork (ufs-community/ufs-weather-model#1381) at EMC. I also have merged the C-grid changes. We typically add these optional features into our system even if nobody is actively experimenting with them.

The ESCOMP and EMC forks will be sync'd and then added to the Consortium/main. Since ESCOMP is considered the "home" of the NUOPC cap, I generally prefer that Dave Bailey take the lead on that last step.

For WW3, at EMC the mesh cap currently resides w/in a branch (https://github.com/NOAA-EMC/WW3/tree/dev/ufs-weather-model). The feature resides w/in the mesh cap source code (the files named "wav_something.F90"). It's not completely functional w/in that branch.

We are working on a PR to update that branch (resolving other, unrelated issues) but in this case, those changes will probably make it into the ESCOMP fork first.

[aekiss]

@dabail10's comments on MOM6-CICE6 coupling, Oct 5, 2022:

We are aiming to have ESCOMP as the authoritative repository for the CESM NUOPC CICE cap. This will be here: https://github.com/ESCOMP/CESM_CICE

The actual source code (under src) will be kept up to date with the CICE Consortium as time allows. We will keep a fork of CICE at: https://github.com/ESCOMP/CICE

The main branch will be the same as Consortium main, but we will have a separate CESM branch to allow us to issue pull requests back to the Consortium.

Similarly for MOM, the CESM NUOPC interface will be here: https://github.com/ESCOMP/MOM_interface

They also have a MOM6 source code fork.

@gustavo-marques added:

The MOM6 repository for CESM is https://github.com/NCAR/MOM6 (branch dev/ncar).

[aekiss]

From Mariana (@mvertens) 7 Oct:

it would be really helpful if we could have a call where I could clarify the need for a workflow to actually set up the coupled configurations.
In our case we use the CIME Case Control System (https://github.com/ESMCI/cime) which accompanies a CESM checkout.
I sent Martin separate instructions for how to do this in a container application with all data model configurations as well as CICE6/MOM6/DATM out of the box.
If we could set up such a chat it would also help me to understand how Martin got the unified model working with NUOPC. In our case we need a separate driver that is model specific to drive the whole system and our atmosphere component (CAM or DATM) is just a component of the system not the driver. In addition, the CMEPS mediator is also just a component that can be shared between different models (e.g. the NOAA/UFS and CESM).

and 12 Oct:

I do want to mention that I had trouble compiling the CICE6/MOM6/WW3 application in the container and we have a work around and explanation. It was a problem with the FMS build. If you do start using the container and run into problems - please let me know and I'll send you the fix.

[aekiss]

See discussion re. CIME: #8

[aekiss]

Elizabeth Hunke (@eclare108213) gave a presentation on CICE6 at the Bluelink science meeting today. She said C-grid work is progressing well and testing is underway, particularly for a fix to checkerboarding at the corners of a rectangular domain. Test runs have not yet been done with tripolar grids.
(@ofa001 have I missed anything?)

[aekiss]

C-grid code has not yet been tested with EAP, though I don't think we were considering using that ice rheology.

[ofa001]

No we haven't yet used the EAP rheology, the other two rheologies that Ed Blockley mentioned that are causing checkerbording are in SI-cubed but inclued the elastic-brittle one which is one of the newer ones.

[aekiss]

an update from @ofa001:

As I replied to Paul last week and in our discussion at the TWG I hadn’t been on the CICE6 website for a few months, he asked about C-grid problems which impact the concentration fields which come from solving the rheology on the C-grid and near the tripole Found this intriguing and long discussions there, but it looks like they have reduced some of the C-grid chequereboarding after som e input form Bill Lipscomb

C grid noise in corners · Issue #792 · CICE-Consortium/CICE · GitHub

There are a number of items on the github (some now closed)

The open one still under discussion is this one started a couple of weeks ago by Jean-Francois Lemieux (Canada) talked about and an idealised case.

Simplify problem to explain the C grid/remapping checkerboard instability · Issue #835 · CICE-Consortium/CICE · GitHub

In the release notes for CICE6.4.1 so one set of bug fixes up from the one I have been looking at.

It says

The C-grid implementation is available. However validation is ongoing. Some noise is generated in the solution when incremental remapping advection is used with the C-grid. One solution seems to be to use the upwind advection scheme instead which is more diffusive. The Consortium would appreciate any feedback from the community as users test the C-grid in their applications both with respect to results and possible improvements.

[eclare108213]

To clarify: this issue #835 is not showing a current problem in the code, but rather exploring simplified configurations that we can use to demonstrate what is driving the checkerboarding in a publication, and with which we can explain how to fix it. This PR #833 fixes the checkerboarding but is on hold because the code crashes in longer simulations using the global grids. We are now using a slightly different method for computing the departure area that is remapped, which is more natural for advection on the C-grid, and occasionally there are rare cases (e.g. tiny slivers of triangles hanging over edges, land mask being applied when it shouldn't be, etc) that we have to adjust the logic to handle. We are getting close! We went through this process with the original incremental remapping scheme -- it takes a while to hit all the rare cases in our tests and get them fixed.

[aekiss]

Thanks for the clarification and update @eclare108213 - great to hear how much progress has been made.

[ofa001]

Thanks @eclare108213 I ralised it was I simplified example that you were using to demonstrate efffects you were seeing in the full model, and there was mention of a publication in the discussion, I was just updating aekiss in an email on progress after a local meeting here and he wanted to post it to the wider group and that there were a numbe rof other threads. I put in the email as it had some good graphical pictures of issues.