Enhance the option for inactive top cells

[cbegeman]

In MPAS-Dev#825, we introduced the option for inactive top cells, by changing the k-index for the surface cell to minLevelCell (and correponding edge and vertex variables). This PR extends the work to enable this option for more configurations. More justification for this work and detail about the scope of this Phase 2 development can be found in the design doc MPAS-Dev#811.

This PR has been migrated to MPAS-Dev#840.

[cbegeman]

If the following notation is given next to the routines in the checklist below that indicates a PASS:

"E3SM Cori intel" encompasses the following tests:

• SMS_D.T62_oQU120_ais20.MPAS_LISIO_TEST.cori-knl_intel (comparison with e3sm/master)
• PEM_Ln9.ne30_oECv3.A_WCYCL1850S.cori-knl_intel (comparison with e3sm/master)
• PET_Ln9.T62_oQU240.GMPAS-IAF.cori-knl_intel
• PET_Ln3.T62_oEC60to30v3wLI.GMPAS-DIB-IAF-ISMF.cori-haswell_intel

"E3SM Compy intel WC" encompasses the following tests:

• SMS_Ld1.ne30pg2_r05_EC30to60E2r2.A_WCYCL1850S_CMIP6.compy_intel.allactive-wcprod (comparison with e3sm/master)
• SMS_PS.northamericax4v1pg2_WC14to60E2r3.A_WCYCL1850S_CMIP6.compy_intel.allactive-wcprodrrm (comparison with e3sm/master)

"Standalone" refers to the z_level_dev testing suite. It encompasses the nightly regression suite and four additional ice-shelf test cases (comparison with ocean/develop and MPAS-Dev/compass/legacy + c1650df1 REMOVE LATER!!! Add z-leve dev. test suite).

"Z-level" refers to the sub-ice-shelf-2d z-level test case added here: https://github.com/xylar/compass/tree/add_z_level_sub_ice_shelf_2D. The namelist used has all prognostic terms enabled but should otherwise be assumed to have the options given in the compass branch unless otherwise specified. A pass indicates changes over 3 timesteps of less than 1e-8 in normalVelocity, less than 1e-14 in temperature, and less than 1e-6 in salinity from the initial state.

Init mode:

• ocn_init_setup_isomip_plus: Standalone

Forward:

• ocn_time_integration_si: Nightly test suite with config_time_integrator = 'semi_implicit'

  • ocn_time_integrator_si
  • ocn_time_integrator_si_init
  • ocn_time_integrator_si_preconditioner

• ocn_time_integration_rk4: Standalone, Z-level

  • ocn_time_integrator_rk4
  • ocn_time_integrator_rk4_diagnostic_update
  • ocn_time_integrator_rk4_accumulate_update
  • ocn_time_integrator_rk4_cleanup

• ocn_diagnostics: E3SM Cori intel, E3SM Compy intel WC, Standalone

  • ocn_filter_btr_mode_tend_vel()

• ocn_tendency: E3SM Cori intel, E3SM Compy intel WC, Standalone

  • ocn_tend_freq_filtered_thickness
  • PASS QU240/performance_test with config_use_freq_filtered_thickness = .true.

• ocn_tracer_advection_std: E3SM Cori intel, E3SM Compy intel WC, Standalone (includes baroclinic_channel/default with config_monotonic=.false.)

  • ocn_tracer_advection_std_tend:

• ocn_tracer_DMS: E3SM Cori intel, E3SM Compy intel WC, Standalone

  • ocn_tracer_DMS_compute
  • ocn_tracer_DMS_surface_flux_compute

• ocn_tracer_ecosys: E3SM Cori intel, E3SM Compy intel WC, Standalone

  • ocn_tracer_ecosys_compute
  • ocn_tracer_ecosys_surface_flux_compute

• ocn_tracer_MacroMolecules: E3SM Cori intel, E3SM Compy intel WC, Standalone

  • ocn_tracer_MacroMolecules_compute

• ocn_tracer_short_wave_absorption_jerlov: E3SM Cori intel, E3SM Compy intel WC, Standalone

  • ocn_tracer_short_wave_absorption_jerlov_tend
  • QU240/performance-test with config_sw_absorption_type='jerlov' is bit-for-bit with QU240/performance-test with config_sw_absorption_type='none'

• ocn_tracer_short_wave_absorption_variable: E3SM Cori intel, E3SM Compy intel WC, Standalone

  • ocn_tracer_short_wave_absorption_variable_tend
  • QU240/performance-test with config_sw_absorption_type='ohlmann0' requires a forcing file so this option hasn't been fully tested

• ocn_tracer_interior_restoring: E3SM Cori intel, E3SM Compy intel WC, Standalone

  • ocn_tracer_interior_restoring_compute
  • QU240/performance-test with config_use_activeTracers_interior_restoring=.true. is bit-for bit with QU240/performance-test with config_use_activeTracers_interior_restoring=.false.

• ocn_frazil_forcing: E3SM Cori intel, E3SM Compy intel WC, Standalone (Note: z-level-dev includes ice-shelf cavity with frazil)

  • ocn_frazil_forcing_layer_thickness
  • ocn_frazil_forcing_active_tracers
  • ocn_frazil_forcing_build_arrays

• ocn_diagnostics: E3SM Cori intel, E3SM Compy intel WC, Standalone

  • ocn_compute_KPP_input_fields()

• ocn_vmix: E3SM Cori intel, E3SM Compy intel WC, Standalone

  • ocn_compute_kpp_rhs
  • PASS single_column_model/sphere bfb
  • PASS single_column_model/planar non-bfb

• ocn_vmix_cvmix: Standalone

  • ocn_vmix_coef_cvmix_build
  • Test cvmixbackgroundtypeBryanLewis

• ocn_gm: Standalone, z-level

  • ocn_GM_compute_Bolus_velocity()

• ocn_tracer_hmix_del2: Standalone, Z-level with config_use_tracer_del2=.true.

  • ocn_tracer_hmix_del2_tend

• ocn_tracer_hmix_del4: Standalone, Z-level with config_use_tracer_del4=.true.

  • ocn_tracer_hmix_del4_tend

• ocn_tracer_hmix_Redi: Standalone, Z-level with config_use_Redi=.true.

  • ocn_tracer_hmix_Redi_tend
  • PASS Redi_verification/all

• ocn_tracer_nonlocalflux: E3SM Cori intel, E3SM Compy intel WC, Standalone

  • ocn_tracer_nonlocalflux_tend

• ocn_high_freq_thickness_hmix_del2: Standalone

  • ocn_high_freq_thickness_hmix_del2_tend
  • PASS QU240/performance_test with config_use_highFreqThick_del2 = .true.

• ocn_vel_hmix_del4: Standalone, Z-level with config_use_Redi=.true.

  • ocn_vel_hmix_del4_tend

• ocn_vel_hmix_leith: Standalone, Z-level with config_use_Redi=.true.

  • ocn_vel_hmix_leith_tend

• ocn_tracer_exponential_decay: E3SM Cori intel, E3SM Compy intel WC, Standalone

  • ocn_tracer_exponential_decay_compute
  • QU240/performance_test with config_use_activeTracers_exponential_decay=.true. has state validation error for both baseline and this branch

• ocn_tracer_ideal_age:

  • ocn_tracer_ideal_age_compute
  • PASS QU240/performance_test with config_use_activeTracers_idealAge_forcing=.true.

• ocn_vmix:

  • ocn_vel_vmix_tend_implicit_spatially_variable
    • PASS drying_slope/zstar_variableCd/1km
  • ocn_vel_vmix_tend_implicit_spatially_variable_mannings
    • PASS drying_slope/zstar_variableManningn/1km, non-bfb with baseline but changes identical to other routines
  • ocn_vel_vmix_tend_implicit_rayleigh
    • Unaware of existing test case, but changes identical to other routines

• ocn_tidal_forcing:

  • ocn_tidal_forcing_build_array
  • ocn_tidal_forcing_layer_thickness
  • PASS drying_slope/z_level/1km

• ocn_tidal_potential forcing:

  • ocn_compute_tidal_potential_forcing
  • PASS tides/USDEQU120at30cr10/harmonic_analysis_test

• ocn_wetting_drying:

  • ocn_wetting_drying_verify
  • ocn_prevent_drying_rk4
  • ocn_wetting_drying_wettingVelocity
  • PASS drying_slope/z_level/1km

• ocn_vel_pressure_grad

  • PASS QU240/performance_test is bfb when config_pressure_gradient_type=ssh_gradient
  • PASS QU240/performance_test is bfb when config_pressure_gradient_type=pressure_and_zmid
  • PASS QU240/performance_test is bfb when config_pressure_gradient_type=Jacobian_from_density

[dengwirda]

This is all looking really good @cbegeman!
I won't finalise this review for now --- I'll try to work through additional changes as they come in.

I agree with some of the comments added throughout, that in various cases it might be necessary to check the minLevelCell != 1 case to see whether additional offsets, etc are needed. Is this something you'd like to attempt in this PR, or just stick with the minLevelCell == 1 do no harm constraint?

[cbegeman]

@dengwirda Thanks for looking over the code changes so far!

I'm leaning toward leaving off the init mode changes in this PR, and then once we have the init mode changes in we can run a global case where minLevelCell != 1. That is, sticking with the do no harm standard. I am testing the sub-ice-shelf-2d case where minLevelCell != 1 along the way, but obviously that won't catch bugs outside of the code touched by those config options.

[dengwirda]

Yes, agree all round, I think leaving off init-mode and focusing on the ice-shelf cases is a good plan.