Skip to content

Fix error reporting when config_nfglevels is insufficiently large #1291

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

abishekg7
Copy link
Collaborator

In init_atmosphere, during the interpolation of first-guess meteorological fields from the intermediate files (config_init_case = 7 or 9), the value of config_nfglevels must be set to a minimum of the number of vertical levels in the intermediate file. Currently, the routines init_atm_case_gfs and init_atm_case_lbc have a logic to throw an error and exit if config_nfglevels is not sufficiently large. In practice, this error message is not reported consistently with intermediate files containing many levels and/or many MPI ranks being used.

This PR fixes the inconsistent error reporting issue by adding the required deallocate statements before the repeated calls to read_next_met_field in the case of config_nfglevels being insufficiently large. Without the deallocate one or more MPI ranks crash during the read of the field % slab from the intermediate file. This typically results in a crash of init_atmosphere without any useful error messages in the log files.

@abishekg7
Copy link
Collaborator Author

Tested config_init_case=7 with 24km and 480km grids using an ERA5:2010-10-23_00 intermediate file with 138 levels, both cases running 128 MPI ranks on 1 node. The tests with the develop branch failed with no error messages, whereas the test with this PR crashed after correctly reporting the error message about config_nfglevels being smaller than 138.

@mgduda mgduda self-requested a review February 26, 2025 18:38
@mgduda mgduda changed the base branch from develop to hotfix-v8.2.3 February 28, 2025 19:39
In init_atmosphere, during the interpolation of first-guess meteorological
fields from the intermediate files (config_init_case = 7 or 9), the value of
config_nfglevels must be set to a minimum of the number of vertical levels in
the intermediate file. Currently, the routines init_atm_case_gfs and
init_atm_case_lbc have a logic to throw an error and exit if config_nfglevels
is not sufficiently large. In practice, this error message is not reported
consistently with intermediate files containing many levels and/or many MPI
ranks being used.

This commit fixes the inconsistent error reporting issue by adding the required
deallocate statements before the repeated calls to read_next_met_field in the
case of config_nfglevels being insufficiently large. Without the deallocate,
one or more MPI ranks crash during the read of the field % slab from the
intermediate file. This typically results in a crash of init_atmosphere without
any useful error messages in the log files.
@abishekg7 abishekg7 force-pushed the init_atmosphere/fix_config_nfglevels_msg branch from ea3eec1 to 054cccd Compare February 28, 2025 20:42
@mgduda mgduda merged commit 2073976 into MPAS-Dev:hotfix-v8.2.3 Feb 28, 2025
mgduda added a commit that referenced this pull request May 23, 2025
This merge addresses several issues in the MPAS-Atmosphere model and in the MPAS
infrastructure. Specific changes include:

 * Correction of the pool from which lbc_scalar constituent indices are obtained
   in the init_atm_thompson_aerosols_lbc routine. Rather than obtaining
   index_nifa and index_nwfa from the state pool, the indices of lbc_nifa and
   lbc_nwfa should be obtained from the lbc_state pool. (PR #1249)

 * Correction to the computation of the soil temperature (TSLB) in the Noah-MP
   land surface scheme through the addition of initialization of the soil liquid
   water (SH2O) in the noahmp_init subroutine in module
   mpas_atmphys_lsm_noahmpinit.F prior to calling NoahmpInitMain. (PR #1244)

 * Correction of the units of the fields 'greenfrac', 'shdmin', 'shdmax',
   'vegfra', and 'albedo12m' from "unitless" to "percent" in the
   init_atmosphere and atmosphere core Registry.xml files. Also, a correction to
   the spelling of 'greenness' in several places. (PR #1248)

 * Removal of a duplicate allocation of indexToEdgeID % array in the
   mpas_io_setup_edge_block_fields routine that was the source of a memory leak.
   (PR #1258)

 * Fix for a memory leak in mpas_block_creator_build_cell_halos by deallocating
   the cellLimitField field before the routine returns. (PR #1264)

 * Fix for a bug in the logic for determining when decompositions can be reused
   by the SMIOL library. In almost any practical situation, however, this bug
   created no issues. (PR #1288)

 * Changes in the init_atmosphere core to provide more reliable error messages
   in case config_nfglevels is not set to a value that is at least as large as
   the number of vertical levels in the first-guess intermediate file. (PR #1291)

 * Correction of the loop for Noah-MP snow initialization, capping snow water
   equivalent maximum at 2000 mm. (PR #1300)

 * Fix for a bug in the horizontal 2nd-order filter for the CAM upper absorbing
   layer, where the wrong level in the kdiff field was being used when enforcing
   a lower-bound on kdiff. This absorbing layer is active only when
   config_mpas_cam_coef > 0.0. (PR #1302)

 * Fix in the mountain wave idealized test case initialization when multiple MPI
   tasks are used. The 'xc' variable, which represents the center-point location
   of the mountain, was previously computed based on the maximum xCell values
   local to an MPI task, leading to inconsistent values on each MPI rank. By
   finding the maximum of xCell over all MPI ranks and ensuring that all MPI
   ranks use this global maximum, the terrain field is computed consistently
   between serial and parallel runs of the init_atmosphere_model program for the
   mountain wave test case (config_init_case = 6). (PR #1312)

 * Correction to the calculation of the 2-meter diagnostics (T2M, TH2M, and Q2)
   when using the Noah-MP land surface scheme. While the computation of 2-meter
   diagnostics is the same for Noah and Noah-MP over oceans, it is different
   between the two land surface schemes over land. In Noah-MP, the 2-meter
   diagnostics are weighted as functions of their respective diagnostics over
   bare soil and over vegetation. The updated diagnostics for Noah and Noah-MP
   are now computed in the new file mpas_atmphys_sfc_diagnostics.F. (PR #1242)

 * Fix to provide consistency in the ringing behavior of recurring alarms after
   their reference time has been adjusted with a call to
   mpas_adjust_alarm_to_reference_time. Now, adjusting the reference time for an
   alarm will always leave that alarm in a state such that it is considered by
   the mpas_is_alarm_ringing routine to be ringing at the current time. With
   this fix, limited-area simulations can be restarted at times between LBC
   updates, provided the reference_time attribute for the 'lbc_in' stream is set
   to the simulation initial time in the streams.atmosphere file. (PR #1290).

 * Correction of an indexing error for rvcuten in code blocks specific to the
   Grell-Freitas scheme in the convection driver. Specifically, in the
   convection_from_MPAS and convection_to_MPAS routines, rvcuten used (k,k) as
   indexing in a loop, where (k,i) is needed. Since the Grell-Freitas scheme
   does not provide momentum tendencies, the changes in this merge have no
   impact on results. (PR #1283)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants