Skip to content

Enable the use of the more modern 'mpi_f08' module for MPI #1142

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Feb 21, 2024

Conversation

mgduda
Copy link
Contributor

@mgduda mgduda commented Feb 20, 2024

This PR enables MPAS to make use of the more modern mpi_f08 module introduced in MPI-3.0.

The top-level Makefile now tries to compile a test program that uses mpi_f08, and if that test is
successful, the macro MPAS_USE_MPI_F08 is defined in the CPPFLAGS used in the build; otherwise,
MPAS will make use of the mpi module as it previously did.

With the use of the mpi_f08 module, certain MPI types are no longer integers in Fortran, but are derived
types; e.g., MPI_Comm, MPI_Request, MPI_Datatype, and MPI_Info. However, in some instances,
an integer-typed MPI type is still needed for interoperability, and the MPI standard permits this to be done
through the mpi_val member of all MPI derived types, e.g., MPI_Comm % mpi_val.

By defining the MPAS_USE_MPI_F08 macro at compile time, MPAS can now use the
more modern 'mpi_f08' module for MPI. If the MPAS_USE_MPI_F08 macro is not
defined, MPAS uses the 'mpi' module as prior to this commit.

With the use of the 'mpi_f08' module, certain MPI types are no longer integers
in Fortran, but are derived types; e.g., MPI_Comm, MPI_Request, MPI_Datatype,
MPI_Info.

In some instances, an integer-typed MPI type is still needed for
interoperability, and the MPI standard permits this to be done through the
'mpi_val' member of all MPI derived types, e.g., MPI_Comm % mpi_val.
By trying to compile a test program that uses the 'mpi_f08' module, the
top-level Makefile can now detect whether the MPI implementation provides an
'mpi_f08' module and, if so, adds a definition of the MPAS_USE_MPI_F08 macro to
the CPPFLAGS used in the build.
@mgduda
Copy link
Contributor Author

mgduda commented Feb 20, 2024

The motivation for the changes in this PR arises from compilation errors with newer GNU compilers when using the mpi module from the cray-mpich library that is available on NCAR's Derecho system:

mpas_dmpar.F:5115:25:

 5115 |           call MPI_Irecv(commListPtr % ibuffer, commListPtr % nList, MPI_INTEGERKIND, commListPtr % procID, commListPtr % procID, dminfo % comm, commListPtr % reqID, mpi_ierr)
      |                         1
......
 7892 |             call MPI_Irecv(commListPtr % rbuffer, commListPtr % nList, MPI_REALKIND, commListPtr % procID, commListPtr % procID, &
      |                           2
Error: Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(4)/REAL(4)).

While these specific errors can be avoided by adding the -fallow-argument-mismatch flag for the gfortran compiler, it was deemed sub-optimal to simply demote errors to warnings.

Since there are combinations of compilers and MPI implementations on Derecho that still don't provide an mpi_f08 module (e.g., nvhpc/24.1 and cray-mpich/8.1.27), we still require support for the older mpi module, justifying the use of preprocessing directives to activate the use of either mpi or mpi_f08.

@mgduda mgduda requested a review from gdicker1 February 20, 2024 20:27
Copy link
Collaborator

@gdicker1 gdicker1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works!

I confirmed that builds now succeed on Derecho with make gnu CORE=atmosphere when using the appropriate modules.

@mgduda mgduda merged commit b603822 into MPAS-Dev:develop Feb 21, 2024
favba added a commit to pedrospeixoto/MPAS-BR that referenced this pull request Feb 26, 2024
Merge branch 'framework/mpas_use_mpi_f08' into develop (PR MPAS-Dev#1142)

This merge enables MPAS to make use of the more modern mpi_f08 module introduced
in MPI-3.0.

The top-level Makefile now tries to compile a test program that uses mpi_f08,
and if that test is successful, the macro MPAS_USE_MPI_F08 is defined in the
CPPFLAGS used in the build; otherwise, MPAS will make use of the mpi module as
it previously did.

With the use of the mpi_f08 module, certain MPI types are no longer integers in
Fortran, but are derived types; e.g., MPI_Comm, MPI_Request, MPI_Datatype, and
MPI_Info. However, in some instances, an integer-typed MPI type is still needed
for interoperability, and the MPI standard permits this to be done through the
mpi_val member of all MPI derived types, e.g., MPI_Comm % mpi_val.

* framework/mpas_use_mpi_f08:
  Add logic in top-level Makefile to detect mpi_f08 module availability
  Allow MPAS to use either of 'mpi_f08' or 'mpi' modules
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants