Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(*)Add parentheses for FMA rotational symmetry #1634

Merged

Commits on Jul 29, 2024

  1. (*)Parenthesize squares of wind stresses for FMAs

      Added parentheses to expressions taking the squares of the wind stress
    components in 70 lines in 7 files so that these expressions will be rotationally
    invariant when fused-multiply-adds are enabled.  All answers are bitwise
    identical in cases without FMAs, but answers could change with FMAs.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    6628dbf View commit details
    Browse the repository at this point in the history
  2. (*)Parenthesize continuity_PPM curv_3 expressions

      Added parentheses to 9 expressions like `curv_3 = h_W(i) + h_E(i) - 2.0*h(i)`
    in PPM_limit_pos, zonal_flux_layer, zonal_flux_thickness, merid_flux_layer and
    merid_flux_thickness, changing them to `curv_3 = (h_W(i) + h_E(i)) - 2.0*h(i)`.
    This change enforces the order of arithmetic that is required to give rotational
    symmetry, but it also is the order that the Intel, GNU, and Nvidia compliers
    were all already using in these expressions.  Moreover, had the order of
    arithmetic ever been anything else, this would have led to failures in our
    rotational consistency and redundant point consistency testing, and almost
    certainly would have been detected before. However, by adding these parentheses,
    there is a remote chance that the addition of these parentheses could change
    answers for some compiler or compiler settings we have never tested before.
    This change should not impact any FMA-enabled calculations.  All answers are
    bitwise identical in the MOM6-examples regression suite as run on Gaea.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    f0e61f3 View commit details
    Browse the repository at this point in the history
  3. (*)Add parentheses for oblique OBCs with FMAs

      Added parentheses to 16 expressions setting the grad_gradient arrays with
    oblique_grad open boundary conditions and setting cff_new with all kinds of
    oblique boundary conditions so that they will be rotationally invariant when
    fused-multiply-adds are enabled.  All answers are bitwise identical in cases
    without FMAs, but answers with certain types of open boundary conditions could
    change with FMAs.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    4f710ef View commit details
    Browse the repository at this point in the history
  4. (*)Add parentheses for density_integrals with FMAs

      Added parentheses to 150 lines in the 5 generic density integral routines
    (int_density_dz_generic_pcm, int_density_dz_generic_plm,
    int_density_dz_generic_ppm, int_spec_vol_dp_generic_pcm and
    int_spec_vol_dp_generic_plm) in the MOM_density_integrals module so that they
    will be rotationally invariant when fused-multiply-adds are enabled.  All
    answers are bitwise identical in cases without FMAs, but answers could change
    with FMAs.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    9172cd5 View commit details
    Browse the repository at this point in the history
  5. (*)Add parentheses to 4 EOS int_density routines

      Added parentheses to 140 lines in 8 int_density_dz and int_spec_vol_dp
    routines for the linear, Wright, Wright_full and Wright_red equations of state
    so that they will be rotationally invariant when fused-multiply-adds are
    enabled.  All answers are bitwise identical in cases without FMAs, but answers
    could change with FMAs.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    24091cc View commit details
    Browse the repository at this point in the history
  6. (*)Simplify density integral parentheses

      Removed recently added parentheses around expressions like '+ (hL*hR)' in 110
    lines in MOM_density_integrals and 4 equation of state module to reflect that
    these parentheses are not necessary for rotational symmetry in
    FMAs.  All answers are bitwise identical in cases without FMAs, but
    answers could change with FMAs.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    8066a3d View commit details
    Browse the repository at this point in the history
  7. (*)Parenthesize PressureForce_Montgomery for FMAs

      Added parentheses to 4 lines in PressureForce_Mont_nonBouss and
    PressureForce_Mont_Bouss so that they will be rotationally invariant when
    fused-multiply-adds are enabled.  All answers are bitwise identical in cases
    without FMAs, but answers could change with FMAs in cases that use the
    Montgomery potential form of the pressure gradient accelerations.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    99fd957 View commit details
    Browse the repository at this point in the history
  8. (*)Parenthesize calc_isoneutral_slopes for FMAs

      Added parentheses to 4 lines in calc_isoneutral_slopes so that they will be rotationally invariant when
    fused-multiply-adds are enabled.  All answers are bitwise identical in cases
    without FMAs, but answers could change with FMAs.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    307a4e2 View commit details
    Browse the repository at this point in the history
  9. (*)Parenthesize MOM_calc_varT for FMAs

      Added parentheses to 2 lines in MOM_calc_varT so that they will be
    rotationally invariant when fused-multiply-adds are enabled.  In this case, FMAs
    can still be applied to the impacted lines, exploiting that the masks are always
    0 or 1.  Also added parentheses to 2 other lines used to generate the stochastic
    pattern for rotational symmetry with FMAs.  All answers are bitwise identical in
    cases without FMAs, but answers could change with FMAs.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    ce559ce View commit details
    Browse the repository at this point in the history
  10. (*)Parenthesize tracer_hordiff for FMAs

      Added parentheses to the calculation of the diffusive temperature changes in
    tracer_hordiff so that it will be rotationally invariant when
    fused-multiply-adds are enabled.  All answers are bitwise identical in cases
    without FMAs, but answers could change with FMAs.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    c344a11 View commit details
    Browse the repository at this point in the history
  11. (*)Parenthesize iceberg_forces for FMAs

      Added parentheses to the calculation of the iceberg contribution to the
    fractional area of ice shelves in iceberg_forces so that it will be rotationally
    invariant when fused-multiply-adds are enabled.  All answers are bitwise
    identical in cases without FMAs, but answers could change with FMAs enabled in
    cases with tabular icebergs.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    b2beab2 View commit details
    Browse the repository at this point in the history
  12. (*)Parenthesize CoriolisStokes and LA_Stk for FMAs

      Added parentheses to the calculation of the Stokes-drift Coriolis velocity
    increments in CoriolisStokes so that it will be rotationally invariant when
    fused-multiply-adds are enabled.  All answers are bitwise identical because
    CoriolisStokes is still under development and is never called, with a fatal
    error occurring if anyone tries to use it.  Also added parentheses to two
    expressions calculating the magnitude of the Stokes velocity in
    get_Langmuir_Number.  Answers could change for some cases that use Langmuir
    turbulence parameterizations with FMAs enabled.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    5398e6f View commit details
    Browse the repository at this point in the history
  13. +(*)Add and use G%Coriolis2Bu

      Added the new element Coriolis2Bu to the ocean_grid_type and the
    dyn_horgrid_type to hold the square of the Coriolis parameter, and use this
    array in 10 routines (including btstep, set_dtbt, calculate_diagnostic_fields,
    VarMix_init, propagate_int_tide, Calculate_kappa_shear, Calc_kappa_shear_vertex
    and add_MLrad_diffusivity) that had been calculating and averaging the square of
    the Coriolis parameter.  This could change some answers with FMAs enabled
    because the compilers were previously free to split up some of the squares
    when averaging the squared Coriolis parameter, but without FMAs all answers are
    bitwise identical.  This commit does add a new element to two transparent
    types.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    56d053a View commit details
    Browse the repository at this point in the history
  14. (*)Parenthesize thickness_diffuse for FMAs

      Added parentheses to 17 expressions in thickness_diffuse_full,
    thickness_diffuse and thickness_diffuse_init to give rotationally consistent
    solutions when fused-multiply-adds are enabled.  One comment was also added to
    note that the calculation of PE_release_h is does not exhibit rotational
    symmetry when MEKE_GM_SRC_ALT is set to true.  All answers are bitwise identical
    in cases without FMAs, but answers could change when FMAs are enabled.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    49419f7 View commit details
    Browse the repository at this point in the history
  15. (*)Parenthesize Zanna_Bolton for FMAs

      Added parentheses to 2 expressions in the Zanna_Bolton code and rearranged
    another line so that the u- and v-discretizations introduce terms in the same
    order so that the Zanna_Bolton code will exhibit rotationally consistent
    solutions when fused-multiply-adds are enabled.  All answers are bitwise
    identical in cases without FMAs, but answers could change with FMAs enabled in
    cases that use the Zanna-Bolton parameterization.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    03dc6f9 View commit details
    Browse the repository at this point in the history
  16. (*)Parenthesize MOM_internal_tides for FMAs

      Added parentheses to 19 expressions in the MOM_internal_tides propagation code
    to exhibit rotationally consistent solutions when fused-multiply-adds are
    enabled.  All answers are bitwise identical in cases without FMAs, but answers
    could change when FMAs are enabled in models that use the ray-tracing based
    internal tides code.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    654cd4a View commit details
    Browse the repository at this point in the history
  17. (*)Parenthesize find_uv_at_h for FMAs

      Added parentheses to 10 expressions in find_uv_at_h to exhibit rotationally
    consistent solutions and treat the velocities at both edges of a tracer cell
    equivalently when fused-multiply-adds are enabled. All answers are bitwise
    identical in cases without FMAs, but answers could change when FMAs are enabled.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    ebf02a9 View commit details
    Browse the repository at this point in the history
  18. (*)Parenthesize set_viscous_ML for FMAs

      Added parentheses to 19 expressions in set_viscous_ML, set_u_at_v and
    set_v_at_u to treat the velocities at both edges of a tracer cell equivalently
    when fused-multiply-adds are enabled, and thereby to exhibit exhibit
    rotationally consistent solutions.  Also swapped the order of the u- and
    v-components in the u-point calculation of Uh2 to mirror the order of the
    corresponging v-point calculation for the same purpose.  All answers are bitwise
    identical in cases without FMAs, but answers could change when FMAs are enabled.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    c0bef18 View commit details
    Browse the repository at this point in the history
  19. (*)Rearrange calc_kappa_shear_vertex for FMAs

      Added mathematically equivalent rearrangements of the code in
    calc_kappa_shear_vertex that interpolates velocities, temperatures and
    salinities to the vertices to expose the mask variables while ensuring that the
    other multiplications occur within parentheses so that they will exhibit
    rotational symmetry when fused-multiply-adds are enabled.  FMAs can still occur,
    but it will be multiplication by the 0-or-1 masks that are fused with an
    addition.  Also added parentheses to 3 expressions calculating the squared shear
    in calculate_projected_state for rotational symmetry with FMAs.  All answers are
    bitwise identical in cases without FMAs, but answers could change when FMAs are
    enabled.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    0b50a15 View commit details
    Browse the repository at this point in the history
  20. (*)Parenthesize MOM_set_diffusivity for FMAs

      Added parentheses to 4 expressions in add_drag_diffusivity, set_BBL_TKE and
    add_LOTW_BBL_diffusivity setting the bottom-drag contributions to TKE and
    friction velocity so that they will exhibit rotationally consistent solutions
    when fused-multiply-adds are enabled.  All answers are bitwise identical in
    cases without FMAs, but answers could change when FMAs are enabled.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    f0c52dd View commit details
    Browse the repository at this point in the history
  21. (*)Parenthesize CorAdCalc for FMAs

      Added parentheses to 20 expressions in CorAdCalc and one in gradKE to exhibit
    rotationally consistent solutions when fused-multiply-adds are enabled.  All
    answers are bitwise identical in cases without FMAs, but answers could change
    when FMAs are enabled.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    64b851c View commit details
    Browse the repository at this point in the history
  22. (*)Parenthesize MOM_barotropic for FMAs

      Added parentheses to 18 expressions in btstep, and one more each in set_dtbt
    and barotropic_init to exhibit rotationally consistent solutions when
    fused-multiply-adds are enabled.  All answers are bitwise identical in cases
    without FMAs, but answers could change when FMAs are enabled.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    46e8b66 View commit details
    Browse the repository at this point in the history
  23. (*)Parenthesize MOM_lateral_mixing_coeffs for FMAs

      Added parentheses to 19 expressions in 5 routines (calc_Visbeck_coeffs_old,
    calc_Eady_growth_rate_2D, calc_slope_functions_using_just_e,
    calc_QG_Leith_viscosity VarMix_init) in MOM_lateral_mixing_coeffs.F90 to give
    rotationally consistent solutions when fused-multiply-adds are enabled.  Also
    reordered terms in a sum in the calculation of beta_dx2_u to mirror that of
    beta_dx2_v, also for rotational symmetry with FMAs.  All answers are bitwise
    identical in cases without FMAs, but answers could change for some parameter
    settings when FMAs are enabled.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    6216fa1 View commit details
    Browse the repository at this point in the history
  24. (*)Parenthesize MOM_hor_visc for FMAs

      Added parentheses to 40 expressions horizontal_viscosity and another 14
    expressions in in hor_visc_init and 3 more in align_aniso_tensor_to_grid to give
    rotationally consistent solutions when fused-multiply-adds are enabled.   Also
    swapped the order of two terms in the expression for Del2u to mirror the order
    of the corresponding terms in Del2v for rotational symmetry with FMAs.  All
    answers are bitwise identical in cases without FMAs, but answers could change
    when FMAs are enabled.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    ffef92f View commit details
    Browse the repository at this point in the history
  25. (*)Parenthesize initialization squares for FMAs

      Added parentheses to 20 sums of squares of x- and y- distances or velocity
    components used for initialization in 8 modules to give rotationally consistent
    solutions when fused-multiply-adds are enabled.  All answers are bitwise
    identical in cases without FMAs, but answers could change when FMAs are enabled.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    fc2af28 View commit details
    Browse the repository at this point in the history
  26. (*)Parenthesize parameterization squares for FMAs

      Added parentheses to 29 sums of squares of velocity or other vector components
    used in parameterizations in 9 modules to give rotationally consistent solutions
    when fused-multiply-adds are enabled.  All answers are bitwise identical in
    cases without FMAs, but answers could change when FMAs are enabled.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    44f1130 View commit details
    Browse the repository at this point in the history
  27. (*)Parenthesize diagnostics for FMAs

      Added parentheses to 9 diagnostics of Coriolis accelerations or expressions
    used in the kinetic energy budgets to give rotationally consistent solutions
    when fused-multiply-adds are enabled.  All answers are bitwise identical in
    cases without FMAs, but answers could change when FMAs are enabled.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    182223c View commit details
    Browse the repository at this point in the history
  28. (*)Parenthesize tracer_advect PPM edge values

      Added parentheses to 4 tracer edge value calculations used with PPM tracer
    advection to give rotationally consistent solutions when fused-multiply-adds are
    enabled.  Although these lines may not appear to need parentheses, some
    compliers appear to be putting these expressions directly into others, where the
    direction of the flow seems to determine which multiplications are incorporated
    into FMAs.  All answers are bitwise identical in cases without FMAs, but answers
    could change when FMAs are enabled.
    Hallberg-NOAA committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    e810ac5 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2024

  1. (*)More parentheses in density_integrals for FMAs

      Added parentheses around the full expressions for intz and intp in 10 more
    lines in 4 generic density integral routines (int_density_dz_generic_pcm,
    int_density_dz_generic_ppm, int_spec_vol_dp_generic_pcm and
    int_spec_vol_dp_generic_plm) in the MOM_density_integrals module so that
    non-Boussinesq cases will be rotationally invariant when fused-multiply-adds are
    enabled.  Although this might not seem to do anything, these parentheses do
    matter if these expressions are in-lined in the sums where they are used a few
    lines later.  The analogous parentheses had previously been added to
    int_density_dz_generic_plm.  All answers are bitwise identical in cases without
    FMAs, but answers could change with FMAs.
    Hallberg-NOAA committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    ffa766b View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2024

  1. (*)Add parentheses in end_value_h4 for FMAs

      Added parentheses to prevent FMAs in 4 expressions in end_value_h4 that rely
    on exact vertical symmetry in order to get the cancellations that are necessary
    to pass the vertical remapping unit testing.  Before this change, the MOM6
    unit-testing was failing when FMAs are enabled, but with it the unit-testing is
    passing even when FMAs are enabled.  All answers are bitwise identical in cases
    without FMAs, but answers could change with FMAs.
    Hallberg-NOAA committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    fd82861 View commit details
    Browse the repository at this point in the history