Skip to content

aarch64 op: Enable two‑stage SVE detection in component configuration #13204

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

vogma
Copy link

@vogma vogma commented Apr 22, 2025

This pull request refactors the build configuration for the OpenMPI aarch64 op component, aligning it with the existing approach used by the avx component.

Currently, the avx component configuration systematically tests SIMD instruction support (e.g., AVX2, AVX512) by incrementally applying compiler flags until compilation succeeds, independent from the host CPU's capabilities. In contrast, the existing aarch64 configuration lacks this mechanism, performing only basic compilation checks without utilizing additional flags or function attributes.

To address this gap, this pull request enhances the aarch64 configuration by incorporating checks using compiler function attributes (specifically, __attribute__((__target__("+sve")))). This enables SVE code and includes it in OpenMPI regardless of compiler flags provided explicitly via the command line, just like the avx component already does.

If compilation via function attribute is successful, the attribute is automatically inserted via macro into the SVE function templates within op_aarch64_functions.c. Runtime detection of processor capabilities (NEON or SVE) remains unchanged. No API's have been changed, only the build systems is modified along with a macro definition in the source files.

There is a discussion that goes into more detail in the developer user group.

Copy link
Member

@jsquyres jsquyres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless I'm missing something, it looks ok to squash the code changes in this PR down to a single commit.

If you'd like to preserve your whitespace changes, please separate those into a separate commit and label them as such (e.g., 11ff7f0).

@vogma vogma force-pushed the aarch64_build_update branch 3 times, most recently from 4d6bf1d to 1268c51 Compare April 22, 2025 17:12
bosilca
bosilca previously approved these changes Apr 22, 2025
@vogma
Copy link
Author

vogma commented Apr 22, 2025

I'm working on the failed test. On my AWS Graviton Instance the Tests run fine but i get an Illegal Instruction error when openmpi was compiled with lower -march flags. Looking into it..
EDIT: Actually i have accidentally set the -march parameter too high, with -march=armv8-a the examples run

Copy link
Member

@jsquyres jsquyres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vogma The macos CI error appears to be legitimate -- I can reproduce it manually. When I build on my Mac (M3) and try to run the ring_c example:

$ mpirun --map-by ppr:1:core examples/ring_c
--------------------------------------------------------------------------
prterun noticed that process rank 8 with PID 60263 on node Little-Beezle exited on
signal 4 (Illegal instruction: 4).
--------------------------------------------------------------------------

Can you investigate?

@vogma
Copy link
Author

vogma commented May 4, 2025

Thank you for testing it on your machine! I will try to find out why this happens.

@vogma
Copy link
Author

vogma commented May 5, 2025

I do not have access to Apple hardware but the macOS failure likely stems from executing this line in user mode:

__asm__("mrs %0, ID_AA64PFR0_EL1" : "=r"(id_aa64pfr0_el1));

On AArch64, MRS …, ID_AA64PFR0_EL1 is a privileged EL1 register access and will raise SIGILL if executed from EL0.

Linux’s kernel transparently traps SIGILL for certain system registers (including the SVE feature bits) and emulates their values in user space (see https://www.kernel.org/doc/html/latest/arch/arm64/cpu-feature-registers.html#implementation). To my knowledge macOS offers no equivalent SIGILL‐hook emulation of EL1 registers, so any direct MRS ID_AA64PFR0_EL1 from user code will fault. In this PR, the build-system refactoring enabled runtime SVE detection even when the hardware used during compilation did not support it, exercising the inline-asm path that was previously bypassed - hence the new SIGILL.

I updated the configure logic to skip the EL1‐based runtime feature probe on Apple platforms and tested the failed examples in the CI/CD pipeline on all available AWS Graviton generations (Graviton 2-4) and had no problems.

Could you rerun the tests please?

@vogma vogma requested a review from jsquyres May 5, 2025 17:01
@jsquyres
Copy link
Member

jsquyres commented May 5, 2025

Also, could you squash down to 1 commit? We prefer rebasing and squashing over merging from main. Thanks!

@jsquyres
Copy link
Member

jsquyres commented May 5, 2025

I forgot to mention: testing with the PR code today works on my Mac M3. 🎉

- Introduce AC_CACHE_CHECK probes for ARM Scalable Vector Extension (SVE)
  using both a default compile test and a second test with __attribute__((__target__("+sve"))).
- Define variables op_cv_sve_support and op_cv_sve_add_flags
- Update AM_CONDITIONAL and AC_DEFINE to expose SVE support macros
  (OMPI_MCA_OP_HAVE_SVE, OMPI_MCA_OP_SVE_EXTRA_FLAGS).
- Extend final AS_IF to enable the component when either NEON or SVE is available.
- Add a preprocessor guard around SVE-specific function attributes
- Encapsulate the +sve attribute behind OMPI_MCA_OP_SVE_EXTRA_FLAGS, ensuring
  that only builds which detected and enabled compiler SVE support will compile with
  SVE-targeted code paths.
- Simplifies later code by using SVE_ATTR in function declarations instead of
  repeating the attribute clause.
- apply SVE_ATTR macro in C source for conditional +sve targeting
- sve feature detection only on linux
- code review feedback

Signed-off-by: Marco Vogel <marco.vogel@fernuni-hagen.de>
@vogma vogma force-pushed the aarch64_build_update branch from 08ccb4b to 16cb214 Compare May 5, 2025 18:58
@vogma vogma requested a review from jsquyres May 5, 2025 19:01
@jsquyres
Copy link
Member

jsquyres commented May 5, 2025

Latest PR works for me on my Mac.

@bosilca Can you review?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants