Help needed with parallel performance of ABACUS with LCAO

[wxwth]

Details

Dear ABACUS developers,

I am a new user of ABACUS, and am trying to run ABACUS on a cluster with SR645 compute nodes. Each node has 128 AMD CPU cores, with the following configuration: 2 AMD Rome Zen2 7H12 CPUs (64 cores each, 2.60 GHz) and 512 GB RAM.

I installed ABACUS (LTSv3.10.1) using the toolchain, and tried both GNU (with GCC 8.3.1 on Red Hat Enterprise Linux) and Intel-oneAPI (with intel/21U2/suite module provided by the cluster administrator). The installations completed without any obvious errors or warnings.

However, as shown in the attached figure, I encountered parallel efficiency issues when running simulations of a spin-polarized system containing 153 atoms with the LCAO method (I did not test the PW method). I have tried various combinations of OpenMP threads and MPI processes, but I have not been able to fully use the 128 cores on a single node (when installed with the GNU toolchain) or achieve efficient multi-node parallelism (when installed with the Intel-oneAPI toolchain).

Because I plan to perform molecular dynamics simulations for this system using the HSE06 hybrid functional, parallel performance is crucial for me. I have attached two folders containing all input and output files for reference. Could you please kindly provide some guidance? Any suggestions regarding hybrid functional calculations for this system, especially on obtaining correct magnetization during molecular dynamics simulations, would also be greatly appreciated (at present, I only know to change “dft_functional pbe” to “dft_functional hse”). Thank you very much!

Sincerely,
Xu

GNU_1-thread_32-processes.tar.gz

Intel_8-threads_16-processes.tar.gz

Task list for Issue attackers (only for developers)

• Reproduce the performance issue on a similar system or environment.
• Identify the specific section of the code causing the performance issue.
• Investigate the issue and determine the root cause.
• Research best practices and potential solutions for the identified performance issue.
• Implement the chosen solution to address the performance issue.
• Test the implemented solution to ensure it improves performance without introducing new issues.
• Optimize the solution if necessary, considering trade-offs between performance and other factors (e.g., code complexity, readability, maintainability).
• Review and incorporate any relevant feedback from users or developers.
• Merge the improved solution into the main codebase and notify the issue reporter.