-
Notifications
You must be signed in to change notification settings - Fork 625
Compiling FDS with Intel oneAPI
oneAPI is the name of an open, standards-based unified programming model. The versions of FDS and Smokeview released by NIST for Windows, Linux, and macOS all use the Fortran (FDS) and C (Smokeview) oneAPI compilers provided by Intel, plus the Intel MPI libraries.
To build FDS with the Intel oneAPI compilers, download and install the Base and HPC toolkits at the links below. After clicking on the links, select the platform (Windows, Linux or macOS) you wish to build FDS on, and then the type of download. The download webpage will then give you instructions on how to obtain the installation file and how to install it. Install the base toolkit first, then the HPC toolkit.
The Windows oneAPI toolkit installers create the environment variables needed by the FDS build scripts.
Reboot your PC after installing the toolkits. To build FDS, cd into an FDS build directory and type make_fds.
For more details see these instructions.
If you are installing on a Linux system and using a bash shell, add the following lines to your .bashrc file
# oneAPI setup
source /opt/intel/oneapi/setvars.sh >& /dev/null
Next, clone the fds GitHub repository. Because FDS uses other GitHub repositories all stored under the name firemodels, it is best that you clone the fds repository into a directory named firemodels. If you also want to compile Smokeview, you can clone the smv repository into firemodels as well.
git clone git@github.com:firemodels/fds.git
cd to fds/Build/impi_intel_linux and type ./make_fds.sh
After the compilation is done, you can follow the procedure for running cases under linux.. Note that the name of the fds executable is not fds, but rather fds_impi_intel_linux to denote that it was compiled with the Intel oneAPI compilers and libraries.
Instructions for compiling FDS with Intel oneAPI on macOS:
NOTE: These steps are unnecessary if you are using the precompiled executable FDS/FDS6/bin/fds. In that case, use the instructions posted here.
Go to the Apple Store, search for the Xcode app and install. It is 12GB and may take more than an hour.
Note: Using the Apple Store to update Xcode seems to be a critical step. Just doing a command line update does not get the latest version.
Download the latest OpenMPI from https://www.open-mpi.org/. In this example, we will use openmpi4.1.2. Unzip the file in a local directory of your choice.
To configure and build OpenMPI, we recommend creating a script as follows. Here we will name this script confmake.sh and save it within the local OpenMPI directory (should be at the same level as configure). Note that we will be installing the libraries into /opt/openmpi412_oneapiu2022u1 (we are here using OneAPI version 2022.1).
#!/bin/bash
./configure --prefix /opt/openmpi412_oneapi2022u1 \
CC=icc CXX=icpc F77=ifort FC=ifort \
CFLAGS="-m64 -O2" \
CXXFLAGS="-m64 -O2" \
FFLAGS="-m64 -O2" FCFLAGS="-m64 -O2" \
LDFLAGS="-m64" \
--without-tm \
--without-psm\
--enable-mpirun-prefix-by-default \
--without-verbs \
--enable-static --disable-shared | tee CONFIGURE.out
make -j 4
From the terminal in the local directory where you unzipped OpenMPI, run the confmake.sh script. This may take from 1 to 3 hours depending on your system.
$ cd /Users/rmcdermo/Documents/Software/openmpi-4.1.2
$ ./confmake.sh
Once finished, create the install directory and make install.
$ sudo mkdir /opt/openmpi412_oneapi2022u1 # must match --prefix from OpenMPI configuration above
$ sudo make install
Once OpenMPI has been installed add the following in your ~/.bash_profile:
# Intel Compilers:
source /opt/intel/oneapi/setvars.sh >& /dev/null
export IFORT_COMPILER_LIB=/opt/intel/oneapi/compiler/latest/mac/compiler/lib
export MPIDIST=/opt/openmpi412_oneapi2022u1
export PATH=$MPIDIST/bin:$PATH
export LD_LIBRARY_PATH=$MPIDIST/lib:$LD_LIBRARY_PATH
Open a new terminal window and do:
$ cd /opt/intel/oneapi/mkl/latest/interfaces/mklmpi/
$ sudo make libintel64
This compiles quickly.
From the top level of your FDS repository do:
$ cd Build/mpi_intel_osx_64XX
$ ./make_fds.sh
...
You should notice the flag -WITH_MKL in the compile options if the MKL library has been correctly linked.
Last, we want to check that the MKL libraries are working correctly. Navigate to the Pressure_Solver verification folder and run the duct_flow_uglmat.fds case. The commands below assume you have clone the fds repository into a directory ~/firemodels/. Also, the -n 8 assumes you have 8 processes available on your mac. You might need to reduce this number or (as I have done) use -oversubscribe (not recommended, in general, because it is inefficient). This test case uses 8 meshes and therefore tests the sparse cluster solver from MKL.
cd ~/firemodels/fds/Verification/Pressure_Solver/
mpirun -n 8 -oversubscribe ~/firemodels/fds/Build/mpi_intel_osx_64/fds_mpi_intel_osx_64 duct_flow_uglmat.fds
If this runs, you have successfully built FDS with Intel oneAPI and MKL!