Build system improvements for CABLE offline

[SeanBryan51]

In preparation for building CABLE with spack, @harshula, @ccarouge and I have found areas where the current build system for CABLE offline can be improved.

• Move serial_build and parallel_build functionality into Makefile #189
  • The serial_build and parallel_build build scripts can go in the Makefile by creating rules specific to serial or mpi. This simplifies the build system by reducing the number of scripts. See here as an example.
• Remove temporary source directory by improving the Makefile #190
  • The current build system copies the source code into a temporary directory and executes make in that directory. This can make debugging confusing as debug information will point to the source code in the temporary directory rather than the actual source code. The Makefile should be able to handle the compilation without the need for a temporary directory (see a draft implementation of this here). Removing this from the build system will also simplify the build3.sh build script.
• Improve build3.sh CLI by creating switches for serial Vs mpi and debug Vs non-debug #191
  • For example, it is currently not possible to enable debug mode and mpi mode simultaneously.
• Solve dependency problems between targets/dependencies and ensure parallel builds succeed #192
  • Many of the rules in the current Makefile are missing dependencies. This causes the compilation to fail when we compile in parallel via make -j.

[SeanBryan51]

@harshula and I have scoped out solutions to address these issues. In short, we have concluded that moving to a CMake based build system is the best approach.

Addressing the requirements with CMake

• Remove temporary source directory by improving the Makefile #190
  Although this problem can be solved in the Makefile by specifying the relative paths to each source file, it requires updating all the Makefile Fortran module dependency rules which is a pain. CMake automatically handles Fortran module dependencies out of the box so doing this with CMake is much easier.
• Improve build3.sh CLI by creating switches for serial Vs mpi and debug Vs non-debug #191
  When invoking cmake, custom options, compilers and compiler flags can be configurable by setting the appropriate variables via the -D flag, for example: cmake -DCMAKE_Fortran_COMPILER="mpif90" -DCMAKE_Fortran_FLAGS="-O2 -fp-model precise" -S . -B build. These can be used for doing MPI only builds and enabling specific debug flags.
• Solve dependency problems between targets/dependencies and ensure parallel builds succeed #192
  As mentioned above, CMake automatically handles Fortran module dependencies out of the box. Parallelised builds can be done by running cmake --build -j <jobs>.

CMake advantages:

• Fortran module dependency resolution is completely automated with CMake. There are tools to generate Makefile dependency rules (see here, however these approaches often require maintenance - CMake does not require any maintenance.
• CMake is a build system generator. It can create a faster build system to use like ninja to speed up compilation time.
• CMake is less prone to build system inconsistencies due to it being more standardised and automated than tools such as GNU Make.
• A CMake build system could provide a basis for building libraries which could be linked against in a coupled model context.

CMake disadvantages:

• There may be a lack of CMake expertise in the community. This raises the need for a well documented guide for developers to assist those who are unfamiliar with CMake.

Consequences of moving to CMake

It is currently not possible to transition to CMake without breaking binary equivalence with the current build due to parts of the build system which are automated in CMake. To demonstrate binary equivalence when moving to CMake, the current build system will have to be massaged into the CMake way of doing things. The required changes include:

• Spurious usage of -O0 in LDFLAGS #196
• ld flags used in offline build are incorrect #198
  CMake provides a standardised way of linking against external dependencies via find_package which can be used in combination with pkg-config.

[JhanSrbinovsky]

Just a general comment here. The make tools will only apply to offline/. The future of which is less than certain, however likely to be around for at least 5 years even if we do adopt the JAC-Standalone route. But my main comment is that the intention of the existing Makefile(s), build scripts (apart from building an execuable) was to be transparent and easy to amend by developers. Potential developers/users are familiar with 1970s Makefiles, shell scripts etc. A large cohort of users will be on the NCI platform, where I suspect you have been developing this system so obviously the tools are available. However, an equally large cohort are not. In the past questions from new users about building etc have been unrelated to available tools as there is (I doubt) any UNIX/linux system that does not have bash, make etc. Is this going to be a problem? Docs on how to build will require updating to describe all of this.

[harshula]

Hi, While reviewing the existing build system, our observation was that the Makefile build system has been improperly amended by developers over time. Our understanding is that CMake automates a number of important steps, particularly module dependencies for Fortran projects. This may allow model developers to focus more on the Fortran code and reduce the changes to the build system.

[ccarouge]

@JhanSrbinovsky for tools availability, it is true that CMake isn't always available on all systems by default but it is very easy to install. I would think it is available on all supercomputers nowadays and it is available via Homebrew for Macs.

For ease of development, as Harshula said, if it is well done it might reduce the need for changing the build system following the development of CABLE hence making it simpler for development work.

[JhanSrbinovsky]

@ccarouge, fine with me. I gave up long ago on running CABLE locally anyway. In fact, we officially gave up on trying to support CABLE on anything less than at least a dedicated HPC system. The overwhelming majority of build issues were related to external tools/libs