Compiler flag handlers (spack#6415)

This adds the ability for packages to apply compiler flags in one of
three ways: by injecting them into the compiler wrapper calls (the
default in this PR and previously the only automated choice);
exporting environment variable definitions for variables with
corresponding names (e.g. CPPFLAGS=...); providing them as arguments
to the build system (e.g. configure).

When applying compiler flags using build system arguments, a package
must implement the 'flags_to_build_system_args" function. This is
provided for CMake and autotools packages, so for packages which
subclass those build systems, they need only update their flag
handler method specify which compiler flags should be specified as
arguments to the build system.

Convenience methods are provided to specify that all flags be applied
in one of the 3 available ways, so a custom implementation is only
required if more than one method of applying compiler flags is
needed.

This also removes redundant build system definitions from tutorial
examples

lib/spack/docs/packaging_guide.rst

@@ -2536,98 +2536,104 @@ build system.
 Compiler flags
 ^^^^^^^^^^^^^^
 
-Compiler flags set by the user through the Spec object can be passed to
-the build in one of two ways. For packages inheriting from the
-``CmakePackage`` or ``AutotoolsPackage`` classes, the build environment
-passes those flags to the relevant environment variables (``CFLAGS``,
-``CXXFLAGS``, etc) that are respected by the build system. For all other
-packages, the default behavior is to inject the flags directly into the
-compiler commands using Spack's compiler wrappers.
+Compiler flags set by the user through the Spec object can be passed
+to the build in one of three ways. By default, the build environment
+injects these flags directly into the compiler commands using Spack's
+compiler wrappers. In cases where the build system requires knowledge
+of the compiler flags, they can be registered with the build system by
+alternatively passing them through environment variables or as build
+system arguments. The flag_handler method can be used to change this
+behavior.
+
+Packages can override the flag_handler method with one of three
+built-in flag_handlers. The built-in flag_handlers are named
+``inject_flags``, ``env_flags``, and ``build_system_flags``. The
+``inject_flags`` method is the default. The ``env_flags`` method puts
+all of the flags into the environment variables that ``make`` uses as
+implicit variables ('CFLAGS', 'CXXFLAGS', etc.). The
+``build_system_flags`` method adds the flags as
+arguments to the invocation of ``configure`` or ``cmake``,
+respectively.
 
 .. warning::
 
-   The flag handling methods described in this section are in beta.
-   The exact semantics are liable to change to improve usability.
+   Passing compiler flags using build system arguments is only
+   supported for CMake and Autotools packages. Individual packages may
+   also differ in whether they properly respect these arguments.
 
-Individual packages can override the default behavior for the flag
-handling.  Packages can define a ``default_flag_handler`` method that
-applies to all sets of flags handled by Spack, or may define
-individual methods ``cflags_handler``, ``cxxflags_handler``,
-etc. Spack will apply the individual method for a flag set if it
-exists, otherwise the ``default_flag_handler`` method if it exists,
-and fall back on the default for that package class if neither exists.
+Individual packages may also define their own ``flag_handler``
+methods. The ``flag_handler`` method takes the package instance
+(``self``), the name of the flag, and a list of the values of the
+flag. It will be called on each of the six compiler flags supported in
+Spack. It should return a triple of ``(injf, envf, bsf)`` where
+``injf`` is a list of flags to inject via the Spack compiler wrappers,
+``envf`` is a list of flags to set in the appropriate environment
+variables, and ``bsf`` is a list of flags to pass to the build system
+as arguments.
 
-These methods are defined on the package class, and take two
-parameters in addition to the packages itself. The ``env`` parameter
-is an ``EnvironmentModifications`` object that can be used to change
-the build environment. The ``flag_val`` parameter is a tuple. Its
-first entry is the name of the flag (``cflags``, ``cxxflags``, etc.)
-and its second entry is a list of the values for that flag.
+.. warning::
 
-There are three primary idioms that can be combined to create whatever
-behavior the package requires.
+   Passing a non-empty list of flags to ``bsf`` for a build system
+   that does not support build system arguments will result in an
+   error.
 
-1. The default behavior for packages inheriting from
-``AutotoolsPackage`` or ``CmakePackage``.
+Here are the definitions of the three built-in flag handlers:
 
 .. code-block:: python
 
-    def default_flag_handler(self, env, flag_val):
-        env.append_flags(flag_val[0].upper(), ' '.join(flag_val[1]))
-        return []
+   def inject_flags(self, name, flags):
+       return (flags, None, None)
 
-2. The default behavior for other packages
+   def env_flags(self, name, flags):
+       return (None, flags, None)
 
-.. code-block:: python
+   def build_system_flags(self, name, flags):
+       return (None, None, flags)
 
-    def default_flag_handler(self, env, flag_val):
-        return flag_val[1]
+.. note::
 
+   Returning ``[]`` and ``None`` are equivalent in a ``flag_handler``
+   method.
 
-3. Packages may have additional flags to add to the build. These flags
-can be added to either idiom above. For example:
+Packages can override the default behavior either by specifying one of
+the built-in flag handlers,
 
 .. code-block:: python
 
-    def default_flag_handler(self, env, flag_val):
-        flags = flag_val[1]
-        flags.append('-flag')
-        return flags
-
-or
-
-.. code-block:: python
+   flag_handler = <PackageClass>.env_flags
 
-    def default_flag_handler(self, env, flag_val):
-        env.append_flags(flag_val[0].upper(), ' '.join(flag_val[1]))
-        env.append_flags(flag_val[0].upper(), '-flag')
-        return []
+where ``<PackageClass>`` can be any of the subclasses of PackageBase
+discussed in :ref:`installation_procedure`,
 
-Packages may also opt for methods that include aspects of any of the
-idioms above. E.g.
+or by implementing the flag_handler method. Suppose for a package
+``Foo`` we need to pass ``cflags``, ``cxxflags``, and ``cppflags``
+through the environment, the rest of the flags through compiler
+wrapper injection, and we need to add ``-lbar`` to ``ldlibs``. The
+following flag handler method accomplishes that.
 
 .. code-block:: python
 
-    def default_flag_handler(self, env, flag_val):
-        flags = []
-        if len(flag_val[1]) > 3:
-            env.append_flags(flag_val[0].upper(), ' '.join(flag_val[1][3:]))
-            flags = flag_val[1][:3]
-        else:
-            flags = flag_val[1]
-        flags.append('-flag')
-        return flags
+   def flag_handler(self, name, flags):
+       if name in ['cflags', 'cxxflags', 'cppflags']:
+           return (None, flags, None)
+       elif name == 'ldlibs':
+           flags.append('-lbar')
+       return (flags, None, None)
 
 Because these methods can pass values through environment variables,
-it is important not to override these variables unnecessarily in other
-package methods. In the ``setup_environment`` and
+it is important not to override these variables unnecessarily
+(E.g. setting ``env['CFLAGS']``) in other package methods when using
+non-default flag handlers. In the ``setup_environment`` and
 ``setup_dependent_environment`` methods, use the ``append_flags``
 method of the ``EnvironmentModifications`` class to append values to a
-list of flags whenever there is no good reason to override the
-existing value. In the ``install`` method and other methods that can
-operate on the build environment directly through the ``env``
-variable, test for environment variable existance before overriding
-values to add compiler flags.
+list of flags whenever the flag handler is ``env_flags``. If the
+package passes flags through the environment or the build system
+manually (in the install method, for example), we recommend using the
+default flag handler, or removind manual references and implementing a
+custom flag handler method that adds the desired flags to export as
+environment variables or pass to the build system. Manual flag passing
+is likely to interfere with the ``env_flags`` and
+``build_system_flags`` methods.
 
 In rare circumstances such as compiling and running small unit tests, a
 package developer may need to know what are the appropriate compiler