Skip to content

Commit

Permalink
Refined INSTALL text; added 'showconfig' target.
Browse files Browse the repository at this point in the history 
Details:
- Added 'showconfig' target to Makefile.
- Added header files and ./config/<configname>/make_defs.mk as prerequisites
  to object file rules.
- Added config.mk as prerequisite to library install rules.
- Edited and added to INSTALL file.
  • Loading branch information
@fgvanzee
fgvanzee committed Dec 7, 2012
1 parent 26cb659 commit 19bb507
Show file tree
Hide file tree
Showing 3 changed files with 203 additions and 70 deletions.
256 changes: 191 additions & 65 deletions INSTALL
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,20 @@ INSTALL

INTRODUCTION

This file is contains an abbreiviated description of how to configure,
compile, and install a BLIS library on your local system. It is mainly for
those of us who are impatient! PLEASE check the BLIS homepage for a wiki
page that describes the build process in much more detail.
This file contains an abbreviated description of how to configure, compile,
and install a BLIS library on your local system. Note that this file is
mainly for those of us who are impatient! If you have time, PLEASE check the
BLIS homepage for a wiki page that describes the build process in much more
detail. The BLIS homepage is located at:

http://code.google.com/p/blis/

The BLIS build system was designed for use with GNU/Linux (or some other
sane UNIX). Other requirements are:

- GNU/Linux or some other sane UNIX
- bash (2.0 or later)
- GNU make
- a working C compiler
- bash (2.0 or later)
- GNU make
- a working C compiler

We also require various other shell utilities that are so ubiquitous that
they are not worth mentioning (such as 'mv', 'mkdir', 'find', and so forth).
Expand All @@ -40,74 +42,93 @@ configuration as a template:
> ls config/x86-64opt
bl2_arch.h bl2_config.h bl2_kernel.h make_defs.mk

Now edit each of these four files. Here are some special notes about each
file:
Then you would edit each of these four files. Here are some special notes
about each file:

o bl2_arch.h
- This is where you specify architectural details such as register and
cache blocksizes.
cache blocksizes used by the micro-kernel(s).

o bl2_config.h
- This file is where you could place C preprocessor macros that would
typically be defined by autoconf/configure. It contains just a few
basic definitions such as whether to use memory alignment and, if so,
what boundary multiple to use.
typically be defined by the arguments to an autoconf-style configure
script. It contains just a few basic definitions such as whether to
use memory alignment and, if so, what boundary multiple to use.

o bl2_kernel.h
- This file defines various kernels and micro-kernels. The reference
configuration defines all kernels to use reference implementations
(which are provided as part of the framework). If you end up
writing your own optimized kernel for some operation, be sure and
set it here. Notice that you only have to set ONE definition for
each operation, as BLIS appends s,d,c,z (as well as 'bl2_') to the
names to create the actual datatype instances. IMPORTANT: If you
add your own kernels, OR if you use kernels provided with the BLIS
distribution (i.e., in the 'kernels' directory), you MUST add a
symbolic link to those kernels to the configuration sub-directory.
So, for example, if you were going to use kernels for the x86-64
architecture that are included with BLIS, you would run:
- This file defines C preprocessor macros associated with various kernels
and micro-kernels. The reference configuration defines all kernels to
use reference implementations (which are provided as part of the BLIS
framework). If you end up writing your own optimized kernel for some
operation, be sure and set it here. Notice that you only have to set
ONE definition for each operation, as BLIS prepends s,d,c,z (as well as
the BLIS function prefix, currently 'bl2_') to the names to create the
actual datatype instances.
- IMPORTANT: If you add your own kernels, OR if you use kernels provided
with the BLIS framework distribution (i.e., in the 'kernels' directory),
you MUST add a symbolic link to those kernels to the configuration
sub-directory so that 'make' can find and compile the source code. For
example, if you were going to use kernels for the x86-64 architecture
that are included with BLIS, you would run:

> pwd
/home/field/google_code/blis/config/x86-64opt
> ls
bl2_arch.h bl2_config.h bl2_kernel.h make_defs.mk
> ln -s ../../kernels/x86_64
> ls ../../kernels
x86 x86_64
> ln -s ../../kernels/x86_64 kernels
> ls bl2_arch.h bl2_config.h bl2_kernel.h kernels make_defs.mk
> ls -l kernels
lrwxrwxrwx 1 field dept 20 Dec 1 18:13 kernels -> ../../kernels/x86_64

If you are building new/custom kernels, nothing stops you from
storing your source code in an actual directory inside your
configuration sub-directory (rather than somewhere else that is then
symbolically linked to). But if you do that, it might get a little
confusing if you create *new* configurations that use the *same*
kernels. Wherever you choose to store your kernels, make sure that
it makes sense for your workflow!
If you are building new/custom kernels, nothing stops you from storing
your source code in an actual directory inside your configuration sub-
directory (rather than somewhere else that is then symbolically linked
to). But if you do that, it might get a little confusing if you create
*new* configurations that use the *same* kernels. Wherever you choose
to store your kernels, make sure that it makes sense for your workflow!

o make_defs.mk
- This file contains general make definitions. To specify things such
- This file contains general 'make' definitions. To specify things such
as your C compiler and which compiler options to use, edit this
file.
- If you use the standalone test drivers in the 'test' directory, some of
the definitions defined in make_defs.mk (i.e., those used for building
the BLIS library instantiation) will also be used when building the
test drivers. If you want to use different compilers and/or flags,
simply edit test/Makefile so that the correct definitions are used.

Once you have you configuration sub-directory in-place, you are done
Once you have your configuration sub-directory in place, you are done
configuring the framework!


STEP 2: MAKE CONFIGURATION

This step should be familiar to many in the open source software community.
Simply run:
This step should be somewhat familiar to many people in the open source
software community. Simply run:

> ./configure <configname>

where <configname> is the configuration sub-directory name you chose in
step 1. If <configname> is not given, it defaults to 'reference'.

Upon running configure, you will get output similar to the following:

configure: starting configuration of BLIS 0.1.0.
Upon running configure, you will get output similar to the following.
The exact output will depend on whether you cloned BLIS from a 'git'
repository or whether you obtained BLIS via a downloadable release
from the BLIS website.

> ./configure
configure: checking whether we need to update the version file.
configure: checking version file './version'.
configure: found .git directory; assuming git clone.
configure: executing git describe --tags.
configure: got back 0.1.3-2-g26cb659.
configure: truncating to 0.1.3-2.
configure: updating version file './version'.
configure: starting configuration of BLIS 0.1.3-2.
configure: no configuration sub-directory given; defaulting to 'reference'.
configure: no install prefix given; defaulting to /home/field/blis.
configure: no install prefix given; defaulting to '/home/field/blis'.
configure: creating ./config.mk from build/config.mk.in
configure: creating ./obj/reference
configure: creating ./obj/reference/config
Expand All @@ -127,25 +148,29 @@ them to be installed somewhere else, use the '-p <PREFIX>' option, where

> ./configure -p /some/other/path <configname>

Note that 'configure' will create a 'lib' and 'include-<id>' directory inside
<PREFIX> if they do not already exist, as well as some symbolic links for
convenient references in application makefiles. (Here, <id> is an identifier
consisting of the BLIS distribution's version number and the configuration
name you chose in step 1.)
Note that 'configure' will create a 'lib' and 'include-<versconf>' directory
inside <PREFIX> if they do not already exist, as well as some symbolic links
for convenient references in application makefiles. (Here, <versconf> is an
identifier consisting of the BLIS distribution's version number and the
configuration name you chose in step 1.)

For more help information, run 'configure' with the '-h' option:
On last note: for a complete list of supported 'configure' options and
arguments, run 'configure' with the '-h' option:

> ./configure -h

The output from this invocation of 'configure' should give you an
up-to-date list of options and their descriptions.


STEP 3: COMPILATION

Now your BLIS configuration is ready to be compiled into a library:
Once 'configure' is finished, you are ready to instantiate (compile) BLIS into
a library by running 'make':

> make

Running make will result in output similar to:
Running 'make' will result in output similar to:

Compiling frame/1/axpyv/bl2_axpyv.c
Compiling frame/1/axpyv/bl2_axpyv_check.c
Expand All @@ -156,12 +181,12 @@ Running make will result in output similar to:
Compiling frame/1/copyv/bl2_copyv.c
Compiling frame/1/copyv/bl2_copyv_check.c

If you want to see the individual compiler invocations, edit your
configuration's make_defs.mk to contain:
If you want to see the individual command line invocations of the compiler,
edit your configuration's make_defs.mk to contain:

BLIS_ENABLE_VERBOSE_MAKE_OUTPUT=yes

or, you can override that value from the command line:
or, you can temporarily override that value from the command line:

> make BLIS_ENABLE_VERBOSE_MAKE_OUTPUT=yes

Expand All @@ -170,9 +195,9 @@ via:

> make -j<n>

where <n> is the number of parallel jobs to run. You should typically limit
<n> to p+1, where p is the number of processor cores on your system.

where <n> is the number of jobs 'make' is allowed to run simultaneously.
Generally, you should typically limit <n> to p+1, where p is the number of
processor cores on your system.


STEP 4: INSTALLATION
Expand All @@ -187,37 +212,138 @@ Toward the end of compilation, you should get output similar to:
Compiling frame/base/noopt/bl2_slamch.c (NOTE: optimizations disabled)
Archiving lib/reference/libblis.a

Now you have a BLIS library sitting in 'lib/<configname>/'. To install it and
the header files associated with it, simply execute:
Now you have a BLIS library sitting in the 'lib/<configname>/' directory. To
install the library and the header files associated with it, simply execute:

> make install

This not only installs copies of the library and header files, but also
creates convenient symbolic links:

Installing libblis-0.1.0-reference.a into /home/field/blis/lib/
Installing C header files into /home/field/blis/include-0.1.0-reference
Installing libblis-0.1.3-2-reference.a into /home/field/blis/lib/
Installing C header files into /home/field/blis/include-0.1.3-2-reference
Installing symlink libblis.a into /home/field/blis/lib/
Installing symlink include into /home/field/blis/

This results in your <PREFIX> directory looking like:

> ls -l /home/field/blis
lrwxrwxrwx 1 field dept 29 Dec 6 14:19 include -> include-0.1.0-reference
drwxr-xr-x 2 field dept 32768 Dec 6 14:19 include-0.1.0-reference
lrwxrwxrwx 1 field dept 29 Dec 6 14:19 include -> include-0.1.3-2-reference
drwxr-xr-x 2 field dept 32768 Dec 6 14:19 include-0.1.3-2-reference
drwxr-xr-x 2 field dept 4096 Dec 6 14:19 lib
> ls -l /home/field/blis/lib
-rw-r--r-- 1 field dept 3919726 Dec 6 14:19 libblis-0.1.0-reference.a
lrwxrwxrwx 1 field dept 31 Dec 6 14:19 libblis.a -> libblis-0.1.0-reference.a
-rw-r--r-- 1 field dept 3919726 Dec 6 14:19 libblis-0.1.3-2-reference.a
lrwxrwxrwx 1 field dept 31 Dec 6 14:19 libblis.a -> libblis-0.1.3-2-reference.a

If you were to build a new configuration or version of BLIS and install that,
'make' would update your symbolic links automatically. This means that simply
you don't need to use unwieldy names such as 'libblis-0.1.0-reference.a'.
you don't need to use unwieldy names such as 'libblis-0.1.3-2-reference.a'.
Linking to 'libblis.a' and including '<PREFIX>/include' in your application's
makefile will always pull in the BLIS library and headers that were most
recently installed.


FURTHER DEVELOPMENT TOPCIS

-- Cleaning out build products --

If you want to remove various build products, you can use one of the 'make'
targets already defined for you in the BLIS Makefile:

> make clean
Removing .o files from ./obj/reference.
Removing .a files from ./lib/reference.

Executing the 'clean' target will remove all binary object files and library
builds from the 'obj' and 'lib' directories, respectively. Notice, though,
that only object files for the current configuration are removed. Any other
configurations' build products are left untouched.

> make cleanmk
Removing makefile fragments from ./config/reference.
Removing makefile fragments from ./frame.

The 'cleanmk' target results in removal of all makefile fragments from the
framework source tree. Makefile fragments are also removed from the
configuration sub-directory, since the build system also compiles any source
code residing there.

> make distclean
Removing makefile fragments from ./config/reference.
Removing makefile fragments from ./frame.
Removing .o files from ./obj/reference.
Removing .a files from ./lib/reference.
Removing ./config.mk.
Removing ./obj.
Removing ./lib.

Running the 'distclean' target is like saying, "Remove anything ever created
by the build system." In addition to invoking 'clean' and 'cleanmk', the
'distclean' target removes the 'config.mk' file as well as the top-level 'obj'
and 'lib' directories.

-- Switching configurations --

The BLIS build system was designed to allow a developer to be working with
multiple framework configurations simultaneously. For example, the build
system allows you to create a configuration that is oriented towards
debugging and then another similar configuration that requests a full slate
of optimizations to be applied by the compiler. Or, you might have yet another
configuration to test out a new kernel that you just wrote. All of these
configurations can co-exist together in the 'config' directory. (The BLIS
build system will NEVER delete any of these directories, not even when
executing a full clean via 'make distclean'.) And switching between these
configurations is easy; simply run './configure <configname>', where
<configname> is the configuration you want to switch to. The script will
prepare the build system for compiling for that configuration. If you are
ever unsure which configuration is "active", simply run:

> make showconfig
Current configuration is 'reference', located in './config/reference'

This will tell you which configuration is specified by the config.mk file.

Another benefit to switching configuration is that a previous configuration's
object files are saved from the previous build. For example, suppose you
configure a configuration named 'debug'. You run 'make' followed by 'make
install'. You use that debug-enabled library for a while and then you create
a configuration named 'opt', which is the same configuration in optimized
form. You compile, install, and test that library. But then you want to return
to developing with the 'debug' configuration. Simply run './configure debug'
again to switch to the 'debug' configuration. If haven't run any of the
'clean' targets in the interim, then the previously created object files for
the 'debug' configuration will still be there (inside './obj/debug'). If you
change a .c source file (not a header file; see "Caveats" section below), you
won't have to recompile the whole library. Rather, only the source files that
changed since the previous configuration and compilation will need to be
recompiled, because the object files created by the initial 'debug' build (and
their modification times) are preserved even after you switch away to another
configuration.

Now, while the object files are "remembered" from previous configurations,
the library archives (and headers) are not. So, 'make install' will always
re-install the build products after returning to a previous configuration,
even if no source files (or header files) changed. We must re-install the
libraries unconditionally after switching back to a previous configuration
because while the configuration name might be the same, the installation
prefix might have changed. Thus, we must install the build products just
in case. But the installation itself is a relatively low overhead task,
and so shouldn't be much of a burden to a developer who switches his or her
configurations.

-- Caveats --

Due to the way the BLIS framework handles header files, any change to any
header file will result in the entire library being rebuilt. This policty is
mostly out of an abundance of caution. If two or more files use definitions in
a header that is modified, and one or more of those files somehow does not get
recompiled to reflect the updated definitions, you could end up sinking hours
of time trying to track down a bug that didn't ever need to be an issue to
begin with. Thus, to prevent developers from shooting themselves in the foot
with this problem, the BLIS build system recompiles *all* object files if
a header file--any header file--is touched.


CONCLUSION

That's it! The BLIS framework's build system adheres to the familiar
Expand Down
Loading

0 comments on commit 19bb507

Please sign in to comment.