-
Notifications
You must be signed in to change notification settings - Fork 67
Description
There have been a number of discussions about replacing the build system, but I can't seem to find an issue for it. (Apologies for the length; deficiencies are probably overly-detailed after I've encountered scepticism about whether the current build system in fact even has any major problems.)
The problem
The build system is decrepit and needs replacing. The problems of the build system can be explained in the context of its origins.
- Began with the original, plain
Makefilebuild system with Linux Kbuild integration$EMC_HOMEenvironment variable configures both build and run-time systems- This resembles other CAD and EDA packages from the '90s (TCL was also a favorite of the era)
- Each type of build artifact must be dealt with manually
- Introducing a new type of artifacts requires writing new rules
- No out-of-the-box macros or scripts, like in high-level build systems
- Additional complexity to integrate with the Linux Kbuild build system for building
.komodules becomes an issue later - "Shotgun approach" to includes: Simply include every subdirectory with
gcc -Iin the%.o: %.c:target!
- GNU Autoconf added later to ease build parameter configuration
$EMC_HOMEconflated with$prefix, but semantic differences- Makes it impossible to
./configure --prefix=/usr(the default in normal Autotools build systems), run tests or run the application, and thenmake installfor a system install - Instead, need separate "RIP" build to run locally;
$prefixartificially set to$EMC_HOME, the source directory by default- Setting
$prefixinsideconfigure.acintroduces subtle problems, e.g. requires crazy hacks to template files that include variables based on$prefix
- Setting
- Makes it impossible to
- Automake support was not built in at the same time
- Perhaps because it's difficult to integrate with Kbuild, or perhaps just to avoid a lot of rework with
$EMC_HOME - The build rules and
libtoolsupport are another ingredient missing to run out of the source directory and thenmake installto/usrwithout rebuilding from zero
- Perhaps because it's difficult to integrate with Kbuild, or perhaps just to avoid a lot of rework with
- As long as a separate RIP build is required to run from the source tree, tests cannot be run during the package build
- (Somewhere in there)
pthreads"sim" build was introduced- The "sim" build is the granddaddy of today's RT_PREEMPT, etc. user-space threads
- Build system now has to build userland
.soplugins- Userland threads share module sources but build them differently
- The Kbuild integration became even more complex
- New build rules and new build configuration settings
- The "unified build" came in along with Machinekit (an atrocity committed by YT)
- Introduced a disgusting recursive
make modulesfor each configured thread flavor - Needed crazy hacks to build differently configured
.ofiles from the same.cfiles to integrate with the "sim" module build rules - Introduced some build dependency bugs that were never resolved
- Introduced a disgusting recursive
- All along the way, dozens of contributors added their own rules
- Trivial changes to add a new module with no external dependencies are easy to add
- But non-trivial changes take special knowledge to integrate cleanly
- How to integrate new rules with Kbuild?
src/subdir/Submakefileorsrc/Makefile? - Which
CFLAGSorLDFLAGSvariable to use?- Userspace or Kbuild? Where are all those
*FLAGSvariables documented? - Solution: Make a new variable!
- Userspace or Kbuild? Where are all those
- Where do build artifacts for new types of builds go?
- Normally, build artifacts go in the
src/objectsdirectory - But sometimes it's just easier to put them next to the sources!
- Normally, build artifacts go in the
- Where do the artifacts' dependencies get plugged in?
- Worst case: shotgun approach (cf.
make clean; make threads=posix objects/posix/hal/components/encoder.o)
- Worst case: shotgun approach (cf.
- How to integrate new rules with Kbuild?
- As a result, the build system's complexity and disorganization only leads to increasing disorganization over time
The end result, the build system of today, is very difficult to modify. It's difficult to understand the structure of dependencies, files and variables in order to know how to introduce the change at all, much less where to add it so that it doesn't make the structure even more disorganized and hard to understand for the next person.
The only way to fix it at this point is by replacing it. Fortunately, this will be easier today than ever before. The split machinekit-hal repository reduces the size of the task. The upcoming per-flavor module merge will simplify build rules. Eliminating kernel threads also eliminates the requirement to support kbuild. And today there are two options that are already 50% complete.
Possible build system replacements
Among alternatives, these make the best sense, given the enormous head start, and given that both are widely-used and well-supported in general, and that the Machinekit community has more or less experience with both.
- CMake:
- Several top Machinekit contributors have experience already
- There have been a few attempts already that would get us most of the way
- @bobvanderlinden's branch
- @kinsamanka's branch
- @ArcEye's branch
- my branch (probably nothing useful)
- Needs selecting the most complete one, pulling from others where possible, and finishing the configuration
- Autotools:
- Machinekit community already familiar with Autoconf portion
- The existing Autoconf
configure.acgets us halfway to a full Autotools configuration - Needs developing the Automake configuration; starting point is zero, AFAIK