Tutorial: Speeding up Nelua compilation

Here are some tips to speedup compilation.

Debugging compilation time

First learn how to debug Nelua compilation time with --timing or in short -t. Let's first establish a baseline of time to compile tests/all_test.nelua, this is a big file with contains tests for all the Nelua standard library, it contains hundreds of code lines:

$ nelua -btC tests/all_test.nelua
startup      30.2 ms
parse        114.3 ms
preprocess   39.0 ms
analyze      795.7 ms
generate     360.8 ms
compile      5422.6 ms
total build  6762.8 ms

Looks like it took about 6762 milliseconds to compile this file, we will use this as our baseline and further optimize using some tips. The extra flag -C is used to ignore caches so we always recompile everything in our measurements, and -b is used to only compile the binary (don't run).

Using optimized Lua interpreter

The Nelua compiler is built and runs with a Lua interpreter, and after cloning Nelua you usually compile its Lua interpreter using make, by default it will compile with a general optimization flags to work on most systems. However you can compile it optimized your system using make optimized-nelua-lua, this should speedup the compiler in general, though this will only optimize Nelua compiler, not the C compiler, so lets look at time to generate the C file this time (using -c instead of -b)

$ nelua -tcC tests/all_test.nelua
startup      30.5 ms
parse        116.8 ms
preprocess   40.1 ms
analyze      807.6 ms
generate     373.9 ms
total build  1376.7 ms

$ make optimized-nelua-lua

$ nelua -tcC tests/all_test.nelua
startup      28.7 ms
parse        98.4 ms
preprocess   34.2 ms
analyze      726.3 ms
generate     330.2 ms
total build  1224.7 m

Looks like we generated the code in about 90% of the time, good!

Note that this may not work on all systems, make optimized-nelua-lua requires GCC and will use profile guided optimization to optimize branch predictions.

Using other C compiler

By default we are using the system's C compiler, which is probably GCC, we could use a faster C compiler like TinyCC to speed up compile time:

$ nelua -btC --cc=gcc tests/all_test.nelua
startup      28.0 ms
parse        97.0 ms
preprocess   36.3 ms
analyze      731.5 ms
generate     336.3 ms
compile      5426.9 ms
total build  6656.0 ms

$ nelua -btC --cc=tcc tests/all_test.nelua
startup      23.1 ms
parse        97.8 ms
preprocess   35.1 ms
analyze      739.4 ms
generate     330.7 ms
compile      67.6 ms
total build  1293.7 ms

As you can notice, TCC is much faster to compile, nice for quick development, we are already below 19% from the baseline in time to compile (from ~6.8s to ~1.3s). But TCC is not ideal when you want to debug your code, if you are using --debug or --sanitize flags then please stick to GCC or Clang.

Compiling single header libraries

If you are using bindings from nelua-decl you will notice that some are single header C libraries, while they are easy to use they can slowdown compile-time by a good amount, even though you never edit them, so it's best to precompile them to avoid redundant work from the C compiler, let's do this with the minilua library for the minilua-test.nelua example:

$ nelua -btC minilua-test.nelua
startup      28.5 ms
parse        5.9 ms
preprocess   2.4 ms
analyze      20.6 ms
generate     0.7 ms
compile      1545.1 ms
total build  1603.4 ms

$ gcc -c -o minilua.o -x c minilua.h -DLUA_IMPL -O2
$ nelua -btC --cflags=minilua.o -DMINILUA_NO_IMPL minilua-test.nelua
startup      28.6 ms
parse        7.7 ms
preprocess   4.1 ms
analyze      18.6 ms
generate     0.8 ms
compile      65.0 ms
total build  124.9 ms

Here GCC was used, but notice that it compiled much faster after we compiled the minilua.h into the object file minilua.o. You could use some build system to precompile C files for you like a plain Makefile, or even make a minor build system in the Nelua preprocessor.

To simplify the setup you could also move the flags --cflags=minilua.o -DMINILUA_NO_IMPL to your source file before require 'minilua':

## MINILUA_NO_IMPL = true
## cfile 'minilua.o'
require 'minilua'