standalone flisp build

[StefanKarpinski]

Going into src/flisp and running make once built flisp but doesn't anymore. Not super crucial for Julia itself, but it seems like a good idea for this to work. We may want to pull flisp and support out into their own submodules?

[Background motivation: I'm at Hacker School and someone here wants to study a small and clean but non-toy programming language implementation. This was the best body of code I could think of looking at since it implements a fast, complete Scheme in under 8000 lines of C code.]

[JeffBezanson]

Available here: https://github.com/JeffBezanson/femtolisp

[JeffBezanson]

For the smallest version of it: https://github.com/JeffBezanson/femtolisp/blob/master/tiny/lisp.c

[JeffBezanson]

Also the flisp in julia has some julia-specific customizations.

[JeffBezanson]

Could you send the output you get, since when I do make cleanall && cd src/flisp && make it seems to work fine for me.

[sent-hil]

Hey Jeff,

sent-hil:others/ $ git clone git://github.com/JuliaLang/julia.git
...
sent-hil:julia/ (master) $ make cleanall && cd src/flisp && make
rm: /Users/sent-hil/Documents/others/julia/usr/lib/julia/extras: No such file or directory
make: [clean-/Users/sent-hil/Documents/others/julia/usr/lib/julia/extras] Error 1 (ignored)
rm: /Users/sent-hil/Documents/others/julia/usr/lib/julia/base: No such file or directory
make: [clean-/Users/sent-hil/Documents/others/julia/usr/lib/julia/base] Error 1 (ignored)
rm: /Users/sent-hil/Documents/others/julia/usr/lib/julia/ui: No such file or directory
make: [clean-/Users/sent-hil/Documents/others/julia/usr/lib/julia/ui] Error 1 (ignored)
rm: /Users/sent-hil/Documents/others/julia/usr/lib/julia/test: No such file or directory
make: [clean-/Users/sent-hil/Documents/others/julia/usr/lib/julia/test] Error 1 (ignored)
rm: /Users/sent-hil/Documents/others/julia/usr/share/julia/doc: No such file or directory
make: [clean-/Users/sent-hil/Documents/others/julia/usr/share/julia/doc] Error 1 (ignored)
rm: /Users/sent-hil/Documents/others/julia/usr/share/julia/examples: No such file or directory
make: [clean-/Users/sent-hil/Documents/others/julia/usr/share/julia/examples] Error 1 (ignored)
rm: /Users/sent-hil/Documents/others/julia/usr/lib/julia/website: No such file or directory
clang -mmacosx-version-min=10.6 -pipe -fPIC -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_DARWIN_USE_64_BIT_INODE=1 '-DJL_SYSTEM_IMAGE_PATH="../lib/julia/sys.ji"'   -c -o flisp.o flisp.c
flisp.c:47:10: fatal error: 'libsupport.h' file not found
#include "libsupport.h"
         ^
1 error generated.
make: *** [flisp.o] Error 1

I've tried cloning your libsupport repo, fixed a xcode 4.3 related error and ran it again, the make succeeds with this output:

sent-hil:flisp/ (master✗) $ make                            [21:23:06]
cc -O3 -DNDEBUG -falign-functions -momit-leaf-frame-pointer -std=gnu99 -fPIC -Wall -Wno-strict-aliasing  -c dirpath.c -o dirpath.o
clang: warning: argument unused during compilation: '-falign-functions'
dirpath.c:114:9: warning: 'FSRefMakePath' is deprecated: first
      deprecated in Mac OS X 10.8 [-Wdeprecated-declarations]
        FSRefMakePath(&ref, (uint8_t*)buf, size) < 0)
        ^
/System/Library/Frameworks/CoreServices.framework/Frameworks/CarbonCore.framework/Headers/Files.h:4081:18: note: 
      'FSRefMakePath' declared here
extern OSStatus  FSRefMakePath(const FSRef *ref, UInt8 *path, ...
                 ^
1 warning generated.
cc -O3 -DNDEBUG -falign-functions -momit-leaf-frame-pointer -std=gnu99 -fPIC -Wall -Wno-strict-aliasing  -c htable.c -o htable.o
clang: warning: argument unused during compilation: '-falign-functions'
cc -O3 -DNDEBUG -falign-functions -momit-leaf-frame-pointer -std=gnu99 -fPIC -Wall -Wno-strict-aliasing  -c bitvector.c -o bitvector.o
clang: warning: argument unused during compilation: '-falign-functions'
cc -O3 -DNDEBUG -falign-functions -momit-leaf-frame-pointer -std=gnu99 -fPIC -Wall -Wno-strict-aliasing  -c bitvector-ops.c -o bitvector-ops.o
clang: warning: argument unused during compilation: '-falign-functions'
cc -O3 -DNDEBUG -falign-functions -momit-leaf-frame-pointer -std=gnu99 -fPIC -Wall -Wno-strict-aliasing  -c int2str.c -o int2str.o
clang: warning: argument unused during compilation: '-falign-functions'
cc -O3 -DNDEBUG -falign-functions -momit-leaf-frame-pointer -std=gnu99 -fPIC -Wall -Wno-strict-aliasing  -c dump.c -o dump.o
clang: warning: argument unused during compilation: '-falign-functions'
cc -O3 -DNDEBUG -falign-functions -momit-leaf-frame-pointer -std=gnu99 -fPIC -Wall -Wno-strict-aliasing  -c libsupportinit.c -o libsupportinit.o
clang: warning: argument unused during compilation: '-falign-functions'
cc -O3 -DNDEBUG -falign-functions -momit-leaf-frame-pointer -std=gnu99 -fPIC -Wall -Wno-strict-aliasing  -c arraylist.c -o arraylist.o
clang: warning: argument unused during compilation: '-falign-functions'
rm -rf libsupport.a
ar -rcs libsupport.a hashing.o timefuncs.o ptrhash.o socket.o utf8.o ios.o dirpath.o htable.o bitvector.o bitvector-ops.o int2str.o dump.o libsupportinit.o arraylist.o

But I don't see a flisp executable. This is what the ls produces:

sent-hil:flisp/ (master✗) $ ls                              [21:23:11]
Makefile           equalhash.h        libsupportinit.o
MurmurHash3.c      extra              mkboot0.lsp
MurmurHash3.h      flisp.boot         mkboot1.lsp
README.md          flisp.c            opcodes.h
aliases.scm        flisp.h            print.c
arraylist.c        flmain.c           ptrhash.c
arraylist.h        hashing.c          ptrhash.h
arraylist.o        hashing.h          ptrhash.o
bitvector-ops.c    hashing.o          read.c
bitvector-ops.o    htable.c           socket.c
bitvector.c        htable.h           socket.h
bitvector.h        htable.inc         socket.o
bitvector.o        htable.o           string.c
bootstrap.sh       htableh.inc        system.lsp
builtins.c         ieee754.h          table.c
color.lsp          int2str.c          timefuncs.c
compiler.lsp       int2str.o          timefuncs.h
cvalues.c          ios.c              timefuncs.o
dirpath.c          ios.h              types.c
dirpath.h          ios.o              unittest.lsp
dirpath.o          iostream.c         utf8.c
dtypes.h           julia_extensions.c utf8.h
dump.c             libsupport         utf8.o
dump.o             libsupport.a       utils.h
equal.c            libsupport.h
equalhash.c        libsupportinit.c

The tiny/lisp.c looks very interesting, will be sinking my teeth into it tomorrow. Thanks!

[JeffBezanson]

Hmm, strange. src/flisp/Makefile:19 specifies flags for the correct include directory, but these flags aren't being used (they don't show up on the compile line for flisp.c). Is this GNU make?

[sent-hil]

Yep, came with the OS.

sent-hil:~/ $ make -v                                       [21:54:11]
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i386-apple-darwin11.3.0

[JeffBezanson]

Something must be different, perhaps in your Make.inc. I notice that QUIET_CC doesn't seem to be in effect either. That directory listing is also strange; it looks like the files of libsupport copied into the flisp directory, possibly overwriting the makefile?

[sent-hil]

Oh right sorry for not mentioning it, I did have to copy over the files from libsupport/* into src/flisp to fix fatal error: 'libsupport.h' file not found. I'm very new to C, do I've to compile libsupport first and then move the files to src/flisp, or does compiling flisp take care of that?

[JeffBezanson]

Actually neither the standalone femtolisp repo nor flisp in julia require cloning libsupport or copying any files; it is supposed to just work but the FLAGS in the makefile are not taking effect for some reason.

[JeffBezanson]

Have you tried cloning https://github.com/JeffBezanson/femtolisp? At least the build there should be simpler than julia's.

[sent-hil]

I just did. More errors :(

sent-hil:others/ $ git clone git://github.com/JeffBezanson/femtolisp.git
...
sent-hil:others/ $ cd femtolisp                             [22:26:20]
sent-hil:femtolisp/ (master) $ make                         [22:26:22]
gcc -O2 -DNDEBUG -falign-functions -Wall -Wno-strict-aliasing -Illt  -DUSE_COMPUTED_GOTO -DLINUX -DARCH_X86_64 -DBITS64 -D__CPU__=686 -c flisp.c -o flisp.o
In file included from llt/llt.h:5,
                 from flisp.c:45:
llt/dtypes.h:32:22: error: features.h: No such file or directory
llt/dtypes.h:33:20: error: endian.h: No such file or directory
llt/dtypes.h:34:1: warning: "LITTLE_ENDIAN" redefined
In file included from /usr/include/machine/endian.h:35,
                 from /usr/include/sys/wait.h:193,
                 from /usr/include/stdlib.h:65,
                 from flisp.c:32:
/usr/include/i386/endian.h:93:1: warning: this is the location of the previous definition
In file included from llt/llt.h:5,
                 from flisp.c:45:
llt/dtypes.h:35:1: warning: "BIG_ENDIAN" redefined
In file included from /usr/include/machine/endian.h:35,
                 from /usr/include/sys/wait.h:193,
                 from /usr/include/stdlib.h:65,
                 from flisp.c:32:
/usr/include/i386/endian.h:94:1: warning: this is the location of the previous definition
In file included from llt/llt.h:5,
                 from flisp.c:45:
llt/dtypes.h:36:1: warning: "PDP_ENDIAN" redefined
In file included from /usr/include/machine/endian.h:35,
                 from /usr/include/sys/wait.h:193,
                 from /usr/include/stdlib.h:65,
                 from flisp.c:32:
/usr/include/i386/endian.h:95:1: warning: this is the location of the previous definition
In file included from llt/llt.h:5,
                 from flisp.c:45:
llt/dtypes.h:37:1: warning: "BYTE_ORDER" redefined
In file included from /usr/include/machine/endian.h:35,
                 from /usr/include/sys/wait.h:193,
                 from /usr/include/stdlib.h:65,
                 from flisp.c:32:
/usr/include/i386/endian.h:97:1: warning: this is the location of the previous definition
In file included from operators.c:5,
                 from cvalues.c:1,
                 from flisp.c:390:
llt/ieee754.h:14: error: duplicate member ‘mantissa’
llt/ieee754.h:15: error: duplicate member ‘exponent’
llt/ieee754.h:15: error: duplicate member ‘({anonymous})’
llt/ieee754.h:16: error: duplicate member ‘negative’
llt/ieee754.h:16: error: duplicate member ‘({anonymous})’
llt/ieee754.h:16: error: duplicate member ‘({anonymous})’
llt/ieee754.h:34: error: duplicate member ‘mantissa1’
llt/ieee754.h:35: error: duplicate member ‘mantissa0’
llt/ieee754.h:35: error: duplicate member ‘({anonymous})’
llt/ieee754.h:36: error: duplicate member ‘exponent’
llt/ieee754.h:36: error: duplicate member ‘({anonymous})’
llt/ieee754.h:36: error: duplicate member ‘({anonymous})’
llt/ieee754.h:37: error: duplicate member ‘negative’
llt/ieee754.h:37: error: duplicate member ‘({anonymous})’
llt/ieee754.h:37: error: duplicate member ‘({anonymous})’
llt/ieee754.h:37: error: duplicate member ‘({anonymous})’
llt/ieee754.h:56: error: duplicate member ‘mantissa1’
llt/ieee754.h:57: error: duplicate member ‘mantissa0’
llt/ieee754.h:57: error: duplicate member ‘({anonymous})’
llt/ieee754.h:58: error: duplicate member ‘exponent’
llt/ieee754.h:58: error: duplicate member ‘({anonymous})’
llt/ieee754.h:58: error: duplicate member ‘({anonymous})’
llt/ieee754.h:59: error: duplicate member ‘negative’
llt/ieee754.h:59: error: duplicate member ‘({anonymous})’
llt/ieee754.h:59: error: duplicate member ‘({anonymous})’
llt/ieee754.h:59: error: duplicate member ‘({anonymous})’
llt/ieee754.h:60: error: duplicate member ‘empty’
llt/ieee754.h:60: error: duplicate member ‘({anonymous})’
llt/ieee754.h:60: error: duplicate member ‘({anonymous})’
llt/ieee754.h:60: error: duplicate member ‘({anonymous})’
llt/ieee754.h:60: error: duplicate member ‘({anonymous})’
make: *** [flisp.o] Error 1

I noticed there was a Makefile.macosx. I tried that as well:

sent-hil:femtolisp/ (master) $ mv Makefile m                [22:28:18]
sent-hil:femtolisp/ (master✗) $ mv Makefile.macosx Makefile [22:28:25]
sent-hil:femtolisp/ (master✗) $                             [22:28:30]
sent-hil:femtolisp/ (master✗) $ make                        [22:28:30]
gcc -O2 -DNDEBUG -falign-functions -Wall -Wno-strict-aliasing -Illt  -DUSE_COMPUTED_GOTO -DMACOSX -DARCH_X86_64 -DBITS64 -D__CPU__=686 -c flisp.c -o flisp.o
gcc -O2 -DNDEBUG -falign-functions -Wall -Wno-strict-aliasing -Illt  -DUSE_COMPUTED_GOTO -DMACOSX -DARCH_X86_64 -DBITS64 -D__CPU__=686 -c builtins.c -o builtins.o
gcc -O2 -DNDEBUG -falign-functions -Wall -Wno-strict-aliasing -Illt  -DUSE_COMPUTED_GOTO -DMACOSX -DARCH_X86_64 -DBITS64 -D__CPU__=686 -c string.c -o string.o
gcc -O2 -DNDEBUG -falign-functions -Wall -Wno-strict-aliasing -Illt  -DUSE_COMPUTED_GOTO -DMACOSX -DARCH_X86_64 -DBITS64 -D__CPU__=686 -c equalhash.c -o equalhash.o
gcc -O2 -DNDEBUG -falign-functions -Wall -Wno-strict-aliasing -Illt  -DUSE_COMPUTED_GOTO -DMACOSX -DARCH_X86_64 -DBITS64 -D__CPU__=686 -c table.c -o table.o
gcc -O2 -DNDEBUG -falign-functions -Wall -Wno-strict-aliasing -Illt  -DUSE_COMPUTED_GOTO -DMACOSX -DARCH_X86_64 -DBITS64 -D__CPU__=686 -c iostream.c -o iostream.o
cd llt && make
gcc -O3 -DNDEBUG -Wall -Wno-strict-aliasing  -DLINUX -DARCH_X86_64 -DBITS64 -D__CPU__=686 -c bitvector.c -o bitvector.o
In file included from bitvector.c:36:
dtypes.h:32:22: error: features.h: No such file or directory
dtypes.h:33:20: error: endian.h: No such file or directory
dtypes.h:34:1: warning: "LITTLE_ENDIAN" redefined
In file included from /usr/include/machine/endian.h:35,
                 from /usr/include/sys/wait.h:193,
                 from /usr/include/stdlib.h:65,
                 from bitvector.c:32:
/usr/include/i386/endian.h:93:1: warning: this is the location of the previous definition
In file included from bitvector.c:36:
dtypes.h:35:1: warning: "BIG_ENDIAN" redefined
In file included from /usr/include/machine/endian.h:35,
                 from /usr/include/sys/wait.h:193,
                 from /usr/include/stdlib.h:65,
                 from bitvector.c:32:
/usr/include/i386/endian.h:94:1: warning: this is the location of the previous definition
In file included from bitvector.c:36:
dtypes.h:36:1: warning: "PDP_ENDIAN" redefined
In file included from /usr/include/machine/endian.h:35,
                 from /usr/include/sys/wait.h:193,
                 from /usr/include/stdlib.h:65,
                 from bitvector.c:32:
/usr/include/i386/endian.h:95:1: warning: this is the location of the previous definition
In file included from bitvector.c:36:
dtypes.h:37:1: warning: "BYTE_ORDER" redefined
In file included from /usr/include/machine/endian.h:35,
                 from /usr/include/sys/wait.h:193,
                 from /usr/include/stdlib.h:65,
                 from bitvector.c:32:
/usr/include/i386/endian.h:97:1: warning: this is the location of the previous definition
make[1]: *** [bitvector.o] Error 1
make: *** [llt/libllt.a] Error 2

Just to be clear, I run make with no args right?

[JeffBezanson]

Ok, just pushed a fix to the macos makefiles to the femtolisp repo. You can use the mac makefile without renaming it using make -f Makefile.macosx.

[sent-hil]

Got it working, thanks!

[sent-hil]

On a different note, having trouble getting the tiny lisp up and running.

sent-hil:tiny/ (master✗) $ ls                               [22:50:33]
Makefile       evalt          lisp.c         lispf.c
eval1          flutils.c      lisp2.c        scrap.c
eval2          lisp-nontail.c lisp2.c.bak    system.lsp
sent-hil:tiny/ (master✗) $ make                             [22:50:38]
gcc -O3 -fomit-frame-pointer -Wall -Wextra lisp.c -o lisp 
lisp.c: In function ‘tocons’:
lisp.c:131: warning: cast to pointer from integer of different size
... more warnings
sent-hil:tiny/ (master✗) $ ./lisp                           [22:50:41]
[1]    4567 segmentation fault  ./lisp

Ran it in gdb.

sent-hil:tiny/ (master✗) $ rm -rf lisp                      [22:53:19]
sent-hil:tiny/ (master✗) $ gcc -g lisp.c -o lisp            [22:53:21]
lisp.c: In function ‘tocons’:
...warnings
(gdb) break main
Breakpoint 1 at 0x1000061c1: file lisp.c, line 1005.
(gdb) run
Starting program: /Users/sent-hil/Documents/others/femtolisp/tiny/lisp 
Reading symbols for shared libraries +............................. done

Breakpoint 1, main (argc=1, argv=0x7fff5fbff270) at lisp.c:1005
1005        stack_bottom = ((char*)&v) - PROCESS_STACK_SIZE;
(gdb) n
1006        lisp_init();
(gdb) s
lisp_init () at lisp.c:188
188     fromspace = malloc(heapsize);
(gdb) n
189     tospace   = malloc(heapsize);
(gdb) n
190     curheap = fromspace;
(gdb) n
191     lim = curheap+heapsize-sizeof(cons_t);
(gdb) n
193     NIL = symbol("nil"); setc(NIL, NIL);
(gdb) n