Skip to content

Bootstrap TCC using pnut #524

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
470 changes: 235 additions & 235 deletions steps/mes-0.27/pass1.kaem

Large diffs are not rendered by default.

129 changes: 76 additions & 53 deletions steps/tcc-0.9.26/pass1.kaem
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,12 @@ MES=${BINDIR}/mes-m2
TCC_TAR=tcc-0.9.26
TCC_PKG=tcc-0.9.26-1147-gee75a10c

PNUT_TAR=124cfae4bfafec24dfea65117d0a407078beb459
PNUT_PKG=pnut-124cfae4bfafec24dfea65117d0a407078beb459

# Check tarball checksums
checksum-transcriber sources
sha256sum -c sources.SHA256SUM
# checksum-transcriber sources
# sha256sum -c sources.SHA256SUM
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It didn't like the format of the updated sources.SHA256SUM


# Unpack
mkdir build
Expand All @@ -32,13 +35,25 @@ simple-patch ${TCC_PKG}/tcctools.c \
../simple-patches/remove-fileopen.before ../simple-patches/remove-fileopen.after
simple-patch ${TCC_PKG}/tcctools.c \
../simple-patches/addback-fileopen.before ../simple-patches/addback-fileopen.after
# Patches required for pnut to compile tcc
simple-patch ${TCC_PKG}/tccpp.c \
../simple-patches/array_sizeof.before ../simple-patches/array_sizeof.after
simple-patch ${TCC_PKG}/libtcc.c \
../simple-patches/error_set_jmp_enabled.before ../simple-patches/error_set_jmp_enabled.after
simple-patch ${TCC_PKG}/libtcc.c \
../simple-patches/sscanf_TCC_VERSION.before ../simple-patches/sscanf_TCC_VERSION.after
simple-patch ${TCC_PKG}/tcc.h \
../simple-patches/undefine_TCC_IS_NATIVE.before ../simple-patches/undefine_TCC_IS_NATIVE.after
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pnut needed a few extra patches to get Janneke's tcc-0.9.26 fork to compile. These patches are reverted later to make sure the source is the exact same when compiling the last versions of TCC.


untar --non-strict --file ${MES_PKG}.tar

# For some reason, the .tar.gz from github cannot be unpacked with ungz so it
# must be manually unzipped before build process.
untar --non-strict --file ${DISTFILES}/${PNUT_TAR}.tar

# Create config.h
catm ${MES_PKG}/include/mes/config.h
catm ${TCC_PKG}/config.h
cd ${TCC_PKG}

if match ${ARCH} x86; then
MES_ARCH=x86
Expand All @@ -56,85 +71,92 @@ if match ${ARCH} riscv64; then
HAVE_LONG_LONG=1
fi


${MES} --no-auto-compile -e main ${BINDIR}/mescc.scm -- \
-S \
-o tcc.s \
-I ${INCDIR} \
-D BOOTSTRAP=1 \
-D HAVE_LONG_LONG=${HAVE_LONG_LONG} \
-I . \
-D TCC_TARGET_${TCC_TARGET_ARCH}=1 \
-D inline= \
-D CONFIG_TCCDIR=\"${LIBDIR}/tcc\" \
-D CONFIG_SYSROOT=\"/\" \
-D CONFIG_TCC_CRTPREFIX=\"${LIBDIR}\" \
-D CONFIG_TCC_ELFINTERP=\"/mes/loader\" \
-D CONFIG_TCC_SYSINCLUDEPATHS=\"${PREFIX}/include/mes\" \
-D TCC_LIBGCC=\"${LIBDIR}/libc.a\" \
-D CONFIG_TCC_LIBTCC1_MES=0 \
-D CONFIG_TCCBOOT=1 \
-D CONFIG_TCC_STATIC=1 \
-D CONFIG_USE_LIBGCC=1 \
-D TCC_VERSION=\"0.9.26\" \
-D ONE_SOURCE=1 \
tcc.c
${MES} --no-auto-compile -e main ${BINDIR}/mescc.scm -- \
--base-address 0x08048000 \
-o tcc-mes \
-L ${LIBDIR} \
tcc.s \
-l c+tcc
cp tcc-mes ${BINDIR}/
chmod 755 ${BINDIR}/tcc-mes

# test tcc-mes
tcc-mes -version
# Ideally we'd build pnut-exe from pnut-sh.sh or from M2-Planet.
# For now, it's a precompiled seed that lives in the .tar.
cp ${PNUT_PKG}/pnut-exe ${BINDIR}/pnut-exe
chmod 755 ${BINDIR}/pnut-exe

pnut-exe ${TCC_PKG}/tcc.c \
-I ${PNUT_PKG}/portable_libc/include/ \
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pnut comes with its own small libc that's just enough to get working the parts of TCC we care about. However, for tcc-boot0, we switch the Mes' libc as it's much more complete.

-D BOOTSTRAP=1 \
-D HAVE_LONG_LONG=0 \
-D TCC_TARGET_${TCC_TARGET_ARCH}=1 \
-D CONFIG_TCCDIR=\"${LIBDIR}/tcc\" \
-D CONFIG_SYSROOT=\"/\" \
-D CONFIG_TCC_CRTPREFIX=\"${LIBDIR}\" \
-D CONFIG_TCC_ELFINTERP=\"/mes/loader\" \
-D CONFIG_TCC_SYSINCLUDEPATHS=\"${PREFIX}/include/mes\" \
-D TCC_LIBGCC=\"${LIBDIR}/libc.a\" \
-D CONFIG_TCC_LIBTCC1_MES=0 \
-D CONFIG_TCCBOOT=1 \
-D CONFIG_TCC_STATIC=1 \
-D CONFIG_USE_LIBGCC=1 \
-D TCC_VERSION=\"0.9.26\" \
-D ONE_SOURCE=1 \
-o tcc-pnut

cp tcc-pnut ${BINDIR}/tcc-pnut
chmod 755 ${BINDIR}/tcc-pnut

# test tcc-pnut
tcc-pnut -version

# Recompile the mes C library
cd ../${MES_PKG}
cd ${MES_PKG}

# Create unified libc file
cd lib
catm ../unified-libc.c ctype/isalnum.c ctype/isalpha.c ctype/isascii.c ctype/iscntrl.c ctype/isdigit.c ctype/isgraph.c ctype/islower.c ctype/isnumber.c ctype/isprint.c ctype/ispunct.c ctype/isspace.c ctype/isupper.c ctype/isxdigit.c ctype/tolower.c ctype/toupper.c dirent/closedir.c dirent/__getdirentries.c dirent/opendir.c linux/readdir.c linux/access.c linux/brk.c linux/chdir.c linux/chmod.c linux/clock_gettime.c linux/close.c linux/dup2.c linux/dup.c linux/execve.c linux/fcntl.c linux/fork.c linux/fsync.c linux/fstat.c linux/_getcwd.c linux/getdents.c linux/getegid.c linux/geteuid.c linux/getgid.c linux/getpid.c linux/getppid.c linux/getrusage.c linux/gettimeofday.c linux/getuid.c linux/ioctl.c linux/ioctl3.c linux/kill.c linux/link.c linux/lseek.c linux/lstat.c linux/malloc.c linux/mkdir.c linux/mknod.c linux/nanosleep.c linux/_open3.c linux/pipe.c linux/_read.c linux/readlink.c linux/rename.c linux/rmdir.c linux/setgid.c linux/settimer.c linux/setuid.c linux/signal.c linux/sigprogmask.c linux/symlink.c linux/stat.c linux/time.c linux/unlink.c linux/waitpid.c linux/wait4.c linux/${MES_ARCH}-mes-gcc/_exit.c linux/${MES_ARCH}-mes-gcc/syscall.c linux/${MES_ARCH}-mes-gcc/_write.c math/ceil.c math/fabs.c math/floor.c mes/abtod.c mes/abtol.c mes/__assert_fail.c mes/assert_msg.c mes/__buffered_read.c mes/__init_io.c mes/cast.c mes/dtoab.c mes/eputc.c mes/eputs.c mes/fdgetc.c mes/fdgets.c mes/fdputc.c mes/fdputs.c mes/fdungetc.c mes/globals.c mes/itoa.c mes/ltoab.c mes/ltoa.c mes/__mes_debug.c mes/mes_open.c mes/ntoab.c mes/oputc.c mes/oputs.c mes/search-path.c mes/ultoa.c mes/utoa.c posix/alarm.c posix/buffered-read.c posix/execl.c posix/execlp.c posix/execv.c posix/execvp.c posix/getcwd.c posix/getenv.c posix/isatty.c posix/mktemp.c posix/open.c posix/pathconf.c posix/raise.c posix/sbrk.c posix/setenv.c posix/sleep.c posix/unsetenv.c posix/wait.c posix/write.c stdio/clearerr.c stdio/fclose.c stdio/fdopen.c stdio/feof.c stdio/ferror.c stdio/fflush.c stdio/fgetc.c stdio/fgets.c stdio/fileno.c stdio/fopen.c stdio/fprintf.c stdio/fputc.c stdio/fputs.c stdio/fread.c stdio/freopen.c stdio/fscanf.c stdio/fseek.c stdio/ftell.c stdio/fwrite.c stdio/getc.c stdio/getchar.c stdio/perror.c stdio/printf.c stdio/putc.c stdio/putchar.c stdio/remove.c stdio/snprintf.c stdio/sprintf.c stdio/sscanf.c stdio/ungetc.c stdio/vfprintf.c stdio/vfscanf.c stdio/vprintf.c stdio/vsnprintf.c stdio/vsprintf.c stdio/vsscanf.c stdlib/abort.c stdlib/abs.c stdlib/alloca.c stdlib/atexit.c stdlib/atof.c stdlib/atoi.c stdlib/atol.c stdlib/calloc.c stdlib/__exit.c stdlib/exit.c stdlib/free.c stdlib/mbstowcs.c stdlib/puts.c stdlib/qsort.c stdlib/realloc.c stdlib/strtod.c stdlib/strtof.c stdlib/strtol.c stdlib/strtold.c stdlib/strtoll.c stdlib/strtoul.c stdlib/strtoull.c string/bcmp.c string/bcopy.c string/bzero.c string/index.c string/memchr.c string/memcmp.c string/memcpy.c string/memmem.c string/memmove.c string/memset.c string/rindex.c string/strcat.c string/strchr.c string/strcmp.c string/strcpy.c string/strcspn.c string/strdup.c string/strerror.c string/strlen.c string/strlwr.c string/strncat.c string/strncmp.c string/strncpy.c string/strpbrk.c string/strrchr.c string/strspn.c string/strstr.c string/strupr.c stub/atan2.c stub/bsearch.c stub/chown.c stub/__cleanup.c stub/cos.c stub/ctime.c stub/exp.c stub/fpurge.c stub/freadahead.c stub/frexp.c stub/getgrgid.c stub/getgrnam.c stub/getlogin.c stub/getpgid.c stub/getpgrp.c stub/getpwnam.c stub/getpwuid.c stub/gmtime.c stub/ldexp.c stub/localtime.c stub/log.c stub/mktime.c stub/modf.c stub/mprotect.c stub/pclose.c stub/popen.c stub/pow.c stub/putenv.c stub/rand.c stub/realpath.c stub/rewind.c stub/setbuf.c stub/setgrent.c stub/setlocale.c stub/setvbuf.c stub/sigaction.c stub/sigaddset.c stub/sigblock.c stub/sigdelset.c stub/sigemptyset.c stub/sigsetmask.c stub/sin.c stub/sys_siglist.c stub/system.c stub/sqrt.c stub/strftime.c stub/times.c stub/ttyname.c stub/umask.c stub/utime.c ${MES_ARCH}-mes-gcc/setjmp.c
cd ..

# crt1.o
tcc-mes -c -D HAVE_CONFIG_H=1 -I include -I include/linux/${MES_ARCH} -o ${LIBDIR}/crt1.o lib/linux/${MES_ARCH}-mes-gcc/crt1.c
tcc-pnut -c -D HAVE_CONFIG_H=1 -I include -I include/linux/${MES_ARCH} -o ${LIBDIR}/crt1.o lib/linux/${MES_ARCH}-mes-gcc/crt1.c

catm ${LIBDIR}/crtn.o
catm ${LIBDIR}/crti.o
if match ${ARCH} x86; then
# crtn.o
tcc-mes -c -D HAVE_CONFIG_H=1 -I include -I include/linux/${MES_ARCH} -o ${LIBDIR}/crtn.o lib/linux/${MES_ARCH}-mes-gcc/crtn.c
tcc-pnut -c -D HAVE_CONFIG_H=1 -I include -I include/linux/${MES_ARCH} -o ${LIBDIR}/crtn.o lib/linux/${MES_ARCH}-mes-gcc/crtn.c

# crti.o
tcc-mes -c -D HAVE_CONFIG_H=1 -I include -I include/linux/${MES_ARCH} -o ${LIBDIR}/crti.o lib/linux/${MES_ARCH}-mes-gcc/crti.c
tcc-pnut -c -D HAVE_CONFIG_H=1 -I include -I include/linux/${MES_ARCH} -o ${LIBDIR}/crti.o lib/linux/${MES_ARCH}-mes-gcc/crti.c
fi

# libc+gcc.a
tcc-mes -c -D HAVE_CONFIG_H=1 -I include -I include/linux/${MES_ARCH} -o unified-libc.o unified-libc.c
tcc-mes -ar cr ${LIBDIR}/libc.a unified-libc.o
tcc-pnut -c -D HAVE_CONFIG_H=1 -I include -I include/linux/${MES_ARCH} -o unified-libc.o unified-libc.c
tcc-pnut -ar cr ${LIBDIR}/libc.a unified-libc.o

# libtcc1.a
mkdir ${LIBDIR}/tcc
tcc-mes -c -D HAVE_CONFIG_H=1 -D HAVE_LONG_LONG=1 -D HAVE_FLOAT=1 -I include -I include/linux/${MES_ARCH} -o libtcc1.o lib/libtcc1.c
tcc-pnut -c -D HAVE_CONFIG_H=1 -D HAVE_LONG_LONG=1 -D HAVE_FLOAT=1 -I include -I include/linux/${MES_ARCH} -o libtcc1.o lib/libtcc1.c
if match ${ARCH} riscv64; then
tcc-mes -c -D HAVE_CONFIG_H=1 -D HAVE_LONG_LONG=1 -D HAVE_FLOAT=1 -I include -I include/linux/${MES_ARCH} -o lib-arm64.o ../${TCC_PKG}/lib/lib-arm64.c
tcc-mes -ar cr ${LIBDIR}/tcc/libtcc1.a libtcc1.o lib-arm64.o
tcc-pnut -c -D HAVE_CONFIG_H=1 -D HAVE_LONG_LONG=1 -D HAVE_FLOAT=1 -I include -I include/linux/${MES_ARCH} -o lib-arm64.o ../${TCC_PKG}/lib/lib-arm64.c
tcc-pnut -ar cr ${LIBDIR}/tcc/libtcc1.a libtcc1.o lib-arm64.o
else
tcc-mes -ar cr ${LIBDIR}/tcc/libtcc1.a libtcc1.o
tcc-pnut -ar cr ${LIBDIR}/tcc/libtcc1.a libtcc1.o
fi

# libgetopt.a
tcc-mes -c -D HAVE_CONFIG_H=1 -I include -I include/linux/${MES_ARCH} lib/posix/getopt.c
tcc-mes -ar cr ${LIBDIR}/libgetopt.a getopt.o
tcc-pnut -c -D HAVE_CONFIG_H=1 -I include -I include/linux/${MES_ARCH} lib/posix/getopt.c
tcc-pnut -ar cr ${LIBDIR}/libgetopt.a getopt.o

cd ../${TCC_PKG}
cd ..

# Undo the pnut patches
simple-patch ${TCC_PKG}/tcc.h \
../simple-patches/undefine_TCC_IS_NATIVE.after ../simple-patches/undefine_TCC_IS_NATIVE.before
simple-patch ${TCC_PKG}/libtcc.c \
../simple-patches/sscanf_TCC_VERSION.after ../simple-patches/sscanf_TCC_VERSION.before
simple-patch ${TCC_PKG}/libtcc.c \
../simple-patches/error_set_jmp_enabled.after ../simple-patches/error_set_jmp_enabled.before
simple-patch ${TCC_PKG}/tccpp.c \
../simple-patches/array_sizeof.after ../simple-patches/array_sizeof.before
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps naively, I thought having #ifdef in the patches to disable the changes when compiling with tcc-pnut would be enough to get the same hash. Turns out that the debug information contains line information, so anything that adds the equivalent of new lines can result in a different hash. This means we need to undo the patches.


cd ${TCC_PKG}

# boot0 (ref comments here for all boot*)
# compile
tcc-mes \
tcc-pnut \
-g \
-v \
-static \
Expand Down Expand Up @@ -310,8 +332,7 @@ cd ../..
# Checksums
if match x${UPDATE_CHECKSUMS} xTrue; then
sha256sum -o ${pkg}.${ARCH}.checksums \
/usr/bin/tcc-mes \
/usr/bin/tcc-boot0 \
/usr/bin/pnut-exe \
/usr/bin/tcc-boot1 \
/usr/bin/tcc \
/usr/lib/mes/libc.a \
Expand All @@ -325,3 +346,5 @@ if match x${UPDATE_CHECKSUMS} xTrue; then
else
sha256sum -c ${pkg}.${ARCH}.checksums
fi

exit 1 # Stop the execution so we can check the output
5 changes: 5 additions & 0 deletions steps/tcc-0.9.26/simple-patches/array_sizeof.after
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#ifdef PNUT_CC
char buf1[1024];
#else
char buf1[sizeof file->filename];
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pnut's parser doesn't consider sizeof expressions constants

#endif
1 change: 1 addition & 0 deletions steps/tcc-0.9.26/simple-patches/array_sizeof.before
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
char buf1[sizeof file->filename];
3 changes: 3 additions & 0 deletions steps/tcc-0.9.26/simple-patches/error_set_jmp_enabled.after
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#ifdef PNUT_CC
s1->error_set_jmp_enabled = 0;
#endif
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
s1->error_set_jmp_enabled = 1;
5 changes: 5 additions & 0 deletions steps/tcc-0.9.26/simple-patches/sscanf_TCC_VERSION.after
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#ifdef PNUT_CC
a = 0; b = 9; c = 26;
#else
sscanf(TCC_VERSION, "%d.%d.%d", &a, &b, &c);
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pnut's libc doesn't implement sscanf so we initialize manually the a,b,c variables with the version.

#endif
1 change: 1 addition & 0 deletions steps/tcc-0.9.26/simple-patches/sscanf_TCC_VERSION.before
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
sscanf(TCC_VERSION, "%d.%d.%d", &a, &b, &c);
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
#if defined _WIN32 == defined TCC_TARGET_PE && !defined PNUT_CC
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This masks off the #define TCC_IS_NATIVE which in turn means tccrun.c is not included.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
#if defined _WIN32 == defined TCC_TARGET_PE
2 changes: 2 additions & 0 deletions steps/tcc-0.9.26/sources
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
https://lilypond.org/janneke/tcc/tcc-0.9.26-1147-gee75a10c.tar.gz 6b8cbd0a5fed0636d4f0f763a603247bc1935e206e1cc5bda6a2818bab6e819f tcc-0.9.26.tar.gz
git://github.com/udem-dlteam/pnut.git~124cfae4bfafec24dfea65117d0a407078beb459 https://github.com/udem-dlteam/pnut/archive/124cfae4bfafec24dfea65117d0a407078beb459.tar.gz b7ef75c6b9d647cf91309f52b8b2edf4fb68c2e7cf6c6e7b8a387c9fa447cef0
git://github.com/udem-dlteam/pnut.git~124cfae4bfafec24dfea65117d0a407078beb459 _ 8e82e3b0806abfe4db8224b2ca18cfee0353f1745b2554a24ba10236034f3f28 124cfae4bfafec24dfea65117d0a407078beb459.tar
3 changes: 1 addition & 2 deletions steps/tcc-0.9.26/tcc-0.9.26.x86.checksums
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
096aa0aaf3d2d2742c45683edebbe3ed8ba61d587e1c610e04dab06715824e07 /usr/bin/tcc-mes
2f0209ee3bfe4c65b54c72fe7b5a4f9e85562c31b5b202301d07e040d1f34347 /usr/bin/tcc-boot0
aec2fdd8babb30d4fe4779fcb3f3a76f6438477a01bdb590cae29c56d47355d3 /usr/bin/pnut-exe
edc61c506e4e080ace481d0e2e68ce4e184145721d6e67aab5a9711a7a92695b /usr/bin/tcc-boot1
2204b4625e068b77aaeffc5f94e0ee12e6826c5f7f0639bb53932e2d625939d9 /usr/bin/tcc
bcc6536a4bf27277a2a63430dbf59993dd380ce60ae2feb53dc5ffe5723f7345 /usr/lib/mes/libc.a
Expand Down
Loading