Problem with fdatasync when cross compiling to aarch64 on Mac OS X

[sseefried]

The error output is:

"inplace/bin/ghc-stage1" -hisuf hi -osuf  o -hcsuf hc -static  -H64m -O0    -this-package-key unix_6KxsoniXeYR9o99eChzwLX -hide-all-packages -i -ilibraries/unix/. -ilibraries/unix/dist-install/build -ilibraries/unix/dist-install/build/autogen -Ilibraries/unix/dist-install/build -Ilibraries/unix/dist-install/build/autogen -Ilibraries/unix/include    -optP-include -optPlibraries/unix/dist-install/build/autogen/cabal_macros.h -package-key base_62ctz6fVRDGAMD7GZKNkAJ -package-key bytes_BBgXs4iMVG62xkLTXxSQ78 -package-key time_Ca2vuoP20B2BBf1t5jg5Kb -Wall -XHaskell2010 -O -fllvm  -no-user-package-db -rtsopts      -odir libraries/unix/dist-install/build -hidir libraries/unix/dist-install/build -stubdir libraries/unix/dist-install/build   -c libraries/unix/dist-install/build/System/Posix/Unistd.hs -o libraries/unix/dist-install/build/System/Posix/Unistd.o

/var/folders/pv/pfxcyy117v31vt6lsshtgr_w0000gn/T/ghc3154_0/ghc3154_2.c:9:110:
     error: implicit declaration of function 'fdatasync' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
HsInt32 ghczuwrapperZC0ZCunixzu6KxsoniXeYR9o99eChzzwLXZCSystemziPosixziUnistdZCfdatasync(HsInt32 a1) {return fdatasync(a1);}

The bug is introduced in commit 98eced8.

The compiler I was using was LLVM 3.6's clang in a wrapper script called aarch64-apple-darwin14-clang. The contents of this file are:

#!/bin/sh
TARGET_PLATFORM=`xcrun --show-sdk-path --sdk iphoneos`
TARGET_GCC=/usr/local/clang+llvm-3.6.0-rc2-x86_64-apple-darwin/bin/clang
TARGET_CFLAGS="-isysroot $TARGET_PLATFORM -arch arm64 -miphoneos-version-min=7.0"

exec $TARGET_GCC $TARGET_CFLAGS "$@"

The problem is simply that you cannot use AC_CHECK_FUNCS in configure.ac to
check for the existence of a library function when cross-compiling.

This is detailed here

My work-around to the issue is to run configure manually and then edit include/HsUnixConfig.h
manually and set:

 #define HAVE_FDATASYNC 0

[hvr]

I'm not sure what we can do here. We use AC_CHECK_FUNCS quite extensively for all sorts of functions:

AC_CHECK_FUNCS([getgrgid_r getgrnam_r getpwnam_r getpwuid_r getpwnam getpwuid])
AC_CHECK_FUNCS([getpwent getgrent])
AC_CHECK_FUNCS([lchown setenv sysconf unsetenv clearenv])
AC_CHECK_FUNCS([nanosleep])
AC_CHECK_FUNCS([ptsname])
AC_CHECK_FUNCS([setitimer])
AC_CHECK_FUNCS([readdir_r])
AC_CHECK_FUNCS([telldir seekdir])
AC_CHECK_FUNCS([execvpe])
AC_CHECK_FUNCS([utimensat futimens])
AC_CHECK_FUNCS([lutimes futimes])
AC_CHECK_FUNCS([mkstemps mkdtemp])
AC_CHECK_FUNCS([fsync fdatasync])
AC_CHECK_FUNCS([posix_fadvise posix_fallocate])

So we're definitely not cross-compile safe right now... :-/

[sseefried]

Okay, I've found a work around. We just need to suppress the warning when we are using clang. Just add the following to configure.ac.

AC_EGREP_CPP(we_are_using_clang,
[
#include <unistd.h>
#ifdef __clang__
we_are_using_clang
#endif
],
[AC_MSG_RESULT([yes])
IS_CLANG="yes"],
[AC_MSG_RESULT([no])])

if test x"$IS_CLANG" == x"yes" ; then
  CFLAGS="$CFLAGS -Wno-implicit-function-declaration"
fi

The reason I think this is okay is due to this discussion: Orc/discount#55

At the very least, I don't think this is going to hurt anything. I'll fork and send you a pull request right now.

[rwbarton]

There should be no problem using AC_CHECK_FUNCS when cross-compiling:

"When writing tests in configure.in', you must remember that you want to test the host environment, not the build environment. Macros which use the compiler, such as likeAC_CHECK_FUNCS', will test the host environment. That is because the tests will be done by running the compiler, which is actually a build cross host compiler. If the compiler can find the function, that means that the function is present in the host environment."

This is what we want, since the libraries (like unix) are built to run on the host environment (the target of the stage1 (cross-)compiler, and the system on which the stage2 (native) compiler runs). (Yes, the autoconf terminology is terribly confusing.)

We should figure out what's going on here. Is fdatasync actually provided by unistd.h on the arm64 iOS environment you are targeting?

[argiopetech]

Some searching reveals that OS X has had problems with fdatasync being in the headers but not having an implementation in past. It's possible that it's been fixed in OS X proper but still exists in iOS.

@sseefried Can you check pull request #42 and see if it solves your problem?

[rwbarton]

Oh, interesting. I found this bug report for ruby, which seems to state the opposite of what you wrote, but it's a bit confusing:

This is a regression caused by the revert (r30821) of r30725:
configure.in: Mac OS X wrongly reports it has fdatasync(3).

While fdatasync is defined, there is no header for it.

If that last sentence is accurate, then things fall into place. I looked at what AC_CHECK_FUNCS generates and it defines its own prototype for the function being tested (I guess since it wouldn't know what kind of arguments to pass otherwise). So the configure check succeeded, since it successfully linked the test executable because the iOS libc actually does define an fdatasync function. Then when we tried to build the unix library with the C backend, we got an error about the missing prototype for fdatasync, and failed due to -Werror.

If that is what's going on, then I would think that @argiopetech's patch would have the same problem, unless a missing prototype is an error by default for clang? Worth testing though since apparently it worked for beanstalkd.

[argiopetech]

Yeah, it appears I've crossed my sources here. Not sure why #42 workes for beanstalkd.

According to this issue in picoc, fdatasync is available as a system call but not as a library function under OS X. They use a call to fcntl() as a the alternative.

I'm trying to track down a copy of the iOS documentation to see if this is a viable alternative. I'm honestly not sure how much the iOS libraries mimic those of OS X.

[sseefried]

@argiopetech I have tried out #42 and found that the relevant part of include/HsUnixConfig.h now becomes:

/* If the system defines fdatasync */
/* #undef HAVE_FDATASYNC */

/* Define to 1 if you have the `fsync' function. */
#define HAVE_FSYNC 1

My previous patch is now obviously the wrong approach. Thank you everyone for thinking through this more clearly than I was.

[argiopetech]

Okay, so it appears the beanstalkd patch works is that (as mentioned by @rwbarton) AC_CHECK_FUNCS includes a prototype when it tries to compile. The beanstalkd patch does not include a prototype. As such, the AC_CHECK_FUNCS file compiles (no missing prototypes) and is linked (wrongly) to the system call. Without the prototype it fails to compile.

I'm going to go ahead and close this as fixed. It's going to throw exceptions on OS X (with warning at compile time, of course), but I'll open a new issue to implement the fcntl() fix.