Mac OSX support incomplete

[schmatz]

As per the readme, OS X support is incomplete. What work needs to be done to achieve full compatibility?

[dcsommer]

We haven't tried it yet. Thrift and Folly, our main depencies, are pretty much working under OS X. This line was basically to document that it is simply untested at this point.

[schmatz]

Awesome. When I have time I'll test on OSX.

[schmatz]

Building the dependencies is a bit tricky, at least on Yosemite. @dcsommer, would a guide for building on Mac be something worth investing time in?

[electrum]

Perhaps even better than a guide would be a Homebrew formula. Then everyone could just brew install folly.

A brief search turned up Homebrew/legacy-homebrew#33086.

[rezacute]

I did the build via docker on yosemite , it will works as long as you set virtual box memory to 3 Gigs.
Can i contribute building via docker and docker file ?

[dcsommer]

@rezacute Building via docker isn't really what we're looking for here. We want to get a native compile on OSX working.

[chaitanyav]

There is no support for numa on OSX. I get this error

configure: error: Please install numa library

The rest of the dependencies can be installed via brew
brew install flex bison pkg-config openssl libcapn gperf autoconf-archive libevent wget

[chaitanyav]

May be the dependencies can be detected like this

case `uname -s` in
  Darwin) install dependencies via brew
    ;;
  Linux) install dependencies via apt-get
    ;;
  FreeBSD) install dependencies via pkg
    ;;
  esac

But this might be a overkill. May be do a bunch of wget of the dependencies from a site and do plain old configure, make and make install . The dependencies versions should be known ahead.

[rezacute]

I have succeeded (but still can not run the echo samples) building proxygen on osx yosemite by lot of hacks especially on fbthrift (removing NUMA, and the compiler from the build).
Here is the error whenever i ran echo_server

Syahs-MacBook-Air:echo syahriza$ ./echo_server 
dyld: Symbol not found: __ZTIN6apache6thrift9transport16THeaderTransportE
  Referenced from: /Users/syahriza/Documents/private/Proxygen_dev/proxygen/proxygen/fbthrift/thrift/lib/cpp/.libs//libthrift.14.dylib
  Expected in: flat namespace
 in /Users/syahriza/Documents/private/Proxygen_dev/proxygen/proxygen/fbthrift/thrift/lib/cpp/.libs//libthrift.14.dylib
Trace/BPT trap: 5

Anyone have suggestion?

[chaitanyav]

@rezacute how did you get folly working on osx?

[afrind]

Maybe try again now that the dependency on fbthrift has been removed?

[yfinkelstein]

I was able to build the latest versions of folly and proxygen on Mac OS 10.10 using downloaded llvm

$ clang++ --version
Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn)
Target: x86_64-apple-darwin14.3.0
Thread model: posix

and
having the following options:

./configure CXX=clang++ CXXFLAGS="-std=c++11 -stdlib=libc++ -nostdinc++ -I/opt/clang+llvm-3.6.1-x86_64-apple-darwin/include/c++/v1"

Boost was installed using

brew install boost --c++11

I have tried various other combinations of compilers and c++ libraries but only this configuration have worked and produced valid binaries.

I also struggled with building libproxygenssl using native darwin toolset (ar not gar)
and ended up adding a dummy c++ file with an empty function just to keep ar happy as it does not support creating libraries without specifying at least one object file:
In proxygen/lib/ssl/Makefile.am:

-libproxygenssl_la_SOURCES =
+libproxygenssl_la_SOURCES = dummy.cpp

With this setup "make check" in folly passes all tests except for one:

PASS: sorted_vector_types_test
PASS: foreach_test
PASS: hash_test
PASS: timeout_queue_test
PASS: conv_test
PASS: range_test
PASS: bits_test
PASS: bit_iterator_test
PASS: spin_lock_test
PASS: small_locks_test
PASS: packed_sync_ptr_test
PASS: small_vector_test
PASS: discriminated_ptr_test
PASS: cpuid_test
PASS: fbstring_test_using_jemalloc
PASS: thread_cached_int_test
FAIL: thread_local_test
PASS: fbvector_test
PASS: dynamic_test
PASS: json_test
PASS: scope_guard_test
PASS: endian_test
PASS: rw_spinlock_test
PASS: synchronized_test
PASS: concurrent_skiplist_test
PASS: histogram_test
PASS: group_varint_test
PASS: map_util_test
PASS: string_test
PASS: producer_consumer_queue_test
PASS: atomic_hash_array_test
PASS: atomic_hash_map_test
PASS: format_test
PASS: fingerprint_test
PASS: portability_test
PASS: spooky_hash_v1_test
PASS: spooky_hash_v2_test
============================================================================
Testsuite summary for folly 41.0
============================================================================
# TOTAL: 37
# PASS:  36
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

Details on the failed test in thread-local group:

[----------] 7 tests from ThreadLocalPtr
[ RUN      ] ThreadLocalPtr.BasicDestructor
[       OK ] ThreadLocalPtr.BasicDestructor (0 ms)
[ RUN      ] ThreadLocalPtr.CustomDeleter1
[       OK ] ThreadLocalPtr.CustomDeleter1 (0 ms)
[ RUN      ] ThreadLocalPtr.resetNull
[       OK ] ThreadLocalPtr.resetNull (0 ms)
[ RUN      ] ThreadLocalPtr.TestRelease
[       OK ] ThreadLocalPtr.TestRelease (0 ms)
[ RUN      ] ThreadLocalPtr.CreateOnThreadExit
ThreadLocalTest.cpp:118: Failure
Value of: Widget::totalVal_
  Actual: 1
Expected: 2
[  FAILED  ] ThreadLocalPtr.CreateOnThreadExit (0 ms)
[ RUN      ] ThreadLocalPtr.CustomDeleter2
[       OK ] ThreadLocalPtr.CustomDeleter2 (0 ms)
[ RUN      ] ThreadLocalPtr.AccessAllThreadsCounter
[       OK ] ThreadLocalPtr.AccessAllThreadsCounter (1 ms)
[----------] 7 tests from ThreadLocalPtr (1 ms total)

[----------] 8 tests from ThreadLocal
[ RUN      ] ThreadLocal.BasicDestructor
[       OK ] ThreadLocal.BasicDestructor (0 ms)
[ RUN      ] ThreadLocal.SimpleRepeatDestructor
[       OK ] ThreadLocal.SimpleRepeatDestructor (0 ms)
[ RUN      ] ThreadLocal.InterleavedDestructors
[       OK ] ThreadLocal.InterleavedDestructors (0 ms)
[ RUN      ] ThreadLocal.resetNull
[       OK ] ThreadLocal.resetNull (0 ms)
[ RUN      ] ThreadLocal.Movable1
[       OK ] ThreadLocal.Movable1 (0 ms)
[ RUN      ] ThreadLocal.Movable2
[       OK ] ThreadLocal.Movable2 (0 ms)
[ RUN      ] ThreadLocal.Fork
[       OK ] ThreadLocal.Fork (0 ms)
[ RUN      ] ThreadLocal.Fork2
[       OK ] ThreadLocal.Fork2 (1 ms)
[----------] 8 tests from ThreadLocal (2 ms total)

[----------] Global test environment tear-down
[==========] 15 tests from 2 test cases ran. (3 ms total)
[  PASSED  ] 14 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] ThreadLocalPtr.CreateOnThreadExit

 1 FAILED TEST
FAIL thread_local_test (exit status: 1)

make check proxygen also has failures in util tests:

FAIL: UtilTests
PASS: AsyncTimeoutSetTest
============================================================================
Testsuite summary for proxygen 24.0
============================================================================
# TOTAL: 2
# PASS:  1
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

util test details:

FAIL: UtilTests
===============

[==========] Running 32 tests from 6 test cases.
[----------] Global test environment set-up.
[----------] 12 tests from OwnedGenericFilterTest
[ RUN      ] OwnedGenericFilterTest.empty_chain
[       OK ] OwnedGenericFilterTest.empty_chain (0 ms)
[ RUN      ] OwnedGenericFilterTest.single_elem_chain
[       OK ] OwnedGenericFilterTest.single_elem_chain (0 ms)
[ RUN      ] OwnedGenericFilterTest.multi_elem_chain
[       OK ] OwnedGenericFilterTest.multi_elem_chain (0 ms)
[ RUN      ] OwnedGenericFilterTest.multi_elem_multi_add
[       OK ] OwnedGenericFilterTest.multi_elem_multi_add (0 ms)
[ RUN      ] OwnedGenericFilterTest.wants
[       OK ] OwnedGenericFilterTest.wants (0 ms)
[ RUN      ] OwnedGenericFilterTest.wants_multi_add
[       OK ] OwnedGenericFilterTest.wants_multi_add (0 ms)
[ RUN      ] OwnedGenericFilterTest.wants_multi_add_hard
[       OK ] OwnedGenericFilterTest.wants_multi_add_hard (139 ms)
[ RUN      ] OwnedGenericFilterTest.change_callback
[       OK ] OwnedGenericFilterTest.change_callback (0 ms)
[ RUN      ] OwnedGenericFilterTest.set_null_cb
[       OK ] OwnedGenericFilterTest.set_null_cb (1 ms)
[ RUN      ] OwnedGenericFilterTest.delete_chain
[       OK ] OwnedGenericFilterTest.delete_chain (0 ms)
[ RUN      ] OwnedGenericFilterTest.get_chain_end
[       OK ] OwnedGenericFilterTest.get_chain_end (144 ms)
[ RUN      ] OwnedGenericFilterTest.set_destination
[       OK ] OwnedGenericFilterTest.set_destination (1 ms)
[----------] 12 tests from OwnedGenericFilterTest (287 ms total)

[----------] 3 tests from UnownedGenericFilterTest
[ RUN      ] UnownedGenericFilterTest.all
[       OK ] UnownedGenericFilterTest.all (141 ms)
[ RUN      ] UnownedGenericFilterTest.delete_do
[       OK ] UnownedGenericFilterTest.delete_do (0 ms)
[ RUN      ] UnownedGenericFilterTest.delete_on
[       OK ] UnownedGenericFilterTest.delete_on (0 ms)
[----------] 3 tests from UnownedGenericFilterTest (141 ms total)

[----------] 4 tests from HTTPTimeTests
[ RUN      ] HTTPTimeTests.InvalidTimeTest
[       OK ] HTTPTimeTests.InvalidTimeTest (0 ms)
[ RUN      ] HTTPTimeTests.ValidTimeTest
[       OK ] HTTPTimeTests.ValidTimeTest (0 ms)
[ RUN      ] HTTPTimeTests.EqualTimeTest
HTTPTimeTest.cpp:37: Failure
Value of: c.value()
  Actual: 1362674977
Expected: a.value()
Which is: 1362646177
HTTPTimeTest.cpp:38: Failure
Value of: c.value()
  Actual: 1362674977
Expected: b.value()
Which is: 1362646177
[  FAILED  ] HTTPTimeTests.EqualTimeTest (0 ms)
[ RUN      ] HTTPTimeTests.ReallyOldTimeTest
HTTPTimeTest.cpp:43: Failure
Value of: a.hasValue()
  Actual: false
Expected: true
HTTPTimeTest.cpp:45: Failure
Value of: b.hasValue()
  Actual: false
Expected: true
HTTPTimeTest.cpp:49: Failure
Expected: (a) < (b), actual: 16-byte object <90-70 80-08 01-00 00-00 00-F0 82-08 01-00 00-00> vs 16-byte object <00-51 84-08 01-00 00-00 00-51 84-08 01-00 00-00>
[  FAILED  ] HTTPTimeTests.ReallyOldTimeTest (0 ms)
[----------] 4 tests from HTTPTimeTests (0 ms total)

[----------] 8 tests from ParseURL
[ RUN      ] ParseURL.HostNoBrackets
[       OK ] ParseURL.HostNoBrackets (0 ms)
[ RUN      ] ParseURL.FullyFormedURL
[       OK ] ParseURL.FullyFormedURL (1 ms)
[ RUN      ] ParseURL.NoScheme
[       OK ] ParseURL.NoScheme (0 ms)
[ RUN      ] ParseURL.NoSchemeIP
[       OK ] ParseURL.NoSchemeIP (0 ms)
[ RUN      ] ParseURL.PathOnly
[       OK ] ParseURL.PathOnly (0 ms)
[ RUN      ] ParseURL.QueryIsURL
[       OK ] ParseURL.QueryIsURL (0 ms)
[ RUN      ] ParseURL.InvalidURL
[       OK ] ParseURL.InvalidURL (0 ms)
[ RUN      ] ParseURL.IsHostIPAddress
[       OK ] ParseURL.IsHostIPAddress (0 ms)
[----------] 8 tests from ParseURL (1 ms total)

[----------] 4 tests from Result
[ RUN      ] Result.Copy
[       OK ] Result.Copy (0 ms)
[ RUN      ] Result.Move
[       OK ] Result.Move (0 ms)
[ RUN      ] Result.Assign
[       OK ] Result.Assign (0 ms)
[ RUN      ] Result.Emplace
[       OK ] Result.Emplace (0 ms)
[----------] 4 tests from Result (0 ms total)

[----------] 1 test from UtilTest
[ RUN      ] UtilTest.CaseInsensitiveEqual
[       OK ] UtilTest.CaseInsensitiveEqual (0 ms)
[----------] 1 test from UtilTest (0 ms total)

[----------] Global test environment tear-down
[==========] 32 tests from 6 test cases ran. (430 ms total)
[  PASSED  ] 30 tests.
[  FAILED  ] 2 tests, listed below:
[  FAILED  ] HTTPTimeTests.EqualTimeTest
[  FAILED  ] HTTPTimeTests.ReallyOldTimeTest

 2 FAILED TESTS
FAIL UtilTests (exit status: 1)

Not sure if this a big deal and did not have time to look into the test code itself.
Is this to be expected in the current version in master branch?

But the real problem is the fact the echo sample consistently crashes within seconds with assertion in folly:

Assertion failed: (sp >= tls_stackLimit), function unmapUnusedStack, file detail/MemoryIdler.cpp, line 142.
[New Thread 0x1713 of process 21075]
[New Thread 0x1803 of process 21075]
[New Thread 0x1903 of process 21075]
[New Thread 0x1a03 of process 21075]
[New Thread 0x1b03 of process 21075]
[New Thread 0x1c03 of process 21075]
[New Thread 0x1d03 of process 21075]
[New Thread 0x1e03 of process 21075]
[New Thread 0x1f03 of process 21075]
[New Thread 0x2003 of process 21075]

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x1713 of process 21075]
0x00007fff8ace2286 in __pthread_kill () from /usr/lib/system/libsystem_kernel.dylib
(gdb) bt
#0  0x00007fff8ace2286 in __pthread_kill () from /usr/lib/system/libsystem_kernel.dylib
#1  0x00007fff879b442f in pthread_kill () from /usr/lib/system/libsystem_pthread.dylib
#2  0x00007fff8854bb53 in abort () from /usr/lib/system/libsystem_c.dylib
#3  0x00007fff88513c39 in __assert_rtn () from /usr/lib/system/libsystem_c.dylib
#4  0x0000000100623f57 in folly::detail::MemoryIdler::unmapUnusedStack(unsigned long) () from /usr/local/lib/libfolly.41.dylib
#5  0x00000001007704ed in folly::wangle::MemoryIdlerTimeout::runLoopCallback() () from /usr/local/lib/libfolly.41.dylib
#6  0x000000010077070c in non-virtual thunk to folly::wangle::MemoryIdlerTimeout::runLoopCallback() () from /usr/local/lib/libfolly.41.dylib
#7  0x00000001005f291f in folly::EventBase::loopBody(int) () from /usr/local/lib/libfolly.41.dylib
#8  0x00000001005f2237 in folly::EventBase::loop() () from /usr/local/lib/libfolly.41.dylib
#9  0x00000001005f45a4 in folly::EventBase::loopForever() () from /usr/local/lib/libfolly.41.dylib
#10 0x000000010076c56d in folly::wangle::IOThreadPoolExecutor::threadRun(std::__1::shared_ptr<folly::wangle::ThreadPoolExecutor::Thread>) () from /usr/local/lib/libfolly.41.dylib
#11 0x00000001007a6250 in std::__1::__function::__func<std::__1::__bind<void (folly::wangle::ThreadPoolExecutor::*)(std::__1::shared_ptr<folly::wangle::ThreadPoolExecutor::Thread>), folly::wangle::ThreadPoolExecutor*, std::__1::shared_ptr<folly::wangle::ThreadPoolExecutor::Thread>&>, std::__1::allocator<std::__1::__bind<void (folly::wangle::ThreadPoolExecutor::*)(std::__1::shared_ptr<folly::wangle::ThreadPoolExecutor::Thread>), folly::wangle::ThreadPoolExecutor*, std::__1::shared_ptr<folly::wangle::ThreadPoolExecutor::Thread>&> >, void ()>::operator()() () from /usr/local/lib/libfolly.41.dylib
#12 0x0000000100027498 in ?? () from /usr/local/lib/libproxygenhttpserver.0.dylib
#13 0x0000000105504000 in ?? ()
#14 0x0000000000001603 in ?? ()
#15 0x000000010400f3a0 in ?? ()
#16 0x0000000100027430 in ?? () from /usr/local/lib/libproxygenhttpserver.0.dylib
#17 0x0000000105503f10 in ?? ()
#18 0x00007fff879b2268 in _pthread_body () from /usr/lib/system/libsystem_pthread.dylib
Backtrace stopped: frame did not save the PC
(gdb) quit
A debugging session is active.

    Inferior 1 [process 21075] will be killed.

I tried linking both folly and proxygen with and without jemalloc - the same effect.

Can somebody explain the nature of this assertion and exception?

Thanks
Yuri

[bugok]

@jwatzman - do you might have an idea?

[jwatzman]

I don't, sorry. I have been poking at HHVM's dependencies (of which proxygen and folly are two) as well as HHVM itself to try to get it to build on OS X, but haven't quite succeeded yet. So there are likely issues actually running the code I have yet to get to :) Nothing in the traces above jumps out at me as an obvious likely culprit, sorry.