dlsym not implemented for: __pthread_get_minstack

[petr-tik]

Hey, this is a great project!

I appreciate you are currently not supporting FFI, but thought this might be a useful data point for posterity. I expect you will consider this low on your list of priorities.

I decided to run it on tantivy

** Versions of rustc and miri

petr_tik@merluza:~$ cargo +nightly miri -V
miri 0.1.0 (bc0c76d 2019-05-13)
petr_tik@merluza:~$ rustc +nightly -Vv
rustc 1.36.0-nightly (7d5aa4332 2019-05-16)
binary: rustc
commit-hash: 7d5aa43325ad7629766b1183011f5bf5b2a1ea26
commit-date: 2019-05-16
host: x86_64-unknown-linux-gnu
release: 1.36.0-nightly
LLVM version: 8.0

** How to reproduce

get the current upstream/master of tantivy

petr_tik@merluza:~/Coding/tantivy$ git show -s HEAD
commit bcd7386fc517e8515e3f062f28196f33ff564e23 (HEAD -> master, upstream/master)
Author: Antoine Catton <devel@antoine.catton.fr>
Date:   Tue Jun 18 04:19:06 2019 +0200

    Add crates.io shield to the README (#572)

Run miri

petr_tik@merluza:~/Coding/tantivy$ cargo +nightly miri test -- -Zmiri-seed=42 -- -Zunstable-options --exclude-should-panic
   Compiling tantivy v0.10.0-dev (/home/petr_tik/Coding/tantivy)

running 322 tests
test collector::count_collector::tests::test_count_collect_does_not_requires_scoring ... ok
test collector::count_collector::tests::test_segment_count_collector ... ok
error[E0080]: Miri evaluation error: miri does not support dynamically loading libraries (requested symbol: __pthread_get_minstack)
  --> /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/sys/unix/weak.rs:65:5
   |
65 |     libc::dlsym(libc::RTLD_DEFAULT, name.as_ptr()) as usize
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Miri evaluation error: miri does not support dynamically loading libraries (requested symbol: __pthread_get_minstack)
   |
   = note: inside call to `std::sys::unix::weak::fetch` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/sys/unix/weak.rs:50:33
   = note: inside call to `std::sys::unix::weak::Weak::<unsafe extern "C" fn(*const libc::unix::notbsd::linux::other::b64::x86_64::pthread_attr_t) -> usize>::get` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/sys/unix/thread.rs:404:11
   = note: inside call to `std::sys::unix::thread::min_stack_size` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/sys/unix/thread.rs:48:42
   = note: inside call to `std::sys::unix::thread::Thread::new` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/thread/mod.rs:488:26
   = note: inside call to `std::thread::Builder::spawn_unchecked::<[closure@src/directory/watch_event_router.rs:54:20: 58:14 callbacks:std::vec::Vec<std::sync::Arc<std::boxed::Box<dyn std::ops::Fn() + std::marker::Send + std::marker::Sync>>>], ()>` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/thread/mod.rs:382:18
note: inside call to `std::thread::Builder::spawn::<[closure@src/directory/watch_event_router.rs:54:20: 58:14 callbacks:std::vec::Vec<std::sync::Arc<std::boxed::Box<dyn std::ops::Fn() + std::marker::Send + std::marker::Sync>>>], ()>` at src/directory/watch_event_router.rs:52:25
  --> src/directory/watch_event_router.rs:52:25
   |
52 |           let spawn_res = std::thread::Builder::new()
   |  _________________________^
53 | |             .name("watch-callbacks".to_string())
54 | |             .spawn(move || {
55 | |                 for callback in callbacks {
56 | |                     callback();
57 | |                 }
58 | |             });
   | |______________^
note: inside call to `directory::watch_event_router::WatchCallbackList::broadcast` at src/directory/ram_directory.rs:185:13
  --> src/directory/ram_directory.rs:185:13
   |
185|             self.fs.write().unwrap().watch_router.broadcast();
   |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside call to `<directory::ram_directory::RAMDirectory as directory::directory::Directory>::atomic_write` at src/directory/managed_directory.rs:227:9
  --> src/directory/managed_directory.rs:227:9
   |
227|         self.directory.atomic_write(path, data)
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside call to `<directory::managed_directory::ManagedDirectory as directory::directory::Directory>::atomic_write` at src/indexer/segment_updater.rs:77:5
  --> src/indexer/segment_updater.rs:77:5
   |
77 |     directory.atomic_write(&META_FILEPATH, &buffer[..])?;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside call to `indexer::segment_updater::save_metas` at src/indexer/segment_updater.rs:52:5
  --> src/indexer/segment_updater.rs:52:5
   |
52 | /     save_metas(
53 | |         &IndexMeta {
54 | |             segments: Vec::new(),
55 | |             schema,
...  |
59 | |         directory,
60 | |     )
   | |_____^
note: inside call to `indexer::segment_updater::save_new_metas` at src/core/index.rs:149:9
  --> src/core/index.rs:149:9
   |
149|         save_new_metas(schema.clone(), directory.borrow_mut())?;
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside call to `core::index::Index::from_directory` at src/core/index.rs:142:9
  --> src/core/index.rs:142:9
   |
142|         Index::from_directory(directory, schema)
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside call to `core::index::Index::create::<directory::ram_directory::RAMDirectory>` at src/core/index.rs:92:9
  --> src/core/index.rs:92:9
   |
92 |         Index::create(ram_directory, schema).expect("Creating a RAMDirectory should never fail")
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside call to `core::index::Index::create_in_ram` at src/collector/facet_collector.rs:529:21
  --> src/collector/facet_collector.rs:529:21
   |
529|         let index = Index::create_in_ram(schema);
   |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside call to `collector::facet_collector::tests::test_doc_unsorted_multifacet` at src/collector/facet_collector.rs:525:5
  --> src/collector/facet_collector.rs:525:5
   |
525| /     fn test_doc_unsorted_multifacet() {
526| |         let mut schema_builder = Schema::builder();
527| |         let facet_field = schema_builder.add_facet_field("facets");
528| |         let schema = schema_builder.build();
...  |
545| |         assert_eq!(facets[0].1, 1);
546| |     }
   | |_____^
   = note: inside call to closure at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/ops/function.rs:231:5
   = note: inside call to `<[closure@src/collector/facet_collector.rs:525:5: 546:6] as std::ops::FnOnce<()>>::call_once - shim` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/ops/function.rs:231:5
   = note: inside call to `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libtest/lib.rs:1521:5
   = note: inside call to `functional_test::test::__rust_begin_short_backtrace::<fn()>` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libtest/lib.rs:1512:30
   = note: inside call to closure at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/ops/function.rs:231:5
   = note: inside call to `<[closure@DefId(123:368 ~ test[fa8f]::run_test[0]::{{closure}}[3]) 0:fn()] as std::ops::FnOnce<()>>::call_once - shim` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/ops/function.rs:231:5
   = note: inside call to `<[closure@DefId(123:368 ~ test[fa8f]::run_test[0]::{{closure}}[3]) 0:fn()] as std::ops::FnOnce<()>>::call_once - shim(vtable)` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/liballoc/boxed.rs:704:9
   = note: inside call to `<std::boxed::Box<dyn std::boxed::FnBox<(), Output = ()> + std::marker::Send> as std::ops::FnOnce<()>>::call_once` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panic.rs:309:9
   = note: inside call to `<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::boxed::FnBox<(), Output = ()> + std::marker::Send>> as std::ops::FnOnce<()>>::call_once` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panicking.rs:293:40
   = note: inside call to `std::panicking::try::do_call::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::boxed::FnBox<(), Output = ()> + std::marker::Send>>, ()>` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panicking.rs:289:5
   = note: inside call to `std::panicking::try::<(), std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::boxed::FnBox<(), Output = ()> + std::marker::Send>>>` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panic.rs:388:9
   = note: inside call to `std::panic::catch_unwind::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::boxed::FnBox<(), Output = ()> + std::marker::Send>>, ()>` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libtest/lib.rs:1467:26
   = note: inside call to closure at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libtest/lib.rs:1489:13
   = note: inside call to `functional_test::test::run_test::run_test_inner` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libtest/lib.rs:1508:28
   = note: inside call to `functional_test::test::run_test` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libtest/lib.rs:1167:13
   = note: inside call to `functional_test::test::run_tests::<[closure@DefId(123:317 ~ test[fa8f]::run_tests_console[0]::{{closure}}[2]) 0:&mut functional_test::test::ConsoleTestState, 1:&mut std::boxed::Box<dyn functional_test::test::formatters::OutputFormatter>]>` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libtest/lib.rs:972:5
   = note: inside call to `functional_test::test::run_tests_console` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libtest/lib.rs:293:15
   = note: inside call to `functional_test::test::test_main` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libtest/lib.rs:327:5
   = note: inside call to `functional_test::test::test_main_static`
   = note: inside call to `main` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:64:34
   = note: inside call to closure at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:52:53
   = note: inside call to closure at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panicking.rs:293:40
   = note: inside call to `std::panicking::try::do_call::<[closure@DefId(1:5981 ~ std[ef5b]::rt[0]::lang_start_internal[0]::{{closure}}[0]) 0:&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe], i32>` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panicking.rs:289:5
   = note: inside call to `std::panicking::try::<i32, [closure@DefId(1:5981 ~ std[ef5b]::rt[0]::lang_start_internal[0]::{{closure}}[0]) 0:&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe]>` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panic.rs:388:9
   = note: inside call to `std::panic::catch_unwind::<[closure@DefId(1:5981 ~ std[ef5b]::rt[0]::lang_start_internal[0]::{{closure}}[0]) 0:&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe], i32>` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:52:25
   = note: inside call to `std::rt::lang_start_internal` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:64:5
   = note: inside call to `std::rt::lang_start::<()>`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0080`.
test collector::facet_collector::tests::test_doc_unsorted_multifacet ...
error: Could not compile `tantivy`.

To learn more, run the command again with --verbose.

[RalfJung]

Thanks for your report!

@oli-obk actually had some ideas for how to extend our shim system to handle dlsym without having to do full FFI... I believe that's part of #603. @oli-obk might be worth splitting the dlsym part (after removing the gross hack ;)? It's independent from the rest of that PR.

[RalfJung]

However, looking at your stacktrace, you are trying to access the file system. That's still way further of I think. See #653.

[petr-tik]

thanks for replying - hopefully my report might be useful.

However, looking at your stacktrace, you are trying to access the file system.

Our library offers several interfaces to interact with the index. We use the in-RAM directory abstraction for testing, so I thought we might wing it

[RalfJung]

Oh I misread... it's not the FS code you are coming from, it's thread spawning. That's ways off I am afraid.

[RalfJung]

This will be "fixed" by #808 insofar as the error message will get better:

  --> /home/r/src/rust/rustc/src/libstd/sys/unix/thread.rs:67:19
   |
67 |           let ret = libc::pthread_create(&mut native, &attr, thread_start,
   |  ___________________^
68 | |                                        &*p as *const _ as *mut _);
   | |_________________________________________________________________^ Miri evaluation error: Miri does not support threading

[RalfJung]

Actual concurrency support is tracked at #811.