request: clear instructions on how to handle cpp layout failures

[DrRibosome]

Input C/C++ Header

Im attempting to generate bindings for the cpp api for apache arrow. The header file links to a number of other headers, which is difficult to include here.

Bindgen Invocation

(ignore the raw absolute paths)

extern crate bindgen;

use std::env;
use std::path::PathBuf;

fn main() {
    println!("cargo:rustc-env=LD_LIBRARY_PATH={}", "/Users/crawford/git/rust/arrow-sys/arrow/cpp/debug/debug/");
    let base = "/Users/crawford/git/rust/arrow-sys/arrow/cpp/src";

    let bindings = bindgen::Builder::default()
        .opaque_type("std::.*")
        .whitelist_type("arrow::.*")
        .enable_cxx_namespaces()
        .clang_arg("-xc++") // change to cpp mode
        .clang_arg("-std=c++14")
        .clang_arg(format!("-I{}", base))
        .header(format!("{}/arrow/api.h", base))
        .generate()
        .expect("Unable to generate bindings");

    let out_path = PathBuf::from(env::var("OUT_DIR").unwrap());
    bindings
        .write_to_file(out_path.join("bindings.rs"))
        .expect("Couldn't write bindings!");
}

Actual Results

I successfully generated the bindings, but a number of the layout tests fail. I included a snippet of the failures below. The problem is:

• the bindgen docs dont make clear what the impact of incorrect layout is
• if I avoid using these types, will the bindings still work?
• where is the true size (eg, 40 in the below) from?
• what steps should I take to correct this
  • for instance, what is a likely cause for layout issues

---- root::arrow::bindgen_test_layout_UInt64Type stdout ----
        thread 'root::arrow::bindgen_test_layout_UInt64Type' panicked at 'assertion failed: `(left == right)`
  left: `48`,
 right: `40`: Size of: UInt64Type', /Users/crawford/git/rust/arrow-sys/target/debug/build/arrow-sys-903815425d802955/out/bindings.rs:2541:13

---- root::arrow::bindgen_test_layout_UInt8Type stdout ----
        thread 'root::arrow::bindgen_test_layout_UInt8Type' panicked at 'assertion failed: `(left == right)`
  left: `48`,
 right: `40`: Size of: UInt8Type', /Users/crawford/git/rust/arrow-sys/target/debug/build/arrow-sys-903815425d802955/out/bindings.rs:2415:13


failures:
    root::__bindgen_test_layout_DictionaryBuilder_open0_BinaryType_close0_instantiation
    root::__bindgen_test_layout_DictionaryBuilder_open0_StringType_close0_instantiation
    root::arrow::__bindgen_test_layout_DictionaryBuilder_open0_NullType_close0_instantiation
    root::arrow::bindgen_test_layout_BinaryDictionaryBuilder
    root::arrow::bindgen_test_layout_DecimalType
    root::arrow::bindgen_test_layout_DoubleType
    root::arrow::bindgen_test_layout_FloatType
    root::arrow::bindgen_test_layout_HalfFloatType
    root::arrow::bindgen_test_layout_Int16Type
    root::arrow::bindgen_test_layout_Int32Type
    root::arrow::bindgen_test_layout_Int64Type
    root::arrow::bindgen_test_layout_Int8Type
    root::arrow::bindgen_test_layout_StringDictionaryBuilder
    root::arrow::bindgen_test_layout_UInt16Type
    root::arrow::bindgen_test_layout_UInt32Type
    root::arrow::bindgen_test_layout_UInt64Type
    root::arrow::bindgen_test_layout_UInt8Type

test result: FAILED. 150 passed; 17 failed; 0 ignored; 0 measured; 0 filtered out

A snippet for a failing type (size should be 40 (according to the test), but we have 8 from the vtable + 40 from the _base arrow):

        #[repr(C)]
        pub struct UInt8Type__bindgen_vtable(::std::os::raw::c_void);
        #[repr(C)]
        #[derive(Debug, Copy, Clone)]
        pub struct UInt8Type {
            pub vtable_: *const UInt8Type__bindgen_vtable,
            pub _base: [u64; 5usize],
        }
        #[test]
        fn bindgen_test_layout_UInt8Type() {
            assert_eq!(
                ::std::mem::size_of::<UInt8Type>(),
                40usize,
                concat!("Size of: ", stringify!(UInt8Type))
            );
            assert_eq!(
                ::std::mem::align_of::<UInt8Type>(),
                8usize,
                concat!("Alignment of ", stringify!(UInt8Type))
            );
        }

Expected Results

All the layout tests pass, or a chapter in the bindgen book on why they might not pass for cpp code.

[emilio]

It is true that this is lacking. In your case tests are probably failing because the base also has a vtable itself...

[DrRibosome]

If it picked up another vtable in base, does that mean the test itself is wrong (and using the type should work)?

[emilio]

No, the test is usually always right. We take that number from the clang type information.

If you inherit from a class that has a vtable, the resulting class should also have a vtable, but not two. See #851 for a known example where bindgen gets it wrong. I suspect, given you're using opaque types, that we may also get it wrong when the base is opaque / hidden.