`c2rust transpile` always crashes on `aarch64-unknown-linux-gnu` using clang 10

[kkysen]

When linked to clang 10 (and 7, 8, and 9, all the version supported on Ubuntu 20), as in the Docker image and CI, c2rust transpile always crashes on aarch64-unknown-linux-gnu. This includes crashing on a completely empty C file and a minimal compile_commands.json, as well as any other C files I test. It does not happen in clang >10, but it does in clang 10, 9, 8, and 7, and I'm not sure about <7.

For example, using this compilation database at tests/empty.aarch64/compile_commands.json (you need to adjust the directory since it must be an absolute path):

[
  {
    "arguments": [
      "cc",
      "-c",
      "-target",
      "aarch64-linux-gnu",
      "empty.c"
    ],
    "directory": "/home/kkysen/work/rust/c2rust/tests/empty.aarch64",
    "file": "empty.c"
  }
]

and running:

> touch tests/empty.aarch64/empty.c
> LLVM_CONFIG_PATH=llvm-config-10 RUST_BACKTRACE=1 cargo run transpile tests/empty.aarch64/compile_commands.json
    Finished release [optimized] target(s) in 0.17s
     Running `target/release/c2rust transpile tests/empty.aarch64/compile_commands.json`
warning: c2rust: Encountered unsupported BuiltinType kind 48 for type __SVInt8_t
warning: c2rust: Encountered unsupported BuiltinType kind 49 for type __SVInt16_t
warning: c2rust: Encountered unsupported BuiltinType kind 50 for type __SVInt32_t
warning: c2rust: Encountered unsupported BuiltinType kind 51 for type __SVInt64_t
warning: c2rust: Encountered unsupported BuiltinType kind 52 for type __SVUint8_t
warning: c2rust: Encountered unsupported BuiltinType kind 53 for type __SVUint16_t
warning: c2rust: Encountered unsupported BuiltinType kind 54 for type __SVUint32_t
warning: c2rust: Encountered unsupported BuiltinType kind 55 for type __SVUint64_t
warning: c2rust: Encountered unsupported BuiltinType kind 56 for type __SVFloat16_t
warning: c2rust: Encountered unsupported BuiltinType kind 57 for type __SVFloat32_t
warning: c2rust: Encountered unsupported BuiltinType kind 58 for type __SVFloat64_t
warning: c2rust: Encountered unsupported BuiltinType kind 59 for type __SVBool_t
12 warnings generated.
Transpiling empty.c
thread 'main' panicked at 'Type conversion not implemented for TagTypeUnknown expecting 3', c2rust-transpile/src/c_ast/conversion.rs:829:22
stack backtrace:
   0: rust_begin_unwind
             at /rustc/65f3f8b220f020e562c5dd848ff7319257a7ba45/library/std/src/panicking.rs:498:5
   1: core::panicking::panic_fmt
             at /rustc/65f3f8b220f020e562c5dd848ff7319257a7ba45/library/core/src/panicking.rs:107:14
   2: c2rust_transpile::c_ast::conversion::ConversionContext::visit_node
   3: c2rust_transpile::c_ast::conversion::ConversionContext::new
   4: c2rust_transpile::transpile_single
   5: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::fold
   6: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter
   7: c2rust_transpile::transpile
   8: c2rust_transpile::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

you get the above error.

It seems to be due to unrecognized __SV*_t builtin types, which are from arm64 scalable vector extensions. These are unsupported by c2rust, so we should find a way to turn them off as long as we are still building on clang 10.

Note the warning: c2rust: Encountered unsupported BuiltinType kind * for type __SV*_t warnings are from c2rust-ast-exporter/src/AstExporter.cpp:353 in the function TypeEncoder::VisitBuiltinType:

    void VisitBuiltinType(const BuiltinType *T) {
        TypeTag tag;
        auto kind = T->getKind();

        #if CLANG_VERSION_MAJOR >= 11
        // Handle built-in vector types as if they're normal vector types
        if (kind >= BuiltinType::SveInt8 && kind <= BuiltinType::SveBool) {
            auto Info = Context->getBuiltinVectorTypeInfo(T);
            auto t = Info.ElementType;
            auto qt = encodeQualType(t);

            #if CLANG_VERSION_MAJOR >= 12
            auto ElemCount = Info.EC.getKnownMinValue();
            #else
            // getKnownMinValue was added in Clang 12.
            auto ElemCount = Info.EC.Min;
            #endif

            encodeType(T, TagVectorType, [T, qt, Info, ElemCount](CborEncoder *local) {
                cbor_encode_uint(local, qt);
                cbor_encode_uint(local, ElemCount * Info.NumVectors);
            });

            VisitQualType(t);
            return;
        }
        #endif

        // clang-format off
        switch (kind) {
        default: {
            auto pol = clang::PrintingPolicy(Context->getLangOpts());
            auto warning = std::string("Encountered unsupported BuiltinType kind ") +
                           std::to_string((int)kind) + " for type " +
                           T->getName(pol).str();
            printWarning(warning, clang::FullSourceLoc());
            tag = TagTypeUnknown;
            break;
        }

        case BuiltinType::BuiltinFn:  tag = TagBuiltinFn;   break;
        case BuiltinType::UInt128:    tag = TagUInt128;     break;
        case BuiltinType::Int128:     tag = TagInt128;      break;
        case BuiltinType::Short:      tag = TagShort;       break;
        case BuiltinType::Int:        tag = TagInt;         break;
        case BuiltinType::Long:       tag = TagLong;        break;
        case BuiltinType::LongLong:   tag = TagLongLong;    break;
        case BuiltinType::UShort:     tag = TagUShort;      break;
        case BuiltinType::UInt:       tag = TagUInt;        break;
        case BuiltinType::ULong:      tag = TagULong;       break;
        case BuiltinType::ULongLong:  tag = TagULongLong;   break;
        case BuiltinType::Half:       tag = TagHalf;        break;
        #if CLANG_VERSION_MAJOR >= 11
        case BuiltinType::BFloat16:   tag = TagBFloat16;    break;
        #endif
        case BuiltinType::Float:      tag = TagFloat;       break;
        case BuiltinType::Double:     tag = TagDouble;      break;
        case BuiltinType::LongDouble: tag = TagLongDouble;  break;
        case BuiltinType::SChar:      tag = TagSChar;       break;
        case BuiltinType::UChar:      tag = TagUChar;       break;
        case BuiltinType::Char_U:     tag = TagChar;        break;
        case BuiltinType::Char_S:     tag = TagChar;        break;
        case BuiltinType::Void:       tag = TagVoid;        break;
        case BuiltinType::Bool:       tag = TagBool;        break;
        case BuiltinType::WChar_S:    tag = TagSWChar;      break;
        case BuiltinType::WChar_U:    tag = TagUWChar;      break;
        }
        // clang-format on

        encodeType(T, tag);
    }

There's an explicit #if CLANG_VERSION_MAJOR >= 11 check that handles SVE builtin types, so that might be the cause of the panic.

[kkysen]

I think #383 is related to this, in that it fixes the issue on clang >10, but clang 10 doesn't have the Context->getBuiltinVectorTypeInfo API that's needed for this fix. That's a problem if we're still building on clang 10. @fw-immunant, can you look at this?

#378 also seems like it might be related.

[fw-immunant]

#383 is definitely related, as you mention. When writing 294a088 I thought the SVE types were introduced with Clang 11, but they do indeed seem to have been in Clang 10 (despite the API we use to deal with them, as you note, being introduced in Clang 11). I'll do some archaeology and try to see what the right way to handle them in Clang 10 is.

[kkysen]

Is clang 10 the earliest version we're still supporting? It'd be great to at least support clang 10, though, since we use that by default.

[fw-immunant]

I think we support earlier versions (README says 7 or later), but they don't have the SVE types that cause this issue. Does aa0dda8 seem to fix the problem?

[kkysen]

I'll test aa0dda8 now, but LLVM 7-10 all have this same SVE error, not just 10.

[kkysen]

fw/clang-10-aarch64-sve doesn't compile on either clang 10 or 14. I tried to fix a couple things to get it to compile, which I pushed to that branch, but there's a couple things I'm not sure about (I put inline comments for these things, too):

• In

            auto *llvmType = CodeGenTypes::ConvertType(T->desugar());

CodeGenTypes is actually a class (at least it is in clang 15; I can't find older docs). Either way, the "CodeGenTypes.h" header isn't included, so it's not being found. And if it is a class, I'm not sure how to get an instance of it.

• In

                    if (BTy->getKind() == BuiltinType::SveBool)
                        return Ctx.UnsignedCharTy;

Bty is not found. I'm not sure where BTy is supposed to come from.

[fw-immunant]

Ugh, that'll teach me to write code in an #ifdef block that VS Code doesn't check... I don't have a Clang 10 toolchain locally so I was writing from docs and so that code isn't tested.

The BTy->getKind() error is a copy-paste mistake--it should just be kind. The other is a bigger problem--we don't actually have a CodeGenTypes instance at this point, so we'll need a different approach for determining the vector size.

I'm confused how SVE-related errors could be appearing when compiling against Clang 9 and below--as far as I can tell, those types aren't defined until Clang 10. E.g., ripgrep doen't find any relevant code searching for __SV and Sve in the llvm.org source code archive for clang-9.0.0, while the expected results show up in the one for clang-10.0.0.

[kkysen]

I'm realizing the SVE errors in older clangs might actually be a testing mistake on my end. I was running LLVM_CONFIG_PATH=llvm-config-$N cargo run transpile ..., which as I know now, doesn't recompile the c2rust-transpile or its dependencies. That might be the error, it was still seeing the clang 10 errors, so I'll have to recompile and check again.

[kkysen]

I fixed the BTy->getKind() to kind thing now. There's also an error for llvm::ScalableVectorType, which it can't find. I forgot that one.

[fw-immunant]

I've updated the fw/clang-10-aarch64-sve branch.

[kkysen]

Great, it compiles now! I get an error running it, though:

warning: c2rust: Encountered unsupported BuiltinType kind 109 for type _Float16
1 warning generated.
Transpiling empty.c
thread 'main' panicked at 'Type conversion not implemented for TagTypeUnknown expecting 3', c2rust-transpile/src/c_ast/conversion.rs:829:22

Let me look into that.

[kkysen]

That's weird; I can't reproduce that error again anymore.

[kkysen]

And it works in clang 9 as well (so I assume older version are fine, too). The error I saw earlier must've been because of cargo run not recompiling c2rust-transpile.