ICE inside nested lambdas, with reference captures, when the inner lambda is recursive using c++23s deducing this

[apache-hb]

ICE inside nested lambdas, when lambdas capture by reference, and specifically when the inner lambda both captures by reference and is recursive using c++23s deducing this
https://godbolt.org/z/vbcTjKcvY

[llvmbot]

@llvm/issue-subscribers-clang-codegen

Author: Elliot (apache-hb)

ICE inside nested lambdas, when lambdas capture by reference, and specifically when the inner lambda both captures by reference and is recursive using c++23s deducing this https://godbolt.org/z/vbcTjKcvY

[shafik]

Greatly reduced: https://godbolt.org/z/6M1d91Es7

void do_thing(int x) {
    auto second = [&](this auto const& self, int b) -> int {
        if (x) return x;
        else return self(x);
    };

     second(1);
}

Assertion:

clang++: /root/llvm-project/clang/lib/CodeGen/CGExpr.cpp:5529:
clang::CodeGen::CGCallee clang::CodeGen::CodeGenFunction::EmitCallee(const clang::Expr*): 
Assertion `functionType->isFunctionType()' failed.

Backtrace:

PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.	Program arguments: /opt/compiler-explorer/clang-assertions-trunk/bin/clang++ -gdwarf-4 -g -o /app/output.s -mllvm --x86-asm-syntax=intel -S --gcc-toolchain=/opt/compiler-explorer/gcc-snapshot -fcolor-diagnostics -fno-crash-diagnostics -std=c++2c <source>
1.	<eof> parser at end of file
2.	Per-file LLVM IR generation
3.	<source>:2:19: Generating code for declaration 'do_thing(int)::(anonymous class)::operator()'
 #0 0x00000000038ee468 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x38ee468)
 #1 0x00000000038ec14c llvm::sys::CleanupOnSignal(unsigned long) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x38ec14c)
 #2 0x0000000003833fb8 CrashRecoverySignalHandler(int) CrashRecoveryContext.cpp:0:0
 #3 0x00007f33c4e42520 (/lib/x86_64-linux-gnu/libc.so.6+0x42520)
 #4 0x00007f33c4e969fc pthread_kill (/lib/x86_64-linux-gnu/libc.so.6+0x969fc)
 #5 0x00007f33c4e42476 gsignal (/lib/x86_64-linux-gnu/libc.so.6+0x42476)
 #6 0x00007f33c4e287f3 abort (/lib/x86_64-linux-gnu/libc.so.6+0x287f3)
 #7 0x00007f33c4e2871b (/lib/x86_64-linux-gnu/libc.so.6+0x2871b)
 #8 0x00007f33c4e39e96 (/lib/x86_64-linux-gnu/libc.so.6+0x39e96)
 #9 0x000000000401d58f clang::CodeGen::CodeGenFunction::EmitCallee(clang::Expr const*) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x401d58f)
#10 0x000000000401d98d clang::CodeGen::CodeGenFunction::EmitCallExpr(clang::CallExpr const*, clang::CodeGen::ReturnValueSlot) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x401d98d)
#11 0x0000000004067f19 (anonymous namespace)::ScalarExprEmitter::VisitCallExpr(clang::CallExpr const*) CGExprScalar.cpp:0:0
#12 0x000000000405df50 clang::StmtVisitorBase<std::add_pointer, (anonymous namespace)::ScalarExprEmitter, llvm::Value*>::Visit(clang::Stmt*) CGExprScalar.cpp:0:0
#13 0x000000000406476c clang::CodeGen::CodeGenFunction::EmitScalarExpr(clang::Expr const*, bool) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x406476c)
#14 0x0000000003c0b4a4 clang::CodeGen::CodeGenFunction::EmitReturnStmt(clang::ReturnStmt const&) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x3c0b4a4)
#15 0x0000000003c12c76 clang::CodeGen::CodeGenFunction::EmitStmt(clang::Stmt const*, llvm::ArrayRef<clang::Attr const*>) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x3c12c76)
#16 0x0000000003c19479 clang::CodeGen::CodeGenFunction::EmitIfStmt(clang::IfStmt const&) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x3c19479)
#17 0x0000000003c12c26 clang::CodeGen::CodeGenFunction::EmitStmt(clang::Stmt const*, llvm::ArrayRef<clang::Attr const*>) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x3c12c26)
#18 0x0000000003c1999c clang::CodeGen::CodeGenFunction::EmitCompoundStmtWithoutScope(clang::CompoundStmt const&, bool, clang::CodeGen::AggValueSlot) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x3c1999c)
#19 0x0000000003c7a4f2 clang::CodeGen::CodeGenFunction::EmitFunctionBody(clang::Stmt const*) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x3c7a4f2)
#20 0x0000000003c8d884 clang::CodeGen::CodeGenFunction::GenerateCode(clang::GlobalDecl, llvm::Function*, clang::CodeGen::CGFunctionInfo const&) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x3c8d884)
#21 0x0000000003cebb82 clang::CodeGen::CodeGenModule::EmitGlobalFunctionDefinition(clang::GlobalDecl, llvm::GlobalValue*) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x3cebb82)
#22 0x0000000003ce64a5 clang::CodeGen::CodeGenModule::EmitGlobalDefinition(clang::GlobalDecl, llvm::GlobalValue*) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x3ce64a5)
#23 0x0000000003cf1f9d clang::CodeGen::CodeGenModule::EmitDeferred() (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x3cf1f9d)
#24 0x0000000003cf4e63 clang::CodeGen::CodeGenModule::Release() (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x3cf4e63)
#25 0x00000000041869fa (anonymous namespace)::CodeGeneratorImpl::HandleTranslationUnit(clang::ASTContext&) ModuleBuilder.cpp:0:0
#26 0x000000000418551d clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x418551d)
#27 0x0000000006160d5c clang::ParseAST(clang::Sema&, bool, bool) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x6160d5c)
#28 0x0000000004184de8 clang::CodeGenAction::ExecuteAction() (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x4184de8)
#29 0x0000000004400319 clang::FrontendAction::Execute() (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x4400319)
#30 0x000000000437d84e clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x437d84e)
#31 0x00000000044e246e clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x44e246e)
#32 0x0000000000c25566 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0xc25566)
#33 0x0000000000c1d2ca ExecuteCC1Tool(llvm::SmallVectorImpl<char const*>&, llvm::ToolContext const&) driver.cpp:0:0
#34 0x00000000041c64d9 void llvm::function_ref<void ()>::callback_fn<clang::driver::CC1Command::Execute(llvm::ArrayRef<std::optional<llvm::StringRef>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, bool*) const::'lambda'()>(long) Job.cpp:0:0
#35 0x0000000003834464 llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x3834464)
#36 0x00000000041c6acf clang::driver::CC1Command::Execute(llvm::ArrayRef<std::optional<llvm::StringRef>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, bool*) const (.part.0) Job.cpp:0:0
#37 0x000000000418e675 clang::driver::Compilation::ExecuteCommand(clang::driver::Command const&, clang::driver::Command const*&, bool) const (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x418e675)
#38 0x000000000418f0dd clang::driver::Compilation::ExecuteJobs(clang::driver::JobList const&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&, bool) const (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x418f0dd)
#39 0x0000000004197015 clang::driver::Driver::ExecuteCompilation(clang::driver::Compilation&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x4197015)
#40 0x0000000000c22a4d clang_main(int, char**, llvm::ToolContext const&) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0xc22a4d)
#41 0x0000000000b17e04 main (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0xb17e04)
#42 0x00007f33c4e29d90 (/lib/x86_64-linux-gnu/libc.so.6+0x29d90)
#43 0x00007f33c4e29e40 __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29e40)
#44 0x0000000000c1cd8e _start (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0xc1cd8e)
clang++: error: clang frontend command failed with exit code 134 (use -v to see invocation)
Compiler returned: 134

a little more I obtain a different crash: https://godbolt.org/z/c3d7ses1n

void do_thing(int x) {
    auto second = [&](this auto const& self)  {
        if (true) return x;
        else return x;
    };

     second();
}

CC @cor3ntin @erichkeane

[Sirraide]

Currently working on a bug that also involves capturing lambdas w/ explicit object parameters, so I’ll see if I can take a look at this one as well.