Concurrency issue with TClassEdit::ResolveTypedef and TClass::GetListOfMethods

[Dr15Jones]

Check duplicate issues.

• Checked for duplicates

Description

While testing RNTuple code with CMS release CMSSW_15_1_RNTUPLE_X_2025-04-24-2300 we are seeing lots of failures when running the system with multiple threads. The tracebacks all involve one thread calling RNTuple routines which include TClassEdit::ResolveTypedef in the traceback and the others involve code that is asking TClass about member functions of classes.

Reproducer

setup work area using CMSSW_15_1_RNTUPLE_X_2025-04-24-2300.

In the /src/ directory do

mkdir Test/Test/bin

In that directory add the file main.cc containing

#include "TClassEdit.h"

#include "FWCore/Reflection/interface/TypeWithDict.h"

#include "DataFormats/Candidate/interface/LeafCandidate.h"

#include <thread>
#include <atomic>


int main() {

  std::atomic<int> countDown{2};

  std::thread t1([](std::atomic<int>& count) {
                   --count;
                   while(count != 0);
                   TClassEdit::ResolveTypedef("std::vector<unsigned int>", false);
                 }, std::ref(countDown));
  std::thread t2([](std::atomic<int>& count) {
                   edm::TypeWithDict t(typeid(reco::LeafCandidate));
                   --count;
                   while(count!=0);
                   t.functionMemberByName("eta");
                 }, std::ref(countDown));

  t1.join();
  t2.join();
  return 0;
}

also create the file BuildFile.xml containing

<bin name="testRoot" file="test.cc">
  <use name="root"/>
  <use name="DataFormats/Candidate"/>
  <use name="FWCore/Reflection"/>
</bin>

build using scram b then from the src directory run the program ../bin/el8_amd64_gcc12/testRoot

Several tries are normally necessary before a failure is seen. One such failure while running in the debugger gave an assertion failure with the message

testRoot: /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/interpreter/llvm-project/clang/include/clang/Lex/Preprocessor.h:1877: const clang::Token& 
clang::Preprocessor::LookAhead(unsigned int): Assertion `LexLevel == 0 && "cannot use lookahead while lexing"' failed.

and with the following stack trace

Thread 3 (Thread 0x7ffff579c700 (LWP 411887) "testRoot"):                                                                                                                                                                                          
#0  0x00007ffff7027b09 in __xstat64 () from /lib64/libc.so.6                                                                                                                                                                                       
#1  0x00007fffeda9e2fc in llvm::sys::fs::status(llvm::Twine const&, llvm::sys::fs::file_status&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/li
bCling.so                                                                                                                                                                                                                                          
#2  0x00007fffeda5f65c in (anonymous namespace)::RealFileSystem::status(llvm::Twine const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so  
#3  0x00007fffeda5f7fc in llvm::vfs::RedirectingFileSystem::getExternalStatus(llvm::Twine const&, llvm::Twine const&) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/e
l8_amd64_gcc12/lib/libCling.so                                                                                                                                                                                                                     
#4  0x00007fffeda62e71 in llvm::vfs::RedirectingFileSystem::status(llvm::Twine const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so       
#5  0x00007fffeda5f4c9 in llvm::vfs::OverlayFileSystem::status(llvm::Twine const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so           
#6  0x00007fffec76b897 in clang::FileSystemStatCache::get(llvm::StringRef, llvm::vfs::Status&, bool, std::unique_ptr<llvm::vfs::File, std::default_delete<llvm::vfs::File> >*, clang::FileSystemStatCache*, llvm::vfs::FileSystem&) () from /cvmfs/
cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                                                                                                             
#7  0x00007fffec76441d in clang::FileManager::getStatValue(llvm::StringRef, llvm::vfs::Status&, bool, std::unique_ptr<llvm::vfs::File, std::default_delete<llvm::vfs::File> >*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/c
mssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                                                                                                                                                                 
#8  0x00007fffec766752 in clang::FileManager::getDirectoryRef(llvm::StringRef, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so         
#9  0x00007fffec76784e in getDirectoryFromFile(clang::FileManager&, llvm::StringRef, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so   
#10 0x00007fffec769694 in clang::FileManager::getFileRef(llvm::StringRef, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so        
#11 0x00007fffec683f9a in clang::HeaderSearch::getFileAndSuggestModule(llvm::StringRef, clang::SourceLocation, clang::DirectoryEntry const*, bool, clang::Module*, clang::ModuleMap::KnownHeader*, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86
_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                                                                                                                                  
#12 0x00007fffec686737 in clang::HeaderSearch::LookupFile(llvm::StringRef, clang::SourceLocation, bool, clang::detail::SearchDirIteratorImpl<true>, clang::detail::SearchDirIteratorImpl<true>*, llvm::ArrayRef<std::pair<clang::CustomizableOption
al<clang::FileEntryRef>, clang::DirectoryEntryRef> >, llvm::SmallVectorImpl<char>*, llvm::SmallVectorImpl<char>*, clang::Module*, clang::ModuleMap::KnownHeader*, bool*, bool*, bool, bool, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/wee
k0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                                                                                                                                         
#13 0x00007fffec6d3544 in clang::Preprocessor::LookupFile(clang::SourceLocation, llvm::StringRef, bool, clang::detail::SearchDirIteratorImpl<true>, clang::FileEntry const*, clang::detail::SearchDirIteratorImpl<true>*, llvm::SmallVectorImpl<cha
r>*, llvm::SmallVectorImpl<char>*, clang::ModuleMap::KnownHeader*, bool*, bool*, bool, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCli
ng.so                                                                                                                                                                                                                                              
#14 0x00007fffec6d693d in clang::Preprocessor::LookupHeaderIncludeOrImport(clang::detail::SearchDirIteratorImpl<true>*, llvm::StringRef&, clang::SourceLocation, clang::CharSourceRange, clang::Token const&, bool&, bool, bool&, clang::detail::Se
archDirIteratorImpl<true>, clang::FileEntry const*, llvm::StringRef&, llvm::SmallVectorImpl<char>&, llvm::SmallVectorImpl<char>&, clang::ModuleMap::KnownHeader&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMS
SW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                                                                                                                                                                         
#15 0x00007fffec6dc9d6 in clang::Preprocessor::HandleHeaderIncludeOrImport(clang::SourceLocation, clang::Token&, clang::Token&, clang::SourceLocation, clang::detail::SearchDirIteratorImpl<true>, clang::FileEntry const*) () from /cvmfs/cms-ib.c
ern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                                                                                                                     
#16 0x00007fffec6df094 in clang::Preprocessor::HandleIncludeDirective(clang::SourceLocation, clang::Token&, clang::detail::SearchDirIteratorImpl<true>, clang::FileEntry const*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/
cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                                                                                                                                                                
#17 0x00007fffec6dfde1 in clang::Preprocessor::HandleDirective(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                
#18 0x00007fffec6993c2 in clang::Lexer::LexTokenInternal(clang::Token&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                
#19 0x00007fffec69ad5f in clang::Lexer::Lex(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                                   
#20 0x00007fffec719004 in clang::Preprocessor::Lex(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                            
#21 0x00007fffec6cc840 in clang::Preprocessor::PeekAhead(unsigned int) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                       
#22 0x00007fffe8a79ea0 in clang::Preprocessor::LookAhead (this=0x7ffff0083790, N=0) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/interpreter/l
lvm-project/clang/include/clang/Lex/Preprocessor.h:1881                                                                                                                                                                                            
#23 0x00007fffe977e14a in clang::Parser::ParseOptionalCXXScopeSpecifier(clang::CXXScopeSpec&, clang::OpaquePtr<clang::QualType>, bool, bool, bool*, bool, clang::IdentifierInfo**, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_am
d64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                                                                                                                                                  
#24 0x00007fffe97feed7 in clang::Parser::isCXXDeclarationStatement(bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                     
#25 0x00007fffe980eeb6 in clang::Parser::ParseExternalDeclaration(clang::ParsedAttributes&, clang::ParsedAttributes&, clang::ParsingDeclSpec*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04
-24-2300/external/el8_amd64_gcc12/lib/libCling.so                                                                                                                                                                                                  
#26 0x00007fffe980fe95 in clang::Parser::ParseTopLevelDecl(clang::OpaquePtr<clang::DeclGroupRef>&, clang::Sema::ModuleImportState&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/ex
ternal/el8_amd64_gcc12/lib/libCling.so                                                                                                                                                                                                             
#27 0x00007fffe981033f in clang::Parser::ParseFirstTopLevelDecl(clang::OpaquePtr<clang::DeclGroupRef>&, clang::Sema::ModuleImportState&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-23
00/external/el8_amd64_gcc12/lib/libCling.so                                                                                                                                                                                                        
#28 0x00007fffe8bbc219 in cling::IncrementalParser::ParseOrWrapTopLevelDecl() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                
#29 0x00007fffe8bbcb08 in cling::IncrementalParser::ParseInternal(llvm::StringRef) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so           
#30 0x00007fffe8bbd79c in cling::IncrementalParser::Compile(llvm::StringRef, cling::CompilationOptions const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc1
2/lib/libCling.so                                                                                                                                                                                                                                  
#31 0x00007fffe8b29642 in cling::Interpreter::parseForModule(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2
025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                                                                                                                                                                                            
#32 0x00007fffe88dca8e in ExecAutoParse (what=0x7ffff7fa4810 "\n#line 1 \"DataFormatsCandidate_xr dictionary payload\"\n\n#ifndef CMS_DICT_IMPL\n  #define CMS_DICT_IMPL 1\n#endif\n#ifndef _REENTRANT\n  #define _REENTRANT 1\n#endif\n#ifndef GNU
SOURCE\n  #define GNUSOURCE 1\n#"..., header=false, interpreter=0x7ffff00293e0) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/metacling/sr
c/TCling.cxx:6392                                                                                                                                                                                                                                  
#33 0x00007fffe88dd31e in TCling::AutoParseImplRecurse (this=0x7ffff00279a0, cls=0x7ffff2840280 "reco::LeafCandidate", topLevel=true) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f$1d83322604af89ae12687a2c/root-6.35.1/core/metacling/src/TCling.cxx:6497
#34 0x00007fffe88dd9a3 in TCling::AutoParse (this=0x7ffff00279a0, cls=0x7ffff2840280 "reco::LeafCandidate") at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/metacling/src/TCling.cxx:6582
#35 0x00007ffff7bcb002 in TClass::LoadClassInfo (this=0x7ffff2767150) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/meta/src/TClass.cxx:5962
#36 0x00007ffff7af41e8 in TClass::GetClassInfo (this=0x7ffff2767150) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/meta/inc/TClass.h:442 
#37 0x00007ffff7bf81e8 in TListOfFunctions::Load (this=0x7ffff28728a0) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/meta/src/TListOfFunctions.cxx:382
#38 0x00007ffff7bc422c in TClass::GetListOfMethods (this=0x7ffff2767150, load=true) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/meta/src/TClass.cxx:3933
#39 0x00007ffff7f2b94f in edm::TypeWithDict::functionMemberByName (this=0x7ffff57979b0, member=...) at src/FWCore/Reflection/src/TypeWithDict.cc:602                                                                                              
#40 0x0000000000402555 in operator() (__closure=0x4e28a8, count=...) at src/Test/Test/bin/test.cc:24
#41 0x0000000000403cff in std::__invoke_impl<void, main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > >(std::__invoke_other, struct {...} &&, std::reference_wrapper<std::atomic<int> > &&) (__f=...) at /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/invoke.h:61                                                                                                            
--Type <RET> for more, q to quit, c to continue without paging--
#42 0x0000000000403c3f in std::__invoke<main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > >(struct {...} &&, std::reference_wrapper<std::atomic<int> > &&) (__fn=...) at /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/invoke.h:96
#43 0x0000000000403b95 in std::thread::_Invoker<std::tuple<main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > > >::_M_invoke<0, 1>(std::_Index_tuple<0, 1>) (this=0x4e28a8) at /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/std_thread.h:279
#44 0x0000000000403b32 in std::thread::_Invoker<std::tuple<main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > > >::operator()(void) (this=0x4e28a8) at /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/std_thread.h:286
#45 0x0000000000403af6 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > > > >::_M_run(void) (this=0x4e28a0) at /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/std_thread.h:231
#46 0x00007ffff75c8a73 in std::execute_native_thread_routine (__p=0x4e28a0) at ../../../../../libstdc++-v3/src/c++11/thread.cc:82                                                                                                                 
#47 0x00007ffff72d81ca in start_thread () from /lib64/libpthread.so.0
#48 0x00007ffff6f338d3 in clone () from /lib64/libc.so.6

Thread 2 (Thread 0x7ffff5f9d700 (LWP 411886) "testRoot"):
#0  0x00007ffff6f4852f in raise () from /lib64/libc.so.6
#1  0x00007ffff6f1be65 in abort () from /lib64/libc.so.6
#2  0x00007ffff6f1bd39 in __assert_fail_base.cold.0 () from /lib64/libc.so.6
#3  0x00007ffff6f40e86 in __assert_fail () from /lib64/libc.so.6
#4  0x00007fffe8a79e36 in clang::Preprocessor::LookAhead (this=0x7ffff0083790, N=0) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/interpreter/llvm-project/clang/include/clang/Lex/Preprocessor.h:1877
#5  0x00007fffe977e14a in clang::Parser::ParseOptionalCXXScopeSpecifier(clang::CXXScopeSpec&, clang::OpaquePtr<clang::QualType>, bool, bool, bool*, bool, clang::IdentifierInfo**, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so
#6  0x00007fffe98064eb in clang::Parser::TryAnnotateCXXScopeToken(bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                     
#7  0x00007fffe8b3c3b1 in cling::LookupHelper::findScope(llvm::StringRef, cling::LookupHelper::DiagSetting, clang::Type const**, bool) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so
#8  0x00007fffe88dfba5 in GetClassSharedLibsForModule (cls=0x7fffcc000b60 "std::vector<unsigned int>", LH=..., skipCore=true) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/metacling/src/TCling.cxx:7037
#9  0x00007fffe88dff5d in TCling::GetClassSharedLibs (this=0x7ffff00279a0, cls=0x7fffcc000b60 "std::vector<unsigned int>", skipCore=true) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/metacling/src/TCling.cxx:7149
#10 0x00007fffe88c4e78 in TClingLookupHelper__ExistingTypeCheck (tname=..., result=...) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/metacling/src/TCling.cxx:990
#11 0x00007fffe86f225e in ROOT::TMetaUtils::TClingLookupHelper::ExistingTypeCheck (this=0x7ffff193da90, tname=..., result=...) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/clingutils/src/TClingUtils.cxx:547
#12 0x00007ffff7ba3aa2 in TClassEdit::ResolveTypedef[abi:cxx11](char const*, bool) (tname=0x404200 "std::vector<unsigned int>") at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/foundation/src/TClassEdit.cxx:1778
#13 0x00000000004024ab in operator() (__closure=0x4e2718, count=...) at src/Test/Test/bin/test.cc:18
#14 0x0000000000403d45 in std::__invoke_impl<void, main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > >(std::__invoke_other, struct {...} &&, std::reference_wrapper<std::atomic<int> > &&) (__f=...) at /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/invoke.h:61                                                                                                            
#15 0x0000000000403c9f in std::__invoke<main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > >(struct {...} &&, std::reference_wrapper<std::atomic<int> > &&) (__fn=...) at /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/invoke.h:96
#16 0x0000000000403bdf in std::thread::_Invoker<std::tuple<main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > > >::_M_invoke<0, 1>(std::_Index_tuple<0, 1>) (this=0x4e2718) at /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/std_thread.h:279
#17 0x0000000000403b4e in std::thread::_Invoker<std::tuple<main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > > >::operator()(void) (this=0x4e2718) at /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/std_thread.h:286
#18 0x0000000000403b16 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > > > >::_M_run(void) (this=0x4e2710) at /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/std_thread.h:231
#19 0x00007ffff75c8a73 in std::execute_native_thread_routine (__p=0x4e2710) at ../../../../../libstdc++-v3/src/c++11/thread.cc:82                                                                                                                 
#20 0x00007ffff72d81ca in start_thread () from /lib64/libpthread.so.0
#21 0x00007ffff6f338d3 in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x7ffff665cb00 (LWP 411863) "testRoot"):
#0  0x00007ffff72d96bd in __pthread_timedjoin_ex () from /lib64/libpthread.so.0
#1  0x00007ffff75c8d77 in __gthread_join (__value_ptr=0x0, __threadid=<optimized out>) at /data/cmsbld/jenkins/workspace/auto-builds/CMSSW_13_2_0_pre2-el8_amd64_gcc12/build/CMSSW_13_2_0_pre2-build/BUILD/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/gcc-12.3.1/obj/x86_64-redhat-linux-gnu/libstdc++-v3/include/x86_64-redhat-linux-gnu/bits/gthr-default.h:669                                                                                                          
#2  std::thread::join (this=0x7fffffff1ec0) at ../../../../../libstdc++-v3/src/c++11/thread.cc:112
#3  0x0000000000402628 in main () at src/Test/Test/bin/test.cc:27

ROOT version

---

| Welcome to ROOT 6.35.01 https://root.cern |
| (c) 1995-2024, The ROOT Team; conception: R. Brun, F. Rademakers |
| Built for linuxx8664gcc on Apr 17 2025, 15:35:31 |
| From heads/master@tags/v6-35-01 |
| With g++ (GCC) 12.3.1 20230527 |
| Try '.help'/'.?', '.demo', '.license', '.credits', '.quit'/'.q' |

Installation method

build from source

Operating system

Linux

Additional context

No response

[pcanal]

Even-though cms-sw/cmssw#47953 (comment) shows a real error, the one seen is most likely due to the missing call to:

ROOT::EnableThreadSafety();

TCling::GetClassSharedLibs which is in stack trace is the one that does take the lock.

[Dr15Jones]

@pcanal thanks for noticing the omission. With the following changes I do see a data race crash

#include "TClassEdit.h"

#include "FWCore/Reflection/interface/TypeWithDict.h"

#include "DataFormats/Candidate/interface/LeafCandidate.h"

#include "TROOT.h"

#include <thread>
#include <atomic>
#include <iostream>

int main() {

  ROOT::EnableThreadSafety();
  std::atomic<int> countDown{2};

  std::thread t1([](std::atomic<int>& count) {
                   --count;
                   while(count != 0);
                   std::cerr<<"typedef v unsigned int\n";
                   TClassEdit::ResolveTypedef("std::vector<unsigned int>", false);
                   std::cerr<<"typedef v reco::Track\n";
                   TClassEdit::ResolveTypedef("std::vector<reco::Track>",false);
                   std::cerr<<"typedef v reco::Electron\n";
                   TClassEdit::ResolveTypedef("std::vector<reco::Electron>",false);
                   std::cerr<<"typedef v reco::Muon\n";
                   TClassEdit::ResolveTypedef("std::vector<reco::Muon>",false);
                   std::cerr<<"typedef done'n";
                 }, std::ref(countDown));
  std::thread t2([](std::atomic<int>& count) {
                   edm::TypeWithDict t(typeid(reco::LeafCandidate));
                   --count;
                   while(count!=0);
                   std::cerr<<"func eta\n";
                   t.functionMemberByName("eta");
                   std::cerr<<"func phi\n";
                   t.functionMemberByName("phi");
                   std::cerr<<"func charge\n";
                   t.functionMemberByName("charge");
                   std::cerr<<"func momentum\n";
                   t.functionMemberByName("momentum");
                 }, std::ref(countDown));

  t1.join();
  t2.join();
  return 0;
}

with the ROOT based crash log of

===========================================================                                                                                                                                                                                        
There was a crash (kSigSegmentationViolation).                                                                                                                                                                                                     
This is the entire stack trace of all threads:                                                                                                                                                                                                     
===========================================================                                                                                                                                                                                        
                                                                                                                                                                                                                                                   
Thread 3 (Thread 0x7f33aed77700 (LWP 557705) "testRoot"):                                                                                                                                                                                          
#0  0x00007f33cd084371 in pthread_cond_wait                                                                                                                                                                                                        
                                                                                                                                                                                                                                                   
GLIBC_2.3.2 () from /lib64/libpthread.so.0                                                                                                                                                                                                         
#1  0x00007f33cb64ecec in std::_V2::condition_variable_any::wait<std::unique_lock<std::mutex> > (this=0x3b457b0, __lock=...) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3
947a6e575ca28/include/c++/12.3.1/condition_variable:312                                                                                                                                                                                            
#2  0x00007f33cb64dd39 in std::_V2::condition_variable_any::wait<std::unique_lock<std::mutex>, ROOT::TReentrantRWLock<std::mutex, ROOT::Internal::RecurseCountsTBBUnique>::WriteLock()::{lambda()#1}>(std::unique_lock<std::mutex>&, ROOT::TReentra
ntRWLock<std::mutex, ROOT::Internal::RecurseCountsTBBUnique>::WriteLock()::{lambda()#1}) (this=0x3b457b0, __lock=..., __p=...) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30
e3947a6e575ca28/include/c++/12.3.1/condition_variable:321                                                                                                                                                                                          
#3  0x00007f33cb649c8c in ROOT::TReentrantRWLock<std::mutex, ROOT::Internal::RecurseCountsTBBUnique>::WriteLock (this=0x3b45778) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d833
22604af89ae12687a2c/root-6.35.1/core/thread/src/TReentrantRWLock.cxx:190                                                                                                                                                                           
#4  0x00007f33cb65c1e0 in ROOT::TRWMutexImp<std::mutex, ROOT::Internal::RecurseCountsTBBUnique>::WriteLock (this=0x3b45770) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604
af89ae12687a2c/root-6.35.1/core/thread/src/TRWMutexImp.cxx:42                                                                                                                                                                                      
#5  0x00007f33cd80da5d in ROOT::TVirtualRWMutex::Lock (this=0x3b45770) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/build/include/TVirtualRWMutex.h:79    
#6  0x00007f33c3fec85e in TLockGuard::TLockGuard (this=0x7f33aed71e68, mutex=0x3b45770) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/base
/inc/TVirtualMutex.h:80                                                                                                                                                                                                                            
#7  0x00007f33c3e94aea in TCling::HandleNewTransaction (this=0x1c30100, T=...) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/metacling/src
/TCling.cxx:3691                                                                                                                                                                                                                                   
#8  0x00007f33c3ea17e4 in TCling::UpdateListsOnCommitted (this=0x1c30100, T=...) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/metacling/s
rc/TCling.cxx:6804                                                                                                                                                                                                                                 
#9  0x00007f33c3e85895 in TCling__UpdateListsOnCommitted (T=...) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/metacling/src/TCling.cxx:56
7                                                                                                                                                                                                                                                  
#10 0x00007f33c403bc55 in TClingCallbacks::TransactionCommitted (this=0x313d430, T=...) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/meta
cling/src/TClingCallbacks.cxx:974                                                                                                                                                                                                                  
#11 0x00007f33c40f3e88 in cling::MultiplexInterpreterCallbacks::TransactionCommitted(cling::Transaction const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc
12/lib/libCling.so                                                                                                                                                                                                                                 
#12 0x00007f33c417d3a8 in cling::IncrementalParser::commitTransaction(llvm::PointerIntPair<cling::Transaction*, 2u, cling::IncrementalParser::EParseResult, llvm::PointerLikeTypeTraits<cling::Transaction*>, llvm::PointerIntPairInfo<cling::Trans
action*, 2u, llvm::PointerLikeTypeTraits<cling::Transaction*> > >&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                    
#13 0x00007f33c417d0d5 in cling::IncrementalParser::commitTransaction(llvm::PointerIntPair<cling::Transaction*, 2u, cling::IncrementalParser::EParseResult, llvm::PointerLikeTypeTraits<cling::Transaction*>, llvm::PointerIntPairInfo<cling::Trans
action*, 2u, llvm::PointerLikeTypeTraits<cling::Transaction*> > >&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                    
#14 0x00007f33c417ce53 in cling::IncrementalParser::codeGenTransaction(cling::Transaction*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so  
#15 0x00007f33c417d335 in cling::IncrementalParser::commitTransaction(llvm::PointerIntPair<cling::Transaction*, 2u, cling::IncrementalParser::EParseResult, llvm::PointerLikeTypeTraits<cling::Transaction*>, llvm::PointerIntPairInfo<cling::Trans
action*, 2u, llvm::PointerLikeTypeTraits<cling::Transaction*> > >&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so                    
#16 0x00007f33c40e9a6a in cling::Interpreter::PushTransactionRAII::~PushTransactionRAII() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/lib/libCling.so    
#17 0x00007f33c40ffa8d in cling::LookupHelper::findScope(llvm::StringRef, cling::LookupHelper::DiagSetting, clang::Type const**, bool) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-2
4-2300/external/el8_amd64_gcc12/lib/libCling.so                                                                                                                                                                                                    
#18 0x00007f33c3cb55e5 in ROOT::TMetaUtils::TClingLookupHelper::IsDeclaredScope (this=0x3564270, base=..., isInlined=                                                                                                                              
0x7f33aed725af: false) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/clingutils/src/TClingUtils.cxx:588                                   
#19 0x00007f33cd94872e in ResolveTypedefImpl (tname=0x405218 "std::vector<unsigned int>", len=25, cursor=                                                                                                                                          
0x7f33aed7292c: 3, modified=                                                                                                                                                                                                                       
0x7f33aed7292b: false, result=...) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/foundation/src/TClassEdit.cxx:1605                       
#20 0x00007f33cd949b63 in TClassEdit::ResolveTypedef[abi:cxx11](char const*, bool) (tname=0x405218 "std::vector<unsigned int>") at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d8332
2604af89ae12687a2c/root-6.35.1/core/foundation/src/TClassEdit.cxx:1788                                                                                                                                                                             
#21 0x00000000004024f3 in operator() (__closure=0x3c980b8, count=...) at src/Test/Test/bin/test.cc:22                                                                                                                                              
#22 0x000000000040408b in std::__invoke_impl<void, main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > >(std::__invoke_other, struct {...} &&, std::reference_wrapper<std::atomic<int> > &&) (__f=...) at /cvmfs/cms-ib.
cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/invoke.h:61                                                                                                             
#23 0x0000000000403fe5 in std::__invoke<main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > >(struct {...} &&, std::reference_wrapper<std::atomic<int> > &&) (__fn=...) at /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/e
l8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/invoke.h:96                                                                                                                                            
#24 0x0000000000403f25 in std::thread::_Invoker<std::tuple<main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > > >::_M_invoke<0, 1>(std::_Index_tuple<0, 1>) (this=0x3c980b8) at /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-0
2886/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/std_thread.h:279                                                                                                                                 
#25 0x0000000000403e94 in std::thread::_Invoker<std::tuple<main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > > >::operator()(void) (this=0x3c980b8) at /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/ext
ernal/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/std_thread.h:286                                                                                                                                                         
#26 0x0000000000403e5c in std::thread::_State_impl<std::thread::_Invoker<std::tuple<main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > > > >::_M_run(void) (this=0x3c980b0) at /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02
886/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/std_thread.h:231                                                                                                                                  
#27 0x00007f33cd36ea73 in std::execute_native_thread_routine (__p=0x3c980b0) at ../../../../../libstdc++-v3/src/c++11/thread.cc:82                                                                                                                 
#28 0x00007f33cd07e1ca in start_thread () from /lib64/libpthread.so.0                                                                                                                                                                              
#29 0x00007f33cccd98d3 in clone () from /lib64/libc.so.6                                                                                                                                                                                           

Thread 2 (Thread 0x7f33ae576700 (LWP 557706) "testRoot"):                                                                                                                                                                                          
#0  0x00007f33ccda7e42 in waitpid () from /lib64/libc.so.6                                                                                                                                                                                         
#1  0x00007f33cccfb747 in do_system () from /lib64/libc.so.6                                                                                                                                                                                       
#2  0x00007f33cd9e52a2 in TUnixSystem::Exec (this=0x1ba3e40, shellcmd=0x7f3398a160d0 "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/etc/gdb-backtrace.sh 557687 1>&2") at /data/cmsb
ld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/unix/src/TUnixSystem.cxx:2157                                                                            
#3  0x00007f33cd9e5b45 in TUnixSystem::StackTrace (this=0x1ba3e40) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/unix/src/TUnixSystem.cxx:
2448
#4  0x00007f33cd9e9590 in TUnixSystem::DispatchSignals (this=0x1ba3e40, sig=kSigSegmentationViolation) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6
.35.1/core/unix/src/TUnixSystem.cxx:3668
#5  0x00007f33cd9e13b0 in SigHandler (sig=kSigSegmentationViolation) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/unix/src/TUnixSystem.cx
x:410
#6  0x00007f33cd9e9493 in sighandler (sig=11) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/unix/src/TUnixSystem.cxx:3639
#7  <signal handler called>
#8  0x00007f33c40ecf03 in cling::Interpreter::DeclareCFunction(llvm::StringRef, llvm::StringRef, bool, cling::Transaction*&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/
el8_amd64_gcc12/lib/libCling.so
#9  0x00007f33c40ed103 in cling::Interpreter::compileFunction(llvm::StringRef, llvm::StringRef, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_RNTUPLE_X_2025-04-24-2300/external/el8_amd64_gcc12/l
ib/libCling.so
#10 0x00007f33c40492d3 in TClingCallFunc::compile_wrapper (this=0x7f33a835d750, wrapper_name=..., wrapper=..., withAccessControl=true) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f
51d83322604af89ae12687a2c/root-6.35.1/core/metacling/src/TClingCallFunc.cxx:138
#11 0x00007f33c404c616 in TClingCallFunc::make_wrapper (this=0x7f33a835d750) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/metacling/src/T
ClingCallFunc.cxx:1031
#12 0x00007f33c404f780 in TClingCallFunc::IFacePtr (this=0x7f33a835d750) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/metacling/src/TClin
gCallFunc.cxx:1701
#13 0x00007f33c3ea6035 in TCling::CallFunc_IFacePtr (this=0x1c30100, func=0x7f33a835d750) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/root-6.35.1/core/me
tacling/src/TCling.cxx:7996
#14 0x00007f33cdcd0410 in edm::FunctionWithDict::FunctionWithDict (this=<optimized out>, meth=<optimized out>, this=<optimized out>, meth=<optimized out>) at src/FWCore/Reflection/src/FunctionWithDict.cc:25
#15 0x00007f33cdcd799f in edm::TypeWithDict::functionMemberByName (this=0x7f33ae5718c0, member=...) at src/FWCore/Reflection/src/TypeWithDict.cc:606
#16 0x00000000004027a5 in operator() (__closure=0x41af9f8, count=...) at src/Test/Test/bin/test.cc:40
#17 0x0000000000404045 in std::__invoke_impl<void, main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > >(std::__invoke_other, struct {...} &&, std::reference_wrapper<std::atomic<int> > &&) (__f=...) at /cvmfs/cms-ib.
cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/invoke.h:61
#18 0x0000000000403f85 in std::__invoke<main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > >(struct {...} &&, std::reference_wrapper<std::atomic<int> > &&) (__fn=...) at /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/e
l8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/invoke.h:96
#19 0x0000000000403edb in std::thread::_Invoker<std::tuple<main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > > >::_M_invoke<0, 1>(std::_Index_tuple<0, 1>) (this=0x41af9f8) at /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-0
2886/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/std_thread.h:279
#20 0x0000000000403e78 in std::thread::_Invoker<std::tuple<main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > > >::operator()(void) (this=0x41af9f8) at /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/ext
ernal/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/std_thread.h:286
#21 0x0000000000403e3c in std::thread::_State_impl<std::thread::_Invoker<std::tuple<main()::<lambda(std::atomic<int>&)>, std::reference_wrapper<std::atomic<int> > > > >::_M_run(void) (this=0x41af9f0) at /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02
886/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/std_thread.h:231
#22 0x00007f33cd36ea73 in std::execute_native_thread_routine (__p=0x41af9f0) at ../../../../../libstdc++-v3/src/c++11/thread.cc:82
#23 0x00007f33cd07e1ca in start_thread () from /lib64/libpthread.so.0
#24 0x00007f33cccd98d3 in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x7f33cc404b00 (LWP 557687) "testRoot"):
#0  0x00007f33cd07f6bd in __pthread_timedjoin_ex () from /lib64/libpthread.so.0
#1  0x00007f33cd36ed77 in __gthread_join (__value_ptr=0x0, __threadid=<optimized out>) at /data/cmsbld/jenkins/workspace/auto-builds/CMSSW_13_2_0_pre2-el8_amd64_gcc12/build/CMSSW_13_2_0_pre2-build/BUILD/el8_amd64_gcc12/external/gcc/12.3.1-40d5
04be6370b5a30e3947a6e575ca28/gcc-12.3.1/obj/x86_64-redhat-linux-gnu/libstdc++-v3/include/x86_64-redhat-linux-gnu/bits/gthr-default.h:669
#2  std::thread::join (this=0x7ffe5beea930) at ../../../../../libstdc++-v3/src/c++11/thread.cc:112
#3  0x000000000040296e in main () at src/Test/Test/bin/test.cc:45
===========================================================

[pcanal]

Good. Now this is indeed exactly cms-sw/cmssw#47953 (comment).
And this issue and #18520 complement each other.

[Dr15Jones]

Running Helgrind on the new program gives many hits within the IncrementalParser, e.g.

==558597== Possible data race during read of size 8 at 0x637A0F0 by thread #3
==558597== Locks held: none
==558597==    at 0x81DB1F3: cling::IncrementalParser::beginTransaction(cling::CompilationOptions const&) (in /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/lib/libCling.so)
==558597==    by 0x8149A03: cling::Interpreter::PushTransactionRAII::PushTransactionRAII(cling::Interpreter const*) (in /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/lib/libCling.s
o)
==558597==    by 0x80C60E3: TClingMethodInfo::TClingMethodInfo(cling::Interpreter*, TClingClassInfo*) (TClingMethodInfo.cxx:256)
==558597==    by 0x7F08611: TCling::MethodInfo_Factory(ClassInfo_t*) const (TCling.cxx:9064)
==558597==    by 0x4F192BF: TListOfFunctions::Load() (TListOfFunctions.cxx:396)
==558597==    by 0x4EE522B: TClass::GetListOfMethods(bool) (TClass.cxx:3933)
==558597==    by 0x412494E: edm::TypeWithDict::functionMemberByName(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const (TypeWithDict.cc:602)
==558597==    by 0x402724: main::{lambda(std::atomic<int>&)#2}::operator()(std::atomic<int>&) const (test.cc:38)
==558597==    by 0x404044: void std::__invoke_impl<void, main::{lambda(std::atomic<int>&)#2}, std::reference_wrapper<std::atomic<int> > >(std::__invoke_other, main::{lambda(std::atomic<int>&)#2}&&, std::reference_wrapper<std::atomic<int> >&&) 
(invoke.h:61)
==558597==    by 0x403F84: std::__invoke_result<main::{lambda(std::atomic<int>&)#2}, std::reference_wrapper<std::atomic<int> > >::type std::__invoke<main::{lambda(std::atomic<int>&)#2}, std::reference_wrapper<std::atomic<int> > >(main::{lambda
(std::atomic<int>&)#2}&&, std::reference_wrapper<std::atomic<int> >&&) (invoke.h:96)
==558597==    by 0x403EDA: void std::thread::_Invoker<std::tuple<main::{lambda(std::atomic<int>&)#2}, std::reference_wrapper<std::atomic<int> > > >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) (std_thread.h:279)
==558597==    by 0x403E77: std::thread::_Invoker<std::tuple<main::{lambda(std::atomic<int>&)#2}, std::reference_wrapper<std::atomic<int> > > >::operator()() (std_thread.h:286)
==558597== 
==558597== This conflicts with a previous write of size 8 by thread #2
==558597== Locks held: none
==558597==    at 0x81DB266: cling::IncrementalParser::beginTransaction(cling::CompilationOptions const&) (in /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/lib/libCling.so)
==558597==    by 0x8149A03: cling::Interpreter::PushTransactionRAII::PushTransactionRAII(cling::Interpreter const*) (in /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/lib/libCling.s
o)
==558597==    by 0x815EE21: cling::LookupHelper::findScope(llvm::StringRef, cling::LookupHelper::DiagSetting, clang::Type const**, bool) const (in /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af
89ae12687a2c/lib/libCling.so)
==558597==    by 0x7D155E4: ROOT::TMetaUtils::TClingLookupHelper::IsDeclaredScope(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool&) (TClingUtils.cxx:588)
==558597==    by 0x4EC372D: ResolveTypedefImpl(char const*, unsigned int, unsigned int&, bool&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) (TClassEdit.cxx:1605)
==558597==    by 0x4EC4B62: TClassEdit::ResolveTypedef[abi:cxx11](char const*, bool) (TClassEdit.cxx:1788)
==558597==    by 0x4024F2: main::{lambda(std::atomic<int>&)#1}::operator()(std::atomic<int>&) const (test.cc:22)
==558597==    by 0x40408A: void std::__invoke_impl<void, main::{lambda(std::atomic<int>&)#1}, std::reference_wrapper<std::atomic<int> > >(std::__invoke_other, main::{lambda(std::atomic<int>&)#1}&&, std::reference_wrapper<std::atomic<int> >&&) 
(invoke.h:61)
==558597==  Address 0x637a0f0 is 80 bytes inside a block of size 96 alloc'd
==558597==    at 0x403DF21: operator new(unsigned long) (in /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/external/valgrind/3.24.0-f85a1303334507f502ed8242a93c05bd/libexec/valgrind/vgpreload_helgrind-amd64-linux.so)
==558597==    by 0x81DD4AB: cling::IncrementalParser::IncrementalParser(cling::Interpreter*, char const*, std::vector<std::shared_ptr<clang::ModuleFileExtension>, std::allocator<std::shared_ptr<clang::ModuleFileExtension> > > const&) (in /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/lib/libCling.so)
==558597==    by 0x8150A51: cling::Interpreter::Interpreter(int, char const* const*, char const*, std::vector<std::shared_ptr<clang::ModuleFileExtension>, std::allocator<std::shared_ptr<clang::ModuleFileExtension> > > const&, void*, bool, cling::Interpreter const*) (in /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02886/el8_amd64_gcc12/lcg/root/6.35.1-6c0d80f51d83322604af89ae12687a2c/lib/libCling.so)
==558597==    by 0x7E5C3E8: cling::Interpreter::Interpreter(int, char const* const*, char const*, std::vector<std::shared_ptr<clang::ModuleFileExtension>, std::allocator<std::shared_ptr<clang::ModuleFileExtension> > > const&, void*, bool) (Interpreter.h:368)
==558597==    by 0x805CF2D: std::__detail::_MakeUniq<cling::Interpreter>::__single_object std::make_unique<cling::Interpreter, unsigned long, char const**, TString&, std::vector<std::shared_ptr<clang::ModuleFileExtension>, std::allocator<std::shared_ptr<clang::ModuleFileExtension> > >&, void*&>(unsigned long&&, char const**&&, TString&, std::vector<std::shared_ptr<clang::ModuleFileExtension>, std::allocator<std::shared_ptr<clang::ModuleFileExtension> > >&, void*&) (unique_ptr.h:1065)
==558597==    by 0x7EEB63A: TCling::TCling(char const*, char const*, char const* const*, void*) (TCling.cxx:1537)
==558597==    by 0x7EE5A57: CreateInterpreter (TCling.cxx:613)
==558597==    by 0x4E22F19: TROOT::InitInterpreter() (TROOT.cxx:2127)
==558597==    by 0x4E1C630: ROOT::Internal::GetROOT2() (TROOT.cxx:391)
==558597==    by 0x4E1C992: ROOT::GetROOT() (TROOT.cxx:473)
==558597==    by 0x4F118EF: TInterpreter::Instance() (TInterpreter.cxx:61)
==558597==    by 0x4E49289: TSystem::GetLibraries(char const*, char const*, bool) (TSystem.cxx:2160)
==558597==  Block was alloc'd by thread #1

[pcanal]

Hellgrind is right in this case but I don't know if it knows about the ROOT lock. For example:

==558597==    by 0x7F08611: TCling::MethodInfo_Factory(ClassInfo_t*) const (TCling.cxx:9064)

does that the ROOT lock.

[pcanal]

Should be fixed by #18522