Extractor exiting with code 1 (error: parameter pack "BoundArgs" was referenced but not expanded)

[flowerhack]

Hi hello,

I'm a committer for the Chromium project & we've been experimenting with building CodeQL databases of Chromium.

Context

While building the Chromium CodeQL database, in addition to the previously-reported "catastrophic" errors ([1], [2]), we get many thousands of errors that, while they do not seem to cross the threshold to be logged as "catastrophic," they nonetheless cause the extractor to terminate with exit code 1 & lead to incomplete Chromium databases being created.

I've investigated these errors and have classed them into nine unique bug types. I intend to report all nine (this report is bug 6 of 9), with a reproducing test case for each.

The hope is that, if these bugs + the catastrophic errors are fixed, we will be able to have a complete build of a Chromium CodeQL database (barring, of course, the scenario where fixing these bugs serves to unmask new ones...!).

The Bug

When building the Chromium CodeQL database, we see ~6,000 errors of the following type:

Warning[extractor-c++]: In construct_text_message: "../../base/functional/bind_internal.h", line 1309: error: parameter pack "BoundArgs" was referenced but not expanded
    template <bool v = !HasRefCountedTypeAsRawPtr<std::decay_t<BoundArgs>...>>
                                                               ^

Reproducing The Bug

I have created a standalone file which can be used to reproduce this bug, which is attached here as webgl_depth_texture_ii.cc.txt (please remove the .txt extension; this was to make the Github attachment uploader happy).

Reproduction steps (assumes that webgl_depth_texture_ii.cc is in /YOUR/ROOT/HERE; assumes Clang 19 (Chrome uses the latest upstream Clang, generally speaking); assumes Linux):

(1) `codeql database init --language=cpp --source-root=/YOUR/ROOT/HERE/SOME-EMPTY-DIRECTORY /YOUR/ROOT/HERE/repro-boundargs-bug-db --overwrite

(2) codeql database trace-command /YOUR_ROOT_HERE/repro-boundargs-bug-db --working-dir=/YOUR/ROOT/HERE -- clang++ -DDCHECK_ALWAYS_ON=1 -DUSE_UDEV -DUSE_AURA=1 -DUSE_GLIB=1 -DUSE_OZONE=1 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_EXTENSIVE -DCR_CLANG_REVISION=\"llvmorg-19-init-14561-gecea8371-1\" -DCOMPONENT_BUILD -DCR_LIBCXX_REVISION=09b99fd8ab300c93ff7b8df6688cafb27bd3db28 -DCR_SYSROOT_KEY=20230611T210420Z-2 -D_DEBUG -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DBLINK_MODULES_IMPLEMENTATION=1 -DBLINK_IMPLEMENTATION=1 -DINSIDE_BLINK -DGLIB_VERSION_MAX_ALLOWED=GLIB_VERSION_2_56 -DGLIB_VERSION_MIN_REQUIRED=GLIB_VERSION_2_56 -DSK_ENABLE_SKSL -DSK_UNTIL_CRBUG_1187654_IS_FIXED -DSK_WIN_FONTMGR_NO_SIMULATIONS -DSK_DISABLE_LEGACY_INIT_DECODERS -DSK_SLUG_DISABLE_LEGACY_DESERIALIZE -DSK_DISABLE_LEGACY_VULKAN_BACKENDSEMAPHORE -DSK_DISABLE_LEGACY_CREATE_CHARACTERIZATION -DSK_DISABLE_LEGACY_VULKAN_MUTABLE_TEXTURE_STATE -DSK_CODEC_DECODES_JPEG -DSK_ENCODE_JPEG -DSK_ENCODE_PNG -DSK_ENCODE_WEBP -DSKIA_DLL -DSKCMS_API=__attribute__\(\(visibility\(\"default\"\)\)\) -DSK_GANESH -DSK_GL -DSK_VULKAN=1 -DSK_GRAPHITE -DSK_DAWN -DVK_USE_PLATFORM_XCB_KHR -DVK_USE_PLATFORM_WAYLAND_KHR -DLIBYUV_DISABLE_NEON -DLIBYUV_DISABLE_LSX -DLIBYUV_DISABLE_LASX -DABSL_CONSUME_DLL -DABSL_FLAGS_STRIP_NAMES=0 -DBORINGSSL_SHARED_LIBRARY -DU_USING_ICU_NAMESPACE=0 -DU_ENABLE_DYLOAD=0 -DUSE_CHROMIUM_ICU=1 -DU_ENABLE_TRACING=1 -DU_ENABLE_RESOURCE_TRACING=0 -DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_FILE -DUSING_V8_SHARED -DUSING_V8_SHARED_PRIVATE -DV8_ARRAY_BUFFER_INTERNAL_FIELD_COUNT=0 -DV8_ARRAY_BUFFER_VIEW_INTERNAL_FIELD_COUNT=0 -DV8_PROMISE_INTERNAL_FIELD_COUNT=0 -DV8_ENABLE_CHECKS -DV8_COMPRESS_POINTERS -DV8_COMPRESS_POINTERS_IN_SHARED_CAGE -DV8_31BIT_SMIS_ON_64BIT_ARCH -DV8_ENABLE_SANDBOX -DV8_DEPRECATION_WARNINGS -DV8_USE_PERFETTO -DV8_HAVE_TARGET_OS -DV8_TARGET_OS_LINUX -DCPPGC_CAGED_HEAP -DCPPGC_YOUNG_GENERATION -DCPPGC_POINTER_COMPRESSION -DCPPGC_ENABLE_LARGER_CAGE -DCPPGC_SLIM_WRITE_BARRIER -DGOOGLE_PROTOBUF_NO_RTTI -DGOOGLE_PROTOBUF_NO_STATIC_INITIALIZER -DGOOGLE_PROTOBUF_INTERNAL_DONATE_STEAL_INLINE=0 -DHAVE_PTHREAD -DPROTOBUF_USE_DLLS -DWGPU_SHARED_LIBRARY -DWEBRTC_ENABLE_SYMBOL_EXPORT -DWEBRTC_ENABLE_AVX2 -DWEBRTC_CHROMIUM_BUILD -DWEBRTC_POSIX -DWEBRTC_LINUX -DABSL_ALLOCATOR_NOTHROW=1 -DWEBRTC_USE_X11 -DWEBRTC_USE_PIPEWIRE -DWEBRTC_USE_GIO -DLOGGING_INSIDE_WEBRTC -DLEVELDB_PLATFORM_CHROMIUM=1 -DLEVELDB_SHARED_LIBRARY -DCRASHPAD_ZLIB_SOURCE_EXTERNAL -DWTF_USE_WEBAUDIO_PFFFT=1 -DUSE_LIBJPEG_TURBO=1 -DMANGLE_JPEG_NAMES -DWEBP_EXTERN=extern -DUSING_V8_BASE_SHARED -DUSING_V8_PLATFORM_SHARED -DLIBXSLT_STATIC -fno-delete-null-pointer-checks -fno-ident -fno-strict-aliasing -fstack-protector -funwind-tables -fPIC -pthread -fcolor-diagnostics -fmerge-all-constants -fno-sized-deallocation -mllvm -instcombine-lower-dbg-declare=0 -mllvm -split-threshold-for-reg-with-hint=0 -ffp-contract=off -fcomplete-member-pointers -m64 -msse3 -D__DATE__= -D__TIME__= -D__TIMESTAMP__= -ffile-compilation-dir=. -no-canonical-prefixes -ftrivial-auto-var-init=pattern -O0 -fno-omit-frame-pointer -fvisibility=hidden -DUNSAFE_BUFFERS_BUILD -gdwarf-4 -g2 -gdwarf-aranges -gsplit-dwarf -ggnu-pubnames -DPROTOBUF_ALLOW_DEPRECATED=1 -DLIBXML_STATIC= -Wno-everything -gsimple-template-names -fno-exceptions -fno-rtti -nostdinc++ -fvisibility-inlines-hidden -std=c++20 -c /YOUR/ROOT/HERE/webgl_depth_texture_ii.cc -o /YOUR/ROOT/HERE/webgl_depth_texture.o

(3) codeql database finalize -j=-1 /YOUR/ROOT/HERE/repro-boundargs-bug-db.

At the conclusion of these steps there should be logs in build-tracer.log and logs/extractor/ indicating the failure.

In addition to (1) webgl_depth_texture_ii.cc.txt (the reproducer file), please find attached (2) the build-tracer.log and (3) the relevant extractor logfile (ecc0f.log) from running this on my own machine, which will hopefully be useful for debugging/triage.

I do have the logs for the entire Chromium build available upon request, but as you might imagine, those files are very large and may not be as useful to you as this standalone reproducer.

A fix for this bug (or, guidance on how we might be holding it wrong!) would be extremely helpful for us here in Chromium. Please let me know if you need any more information. Thank you!

build-tracer.log
ecc0f.log
webgl_depth_texture_ii.cc.txt

[jketema]

Hi @flowerhack

Thanks for the report. We'll try to have a look at this fairly soon.

[jketema]

Reported to our frontend provider.

[jketema]

@flowerhack This should be fixed in CodeQL 2.19.3. I believe this was the last open Chromium issue that you reported that came with a reproducing test case. We would be happy to continue to work on Chromium coverage if you're able to submit more issues with reproducers (ideally reproducible with the latest CodeQL version).