C++: crash when reaching maximum token chain size.

[Quipyowert2]

Crash when reaching max token chain size 2026-01-08
Crash #264
8191 repetitions of "0^U", then :class<
To create the crashing testcase, run perl -e 'print "0\cU" x 8191, ":class<"' > ./crash_264.cpp

In cxxParserParseNextToken(), when the token chain size limit is reached, the last token is destroyed without notifying the calling function, then the last token is not in the token chain when calling cxxParserCleanupEnumStructClassOrUnionPrefixChain(). This function removes all of the prefix tokens (since none of them are const or volatile) and then sets pToken to the first token, which has just been freed. Then in the caller, cxxParserParseClassStructOrUnionInternal(), the next line checks if g_cxx.pToken is a semicolon, but the token is now pointing at freed memory (as all prefix tokens have been freed with this particular invalid input), so AddressSanitizer prints an error and closes the program.

Found with AFL++ running Docker CE running in openSUSE on Windows Subsystem for Linux running on a Windows 10 host.

This is the third bug I found with AFL++ just in the C++ parser part of u-ctags.

Reproducible: happens every time with AddressSanitizer enabled. The
bug is not detected when running without AddressSanitizer
instrumentation.

u-ctags version: p6.2.20251130.0-2-g6858451dc
AFL version: afl-clang-lto
Clang version: 19.1.7
AFL++ version: 4.35a
Configuration: (export CC=afl-clang-lto CXX=afl-clang-lto++ CFLAGS='-O3 -ffast-math -fno-finite-math-only -g -fno-omit-frame-pointer -mtune=skylake' AFL_USE_ASAN=1;time dash ./configure && time make -j6)
uname -a: Linux MSI 6.6.87.2-microsoft-standard-WSL2 #1 SMP PREEMPT_DYNAMIC Thu Jun 5 18:30:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Hardware: MSI GS63 Stealth 8RE

Host: Windows 10 22H2 (OS Build 19045.6691) x86_64-v3
WSL distro: openSUSE 16.0
Windows Terminal version: 1.23.13503.0
Docker CE version: 28.3.2-ce, build e77ff99ed
Docker image: custom build of aflplusplus Dockerfile, I think(?).
WSL version:

WSL version: 2.6.3.0
Kernel version: 6.6.87.2-1
WSLg version: 1.0.71
MSRDC version: 1.2.6353
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.26100.1-240331-1435.ge-release
Windows version: 10.0.19045.6691

AddressSanitizer crash and gdb backtrace

> ./ctags ./crash_264.cpp
=================================================================
==898544==ERROR: AddressSanitizer: heap-use-after-free on address 0x7caff5da73c0 at pc 0x000000506698 bp 0x7fffffffd420 sp 0x7fffffffd418
READ of size 4 at 0x7caff5da73c0 thread T0
    #0 0x000000506697 in cxxParserParseClassStructOrUnionInternal parsers/cxx/cxx_parser.c:1203
    #1 0x000000506697 in cxxParserParseClassStructOrUnion parsers/cxx/cxx_parser.c:1512
    #2 0x00000050c382 in cxxParserParseBlockInternal parsers/cxx/cxx_parser_block.c:431
    #3 0x00000050e718 in cxxParserParseBlockFull parsers/cxx/cxx_parser_block.c:893
    #4 0x00000050e718 in cxxParserParseBlock parsers/cxx/cxx_parser_block.c:904
    #5 0x0000004ff437 in cxxParserMain parsers/cxx/cxx_parser.c:2019
    #6 0x0000005084ad in cxxCppParserMain parsers/cxx/cxx_parser.c:2078
    #7 0x000000445481 in createTagsForFile main/parse.c:4395
    #8 0x000000445481 in createTagsWithFallback1 main/parse.c:4521
    #9 0x000000445f7e in createTagsWithFallback main/parse.c:4612
    #10 0x000000445f7e in parseMio main/parse.c:4774
    #11 0x00000045ee13 in parseFileWithMio main/parse.c:4821
    #12 0x00000045f53b in parseFile main/parse.c:4757
    #13 0x00000041caff in createTagsForEntry main/main.c:221
    #14 0x00000041d1cf in createTagsForArgs main/main.c:266
    #15 0x00000041d1cf in batchMakeTags main/main.c:360
    #16 0x00000041da74 in runMainLoop main/main.c:332
    #17 0x00000041da74 in ctags_cli_main main/main.c:586
    #18 0x7ffff6c3033f in __libc_start_call_main (/lib64/libc.so.6+0x2a33f) (BuildId: 41d443e8a12973d20e705f4efb33ee3bed03f727)
    #19 0x7ffff6c30408 in __libc_start_main_alias_1 (/lib64/libc.so.6+0x2a408) (BuildId: 41d443e8a12973d20e705f4efb33ee3bed03f727)
    #20 0x00000041c534 in _start ../sysdeps/x86_64/start.S:115

0x7caff5da73c0 is located 0 bytes inside of 104-byte region [0x7caff5da73c0,0x7caff5da7428)
freed by thread T0 here:
    #0 0x7ffff792168b  (/lib64/libasan.so.8+0x12168b) (BuildId: 0ca023704e42220944eaf9284f2795a514ab2fed)
    #1 0x0000004ff1ac in cxxParserCleanupEnumStructClassOrUnionPrefixChain parsers/cxx/cxx_parser.c:485
    #2 0x000000505015 in cxxParserParseClassStructOrUnionInternal parsers/cxx/cxx_parser.c:1201
    #3 0x000000505015 in cxxParserParseClassStructOrUnion parsers/cxx/cxx_parser.c:1512
    #4 0x00000050c382 in cxxParserParseBlockInternal parsers/cxx/cxx_parser_block.c:431
    #5 0x00000050e718 in cxxParserParseBlockFull parsers/cxx/cxx_parser_block.c:893
    #6 0x00000050e718 in cxxParserParseBlock parsers/cxx/cxx_parser_block.c:904
    #7 0x0000004ff437 in cxxParserMain parsers/cxx/cxx_parser.c:2019
    #8 0x0000005084ad in cxxCppParserMain parsers/cxx/cxx_parser.c:2078
    #9 0x000000445481 in createTagsForFile main/parse.c:4395
    #10 0x000000445481 in createTagsWithFallback1 main/parse.c:4521
    #11 0x000000445f7e in createTagsWithFallback main/parse.c:4612
    #12 0x000000445f7e in parseMio main/parse.c:4774
    #13 0x00000045ee13 in parseFileWithMio main/parse.c:4821
    #14 0x00000045f53b in parseFile main/parse.c:4757
    #15 0x00000041caff in createTagsForEntry main/main.c:221
    #16 0x00000041d1cf in createTagsForArgs main/main.c:266
    #17 0x00000041d1cf in batchMakeTags main/main.c:360
    #18 0x00000041da74 in runMainLoop main/main.c:332
    #19 0x00000041da74 in ctags_cli_main main/main.c:586
    #20 0x7ffff6c3033f in __libc_start_call_main (/lib64/libc.so.6+0x2a33f) (BuildId: 41d443e8a12973d20e705f4efb33ee3bed03f727)

previously allocated by thread T0 here:
    #0 0x7ffff79229cb in malloc (/lib64/libasan.so.8+0x1229cb) (BuildId: 0ca023704e42220944eaf9284f2795a514ab2fed)
    #1 0x000000b66f00 in eMalloc main/routines.c:220
    #2 0x0000005546d2 in createToken parsers/cxx/cxx_token.c:31
    #3 0x000000be3ff6 in objPoolGet main/objpool.c:66
    #4 0x00000052edc4 in cxxParserParseNextToken parsers/cxx/cxx_parser_tokenizer.c:1287
    #5 0x0000004ff632 in cxxParserParseAndCondenseSubchainsUpToOneOf parsers/cxx/cxx_parser.c:202
    #6 0x0000005045de in cxxParserParseUpToOneOf parsers/cxx/cxx_parser.c:339
    #7 0x0000005045de in cxxParserParseClassStructOrUnionInternal parsers/cxx/cxx_parser.c:1082
    #8 0x0000005045de in cxxParserParseClassStructOrUnion parsers/cxx/cxx_parser.c:1512
    #9 0x00000050c382 in cxxParserParseBlockInternal parsers/cxx/cxx_parser_block.c:431
    #10 0x00000050e718 in cxxParserParseBlockFull parsers/cxx/cxx_parser_block.c:893
    #11 0x00000050e718 in cxxParserParseBlock parsers/cxx/cxx_parser_block.c:904
    #12 0x0000004ff437 in cxxParserMain parsers/cxx/cxx_parser.c:2019
    #13 0x0000005084ad in cxxCppParserMain parsers/cxx/cxx_parser.c:2078
    #14 0x000000445481 in createTagsForFile main/parse.c:4395
    #15 0x000000445481 in createTagsWithFallback1 main/parse.c:4521
    #16 0x000000445f7e in createTagsWithFallback main/parse.c:4612
    #17 0x000000445f7e in parseMio main/parse.c:4774
    #18 0x00000045ee13 in parseFileWithMio main/parse.c:4821
    #19 0x00000045f53b in parseFile main/parse.c:4757
    #20 0x00000041caff in createTagsForEntry main/main.c:221
    #21 0x00000041d1cf in createTagsForArgs main/main.c:266
    #22 0x00000041d1cf in batchMakeTags main/main.c:360
    #23 0x00000041da74 in runMainLoop main/main.c:332
    #24 0x00000041da74 in ctags_cli_main main/main.c:586
    #25 0x7ffff6c3033f in __libc_start_call_main (/lib64/libc.so.6+0x2a33f) (BuildId: 41d443e8a12973d20e705f4efb33ee3bed03f727)

SUMMARY: AddressSanitizer: heap-use-after-free parsers/cxx/cxx_parser.c:1203 in cxxParserParseClassStructOrUnionInternal
Shadow bytes around the buggy address:
  0x7caff5da7100: fd fd fd fd fd fd fd fd fd fd fd fd fd fa fa fa
  0x7caff5da7180: fa fa fa fa fa fa fd fd fd fd fd fd fd fd fd fd
  0x7caff5da7200: fd fd fd fa fa fa fa fa fa fa fa fa 00 00 00 00
  0x7caff5da7280: 00 00 00 00 00 00 00 00 00 fa fa fa fa fa fa fa
  0x7caff5da7300: fa fa 00 00 00 00 00 00 00 00 00 00 00 00 00 fa
=>0x7caff5da7380: fa fa fa fa fa fa fa fa[fd]fd fd fd fd fd fd fd
  0x7caff5da7400: fd fd fd fd fd fa fa fa fa fa fa fa fa fa fa fa
  0x7caff5da7480: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x7caff5da7500: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x7caff5da7580: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x7caff5da7600: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==898544==ABORTING

Program received signal SIGABRT, Aborted.
0x00007ffff6c9a45c in __pthread_kill_implementation () from /lib64/libc.so.6
Missing separate debuginfos, use: zypper install glibc-debuginfo-2.40-160000.2.2.x86_64 libasan8-debuginfo-15.1.1+git9973-160000.2.2.x86_64 libxml2-2-debuginfo-2.13.8-160000.2.2.x86_64 libyaml-0-2-debuginfo-0.2.5-160000.3.2.x86_64 libpcre2-8-0-debuginfo-10.45-160000.2.2.x86_64 libubsan1-debuginfo-15.1.1+git9973-160000.2.2.x86_64 libstdc++6-debuginfo-15.1.1+git9973-160000.2.2.x86_64 gcc15-debuginfo-15.1.1+git9973-160000.2.2.x86_64 libz1-debuginfo-1.2.13-160000.2.2.x86_64 liblzma5-debuginfo-5.8.1-160000.2.2.x86_64
(gdb) bt
#0  0x00007ffff6c9a45c in __pthread_kill_implementation () from /lib64/libc.so.6
#1  0x00007ffff6c46fc6 in raise () from /lib64/libc.so.6
#2  0x00007ffff6c2e917 in abort () from /lib64/libc.so.6
#3  0x00007ffff7944d5f in ?? () from /lib64/libasan.so.8
#4  0x00007ffff7955a4c in ?? () from /lib64/libasan.so.8
#5  0x00007ffff792e5b1 in ?? () from /lib64/libasan.so.8
#6  0x00007ffff792db01 in ?? () from /lib64/libasan.so.8
#7  0x00007ffff792ed9c in __asan_report_load4 () from /lib64/libasan.so.8
#8  0x0000000000506698 in cxxParserParseClassStructOrUnionInternal (eKeyword=CXXKeywordCLASS, uTagKind=16,
    uScopeType=2) at parsers/cxx/cxx_parser.c:1203
#9  cxxParserParseClassStructOrUnion (eKeyword=eKeyword@entry=CXXKeywordCLASS, uTagKind=uTagKind@entry=16,
    uScopeType=uScopeType@entry=2) at parsers/cxx/cxx_parser.c:1512
#10 0x000000000050c383 in cxxParserParseBlockInternal (bExpectClosingBracket=bExpectClosingBracket@entry=false,
    bExported=bExported@entry=false) at parsers/cxx/cxx_parser_block.c:431
#11 0x000000000050e719 in cxxParserParseBlockFull (bExpectClosingBracket=false, bExported=false)
    at parsers/cxx/cxx_parser_block.c:893
#12 cxxParserParseBlock (bExpectClosingBracket=bExpectClosingBracket@entry=false) at parsers/cxx/cxx_parser_block.c:904
#13 0x00000000004ff438 in cxxParserMain (passCount=passCount@entry=1) at parsers/cxx/cxx_parser.c:2019
#14 0x00000000005084ae in cxxCppParserMain (passCount=1) at parsers/cxx/cxx_parser.c:2078
#15 0x0000000000445482 in createTagsForFile (language=-173136832, passCount=1) at main/parse.c:4395
#16 createTagsWithFallback1 (language=language@entry=23, exclusive_subparser=exclusive_subparser@entry=0x7bfff48fabc0)
    at main/parse.c:4521
#17 0x0000000000445f7f in createTagsWithFallback (fileName=0x7c2ff5ae0250 "./crash_264-2026-01-02.cpp", language=23,
    mio=<optimized out>, mtime=<optimized out>, failureInOpenning=0x7bfff48fabb0) at main/parse.c:4612
#18 parseMio (fileName=fileName@entry=0x7c2ff5ae0250 "./crash_264-2026-01-02.cpp", language=language@entry=23,
    mio=<optimized out>, mtime=<optimized out>, useSourceFileTagPath=useSourceFileTagPath@entry=true,
    clientData=clientData@entry=0x0) at main/parse.c:4774
#19 0x000000000045ee14 in parseFileWithMio (fileName=fileName@entry=0x7c2ff5ae0250 "./crash_264-2026-01-02.cpp",
    mio=mio@entry=0x0, clientData=clientData@entry=0x0) at main/parse.c:4821
#20 0x000000000045f53c in parseFile (fileName=fileName@entry=0x7c2ff5ae0250 "./crash_264-2026-01-02.cpp")
    at main/parse.c:4757
#21 0x000000000041cb00 in createTagsForEntry (entryName=0x7c2ff5ae0250 "./crash_264-2026-01-02.cpp") at main/main.c:221
#22 0x000000000041d1d0 in createTagsForArgs (args=0x7c3ff5ae0010) at main/main.c:266
#23 batchMakeTags (args=0x7c3ff5ae0010, user=<optimized out>) at main/main.c:360
#24 0x000000000041da75 in runMainLoop (args=0x7c3ff5ae0010) at main/main.c:332
#25 ctags_cli_main (argc=<optimized out>, argv=0x7fffffffdb30) at main/main.c:586
#26 0x00007ffff6c30340 in __libc_start_call_main () from /lib64/libc.so.6
#27 0x00007ffff6c30409 in __libc_start_main_impl () from /lib64/libc.so.6
#28 0x000000000041c535 in _start () at ../sysdeps/x86_64/start.S:115
(gdb) frame 8
#8  0x0000000000506698 in cxxParserParseClassStructOrUnionInternal (eKeyword=CXXKeywordCLASS, uTagKind=16,
    uScopeType=2) at parsers/cxx/cxx_parser.c:1203
1203            if(cxxTokenTypeIs(g_cxx.pToken,CXXTokenTypeSemicolon))
(gdb) p iInitialTokenCount
$1 = 16384

Here's where parseNextToken() is called:

#0  cxxParserParseNextToken () at parsers/cxx/cxx_parser_tokenizer.c:1273
#1  0x00000000004ff693 in cxxParserParseAndCondenseSubchainsUpToOneOf (uTokenTypes=uTokenTypes@entry=273154133,
    uInitialSubchainMarkerTypes=uInitialSubchainMarkerTypes@entry=3670016,
    bCanReduceInnerElements=bCanReduceInnerElements@entry=false) at parsers/cxx/cxx_parser.c:202
#2  0x000000000050463f in cxxParserParseUpToOneOf (uTokenTypes=273154133, bCanReduceInnerElements=false)
    at parsers/cxx/cxx_parser.c:339
#3  cxxParserParseClassStructOrUnionInternal (eKeyword=CXXKeywordCLASS, uTagKind=16, uScopeType=2)
    at parsers/cxx/cxx_parser.c:1082
#4  cxxParserParseClassStructOrUnion (eKeyword=eKeyword@entry=CXXKeywordCLASS, uTagKind=uTagKind@entry=16,
    uScopeType=uScopeType@entry=2) at parsers/cxx/cxx_parser.c:1511
#5  0x000000000050c3d3 in cxxParserParseBlockInternal (bExpectClosingBracket=bExpectClosingBracket@entry=false,
    bExported=bExported@entry=false) at parsers/cxx/cxx_parser_block.c:431
#6  0x000000000050e769 in cxxParserParseBlockFull (bExpectClosingBracket=false, bExported=false)
    at parsers/cxx/cxx_parser_block.c:893
#7  cxxParserParseBlock (bExpectClosingBracket=bExpectClosingBracket@entry=false) at parsers/cxx/cxx_parser_block.c:904
#8  0x00000000004ff498 in cxxParserMain (passCount=passCount@entry=1) at parsers/cxx/cxx_parser.c:2018
#9  0x00000000005084fe in cxxCppParserMain (passCount=1) at parsers/cxx/cxx_parser.c:2077
#10 0x0000000000445472 in createTagsForFile (language=1783505984, passCount=1) at main/parse.c:4396
#11 createTagsWithFallback1 (language=language@entry=23, exclusive_subparser=exclusive_subparser@entry=0x7a97692fabc0)
    at main/parse.c:4522
#12 0x0000000000445f6f in createTagsWithFallback (fileName=0x7ad76a4e0090 "./crashers/crash_264-2026-01-02.cpp",
    language=23, mio=<optimized out>, mtime=<optimized out>, failureInOpenning=0x7a97692fabb0) at main/parse.c:4613
#13 parseMio (fileName=fileName@entry=0x7ad76a4e0090 "./crashers/crash_264-2026-01-02.cpp",
    language=language@entry=23, mio=<optimized out>, mtime=<optimized out>,
    useSourceFileTagPath=useSourceFileTagPath@entry=true, clientData=clientData@entry=0x0) at main/parse.c:4775
#14 0x000000000045ee04 in parseFileWithMio (
    fileName=fileName@entry=0x7ad76a4e0090 "./crashers/crash_264-2026-01-02.cpp", mio=mio@entry=0x0,
    clientData=clientData@entry=0x0) at main/parse.c:4824
#15 0x000000000045f58c in parseFile (fileName=fileName@entry=0x7ad76a4e0090 "./crashers/crash_264-2026-01-02.cpp")
--Type <RET> for more, q to quit, c to continue without paging--c
    at main/parse.c:4758
#16 0x000000000041cb10 in createTagsForEntry (entryName=0x7ad76a4e0090 "./crashers/crash_264-2026-01-02.cpp")
    at main/main.c:221
#17 0x000000000041d1e0 in createTagsForArgs (args=0x7ad76a4e0010) at main/main.c:266
#18 batchMakeTags (args=0x7ad76a4e0010, user=<optimized out>) at main/main.c:360
#19 0x000000000041da85 in runMainLoop (args=0x7ad76a4e0010) at main/main.c:332
#20 ctags_cli_main (argc=<optimized out>, argv=0x7fffa08f6080) at main/main.c:586
#21 0x00007e976b630340 in __libc_start_call_main () from /lib64/libc.so.6
#22 0x00007e976b630409 in __libc_start_main_impl () from /lib64/libc.so.6
#23 0x000000000041c545 in _start () at ../sysdeps/x86_64/start.S:115

PATCH

Since cxxParserParseUpToOneOf() can free the last token, we need to refresh the pLastToken pointer before cleaning up the prefix chain.

diff --git a/parsers/cxx/cxx_parser.c b/parsers/cxx/cxx_parser.c
index 393ef2e86..cf40d3dfb 100644
--- a/parsers/cxx/cxx_parser.c
+++ b/parsers/cxx/cxx_parser.c
@@ -768,7 +768,10 @@ bool cxxParserParseEnum(void)
        }

        if(iInitialTokenCount > 1)
+       {
+               pLastToken = cxxTokenChainLast(g_cxx.pTokenChain);
                cxxParserCleanupEnumStructClassOrUnionPrefixChain(CXXKeywordENUM,pLastToken);
+       }

        if(cxxTokenTypeIs(g_cxx.pToken,CXXTokenTypeSemicolon))
        {
@@ -1197,7 +1200,10 @@ static bool cxxParserParseClassStructOrUnionInternal(
        }

        if(iInitialTokenCount > 1)
+       {
+               pLastToken = cxxTokenChainLast(g_cxx.pTokenChain);
                cxxParserCleanupEnumStructClassOrUnionPrefixChain(eKeyword,pLastToken);
+       }

        if(cxxTokenTypeIs(g_cxx.pToken,CXXTokenTypeSemicolon))
        {