[clang-format] Correctly annotate token-pasted function decl names

[owenca]

Fix #142178

[llvmbot]

@llvm/pr-subscribers-clang-format

Author: Owen Pan (owenca)

Changes

Fix #142178

---

Full diff: https://github.com/llvm/llvm-project/pull/142337.diff

2 Files Affected:

• (modified) clang/lib/Format/TokenAnnotator.cpp (+2)
• (modified) clang/unittests/Format/TokenAnnotatorTest.cpp (+7)

diff --git a/clang/lib/Format/TokenAnnotator.cpp b/clang/lib/Format/TokenAnnotator.cpp
index da1b6dd5254b8..da279d07b5918 100644
--- a/clang/lib/Format/TokenAnnotator.cpp
+++ b/clang/lib/Format/TokenAnnotator.cpp
@@ -3848,6 +3848,8 @@ static bool isFunctionDeclarationName(const LangOptions &LangOpts,
   } else {
     if (Current.isNot(TT_StartOfName) || Current.NestingLevel != 0)
       return false;
+    while (Next && Next->startsSequence(tok::hashhash, tok::identifier))
+      Next = Next->Next->Next;
     for (; Next; Next = Next->Next) {
       if (Next->is(TT_TemplateOpener) && Next->MatchingParen) {
         Next = Next->MatchingParen;
diff --git a/clang/unittests/Format/TokenAnnotatorTest.cpp b/clang/unittests/Format/TokenAnnotatorTest.cpp
index 1a5ed4b9040c2..9637085c70502 100644
--- a/clang/unittests/Format/TokenAnnotatorTest.cpp
+++ b/clang/unittests/Format/TokenAnnotatorTest.cpp
@@ -2257,6 +2257,13 @@ TEST_F(TokenAnnotatorTest, UnderstandsFunctionDeclarationNames) {
   EXPECT_TOKEN(Tokens[1], tok::identifier, TT_FunctionDeclarationName);
   EXPECT_TOKEN(Tokens[2], tok::l_paren, TT_FunctionDeclarationLParen);
 
+  Tokens = annotate("#define FUNC(foo, bar) \\\n"
+                    "  auto foo##bar() -> Type {}");
+  ASSERT_EQ(Tokens.size(), 19u) << Tokens;
+  EXPECT_TOKEN(Tokens[9], tok::identifier, TT_FunctionDeclarationName);
+  EXPECT_TOKEN(Tokens[12], tok::l_paren, TT_FunctionDeclarationLParen);
+  EXPECT_TOKEN(Tokens[14], tok::arrow, TT_TrailingReturnArrow);
+
   Tokens = annotate("int iso_time(time_t);");
   ASSERT_EQ(Tokens.size(), 7u) << Tokens;
   EXPECT_TOKEN(Tokens[1], tok::identifier, TT_FunctionDeclarationName);

[prj-]

Looking forward to testing this out!

[owenca]

Looking forward to testing this out!

I already tested it with your reproducer in #142251 (comment).

[prj-]

Unfortunately, we have many other use-cases for triggering regressions in our code (hence why we are having this discussion in the first place 😅), but hopefully all will be good afterwards.

[owenca]

@prj- I understand. If you use AlignFunctionDeclarations, you might see formatting changes as the first token in a token-pasted function decl name was not annotated correctly before this patch.

[owenca]

/cherry-pick 7bf5862

[llvmbot]

/pull-request #142482