Skip to content

[Clang] Unify interface for accessing template arguments as written for class/variable template specializations #81642

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
May 7, 2024

Conversation

sdkrystian
Copy link
Member

@sdkrystian sdkrystian commented Feb 13, 2024

Our current method of storing the template arguments as written for (Class/Var)Template(Partial)SpecializationDecl suffers from a number of flaws:

  • We use TypeSourceInfo to store TemplateArgumentLocs for class template/variable template partial/explicit specializations. For variable template specializations, this is a rather unintuitive hack (as we store a non-type specialization as a type). Moreover, we don't ever need the type as written -- in almost all cases, we only want the template arguments (e.g. in tooling use-cases).
  • The template arguments as written are stored in a number of redundant data members. For example, (Class/Var)TemplatePartialSpecialization have their own ArgsAsWritten member that stores an ASTTemplateArgumentListInfo (the template arguments). VarTemplateSpecializationDecl has yet another redundant member "TemplateArgsInfo" that also stores an ASTTemplateArgumentListInfo.

This patch eliminates all (Class/Var)Template(Partial)SpecializationDecl members which store the template arguments as written, and turns the ExplicitInfo member into a llvm::PointerUnion<const ASTTemplateArgumentListInfo*, ExplicitInstantiationInfo*> (to avoid unnecessary allocations when the declaration isn't an explicit instantiation). The template arguments as written are now accessed via getTemplateArgsWritten in all cases.

This patch is near-complete. I still need to eliminate any instances of the template keyword location being stored for non-explicit-instantiation declarations (since we already store the outer template parameter lists). With respect to the clang Index tests, I still need to update the tests such that we check for references to entities in the explicit template argument list of specializations (UPDATE: now done).

The "most breaking" change is to AST Matchers, insofar that hasTypeLoc will no longer match class template specializations (since they no longer store the type as written). I wasn't entirely sure what to replace the tests for this behavior with, so I just commented them out... guidance/feedback would be appreciated on this.

@sdkrystian sdkrystian requested a review from Endilll as a code owner February 13, 2024 18:18
@llvmbot llvmbot added clang Clang issues not falling into any other category clang:frontend Language frontend issues, e.g. anything involving "Sema" clang:modules C++20 modules and Clang Header Modules labels Feb 13, 2024
@llvmbot
Copy link
Member

llvmbot commented Feb 13, 2024

@llvm/pr-subscribers-clang-tools-extra
@llvm/pr-subscribers-clangd
@llvm/pr-subscribers-clang-modules

@llvm/pr-subscribers-clang

Author: Krystian Stasiowski (sdkrystian)

Changes

Our current method of storing the template arguments as written for (Class/Var)Template(Partial)SpecializationDecl suffers from a number of flaws:

  • We use TypeSourceInfo to store TemplateArgumentLocs for class template/variable template partial/explicit specializations. For variable template specializations, this is a rather unintuitive hack (as we store a non-type specialization as a type). Moreover, we don't ever need the type as written -- in almost all cases, we only want the template arguments (e.g. in tooling use-cases).
  • The template arguments as written are stored in a number of redundant data members. For example, (Class/Var)TemplatePartialSpecialization have their own ArgsAsWritten member that stores an ASTTemplateArgumentListInfo (the template arguments). VarTemplateSpecializationDecl has yet another redundant member "TemplateArgsInfo" that also stores an ASTTemplateArgumentListInfo.

This patch eliminates all (Class/Var)Template(Partial)SpecializationDecl members which store the template arguments as written, and turns the ExplicitInfo member into a llvm::PointerUnion&lt;const ASTTemplateArgumentListInfo*, ExplicitInstantiationInfo*&gt; (to avoid unnecessary allocations when the declaration isn't an explicit instantiation). The template arguments as written are now accessed via getTemplateArgsWritten in all cases.

This patch is near-complete. I still need to eliminate any instances of the template keyword location being stored for non-explicit-instantiation declarations (since we already store the outer template parameter lists). With respect to the clang Index tests, I still need to update the tests such that we check for references to entities in the explicit template argument list of specializations.

The "most breaking" change is to AST Matchers, insofar that hasTypeLoc will no longer match class template specializations (since they no longer store the type as written). I wasn't entirely sure what to replace the tests for this behavior with, so I just commented them out... guidance/feedback would be appreciated on this.


Patch is 73.66 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/81642.diff

21 Files Affected:

  • (modified) clang/include/clang/AST/DeclTemplate.h (+78-108)
  • (modified) clang/include/clang/AST/RecursiveASTVisitor.h (+15-12)
  • (modified) clang/include/clang/ASTMatchers/ASTMatchers.h (+3-4)
  • (modified) clang/include/clang/ASTMatchers/ASTMatchersInternal.h (-4)
  • (modified) clang/lib/AST/ASTImporter.cpp (+23-32)
  • (modified) clang/lib/AST/DeclPrinter.cpp (+7-8)
  • (modified) clang/lib/AST/DeclTemplate.cpp (+85-60)
  • (modified) clang/lib/AST/TypePrinter.cpp (+10-14)
  • (modified) clang/lib/Index/IndexDecl.cpp (+6-3)
  • (modified) clang/lib/Sema/Sema.cpp (+1-1)
  • (modified) clang/lib/Sema/SemaTemplate.cpp (+19-28)
  • (modified) clang/lib/Sema/SemaTemplateInstantiateDecl.cpp (+58-97)
  • (modified) clang/lib/Serialization/ASTReaderDecl.cpp (+12-8)
  • (modified) clang/lib/Serialization/ASTWriterDecl.cpp (+10-6)
  • (modified) clang/test/AST/ast-dump-template-decls.cpp (+7-11)
  • (modified) clang/test/CXX/drs/dr7xx.cpp (+1-1)
  • (modified) clang/test/Index/Core/index-source.cpp (-11)
  • (modified) clang/test/Index/index-refs.cpp (-1)
  • (modified) clang/tools/libclang/CIndex.cpp (+14-15)
  • (modified) clang/unittests/ASTMatchers/ASTMatchersNodeTest.cpp (-12)
  • (modified) clang/unittests/ASTMatchers/ASTMatchersTraversalTest.cpp (+12-6)
diff --git a/clang/include/clang/AST/DeclTemplate.h b/clang/include/clang/AST/DeclTemplate.h
index e3b6a7efb1127a..b5c2d459715120 100644
--- a/clang/include/clang/AST/DeclTemplate.h
+++ b/clang/include/clang/AST/DeclTemplate.h
@@ -1792,10 +1792,9 @@ class ClassTemplateSpecializationDecl
   llvm::PointerUnion<ClassTemplateDecl *, SpecializedPartialSpecialization *>
     SpecializedTemplate;
 
-  /// Further info for explicit template specialization/instantiation.
-  struct ExplicitSpecializationInfo {
-    /// The type-as-written.
-    TypeSourceInfo *TypeAsWritten = nullptr;
+  struct ExplicitInstantiationInfo {
+    /// The template arguments as written..
+    const ASTTemplateArgumentListInfo *TemplateArgsAsWritten = nullptr;
 
     /// The location of the extern keyword.
     SourceLocation ExternLoc;
@@ -1803,12 +1802,14 @@ class ClassTemplateSpecializationDecl
     /// The location of the template keyword.
     SourceLocation TemplateKeywordLoc;
 
-    ExplicitSpecializationInfo() = default;
+    ExplicitInstantiationInfo() = default;
   };
 
   /// Further info for explicit template specialization/instantiation.
   /// Does not apply to implicit specializations.
-  ExplicitSpecializationInfo *ExplicitInfo = nullptr;
+  llvm::PointerUnion<const ASTTemplateArgumentListInfo *,
+                     ExplicitInstantiationInfo *>
+      ExplicitInfo = nullptr;
 
   /// The template arguments used to describe this specialization.
   const TemplateArgumentList *TemplateArgs;
@@ -1985,44 +1986,45 @@ class ClassTemplateSpecializationDecl
     SpecializedTemplate = TemplDecl;
   }
 
-  /// Sets the type of this specialization as it was written by
-  /// the user. This will be a class template specialization type.
-  void setTypeAsWritten(TypeSourceInfo *T) {
-    if (!ExplicitInfo)
-      ExplicitInfo = new (getASTContext()) ExplicitSpecializationInfo;
-    ExplicitInfo->TypeAsWritten = T;
-  }
-
-  /// Gets the type of this specialization as it was written by
-  /// the user, if it was so written.
-  TypeSourceInfo *getTypeAsWritten() const {
-    return ExplicitInfo ? ExplicitInfo->TypeAsWritten : nullptr;
+  const ASTTemplateArgumentListInfo *getTemplateArgsAsWritten() const {
+    if (auto *Info = ExplicitInfo.dyn_cast<ExplicitInstantiationInfo *>())
+      return Info->TemplateArgsAsWritten;
+    return ExplicitInfo.get<const ASTTemplateArgumentListInfo *>();
   }
 
   /// Gets the location of the extern keyword, if present.
   SourceLocation getExternLoc() const {
-    return ExplicitInfo ? ExplicitInfo->ExternLoc : SourceLocation();
+    if (auto *Info = ExplicitInfo.dyn_cast<ExplicitInstantiationInfo *>())
+      return Info->ExternLoc;
+    return SourceLocation();
   }
 
-  /// Sets the location of the extern keyword.
-  void setExternLoc(SourceLocation Loc) {
-    if (!ExplicitInfo)
-      ExplicitInfo = new (getASTContext()) ExplicitSpecializationInfo;
-    ExplicitInfo->ExternLoc = Loc;
+  /// Gets the location of the template keyword, if present.
+  SourceLocation getTemplateKeywordLoc() const {
+    if (auto *Info = ExplicitInfo.dyn_cast<ExplicitInstantiationInfo *>())
+      return Info->TemplateKeywordLoc;
+    return SourceLocation();
   }
 
-  /// Sets the location of the template keyword.
-  void setTemplateKeywordLoc(SourceLocation Loc) {
-    if (!ExplicitInfo)
-      ExplicitInfo = new (getASTContext()) ExplicitSpecializationInfo;
-    ExplicitInfo->TemplateKeywordLoc = Loc;
+  void
+  setTemplateArgsAsWritten(const ASTTemplateArgumentListInfo *ArgsWritten) {
+    if (auto *Info = ExplicitInfo.dyn_cast<ExplicitInstantiationInfo *>())
+      Info->TemplateArgsAsWritten = ArgsWritten;
+    else
+      ExplicitInfo = ArgsWritten;
   }
 
-  /// Gets the location of the template keyword, if present.
-  SourceLocation getTemplateKeywordLoc() const {
-    return ExplicitInfo ? ExplicitInfo->TemplateKeywordLoc : SourceLocation();
+  void setTemplateArgsAsWritten(const TemplateArgumentListInfo &ArgsInfo) {
+    setTemplateArgsAsWritten(
+        ASTTemplateArgumentListInfo::Create(getASTContext(), ArgsInfo));
   }
 
+  /// Sets the location of the extern keyword.
+  void setExternLoc(SourceLocation Loc);
+
+  /// Sets the location of the template keyword.
+  void setTemplateKeywordLoc(SourceLocation Loc);
+
   SourceRange getSourceRange() const override LLVM_READONLY;
 
   void Profile(llvm::FoldingSetNodeID &ID) const {
@@ -2050,10 +2052,6 @@ class ClassTemplatePartialSpecializationDecl
   /// The list of template parameters
   TemplateParameterList* TemplateParams = nullptr;
 
-  /// The source info for the template arguments as written.
-  /// FIXME: redundant with TypeAsWritten?
-  const ASTTemplateArgumentListInfo *ArgsAsWritten = nullptr;
-
   /// The class template partial specialization from which this
   /// class template partial specialization was instantiated.
   ///
@@ -2062,15 +2060,11 @@ class ClassTemplatePartialSpecializationDecl
   llvm::PointerIntPair<ClassTemplatePartialSpecializationDecl *, 1, bool>
       InstantiatedFromMember;
 
-  ClassTemplatePartialSpecializationDecl(ASTContext &Context, TagKind TK,
-                                         DeclContext *DC,
-                                         SourceLocation StartLoc,
-                                         SourceLocation IdLoc,
-                                         TemplateParameterList *Params,
-                                         ClassTemplateDecl *SpecializedTemplate,
-                                         ArrayRef<TemplateArgument> Args,
-                               const ASTTemplateArgumentListInfo *ArgsAsWritten,
-                               ClassTemplatePartialSpecializationDecl *PrevDecl);
+  ClassTemplatePartialSpecializationDecl(
+      ASTContext &Context, TagKind TK, DeclContext *DC, SourceLocation StartLoc,
+      SourceLocation IdLoc, TemplateParameterList *Params,
+      ClassTemplateDecl *SpecializedTemplate, ArrayRef<TemplateArgument> Args,
+      ClassTemplatePartialSpecializationDecl *PrevDecl);
 
   ClassTemplatePartialSpecializationDecl(ASTContext &C)
     : ClassTemplateSpecializationDecl(C, ClassTemplatePartialSpecialization),
@@ -2085,11 +2079,8 @@ class ClassTemplatePartialSpecializationDecl
   static ClassTemplatePartialSpecializationDecl *
   Create(ASTContext &Context, TagKind TK, DeclContext *DC,
          SourceLocation StartLoc, SourceLocation IdLoc,
-         TemplateParameterList *Params,
-         ClassTemplateDecl *SpecializedTemplate,
-         ArrayRef<TemplateArgument> Args,
-         const TemplateArgumentListInfo &ArgInfos,
-         QualType CanonInjectedType,
+         TemplateParameterList *Params, ClassTemplateDecl *SpecializedTemplate,
+         ArrayRef<TemplateArgument> Args, QualType CanonInjectedType,
          ClassTemplatePartialSpecializationDecl *PrevDecl);
 
   static ClassTemplatePartialSpecializationDecl *
@@ -2120,11 +2111,6 @@ class ClassTemplatePartialSpecializationDecl
     return TemplateParams->hasAssociatedConstraints();
   }
 
-  /// Get the template arguments as written.
-  const ASTTemplateArgumentListInfo *getTemplateArgsAsWritten() const {
-    return ArgsAsWritten;
-  }
-
   /// Retrieve the member class template partial specialization from
   /// which this particular class template partial specialization was
   /// instantiated.
@@ -2596,10 +2582,9 @@ class VarTemplateSpecializationDecl : public VarDecl,
   llvm::PointerUnion<VarTemplateDecl *, SpecializedPartialSpecialization *>
   SpecializedTemplate;
 
-  /// Further info for explicit template specialization/instantiation.
-  struct ExplicitSpecializationInfo {
-    /// The type-as-written.
-    TypeSourceInfo *TypeAsWritten = nullptr;
+  struct ExplicitInstantiationInfo {
+    /// The template arguments as written..
+    const ASTTemplateArgumentListInfo *TemplateArgsAsWritten = nullptr;
 
     /// The location of the extern keyword.
     SourceLocation ExternLoc;
@@ -2607,12 +2592,14 @@ class VarTemplateSpecializationDecl : public VarDecl,
     /// The location of the template keyword.
     SourceLocation TemplateKeywordLoc;
 
-    ExplicitSpecializationInfo() = default;
+    ExplicitInstantiationInfo() = default;
   };
 
   /// Further info for explicit template specialization/instantiation.
   /// Does not apply to implicit specializations.
-  ExplicitSpecializationInfo *ExplicitInfo = nullptr;
+  llvm::PointerUnion<const ASTTemplateArgumentListInfo *,
+                     ExplicitInstantiationInfo *>
+      ExplicitInfo = nullptr;
 
   /// The template arguments used to describe this specialization.
   const TemplateArgumentList *TemplateArgs;
@@ -2670,14 +2657,6 @@ class VarTemplateSpecializationDecl : public VarDecl,
   /// specialization.
   const TemplateArgumentList &getTemplateArgs() const { return *TemplateArgs; }
 
-  // TODO: Always set this when creating the new specialization?
-  void setTemplateArgsInfo(const TemplateArgumentListInfo &ArgsInfo);
-  void setTemplateArgsInfo(const ASTTemplateArgumentListInfo *ArgsInfo);
-
-  const ASTTemplateArgumentListInfo *getTemplateArgsInfo() const {
-    return TemplateArgsInfo;
-  }
-
   /// Determine the kind of specialization that this
   /// declaration represents.
   TemplateSpecializationKind getSpecializationKind() const {
@@ -2781,44 +2760,45 @@ class VarTemplateSpecializationDecl : public VarDecl,
     SpecializedTemplate = TemplDecl;
   }
 
-  /// Sets the type of this specialization as it was written by
-  /// the user.
-  void setTypeAsWritten(TypeSourceInfo *T) {
-    if (!ExplicitInfo)
-      ExplicitInfo = new (getASTContext()) ExplicitSpecializationInfo;
-    ExplicitInfo->TypeAsWritten = T;
-  }
-
-  /// Gets the type of this specialization as it was written by
-  /// the user, if it was so written.
-  TypeSourceInfo *getTypeAsWritten() const {
-    return ExplicitInfo ? ExplicitInfo->TypeAsWritten : nullptr;
+  const ASTTemplateArgumentListInfo *getTemplateArgsAsWritten() const {
+    if (auto *Info = ExplicitInfo.dyn_cast<ExplicitInstantiationInfo *>())
+      return Info->TemplateArgsAsWritten;
+    return ExplicitInfo.get<const ASTTemplateArgumentListInfo *>();
   }
 
   /// Gets the location of the extern keyword, if present.
   SourceLocation getExternLoc() const {
-    return ExplicitInfo ? ExplicitInfo->ExternLoc : SourceLocation();
+    if (auto *Info = ExplicitInfo.dyn_cast<ExplicitInstantiationInfo *>())
+      return Info->ExternLoc;
+    return SourceLocation();
   }
 
-  /// Sets the location of the extern keyword.
-  void setExternLoc(SourceLocation Loc) {
-    if (!ExplicitInfo)
-      ExplicitInfo = new (getASTContext()) ExplicitSpecializationInfo;
-    ExplicitInfo->ExternLoc = Loc;
+  /// Gets the location of the template keyword, if present.
+  SourceLocation getTemplateKeywordLoc() const {
+    if (auto *Info = ExplicitInfo.dyn_cast<ExplicitInstantiationInfo *>())
+      return Info->TemplateKeywordLoc;
+    return SourceLocation();
   }
 
-  /// Sets the location of the template keyword.
-  void setTemplateKeywordLoc(SourceLocation Loc) {
-    if (!ExplicitInfo)
-      ExplicitInfo = new (getASTContext()) ExplicitSpecializationInfo;
-    ExplicitInfo->TemplateKeywordLoc = Loc;
+  void
+  setTemplateArgsAsWritten(const ASTTemplateArgumentListInfo *ArgsWritten) {
+    if (auto *Info = ExplicitInfo.dyn_cast<ExplicitInstantiationInfo *>())
+      Info->TemplateArgsAsWritten = ArgsWritten;
+    else
+      ExplicitInfo = ArgsWritten;
   }
 
-  /// Gets the location of the template keyword, if present.
-  SourceLocation getTemplateKeywordLoc() const {
-    return ExplicitInfo ? ExplicitInfo->TemplateKeywordLoc : SourceLocation();
+  void setTemplateArgsAsWritten(const TemplateArgumentListInfo &ArgsInfo) {
+    setTemplateArgsAsWritten(
+        ASTTemplateArgumentListInfo::Create(getASTContext(), ArgsInfo));
   }
 
+  /// Sets the location of the extern keyword.
+  void setExternLoc(SourceLocation Loc);
+
+  /// Sets the location of the template keyword.
+  void setTemplateKeywordLoc(SourceLocation Loc);
+
   SourceRange getSourceRange() const override LLVM_READONLY;
 
   void Profile(llvm::FoldingSetNodeID &ID) const {
@@ -2846,10 +2826,6 @@ class VarTemplatePartialSpecializationDecl
   /// The list of template parameters
   TemplateParameterList *TemplateParams = nullptr;
 
-  /// The source info for the template arguments as written.
-  /// FIXME: redundant with TypeAsWritten?
-  const ASTTemplateArgumentListInfo *ArgsAsWritten = nullptr;
-
   /// The variable template partial specialization from which this
   /// variable template partial specialization was instantiated.
   ///
@@ -2862,8 +2838,7 @@ class VarTemplatePartialSpecializationDecl
       ASTContext &Context, DeclContext *DC, SourceLocation StartLoc,
       SourceLocation IdLoc, TemplateParameterList *Params,
       VarTemplateDecl *SpecializedTemplate, QualType T, TypeSourceInfo *TInfo,
-      StorageClass S, ArrayRef<TemplateArgument> Args,
-      const ASTTemplateArgumentListInfo *ArgInfos);
+      StorageClass S, ArrayRef<TemplateArgument> Args);
 
   VarTemplatePartialSpecializationDecl(ASTContext &Context)
       : VarTemplateSpecializationDecl(VarTemplatePartialSpecialization,
@@ -2880,8 +2855,8 @@ class VarTemplatePartialSpecializationDecl
   Create(ASTContext &Context, DeclContext *DC, SourceLocation StartLoc,
          SourceLocation IdLoc, TemplateParameterList *Params,
          VarTemplateDecl *SpecializedTemplate, QualType T,
-         TypeSourceInfo *TInfo, StorageClass S, ArrayRef<TemplateArgument> Args,
-         const TemplateArgumentListInfo &ArgInfos);
+         TypeSourceInfo *TInfo, StorageClass S,
+         ArrayRef<TemplateArgument> Args);
 
   static VarTemplatePartialSpecializationDecl *CreateDeserialized(ASTContext &C,
                                                                   unsigned ID);
@@ -2897,11 +2872,6 @@ class VarTemplatePartialSpecializationDecl
     return TemplateParams;
   }
 
-  /// Get the template arguments as written.
-  const ASTTemplateArgumentListInfo *getTemplateArgsAsWritten() const {
-    return ArgsAsWritten;
-  }
-
   /// \brief All associated constraints of this partial specialization,
   /// including the requires clause and any constraints derived from
   /// constrained-parameters.
diff --git a/clang/include/clang/AST/RecursiveASTVisitor.h b/clang/include/clang/AST/RecursiveASTVisitor.h
index 5080551ada4fc6..562a304cb23b1e 100644
--- a/clang/include/clang/AST/RecursiveASTVisitor.h
+++ b/clang/include/clang/AST/RecursiveASTVisitor.h
@@ -2009,6 +2009,15 @@ DEF_TRAVERSE_DECL(RecordDecl, { TRY_TO(TraverseRecordHelper(D)); })
 
 DEF_TRAVERSE_DECL(CXXRecordDecl, { TRY_TO(TraverseCXXRecordHelper(D)); })
 
+template <typename Derived>
+bool RecursiveASTVisitor<Derived>::TraverseTemplateArgumentLocsHelper(
+    const TemplateArgumentLoc *TAL, unsigned Count) {
+  for (unsigned I = 0; I < Count; ++I) {
+    TRY_TO(TraverseTemplateArgumentLoc(TAL[I]));
+  }
+  return true;
+}
+
 #define DEF_TRAVERSE_TMPL_SPEC_DECL(TMPLDECLKIND, DECLKIND)                    \
   DEF_TRAVERSE_DECL(TMPLDECLKIND##TemplateSpecializationDecl, {                \
     /* For implicit instantiations ("set<int> x;"), we don't want to           \
@@ -2018,9 +2027,12 @@ DEF_TRAVERSE_DECL(CXXRecordDecl, { TRY_TO(TraverseCXXRecordHelper(D)); })
        TemplateSpecializationType).  For explicit instantiations               \
        ("template set<int>;"), we do need a callback, since this               \
        is the only callback that's made for this instantiation.                \
-       We use getTypeAsWritten() to distinguish. */                            \
-    if (TypeSourceInfo *TSI = D->getTypeAsWritten())                           \
-      TRY_TO(TraverseTypeLoc(TSI->getTypeLoc()));                              \
+       We use getTemplateArgsAsWritten() to distinguish. */                    \
+    if (const auto *ArgsWritten = D->getTemplateArgsAsWritten()) {             \
+      /* The args that remains unspecialized. */                               \
+      TRY_TO(TraverseTemplateArgumentLocsHelper(                               \
+          ArgsWritten->getTemplateArgs(), ArgsWritten->NumTemplateArgs));      \
+    }                                                                          \
                                                                                \
     if (getDerived().shouldVisitTemplateInstantiations() ||                    \
         D->getTemplateSpecializationKind() == TSK_ExplicitSpecialization) {    \
@@ -2040,15 +2052,6 @@ DEF_TRAVERSE_DECL(CXXRecordDecl, { TRY_TO(TraverseCXXRecordHelper(D)); })
 DEF_TRAVERSE_TMPL_SPEC_DECL(Class, CXXRecord)
 DEF_TRAVERSE_TMPL_SPEC_DECL(Var, Var)
 
-template <typename Derived>
-bool RecursiveASTVisitor<Derived>::TraverseTemplateArgumentLocsHelper(
-    const TemplateArgumentLoc *TAL, unsigned Count) {
-  for (unsigned I = 0; I < Count; ++I) {
-    TRY_TO(TraverseTemplateArgumentLoc(TAL[I]));
-  }
-  return true;
-}
-
 #define DEF_TRAVERSE_TMPL_PART_SPEC_DECL(TMPLDECLKIND, DECLKIND)               \
   DEF_TRAVERSE_DECL(TMPLDECLKIND##TemplatePartialSpecializationDecl, {         \
     /* The partial specialization. */                                          \
diff --git a/clang/include/clang/ASTMatchers/ASTMatchers.h b/clang/include/clang/ASTMatchers/ASTMatchers.h
index dc1f49525a004a..4cdc9ec4cc5178 100644
--- a/clang/include/clang/ASTMatchers/ASTMatchers.h
+++ b/clang/include/clang/ASTMatchers/ASTMatchers.h
@@ -4066,7 +4066,7 @@ AST_POLYMORPHIC_MATCHER_P_OVERLOAD(
 ///   Matcher<CXXCtorInitializer>, Matcher<CXXFunctionalCastExpr>,
 ///   Matcher<CXXNewExpr>, Matcher<CXXTemporaryObjectExpr>,
 ///   Matcher<CXXUnresolvedConstructExpr>,
-///   Matcher<ClassTemplateSpecializationDecl>, Matcher<CompoundLiteralExpr>,
+///   Matcher<CompoundLiteralExpr>,
 ///   Matcher<DeclaratorDecl>, Matcher<ExplicitCastExpr>,
 ///   Matcher<ObjCPropertyDecl>, Matcher<TemplateArgumentLoc>,
 ///   Matcher<TypedefNameDecl>
@@ -4075,9 +4075,8 @@ AST_POLYMORPHIC_MATCHER_P(
     AST_POLYMORPHIC_SUPPORTED_TYPES(
         BlockDecl, CXXBaseSpecifier, CXXCtorInitializer, CXXFunctionalCastExpr,
         CXXNewExpr, CXXTemporaryObjectExpr, CXXUnresolvedConstructExpr,
-        ClassTemplateSpecializationDecl, CompoundLiteralExpr, DeclaratorDecl,
-        ExplicitCastExpr, ObjCPropertyDecl, TemplateArgumentLoc,
-        TypedefNameDecl),
+        CompoundLiteralExpr, DeclaratorDecl, ExplicitCastExpr, ObjCPropertyDecl,
+        TemplateArgumentLoc, TypedefNameDecl),
     internal::Matcher<TypeLoc>, Inner) {
   TypeSourceInfo *source = internal::GetTypeSourceInfo(Node);
   if (source == nullptr) {
diff --git a/clang/include/clang/ASTMatchers/ASTMatchersInternal.h b/clang/include/clang/ASTMatchers/ASTMatchersInternal.h
index 47d912c73dd7eb..dca642acfa38bc 100644
--- a/clang/include/clang/ASTMatchers/ASTMatchersInternal.h
+++ b/clang/include/clang/ASTMatchers/ASTMatchersInternal.h
@@ -186,10 +186,6 @@ inline TypeSourceInfo *GetTypeSourceInfo(const BlockDecl &Node) {
 inline TypeSourceInfo *GetTypeSourceInfo(const CXXNewExpr &Node) {
   return Node.getAllocatedTypeSourceInfo();
 }
-inline TypeSourceInfo *
-GetTypeSourceInfo(const ClassTemplateSpecializationDecl &Node) {
-  return Node.getTypeAsWritten();
-}
 
 /// Unifies obtaining the FunctionProtoType pointer from both
 /// FunctionProtoType and FunctionDecl nodes..
diff --git a/clang/lib/AST/ASTImporter.cpp b/clang/lib/AST/ASTImporter.cpp
index 023aaa7f0572b4..d1b141026497ed 100644
--- a/clang/lib/AST/ASTImporter.cpp
+++ b/clang/lib/AST/ASTImporter.cpp
@@ -6148,15 +6148,16 @@ ExpectedDecl ASTNodeImporter::VisitClassTemplateSpecializationDecl(
   if (!IdLocOrErr)
     return IdLocOrErr.takeError();
 
+  // Import TemplateArgumentListInfo.
+  TemplateArgumentListInfo ToTAInfo;
+  if (const auto *ASTTemplateArgs = D->getTemplateArgsAsWritten()) {
+    if (Error Err = ImportTemplateArgumentListInfo(*ASTTemplateArgs, ToTAInfo))
+      return std::move(Err);
+  }
+
   // Create the specialization.
   ClassTemplateSpecializationDecl *D2 = nullptr;
   if (PartialSpec) {
-    // Import TemplateArgumentListInfo.
-    TemplateArgumentListInfo ToTAInfo;
-    const auto &ASTTemplateArgs = *PartialSpec->getTemplateArgsAsWritten();
-    if (Error Err = ImportTemplateArgumentListInfo(ASTTemplateArgs, ToTAInfo))
-      return std::move(Err);
-
     QualType CanonInjType;
     if (Error Err = imp...
[truncated]

@erichkeane
Copy link
Collaborator

The "most breaking" change is to AST Matchers, insofar that hasTypeLoc will no longer match class template specializations (since they no longer store the type as written). I wasn't entirely sure what to replace the tests for this behavior with, so I just commented them out... guidance/feedback would be appreciated on this.

This part has implications that I'm not sure about, and likely needs to A: be in the 'breaking changes' doc, and B: Have Aaron comment on it.

@@ -222,7 +220,7 @@ int binTempl<int, U>;

template<class U>
float binTempl<float, U> = 1;
// CHECK: VarTemplatePartialSpecializationDecl 0x{{[^ ]*}} <line:{{[0-9]+}}:1, line:{{[0-9]+}}:24> col:7 binTempl 'float' cinit
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we losing 'const-int' (Is that what cinit means?) here? Is that perhaps important?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed cinit because the output will contain "explicit_specialization" before it.

Despite the output being identical to our current output, this somehow seems to affect whether the tests pass if --check-prefix=... is removed from the test command line arguments. If the check prefix arguments are retained and we instead make the expected output lines for DIRECT/SERIALIZED the same, the tests pass. Merging the two lines into a single CHECK line results in the change being necessary for the tests to ass...

refersToType(asString("int")))))));
}

#if 0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still a WIP, or are these from local testing?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, hah. I was checking whether we could still match class template specializations based on template arguments. The commented out tests should be replaced with equivalents written using the above syntax (where it makes sense).

So to answer your question, it's from local testing :)

@@ -6390,6 +6394,7 @@ TEST(HasTemplateArgumentLoc, BindsToSpecializationWithDoubleArgument) {
0, hasTypeLoc(loc(asString("double")))))))))));
}

#if 0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question here :)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ping on this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@erichkeane I just pushed a commit which converts all the #if'd out test to use hasTemplateArgument/hasAnyTemplateArgument instead of hasTemplateArgumentLoc/hasAnyTemplateArgumentLoc. However, this is a temporary measure.

I want to add support for ClassTemplateSpecializationDecl, VarTemplateSpecializationDecl, and FunctionDecl to hasTemplateArgumentLoc/hasAnyTemplateArgumentLoc so these tests can be switched back to using the TemplateArgumentLoc matchers, but I'm not sure whether it should be included in this PR, or have it's own follow-up PR (I would prefer to include it in this one).

Copy link
Member

@ChuanqiXu9 ChuanqiXu9 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. My instinct reaction to the title is that you're going to unify the annoying duplicated interfaces for the 5 specialization classes (function specialization, class/var (partial) specializations). But it is still good to merge these things.

Comment on lines 1812 to 1831
llvm::PointerUnion<const ASTTemplateArgumentListInfo *,
ExplicitInstantiationInfo *>
ExplicitInfo = nullptr;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spent some time to understand const ASTTemplateArgumentListInfo * is only meaningful for partial specializations. Can we reuse ExplicitInstantiationInfo::TemplateArgsAsWrittenTemplateArgsAsWritten? And make ExplicitInstantiationInfo to InstantiationInfo so that we can use it generally?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spent some time to understand const ASTTemplateArgumentListInfo * is only meaningful for partial specializations.

Not quite... const ASTTemplateArgumentListInfo * is also used for explicit specializations. ExplicitInstantiationInfo * (should) only be used for explicit instantiations, I just need to remove any instances of TemplateKeywordLoc being set for non-explicit-instantiations specializations (planning to do this prior to merging).

@sdkrystian sdkrystian force-pushed the spec-type-as-written branch from 85fe29f to 0185834 Compare April 1, 2024 15:00
Copy link

github-actions bot commented Apr 1, 2024

✅ With the latest revision this PR passed the C/C++ code formatter.

Copy link
Collaborator

@erichkeane erichkeane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more suggestions, also I like Shafik's suggestion on the variable name.

@sdkrystian
Copy link
Member Author

@erichkeane Regarding "Still waiting on this?", I pushed a commit that addresses the fixme. I'll be addressing the rest of your review comments in a subsequent commit..

Also, if we are renaming ExplicitInstantiationInfo::ExternLoc to ExternKeywordLoc, should we rename the getters and setters to getExternKeywordLoc/setExternKeywordLoc as well?

@erichkeane
Copy link
Collaborator

@erichkeane Regarding "Still waiting on this?", I pushed a commit that addresses the fixme. I'll be addressing the rest of your review comments in a subsequent commit..

Also, if we are renaming ExplicitInstantiationInfo::ExternLoc to ExternKeywordLoc, should we rename the getters and setters to getExternKeywordLoc/setExternKeywordLoc as well?

I think that rename makes sense as well.

@sdkrystian sdkrystian force-pushed the spec-type-as-written branch from 34acf0c to 10ee471 Compare April 17, 2024 14:23
@sdkrystian sdkrystian force-pushed the spec-type-as-written branch 3 times, most recently from 230bf12 to c7d84c5 Compare April 17, 2024 17:52
@sdkrystian
Copy link
Member Author

I added a commit which adds support for matching the TemplateArgumentLocs of explicit specializations directly.

@sdkrystian sdkrystian force-pushed the spec-type-as-written branch 3 times, most recently from 11097fe to 73b5f0a Compare April 19, 2024 17:51
@sdkrystian sdkrystian requested a review from erichkeane April 19, 2024 17:52
@sdkrystian sdkrystian force-pushed the spec-type-as-written branch from 627af68 to eb0f392 Compare May 7, 2024 16:24
@sdkrystian sdkrystian merged commit 7115ed0 into llvm:main May 7, 2024
5 of 7 checks passed
@sdkrystian
Copy link
Member Author

This seems to cause a crash in lldb in ClassTemplateSpecializationDecl::getSourceRange()... investigating

@adrian-prantl
Copy link
Collaborator

This broke several tests in the LLDB testsuite, I'm going to have to revert this.
https://green.lab.llvm.org/job/llvm.org/view/LLDB/job/as-lldb-cmake/3480/

adrian-prantl added a commit that referenced this pull request May 7, 2024
…ritten for class/variable template specializations (#81642)"

This reverts commit 7115ed0.

This commit broke several LLDB tests.

https://green.lab.llvm.org/job/llvm.org/view/LLDB/job/as-lldb-cmake/3480/
@adrian-prantl adrian-prantl requested a review from Michael137 May 7, 2024 20:04
sdkrystian added a commit to sdkrystian/llvm-project that referenced this pull request May 7, 2024
…written for class/variable template specializations (llvm#81642)"

Our current method of storing the template arguments as written for
`(Class/Var)Template(Partial)SpecializationDecl` suffers from a number
of flaws:
- We use `TypeSourceInfo` to store `TemplateArgumentLocs` for class
template/variable template partial/explicit specializations. For
variable template specializations, this is a rather unintuitive hack (as
we store a non-type specialization as a type). Moreover, we don't ever
*need* the type as written -- in almost all cases, we only want the
template arguments (e.g. in tooling use-cases).
- The template arguments as written are stored in a number of redundant
data members. For example, `(Class/Var)TemplatePartialSpecialization`
have their own `ArgsAsWritten` member that stores an
`ASTTemplateArgumentListInfo` (the template arguments).
`VarTemplateSpecializationDecl` has yet _another_ redundant member
"`TemplateArgsInfo`" that also stores an `ASTTemplateArgumentListInfo`.

This patch eliminates all
`(Class/Var)Template(Partial)SpecializationDecl` members which store the
template arguments as written, and turns the `ExplicitInfo` member into
a `llvm::PointerUnion<const ASTTemplateArgumentListInfo*,
ExplicitInstantiationInfo*>` (to avoid unnecessary allocations when the
declaration isn't an explicit instantiation). The template arguments as
written are now accessed via `getTemplateArgsWritten` in all cases.

The "most breaking" change is to AST Matchers, insofar that `hasTypeLoc`
will no longer match class template specializations (since they no
longer store the type as written).
sdkrystian added a commit that referenced this pull request May 14, 2024
…written for class/variable template specializations (#81642)" (#91393)

Reapplies #81642, fixing the crash which occurs when running the lldb test suite.
@kimgr
Copy link
Contributor

kimgr commented May 26, 2024

@sdkrystian It looks like this PR broke IWYU with respect to this assumption:

Moreover, we don't ever need the type as written -- in almost all cases, we only want the template arguments (e.g. in tooling use-cases).

As I understand it, IWYU did use the type as written to do resugaring of template types, here: https://github.com/include-what-you-use/include-what-you-use/blob/master/iwyu_ast_util.cc#L935 Do you have any tips for how to implement that on top of the new model? Manually type-switch all TemplateArgument kinds and map to a Type, possibly via Expr?

Uh, oops. I mixed this up with another breaking change.

The breakage is here: https://github.com/include-what-you-use/include-what-you-use/blob/bc86a8fd570cb8344795564f74f2a5a27a6ea925/iwyu.cc#L4080. I'm actually not sure what we're trying to do there, it looks very roundabout.

@sdkrystian
Copy link
Member Author

sdkrystian commented May 27, 2024

@kimgr The linked code seems to only be concerned with the SourceLocation of the Decl (which should be the same SourceLocation returned by Decl::getLocation), and the template arguments as written (which can be accessed via getTemplateArgsAsWritten per this patch). I would imagine the fix here would be to handle explicit instantiations of class templates the same way you handle explicit instantiations of function templates (which also do not store a TypeLoc but can have their template arguments as written accessed in a similar manner).

@kimgr
Copy link
Contributor

kimgr commented May 27, 2024

@sdkrystian Thank you! I think there's some more subtleties to it, but I don't really have my head around it yet. I'll first try to understand what we're trying to achieve and then ping back of I need help with something concrete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clang:frontend Language frontend issues, e.g. anything involving "Sema" clang:modules C++20 modules and Clang Header Modules clang Clang issues not falling into any other category clang-tools-extra clangd
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants