Skip to content

[clang] Serialization: support hashing null template arguments #141890

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 29, 2025

Conversation

mizvekov
Copy link
Contributor

@mizvekov mizvekov commented May 29, 2025

When performing overload resolution during code completion, clang will allow incomplete substitutions in more places than would be allowed for valid code, because for completion to work well, it needs clang to keep going so it can explore the space of possibilities.

Notably, we accept instantiating declarations with null template arguments, and this works fine, except that when lazily loading serialized templated declarations, the template argument hasher assumes null arguments can't be used.

This patch makes the hasher happily accept that.

Fixes #139019

When performing overload resolution during code completion,
clang will allow incomplete substitutions in more places
than would be allowed for valid code, because for completion to work well,
it needs clang to keep going so it can explore the space of possibilities.

Notably, we accept instantiating declarations will null template arguments,
and this works fine, except that when lazily loading serialzied templated
declarations, the template argument hasher assumes null arguments can't
be used.

This patch makes the hasher happily accept that.

Fixes #139019
@llvmbot llvmbot added clang Clang issues not falling into any other category clang:modules C++20 modules and Clang Header Modules labels May 29, 2025
@llvmbot
Copy link
Member

llvmbot commented May 29, 2025

@llvm/pr-subscribers-clang

Author: Matheus Izvekov (mizvekov)

Changes

When performing overload resolution during code completion, clang will allow incomplete substitutions in more places than would be allowed for valid code, because for completion to work well, it needs clang to keep going so it can explore the space of possibilities.

Notably, we accept instantiating declarations will null template arguments, and this works fine, except that when lazily loading serialzied templated declarations, the template argument hasher assumes null arguments can't be used.

This patch makes the hasher happily accept that.

Fixes #139019


Full diff: https://github.com/llvm/llvm-project/pull/141890.diff

3 Files Affected:

  • (modified) clang/docs/ReleaseNotes.rst (+1)
  • (modified) clang/lib/Serialization/TemplateArgumentHasher.cpp (+3-1)
  • (added) clang/test/CodeCompletion/GH139019.cpp (+26)
diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst
index 540552bc47e8d..32266fce4d3cb 100644
--- a/clang/docs/ReleaseNotes.rst
+++ b/clang/docs/ReleaseNotes.rst
@@ -821,6 +821,7 @@ Miscellaneous Clang Crashes Fixed
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 - Fixed crash when ``-print-stats`` is enabled in compiling IR files. (#GH131608)
+- Fix code completion crash involving PCH serialzied templates. (#GH139019)
 
 OpenACC Specific Changes
 ------------------------
diff --git a/clang/lib/Serialization/TemplateArgumentHasher.cpp b/clang/lib/Serialization/TemplateArgumentHasher.cpp
index 3c7177b83ba52..aa61496d4aa0c 100644
--- a/clang/lib/Serialization/TemplateArgumentHasher.cpp
+++ b/clang/lib/Serialization/TemplateArgumentHasher.cpp
@@ -65,7 +65,9 @@ void TemplateArgumentHasher::AddTemplateArgument(TemplateArgument TA) {
 
   switch (Kind) {
   case TemplateArgument::Null:
-    llvm_unreachable("Expected valid TemplateArgument");
+    // These can occur in incomplete substitutions performed with code
+    // completion (see PartialOverloading).
+    break;
   case TemplateArgument::Type:
     AddQualType(TA.getAsType());
     break;
diff --git a/clang/test/CodeCompletion/GH139019.cpp b/clang/test/CodeCompletion/GH139019.cpp
new file mode 100644
index 0000000000000..fed35b38362a1
--- /dev/null
+++ b/clang/test/CodeCompletion/GH139019.cpp
@@ -0,0 +1,26 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+// RUN: split-file %s %t
+//
+// RUN: %clang_cc1 -std=c++20 %t/test.hpp -emit-pch -o %t/1.pch
+// RUN: %clang_cc1 -std=c++20 %t/test.cpp -include-pch %t/1.pch -code-completion-at=%t/test.cpp:7:17
+
+//--- test.hpp
+#pragma once
+class provider_t
+{
+  public:
+    template<class T>
+    void emit(T *data)
+    {}
+};
+
+//--- test.cpp
+#include "test.hpp"
+
+void test()
+{
+    provider_t *focus;
+    void *data;
+    focus->emit(&data);
+}

@llvmbot
Copy link
Member

llvmbot commented May 29, 2025

@llvm/pr-subscribers-clang-modules

Author: Matheus Izvekov (mizvekov)

Changes

When performing overload resolution during code completion, clang will allow incomplete substitutions in more places than would be allowed for valid code, because for completion to work well, it needs clang to keep going so it can explore the space of possibilities.

Notably, we accept instantiating declarations will null template arguments, and this works fine, except that when lazily loading serialzied templated declarations, the template argument hasher assumes null arguments can't be used.

This patch makes the hasher happily accept that.

Fixes #139019


Full diff: https://github.com/llvm/llvm-project/pull/141890.diff

3 Files Affected:

  • (modified) clang/docs/ReleaseNotes.rst (+1)
  • (modified) clang/lib/Serialization/TemplateArgumentHasher.cpp (+3-1)
  • (added) clang/test/CodeCompletion/GH139019.cpp (+26)
diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst
index 540552bc47e8d..32266fce4d3cb 100644
--- a/clang/docs/ReleaseNotes.rst
+++ b/clang/docs/ReleaseNotes.rst
@@ -821,6 +821,7 @@ Miscellaneous Clang Crashes Fixed
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 - Fixed crash when ``-print-stats`` is enabled in compiling IR files. (#GH131608)
+- Fix code completion crash involving PCH serialzied templates. (#GH139019)
 
 OpenACC Specific Changes
 ------------------------
diff --git a/clang/lib/Serialization/TemplateArgumentHasher.cpp b/clang/lib/Serialization/TemplateArgumentHasher.cpp
index 3c7177b83ba52..aa61496d4aa0c 100644
--- a/clang/lib/Serialization/TemplateArgumentHasher.cpp
+++ b/clang/lib/Serialization/TemplateArgumentHasher.cpp
@@ -65,7 +65,9 @@ void TemplateArgumentHasher::AddTemplateArgument(TemplateArgument TA) {
 
   switch (Kind) {
   case TemplateArgument::Null:
-    llvm_unreachable("Expected valid TemplateArgument");
+    // These can occur in incomplete substitutions performed with code
+    // completion (see PartialOverloading).
+    break;
   case TemplateArgument::Type:
     AddQualType(TA.getAsType());
     break;
diff --git a/clang/test/CodeCompletion/GH139019.cpp b/clang/test/CodeCompletion/GH139019.cpp
new file mode 100644
index 0000000000000..fed35b38362a1
--- /dev/null
+++ b/clang/test/CodeCompletion/GH139019.cpp
@@ -0,0 +1,26 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+// RUN: split-file %s %t
+//
+// RUN: %clang_cc1 -std=c++20 %t/test.hpp -emit-pch -o %t/1.pch
+// RUN: %clang_cc1 -std=c++20 %t/test.cpp -include-pch %t/1.pch -code-completion-at=%t/test.cpp:7:17
+
+//--- test.hpp
+#pragma once
+class provider_t
+{
+  public:
+    template<class T>
+    void emit(T *data)
+    {}
+};
+
+//--- test.cpp
+#include "test.hpp"
+
+void test()
+{
+    provider_t *focus;
+    void *data;
+    focus->emit(&data);
+}

Copy link
Contributor

@zyn0217 zyn0217 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@github-project-automation github-project-automation bot moved this from Needs Triage to Needs Merge in LLVM Release Status May 29, 2025
Copy link
Member

@ChuanqiXu9 ChuanqiXu9 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mizvekov mizvekov merged commit 6131407 into main May 29, 2025
15 checks passed
@github-project-automation github-project-automation bot moved this from Needs Merge to Done in LLVM Release Status May 29, 2025
@mizvekov mizvekov deleted the users/mizvekov/GH139019 branch May 29, 2025 05:08
@llvm-ci
Copy link
Collaborator

llvm-ci commented May 29, 2025

LLVM Buildbot has detected a new failure on builder lldb-x86_64-debian running on lldb-x86_64-debian while building clang at step 6 "test".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/162/builds/23407

Here is the relevant piece of the build log for the reference
Step 6 (test) failure: build (failure)
...
UNSUPPORTED: lldb-shell :: ScriptInterpreter/Lua/io.test (2954 of 2965)
UNSUPPORTED: lldb-shell :: ScriptInterpreter/Python/Crashlog/app_specific_backtrace_crashlog.test (2955 of 2965)
UNSUPPORTED: lldb-shell :: ScriptInterpreter/Python/Crashlog/no-args.test (2956 of 2965)
UNSUPPORTED: lldb-shell :: ScriptInterpreter/Python/Crashlog/altered_threadState.test (2957 of 2965)
UNSUPPORTED: lldb-shell :: ScriptInterpreter/Lua/persistent_state.test (2958 of 2965)
UNSUPPORTED: lldb-shell :: ScriptInterpreter/Python/Crashlog/parser_text.test (2959 of 2965)
UNSUPPORTED: lldb-shell :: Register/x86-gp-read.test (2960 of 2965)
UNSUPPORTED: lldb-shell :: ScriptInterpreter/Lua/bindings.test (2961 of 2965)
PASS: lldb-api :: terminal/TestEditlineCompletions.py (2962 of 2965)
UNRESOLVED: lldb-api :: tools/lldb-dap/launch/TestDAP_launch.py (2963 of 2965)
******************** TEST 'lldb-api :: tools/lldb-dap/launch/TestDAP_launch.py' FAILED ********************
Script:
--
/usr/bin/python3 /home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/test/API/dotest.py -u CXXFLAGS -u CFLAGS --env LLVM_LIBS_DIR=/home/worker/2.0.1/lldb-x86_64-debian/build/./lib --env LLVM_INCLUDE_DIR=/home/worker/2.0.1/lldb-x86_64-debian/build/include --env LLVM_TOOLS_DIR=/home/worker/2.0.1/lldb-x86_64-debian/build/./bin --arch x86_64 --build-dir /home/worker/2.0.1/lldb-x86_64-debian/build/lldb-test-build.noindex --lldb-module-cache-dir /home/worker/2.0.1/lldb-x86_64-debian/build/lldb-test-build.noindex/module-cache-lldb/lldb-api --clang-module-cache-dir /home/worker/2.0.1/lldb-x86_64-debian/build/lldb-test-build.noindex/module-cache-clang/lldb-api --executable /home/worker/2.0.1/lldb-x86_64-debian/build/./bin/lldb --compiler /home/worker/2.0.1/lldb-x86_64-debian/build/./bin/clang --dsymutil /home/worker/2.0.1/lldb-x86_64-debian/build/./bin/dsymutil --make /usr/bin/gmake --llvm-tools-dir /home/worker/2.0.1/lldb-x86_64-debian/build/./bin --lldb-obj-root /home/worker/2.0.1/lldb-x86_64-debian/build/tools/lldb --lldb-libs-dir /home/worker/2.0.1/lldb-x86_64-debian/build/./lib --cmake-build-type Release -t /home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/test/API/tools/lldb-dap/launch -p TestDAP_launch.py
--
Exit Code: 1

Command Output (stdout):
--
lldb version 21.0.0git (https://github.com/llvm/llvm-project.git revision 61314076f78327ffd5e1463c35373c7f4e52d30f)
  clang revision 61314076f78327ffd5e1463c35373c7f4e52d30f
  llvm revision 61314076f78327ffd5e1463c35373c7f4e52d30f
Skipping the following test categories: ['libc++', 'dsym', 'gmodules', 'debugserver', 'objc']

--
Command Output (stderr):
--
Change dir to: /home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/test/API/tools/lldb-dap/launch
runCmd: settings clear --all

output: 

runCmd: settings set symbols.enable-external-lookup false

output: 

runCmd: settings set target.inherit-tcc true

output: 

runCmd: settings set target.disable-aslr false

output: 

runCmd: settings set target.detach-on-error false

output: 

runCmd: settings set target.auto-apply-fixits false

@zyn0217
Copy link
Contributor

zyn0217 commented May 29, 2025

/cherry-pick 6131407

@llvmbot
Copy link
Member

llvmbot commented May 29, 2025

Failed to cherry-pick: 6131407

https://github.com/llvm/llvm-project/actions/runs/15317560723

Please manually backport the fix and push it to your github fork. Once this is done, please create a pull request

@@ -65,7 +65,9 @@ void TemplateArgumentHasher::AddTemplateArgument(TemplateArgument TA) {

switch (Kind) {
case TemplateArgument::Null:
llvm_unreachable("Expected valid TemplateArgument");
// These can occur in incomplete substitutions performed with code
// completion (see PartialOverloading).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see PartialOverloading where? Is there a specific method or API doc we should look at?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's the term of art within clang, and the name of the flag as used in source code.
A search on doxygen for this term should lead to:

  /// AddOverloadCandidate - Adds the given function to the set of
  /// candidate functions, using the given function call arguments.  If
  /// @p SuppressUserConversions, then don't allow user-defined
  /// conversions via constructors or conversion operators.
  ///
  /// \param PartialOverloading true if we are performing "partial" overloading
  /// based on an incomplete set of function arguments. This feature is used by
  /// code completion.

This term can also easily be greped for, and there are no irrelevant results.
But this is mostly undocumented.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I am using Visual Studio code and did a naive search for PartialOverloading and it was definitely not obvious where I specifically should look for this term. I did spend some time trying to dig around.

I think my point is that the comment is not super helpful unless you already have some understanding. So it requires a bit more elaboration so that random folks who come upon can get it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this name is the best stable reference to this functionality.
And I don't think mentioning file names is a good idea, and there is no documentation to point to.

Is there any specific thing this comment should say so you think this would be easier to find?

@nikic
Copy link
Contributor

nikic commented May 30, 2025

For the record, the manual backport PR is at: #141957

@nikic nikic moved this from Needs Backport PR to Done in LLVM Release Status May 30, 2025
sivan-shani pushed a commit to sivan-shani/llvm-project that referenced this pull request Jun 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clang:modules C++20 modules and Clang Header Modules clang Clang issues not falling into any other category release:cherry-pick-failed
Projects
Development

Successfully merging this pull request may close these issues.

[clangd] "Expected valid TemplateArgument" for std::min-like template
7 participants