Skip to content

[ASTWriter] Do not write ObjCCategories if empty. #141841

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 2, 2025

Conversation

davemgreen
Copy link
Collaborator

This is a fix for a completely unrelated patch, that started to cause failures in the explicit-build.cpp test because the size of the b.pcm and b-not-a.pcm files became the same. The alignment added by empty ObjCCategory blobs being written to the file causes them to become the same size, and the error 'module file has a different size than expected' will not be emitted as the pcms only track module size, not content, for whether they are valid.

This prevents that issue by not saving the ObjCCategories if it is empty. The change in clang/lib/Serialization/ASTReaderDecl.cpp is just a format, but shows that the only use of ObjCCategoriesMap loaded from the file will be OK with null (never loaded) data. It is a bit of a weird fix, but should help decrease the size of the modules for objects that are not used.

@llvmbot llvmbot added clang Clang issues not falling into any other category clang:modules C++20 modules and Clang Header Modules labels May 28, 2025
@llvmbot
Copy link
Member

llvmbot commented May 28, 2025

@llvm/pr-subscribers-clang

@llvm/pr-subscribers-clang-modules

Author: David Green (davemgreen)

Changes

This is a fix for a completely unrelated patch, that started to cause failures in the explicit-build.cpp test because the size of the b.pcm and b-not-a.pcm files became the same. The alignment added by empty ObjCCategory blobs being written to the file causes them to become the same size, and the error 'module file has a different size than expected' will not be emitted as the pcms only track module size, not content, for whether they are valid.

This prevents that issue by not saving the ObjCCategories if it is empty. The change in clang/lib/Serialization/ASTReaderDecl.cpp is just a format, but shows that the only use of ObjCCategoriesMap loaded from the file will be OK with null (never loaded) data. It is a bit of a weird fix, but should help decrease the size of the modules for objects that are not used.


Full diff: https://github.com/llvm/llvm-project/pull/141841.diff

2 Files Affected:

  • (modified) clang/lib/Serialization/ASTReaderDecl.cpp (+4-5)
  • (modified) clang/lib/Serialization/ASTWriter.cpp (+3)
diff --git a/clang/lib/Serialization/ASTReaderDecl.cpp b/clang/lib/Serialization/ASTReaderDecl.cpp
index e84932c765663..237b8b9d6ef6a 100644
--- a/clang/lib/Serialization/ASTReaderDecl.cpp
+++ b/clang/lib/Serialization/ASTReaderDecl.cpp
@@ -4634,11 +4634,10 @@ namespace {
 
       // Perform a binary search to find the local redeclarations for this
       // declaration (if any).
-      const ObjCCategoriesInfo Compare = { LocalID, 0 };
-      const ObjCCategoriesInfo *Result
-        = std::lower_bound(M.ObjCCategoriesMap,
-                           M.ObjCCategoriesMap + M.LocalNumObjCCategoriesInMap,
-                           Compare);
+      const ObjCCategoriesInfo Compare = {LocalID, 0};
+      const ObjCCategoriesInfo *Result = std::lower_bound(
+          M.ObjCCategoriesMap,
+          M.ObjCCategoriesMap + M.LocalNumObjCCategoriesInMap, Compare);
       if (Result == M.ObjCCategoriesMap + M.LocalNumObjCCategoriesInMap ||
           LocalID != Result->getDefinitionID()) {
         // We didn't find anything. If the class definition is in this module
diff --git a/clang/lib/Serialization/ASTWriter.cpp b/clang/lib/Serialization/ASTWriter.cpp
index cc9916a75d4b4..a92e9c37f6a7c 100644
--- a/clang/lib/Serialization/ASTWriter.cpp
+++ b/clang/lib/Serialization/ASTWriter.cpp
@@ -4978,6 +4978,9 @@ void ASTWriter::WriteCUDAPragmas(Sema &SemaRef) {
 }
 
 void ASTWriter::WriteObjCCategories() {
+  if (ObjCClassesWithCategories.empty())
+    return;
+
   SmallVector<ObjCCategoriesInfo, 2> CategoriesMap;
   RecordData Categories;
 

@jansvoboda11
Copy link
Contributor

This change makes sense to me. I have two questions:

  1. Can we add a test that checks that the block is omitted? (maybe via llvm-bcanalyzer -dump)
  2. Can we make the explicit-build.cpp test more robust? (possibly by writing out import signatures)

@davemgreen
Copy link
Collaborator Author

This change makes sense to me. I have two questions:

  1. Can we add a test that checks that the block is omitted? (maybe via llvm-bcanalyzer -dump)
  2. Can we make the explicit-build.cpp test more robust? (possibly by writing out import signatures)

Thanks - For 1 I can do. I'll see about adding a test.

For 2 I'm not sure. I was thinking of adding a triple to make sure it more reliable and less likely to go wrong on different architectures, but excluding the objc info was enough to get it working again for me. As for adding a import signature I'm not an expert, it certainly sounds sensible to have something more precise than the timestamp if what the test is testing is an important thing to get right.

@davemgreen davemgreen force-pushed the gh-clang-donotwriteobjc branch from 8e7522b to a811e8b Compare May 29, 2025 13:32
This is a fix for a completely unrelated patch, that started to fail the
explicit-build.cpp test because the size of the b.pcm and b-not-a.pcm files
became the same. The alignment added by empty ObjCCategory blobs being written
to the file causes them to be the same size, and the error 'module file has a
different size than expected' will not be emitted.

This prevents that issue by not saving the ObjCCategories if it is empty. The
change in clang/lib/Serialization/ASTReaderDecl.cpp is just a format, but shows
that the only use of ObjCCategoriesMap loaded from the file will be OK with
null (never loaded) data.
@davemgreen davemgreen force-pushed the gh-clang-donotwriteobjc branch from a811e8b to 696c8b3 Compare June 2, 2025 09:30
@davemgreen davemgreen merged commit 847e403 into llvm:main Jun 2, 2025
11 checks passed
@davemgreen davemgreen deleted the gh-clang-donotwriteobjc branch June 2, 2025 15:42
@llvm-ci
Copy link
Collaborator

llvm-ci commented Jun 2, 2025

LLVM Buildbot has detected a new failure on builder sanitizer-aarch64-linux running on sanitizer-buildbot7 while building clang at step 2 "annotate".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/51/builds/17220

Here is the relevant piece of the build log for the reference
Step 2 (annotate) failure: 'python ../sanitizer_buildbot/sanitizers/zorg/buildbot/builders/sanitizers/buildbot_selector.py' (failure)
...
[182/186] Generating MSAN_INST_TEST_OBJECTS.msan_test.cpp.aarch64-with-call.o
[183/186] Generating Msan-aarch64-with-call-Test
[184/186] Generating MSAN_INST_TEST_OBJECTS.msan_test.cpp.aarch64.o
[185/186] Generating Msan-aarch64-Test
[185/186] Running compiler_rt regression tests
llvm-lit: /home/b/sanitizer-aarch64-linux/build/llvm-project/llvm/utils/lit/lit/discovery.py:276: warning: input '/home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/interception/Unit' contained no tests
llvm-lit: /home/b/sanitizer-aarch64-linux/build/llvm-project/llvm/utils/lit/lit/discovery.py:276: warning: input '/home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/Unit' contained no tests
llvm-lit: /home/b/sanitizer-aarch64-linux/build/llvm-project/llvm/utils/lit/lit/main.py:73: note: The test suite configuration requested an individual test timeout of 0 seconds but a timeout of 900 seconds was requested on the command line. Forcing timeout to be 900 seconds.
-- Testing: 5923 tests, 72 workers --
Testing:  0.. 10.. 20
FAIL: DataFlowSanitizer-aarch64 :: origin_memmove.c (1411 of 5923)
******************** TEST 'DataFlowSanitizer-aarch64 :: origin_memmove.c' FAILED ********************
Exit Code: 139

Command Output (stderr):
--
/home/b/sanitizer-aarch64-linux/build/build_default/./bin/clang  -fsanitize=dataflow   -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta   -gmlt -DOFFSET=0 -mllvm -dfsan-track-origins=1 /home/b/sanitizer-aarch64-linux/build/llvm-project/compiler-rt/test/dfsan/origin_memmove.c -o /home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/dfsan/AARCH64Config/Output/origin_memmove.c.tmp &&       /home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/dfsan/AARCH64Config/Output/origin_memmove.c.tmp >/home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/dfsan/AARCH64Config/Output/origin_memmove.c.tmp.out 2>&1 # RUN: at line 1
+ /home/b/sanitizer-aarch64-linux/build/build_default/./bin/clang -fsanitize=dataflow -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta -gmlt -DOFFSET=0 -mllvm -dfsan-track-origins=1 /home/b/sanitizer-aarch64-linux/build/llvm-project/compiler-rt/test/dfsan/origin_memmove.c -o /home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/dfsan/AARCH64Config/Output/origin_memmove.c.tmp
+ /home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/dfsan/AARCH64Config/Output/origin_memmove.c.tmp
/home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/dfsan/AARCH64Config/Output/origin_memmove.c.script: line 4: 1665508 Segmentation fault      (core dumped) /home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/dfsan/AARCH64Config/Output/origin_memmove.c.tmp > /home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/dfsan/AARCH64Config/Output/origin_memmove.c.tmp.out 2>&1

--

********************
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.. 

2 warning(s) in tests
Slowest Tests:
--------------------------------------------------------------------------
37.13s: ThreadSanitizer-aarch64 :: bench_threads.cpp
28.49s: ThreadSanitizer-aarch64 :: restore_stack.cpp
20.60s: ThreadSanitizer-aarch64 :: signal_thread.cpp
18.67s: libFuzzer-aarch64-default-Linux :: minimize_crash.test
17.34s: libFuzzer-aarch64-default-Linux :: value-profile-switch.test
16.52s: HWAddressSanitizer-aarch64 :: TestCases/Linux/create-thread-stress.cpp
16.06s: libFuzzer-aarch64-libcxx-Linux :: minimize_crash.test
15.39s: libFuzzer-aarch64-libcxx-Linux :: value-profile-switch.test
14.68s: libFuzzer-aarch64-static-libcxx-Linux :: minimize_crash.test
13.81s: Profile-aarch64 :: Posix/instrprof-value-prof-shared.test
13.67s: libFuzzer-aarch64-static-libcxx-Linux :: value-profile-switch.test
12.08s: libFuzzer-aarch64-default-Linux :: minimize_timeout.test
11.77s: ThreadSanitizer-aarch64 :: stress.cpp
11.74s: libFuzzer-aarch64-static-libcxx-Linux :: minimize_timeout.test
11.52s: DataFlowSanitizer-aarch64 :: custom.cpp
11.28s: libFuzzer-aarch64-default-Linux :: large.test
11.10s: ThreadSanitizer-aarch64 :: deadlock_detector_stress_test.cpp
10.78s: libFuzzer-aarch64-libcxx-Linux :: minimize_timeout.test
10.29s: libFuzzer-aarch64-default-Linux :: fuzzer-timeout.test
10.09s: libFuzzer-aarch64-default-Linux :: value-profile-cmp4.test
Step 14 (test compiler-rt default) failure: test compiler-rt default (failure)
...
[182/186] Generating MSAN_INST_TEST_OBJECTS.msan_test.cpp.aarch64-with-call.o
[183/186] Generating Msan-aarch64-with-call-Test
[184/186] Generating MSAN_INST_TEST_OBJECTS.msan_test.cpp.aarch64.o
[185/186] Generating Msan-aarch64-Test
[185/186] Running compiler_rt regression tests
llvm-lit: /home/b/sanitizer-aarch64-linux/build/llvm-project/llvm/utils/lit/lit/discovery.py:276: warning: input '/home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/interception/Unit' contained no tests
llvm-lit: /home/b/sanitizer-aarch64-linux/build/llvm-project/llvm/utils/lit/lit/discovery.py:276: warning: input '/home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/Unit' contained no tests
llvm-lit: /home/b/sanitizer-aarch64-linux/build/llvm-project/llvm/utils/lit/lit/main.py:73: note: The test suite configuration requested an individual test timeout of 0 seconds but a timeout of 900 seconds was requested on the command line. Forcing timeout to be 900 seconds.
-- Testing: 5923 tests, 72 workers --
Testing:  0.. 10.. 20
FAIL: DataFlowSanitizer-aarch64 :: origin_memmove.c (1411 of 5923)
******************** TEST 'DataFlowSanitizer-aarch64 :: origin_memmove.c' FAILED ********************
Exit Code: 139

Command Output (stderr):
--
/home/b/sanitizer-aarch64-linux/build/build_default/./bin/clang  -fsanitize=dataflow   -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta   -gmlt -DOFFSET=0 -mllvm -dfsan-track-origins=1 /home/b/sanitizer-aarch64-linux/build/llvm-project/compiler-rt/test/dfsan/origin_memmove.c -o /home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/dfsan/AARCH64Config/Output/origin_memmove.c.tmp &&       /home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/dfsan/AARCH64Config/Output/origin_memmove.c.tmp >/home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/dfsan/AARCH64Config/Output/origin_memmove.c.tmp.out 2>&1 # RUN: at line 1
+ /home/b/sanitizer-aarch64-linux/build/build_default/./bin/clang -fsanitize=dataflow -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta -gmlt -DOFFSET=0 -mllvm -dfsan-track-origins=1 /home/b/sanitizer-aarch64-linux/build/llvm-project/compiler-rt/test/dfsan/origin_memmove.c -o /home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/dfsan/AARCH64Config/Output/origin_memmove.c.tmp
+ /home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/dfsan/AARCH64Config/Output/origin_memmove.c.tmp
/home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/dfsan/AARCH64Config/Output/origin_memmove.c.script: line 4: 1665508 Segmentation fault      (core dumped) /home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/dfsan/AARCH64Config/Output/origin_memmove.c.tmp > /home/b/sanitizer-aarch64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/dfsan/AARCH64Config/Output/origin_memmove.c.tmp.out 2>&1

--

********************
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..

2 warning(s) in tests
Slowest Tests:
--------------------------------------------------------------------------
37.13s: ThreadSanitizer-aarch64 :: bench_threads.cpp
28.49s: ThreadSanitizer-aarch64 :: restore_stack.cpp
20.60s: ThreadSanitizer-aarch64 :: signal_thread.cpp
18.67s: libFuzzer-aarch64-default-Linux :: minimize_crash.test
17.34s: libFuzzer-aarch64-default-Linux :: value-profile-switch.test
16.52s: HWAddressSanitizer-aarch64 :: TestCases/Linux/create-thread-stress.cpp
16.06s: libFuzzer-aarch64-libcxx-Linux :: minimize_crash.test
15.39s: libFuzzer-aarch64-libcxx-Linux :: value-profile-switch.test
14.68s: libFuzzer-aarch64-static-libcxx-Linux :: minimize_crash.test
13.81s: Profile-aarch64 :: Posix/instrprof-value-prof-shared.test
13.67s: libFuzzer-aarch64-static-libcxx-Linux :: value-profile-switch.test
12.08s: libFuzzer-aarch64-default-Linux :: minimize_timeout.test
11.77s: ThreadSanitizer-aarch64 :: stress.cpp
11.74s: libFuzzer-aarch64-static-libcxx-Linux :: minimize_timeout.test
11.52s: DataFlowSanitizer-aarch64 :: custom.cpp
11.28s: libFuzzer-aarch64-default-Linux :: large.test
11.10s: ThreadSanitizer-aarch64 :: deadlock_detector_stress_test.cpp
10.78s: libFuzzer-aarch64-libcxx-Linux :: minimize_timeout.test
10.29s: libFuzzer-aarch64-default-Linux :: fuzzer-timeout.test
10.09s: libFuzzer-aarch64-default-Linux :: value-profile-cmp4.test

sallto pushed a commit to sallto/llvm-project that referenced this pull request Jun 3, 2025
This is a fix for a completely unrelated patch, that started to cause
failures in the explicit-build.cpp test because the size of the b.pcm
and b-not-a.pcm files became the same. The alignment added by empty
ObjCCategory blobs being written to the file causes them to become the
same size, and the error 'module file has a different size than
expected' will not be emitted as the pcms only track module size, not
content, for whether they are valid.

This prevents that issue by not saving the ObjCCategories if it is
empty. The change in clang/lib/Serialization/ASTReaderDecl.cpp is just
formatting, but shows that the only use of ObjCCategoriesMap loaded from
the file will be OK with null (never loaded) data. It is a bit of a weird
fix, but should help decrease the size of the modules for objects that
are not used.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clang:modules C++20 modules and Clang Header Modules clang Clang issues not falling into any other category
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants