Skip to content

[DWARFLinker][DWARFLinkerParallel][NFC] Refactor DWARFLinker&DWARFLinkerParallel to have a common library. Part 1. #75925

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 9, 2024

Conversation

avl-llvm
Copy link
Collaborator

This patch is extracted from #74725 .
It creates DWARFLinkerBase library, places DWARFLinker code into DWARFLinker\Apple directory, puts code from DWARFLinker\Apple into 'dwarflinker' namespace, places DWARFLinkerParallel into DWARFLinker\LLVM directory, updates BOLT to use new library. This patch is NFC.

@llvmbot
Copy link
Member

llvmbot commented Dec 19, 2023

@llvm/pr-subscribers-debuginfo

Author: None (avl-llvm)

Changes

This patch is extracted from #74725 .
It creates DWARFLinkerBase library, places DWARFLinker code into DWARFLinker\Apple directory, puts code from DWARFLinker\Apple into 'dwarflinker' namespace, places DWARFLinkerParallel into DWARFLinker\LLVM directory, updates BOLT to use new library. This patch is NFC.


Patch is 126.76 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/75925.diff

60 Files Affected:

  • (modified) bolt/lib/Rewrite/CMakeLists.txt (+1)
  • (modified) bolt/lib/Rewrite/DWARFRewriter.cpp (+6-6)
  • (renamed) llvm/include/llvm/DWARFLinker/AddressesMap.h (+5-5)
  • (renamed) llvm/include/llvm/DWARFLinker/Apple/DWARFLinker.h (+52-162)
  • (renamed) llvm/include/llvm/DWARFLinker/Apple/DWARFLinkerCompileUnit.h (+5-3)
  • (renamed) llvm/include/llvm/DWARFLinker/Apple/DWARFLinkerDeclContext.h (+5-3)
  • (renamed) llvm/include/llvm/DWARFLinker/Apple/DWARFStreamer.h (+10-6)
  • (renamed) llvm/include/llvm/DWARFLinker/DWARFFile.h (+9-8)
  • (added) llvm/include/llvm/DWARFLinker/DWARFLinkerBase.h (+98)
  • (renamed) llvm/include/llvm/DWARFLinker/LLVM/DWARFLinker.h (+8-94)
  • (renamed) llvm/include/llvm/DWARFLinker/StringPool.h (+5-5)
  • (modified) llvm/include/llvm/DebugInfo/DWARF/DWARFDebugMacro.h (+5-2)
  • (modified) llvm/include/llvm/DebugInfo/DWARF/DWARFUnit.h (+3-1)
  • (modified) llvm/lib/CMakeLists.txt (-1)
  • (added) llvm/lib/DWARFLinker/Apple/CMakeLists.txt (+24)
  • (renamed) llvm/lib/DWARFLinker/Apple/DWARFLinker.cpp (+29-36)
  • (renamed) llvm/lib/DWARFLinker/Apple/DWARFLinkerCompileUnit.cpp (+4-2)
  • (renamed) llvm/lib/DWARFLinker/Apple/DWARFLinkerDeclContext.cpp (+4-2)
  • (renamed) llvm/lib/DWARFLinker/Apple/DWARFStreamer.cpp (+4-5)
  • (modified) llvm/lib/DWARFLinker/CMakeLists.txt (+5-10)
  • (renamed) llvm/lib/DWARFLinker/LLVM/AcceleratorRecordsSaver.cpp (+3-5)
  • (renamed) llvm/lib/DWARFLinker/LLVM/AcceleratorRecordsSaver.h (+5-3)
  • (renamed) llvm/lib/DWARFLinker/LLVM/ArrayList.h (+5-3)
  • (renamed) llvm/lib/DWARFLinker/LLVM/CMakeLists.txt (+1-2)
  • (renamed) llvm/lib/DWARFLinker/LLVM/DIEAttributeCloner.cpp (+3-5)
  • (renamed) llvm/lib/DWARFLinker/LLVM/DIEAttributeCloner.h (+5-3)
  • (renamed) llvm/lib/DWARFLinker/LLVM/DIEGenerator.h (+5-3)
  • (renamed) llvm/lib/DWARFLinker/LLVM/DWARFEmitterImpl.cpp (+4-6)
  • (renamed) llvm/lib/DWARFLinker/LLVM/DWARFEmitterImpl.h (+6-4)
  • (renamed) llvm/lib/DWARFLinker/LLVM/DWARFLinker.cpp (+8-4)
  • (renamed) llvm/lib/DWARFLinker/LLVM/DWARFLinkerCompileUnit.cpp (+3-2)
  • (renamed) llvm/lib/DWARFLinker/LLVM/DWARFLinkerCompileUnit.h (+6-4)
  • (renamed) llvm/lib/DWARFLinker/LLVM/DWARFLinkerGlobalData.h (+6-4)
  • (renamed) llvm/lib/DWARFLinker/LLVM/DWARFLinkerImpl.cpp (+93-99)
  • (renamed) llvm/lib/DWARFLinker/LLVM/DWARFLinkerImpl.h (+7-5)
  • (renamed) llvm/lib/DWARFLinker/LLVM/DWARFLinkerTypeUnit.cpp (+2-1)
  • (renamed) llvm/lib/DWARFLinker/LLVM/DWARFLinkerTypeUnit.h (+5-3)
  • (renamed) llvm/lib/DWARFLinker/LLVM/DWARFLinkerUnit.cpp (+16-17)
  • (renamed) llvm/lib/DWARFLinker/LLVM/DWARFLinkerUnit.h (+7-5)
  • (renamed) llvm/lib/DWARFLinker/LLVM/DebugLineSectionEmitter.h (+7-5)
  • (renamed) llvm/lib/DWARFLinker/LLVM/DependencyTracker.cpp (+3-5)
  • (renamed) llvm/lib/DWARFLinker/LLVM/DependencyTracker.h (+5-3)
  • (renamed) llvm/lib/DWARFLinker/LLVM/IndexedValuesMap.h (+5-3)
  • (renamed) llvm/lib/DWARFLinker/LLVM/OutputSections.cpp (+5-18)
  • (renamed) llvm/lib/DWARFLinker/LLVM/OutputSections.h (+19-7)
  • (renamed) llvm/lib/DWARFLinker/LLVM/StringEntryToDwarfStringPoolEntryMap.h (+6-4)
  • (renamed) llvm/lib/DWARFLinker/LLVM/SyntheticTypeNameBuilder.cpp (+3-5)
  • (renamed) llvm/lib/DWARFLinker/LLVM/SyntheticTypeNameBuilder.h (+5-3)
  • (renamed) llvm/lib/DWARFLinker/LLVM/TypePool.h (+5-3)
  • (renamed) llvm/lib/DWARFLinker/LLVM/Utils.h (+5-3)
  • (renamed) llvm/lib/DWARFLinker/Utils.cpp (+1-3)
  • (removed) llvm/lib/DWARFLinkerParallel/DWARFFile.cpp (-17)
  • (modified) llvm/tools/dsymutil/CMakeLists.txt (+1)
  • (modified) llvm/tools/dsymutil/DwarfLinkerForBinary.cpp (+58-85)
  • (modified) llvm/tools/dsymutil/DwarfLinkerForBinary.h (+14-16)
  • (modified) llvm/tools/dsymutil/LinkUtils.h (+4-3)
  • (modified) llvm/tools/dsymutil/dsymutil.cpp (+1)
  • (modified) llvm/tools/llvm-dwarfutil/CMakeLists.txt (+1)
  • (modified) llvm/tools/llvm-dwarfutil/DebugInfoLinker.cpp (+15-18)
  • (modified) llvm/unittests/DWARFLinkerParallel/StringPoolTest.cpp (+2-2)
diff --git a/bolt/lib/Rewrite/CMakeLists.txt b/bolt/lib/Rewrite/CMakeLists.txt
index b0e2b7f46bef4c..fb21c13c654b3e 100644
--- a/bolt/lib/Rewrite/CMakeLists.txt
+++ b/bolt/lib/Rewrite/CMakeLists.txt
@@ -5,6 +5,7 @@ set(LLVM_LINK_COMPONENTS
   MC
   Object
   Support
+  DWARFLinkerBase
   DWARFLinker
   AsmPrinter
   TargetParser
diff --git a/bolt/lib/Rewrite/DWARFRewriter.cpp b/bolt/lib/Rewrite/DWARFRewriter.cpp
index 05fb3e8fafe2f7..8dd2ac342ee455 100644
--- a/bolt/lib/Rewrite/DWARFRewriter.cpp
+++ b/bolt/lib/Rewrite/DWARFRewriter.cpp
@@ -21,7 +21,7 @@
 #include "llvm/BinaryFormat/Dwarf.h"
 #include "llvm/CodeGen/AsmPrinter.h"
 #include "llvm/CodeGen/DIE.h"
-#include "llvm/DWARFLinker/DWARFStreamer.h"
+#include "llvm/DWARFLinker/Apple/DWARFStreamer.h"
 #include "llvm/DebugInfo/DWARF/DWARFContext.h"
 #include "llvm/DebugInfo/DWARF/DWARFDebugAbbrev.h"
 #include "llvm/DebugInfo/DWARF/DWARFDebugLoc.h"
@@ -181,7 +181,7 @@ translateInputToOutputLocationList(const BinaryFunction &BF,
 namespace llvm {
 namespace bolt {
 /// Emits debug information into .debug_info or .debug_types section.
-class DIEStreamer : public DwarfStreamer {
+class DIEStreamer : public dwarflinker::DwarfStreamer {
   DIEBuilder *DIEBldr;
   DWARFRewriter &Rewriter;
 
@@ -278,10 +278,10 @@ class DIEStreamer : public DwarfStreamer {
 
 public:
   DIEStreamer(DIEBuilder *DIEBldr, DWARFRewriter &Rewriter,
-              DWARFLinker::OutputFileType OutFileType,
+              dwarflinker::DWARFLinker::OutputFileType OutFileType,
               raw_pwrite_stream &OutFile,
               std::function<StringRef(StringRef Input)> Translator,
-              DWARFLinker::messageHandler Warning)
+              dwarflinker::DWARFLinker::MessageHandlerTy Warning)
       : DwarfStreamer(OutFileType, OutFile, Translator, Warning),
         DIEBldr(DIEBldr), Rewriter(Rewriter){};
 
@@ -457,8 +457,8 @@ createDIEStreamer(const Triple &TheTriple, raw_pwrite_stream &OutFile,
                   DWARFRewriter &Rewriter) {
 
   std::unique_ptr<DIEStreamer> Streamer = std::make_unique<DIEStreamer>(
-      &DIEBldr, Rewriter, llvm::DWARFLinker::OutputFileType::Object, OutFile,
-      [](StringRef Input) -> StringRef { return Input; },
+      &DIEBldr, Rewriter, dwarflinker::DWARFLinker::OutputFileType::Object,
+      OutFile, [](StringRef Input) -> StringRef { return Input; },
       [&](const Twine &Warning, StringRef Context, const DWARFDie *) {});
   Error Err = Streamer->init(TheTriple, Swift5ReflectionSegmentName);
   if (Err)
diff --git a/llvm/include/llvm/DWARFLinkerParallel/AddressesMap.h b/llvm/include/llvm/DWARFLinker/AddressesMap.h
similarity index 97%
rename from llvm/include/llvm/DWARFLinkerParallel/AddressesMap.h
rename to llvm/include/llvm/DWARFLinker/AddressesMap.h
index b451fee4e0b723..dec95cd60af37f 100644
--- a/llvm/include/llvm/DWARFLinkerParallel/AddressesMap.h
+++ b/llvm/include/llvm/DWARFLinker/AddressesMap.h
@@ -6,8 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
-#ifndef LLVM_DWARFLINKERPARALLEL_ADDRESSESMAP_H
-#define LLVM_DWARFLINKERPARALLEL_ADDRESSESMAP_H
+#ifndef LLVM_DWARFLINKER_ADDRESSESMAP_H
+#define LLVM_DWARFLINKER_ADDRESSESMAP_H
 
 #include "llvm/ADT/AddressRanges.h"
 #include "llvm/DebugInfo/DWARF/DWARFContext.h"
@@ -17,7 +17,7 @@
 #include <cstdint>
 
 namespace llvm {
-namespace dwarflinker_parallel {
+namespace dwarflinker {
 
 /// Mapped value in the address map is the offset to apply to the
 /// linked address.
@@ -186,7 +186,7 @@ class AddressesMap {
   }
 };
 
-} // end of namespace dwarflinker_parallel
+} // end of namespace dwarflinker
 } // end namespace llvm
 
-#endif // LLVM_DWARFLINKERPARALLEL_ADDRESSESMAP_H
+#endif // LLVM_DWARFLINKER_ADDRESSESMAP_H
diff --git a/llvm/include/llvm/DWARFLinker/DWARFLinker.h b/llvm/include/llvm/DWARFLinker/Apple/DWARFLinker.h
similarity index 81%
rename from llvm/include/llvm/DWARFLinker/DWARFLinker.h
rename to llvm/include/llvm/DWARFLinker/Apple/DWARFLinker.h
index 2bd85e30d3b13b..05cee779179cda 100644
--- a/llvm/include/llvm/DWARFLinker/DWARFLinker.h
+++ b/llvm/include/llvm/DWARFLinker/Apple/DWARFLinker.h
@@ -6,14 +6,15 @@
 //
 //===----------------------------------------------------------------------===//
 
-#ifndef LLVM_DWARFLINKER_DWARFLINKER_H
-#define LLVM_DWARFLINKER_DWARFLINKER_H
+#ifndef LLVM_DWARFLINKER_APPLE_DWARFLINKER_H
+#define LLVM_DWARFLINKER_APPLE_DWARFLINKER_H
 
 #include "llvm/ADT/AddressRanges.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/CodeGen/AccelTable.h"
 #include "llvm/CodeGen/NonRelocatableStringpool.h"
-#include "llvm/DWARFLinker/DWARFLinkerCompileUnit.h"
+#include "llvm/DWARFLinker/Apple/DWARFLinkerCompileUnit.h"
+#include "llvm/DWARFLinker/DWARFLinkerBase.h"
 #include "llvm/DebugInfo/DWARF/DWARFContext.h"
 #include "llvm/DebugInfo/DWARF/DWARFDebugLine.h"
 #include "llvm/DebugInfo/DWARF/DWARFDebugRangeList.h"
@@ -25,73 +26,10 @@ namespace llvm {
 class DWARFExpression;
 class DWARFUnit;
 class DataExtractor;
-class DeclContextTree;
 template <typename T> class SmallVectorImpl;
 
-enum class DwarfLinkerClient { Dsymutil, LLD, General };
-
-/// AddressesMap represents information about valid addresses used
-/// by debug information. Valid addresses are those which points to
-/// live code sections. i.e. relocations for these addresses point
-/// into sections which would be/are placed into resulting binary.
-class AddressesMap {
-public:
-  virtual ~AddressesMap();
-
-  /// Checks that there are valid relocations against a .debug_info
-  /// section.
-  virtual bool hasValidRelocs() = 0;
-
-  /// Checks that the specified DWARF expression operand \p Op references live
-  /// code section and returns the relocation adjustment value (to get the
-  /// linked address this value might be added to the source expression operand
-  /// address).
-  /// \returns relocation adjustment value or std::nullopt if there is no
-  /// corresponding live address.
-  virtual std::optional<int64_t>
-  getExprOpAddressRelocAdjustment(DWARFUnit &U,
-                                  const DWARFExpression::Operation &Op,
-                                  uint64_t StartOffset, uint64_t EndOffset) = 0;
-
-  /// Checks that the specified subprogram \p DIE references the live code
-  /// section and returns the relocation adjustment value (to get the linked
-  /// address this value might be added to the source subprogram address).
-  /// Allowed kinds of input DIE: DW_TAG_subprogram, DW_TAG_label.
-  /// \returns relocation adjustment value or std::nullopt if there is no
-  /// corresponding live address.
-  virtual std::optional<int64_t>
-  getSubprogramRelocAdjustment(const DWARFDie &DIE) = 0;
-
-  /// Returns the file name associated to the AddessesMap
-  virtual std::optional<StringRef> getLibraryInstallName() = 0;
-
-  /// Apply the valid relocations to the buffer \p Data, taking into
-  /// account that Data is at \p BaseOffset in the .debug_info section.
-  ///
-  /// \returns true whether any reloc has been applied.
-  virtual bool applyValidRelocs(MutableArrayRef<char> Data, uint64_t BaseOffset,
-                                bool IsLittleEndian) = 0;
-
-  /// Check if the linker needs to gather and save relocation info.
-  virtual bool needToSaveValidRelocs() = 0;
-
-  /// Update and save original relocations located in between StartOffset and
-  /// EndOffset. LinkedOffset is the value which should be added to the original
-  /// relocation offset to get new relocation offset in linked binary.
-  virtual void updateAndSaveValidRelocs(bool IsDWARF5,
-                                        uint64_t OriginalUnitOffset,
-                                        int64_t LinkedOffset,
-                                        uint64_t StartOffset,
-                                        uint64_t EndOffset) = 0;
-
-  /// Update the valid relocations that used OriginalUnitOffset as the compile
-  /// unit offset, and update their values to reflect OutputUnitOffset.
-  virtual void updateRelocationsWithUnitOffset(uint64_t OriginalUnitOffset,
-                                               uint64_t OutputUnitOffset) = 0;
-
-  /// Erases all data.
-  virtual void clear() = 0;
-};
+namespace dwarflinker {
+class DeclContextTree;
 
 using Offset2UnitMap = DenseMap<uint64_t, CompileUnit *>;
 
@@ -117,7 +55,7 @@ struct DebugDieValuePool {
 /// DwarfEmitter presents interface to generate all debug info tables.
 class DwarfEmitter {
 public:
-  virtual ~DwarfEmitter();
+  virtual ~DwarfEmitter() = default;
 
   /// Emit section named SecName with data SecData.
   virtual void emitSectionContents(StringRef SecData, StringRef SecName) = 0;
@@ -282,44 +220,6 @@ class DwarfEmitter {
 class DwarfStreamer;
 using UnitListTy = std::vector<std::unique_ptr<CompileUnit>>;
 
-/// This class represents DWARF information for source file
-/// and its address map.
-class DWARFFile {
-public:
-  using UnloadCallbackTy = std::function<void(StringRef FileName)>;
-  DWARFFile(StringRef Name, std::unique_ptr<DWARFContext> Dwarf,
-            std::unique_ptr<AddressesMap> Addresses,
-            UnloadCallbackTy UnloadFunc = nullptr)
-      : FileName(Name), Dwarf(std::move(Dwarf)),
-        Addresses(std::move(Addresses)), UnloadFunc(UnloadFunc) {}
-
-  /// The object file name.
-  StringRef FileName;
-
-  /// The source DWARF information.
-  std::unique_ptr<DWARFContext> Dwarf;
-
-  /// Helpful address information(list of valid address ranges, relocations).
-  std::unique_ptr<AddressesMap> Addresses;
-
-  /// Callback to the module keeping object file to unload.
-  UnloadCallbackTy UnloadFunc;
-
-  /// Unloads object file and corresponding AddressesMap and Dwarf Context.
-  void unload() {
-    Addresses.reset();
-    Dwarf.reset();
-
-    if (UnloadFunc)
-      UnloadFunc(FileName);
-  }
-};
-
-typedef std::map<std::string, std::string> swiftInterfacesMap;
-typedef std::map<std::string, std::string> objectPrefixMap;
-
-typedef function_ref<void(const DWARFUnit &Unit)> CompileUnitHandler;
-
 /// The core of the Dwarf linking logic.
 ///
 /// The generation of the dwarf information from the object files will be
@@ -334,41 +234,20 @@ typedef function_ref<void(const DWARFUnit &Unit)> CompileUnitHandler;
 /// a variable). These relocations are called ValidRelocs in the
 /// AddressesInfo and are gathered as a very first step when we start
 /// processing a object file.
-class DWARFLinker {
+class DWARFLinker : public DWARFLinkerBase {
 public:
-  typedef std::function<void(const Twine &Warning, StringRef Context,
-                             const DWARFDie *DIE)>
-      messageHandler;
-  DWARFLinker(messageHandler ErrorHandler, messageHandler WarningHandler,
+  DWARFLinker(MessageHandlerTy ErrorHandler, MessageHandlerTy WarningHandler,
               std::function<StringRef(StringRef)> StringsTranslator)
-      : DwarfLinkerClientID(DwarfLinkerClient::Dsymutil),
-        StringsTranslator(StringsTranslator), ErrorHandler(ErrorHandler),
+      : StringsTranslator(StringsTranslator), ErrorHandler(ErrorHandler),
         WarningHandler(WarningHandler) {}
 
   static std::unique_ptr<DWARFLinker> createLinker(
-      messageHandler ErrorHandler, messageHandler WarningHandler,
+      MessageHandlerTy ErrorHandler, MessageHandlerTy WarningHandler,
       std::function<StringRef(StringRef)> StringsTranslator = nullptr) {
     return std::make_unique<DWARFLinker>(ErrorHandler, WarningHandler,
                                          StringsTranslator);
   }
 
-  /// Type of output file.
-  enum class OutputFileType {
-    Object,
-    Assembly,
-  };
-
-  /// The kind of accelerator tables we should emit.
-  enum class AccelTableKind : uint8_t {
-    Apple,     ///< .apple_names, .apple_namespaces, .apple_types, .apple_objc.
-    Pub,       ///< .debug_pubnames, .debug_pubtypes
-    DebugNames ///< .debug_names.
-  };
-  typedef std::function<void(const DWARFFile &File, llvm::StringRef Output)> inputVerificationHandler;
-  typedef std::function<ErrorOr<DWARFFile &>(StringRef ContainerName,
-                                             StringRef Path)>
-      objFileLoader;
-
   Error createEmitter(const Triple &TheTriple, OutputFileType FileType,
                       raw_pwrite_stream &OutFile);
 
@@ -381,73 +260,84 @@ class DWARFLinker {
   ///
   /// \pre NoODR, Update options should be set before call to addObjectFile.
   void addObjectFile(
-      DWARFFile &File, objFileLoader Loader = nullptr,
-      CompileUnitHandler OnCUDieLoaded = [](const DWARFUnit &) {});
+      DWARFFile &File, ObjFileLoaderTy Loader = nullptr,
+      CompileUnitHandlerTy OnCUDieLoaded = [](const DWARFUnit &) {}) override;
 
   /// Link debug info for added objFiles. Object files are linked all together.
-  Error link();
+  Error link() override;
 
   /// A number of methods setting various linking options:
 
   /// Allows to generate log of linking process to the standard output.
-  void setVerbosity(bool Verbose) { Options.Verbose = Verbose; }
+  void setVerbosity(bool Verbose) override { Options.Verbose = Verbose; }
 
   /// Print statistics to standard output.
-  void setStatistics(bool Statistics) { Options.Statistics = Statistics; }
+  void setStatistics(bool Statistics) override {
+    Options.Statistics = Statistics;
+  }
 
   /// Verify the input DWARF.
-  void setVerifyInputDWARF(bool Verify) { Options.VerifyInputDWARF = Verify; }
+  void setVerifyInputDWARF(bool Verify) override {
+    Options.VerifyInputDWARF = Verify;
+  }
 
   /// Do not unique types according to ODR.
-  void setNoODR(bool NoODR) { Options.NoODR = NoODR; }
+  void setNoODR(bool NoODR) override { Options.NoODR = NoODR; }
 
   /// Update index tables only(do not modify rest of DWARF).
-  void setUpdateIndexTablesOnly(bool Update) { Options.Update = Update; }
+  void setUpdateIndexTablesOnly(bool Update) override {
+    Options.Update = Update;
+  }
 
   /// Allow generating valid, but non-deterministic output.
-  void setAllowNonDeterministicOutput(bool) { /* Nothing to do. */
+  void setAllowNonDeterministicOutput(bool) override { /* Nothing to do. */
   }
 
   /// Set whether to keep the enclosing function for a static variable.
-  void setKeepFunctionForStatic(bool KeepFunctionForStatic) {
+  void setKeepFunctionForStatic(bool KeepFunctionForStatic) override {
     Options.KeepFunctionForStatic = KeepFunctionForStatic;
   }
 
   /// Use specified number of threads for parallel files linking.
-  void setNumThreads(unsigned NumThreads) { Options.Threads = NumThreads; }
+  void setNumThreads(unsigned NumThreads) override {
+    Options.Threads = NumThreads;
+  }
 
   /// Add kind of accelerator tables to be generated.
-  void addAccelTableKind(AccelTableKind Kind) {
+  void addAccelTableKind(AccelTableKind Kind) override {
     assert(!llvm::is_contained(Options.AccelTables, Kind));
     Options.AccelTables.emplace_back(Kind);
   }
 
   /// Set prepend path for clang modules.
-  void setPrependPath(const std::string &Ppath) { Options.PrependPath = Ppath; }
+  void setPrependPath(const std::string &Ppath) override {
+    Options.PrependPath = Ppath;
+  }
 
   /// Set estimated objects files amount, for preliminary data allocation.
-  void setEstimatedObjfilesAmount(unsigned ObjFilesNum) {
+  void setEstimatedObjfilesAmount(unsigned ObjFilesNum) override {
     ObjectContexts.reserve(ObjFilesNum);
   }
 
   /// Set verification handler which would be used to report verification
   /// errors.
-  void setInputVerificationHandler(inputVerificationHandler Handler) {
+  void
+  setInputVerificationHandler(InputVerificationHandlerTy Handler) override {
     Options.InputVerificationHandler = Handler;
   }
 
   /// Set map for Swift interfaces.
-  void setSwiftInterfacesMap(swiftInterfacesMap *Map) {
+  void setSwiftInterfacesMap(SwiftInterfacesMapTy *Map) override {
     Options.ParseableSwiftInterfaces = Map;
   }
 
   /// Set prefix map for objects.
-  void setObjectPrefixMap(objectPrefixMap *Map) {
+  void setObjectPrefixMap(ObjectPrefixMapTy *Map) override {
     Options.ObjectPrefixMap = Map;
   }
 
   /// Set target DWARF version.
-  Error setTargetDWARFVersion(uint16_t TargetDWARFVersion) {
+  Error setTargetDWARFVersion(uint16_t TargetDWARFVersion) override {
     if ((TargetDWARFVersion < 1) || (TargetDWARFVersion > 5))
       return createStringError(std::errc::invalid_argument,
                                "unsupported DWARF version: %d",
@@ -619,16 +509,17 @@ class DWARFLinker {
   /// pointing to the module, and a DW_AT_gnu_dwo_id with the module
   /// hash.
   bool registerModuleReference(const DWARFDie &CUDie, LinkContext &Context,
-                               objFileLoader Loader,
-                               CompileUnitHandler OnCUDieLoaded,
+                               ObjFileLoaderTy Loader,
+                               CompileUnitHandlerTy OnCUDieLoaded,
                                unsigned Indent = 0);
 
   /// Recursively add the debug info in this clang module .pcm
   /// file (and all the modules imported by it in a bottom-up fashion)
   /// to ModuleUnits.
-  Error loadClangModule(objFileLoader Loader, const DWARFDie &CUDie,
+  Error loadClangModule(ObjFileLoaderTy Loader, const DWARFDie &CUDie,
                         const std::string &PCMFile, LinkContext &Context,
-                        CompileUnitHandler OnCUDieLoaded, unsigned Indent = 0);
+                        CompileUnitHandlerTy OnCUDieLoaded,
+                        unsigned Indent = 0);
 
   /// Clone specified Clang module unit \p Unit.
   Error cloneModuleUnit(LinkContext &Context, RefModuleUnit &Unit,
@@ -911,18 +802,16 @@ class DWARFLinker {
   /// Mapping the PCM filename to the DwoId.
   StringMap<uint64_t> ClangModules;
 
-  DwarfLinkerClient DwarfLinkerClientID;
-
   std::function<StringRef(StringRef)> StringsTranslator = nullptr;
 
   /// A unique ID that identifies each compile unit.
   unsigned UniqueUnitID = 0;
 
   // error handler
-  messageHandler ErrorHandler = nullptr;
+  MessageHandlerTy ErrorHandler = nullptr;
 
   // warning handler
-  messageHandler WarningHandler = nullptr;
+  MessageHandlerTy WarningHandler = nullptr;
 
   /// linking options
   struct DWARFLinkerOptions {
@@ -958,20 +847,21 @@ class DWARFLinker {
     std::string PrependPath;
 
     // input verification handler
-    inputVerificationHandler InputVerificationHandler = nullptr;
+    InputVerificationHandlerTy InputVerificationHandler = nullptr;
 
     /// A list of all .swiftinterface files referenced by the debug
     /// info, mapping Module name to path on disk. The entries need to
     /// be uniqued and sorted and there are only few entries expected
     /// per compile unit, which is why this is a std::map.
     /// this is dsymutil specific fag.
-    swiftInterfacesMap *ParseableSwiftInterfaces = nullptr;
+    SwiftInterfacesMapTy *ParseableSwiftInterfaces = nullptr;
 
     /// A list of remappings to apply to file paths.
-    objectPrefixMap *ObjectPrefixMap = nullptr;
+    ObjectPrefixMapTy *ObjectPrefixMap = nullptr;
   } Options;
 };
 
+} // end namespace dwarflinker
 } // end namespace llvm
 
-#endif // LLVM_DWARFLINKER_DWARFLINKER_H
+#endif // LLVM_DWARFLINKER_APPLE_DWARFLINKER_H
diff --git a/llvm/include/llvm/DWARFLinker/DWARFLinkerCompileUnit.h b/llvm/include/llvm/DWARFLinker/Apple/DWARFLinkerCompileUnit.h
similarity index 98%
rename from llvm/include/llvm/DWARFLinker/DWARFLinkerCompileUnit.h
rename to llvm/include/llvm/DWARFLinker/Apple/DWARFLinkerCompileUnit.h
index 08ebd4bc70bc93..915fba7c0bacc2 100644
--- a/llvm/include/llvm/DWARFLinker/DWARFLinkerCompileUnit.h
+++ b/llvm/include/llvm/DWARFLinker/Apple/DWARFLinkerCompileUnit.h
@@ -6,8 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
-#ifndef LLVM_DWARFLINKER_DWARFLINKERCOMPILEUNIT_H
-#define LLVM_DWARFLINKER_DWARFLINKERCOMPILEUNIT_H
+#ifndef LLVM_DWARFLINKER_APPLE_DWARFLINKERCOMPILEUNIT_H
+#define LLVM_DWARFLINKER_APPLE_DWARFLINKERCOMPILEUNIT_H
 
 #include "llvm/ADT/AddressRanges.h"
 #include "llvm/ADT/DenseMap.h"
@@ -16,6 +16,7 @@
 #include <optional>
 
 namespace llvm {
+namespace dwarflinker {
 
 class DeclContext;
 
@@ -327,6 +328,7 @@ class CompileUnit {
   std::string ClangModuleName;
 };
 
+} // end namespace dwarflinker
 } // end namespace llvm
 
-#endif // LLVM_DWARFLINKER_DWARFLINKERCOMPILEUNIT_H
+#endif // LLVM_DWARFLINKER_APPLE_DWARFLINKERCOMPILEUNIT_H
di...
[truncated]

Copy link
Contributor

@felipepiovezan felipepiovezan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we now have a base class for linker implementations, I looked at its documentation from the perspective of "what do I need to do in order to implement a new linker".

Left some wording suggestions! Hopefully others will have a look as well

@avl-llvm
Copy link
Collaborator Author

avl-llvm commented Jan 5, 2024

friendly ping

Copy link
Contributor

@felipepiovezan felipepiovezan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM, but it would be nice to have at least one more person looking into it.
@JDevlieghere ?

@JDevlieghere
Copy link
Member

JDevlieghere commented Jan 5, 2024

The base class looks good but I would prefer a slight change to the directory layout and namespaces. Particularly the distinction between "Apple" and "LLVM" bothers me. I understand how you ended up there, but I think it unnecessary associates the algorithm with particular vendors. How about "Parallel" and "Classic"?

With that naming scheme, what I'd like this to look like would be:

  • The DWARFLinker/ root directory contains shared classes and lives in the dwarflinker namespace. I think it should probably be dwarf_linker but I'm fine with dwarflinker if that's too much churn.
  • The DWARFLinker/Classic directory containing classes needed for the dsymutil lockstep algorithm. Everything here should live under the dwarf(_)linker::classic namespace.
  • The DWARFLinker/Parallel directory containing classes needed for the parallel algorithm. Everything here should live under the dwarf(_)linker::parallel namespace.

@avl-llvm
Copy link
Collaborator Author

avl-llvm commented Jan 5, 2024

The base class looks good but I would prefer a slight change to the directory layout and namespaces. Particularly the distinction between "Apple" and "LLVM" bothers me. I understand how you ended up there, but I think it unnecessary associates the algorithm with particular vendors. How about "Parallel" and "Classic"?

That is perfectly fine for me. Would you like to change command line option also?

dsymutil --linker=[classic,parallel]

Or it would be only directories/namespaces change only?

With that naming scheme, what I'd like this to look like would be:

* The `DWARFLinker/` root directory contains shared classes and lives in the `dwarflinker` namespace. I think it should probably be `dwarf_linker` but I'm fine with `dwarflinker` if that's too much churn.

* The `DWARFLinker/Classic` directory containing classes needed for the dsymutil lockstep algorithm. Everything here should live under the `dwarf(_)linker::classic` namespace.

* The `DWARFLinker/Parallel` directory containing classes needed for the parallel algorithm. Everything here should live under the `dwarf(_)linker::parallel` namespace.

@JDevlieghere
Copy link
Member

That is perfectly fine for me. Would you like to change command line option also?

dsymutil --linker=[classic,parallel]

Or it would be only directories/namespaces change only?

I don't feel strongly about the command line options. We're not using them in production yet so the only churn would be the tests. I'll let you decide whatever you think is best.

@avl-llvm
Copy link
Collaborator Author

avl-llvm commented Jan 5, 2024

That is perfectly fine for me. Would you like to change command line option also?
dsymutil --linker=[classic,parallel]
Or it would be only directories/namespaces change only?

I don't feel strongly about the command line options. We're not using them in production yet so the only churn would be the tests. I'll let you decide whatever you think is best.

I think it would be good to rename options also. Now is the good moment for this(as options are not used widely). Though, I am going to do this in a separate patch.

@@ -9,8 +9,9 @@
#include "DependencyTracker.h"
#include "llvm/Support/FormatVariadic.h"

namespace llvm {
namespace dwarflinker_parallel {
using namespace llvm;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why switch to using here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This style is recommended by guidelines for namespaces in source file : https://llvm.org/docs/CodingStandards.html#use-namespace-qualifiers-to-implement-previously-declared-functions.
Here is the example of using it in codebase -

The patch changes namespace structure so we need to update this place anyway. So it is updated using the recommended form.

…kerParallel to have a common library. Part 1.

This patch creates DWARFLinkerBase library, places DWARFLinker code into
DWARFLinker\Apple, places DWARFLinkerParallel into DWARFLinker\LLVM.
This patch is NFC.
@avl-llvm avl-llvm force-pushed the avl-llvm/refactor-dwarflinker-short-1 branch from 4d71aa8 to 0bcdb5c Compare January 7, 2024 20:52
@avl-llvm
Copy link
Collaborator Author

avl-llvm commented Jan 8, 2024

addressed comments(renamed directories Apple->Classic, LLVM->Parallel), renamed "dwarflinker" namespace into "dwarf_linker", renamed "dwarflinker_parallel" namespace into "parallel", added "classic" namespace as a child of "dwarf_linker").

Copy link
Member

@JDevlieghere JDevlieghere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @avl-llvm, LGTM!

@avl-llvm avl-llvm merged commit 2357e89 into llvm:main Jan 9, 2024
@avl-llvm avl-llvm deleted the avl-llvm/refactor-dwarflinker-short-1 branch January 9, 2024 09:39
@nico
Copy link
Contributor

nico commented Jan 9, 2024

The library names don't match the directory layout, which is really unusual.

How about naming the library in DWARFLinker/Classic LLVMDWARFLinkerClassic to match the name? Also, the one in DWARFLinker should then either be renamed to LLVMDWARFLinker or be moved into DWARFLinker/Base.

@avl-llvm
Copy link
Collaborator Author

avl-llvm commented Jan 9, 2024

The library names don't match the directory layout, which is really unusual.

How about naming the library in DWARFLinker/Classic LLVMDWARFLinkerClassic to match the name? Also, the one in DWARFLinker should then either be renamed to LLVMDWARFLinker or be moved into DWARFLinker/Base.

yep, will rename libraries:

DWARFLinker->LLVMDWARFLinker
DWARFLinker/Classic -> LLVMDWARFLinkerClassic

Thanks!

avl-llvm added a commit that referenced this pull request Jan 12, 2024
…77592)

It was noted that new DWARFLinker libraries do not follow naming
agreement -
#75925 (comment)
This patch rename libraries to match with the agreement.

Rename LLVMDWARFLinkerBase library into the LLVMDWARFLinker. Rename
LLVMDWARFLinker library into the LLVMDWARFLinkerClassic. Correct include
path according to the new directory structure.
justinfargnoli pushed a commit to justinfargnoli/llvm-project that referenced this pull request Jan 28, 2024
…kerParallel to have a common library. Part 1. (llvm#75925)

This patch creates DWARFLinkerBase library, places DWARFLinker code into
DWARFLinker\Classic, places DWARFLinkerParallel into DWARFLinker\Parallel.
updates BOLT to use new library. This patch is NFC.
justinfargnoli pushed a commit to justinfargnoli/llvm-project that referenced this pull request Jan 28, 2024
…lvm#77592)

It was noted that new DWARFLinker libraries do not follow naming
agreement -
llvm#75925 (comment)
This patch rename libraries to match with the agreement.

Rename LLVMDWARFLinkerBase library into the LLVMDWARFLinker. Rename
LLVMDWARFLinker library into the LLVMDWARFLinkerClassic. Correct include
path according to the new directory structure.
felipepiovezan pushed a commit to felipepiovezan/llvm-project that referenced this pull request Feb 2, 2024
…kerParallel to have a common library. Part 1. (llvm#75925)

This patch creates DWARFLinkerBase library, places DWARFLinker code into
DWARFLinker\Classic, places DWARFLinkerParallel into DWARFLinker\Parallel.
updates BOLT to use new library. This patch is NFC.

(cherry picked from commit 2357e89)
felipepiovezan pushed a commit to felipepiovezan/llvm-project that referenced this pull request Feb 2, 2024
…lvm#77592)

It was noted that new DWARFLinker libraries do not follow naming
agreement -
llvm#75925 (comment)
This patch rename libraries to match with the agreement.

Rename LLVMDWARFLinkerBase library into the LLVMDWARFLinker. Rename
LLVMDWARFLinker library into the LLVMDWARFLinkerClassic. Correct include
path according to the new directory structure.

(cherry picked from commit 35708b0)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants