Store test content in a custom metadata section.

grynspan · grynspan · commit f780c96927a5 · 2024-10-04T15:29:40.000-04:00
See also: swiftlang/swift#76698 Resolves #735.
diff --git a/Documentation/ABI/TestContent.md b/Documentation/ABI/TestContent.md
@@ -21,78 +21,117 @@ the testing library are stored in dedicated platform-specific sections:
 
 | Platform | Binary Format | Section Name |
 |-|:-:|-|
-| macOS | Mach-O | `__DATA_CONST,__swift5_tests` |
-| iOS | Mach-O | `__DATA_CONST,__swift5_tests` |
-| watchOS | Mach-O | `__DATA_CONST,__swift5_tests` |
-| tvOS | Mach-O | `__DATA_CONST,__swift5_tests` |
-| visionOS | Mach-O | `__DATA_CONST,__swift5_tests` | 
-| Linux | ELF | `PT_NOTE`[^1] |
-| FreeBSD | ELF | `PT_NOTE`[^1] |
-| Android | ELF | `PT_NOTE`[^1] |
+| macOS, iOS, watchOS, tvOS, visionOS | Mach-O | `__DATA_CONST,__swift5_tests` |
+| Linux, FreeBSD, Android | ELF | `PT_NOTE`[^1] |
 | WASI | Statically Linked | `swift5_tests` |
 | Windows | PE/COFF | `.sw5test` |
 
 [^1]: On platforms that use the ELF binary format natively, test content records
       are stored in ELF program headers of type `PT_NOTE`. Take care not to
       remove these program headers (for example, by invoking [`strip(1)`](https://www.man7.org/linux/man-pages/man1/strip.1.html).)
 
-### Determining the type of test content
-      
+### Record headers
+
 Regardless of platform, all test content records created and discoverable by the
-testing library start have the name `"Swift Testing"` stored in the implied
-`n_name` field of their underlying ELF Notes. Each record's _type_ (stored in
-the underlying ELF Note's `n_type` field) determines how the record will be
-interpreted at runtime:
+testing library have the following structure:
+
+```c
+struct SWTTestContentHeader {
+  int32_t n_namesz;
+  int32_t n_descsz;
+  int32_t n_type;
+  char n_name[n_namesz];
+  // ...
+};
+```
+
+This structure can be represented in Swift as a heterogenous tuple:
+
+```swift
+typealias SWTTestContentHeader = (
+  n_namesz: Int32,
+  n_descsz: Int32,
+  n_type: Int32,
+  n_name: (CChar, CChar, /* ... */),
+  // ...
+)
+```
+
+The size of `n_name` is dynamic and cannot be statically computed. The testing
+library always generates the name `"Swift Testing"` and specifies an `n_namesz`
+value of `20` (the string being null-padded to the correct length), but other
+content may be present in the same section whose header size differs. For more
+information about this structure such as its alignment requirements, see the
+documentation for the [ELF format](https://man7.org/linux/man-pages/man5/elf.5.html).
+
+Each record's _kind_ (stored in the `n_type` field) determines how the record
+will be interpreted at runtime:
 
 | Type Value | Interpretation |
 |-:|-|
-| < `0` | Undefined (**do not use**) |
-| `0` ... `99` | Reserved |
+| `< 0` | Undefined (**do not use**) |
+| `0 ... 99` | Reserved |
 | `100` | Test or suite declaration |
 | `101` | Exit test |
 
 <!-- When adding cases to this enumeration, be sure to also update the
 corresponding enumeration in Discovery.h and TestContentGeneration.swift. -->
 
-### Loading test content from a record
+### Record contents
 
-For all currently-defined record types, the header and name are followed by a
-structure of the following form:
+For all currently-defined record types, the header structure is immediately
+followed by the actual content of the record. A test content record currently
+contains an `accessor` function to load the corresponding Swift content and a
+`flags` field whose value depends on the type of record. The overall structure
+of a record therefore looks like:
 
 ```c
 struct SWTTestContent {
-  bool (* accessor)(void *);
-  uint64_t flags;
+  SWTTestContentHeader header;
+  bool (* accessor)(void *outValue);
+  uint32_t flags;
+  uint32_t reserved;
 };
 ```
 
-#### The accessor field
-
-The function `accessor` is a C function whose signature in Swift can be restated
-as:
+Or, in Swift as a tuple:
 
 ```swift
-@convention(c) (_ outValue: UnsafeMutableRawPointer) -> Bool
+typealias SWTTestContent = (
+  header: SWTTestContentHeader,
+  accessor: @convention(c) (_ outValue: UnsafeMutableRawPointer) -> Bool,
+  flags: UInt32,
+  reserved: UInt32
+)
 ```
 
-When called, it initializes the memory at `outValue` to an instance of some
-Swift type and returns `true`, or returns `false` if it could not generate the
-relevant content. On successful return, the caller is responsible for
-deinitializing the memory at `outValue` when done with it.
+This structure may grow in the future as needed. Check the `header.n_descsz`
+field to determine if there are additional fields present. Do not assume that
+the size of this structure will remain fixed over time or that all discovered
+test content records are the same size.
 
-The concrete Swift type of `accessor`'s result depends on the type of record:
+#### The accessor field
+
+The function `accessor` is a C function. When called, it initializes the memory
+at its argument `outValue` to an instance of some Swift type and returns `true`,
+or returns `false` if it could not generate the relevant content. On successful
+return, the caller is responsible for deinitializing the memory at `outValue`
+when done with it.
+
+The concrete Swift type of the value written to `outValue` depends on the type
+of record:
 
 | Type Value | Return Type |
 |-:|-|
 | < `0` | Undefined (**do not use**) |
-| `0` ... `99` | `nil` |
+| `0` ... `99` | Reserved (**do not use**) |
 | `100` | `@Sendable () async -> Test`[^2] |
-| `101` | `ExitTest` (owned by caller) |
+| `101` | `ExitTest` (consumed by caller) |
 
 [^2]: This signature is not the signature of `accessor`, but of the Swift
-      function reference it returns. This level of indirection is necessary
-      because loading a test or suite declaration is an asynchronous operation,
-      but C functions cannot be `async`.
+      function reference it writes to `outValue`. This level of indirection is
+      necessary because loading a test or suite declaration is an asynchronous
+      operation, but C functions cannot be `async`.
 
 #### The flags field
 
@@ -105,6 +144,10 @@ For test or suite declarations (type `100`), the following flags are defined:
 
 For exit test declarations (type `101`), no flags are currently defined.
 
+#### The reserved field
+
+This field is reserved for future use. Always set it to `0`.
+
 ## Third-party test content
 
 TODO: elaborate how tools can reuse the same `n_name` and `n_type` fields to
diff --git a/Package.swift b/Package.swift
@@ -164,6 +164,15 @@ extension Array where Element == PackageDescription.CXXSetting {
   static var packageSettings: Self {
     var result = Self()
 
+    result += [
+      .define("SWT_TARGET_OS_APPLE", .when(platforms: [.macOS, .iOS, .macCatalyst, .watchOS, .tvOS, .visionOS])),
+
+      .define("SWT_NO_EXIT_TESTS", .when(platforms: [.iOS, .watchOS, .tvOS, .visionOS, .wasi, .android])),
+      .define("SWT_NO_SNAPSHOT_TYPES", .when(platforms: [.linux, .windows, .wasi])),
+      .define("SWT_NO_DYNAMIC_LINKING", .when(platforms: [.wasi])),
+      .define("SWT_NO_PIPES", .when(platforms: [.wasi])),
+    ]
+
     // Capture the testing library's version as a C++ string constant.
     if let git = Context.gitInformation {
       let testingLibraryVersion = if let tag = git.currentTag {
diff --git a/Sources/Testing/ExitTests/ExitTest.swift b/Sources/Testing/ExitTests/ExitTest.swift
@@ -17,20 +17,28 @@ private import _TestingInternals
 
 /// A type describing an exit test.
 ///
-/// Instances of this type describe an exit test defined by the test author and
-/// discovered or called at runtime.
-@_spi(Experimental) @_spi(ForToolsIntegrationOnly)
+/// An instance of this type describes an exit test defined in a test target and
+/// discovered or called at runtime. You do not create instances of this type.
+///
+/// You don't usually need to interact with an instance of this type. To create
+/// an exit test, use the ``expect(exitsWith:_:sourceLocation:performing:)``
+/// or ``require(exitsWith:_:sourceLocation:performing:)`` macro.
+@_spi(Experimental)
 public struct ExitTest: Sendable, ~Copyable {
-  /// The expected exit condition of the exit test.
+  /// This exit test's expected exit condition.
   public var expectedExitCondition: ExitCondition
 
-  /// The source location of the exit test.
+  /// The source location of this exit test.
   ///
   /// The source location is unique to each exit test and is consistent between
   /// processes, so it can be used to uniquely identify an exit test at runtime.
   public var sourceLocation: SourceLocation
 
   /// The body closure of the exit test.
+  ///
+  /// Do not invoke this closure directly. Instead, invoke ``callAsFunction()``
+  /// to run the exit test. Running the exit test will always terminate the
+  /// current process.
   fileprivate var body: @Sendable () async throws -> Void
 
   /// Initialize an exit test at runtime.
@@ -97,6 +105,7 @@ public struct ExitTest: Sendable, ~Copyable {
   /// to terminate the process; if it does not, the testing library will
   /// terminate the process in a way that causes the corresponding expectation
   /// to fail.
+  @_spi(ForToolsIntegrationOnly)
   public consuming func callAsFunction() async -> Never {
     Self._disableCrashReporting()
 
diff --git a/Sources/Testing/Test+Discovery.swift b/Sources/Testing/Test+Discovery.swift
@@ -44,7 +44,9 @@ extension Test {
         nonisolated(unsafe) let imageAddress = imageAddress
         generators.append { @Sendable in
           var result = await generator()
+#if !SWT_NO_DYNAMIC_LINKING
           result.imageAddress = imageAddress
+#endif
           return result
         }
       }
@@ -152,7 +154,7 @@ private func _enumerateTestContent(_ body: _TestContentEnumerator) {
 ///   - stop: An `inout` boolean variable indicating whether type enumeration
 ///     should stop after the function returns. Set `stop` to `true` to stop
 ///     type enumeration.
-typealias TestContentEnumerator<T> = (_ imageAddress: UnsafeRawPointer?, _ content: borrowing T, _ flags: UInt64, _ stop: inout Bool) -> Void where T: ~Copyable
+typealias TestContentEnumerator<T> = (_ imageAddress: UnsafeRawPointer?, _ content: borrowing T, _ flags: UInt32, _ stop: inout Bool) -> Void where T: ~Copyable
 
 /// Enumerate all test content known to Swift and found in the current process.
 ///
diff --git a/Sources/Testing/Test.swift b/Sources/Testing/Test.swift
@@ -53,18 +53,25 @@ public struct Test: Sendable {
   /// The source location of this test.
   public var sourceLocation: SourceLocation
 
+#if !SWT_NO_DYNAMIC_LINKING
   /// The base address of the image containing this test, if available.
   ///
-  /// On platforms that do not support dynamic loading of images, the value of
-  /// this property is `nil`. Otherwise, the value is platform-specific, but
-  /// generally equal to the address of the first byte of the image mapped into
-  /// memory.
+  /// This property's value represents the image that contains this test and is
+  /// equivalent to various platform-specific values:
   ///
-  /// On Apple platforms, this property's value is equivalent to a pointer to a
-  /// `mach_header` value. On Windows, it is equivalent to an `HMODULE`. It is
-  /// never equivalent to the pointer returned from a call to `dlopen()` (on
-  /// platforms that have that function.)
-  nonisolated(unsafe) var imageAddress: UnsafeRawPointer?
+  /// | Platform | Equivalent To |
+  /// |-|-|
+  /// | macOS, iOS, tvOS, visionOS | `UnsafePointer<mach_header_64>` |
+  /// | watchOS | `UnsafePointer<mach_header>` |
+  /// | Linux, FreeBSD, Android (32-bit) | `UnsafePointer<Elf32_Ehdr>` |
+  /// | Linux, FreeBSD, Android (64-bit) | `UnsafePointer<Elf64_Ehdr>` |
+  /// | Windows | `HMODULE` |
+  ///
+  /// The value of this property is distinct from the pointer returned by
+  /// `dlopen()` (on platforms that have that function.)
+  @_spi(ForToolsIntegrationOnly)
+  public nonisolated(unsafe) var imageAddress: UnsafeRawPointer?
+#endif
 
   /// Information about the type containing this test, if any.
   ///
diff --git a/Sources/TestingMacros/SuiteDeclarationMacro.swift b/Sources/TestingMacros/SuiteDeclarationMacro.swift
@@ -126,29 +126,17 @@ public struct SuiteDeclarationMacro: MemberMacro, PeerMacro, Sendable {
     // Parse the @Suite attribute.
     let attributeInfo = AttributeInfo(byParsing: suiteAttribute, on: declaration, in: context)
 
-    // We need an extra trampoline through a static property getter (rather than
-    // just having this logic in the C thunk) because some versions of the Swift
-    // compiler think that the presence of a `try` keyword in `testsBody` means
-    // that it must be throwing (disregarding autoclosures.)
     let accessorName = context.makeUniqueName("")
     result.append(
       """
       @available(*, deprecated, message: "This property is an implementation detail of the testing library. Do not use it directly.")
-      @Sendable private static func \(accessorName)() async -> Testing.Test {
-        .__type(
-          \(declaration.type.trimmed).self,
-          \(raw: attributeInfo.functionArgumentList(in: context))
-        )
-      }
-      """
-    )
-
-    let cAccessorName = context.makeUniqueName("")
-    result.append(
-      """
-      @available(*, deprecated, message: "This property is an implementation detail of the testing library. Do not use it directly.")
-      private static let \(cAccessorName): @convention(c) (UnsafeMutableRawPointer) -> Bool = {
-        $0.initializeMemory(as: (@Sendable () async -> Testing.Test).self, to: \(accessorName))
+      private static let \(accessorName): @convention(c) (UnsafeMutableRawPointer) -> Bool = {
+        $0.initializeMemory(as: (@Sendable () async -> Testing.Test).self) { @Sendable () async in
+          .__type(
+            \(declaration.type.trimmed).self,
+            \(raw: attributeInfo.functionArgumentList(in: context))
+          )
+        }
         return true
       }
       """
@@ -160,7 +148,7 @@ public struct SuiteDeclarationMacro: MemberMacro, PeerMacro, Sendable {
         named: sectionContentName,
         in: declaration.type,
         ofKind: .testDeclaration,
-        accessingWith: cAccessorName,
+        accessingWith: accessorName,
         flags: 1 << 0 /* suite decl */
       )
     )
diff --git a/Sources/TestingMacros/Support/TestContentGeneration.swift b/Sources/TestingMacros/Support/TestContentGeneration.swift
@@ -107,14 +107,16 @@ func makeTestContentRecordDecl(named name: TokenSyntax, in typeName: TypeSyntax?
     type: Int32,
     name: \(elfNoteName.type),
     accessor: @convention(c) (UnsafeMutableRawPointer) -> Bool,
-    flags: UInt64
+    flags: UInt32,
+    reserved: UInt32
   ) = (
     Int32(MemoryLayout<\(elfNoteName.type)>.stride),
-    Int32(MemoryLayout<UnsafeRawPointer>.stride + MemoryLayout<UInt64>.stride),
+    Int32(MemoryLayout<UnsafeRawPointer>.stride + MemoryLayout<UInt32>.stride + MemoryLayout<UInt32>.stride),
     \(raw: kind.rawValue) as Int32,
     \(elfNoteName.expression) as \(elfNoteName.type),
     \(accessorName) as @convention(c) (UnsafeMutableRawPointer) -> Bool,
-    \(raw: flags) as UInt64
+    \(raw: flags) as UInt32,
+    0 as UInt32
   )
   """
 }
diff --git a/Sources/TestingMacros/TestDeclarationMacro.swift b/Sources/TestingMacros/TestDeclarationMacro.swift
@@ -467,26 +467,14 @@ public struct TestDeclarationMacro: PeerMacro, Sendable {
       )
     }
 
-    // We need an extra trampoline through a static property getter (rather than
-    // just having this logic in the C thunk) because some versions of the Swift
-    // compiler think that the presence of a `try` keyword in `testsBody` means
-    // that it must be throwing (disregarding autoclosures.)
     let accessorName = context.makeUniqueName(thunking: functionDecl)
     result.append(
       """
       @available(*, deprecated, message: "This property is an implementation detail of the testing library. Do not use it directly.")
-      @Sendable private \(staticKeyword(for: typeName)) func \(accessorName)() async -> Testing.Test {
-        \(raw: testsBody)
-      }
-      """
-    )
-
-    let cAccessorName = context.makeUniqueName(thunking: functionDecl)
-    result.append(
-      """
-      @available(*, deprecated, message: "This property is an implementation detail of the testing library. Do not use it directly.")
-      private \(staticKeyword(for: typeName)) let \(cAccessorName): @convention(c) (UnsafeMutableRawPointer) -> Bool = {
-        $0.initializeMemory(as: (@Sendable () async -> Testing.Test).self, to: \(accessorName))
+      private \(staticKeyword(for: typeName)) let \(accessorName): @convention(c) (UnsafeMutableRawPointer) -> Bool = {
+        $0.initializeMemory(as: (@Sendable () async -> Testing.Test).self) { @Sendable () async in
+          \(raw: testsBody)
+        }
         return true
       }
       """
@@ -503,7 +491,7 @@ public struct TestDeclarationMacro: PeerMacro, Sendable {
         named: sectionContentName,
         in: typeName,
         ofKind: .testDeclaration,
-        accessingWith: cAccessorName,
+        accessingWith: accessorName,
         flags: flags
       )
     )
diff --git a/Sources/_TestingInternals/Discovery.cpp b/Sources/_TestingInternals/Discovery.cpp
@@ -414,7 +414,8 @@ void swt_enumerateTestContent(void *context, SWTTestContentEnumerator body) {
       // Extract the content of this record now that we know it's ours.
       struct Content {
         bool (* accessor)(void *outValue);
-        uint64_t flags;
+        uint32_t flags;
+        uint32_t reserved;
       };
       auto content = reinterpret_cast<const Content *>(record.getDescription());
       if (!content) {
diff --git a/Sources/_TestingInternals/include/Discovery.h b/Sources/_TestingInternals/include/Discovery.h
diff --git a/Tests/TestingTests/ZipTests.swift b/Tests/TestingTests/ZipTests.swift
