Implement a workaround for ld-prime hitting an assert on large addends - #124721
Conversation
|
Needs more work. I'll investigate the unit test failures first. |
|
I don't get the System.Linq.Expressions test failure on Xcode 26.2 anymore. According the log the compiler/linker version on CI is |
|
Going back to the drawing board now. |
|
@akoeplinger Is there some easy way to switch some CI lane to use newer Xcode? They should be available in the runner images, just not as default. |
|
I assume adding xcode-select to the path mentioned in https://github.com/actions/runner-images/blob/main/images/macos/macos-15-Readme.md#xcode somewhere early in the build should suffice? |
Thanks, seems to pass the smoke test with Xcode 26.2. I'll probably open a separate PR to check what is the behavior just with the Xcode bump alone. |
|
Draft Pull Request was automatically closed for 30 days of inactivity. Please let us know if you'd like to reopen it. |
There was a problem hiding this comment.
Pull request overview
This PR adds a workaround in the Mach-O object writer to avoid Apple ld-prime assertions when emitting IMAGE_REL_BASED_RELPTR32 as *_RELOC_SUBTRACTOR + *_RELOC_UNSIGNED with large addends, by introducing reusable per-section temporary labels to keep addends within the linker’s expected signed 20-bit range.
Changes:
- Track and emit per-section temporary labels to bound relocation addends for
IMAGE_REL_BASED_RELPTR32on ARM64 and x64.eh_frame. - Adjust Mach relocation emission to optionally reference a temporary label symbol (instead of the section base symbol) for SUBTRACTOR relocs.
- Modify
build.shto attempt switching Xcode versions on macOS.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| src/coreclr/tools/Common/Compiler/ObjectWriter/MachObjectWriter.cs | Adds temporary-label generation/reuse and uses label symbol indices to avoid ld-prime large-addend assertions. |
| build.sh | Adds macOS-only logic to switch Xcode via sudo xcode-select. |
Sorry, I was busy for the last two weeks and didn't get to review the failures. The linker in the images is still the old ld-classic one so ideally we shouldn't see any change. I can try to reproduce it locally or we can do a re-run so I can get fresh runfo artifacts. |
There was a problem hiding this comment.
Holistic Review
Motivation: The problem is real and well-documented (issue #119380): Apple's ld-prime asserts (too many large addends) when a SUBTRACTOR/UNSIGNED relocation pair carries an addend that doesn't fit its ~20-bit inline field, which breaks Native AOT publish for large images on macOS ARM64 (and x64 .eh_frame). Fixing the ObjectWriter to keep addends small is the correct layer to address this.
Approach: The fix pre-emits a grid of local anchor symbols every 2^19 bytes in sections that use IMAGE_REL_BASED_RELPTR32, rebases each SUBTRACTOR onto the nearest at-or-below anchor, and biases the inline addend accordingly so it always fits signed-20-bit. Anchors are marked N_ALT_ENTRY so MH_SUBSECTIONS_VIA_SYMBOLS doesn't split atoms at anchor positions — a clever, targeted use of the Mach-O symbol attribute. The anchor-index math (firstAnchor + (offset >> log2) - 1, with anchor 0 mapping to the section symbol) is consistent between symbol emission and both the x64 and ARM64 relocation paths, and symbol-table emission runs before relocation emission in the base driver, so RelocAnchorsFirstSymbolIndex is populated before it is read. The executable-section guard (S_ATTR_PURE_INSTRUCTIONS assert) documents a real ld-classic limitation and matches current usage. This is a notable refactor over the earlier revision that repurposed SymbolicRelocation.Addend; the new NeedsRelocAnchors + RelocAnchorsFirstSymbolIndex design is cleaner and addresses most of the prior Copilot feedback.
Summary: Debug.Assert guarding the 20-bit addend bound (inline finding) should be a Release-safe check. No blocking correctness defect was identified in the changed lines.
Detailed Findings
✅ Anchor index arithmetic and emission ordering — verified consistent
The mapping GetRelocAnchorSymbolIndex uses (anchorIndex == 0 ? sectionIndex : baseIndex + anchorIndex - 1) matches the emission loop, which starts at i = 1 and stores RelocAnchorsFirstSymbolIndex = symbolIndex before appending. Section base symbols (lsectionN) occupy indices 0..sectionCount-1 and are emitted first, so (uint)sectionIndex correctly targets the section symbol for the first grid cell. EmitSymbolTable runs before EmitRelocations in the base ObjectWriter driver, so the base index is always populated before use.
✅ Addend bias arithmetic — correct
labelOffset = offset & ~(granularity-1) selects the nearest anchor at-or-below the site; stored = addend - (offset - labelOffset) shifts the PC-relative bias from the section start to that anchor. Because offset - labelOffset < 2^19 and the original per-target addend is small, the result fits signed-20-bit as intended.
⚠️ Release-safe bound on the biased addend — see inline comment
The signed-20-bit guarantee for stored is enforced only by Debug.Assert; a checked cast or explicit throw would convert an unexpected overflow into a deterministic failure rather than a silently corrupt object file. Flagged inline at the relevant line. Low real-world probability, advisory.
⚠️ Test coverage — no automated regression test
The PR changes object-file generation but adds no test. The linked issue notes the repro object file is too large to attach and requires an actual macOS link, so a hermetic unit test is genuinely hard here. A maintainer should decide whether a synthetic large-section ObjectWriter test (asserting anchors are emitted and addends stay in range) is feasible, or whether on-device manual validation of the #119380 repro is the accepted bar. Not necessarily merge-blocking, but worth an explicit maintainer decision.
💡 Prior review comments target an earlier design
Many existing Copilot inline comments reference _temporaryLabelsBaseIndex and "abuse the addend", which the current head has already refactored away into NeedsRelocAnchors / RelocAnchorsFirstSymbolIndex. Those specific comments appear stale against the current code and can largely be disregarded.
Note
This review was generated by this repository's Holistic Review agentic workflow to complement the built-in Copilot review.
Generated by Holistic Review · 145 AIC · ⌖ 11.3 AIC · ⊞ 10K
I appreciate you looking at this! I'll retrigger the legs. |
|
/azp run runtime-extra-platforms, runtime-nativeaot-outerloop |
|
Azure Pipelines: Successfully started running 2 pipeline(s). |
|
The System.Runtime.Intrinsics.Tests failure is real corruption on IMAGE_REL_BASED_RELPTR32 relocation. I'm trying to track down the specifics. Update: The crash is caused by a synthetic relocation anchor landing inside a four-byte RELPTR32 field. The current version of this PR produces ld-classic still creates a separate atom at every The resulting corrupted relative pointer is copied during NativeAOT rehydration into a generic dictionary slot. Later, |
We now assign the relocation anchors when the addend would overflow based on the actual offset of the relocation instead of fixed grid. This avoids the issue with ld-classic where fixed grid could put the anchor in middle of the relocation value which caused corruption. Instead of reusing and abusing existing fields for tracking temporary labels we now track the relocation anchors explicitly in a separate list with two helper methods managing the list.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
src/coreclr/tools/Common/Compiler/ObjectWriter/MachObjectWriter.cs:680
- PrepareRelptr32Addend only asserts when inlineAddend itself is out of the signed-20-bit range. In Release builds this will still emit an out-of-range addend (and likely reproduce the linker failure) instead of failing fast with a clear error from the object writer.
// An anchor at offset removes the distance term completely. If inlineAddend itself is
// out of range, no source-position anchor can make this relocation encodable. We don't
// expect this to happen in practice since inline addends are usually small offsets, so
// just check with assert.
Debug.Assert(FitsInSigned20Bits(inlineAddend), "The RELPTR32 addend cannot fit in ld-prime's signed 20-bit inline encoding.");
anchors.Add(new RelocAnchor(offset));
return inlineAddend;
|
/azp run runtime-extra-platforms, runtime-nativeaot-outerloop |
|
Azure Pipelines: Successfully started running 2 pipeline(s). |
|
I checked the build/test failures. It definitely deserves a second pair of eyes but I don't think any of the failures are related to the changes in this PR. |
|
/ba-g various timeouts and a #131262 |
|
I guess we can reenable the disabled tests now: runtime/src/libraries/tests.proj Lines 400 to 402 in 8f769f8 and backport to release/10.0 |
We should pin the disabled test against different issue. As I mentioned in #124721 (comment):
Backport to release/10.0 would be nice. That said, I would not mind waiting for this to bake in for a few weeks in |
We translate the IMAGE_REL_BASED_RELPTR32 relocation into ARM64_RELOC_SUBTRACTOR and ARM64_RELOC_UNSIGNED pair on ARM64. To emulate the behavior of PC relative relocation we bake the section-relative PC offset into the addend with negative sign. The ARM64_RELOC_SUBTRACTOR relocation then subtract the base address of the section and finally the ARM64_RELOC_UNSIGNED relocation adds the target symbol address.
This works fine for addends that fit into signed 20-bit integer, but ld-prime hits an assert when the addend is larger. To workaround it we create anchor labels at fixed 2^19 byte offsets in the section and adjust the addend to be relative to the nearest anchor label. This way we can guarantee the addend for ARM64_RELOC_SUBTRACTOR is
always within signed 20-bit range.
Same logic applies to X86_64_RELOC_SUBTRACTOR + X86_64_RELOC_UNSIGNED pair for x64 targets.
Fixes #119380