Skip to content

Implement a workaround for ld-prime hitting an assert on large addends - #124721

Merged
MichalStrehovsky merged 8 commits into
dotnet:mainfrom
filipnavara:ld-prime-addend
Jul 27, 2026
Merged

Implement a workaround for ld-prime hitting an assert on large addends#124721
MichalStrehovsky merged 8 commits into
dotnet:mainfrom
filipnavara:ld-prime-addend

Conversation

@filipnavara

@filipnavara filipnavara commented Feb 22, 2026

Copy link
Copy Markdown
Member

We translate the IMAGE_REL_BASED_RELPTR32 relocation into ARM64_RELOC_SUBTRACTOR and ARM64_RELOC_UNSIGNED pair on ARM64. To emulate the behavior of PC relative relocation we bake the section-relative PC offset into the addend with negative sign. The ARM64_RELOC_SUBTRACTOR relocation then subtract the base address of the section and finally the ARM64_RELOC_UNSIGNED relocation adds the target symbol address.

This works fine for addends that fit into signed 20-bit integer, but ld-prime hits an assert when the addend is larger. To workaround it we create anchor labels at fixed 2^19 byte offsets in the section and adjust the addend to be relative to the nearest anchor label. This way we can guarantee the addend for ARM64_RELOC_SUBTRACTOR is
always within signed 20-bit range.

Same logic applies to X86_64_RELOC_SUBTRACTOR + X86_64_RELOC_UNSIGNED pair for x64 targets.

Fixes #119380

@github-actions github-actions Bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Feb 22, 2026
@dotnet-policy-service dotnet-policy-service Bot added the community-contribution Indicates that the PR has been added by a community member label Feb 22, 2026
@filipnavara filipnavara added os-mac-os-x macOS aka OSX os-ios Apple iOS area-NativeAOT-coreclr and removed area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI labels Feb 22, 2026
@filipnavara
filipnavara marked this pull request as ready for review February 22, 2026 09:04
@filipnavara
filipnavara marked this pull request as draft February 22, 2026 09:47
@filipnavara

filipnavara commented Feb 22, 2026

Copy link
Copy Markdown
Member Author

Needs more work. I'll investigate the unit test failures first.

@filipnavara

filipnavara commented Feb 22, 2026

Copy link
Copy Markdown
Member Author

I don't get the System.Linq.Expressions test failure on Xcode 26.2 anymore. According the log the compiler/linker version on CI is some version of Xcode 26.x Xcode 16.4.

@filipnavara

Copy link
Copy Markdown
Member Author

Going back to the drawing board now.

@filipnavara

Copy link
Copy Markdown
Member Author

@akoeplinger Is there some easy way to switch some CI lane to use newer Xcode? They should be available in the runner images, just not as default.

@akoeplinger

Copy link
Copy Markdown
Member

I assume adding xcode-select to the path mentioned in https://github.com/actions/runner-images/blob/main/images/macos/macos-15-Readme.md#xcode somewhere early in the build should suffice?
You could add it to build.sh (if it's just for testing).

@filipnavara

Copy link
Copy Markdown
Member Author

I assume adding xcode-select to the path mentioned in https://github.com/actions/runner-images/blob/main/images/macos/macos-15-Readme.md#xcode somewhere early in the build should suffice?

Thanks, seems to pass the smoke test with Xcode 26.2. I'll probably open a separate PR to check what is the behavior just with the Xcode bump alone.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Draft Pull Request was automatically closed for 30 days of inactivity. Please let us know if you'd like to reopen it.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a workaround in the Mach-O object writer to avoid Apple ld-prime assertions when emitting IMAGE_REL_BASED_RELPTR32 as *_RELOC_SUBTRACTOR + *_RELOC_UNSIGNED with large addends, by introducing reusable per-section temporary labels to keep addends within the linker’s expected signed 20-bit range.

Changes:

  • Track and emit per-section temporary labels to bound relocation addends for IMAGE_REL_BASED_RELPTR32 on ARM64 and x64 .eh_frame.
  • Adjust Mach relocation emission to optionally reference a temporary label symbol (instead of the section base symbol) for SUBTRACTOR relocs.
  • Modify build.sh to attempt switching Xcode versions on macOS.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
src/coreclr/tools/Common/Compiler/ObjectWriter/MachObjectWriter.cs Adds temporary-label generation/reuse and uses label symbol indices to avoid ld-prime large-addend assertions.
build.sh Adds macOS-only logic to switch Xcode via sudo xcode-select.

Comment thread src/coreclr/tools/Common/Compiler/ObjectWriter/MachObjectWriter.cs Outdated
Comment thread src/coreclr/tools/Common/Compiler/ObjectWriter/MachObjectWriter.cs Outdated
Comment thread build.sh Outdated
@filipnavara

Copy link
Copy Markdown
Member Author

The macOS arm64 failures look related. Maybe we took an Xcode update and their linker has some fresh bugs?

Sorry, I was busy for the last two weeks and didn't get to review the failures. The linker in the images is still the old ld-classic one so ideally we shouldn't see any change. I can try to reproduce it locally or we can do a re-run so I can get fresh runfo artifacts.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Holistic Review

Motivation: The problem is real and well-documented (issue #119380): Apple's ld-prime asserts (too many large addends) when a SUBTRACTOR/UNSIGNED relocation pair carries an addend that doesn't fit its ~20-bit inline field, which breaks Native AOT publish for large images on macOS ARM64 (and x64 .eh_frame). Fixing the ObjectWriter to keep addends small is the correct layer to address this.

Approach: The fix pre-emits a grid of local anchor symbols every 2^19 bytes in sections that use IMAGE_REL_BASED_RELPTR32, rebases each SUBTRACTOR onto the nearest at-or-below anchor, and biases the inline addend accordingly so it always fits signed-20-bit. Anchors are marked N_ALT_ENTRY so MH_SUBSECTIONS_VIA_SYMBOLS doesn't split atoms at anchor positions — a clever, targeted use of the Mach-O symbol attribute. The anchor-index math (firstAnchor + (offset >> log2) - 1, with anchor 0 mapping to the section symbol) is consistent between symbol emission and both the x64 and ARM64 relocation paths, and symbol-table emission runs before relocation emission in the base driver, so RelocAnchorsFirstSymbolIndex is populated before it is read. The executable-section guard (S_ATTR_PURE_INSTRUCTIONS assert) documents a real ld-classic limitation and matches current usage. This is a notable refactor over the earlier revision that repurposed SymbolicRelocation.Addend; the new NeedsRelocAnchors + RelocAnchorsFirstSymbolIndex design is cleaner and addresses most of the prior Copilot feedback.

Summary: ⚠️ Needs Human Review. The approach is sound and the code reads correctly, but this is subtle, platform-specific linker-format code with no accompanying automated regression test, and correctness ultimately depends on behavior of Apple's linker that can only be validated on-device. A maintainer familiar with the ObjectWriter/Mach-O path should confirm the on-device link succeeds for the reproducing scenario and consider whether the sole Debug.Assert guarding the 20-bit addend bound (inline finding) should be a Release-safe check. No blocking correctness defect was identified in the changed lines.


Detailed Findings

✅ Anchor index arithmetic and emission ordering — verified consistent

The mapping GetRelocAnchorSymbolIndex uses (anchorIndex == 0 ? sectionIndex : baseIndex + anchorIndex - 1) matches the emission loop, which starts at i = 1 and stores RelocAnchorsFirstSymbolIndex = symbolIndex before appending. Section base symbols (lsectionN) occupy indices 0..sectionCount-1 and are emitted first, so (uint)sectionIndex correctly targets the section symbol for the first grid cell. EmitSymbolTable runs before EmitRelocations in the base ObjectWriter driver, so the base index is always populated before use.

✅ Addend bias arithmetic — correct

labelOffset = offset & ~(granularity-1) selects the nearest anchor at-or-below the site; stored = addend - (offset - labelOffset) shifts the PC-relative bias from the section start to that anchor. Because offset - labelOffset < 2^19 and the original per-target addend is small, the result fits signed-20-bit as intended.

⚠️ Release-safe bound on the biased addend — see inline comment

The signed-20-bit guarantee for stored is enforced only by Debug.Assert; a checked cast or explicit throw would convert an unexpected overflow into a deterministic failure rather than a silently corrupt object file. Flagged inline at the relevant line. Low real-world probability, advisory.

⚠️ Test coverage — no automated regression test

The PR changes object-file generation but adds no test. The linked issue notes the repro object file is too large to attach and requires an actual macOS link, so a hermetic unit test is genuinely hard here. A maintainer should decide whether a synthetic large-section ObjectWriter test (asserting anchors are emitted and addends stay in range) is feasible, or whether on-device manual validation of the #119380 repro is the accepted bar. Not necessarily merge-blocking, but worth an explicit maintainer decision.

💡 Prior review comments target an earlier design

Many existing Copilot inline comments reference _temporaryLabelsBaseIndex and "abuse the addend", which the current head has already refactored away into NeedsRelocAnchors / RelocAnchorsFirstSymbolIndex. Those specific comments appear stale against the current code and can largely be disregarded.

Note

This review was generated by this repository's Holistic Review agentic workflow to complement the built-in Copilot review.

Generated by Holistic Review · 145 AIC · ⌖ 11.3 AIC · ⊞ 10K

Comment thread src/coreclr/tools/Common/Compiler/ObjectWriter/MachObjectWriter.cs Outdated
@MichalStrehovsky

Copy link
Copy Markdown
Member

Sorry, I was busy for the last two weeks and didn't get to review the failures. The linker in the images is still the old ld-classic one so ideally we shouldn't see any change. I can try to reproduce it locally or we can do a re-run so I can get fresh runfo artifacts.

I appreciate you looking at this! I'll retrigger the legs.

Copilot AI review requested due to automatic review settings July 21, 2026 01:37
@MichalStrehovsky

Copy link
Copy Markdown
Member

/azp run runtime-extra-platforms, runtime-nativeaot-outerloop

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Comment thread src/coreclr/tools/Common/Compiler/ObjectWriter/MachObjectWriter.cs Outdated
Comment thread src/coreclr/tools/Common/Compiler/ObjectWriter/MachObjectWriter.cs Outdated
@filipnavara

filipnavara commented Jul 22, 2026

Copy link
Copy Markdown
Member Author

The System.Runtime.Intrinsics.Tests failure is real corruption on IMAGE_REL_BASED_RELPTR32 relocation. I'm trying to track down the specifics.

Update:

The crash is caused by a synthetic relocation anchor landing inside a four-byte RELPTR32 field.

The current version of this PR produces N_ALT_ENTRY anchors every 512 KiB. In the failing binary, one relocation starts one byte before an anchor:

relocation: [ byte 0 | byte 1 | byte 2 | byte 3 ]
                        ^
                     anchor

ld-classic still creates a separate atom at every N_ALT_ENTRY; it only chains the atoms together. It applies the relocation while writing the first atom, correctly patching all four bytes. It then writes the next atom’s original contents, overwriting the last three patched bytes. Only the first relocated byte survives.

The resulting corrupted relative pointer is copied during NativeAOT rehydration into a generic dictionary slot. Later, RhpNewArrayFast receives constant data instead of an array type handle and segfaults.

We now assign the relocation anchors when the addend would overflow based on the actual offset of the relocation instead of fixed grid. This avoids the issue with ld-classic where fixed grid could put the anchor in middle of the relocation value which caused corruption. Instead of reusing and abusing existing fields for tracking temporary labels we now track the relocation anchors explicitly in a separate list with two helper methods managing the list.
Copilot AI review requested due to automatic review settings July 22, 2026 12:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

src/coreclr/tools/Common/Compiler/ObjectWriter/MachObjectWriter.cs:680

  • PrepareRelptr32Addend only asserts when inlineAddend itself is out of the signed-20-bit range. In Release builds this will still emit an out-of-range addend (and likely reproduce the linker failure) instead of failing fast with a clear error from the object writer.
            // An anchor at offset removes the distance term completely. If inlineAddend itself is
            // out of range, no source-position anchor can make this relocation encodable. We don't
            // expect this to happen in practice since inline addends are usually small offsets, so
            // just check with assert.
            Debug.Assert(FitsInSigned20Bits(inlineAddend), "The RELPTR32 addend cannot fit in ld-prime's signed 20-bit inline encoding.");

            anchors.Add(new RelocAnchor(offset));
            return inlineAddend;

Comment thread src/coreclr/tools/Common/Compiler/ObjectWriter/MachObjectWriter.cs
Comment thread src/coreclr/tools/Common/Compiler/ObjectWriter/MachObjectWriter.cs
@MichalStrehovsky

Copy link
Copy Markdown
Member

/azp run runtime-extra-platforms, runtime-nativeaot-outerloop

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).

@filipnavara

Copy link
Copy Markdown
Member Author

I checked the build/test failures. It definitely deserves a second pair of eyes but I don't think any of the failures are related to the changes in this PR.

@MichalStrehovsky MichalStrehovsky left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@MichalStrehovsky

Copy link
Copy Markdown
Member

/ba-g various timeouts and a #131262

@MichalStrehovsky
MichalStrehovsky enabled auto-merge (squash) July 27, 2026 14:40
@MichalStrehovsky
MichalStrehovsky merged commit 3aa40d2 into dotnet:main Jul 27, 2026
150 of 164 checks passed
@dotnet-milestone-bot dotnet-milestone-bot Bot added this to the 11.0-rc1 milestone Jul 28, 2026
@akoeplinger

Copy link
Copy Markdown
Member

I guess we can reenable the disabled tests now:

<!-- https://github.com/dotnet/runtime/issues/119380 -->
<ProjectExclusions Include="$(MSBuildThisFileDirectory)System.Text.Json\tests\System.Text.Json.SourceGeneration.Tests\System.Text.Json.SourceGeneration.Roslyn4.4.Tests.csproj"
Condition="'$(TargetOS)' == 'osx' and '$(TargetArchitecture)' == 'arm64'" />
?

and backport to release/10.0

@filipnavara

Copy link
Copy Markdown
Member Author

I guess we can reenable the disabled tests now:

We should pin the disabled test against different issue. As I mentioned in #124721 (comment):

The CI for outerloop is still pinned on Xcode 16.4 and ld-classic. The failure in System.Text.Json.SourceGeneration.Roslyn4.4.Tests is thus unrelated to the fix in this PR. I downloaded the Helix payload and double checked that ld64 955.13 was used for the linking. We will need to do separate analysis to see why the unwind information seems to be corrupted there.

Backport to release/10.0 would be nice. That said, I would not mind waiting for this to bake in for a few weeks in main/11.0 first. The issue was around for well over a year and for most cases there are workarounds (eg. forcing classic linker).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-NativeAOT-coreclr community-contribution Indicates that the PR has been added by a community member os-ios Apple iOS os-mac-os-x macOS aka OSX

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ld64 crashing while Native AOT compiling Microsoft MCP for .NET 10

5 participants