Running .NET 7 single file binary on Apple M1 chip causes app to be "killed"

[vitek-karas]

Discussed in #79229

^{Originally posted by rcdailey December 5, 2022}
I have a Github Workflow that compiles my .NET 7 console application using a windows agent. It runs this in a powershell script:

dotnet publish MyProject.csproj `
    --output publish `
    --configuration Release`
    --runtime osx-arm64 `
    --self-contained true `
    -p:PublishSingleFile=true `
    -p:IncludeNativeLibrariesForSelfExtract=true `
    -p:PublishReadyToRunComposite=true `
    -p:PublishReadyToRunShowWarnings=true `
    -p:EnableCompressionInSingleFile=true 

This is the platform I'm running on:

My application is not signed, but Mac lets me bypass that for testing purposes. Eventually it just fails like so:

[1]    37470 killed     ./myconsoleapp --help

I get no further diagnostics. What steps can I take to further diagnose the issue? Why is osx-arm64 not working but osx-x64 works just fine?

Tagging subscribers to this area: @agocke, @vitek-karas, @VSadov
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Discussed in #79229

^{Originally posted by rcdailey December 5, 2022}
I have a Github Workflow that compiles my .NET 7 console application using a windows agent. It runs this in a powershell script:

dotnet publish MyProject.csproj `
    --output publish `
    --configuration Release`
    --runtime osx-arm64 `
    --self-contained true `
    -p:PublishSingleFile=true `
    -p:IncludeNativeLibrariesForSelfExtract=true `
    -p:PublishReadyToRunComposite=true `
    -p:PublishReadyToRunShowWarnings=true `
    -p:EnableCompressionInSingleFile=true 

This is the platform I'm running on:

My application is not signed, but Mac lets me bypass that for testing purposes. Eventually it just fails like so:

[1]    37470 killed     ./myconsoleapp --help

I get no further diagnostics. What steps can I take to further diagnose the issue? Why is osx-arm64 not working but osx-x64 works just fine?

Author:	vitek-karas
Assignees:	-
Labels:	area-Single-File
Milestone:	-

[vitek-karas]

Comment by @am11 :

this seems related to #49855.
I get bus error (138) on osx-arm64 even with hello-world app, e.g. using daily builds:

$ dotnet8 publish -o publish -p:PublishSingleFile=true -p:EnableCompressionInSingleFile=true
$ ./publish/helloworldapp # bus error

$ dotnet8 publish -o publish -p:PublishSingleFile=true -p:EnableCompressionInSingleFile=false
$ ./publish/helloworldapp # works

[vitek-karas]

/cc @elinor-fung

[am11]

It is failing in PEImageLayout::ApplyBaseRelocations:

(lldb) r
Process 95834 launched: '/Users/am11/projects/helloworldapp/publish/helloworldapp' (arm64)
Process 95834 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=2, address=0x1053bdbd4)
    frame #0: 0x0000000100576838 helloworldapp`PEImageLayout::ApplyBaseRelocations(bool) + 488
helloworldapp`PEImageLayout::ApplyBaseRelocations:
->  0x100576838 <+488>: str    x10, [x9], #0x8
    0x10057683c <+492>: cmp    x20, x9
    0x100576840 <+496>: csel   x20, x20, x9, hi
    0x100576844 <+500>: b      0x10057680c               ; <+444>
Target 0: (helloworldapp) stopped.

(lldb) clrstack -f
OS Thread Id: 0x127edfa (1)
        Child SP               IP Call Site
000000016FDFE080 0000000100576838 helloworldapp!PEImageLayout::ApplyBaseRelocations(bool) + 488
000000016FDFE110 0000000100576D74 helloworldapp!ConvertedImageLayout::ConvertedImageLayout(FlatImageLayout*) + 364
000000016FDFE130 0000000100576380 helloworldapp!PEImageLayout::LoadConverted(PEImage*) + 188
000000016FDFE160 00000001000F7BF0 helloworldapp!PEImage::GetOrCreateLayoutInternal(unsigned int) + 68
000000016FDFE190 00000001000F6BCC helloworldapp!PEImage::GetOrCreateLayout(unsigned int) + 160
000000016FDFE1D0 00000001000F4854 helloworldapp!PEAssembly::EnsureLoaded() + 56
000000016FDFE430 000000010005B88C helloworldapp!Assembly::Init(AllocMemTracker*, LoaderAllocator*) + 112
000000016FDFE470 000000010005C650 helloworldapp!Assembly::Create(BaseDomain*, PEAssembly*, DebuggerAssemblyControlFlags, int, AllocMemTracker*, LoaderAllocator*) + 412
000000016FDFE510 000000010008E9C4 helloworldapp!DomainAssembly::Allocate() + 76
000000016FDFE740 000000010008E850 helloworldapp!DomainAssembly::DoIncrementalLoad(FileLoadLevel) + 144
000000016FDFE770 000000010005344C helloworldapp!AppDomain::TryIncrementalLoad(DomainAssembly*, FileLoadLevel, Wrapper<FileLoadLock*, &(void DoNothing<FileLoadLock*>(FileLoadLock*)), &(FileLoadLock::HolderLeave(FileLoadLock*)), 0ul, &(int CompareDefault<FileLoadLock*>(FileLoadLock*, FileLoadLock*)), true>&) + 212
000000016FDFE820 0000000100052908 helloworldapp!AppDomain::LoadDomainAssembly(FileLoadLock*, FileLoadLevel) + 256
000000016FDFE8B0 0000000100052CC4 helloworldapp!AppDomain::LoadDomainAssemblyInternal(AssemblySpec*, PEAssembly*, FileLoadLevel) + 524
000000016FDFE970 000000010004F3AC helloworldapp!SystemDomain::LoadBaseSystemClasses() + 52
000000016FDFE9A0 000000010004F0B0 helloworldapp!SystemDomain::Init() + 580
000000016FDFEA10 000000010057487C helloworldapp!EEStartupHelper() + 1732
000000016FDFEAD0 0000000100573F40 helloworldapp!EEStartup() + 220
000000016FDFEB30 0000000100573E4C helloworldapp!EnsureEEStarted() + 340
000000016FDFEB60 0000000100086728 helloworldapp!CorHost2::Start() + 116
000000016FDFEB90 00000001005783B4 helloworldapp!coreclr_initialize + 724
000000016FDFEC50 000000010001F82C helloworldapp!coreclr_t::create(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, char const*, char const*, coreclr_property_bag_t const&, std::__1::unique_ptr<coreclr_t, std::__1::default_delete<coreclr_t> >&) + 420
000000016FDFED60 000000010002C654 helloworldapp!(anonymous namespace)::create_coreclr() + 432
000000016FDFEDD0 000000010002C128 helloworldapp!corehost_main + 160
000000016FDFEF20 000000010000C8F4 helloworldapp!fx_muxer_t::handle_exec_host_command(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, host_startup_info_t const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::unordered_map<known_options, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >, known_options_hash, std::__1::equal_to<known_options>, std::__1::allocator<std::__1::pair<known_options const, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > > > > const&, int, char const**, int, host_mode_t, bool, char*, int, int*) + 1328
000000016FDFF080 000000010000B9E8 helloworldapp!fx_muxer_t::execute(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int, char const**, host_startup_info_t const&, char*, int, int*) + 860
000000016FDFF190 000000010000848C helloworldapp!hostfxr_main_bundle_startupinfo + 196
000000016FDFF240 000000010004C93C helloworldapp!exe_start(int, char const**) + 1132
000000016FDFF370 000000010004CC20 helloworldapp!main + 152
000000016FDFF3D0 0000000196033E50 dyld!start + 2544

(lldb) disas
helloworldapp`PEImageLayout::ApplyBaseRelocations:
    0x100576650 <+0>:   sub    sp, sp, #0x90
    0x100576654 <+4>:   stp    x28, x27, [sp, #0x30]
    0x100576658 <+8>:   stp    x26, x25, [sp, #0x40]
    0x10057665c <+12>:  stp    x24, x23, [sp, #0x50]
    0x100576660 <+16>:  stp    x22, x21, [sp, #0x60]
    0x100576664 <+20>:  stp    x20, x19, [sp, #0x70]
    0x100576668 <+24>:  stp    x29, x30, [sp, #0x80]
    0x10057666c <+28>:  add    x29, sp, #0x80
    0x100576670 <+32>:  mov    x19, x0
    0x100576674 <+36>:  ldr    x9, [x19, #0x8]!
    0x100576678 <+40>:  ldr    w8, [x19, #0xc]
    0x10057667c <+44>:  orr    w8, w8, #0x4
    0x100576680 <+48>:  str    w8, [x19, #0xc]
    0x100576684 <+52>:  ldrsw  x8, [x9, #0x3c]
    0x100576688 <+56>:  add    x8, x9, x8
    0x10057668c <+60>:  ldrh   w10, [x8, #0x18]
    0x100576690 <+64>:  cmp    w10, #0x10b
    0x100576694 <+68>:  b.ne   0x1005766a8               ; <+88>
    0x100576698 <+72>:  ldr    w11, [x8, #0x34]
    0x10057669c <+76>:  subs   x25, x9, x11
    0x1005766a0 <+80>:  b.ne   0x1005766b4               ; <+100>
    0x1005766a4 <+84>:  b      0x1005768f4               ; <+676>
    0x1005766a8 <+88>:  ldr    x11, [x8, #0x30]
    0x1005766ac <+92>:  subs   x25, x9, x11
    0x1005766b0 <+96>:  b.eq   0x1005768f4               ; <+676>
    0x1005766b4 <+100>: cmp    w10, #0x10b
    0x1005766b8 <+104>: mov    w9, #0x88
    0x1005766bc <+108>: mov    w10, #0x78
    0x1005766c0 <+112>: csel   x9, x10, x9, eq
    0x1005766c4 <+116>: add    x8, x8, x9
    0x1005766c8 <+120>: ldp    w1, w28, [x8, #0x28]
    0x1005766cc <+124>: mov    x0, x19
    0x1005766d0 <+128>: mov    w2, #0x0
    0x1005766d4 <+132>: bl     0x100310e94               ; PEDecoder::GetRvaData(unsigned int, IsNullOK) const
    0x1005766d8 <+136>: cbz    w28, 0x100576914          ; <+708>
    0x1005766dc <+140>: mov    x12, #0x0
    0x1005766e0 <+144>: mov    x13, #0x0
    0x1005766e4 <+148>: mov    w24, #0x0
    0x1005766e8 <+152>: mov    x11, #0x0
    0x1005766ec <+156>: stp    x0, xzr, [sp, #0x20]
    0x1005766f0 <+160>: mov    w27, #0x0
    0x1005766f4 <+164>: mov    x23, #0x1fffffff8
    0x1005766f8 <+168>: str    w28, [sp, #0x1c]
    0x1005766fc <+172>: b      0x100576718               ; <+200>
    0x100576700 <+176>: mov    x11, x21
    0x100576704 <+180>: sub    x8, x20, x11
    0x100576708 <+184>: str    x8, [sp, #0x28]
    0x10057670c <+188>: add    w27, w22, w27
    0x100576710 <+192>: cmp    w27, w28
    0x100576714 <+196>: b.hs   0x1005768a8               ; <+600>
    0x100576718 <+200>: add    x8, x0, w27, uxtw
    0x10057671c <+204>: add    x26, x8, #0x8
    0x100576720 <+208>: ldp    w20, w22, [x8]
    0x100576724 <+212>: ldr    x8, [x19]
    0x100576728 <+216>: add    x21, x8, x20
    0x10057672c <+220>: sub    x8, x21, x12
    0x100576730 <+224>: cmp    x8, x13
    0x100576734 <+228>: b.lo   0x1005767f4               ; <+420>
    0x100576738 <+232>: mov    x28, x11
    0x10057673c <+236>: tst    w24, #0xf0
    0x100576740 <+240>: b.eq   0x10057674c               ; <+252>
    0x100576744 <+244>: mov    w0, #0x0
    0x100576748 <+248>: bl     0x1005fd110               ; PAL_JitWriteProtect
    0x10057674c <+252>: ldrh   w8, [x26]
    0x100576750 <+256>: and    w8, w8, #0xfff
    0x100576754 <+260>: add    w1, w20, w8
    0x100576758 <+264>: mov    x0, x19
    0x10057675c <+268>: bl     0x1003106ec               ; PEDecoder::RvaToSection(unsigned int) const
    0x100576760 <+272>: mov    x24, x0
    0x100576764 <+276>: ldr    w1, [x0, #0xc]
    0x100576768 <+280>: mov    x0, x19
    0x10057676c <+284>: mov    w2, #0x0
    0x100576770 <+288>: bl     0x100310e94               ; PEDecoder::GetRvaData(unsigned int, IsNullOK) const
    0x100576774 <+292>: mov    x12, x0
    0x100576778 <+296>: ldr    w13, [x24, #0x10]
    0x10057677c <+300>: ldr    w8, [x24, #0x24]
    0x100576780 <+304>: tbnz   w8, #0x1f, 0x100576798    ; <+328>
    0x100576784 <+308>: tbnz   w8, #0x1d, 0x1005767a8    ; <+344>
    0x100576788 <+312>: mov    w8, #0x0
    0x10057678c <+316>: ldr    x0, [sp, #0x20]
    0x100576790 <+320>: mov    x11, x28
    0x100576794 <+324>: b      0x1005767dc               ; <+396>
    0x100576798 <+328>: mov    w24, #0x0
    0x10057679c <+332>: ldr    x0, [sp, #0x20]
    0x1005767a0 <+336>: mov    x11, x28
    0x1005767a4 <+340>: b      0x1005767ec               ; <+412>
    0x1005767a8 <+344>: mov    w0, #0x1
    0x1005767ac <+348>: mov    x20, x12
    0x1005767b0 <+352>: mov    x23, x19
    0x1005767b4 <+356>: mov    x19, x13
    0x1005767b8 <+360>: bl     0x1005fd110               ; PAL_JitWriteProtect
    0x1005767bc <+364>: mov    x13, x19
    0x1005767c0 <+368>: mov    x19, x23
    0x1005767c4 <+372>: mov    x12, x20
    0x1005767c8 <+376>: ldr    w9, [x24, #0x24]
    0x1005767cc <+380>: and    w8, w9, #0x20000000
    0x1005767d0 <+384>: ldr    x0, [sp, #0x20]
    0x1005767d4 <+388>: mov    x11, x28
    0x1005767d8 <+392>: tbnz   w9, #0x1f, 0x100576898    ; <+584>
    0x1005767dc <+396>: cmp    w8, #0x0
    0x1005767e0 <+400>: mov    w8, #0x20
    0x1005767e4 <+404>: mov    w9, #0x2
    0x1005767e8 <+408>: csel   w24, w9, w8, eq
    0x1005767ec <+412>: ldr    w28, [sp, #0x1c]
    0x1005767f0 <+416>: mov    x23, #0x1fffffff8
    0x1005767f4 <+420>: add    x8, x22, x23
    0x1005767f8 <+424>: lsr    x9, x8, #1
    0x1005767fc <+428>: cbz    w9, 0x10057670c           ; <+188>
    0x100576800 <+432>: mov    x20, #0x0
    0x100576804 <+436>: ubfx   x8, x8, #1, #32
    0x100576808 <+440>: b      0x100576818               ; <+456>
    0x10057680c <+444>: add    x26, x26, #0x2
    0x100576810 <+448>: subs   x8, x8, #0x1
    0x100576814 <+452>: b.eq   0x100576848               ; <+504>
    0x100576818 <+456>: ldrh   w9, [x26]
    0x10057681c <+460>: and    w10, w9, #0xf000
    0x100576820 <+464>: cmp    w10, #0xa, lsl #12        ; =0xa000 
    0x100576824 <+468>: b.ne   0x10057680c               ; <+444>
    0x100576828 <+472>: and    x9, x9, #0xfff
    0x10057682c <+476>: add    x9, x21, x9
    0x100576830 <+480>: ldr    x10, [x9]
    0x100576834 <+484>: add    x10, x10, x25
->  0x100576838 <+488>: str    x10, [x9], #0x8
    0x10057683c <+492>: cmp    x20, x9
    0x100576840 <+496>: csel   x20, x20, x9, hi
    0x100576844 <+500>: b      0x10057680c               ; <+444>
    0x100576848 <+504>: tst    w24, #0xf0
    0x10057684c <+508>: b.eq   0x10057670c               ; <+188>
    0x100576850 <+512>: cbz    x20, 0x10057670c          ; <+188>
    0x100576854 <+516>: ldr    x8, [sp, #0x28]
    0x100576858 <+520>: add    x8, x11, x8
    0x10057685c <+524>: add    x8, x8, #0x1, lsl #12     ; =0x1000 
    0x100576860 <+528>: cmp    x21, x8
    0x100576864 <+532>: b.hs   0x100576870               ; <+544>
    0x100576868 <+536>: cmp    x21, x11
    0x10057686c <+540>: b.hs   0x100576704               ; <+180>
    0x100576870 <+544>: cbz    x11, 0x100576700          ; <+176>
    0x100576874 <+548>: mov    x26, x11
    0x100576878 <+552>: stp    x13, x12, [sp, #0x8]
    0x10057687c <+556>: bl     0x1006174dc               ; GetCurrentProcess
...

cc @janvorli

[am11]

$ dotnet8 publish -o publish -p:PublishSingleFile=true -p:EnableCompressionInSingleFile=true
$ ./publish/helloworldapp # bus error

Note: if we use x64 (pass -r osx-x64), the error disappears on the same M1 machine (x64 process runs under Rosetta2 emulation). So this is arm64 specific issue with compression enabled. Possibly happening since #67118.

[rcdailey]

Thanks for the quick turnaround on this. Will this be fixed in .net 7 as a hot fix release? Until then, is there a workaround that I can use to get this working?

[rcdailey]

I should also add, I observed this same behavior almost a year ago on .net 6 as well.

[am11]

is there a workaround that I can use to get this working?

If flipping EnableCompressionInSingleFile from true to false fixes the problem in your project, then that'd be the known workaround. 🙂

[rcdailey]

is there a workaround that I can use to get this working?

If flipping EnableCompressionInSingleFile from true to false fixes the problem in your project, then that'd be the known workaround. 🙂

@am11 sadly, this did not work. Here is the build I tested:
https://github.com/recyclarr/recyclarr/actions/runs/3630695323

My project is entirely open source so you are welcome to experiment with it. I'm still looking for a workaround!

[agocke]

@VSadov for investigation

[agocke]

I would definitely take this for back port to 7