Skip to content

[LLD][COFF] Generate X64 thunks for ARM64EC entry points and patchable functions. #105499

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 22, 2024
Merged

[LLD][COFF] Generate X64 thunks for ARM64EC entry points and patchable functions. #105499

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions lld/COFF/Chunks.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1073,4 +1073,9 @@ void AbsolutePointerChunk::writeTo(uint8_t *buf) const {
}
}

void ECExportThunkChunk::writeTo(uint8_t *buf) const {
memcpy(buf, ECExportThunkCode, sizeof(ECExportThunkCode));
write32le(buf + 10, target->getRVA() - rva - 14);
}

} // namespace lld::coff
20 changes: 20 additions & 0 deletions lld/COFF/Chunks.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -749,6 +749,26 @@ class ECCodeMapChunk : public NonSectionChunk {
std::vector<ECCodeMapEntry> &map;
};

static const uint8_t ECExportThunkCode[] = {
0x48, 0x8b, 0xc4, // movq %rsp, %rax
0x48, 0x89, 0x58, 0x20, // movq %rbx, 0x20(%rax)
0x55, // pushq %rbp
0x5d, // popq %rbp
0xe9, 0, 0, 0, 0, // jmp *0x0
0xcc, // int3
0xcc // int3
};

class ECExportThunkChunk : public NonSectionCodeChunk {
public:
explicit ECExportThunkChunk(Defined *targetSym) : target(targetSym) {}
size_t getSize() const override { return sizeof(ECExportThunkCode); };
void writeTo(uint8_t *buf) const override;
MachineTypes getMachine() const override { return AMD64; }

Defined *target;
};

// MinGW specific, for the "automatic import of variables from DLLs" feature.
// This provides the table of runtime pseudo relocations, for variable
// references that turned out to need to be imported from a DLL even though
Expand Down
69 changes: 69 additions & 0 deletions lld/COFF/Driver.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1317,6 +1317,72 @@ void LinkerDriver::convertResources() {
f->includeResourceChunks();
}

void LinkerDriver::maybeCreateECExportThunk(StringRef name, Symbol *&sym) {
Defined *def;
if (!sym)
return;
if (auto undef = dyn_cast<Undefined>(sym))
def = undef->getWeakAlias();
else
def = dyn_cast<Defined>(sym);
if (!def)
return;

if (def->getChunk()->getArm64ECRangeType() != chpe_range_type::Arm64EC)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, getArm64ECRangeType() return a std::optional<> - don't we usually need to somewhat check that it really is set and not nullopt, and then kinda unwrap/dereference it to get the actual value? Or I guess this does that, somewhat, and nullopt obviously returns true for the operator!=?

I.e., I presume this combo actually works on both GCC, Clang and MSVC, but have you tested that it works (and doesn't e.g. assert) on nullopt?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is well-defined, there are comparison operators for comparing std::optional against a value. According to https://en.cppreference.com/w/cpp/utility/optional/operator_cmp:

Compares opt with a value. The values are compared (using the corresponding operator of T) only if opt contains a value. Otherwise, opt is considered less than value.

I will added a test anyway. BTW, it's another difference with MSVC. MSVC linker in this example would generate a thunk, but also produce a bogus redirection data (it has redirection count of 1, but its RVA of redirection table is 0).

return;
StringRef expName;
if (auto mangledName = getArm64ECMangledFunctionName(name))
expName = saver().save("EXP+" + *mangledName);
else
expName = saver().save("EXP+" + name);
sym = addUndefined(expName);
if (auto undef = dyn_cast<Undefined>(sym)) {
if (!undef->getWeakAlias()) {
auto thunk = make<ECExportThunkChunk>(def);
replaceSymbol<DefinedSynthetic>(undef, undef->getName(), thunk);
}
}
}

void LinkerDriver::createECExportThunks() {
// Check if EXP+ symbols have corresponding $hp_target symbols and use them
// to create export thunks when available.
for (Symbol *s : ctx.symtab.expSymbols) {
if (!s->isUsedInRegularObj)
continue;
assert(s->getName().starts_with("EXP+"));
std::string targetName =
(s->getName().substr(strlen("EXP+")) + "$hp_target").str();
Symbol *sym = ctx.symtab.find(targetName);
if (!sym)
continue;
Defined *targetSym;
if (auto undef = dyn_cast<Undefined>(sym))
targetSym = undef->getWeakAlias();
else
targetSym = dyn_cast<Defined>(sym);
if (!targetSym)
continue;

auto *undef = dyn_cast<Undefined>(s);
if (undef && !undef->getWeakAlias()) {
auto thunk = make<ECExportThunkChunk>(targetSym);
replaceSymbol<DefinedSynthetic>(undef, undef->getName(), thunk);
}
if (!targetSym->isGCRoot) {
targetSym->isGCRoot = true;
ctx.config.gcroot.push_back(targetSym);
}
}

if (ctx.config.entry)
maybeCreateECExportThunk(ctx.config.entry->getName(), ctx.config.entry);
for (Export &e : ctx.config.exports) {
if (!e.data)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the case of mingw and exporting all (or practically, most) symbols, I guess ctx.config.exports already has been populated here, and e.data is set according to the heuristic there?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I added a test case for that to be sure too. I also fixed a broken assert from my previous version.

maybeCreateECExportThunk(e.extName.empty() ? e.name : e.extName, e.sym);
}
}

// In MinGW, if no symbols are chosen to be exported, then all symbols are
// automatically exported by default. This behavior can be forced by the
// -export-all-symbols option, so that it happens even when exports are
Expand Down Expand Up @@ -2520,6 +2586,9 @@ void LinkerDriver::linkerMain(ArrayRef<const char *> argsArr) {
if (!wrapped.empty())
wrapSymbols(ctx, wrapped);

if (isArm64EC(config->machine))
createECExportThunks();

// Resolve remaining undefined symbols and warn about imported locals.
ctx.symtab.resolveRemainingUndefines();
if (errorCount())
Expand Down
4 changes: 4 additions & 0 deletions lld/COFF/Driver.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,10 @@ class LinkerDriver {
// Convert Windows resource files (.res files) to a .obj file.
MemoryBufferRef convertResToCOFF(ArrayRef<MemoryBufferRef> mbs,
ArrayRef<ObjFile *> objs);

// Create export thunks for exported and patchable Arm64EC function symbols.
void createECExportThunks();
void maybeCreateECExportThunk(StringRef name, Symbol *&sym);
};

// Create enum with OPT_xxx values for each option in Options.td
Expand Down
3 changes: 3 additions & 0 deletions lld/COFF/SymbolTable.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -551,6 +551,9 @@ std::pair<Symbol *, bool> SymbolTable::insert(StringRef name) {
sym->pendingArchiveLoad = false;
sym->canInline = true;
inserted = true;

if (isArm64EC(ctx.config.machine) && name.starts_with("EXP+"))
expSymbols.push_back(sym);
}
return {sym, inserted};
}
Expand Down
3 changes: 3 additions & 0 deletions lld/COFF/SymbolTable.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,9 @@ class SymbolTable {
// A list of chunks which to be added to .rdata.
std::vector<Chunk *> localImportChunks;

// A list of EC EXP+ symbols.
std::vector<Symbol *> expSymbols;

// Iterates symbols in non-determinstic hash table order.
template <typename T> void forEachSymbol(T callback) {
for (auto &pair : symMap)
Expand Down
11 changes: 11 additions & 0 deletions lld/COFF/Writer.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,7 @@ class Writer {
uint64_t sizeOfHeaders;

OutputSection *textSec;
OutputSection *hexpthkSec;
OutputSection *rdataSec;
OutputSection *buildidSec;
OutputSection *dataSec;
Expand Down Expand Up @@ -984,6 +985,8 @@ void Writer::createSections() {

// Try to match the section order used by link.exe.
textSec = createSection(".text", code | r | x);
if (isArm64EC(ctx.config.machine))
hexpthkSec = createSection(".hexpthk", code | r | x);
createSection(".bss", bss | r | w);
rdataSec = createSection(".rdata", data | r);
buildidSec = createSection(".buildid", data | r);
Expand Down Expand Up @@ -2046,6 +2049,14 @@ void Writer::maybeAddRVATable(SymbolRVASet tableSymbols, StringRef tableSym,

// Create CHPE metadata chunks.
void Writer::createECChunks() {
for (Symbol *s : ctx.symtab.expSymbols) {
auto sym = dyn_cast<Defined>(s);
if (!sym || !sym->getChunk())
continue;
if (auto thunk = dyn_cast<ECExportThunkChunk>(sym->getChunk()))
hexpthkSec->addChunk(thunk);
}

auto codeMapChunk = make<ECCodeMapChunk>(codeMap);
rdataSec->addChunk(codeMapChunk);
Symbol *codeMapSym = ctx.symtab.findUnderscore("__hybrid_code_map");
Expand Down
188 changes: 188 additions & 0 deletions lld/test/COFF/arm64ec-export-thunks.test
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
REQUIRES: aarch64, x86
RUN: split-file %s %t.dir && cd %t.dir

RUN: llvm-mc -filetype=obj -triple=arm64ec-windows arm64ec-data.s -o arm64ec-data.obj
RUN: llvm-mc -filetype=obj -triple=arm64ec-windows arm64ec-func.s -o arm64ec-func.obj
RUN: llvm-mc -filetype=obj -triple=arm64ec-windows antidep-func.s -o antidep-func.obj
RUN: llvm-mc -filetype=obj -triple=arm64ec-windows arm64ec-data-sym.s -o arm64ec-data-sym.obj
RUN: llvm-mc -filetype=obj -triple=x86_64-windows x86_64-func.s -o x86_64-func.obj
RUN: llvm-mc -filetype=obj -triple=arm64ec-windows %S/Inputs/loadconfig-arm64ec.s -o loadconfig-arm64ec.obj

RUN: lld-link -out:exports.dll -machine:arm64ec arm64ec-func.obj x86_64-func.obj loadconfig-arm64ec.obj \
RUN: arm64ec-data.obj -dll -noentry -export:arm64ec_func -export:func=arm64ec_func \
RUN: -export:x86_64_func -export:data_sym,DATA

RUN: llvm-objdump -d exports.dll | FileCheck -check-prefix=EXP-DISASM %s
EXP-DISASM: Disassembly of section .text:
EXP-DISASM-EMPTY:
EXP-DISASM-NEXT: 0000000180001000 <.text>:
EXP-DISASM-NEXT: 180001000: 90000008 adrp x8, 0x180001000 <.text>
EXP-DISASM-NEXT: 180001004: 52800040 mov w0, #0x2
EXP-DISASM-NEXT: 180001008: d65f03c0 ret
EXP-DISASM-NEXT: ...
EXP-DISASM-EMPTY:
EXP-DISASM-NEXT: 0000000180002000 <x86_64_func>:
EXP-DISASM-NEXT: 180002000: e8 fb ef ff ff callq 0x180001000 <.text>
EXP-DISASM-NEXT: 180002005: b8 03 00 00 00 movl $0x3, %eax
EXP-DISASM-NEXT: 18000200a: c3 retq
EXP-DISASM-EMPTY:
EXP-DISASM-NEXT: Disassembly of section .hexpthk:
EXP-DISASM-EMPTY:
EXP-DISASM-NEXT: 0000000180003000 <func>:
EXP-DISASM-NEXT: 180003000: 48 8b c4 movq %rsp, %rax
EXP-DISASM-NEXT: 180003003: 48 89 58 20 movq %rbx, 0x20(%rax)
EXP-DISASM-NEXT: 180003007: 55 pushq %rbp
EXP-DISASM-NEXT: 180003008: 5d popq %rbp
EXP-DISASM-NEXT: 180003009: e9 f2 df ff ff jmp 0x180001000 <.text>
EXP-DISASM-NEXT: 18000300e: cc int3
EXP-DISASM-NEXT: 18000300f: cc int3
EXP-DISASM-EMPTY:
EXP-DISASM-NEXT: 0000000180003010 <arm64ec_func>:
EXP-DISASM-NEXT: 180003010: 48 8b c4 movq %rsp, %rax
EXP-DISASM-NEXT: 180003013: 48 89 58 20 movq %rbx, 0x20(%rax)
EXP-DISASM-NEXT: 180003017: 55 pushq %rbp
EXP-DISASM-NEXT: 180003018: 5d popq %rbp
EXP-DISASM-NEXT: 180003019: e9 e2 df ff ff jmp 0x180001000 <.text>
EXP-DISASM-NEXT: 18000301e: cc int3
EXP-DISASM-NEXT: 18000301f: cc int3

RUN: llvm-objdump -p exports.dll | FileCheck -check-prefix=EXP-EXPORT %s
EXP-EXPORT: Ordinal RVA Name
EXP-EXPORT-NEXT: 1 0x3010 arm64ec_func
EXP-EXPORT-NEXT: 2 0x6000 data_sym
EXP-EXPORT-NEXT: 3 0x3000 func
EXP-EXPORT-NEXT: 4 0x2000 x86_64_func

RUN: llvm-readobj --coff-load-config exports.dll | FileCheck -check-prefix=EXP-CHPE %s
EXP-CHPE: CodeMap [
EXP-CHPE-NEXT: 0x1000 - 0x100C ARM64EC
EXP-CHPE-NEXT: 0x2000 - 0x3020 X64
EXP-CHPE-NEXT: ]

RUN: llvm-objdump -s --section=.test exports.dll | FileCheck --check-prefix=EXP-DATA %s
EXP-DATA: 180006000 00300000 10300000

RUN: lld-link -out:exports2.dll -machine:arm64ec antidep-func.obj x86_64-func.obj loadconfig-arm64ec.obj \
RUN: arm64ec-data.obj -dll -noentry -export:arm64ec_func -export:func=arm64ec_func \
RUN: -export:x86_64_func -export:data_sym,DATA

RUN: llvm-objdump -d exports2.dll | FileCheck -check-prefix=EXP-DISASM %s
RUN: llvm-objdump -p exports2.dll | FileCheck -check-prefix=EXP-EXPORT %s
RUN: llvm-objdump -s --section=.test exports2.dll | FileCheck --check-prefix=EXP-DATA %s
RUN: llvm-readobj --coff-load-config exports2.dll | FileCheck -check-prefix=EXP-CHPE %s

RUN: lld-link -out:entry.dll -machine:arm64ec arm64ec-func.obj loadconfig-arm64ec.obj -dll -entry:arm64ec_func

RUN: llvm-objdump -d entry.dll | FileCheck -check-prefix=ENTRY-DISASM %s
ENTRY-DISASM: Disassembly of section .text:
ENTRY-DISASM-EMPTY:
ENTRY-DISASM-NEXT: 0000000180001000 <.text>:
ENTRY-DISASM-NEXT: 180001000: 90000008 adrp x8, 0x180001000 <.text>
ENTRY-DISASM-NEXT: 180001004: 52800040 mov w0, #0x2 // =2
ENTRY-DISASM-NEXT: 180001008: d65f03c0 ret
ENTRY-DISASM-EMPTY:
ENTRY-DISASM-NEXT: Disassembly of section .hexpthk:
ENTRY-DISASM-EMPTY:
ENTRY-DISASM-NEXT: 0000000180002000 <.hexpthk>:
ENTRY-DISASM-NEXT: 180002000: 48 8b c4 movq %rsp, %rax
ENTRY-DISASM-NEXT: 180002003: 48 89 58 20 movq %rbx, 0x20(%rax)
ENTRY-DISASM-NEXT: 180002007: 55 pushq %rbp
ENTRY-DISASM-NEXT: 180002008: 5d popq %rbp
ENTRY-DISASM-NEXT: 180002009: e9 f2 ef ff ff jmp 0x180001000 <.text>
ENTRY-DISASM-NEXT: 18000200e: cc int3
ENTRY-DISASM-NEXT: 18000200f: cc int3

RUN: llvm-readobj --headers entry.dll | FileCheck -check-prefix=ENTRY %s
ENTRY: AddressOfEntryPoint: 0x2000

RUN: llvm-readobj --coff-load-config entry.dll | FileCheck -check-prefix=ENTRY-CHPE %s
ENTRY-CHPE: CodeMap [
ENTRY-CHPE-NEXT: 0x1000 - 0x100C ARM64EC
ENTRY-CHPE-NEXT: 0x2000 - 0x2010 X64
ENTRY-CHPE-NEXT: ]


Test exporting data symbol as a function:

RUN: lld-link -out:data-func.dll -machine:arm64ec arm64ec-data-sym.obj loadconfig-arm64ec.obj -dll -noentry -export:data_sym

RUN: llvm-readobj --hex-dump=.test data-func.dll | FileCheck --check-prefix=DATAFUNC-TEST %s
DATAFUNC-TEST: Hex dump of section '.test':
DATAFUNC-TEST-NEXT: 0x180003000 00000000 ....

RUN: llvm-readobj --coff-exports --hex-dump=.test data-func.dll | FileCheck --check-prefix=DATAFUNC-EXP %s
DATAFUNC-EXP: Export {
DATAFUNC-EXP-NEXT: Ordinal: 1
DATAFUNC-EXP-NEXT: Name: data_sym
DATAFUNC-EXP-NEXT: RVA: 0x3000
DATAFUNC-EXP-NEXT: }


Test mingw-style auto-export:

RUN: lld-link -out:export-all.dll -machine:arm64ec arm64ec-func.obj loadconfig-arm64ec.obj -dll -noentry -lldmingw
RUN: llvm-objdump -d export-all.dll | FileCheck --check-prefix=EXPORT-ALL %s

EXPORT-ALL: Disassembly of section .text:
EXPORT-ALL-EMPTY:
EXPORT-ALL-NEXT: 0000000180001000 <.text>:
EXPORT-ALL-NEXT: 180001000: 90000008 adrp x8, 0x180001000 <.text>
EXPORT-ALL-NEXT: 180001004: 52800040 mov w0, #0x2 // =2
EXPORT-ALL-NEXT: 180001008: d65f03c0 ret
EXPORT-ALL-EMPTY:
EXPORT-ALL-NEXT: Disassembly of section .hexpthk:
EXPORT-ALL-EMPTY:
EXPORT-ALL-NEXT: 0000000180002000 <arm64ec_func>:
EXPORT-ALL-NEXT: 180002000: 48 8b c4 movq %rsp, %rax
EXPORT-ALL-NEXT: 180002003: 48 89 58 20 movq %rbx, 0x20(%rax)
EXPORT-ALL-NEXT: 180002007: 55 pushq %rbp
EXPORT-ALL-NEXT: 180002008: 5d popq %rbp
EXPORT-ALL-NEXT: 180002009: e9 f2 ef ff ff jmp 0x180001000 <.text>
EXPORT-ALL-NEXT: 18000200e: cc int3
EXPORT-ALL-NEXT: 18000200f: cc int3


#--- arm64ec-func.s
.text
.globl arm64ec_func
.p2align 2, 0x0
arm64ec_func:
adrp x8,arm64ec_func
mov w0, #2
ret

#--- antidep-func.s
.text
.globl "#arm64ec_func"
.p2align 2, 0x0
"#arm64ec_func":
adrp x8,arm64ec_func
mov w0, #2
ret

.weak_anti_dep arm64ec_func
arm64ec_func = "#arm64ec_func"

#--- arm64ec-data.s
.section .test, "r"
.globl data_sym
.p2align 2, 0x0
data_sym:
.rva "EXP+#func"
.rva "EXP+#arm64ec_func"

#--- x86_64-func.s
.text
.globl x86_64_func
.p2align 2, 0x0
x86_64_func:
call arm64ec_func
movl $3, %eax
retq

#--- arm64ec-data-sym.s
.section .test, "r"
.globl data_sym
.p2align 2, 0x0
data_sym:
.word 0
Loading
Loading