-
Notifications
You must be signed in to change notification settings - Fork 10.5k
[android] Move the string and other tags in pointers to the second byte because Android enabled memory tagging #40779
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
cbf36ec
to
2f492c4
Compare
I asked about Android's new tagging in android/ndk#1653 earlier this month and was told the OS reserves the entire top byte of pointers since Android 11, so I reworked this pull to move Swift's tags to the second byte. This pull now gets almost all the same tests to pass in the compiler validation suite as on a device without tagging, with only four more test modules failing on an Android device with tagging enabled. One is expected to fail because of the moved tag, and the other three don't fail if I run the test in-process and a few test cases at a time, so there's something flaky in However, once I build Foundation with this pull and try to run its tests on a device with tagging, I get a segfault when trying to instantiate a testcase array and append to it, in the compiler-generated function @milseman, would you review what I have so far? |
How are you testing these changes on Android? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems like the correct way to go about implementing this direction, modulo some cleanups, if this is the right direction to go.
What's the testing story on Android?
@@ -161,8 +161,13 @@ static_assert(alignof(HeapObject) == alignof(void*), | |||
(__swift_uintptr_t) SWIFT_ABI_ARM64_OBJC_RESERVED_BITS_MASK | |||
#define _swift_abi_ObjCReservedLowBits \ | |||
(unsigned) SWIFT_ABI_ARM64_OBJC_NUM_RESERVED_LOW_BITS | |||
#if defined(__ANDROID__) | |||
#define _swift_BridgeObject_TaggedPointerBits \ | |||
(__swift_uintptr_t) SWIFT_ABI_DEFAULT_BRIDGEOBJECT_TAG_64 >> 8 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we make a separate SWIFT_ABI_DEFAULT_BRIDGEOBJECT_TAG_64
for Android? Something like a SWIFT_ABI_ANDROID_BRIDGEOBJECT_TAG_64
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could, but note that this is only shifted for AArch64, so I'd have to define it only for that. I figured shifting it here is the best way to set that up, but I could add another constant instead if you like.
stdlib/public/core/KeyPath.swift
Outdated
@@ -1764,7 +1764,12 @@ internal struct KeyPathBuffer { | |||
internal mutating func pushRaw(size: Int, alignment: Int) | |||
-> UnsafeMutableRawBufferPointer { | |||
var baseAddress = buffer.baseAddress.unsafelyUnwrapped | |||
#if os(Android) && arch(arm64) | |||
// Android AArch64 may tag the first byte so remove it before checking alignment. | |||
var misalign = (0x00FF_FFFF_FFFF_FFFF & Int(bitPattern: baseAddress)) % alignment |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Someone else should review this impact. We may want/need to tidy up our macros and builtins some to avoid a lot of error-prone #if
logic.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Masking off the top bytes should not affect the alignment computation. Is the problem here the signed-ness of %
? How about one of these instead, which are also probably more accurate on other platforms and don't need to be conditionalized:
var misalign = Int(UInt(bitPattern: baseAddress)) % alignment
or
var misalign = Int(bitPattern: baseAddress) & (alignment - 1)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or if the standard library has since grown standard APIs for pointer alignment, maybe we can adopt those here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@buttaface ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
var misalign = Int(UInt(bitPattern: baseAddress)) % alignment
I just tried it and six stdlib test modules fail, presumably because this doesn't avoid the problem of the top sign bit being set.
var misalign = Int(bitPattern: baseAddress) & (alignment - 1)
All the same stdlib tests pass when built in release mode, but this assumes the alignment is a power of two. I figured you were using the modulo before because you couldn't assume that, can you?
I ran some searches in the stdlib and while there is a Builtin._roundUp()
function, I don't see anything that returns the misalignment, as we're calculating here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jckarter I couldn't tell from the conversation, but has your concern been addressed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can assume the alignment is a power of two. We should use var misalign = Int(bitPattern: baseAddress) & (alignment - 1)
everywhere. The existing code would also be a bug on other 64-bit platforms that might have the high bit set for a valid pointer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will do.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
stdlib/public/core/KeyPath.swift
Outdated
@@ -3242,7 +3247,12 @@ internal struct InstantiateKeyPathBuffer: KeyPathPatternVisitor { | |||
) { | |||
let alignment = MemoryLayout<T>.alignment | |||
var baseAddress = destData.baseAddress.unsafelyUnwrapped | |||
#if os(Android) && arch(arm64) | |||
// Android AArch64 may tag the first byte so remove it before checking alignment. | |||
var misalign = (0x00FF_FFFF_FFFF_FFFF & Int(bitPattern: baseAddress)) % alignment |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated, same as above.
stdlib/public/runtime/HeapObject.cpp
Outdated
#elif defined(__ANDROID__) && defined(__aarch64__) | ||
// Check the top of the second byte instead, since Android AArch64 reserves | ||
// the top byte for its own pointer tagging since Android 11. | ||
return (intptr_t)p << 8 > 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same, someone else should be commenting on the impact here and whether we could clean up how we define our ABI some.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As you've written it, this is undefined behavior if tag bytes are shifted out because of signed overflow. However, the goal on other 64-bit platforms here is to reserve some address space for "free" for small closure contexts and such, since checking for x == 0
and x < 0
are both one CPU instruction, but I don't think you can do the comparison you'd need here without putting another instruction on the swift_retain
hot path. We don't currently take advantage of this optimization, so you could do p != nullptr
like on 32-bit platforms for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@buttaface ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is undefined behavior if tag bytes are shifted out because of signed overflow
I'll look into it, and try your alternative.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you want to use the shifted version, then you can use unsigned rather than signed types. But I don't think any of this is necessary—it's sufficient and cheaper to check for != 0
. We do not currently use the negative bit optimization for anything on existing 64-bit platforms, and when we do, we would need to disable or modify those optimizations for MTE-enabled Android anyway, since there would be fewer bits available for storing data than on other platforms.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you want to use the shifted version, then you can use unsigned rather than signed types.
I'll try that: I'll see if clang accepts shifting it as an unsigned type, then casting the result to a signed type for the comparison.
But I don't think any of this is necessary—it's sufficient and cheaper to check for != 0. We do not currently use the negative bit optimization for anything on existing 64-bit platforms, and when we do, we would need to disable or modify those optimizations for MTE-enabled Android anyway, since there would be fewer bits available for storing data than on other platforms.
You're saying the top bit is currently never intentionally set for pointers as an immortal bit on 64-bit platforms? Or that we should now ignore the immortal bit when set on Android AArch64? Because when I tried your p != nullptr
alternative earlier, several stdlib tests started failing, whereas my two alternatives here that check the immortal bit keep them passing.
I can investigate further why that is, if you believe only checking for null pointers should suffice.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, which tests are those that fail?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I ran a little experiment to see if I can reproduce the stdlib test failures on linux x86_64 and I can. I setup a Ubuntu 20.04 VPS and downloaded the latest Mar. 30 trunk snapshot build and source, before running the following command to build the standalone stdlib and run its tests:
./swift/utils/build-script -RA --build-swift-tools=0 --skip-build-cmark --build-llvm=0
--native-clang-tools-path=/home/butta/swift-DEVELOPMENT-SNAPSHOT-2022-03-30-a-ubuntu20.04/usr/bin
--native-swift-tools-path=/home/butta/swift-DEVELOPMENT-SNAPSHOT-2022-03-30-a-ubuntu20.04/usr/bin
-j9 -T --lit-args=--filter=stdlib/ --skip-test-early-swift-driver
That runs about 530 stdlib tests and only 5 fail, because I don't have needed utilities like llvm-nm installed.
Then I changed the line below from return (intptr_t)p > 0
to return p != nullptr
and reran the above build command, only to see 350 tests failing now. Looking at the results, I see several segfaults like this, ie many test binaries segfault if we only check for the null pointer:
/home/butta/build/Ninja-Release/swift-linux-x86_64/validation-test-linux-x86_64/stdlib/Output/Bitset.swift.script:
line 1: 356022 Segmentation fault (core dumped)
/usr/bin/env LD_LIBRARY_PATH='/home/butta/build/Ninja-Release/swift-linux-x86_64/lib/swift/linux'
/home/butta/build/Ninja-Release/swift-linux-x86_64/validation-test-linux-x86_64/stdlib/Output/Bitset.swift.tmp/a.out
Around the same number of stdlib tests failed on Android AArch64 when I only checked for the null pointer a couple weeks ago as you suggested, which you can reproduce on linux if you like using the above steps: let me know if it is unexpected and if we should investigate further.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've updated this line to shift again with return (intptr_t)((uintptr_t)p << 8) > 0
, which maintains the same shifting three-instruction assembly output above and hopefully threads your UB needle too.
I build the Swift toolchain from scratch on an Android tablet in the Termux app and run the compiler validation suite, currently with the last trunk source snapshot from Jan. 9. |
I put together a small patch for the metadata issue and it got the remaining parts of the trunk toolchain building and passing their tests up through SPM, plus the three flaky test modules from the validation suite that I mentioned above all pass now. It is a hack as it just checks for the static tag |
2f492c4
to
ca127bd
Compare
auto checkNotAndroidTag = subIGF.Builder.CreateICmpNE( | ||
getBitAfterAndroidTag, llvm::ConstantInt::get(IGM.Int64Ty, 0)); | ||
|
||
comparison = subIGF.Builder.CreateAnd(comparison, checkNotAndroidTag); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reworked the metadata patch I linked before and have it working for an arbitrary Android tag in the top byte, by using the scheme described in the comments.
Strictly speaking, only the 24th bit matters on Android AArch64 with this new scheme, but I have it checking the 32nd "sign bit" too.
This pull now gets trunk building and passing all but the one test I linked before, both on devices with and without pointer tagging.
ca127bd
to
0fd5d1d
Compare
SWIFT_ABI_ARM64_SWIFT_SPARE_BITS_MASK); | ||
if (triple.isAndroid()) | ||
setToMask(target.PointerSpareBits, 64, | ||
SWIFT_ABI_ANDROID_ARM64_SWIFT_SPARE_BITS_MASK); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated to use the right tag mask at runtime, as selecting it at compile-time as I had it before wouldn't apply the right mask when cross-compiling, and rebased.
0fd5d1d
to
890f398
Compare
stdlib/public/SwiftShims/System.h
Outdated
// Android AArch64 reserves the top byte for pointer tagging since Android 11, | ||
// so shift these tags to the second byte. | ||
#define SWIFT_ABI_ANDROID_ARM64_SWIFT_SPARE_BITS_MASK 0x00F0000000000007ULL | ||
#define SWIFT_ABI_ANDROID_ARM64_OBJC_RESERVED_BITS_MASK 0x0080000000000000ULL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I only moved the spare bits mask initially, as this ObjC mask appeared to have no effect, but after turning stdlib assertions on to test the new _internalInvariantFailure
assertions below, this mask being in the top byte caused several tests to fail, because of the isNative
checks some assertions added. Because of those stdlib assertion failures alone, I shifted this ObjC mask and all stdlib tests then passed in Debug+assert mode.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What happens if we just omit this definition then? That is, does it have less source impact to omit objc masks and support?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This OBJC mask is completely unused except in some assertions, which simply check isNative
in some invariants, even though that makes no sense with ObjC interoperability turned off? Rather than have all those tests fail when assertions are turned on, I added this mask.
However, I don't really care if this assertions workaround is upstreamed and can remove it if you like.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you elaborate on why those assertions would fail if we reserve no bits for ObjC? I'm ok with Android reserving bits for ObjC, even if it's a bit weird, if it makes sense.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The problem is that without this change the default is to reserve the top bit on all AArch64 platforms, which is then checked against the top bit in Android's memory tag in isNative
, and isNative
is checked by several assertions.
Presumably this is a mistake in checking this ObjC bit even when ObjC interoperability is turned off, so I worked around it by moving the ObjC bit down a byte, out of the Android memory tag in the top byte.
If you'd rather set this mask to zero everywhere on Android AArch64, that might work too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I set this OBJC mask to zero and ran the full compiler validation suite on a Debug+asserts stdlib and got the same passing test results, so I've now set this mask to zero. Since this is just a workaround for stdlib assertions that wrongly check isNative
even when ObjC interop is turned off, I'm fine with not modifying this OBJC mask in any way in this pull and fixing that root cause in a separate pull later instead.
@milseman, just let me know which you prefer.
@milseman, this pull now fixes all the tagging collisions on Android AArch64 and I responded to all your review comments. Let me know what you think. |
… Android enabled memory tagging - pull request swiftlang/swift#40779 - log A/SwiftRuntime: Swift/UnicodeHelpers.swift:266: Fatal error: No foreign strings on Linux in this version of Swift A/libc: Fatal signal 5 (SIGTRAP), code 1 (TRAP_BRKPT), fault addr 0x7102456948 in tid 10696 (ts.todomvvmlive), pid 10696 (ts.todomvvmlive)
@CodaFi, this pull has been ready for three weeks now, but I can't get anyone to respond anymore. Maybe you can take over the review? Since everything is scoped to Android AArch64 only, this should be perfectly safe to merge. |
@milseman and @jckarter, I appreciate the review on this pull, but since Android AArch64 is not an officially supported platform and everything here is exclusively scoped to only affect Android AArch64, do you think we can this into trunk and the 5.6 branch before the 5.6 release? Since this pull requires compiler changes, I was hoping to get my Android cross-compilation SDKs, which work with an official prebuilt Swift compiler, working with this new tagging scheme by getting this pull in before the Swift 5.6 release. If you think it's too late to get this in to 5.6 or just want to tune this more, I'm not in a hurry other than that. I've answered all your review comments above. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall this is LGTM, though I'd like to minimize the number of source changes to areas that aren't supported anyways (ObjC masks and bits, etc).
stdlib/public/SwiftShims/System.h
Outdated
// Android AArch64 reserves the top byte for pointer tagging since Android 11, | ||
// so shift these tags to the second byte. | ||
#define SWIFT_ABI_ANDROID_ARM64_SWIFT_SPARE_BITS_MASK 0x00F0000000000007ULL | ||
#define SWIFT_ABI_ANDROID_ARM64_OBJC_RESERVED_BITS_MASK 0x0080000000000000ULL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What happens if we just omit this definition then? That is, does it have less source impact to omit objc masks and support?
stdlib/public/core/KeyPath.swift
Outdated
@@ -1764,7 +1764,12 @@ internal struct KeyPathBuffer { | |||
internal mutating func pushRaw(size: Int, alignment: Int) | |||
-> UnsafeMutableRawBufferPointer { | |||
var baseAddress = buffer.baseAddress.unsafelyUnwrapped | |||
#if os(Android) && arch(arm64) | |||
// Android AArch64 may tag the first byte so remove it before checking alignment. | |||
var misalign = (0x00FF_FFFF_FFFF_FFFF & Int(bitPattern: baseAddress)) % alignment |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jckarter I couldn't tell from the conversation, but has your concern been addressed?
stdlib/public/runtime/HeapObject.cpp
Outdated
#elif defined(__ANDROID__) && defined(__aarch64__) | ||
// Check the top of the second byte instead, since Android AArch64 reserves | ||
// the top byte for its own pointer tagging since Android 11. | ||
return (intptr_t)p << 8 > 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@buttaface can you address the undefined behavior aspect?
@milseman, been a couple weeks now, can we go ahead and get this in, as I don't want to miss the 5.7 branch? If you can just sign off on the two unresolved issues you raised, and either you decide on the three we're waiting on Joe for or we can just get this in and always change it later whenever he chimes in, since this pull doesn't affect any officially supported platform, only Android AArch64. I will rebase and update this pull once you let me know on those review comments. |
It would be good to avoid introducing UB, fix bugs, and avoid unnecessary conditional compilation, so if we can get the fix to the alignment computation in KeyPath, and remove the UB in the null-or-negative pointer check, then I think we're OK. |
I agree that we don't want to add UB to the swift runtime |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, assuming the runtime stuff looks good and no UB
Looks fine to me now. |
…te because Android enabled memory tagging Starting with Android 11, AArch64 placed a tag in the top byte of pointers to allocations, which has been slowly rolling out to more devices and collides with Swift's tags. Moving these tags to the second byte works around this problem.
Rebased and updated a comment, @artemcm, would you run the CI? |
@kateinoigakukun, I see you're up, mind running the CI on this pull for me? |
@swift-ci Please test |
@milseman, passed CI, would you merge? |
- swiftlang#58975 switched many tests from XFAIL on linux to linux-gnu, so seven fail on the Android CI. They are now explicitly excluded. - swiftlang#42478 removed the @NoEscape attribute for the non-Android SIL/clang-function-types tests, so I remove it for Android too. - My pull swiftlang#40779 moved the Swift pointer tags to the second byte, so SILOptimizer/concat_string_literals.64 will need to be updated for that, disabled it for now. - Compiler-rt moved the directory in which it places those libraries for Android, llvm/llvm-project@a68ccba, so lit.cfg is updated for that.
- swiftlang#58975 switched many tests from XFAIL on linux to linux-gnu, so seven fail on the Android CI. They are now explicitly excluded. - swiftlang#42478 removed the @NoEscape attribute for the non-Android SIL/clang-function-types tests, so I remove it for Android too. - My pull swiftlang#40779 moved the Swift pointer tags to the second byte, so SILOptimizer/concat_string_literals.64 will need to be updated for that, disabled it for now. - Compiler-rt moved the directory in which it places those libraries for Android, llvm/llvm-project@a68ccba, so lit.cfg is updated for that.
- swiftlang#58975 switched many tests from XFAIL on linux to linux-gnu, so seven fail on the Android CI and two natively. They are now explicitly excluded. - swiftlang#39605 added several C++ Interop tests, 11 of which fail on the Android CI, so disable them for now. - swiftlang#42478 removed the @NoEscape attribute for the non-Android SIL/clang-function-types tests, so I remove it for Android too. - My pull swiftlang#40779 moved the Swift pointer tags to the second byte, so SILOptimizer/concat_string_literals.64 will need to be updated for that, disabled it for now. - Compiler-rt moved the directory in which it places those libraries for Android, llvm/llvm-project@a68ccba, so lit.cfg is updated for that.
Android added Software Tagging (ST from here on) to the top byte in their malloc a couple years ago, which has rolled out to some fraction of Android 11+ devices now and collides with Swift's String tag in the top half of the top byte. While the software tagging is in preparation for switching to hardware tagging with ARM's MTE, for some reason ST sets a tag across the entire top byte even though MTE is only supposed to use the bottom half of the top byte.
I was surprised to find that @milseman had reorganized the String tag to accommodate MTE in #21310 three years ago, which was followed by #21453, but since ST writes to the top half too, that doesn't work. Specifically, ST causes two problems with Swift:
B4
written to the top byte when allocated bymalloc
, which means it's tagged as immortal, small, and foreign, with the last one causing errors on Android. Running the Swift compiler validation suite, about 650 test modules have failures, with this earlier patch of mine that only addresses this String issue by moving the small and foreign bits fixing half of those failing tests, though it means ARC is not run on all those "immortal" strings and they leak.@hctim, you added this software tagging a couple years ago: any reason why you didn't confine it to the bottom half of the top byte, as the hardware MTE is supposed to do? If Android sticks with MTE in the future, this pull to work around the current ST won't be necessary.
I'm asking before I go any further to find where those tags are being removed, and slapping this pull up as a draft for feedback in the meantime.