[ABI] Introduce an ABI checker that diffs symbols

[Azoy]

NOTE: This is still a draft, but I didn't want to mark the pull request as such because I wanted it to ping the stdlib reviewers. DO NOT MERGE YET.

This adds two new tests cases that takes a baseline macOS arm64/x86_64 stdlib release no assert symbols and diffs them against the just built stdlib's symbols. If there are additions or removals of symbols it will be caught here (unless symbols were modified on different platforms, archs, or configurations that I haven't created a baseline for yet).

To test that this works, I've added a new API with availability and removed an old legacy ABI symbol to exercise that the test will fail. Once the test fails, I will update the test case with the expected changes and rerun tests. Once the test (hopefully) passes with the expected changes, I'll remove the new API and uncomment the legacy API and just have the tests and baselines added.

One caveat that I think we should fix is that you can build multiple variations of the stdlib on one machine. For example, on macOS you can build both a x86_64 and arm64 stdlib on Apple Silicon, but there are no features indicating that we have both of those configurations available. With that in mind, these tests would fail on build-script invocations that are cross compiling for architectures other than the one your host is on. So it will probably require some testing changes to include such feature before being merged.

It might be easier for maintainability if the script to diff the symbols was in something more portable like a python script than shell scripts.

[Azoy]

@swift-ci please smoke test

[Azoy]

@swift-ci please smoke test

[Azoy]

@swift-ci please test

[Azoy]

preset=buildbot_incremental,tools=RA,stdlib=RD,build
@swift-ci please test with preset macOS

[lorentey]

Yes please! This is looking good 👍

[Azoy]

@swift-ci please smoke test

[Azoy]

@swift-ci please smoke test

[Azoy]

@swift-ci please smoke test macOS

[Azoy]

@swift-ci please smoke test macOS

[Azoy]

******************** TEST 'Swift(macosx-x86_64) :: abi/macOS/x86_64/stdlib-asserts.swift' FAILED ********************
Script:
--
: 'RUN: at line 1';   rm -rf "/Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/build/buildbot_incremental/swift-macosx-x86_64/test-macosx-x86_64/abi/macOS/x86_64/Output/stdlib-asserts.swift.tmp" && mkdir -p "/Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/build/buildbot_incremental/swift-macosx-x86_64/test-macosx-x86_64/abi/macOS/x86_64/Output/stdlib-asserts.swift.tmp"
: 'RUN: at line 2';   /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/build/buildbot_incremental/llvm-macosx-x86_64/bin/llvm-nm -g --defined-only -f just-symbols /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/build/buildbot_incremental/swift-macosx-x86_64/lib/swift/macosx/x86_64/libswiftCore.dylib > /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/build/buildbot_incremental/swift-macosx-x86_64/test-macosx-x86_64/abi/macOS/x86_64/Output/stdlib-asserts.swift.tmp/symbols
: 'RUN: at line 3';   /Applications/Xcode.app/Contents/Developer/usr/bin/python3 /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/utils/swift-abi-symbol-checker.py /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/test/abi/macOS/x86_64/stdlib-asserts.swift /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/build/buildbot_incremental/swift-macosx-x86_64/test-macosx-x86_64/abi/macOS/x86_64/Output/stdlib-asserts.swift.tmp/symbols
: 'RUN: at line 4';   diff -u /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/test/abi/macOS/x86_64/../../Inputs/macOS/x86_64/stdlib/baseline-asserts /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/build/buildbot_incremental/swift-macosx-x86_64/test-macosx-x86_64/abi/macOS/x86_64/Output/stdlib-asserts.swift.tmp/symbols
--
Exit Code: 1

Command Output (stdout):
--
--- /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/test/abi/macOS/x86_64/../../Inputs/macOS/x86_64/stdlib/baseline-asserts	2023-12-05 18:09:34.000000000 +0000
+++ /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/build/buildbot_incremental/swift-macosx-x86_64/test-macosx-x86_64/abi/macOS/x86_64/Output/stdlib-asserts.swift.tmp/symbols	2023-12-05 18:56:58.000000000 +0000
@@ -11339,6 +11339,7 @@
 _$ss5_rintyS2dF
 _$ss5_rintyS2fF
 _$ss5_rintys7Float80VACF
+_$ss5helloyyF
 _$ss5print_9separator10terminator2toyypd_S2Sxzts16TextOutputStreamRzlF
 _$ss5print_9separator10terminatoryypd_S2StF
 _$ss6HasherV11_TailBufferVMa
@@ -12803,7 +12804,6 @@
 _$ss7UnicodeO5UTF16O6decodeys0A14DecodingResultOxzStRzs6UInt16V7ElementRtzlF
 _$ss7UnicodeO5UTF16O6encode_4intoyAB6ScalarV_ys6UInt16VXEtFZ
 _$ss7UnicodeO5UTF16O6encodeys11_UIntBufferVys6UInt16VGSgAB6ScalarVFZ
-_$ss7UnicodeO5UTF16O8_isASCIIySbs6UInt16VFZ
 _$ss7UnicodeO5UTF16O9_isScalarySbs6UInt16VFZ
 _$ss7UnicodeO5UTF16O9transcode_4froms11_UIntBufferVys6UInt16VGSg13EncodedScalarQz_xmts01_A8EncodingRzlFZ
 _$ss7UnicodeO5UTF16OADycfC

--

********************

Ok, this finally has caught the changes I proposed here. Now that we know this can catch these changes, I'm going to edit the exception list to be made aware of those changes. If that passes I will remove the ABI changes and remove the exceptions and commit.

[Azoy]

@swift-ci please smoke test macOS

[Azoy]

PASS: Swift(macosx-x86_64) :: abi/macOS/x86_64/stdlib-asserts.swift (17099 of 17099)

[Azoy]

@swift-ci please test