#expect will hang or crash when comparing unequal ClosedRange values

[ricky-sharp]

Description

Note: I filed this bug in Apple's Feedback (FB14192814) when I first discovered it on July 4th. As it's still an issue with the Beta 6, wanted to bring more awareness to it.

For any Swift Testing test where you use #expect to compare two ClosedRange<Int> instances, and the actual value deviates from the expected value, the test may hang or crash.

Note: While I will present details below for various values, I didn't yet do any testing with ClosedRange<Int8> or any other types. So possible that the issue doesn't just affect ClosedRange<Int>

Expected behavior

As with XCTest (which has no issues when using XCTAssertEqual), tests shouldn't hang or crash for trivial comparisons of ranges where an failure will be generated.

Actual behavior

Depending upon the values stored in lowerBound and/or upperBound, the tests will have three possible outcomes whenever a failure is expected:

1. The test does not hang and reports the failure correctly.
2. The test hangs (Xcode Beta 2 I believe had extremely long hangs. Betas 3 through 6 will hang for several seconds on an M2 Ultra Mac Studio. Though a failure will ultimately be generated.
3. The test crashes (same behavior in Beta 2 through Beta 6) with EXC_BREAKPOINT (Testing.__checkBinaryOperation)

Steps to reproduce

I'm attaching the sample project also attached to Apple's Feedback. Reproduction instructions:

1. Open the ExpectRange project with Xcode 16.0 Beta (I have not tried things with the 16.1 Beta, but expect the same outcome).
2. In the file ExpectRangeTests.swift I have set things up such that if Swift Testing is available, it will run those versions of the tests. Opening the project in Xcode 15.x will execute the equivalent XCTest tests which show no issue.
3. Run the first test "doesNotHangWithCorrectRanges". This shows that when no failure needs to be generated, Int values that could also be stored in Int8, UInt8, ... Int64 all have no issues.
4. Run the next two tests "int8_doesNotHang" and "uint8_doesNotHang". Note that failures should be immediately generated for all lines in the test (where I set just the upperBound to an invalid value, just the lowerBound and both).
5. Run any of "int16_hangs" through "uint32_hangs". These will ultimately generate failures, but run significantly slower that when the Int values are small (can fit within Int8/UInt8)
6. Run "int64_crashes" to see the crash.
7. Run "doesNotHang" which shows the workaround of using #expect to verify lowerBound and upperBound individually.

Note: Even though the test names mention "int8", "uint16", etc. all closed ranges involved are of type ClosedRange<Int>. Not to be confused with the actual types of Int8, UInt16, etc. It's interesting that for very small values stored in an Int that the tests don't hang, medium to semi-large will hang, and the largest values crash.

ExpectRange.zip

swift-testing version/commit hash

No response

Swift & OS version (output of swift --version ; uname -a)

swift-driver version: 1.115 Apple Swift version 6.0 (swiftlang-6.0.0.9.10 clang-1600.0.26.2)
Target: arm64-apple-macosx14.0
Darwin Ariel-Mac-Studio.local 23.6.0 Darwin Kernel Version 23.6.0: Mon Jul 29 21:13:04 PDT 2024; root:xnu-10063.141.2~1/RELEASE_ARM64_T6020 arm64

[ricky-sharp]

So as not require viewers of this bug to download the attached Xcode project, here are a few of the tests.

In all cases below, the tests should execute quickly and immediately report the failures (all tests expected to fail)

import Testing

struct ExpectRangeTests {
    let range_int8: ClosedRange<Int> = Int(Int8.min)...Int(Int8.max)
    let range_uint16: ClosedRange<Int> = Int(UInt16.min)...Int(UInt16.max)
    let range_int64: ClosedRange<Int> = Int(Int64.min)...Int(Int64.max)

    @Test("When working with an Int8 range, test does not hang if values incorrect")
    func int8_doesNotHang() {
        #expect(range_int8 == (-127)...127, "incorrect min")
        #expect(range_int8 == (-128)...128, "incorrect max")
        #expect(range_int8 == 0...0, "both incorrect")
    }

    @Test("When working with a UInt16 range, test hangs if values incorrect")
    func uint16_hangs() {
        #expect(range_uint16 == (-1)...65_535, "incorrect min")
        #expect(range_uint16 == 0...65_534, "incorrect max")
        #expect(range_uint16 == 1...1, "both incorrect")
    }

    @Test("When working with an Int64 range, test crashes if values incorrect")
    func int64_crashes() {
        // These all crash with EXC_BREAKPOINT (Testing.__checkBinaryOperation)
        #expect(range_int64 == (-9_223_372_036_854_775_807)...9_223_372_036_854_775_807, "incorrect min")
        #expect(range_int64 == (-9_223_372_036_854_775_808)...9_223_372_036_854_775_806, "incorrect max")
        #expect(range_int64 == 0...0, "both incorrect")
    }
}

[grynspan]

I see the issue(s). :) Thanks for filing!

[grynspan]

Tracked internally with rdar://131122002.