[Tests] Inline expectations and substitute [[@LINE]] numbers

[modocache]

1. Enhance xctest_checker such that it can parse "CHECK" prefixes that are in the middle of a line (instead of just at the very beginning). This allows us to inline the test expectations.
2. Enhance xctest_checker such that it substitutes the string "[[@line]]" with the current line number of the source file when performing checks. This mirrors the behavior of FileCheck (although that program also handles "[[@line+]]").
3. Inline all checks in the suite, and replace all hardcoded and regex line numbers in the functional test suite with the new substitution.

This allows us to add and remove arbitary lines in the functional test suite without manually updating line numbers! 💯

For example, the old tests looked like this:

// CHECK: Test Case 'ExpectationsTestCase.test_waitingForAnUnfulfilledExpectation_fails' started.
// CHECK: .*/Tests/Functional/Asynchronous/Expectations/main.swift:19: error: ExpectationsTestCase.test_waitingForAnUnfulfilledExpectation_fails : Asynchronous wait failed - Exceeded timeout of 0.2 seconds, with unfulfilled expectations: foo
// CHECK: Test Case 'ExpectationsTestCase.test_waitingForAnUnfulfilledExpectation_fails' failed \(\d+\.\d+ seconds\).
func test_waitingForAnUnfulfilledExpectation_fails() {
    waitForExpectationsWithTimeout(0.2, handler: nil)
}

The new tests look like this (notice we don't hardcode the line number "19"):

func test_waitingForAnUnfulfilledExpectation_fails() { // CHECK: Test Case 'ExpectationsTestCase.test_waitingForAnUnfulfilledExpectation_fails' started.
    expectationWithDescription("foo")
    waitForExpectationsWithTimeout(0.2, handler: nil) // CHECK: .*/Asynchronous/Expectations/main.swift:[[@LINE]]: error: ExpectationsTestCase.test_waitingForAnUnfulfilledExpectation_fails : Asynchronous wait failed - Exceeded timeout of 0.2 seconds, with unfulfilled expectations: foo
} // CHECK: Test Case 'ExpectationsTestCase.test_waitingForAnUnfulfilledExpectation_fails' failed \(\d+\.\d+ seconds\).

[briancroom]

This looks really helpful! Nice Python-fu.

[briancroom]

While the line number substitution is unquestionably a maintainability win, I'm concerned that these changes harm the test legibility a bit. Because the // CHECK: lines aren't lined up anymore, it gets a lot more difficult for my eye to know where to look to start trying to parse the line contents, I'm finding. It starts looking more like an unstructured wall of text. (Possibly relevant - do you have line-wrapping enabled in your editor?)

I'd be interested to hear how others feel about how this effects test readability!

[mike-ferris]

Would supporting [[@line-1]] and/or [[@line+1]] help? I agree that having the comment strain the actual content of the line makes for long lines.

[modocache]

Hmm, interesting. I don't have line-wrapping enabled, and I must admit I do have to do a lot of side scrolling to see the expected text. So I can see where you two are coming from.

I'll try adding support for LINE+, like FileCheck has.

[modocache]

@briancroom @mike-ferris-apple OK, the expectations are no longer inlined, and instead use offsets like [[@LINE+3]]. This allows us to add and remove arbitary lines in the functional test suite without manually updating line numbers in most cases. Line offsets will still need to be updated when the distance between the CHECK and the test failure changes--for example:

diff --git a/Tests/Functional/FailureMessagesTestCase/main.swift b/Tests/Functional/FailureMessagesTestCase/main.swift
index 1f7e4f6..3c50ff1 100644
--- a/Tests/Functional/FailureMessagesTestCase/main.swift
+++ b/Tests/Functional/FailureMessagesTestCase/main.swift
@@ -158,9 +158,10 @@ class FailureMessagesTestCase: XCTestCase {
     }

     // CHECK: Test Case 'FailureMessagesTestCase.testFail' started.
-    // CHECK: test.swift:[[@LINE+3]]: error: FailureMessagesTestCase.testFail : failed - message
+    // CHECK: test.swift:[[@LINE+4]]: error: FailureMessagesTestCase.testFail : failed - message
     // CHECK: Test Case 'FailureMessagesTestCase.testFail' failed \(\d+\.\d+ seconds\).
     func testFail() {
+        // Adding this comment forces me to update the [[@LINE]] offset
         XCTFail("message", file: "test.swift")
     }

I think this makes it much easier to write tests!

[modocache]

I'm closing this to eliminate some noise on this repository's pull requests. The spiritual successor to this pull request is #94. As @briancroom mentions in that discussion, it's probably best to migrate all of this over to LLVM FileCheck.