Parse specifer in swift-test #266

aciidgh · 2016-04-19T17:31:20Z

The specifier was not being parsed so is never passed to xctest.
swift test <test specifier>

ddunbar · 2016-04-19T18:00:15Z

Test case?

aciidgh · 2016-04-19T18:02:45Z

This is in swift-test executable which is not testable right now I think

ddunbar · 2016-04-19T18:03:33Z

Why not? Seems like we should have at least one test that verifies its behavior in a functional style.

aciidgh · 2016-04-19T18:06:41Z

something similar to XCTAssertBuilds ?

kostiakoval · 2016-04-19T18:30:34Z

I don't think we want to actually invoke swift test. It would be better to invoke parse method and test the returned result.

aciidgh · 2016-04-19T18:47:31Z

If I remember correctly importing an executable in test errors out with duplicate symbols for main

aciidgh · 2016-04-20T18:32:35Z

I've updated the PR to include two functional tests for swift-test which executes swift-test executable on a fixture.
I also had to introduce flag -s for specifiers because wasn't able to get a flag with no flag from option parser, @mxcl suggest if there is a way to do that without adding this flag.

mxcl · 2016-04-20T20:06:45Z

Tests/Functional/Utilities.swift

@@ -155,6 +155,44 @@ func executeSwiftBuild(_ chdir: String, configuration: Configuration = .Debug, p
    }
 }

+func swiftTestPath() -> String {
+    #if os(OSX)
+        for bundle in NSBundle.allBundles() where bundle.bundlePath.hasSuffix(".xctest") {


Every time you copy/pasta a kitten dies.

saved the kitten in 852af2e

aciidgh · 2016-04-21T08:41:13Z

rebased fixed conflicts

aciidgh · 2016-04-24T07:40:51Z

Looks like CI is working again, @ddunbar @mxcl can you run it on this

ddunbar · 2016-04-24T07:41:19Z

@swift-ci please test

aciidgh · 2016-04-24T15:55:44Z

This passes tests on my linux VM but Linux CI is failing with this error :

output: Compile Swift Module 'SwiftTestModule' (1 sources)
Compile Swift Module 'SwiftTestModuleTestSuite' (1 sources)
/tmp/ValidLayouts_SwiftTestModule.geloaT/SwiftTestModule/Tests/SwiftTestModule/SwiftTest.swift:2:8: error: no such module 'XCTest'
import XCTest
       ^
<unknown>:0: error: build had 1 command failures
swift-test: error: exit(1): /home/buildnode/jenkins/workspace/swift-package-manager-PR-Linux/buildbot_linux/swiftpm-linux-x86_64/debug/swift-build-tool -f /tmp/ValidLayouts_SwiftTestModule.geloaT/SwiftTestModule/.build/debug.yaml test

swift-test: /home/buildnode/jenkins/workspace/swift-package-manager-PR-Linux/buildbot_linux/swiftpm-linux-x86_64/debug/swift-test
/home/buildnode/jenkins/workspace/swift-package-manager-PR-Linux/swiftpm/Tests/Functional/SwiftTestTests.swift:18: error: SwiftTestTests.testSwiftTestTests : failed - `swift test []' failed:

exit(1): /home/buildnode/jenkins/workspace/swift-package-manager-PR-Linux/buildbot_linux/swiftpm-linux-x86_64/debug/swift-test --chdir /tmp/ValidLayouts_SwiftTestModule.geloaT/SwiftTestModule

Test Case 'SwiftTestTests.testSwiftTestTests' failed (0.708 seconds).

@mxcl @modocache any idea?

briancroom · 2016-04-24T16:54:44Z

Here's what I think is going on.

When executing in the CI environment, no swift toolchain is present, so XCTest is only available in the build directory chosen by the Swift build-script-impl. SwiftPM's bootstrap script receives a parameter with the path to the XCTest build directory (https://github.com/apple/swift/blob/master/utils/build-script-impl#L1417), and passes that through when invoking SwiftPM (https://github.com/apple/swift-package-manager/blob/master/Utilities/bootstrap#L639). I guess a mechanism will need to be introduced here to provide the XCTest build path when running these tests as well.

aciidgh · 2016-04-24T17:17:54Z

hmm if Utilities/bootstrap test also receives the same argument I can probably pass it further? maybe for now I can disable this test for linux.
@mxcl @ddunbar thoughts?

kostiakoval · 2016-04-24T18:05:51Z

It's the same error I experience before when tried to invoke swift test .
#143

aciidgh · 2016-04-24T18:09:53Z

thanks @kostiakoval for digging this out. Quoting Max's reply on that PR:

I've tried to make this work on Linux and my conclusion is we shouldn't bother with this test:

It is hard to make this work in the context of a full build of swift since the paths for XCTest and Foundation are not in the standard "toolchain" shape.
We are always testing swift-test because we use swift-test to test ourselves.
Instead we could add an integration test, but really we don't even need to because we test swift-test when we test ourselves.

So I'd like to close this, if there are no objections.

If this is still valid then can someone suggest another way to test this.

kostiakoval · 2016-04-24T18:16:05Z

@aciidb0mb3r Could you income parse function and check returned mode? if the mode is correct than the option parser works correctly. I think it's enough to unit test this part and we don't need to invoke swift test

aciidgh · 2016-04-24T18:17:14Z

Unit tests are not possible on executables right now due to duplicated main symbols

ddunbar · 2016-04-28T16:31:32Z

Sounds like we should remove the unit test from here, merge this and separately track finding a way to unit test this code? Any objections there?

aciidgh · 2016-04-28T17:13:42Z

@ddunbar no objections, updated PR

aciidgh · 2016-04-29T18:53:19Z

@mxcl @ddunbar ci over here pls 🤖👋

mxcl · 2016-04-29T19:36:45Z

@swift-ci Please test and merge

Upgrade to C API version 5

aciidgh force-pushed the swift-test-fix branch from 32fdb7e to 40f2b10 Compare April 20, 2016 18:29

mxcl reviewed Apr 20, 2016
View reviewed changes

aciidgh force-pushed the swift-test-fix branch 2 times, most recently from 852af2e to db95a20 Compare April 21, 2016 08:41

aciidgh mentioned this pull request Apr 28, 2016

SR-1298: [swift-test] Allow running specific tests #291

Closed

czechboy0 mentioned this pull request Apr 28, 2016

Rename xctest executable to avoid inferring it as module name #282

Merged

[swift-test] Parse specifer in swift-test

1e40127

aciidgh force-pushed the swift-test-fix branch from db95a20 to 1e40127 Compare April 28, 2016 17:13

swift-ci merged commit 311ec17 into swiftlang:master Apr 29, 2016

aciidgh deleted the swift-test-fix branch May 10, 2016 06:07

aciidgh pushed a commit to aciidgh/swift-package-manager that referenced this pull request Jan 11, 2019

Merge pull request swiftlang#266 from dmbryson/capi5

ca0d347

Upgrade to C API version 5

Bouke mentioned this pull request Apr 28, 2016

[SR-1298] swift-test: error: unknown command: TestModule.TestCase #5457

Closed

Parse specifer in swift-test #266

Parse specifer in swift-test #266

Uh oh!

Conversation

aciidgh commented Apr 19, 2016

Uh oh!

ddunbar commented Apr 19, 2016

Uh oh!

aciidgh commented Apr 19, 2016

Uh oh!

ddunbar commented Apr 19, 2016

Uh oh!

aciidgh commented Apr 19, 2016

Uh oh!

kostiakoval commented Apr 19, 2016

Uh oh!

aciidgh commented Apr 19, 2016

Uh oh!

aciidgh commented Apr 20, 2016

Uh oh!

mxcl Apr 20, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aciidgh Apr 20, 2016

Choose a reason for hiding this comment

Uh oh!

aciidgh commented Apr 21, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aciidgh commented Apr 24, 2016

Uh oh!

ddunbar commented Apr 24, 2016

Uh oh!

aciidgh commented Apr 24, 2016

Uh oh!

briancroom commented Apr 24, 2016

Uh oh!

aciidgh commented Apr 24, 2016

Uh oh!

kostiakoval commented Apr 24, 2016

Uh oh!

aciidgh commented Apr 24, 2016

Uh oh!

kostiakoval commented Apr 24, 2016

Uh oh!

aciidgh commented Apr 24, 2016

Uh oh!

ddunbar commented Apr 28, 2016

Uh oh!

aciidgh commented Apr 28, 2016

Uh oh!

aciidgh commented Apr 29, 2016

Uh oh!

mxcl commented Apr 29, 2016

Uh oh!

Uh oh!

mxcl Apr 20, 2016 •

edited

Loading

aciidgh commented Apr 21, 2016 •

edited

Loading