Skip to content

[README] Encourage using the Swift build script #77

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 12, 2016
Merged

[README] Encourage using the Swift build script #77

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions Documentation/Linux.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Additional Considerations for Swift on Linux

When running on the Objective-C runtime, XCTest is able to find all of your tests by simply asking the runtime for the subclasses of `XCTestCase`. It then finds the methods that start with the string `test`. This functionality is not currently present when running on the Swift runtime. Therefore, you must currently provide an additional property, conventionally named `allTests`, in your `XCTestCase` subclass. This method lists all of the tests in the test class. The rest of your test case subclass still contains your test methods.

```swift
class TestNSURL : XCTestCase {
static var allTests : [(String, TestNSURL -> () throws -> Void)] {
return [
("test_URLStrings", test_URLStrings),
("test_fileURLWithPath_relativeToURL", test_fileURLWithPath_relativeToURL),
("test_fileURLWithPath", test_fileURLWithPath),
("test_fileURLWithPath_isDirectory", test_fileURLWithPath_isDirectory),
// Other tests go here
]
}

func test_fileURLWithPath_relativeToURL() {
// Write your test here. Most of the XCTAssert macros you are familiar with are available.
XCTAssertTrue(theBestNumber == 42, "The number is wrong")
}

// Other tests go here
}
```

Also, this version of XCTest does not use the external test runner binary. Instead, create your own executable which links `libXCTest.so`. In your `main.swift`, invoke the `XCTMain` function with an array of the test cases classes that you wish to run, wrapped by the `testCase` helper function. For example:

```swift
XCTMain([testCase(TestNSString.allTests), testCase(TestNSArray.allTests), testCase(TestNSDictionary.allTests)])
```

The `XCTMain` function does not return, and will cause your test app to exit with either `0` for success or `1` for failure. Command line arguments given to the executable can be used to select a particular test or test case to execute. For example:

```sh
./FooTests FooTestCase/testFoo # Run a single test method
./FooTests FooTestCase # Run all the tests in FooTestCase
```

We are currently investigating ideas on how to make these additional steps for test discovery automatic when running on the Swift runtime.
102 changes: 18 additions & 84 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,113 +6,47 @@ This version of XCTest uses the same API as the XCTest you are familiar with fro

## Current Status and Project Goals

This project is the very earliest stages of development. It is scheduled to be part of the Swift 3 release.
This project is in the very earliest stages of development. It is scheduled to be part of the Swift 3 release.

Only the most basic functionality is currently present. This year, we have the following goals for the project:

* Finish implementing support for the most important non-UI testing APIs present in XCTest for Xcode
* Finish implementing support for the most important non-UI testing APIs present in XCTest for Xcode.
* Develop an effective solution to the problem of test discoverability without the Objective-C runtime.
* Provide support for efforts to standardize test functionality across the Swift stack.

For more details, visit the `Documentation` directory.

## Using XCTest

Your tests are organized into a simple hierarchy. Each `XCTestCase` subclass has a set of `test` methods, each of which should test one part of your code.

You can find all kinds of useful information on using XCTest in [Apple's documentation](https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/testing_with_xcode/chapters/03-testing_basics.html).

The rest of this document will focus on how this version of XCTest differs from the one shipped with Xcode.

## Working on XCTest

### On Linux

XCTest can be built as part of the overall Swift package. When following [the instructions for building Swift](http://www.github.com/apple/swift), pass the `--xctest` option to the build script:

```sh
swift/utils/build-script --xctest
```

If you want to build just XCTest, use the `build_script.py` script at the root of the project. The `master` version of XCTest must be built with the `master` version of Swift. XCTest has a dependency upon Foundation, so you must have built the `master` version of that as well.

If your install of Swift is located at `/swift` and you wish to install XCTest into that same location, here is a sample invocation of the build script:

```sh
./build_script.py \
--swiftc="/swift/usr/bin/swiftc" \
--build-dir="/tmp/XCTest_build" \
--foundation-build-dir "/swift/usr/lib/swift/linux" \
--library-install-path="/swift/usr/lib/swift/linux" \
--module-install-path="/swift/usr/lib/swift/linux/x86_64"
```

To run the tests on Linux, use the `--test` option:

```sh
./build_script.py \
--swiftc="/swift/usr/bin/swiftc" \
--foundation-build-dir "/swift/usr/lib/swift/linux" \
--test
```
## Contributing to XCTest

You may add tests for XCTest by including them in the `Tests/Functional/` directory. For an example, see `Tests/Functional/SingleFailingTestCase`.
To contribute, you'll need to be able to build this project and and run its test suite. The easiest way to do so is via the Swift build script.

### On OS X
First, follow [the instructions in the Swift README](https://github.com/apple/swift/blob/master/README.md) to build Swift from source. Confirm you're able to build the Swift project using `utils/build-script -R`.

You may build XCTest via the "SwiftXCTest" scheme in `XCTest.xcworkspace`. The workspace assumes that Foundation and XCTest are checked out from GitHub in sibling directories. For example:
Once you are able to build the Swift project, build XCTest and run its tests:

```
% cd Development
% ls
swift-corelibs-foundation swift-corelibs-xctest
%
$ cd swift-corelibs-xctest
$ ../swift/utils/build-script --preset corelibs-xctest
```

Unlike on Linux, you do not need to build Foundation prior to building XCTest. The "SwiftXCTest" Xcode scheme takes care of that for you.

To run the tests on OS X, build and run the `SwiftXCTestFunctionalTests` target in the Xcode workspace. You may also run them via the command line:
This project is only guaranteed to build with the very latest commit on the Swift and swift-corelibs-foundation `master` branches. You may update to the latest commits using the Swift `utils/update-checkout` script:

```
xcodebuild -workspace XCTest.xcworkspace -scheme SwiftXCTestFunctionalTests
```

When adding tests to the `Tests/Functional` directory, make sure they can be opened in the `XCTest.xcworkspace` by adding references to them, but do not add them to any of the targets.

### Additional Considerations for Swift on Linux

When running on the Objective-C runtime, XCTest is able to find all of your tests by simply asking the runtime for the subclasses of `XCTestCase`. It then finds the methods that start with the string `test`. This functionality is not currently present when running on the Swift runtime. Therefore, you must currently provide an additional property, conventionally named `allTests`, in your `XCTestCase` subclass. This method lists all of the tests in the test class. The rest of your test case subclass still contains your test methods.

```swift
class TestNSURL : XCTestCase {
static var allTests : [(String, TestNSURL -> () throws -> Void)] {
return [
("test_URLStrings", test_URLStrings),
("test_fileURLWithPath_relativeToURL", test_fileURLWithPath_relativeToURL),
("test_fileURLWithPath", test_fileURLWithPath),
("test_fileURLWithPath_isDirectory", test_fileURLWithPath_isDirectory),
// Other tests go here
]
}

func test_fileURLWithPath_relativeToURL() {
// Write your test here. Most of the XCTAssert macros you are familiar with are available.
XCTAssertTrue(theBestNumber == 42, "The number is wrong")
}

// Other tests go here
}
$ ../swift/utils/update-checkout
```

Also, this version of XCTest does not use the external test runner binary. Instead, create your own executable which links `libXCTest.so`. In your `main.swift`, invoke the `XCTMain` function with an array of the test cases classes that you wish to run, wrapped by the `testCase` helper function. For example:
### Using Xcode

```swift
XCTMain([testCase(TestNSString.allTests), testCase(TestNSArray.allTests), testCase(TestNSDictionary.allTests)])
```

The `XCTMain` function does not return, and will cause your test app to exit with either `0` for success or `1` for failure. Command line arguments given to the executable can be used to select a particular test or test case to execute. For example:
To browse files in this project using Xcode, use `XCTest.xcworkspace`. You may build the project using the "SwiftXCTest" scheme. Run the "SwiftXCTestFunctionalTests" scheme to run the tests.

```sh
./FooTests FooTestCase/testFoo # Run a single test method
./FooTests FooTestCase # Run all the tests in FooTestCase
```
However, in order to successfully build the project in Xcode, **you must use an Xcode toolchain with an extremely recent version of Swift**. The Swift website provides [Xcode toolchains to download](https://swift.org/download/#latest-development-snapshots), as well as [instructions on how to use Xcode with those toolchains](https://swift.org/download/#apple-platforms). Swift development moves fairly quickly, and so even a week-old toolchain may no longer work.

We are currently investigating ideas on how to make these additional steps for test discovery automatic when running on the Swift runtime.
> If none of the toolchains available to download are recent enough to build XCTest, you may build your own toolchain by using [the `utils/build-toolchain` script in the Swift repository](https://github.com/apple/swift/blob/master/utils/build-toolchain).
>
> Keep in mind that the build script invocation in "Contributing to XCTest" above will always work, regardless of which Swift toolchains you have installed. The Xcode workspace exists simply for the convenience of contibutors. It is not necessary to successfully build this project in Xcode in order to contribute.
60 changes: 38 additions & 22 deletions build_script.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
import subprocess
import sys
import tempfile
import textwrap

SOURCE_DIR = os.path.dirname(os.path.abspath(__file__))

Expand Down Expand Up @@ -184,40 +185,60 @@ def main(args=sys.argv[1:]):
function.
"""
parser = argparse.ArgumentParser(
description="Builds, tests, and installs XCTest.")
formatter_class=argparse.RawDescriptionHelpFormatter,
description=textwrap.dedent("""
Build, test, and install XCTest.

WARNING: This script is only meant to be used on Linux
environments, and in general should not be invoked directly. The
recommended way to build and test XCTest is via the Swift build
script. See this project's README for details.

The Swift build script invokes this %(prog)s script to build,
test, and install this project on Linux. Assuming you are on a
Linux environment, you may invoke it in the same way to build this
project directly. For example, If your install of Swift is located
at "/swift" and you wish to install XCTest into that same location,
here is a sample invocation of the build script:

$ %(prog)s \\
--swiftc="/swift/usr/bin/swiftc" \\
--build-dir="/tmp/XCTest_build" \\
--foundation-build-dir "/swift/usr/lib/swift/linux" \\
--library-install-path="/swift/usr/lib/swift/linux" \\
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious why you chose to remove the metavar values here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't think they added any value, and I figured I'd just include them in this pull request.

This is the help text diff:

diff --git a/build_script_help_text.txt b/build_script_help_text.txt
index 194df0d..e28cd3b 100755
--- a/build_script_help_text.txt
+++ b/build_script_help_text.txt

$ build_script.py build -h
# ...
optional arguments:
  -h, --help            show this help message and exit
-  --swiftc PATH       Path to the 'swiftc' compiler that will be used to
+  --swiftc SWIFTC     Path to the 'swiftc' compiler that will be used to
                        build XCTest.so, XCTest.swiftmodule, and
                        XCTest.swiftdoc. This will also be used to build the
                        tests for those built products if the --test option is
                        specified.
-  --build-dir PATH
+  --build-dir BUILD_DIR
                        Path to the output build directory. If not specified,
                        a temporary directory is used
-  --foundation-build-dir PATH
+  --foundation-build-dir FOUNDATION_BUILD_DIR
                        Path to swift-corelibs-foundation build products,
                        which the built XCTest.so will be linked against.
-  --swift-build-dir PATH
+  --swift-build-dir SWIFT_BUILD_DIR
                        deprecated, do not use
  --arch ARCH           deprecated, do not use
-  --module-install-path PATH
+  --module-install-path MODULE_PATH
                        Location at which to install XCTest.swiftmodule and
                        XCTest.swiftdoc. This directory will be created if it
                        doesn't already exist.
-  --library-install-path PATH
+  --library-install-path LIB_PATH
                        Location at which to install XCTest.so. This directory
                        will be created if it doesn't already exist.
  --release             builds for release
  --debug               builds for debug (the default)
  --test                Whether to run tests after building. Note that you
                        must have cloned https://github.com/apple/swift-llvm
                        at /Users/bgesiak/GitHub/apple/llvm in order to run
                        this command.

Personally I think it's less code without them, and removing them provides clearer help text. What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation! Looks great.

I'm pretty sure I'd trust your judgement over mine on things Python 😆 Always looking to learn.

--module-install-path="/swift/usr/lib/swift/linux/x86_64"
"""))
subparsers = parser.add_subparsers(
description="Use one of these to specify whether to build, test, "
"or install XCTest. If you don't specify any of these, "
"'build' is executed as a default. You may also use "
"'build' to also test and install the built products. "
"Pass the -h or --help option to any of the subcommands "
"for more information.")
description=textwrap.dedent("""
Use one of these to specify whether to build, test, or install
XCTest. If you don't specify any of these, 'build' is executed as a
default. You may also use 'build' to also test and install the
built products. Pass the -h or --help option to any of the
subcommands for more information."""))

build_parser = subparsers.add_parser(
"build",
description="Build XCTest.so, XCTest.swiftmodule, and XCTest.swiftdoc "
"using the given Swift compiler. This command may also "
"test and install the built products.")
description=textwrap.dedent("""
Build XCTest.so, XCTest.swiftmodule, and XCTest.swiftdoc using the
given Swift compiler. This command may also test and install the
built products."""))
build_parser.set_defaults(func=_build)
build_parser.add_argument(
"--swiftc",
help="Path to the 'swiftc' compiler that will be used to build "
"XCTest.so, XCTest.swiftmodule, and XCTest.swiftdoc. This will "
"also be used to build the tests for those built products if the "
"--test option is specified.",
metavar="PATH",
required=True)
build_parser.add_argument(
"--build-dir",
help="Path to the output build directory. If not specified, a "
"temporary directory is used",
metavar="PATH",
default=tempfile.mkdtemp())
build_parser.add_argument(
"--foundation-build-dir",
help="Path to swift-corelibs-foundation build products, which "
"the built XCTest.so will be linked against.",
metavar="PATH",
required=True)
build_parser.add_argument("--swift-build-dir",
help="deprecated, do not use")
Expand Down Expand Up @@ -262,8 +283,7 @@ def main(args=sys.argv[1:]):
test_parser.add_argument(
"build_dir",
help="An absolute path to a directory containing the built XCTest.so "
"library.",
metavar="PATH")
"library.")
test_parser.add_argument(
"--swiftc",
help="Path to the 'swiftc' compiler used to build and run the tests.",
Expand All @@ -278,7 +298,6 @@ def main(args=sys.argv[1:]):
"--foundation-build-dir",
help="Path to swift-corelibs-foundation build products, which the "
"tests will be linked against.",
metavar="PATH",
required=True)

install_parser = subparsers.add_parser(
Expand All @@ -288,19 +307,16 @@ def main(args=sys.argv[1:]):
install_parser.add_argument(
"build_dir",
help="An absolute path to a directory containing a built XCTest.so, "
"XCTest.swiftmodule, and XCTest.swiftdoc.",
metavar="PATH")
"XCTest.swiftmodule, and XCTest.swiftdoc.")
install_parser.add_argument(
"-m", "--module-install-path",
help="Location at which to install XCTest.swiftmodule and "
"XCTest.swiftdoc. This directory will be created if it doesn't "
"already exist.",
metavar="PATH")
"already exist.")
install_parser.add_argument(
"-l", "--library-install-path",
help="Location at which to install XCTest.so. This directory will be "
"created if it doesn't already exist.",
metavar="PATH")
"created if it doesn't already exist.")

# Many versions of Python require a subcommand must be specified.
# We handle this here: if no known subcommand (or none of the help options)
Expand Down