-
Notifications
You must be signed in to change notification settings - Fork 80
Add infrastructure for testing IndexStoreDB #32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
benlangmuir
merged 1 commit into
swiftlang:master
from
benlangmuir:test-infrastructure-rebased
Aug 1, 2019
Merged
Add infrastructure for testing IndexStoreDB #32
benlangmuir
merged 1 commit into
swiftlang:master
from
benlangmuir:test-infrastructure-rebased
Aug 1, 2019
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The goal of this change is to make it convenient to add tests for the index that test the whole process of producing the raw index data using the compilers in the toolchain, importing that data, and producing/updating an IndexStoreDB. The core infrastructure is also designed to be shared by the SourceKitLSP project. All of this is built for tests written in Swift. This change can be broken down into some inter-dependent high-level components: 1. Tibs (Test Index Build System) 2. ISDBTestSupport 3. Improvements to IndexStoreDB to make it testable 4. Tests --- Tibs ---- Tibs, or the "Test Index Build System" provides a simple and flexible build system for test projects. A test case can use a fixture from the `Tests/INPUTS` directory containing a `project.json` describing any targets of the test. Targets may be Swift modules containing any number of source files, C translations units, or a mix of both, including mixed language targets (subject to the capabilities of the platform - Linux cannot use Objective-C interop, for example). The test can then programatically build index raw data, make modifications to source files, and incrementally rebuild. Tibs is implemented using `ninja`, which introduces a new dependency in IndexStoreDB *only when running tests*. ISDBTestSupport --------------- This module contains a number of useful types and functions for working with test fixtures. In particular, * `TestSources` provides a one-stop shop for working with the source files inside a test project, including modifying them and scanning them for named `TestLocation`s, which can then be used to locate positions within source code during a test. * `TibsTestWorkspace` provides a convenient way to work with a tibs test project, its `TestSources`, and the resulting `IndexStoreDB.. Mutable projects are created with sources and build products copied into a temporary location (and cleared on deinit), and can be edited and rebuilt during the test. An immutable (static) test workspace can use the sources directly from the INPUTS directory, and its build products are stored in the build directory for IndexStoreDB itself, allowing faster incremental builds (or null builds) during development. Improvements to IndexStoreDB ---------------------------- The biggest change here is that the `Symbol` and `SymbolOccurrence` types are now simple struct types rather than wrapping the underlying C API types. This makes it much more convenient to work with them during testing, used along with the new `checkOccurrences` function. There is also a new API to `pollForUnitChangesAndWait`, intended only for testing. This replaces watching for file system changes during testing by polling and comparing to the last known state of the unit directory. Watching for file system changes is unnecessarily asynchronous during testing, and this lets us side step any performance issues with that, as well as determining when an index update is "done". Tests ----- To prove the usefulness of the new testing APIs, several new tests for indexing functionality, were added. These are the first real tests for the indexing code. There is lots of room to improve testing coverage further.
@swift-ci please test |
akyrtzi
approved these changes
Jul 22, 2019
Probably need an update to our build preset in the swift repo. |
@swift-ci please test |
@swift-ci please test |
@swift-ci please test |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The goal of this change is to make it convenient to add tests for the
index that test the whole process of producing the raw index data using
the compilers in the toolchain, importing that data, and
producing/updating an IndexStoreDB. The core infrastructure is also
designed to be shared by the SourceKitLSP project. All of this is built
for tests written in Swift.
This change can be broken down into some inter-dependent high-level
components:
Tibs
Tibs, or the "Test Index Build System" provides a simple and flexible
build system for test projects. A test case can use a fixture from the
Tests/INPUTS
directory containing aproject.json
describing anytargets of the test. Targets may be Swift modules containing any number
of source files, C translations units, or a mix of both, including mixed
language targets (subject to the capabilities of the platform - Linux
cannot use Objective-C interop, for example). The test can then
programatically build index raw data, make modifications to source
files, and incrementally rebuild.
Tibs is implemented using
ninja
, which introduces a new dependency inIndexStoreDB only when running tests.
ISDBTestSupport
This module contains a number of useful types and functions for working
with test fixtures. In particular,
TestSources
provides a one-stop shop for working with the source filesinside a test project, including modifying them and scanning them for
named
TestLocation
s, which can then be used to locate positions withinsource code during a test.
TibsTestWorkspace
provides a convenient way to work with a tibs testproject, its
TestSources
, and the resulting `IndexStoreDB.. Mutableprojects are created with sources and build products copied into a
temporary location (and cleared on deinit), and can be edited and
rebuilt during the test. An immutable (static) test workspace can use
the sources directly from the INPUTS directory, and its build products
are stored in the build directory for IndexStoreDB itself, allowing
faster incremental builds (or null builds) during development.
Improvements to IndexStoreDB
The biggest change here is that the
Symbol
andSymbolOccurrence
types are now simple struct types rather than wrapping the underlying C
API types. This makes it much more convenient to work with them during
testing, used along with the new
checkOccurrences
function.There is also a new API to
pollForUnitChangesAndWait
, intended onlyfor testing. This replaces watching for file system changes during
testing by polling and comparing to the last known state of the unit
directory. Watching for file system changes is unnecessarily
asynchronous during testing, and this lets us side step any performance
issues with that, as well as determining when an index update is "done".
Tests
To prove the usefulness of the new testing APIs, several new tests for
indexing functionality, were added. These are the first real tests for
the indexing code. There is lots of room to improve testing coverage
further.