Dependency solver quickcheck tests #3245
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I started adding quickcheck tests for the solver. There is currently only one test, which checks that parameters like
--reorder-goals
do not affect whether the solver finds an install plan.Here is an example failure: https://gist.github.com/grayjay/86eea907cd08878855ae
I created a new test suite because the test is currently too slow to run as a unit test. The time to find one of the known bugs on
master
ranges from under a second to over a minute on my computer, with a median of about 6 seconds (10,000 tests). We may want to run it even longer after those bugs are fixed. Another problem is that the test occasionally runs out of memory. I haven't tried to debug it yet.There are two other issues. I'm not sure how we should print failing test cases in a human-readable format. The PR currently uses the
pretty-show
package to parse and reformat the results from the derivedShow
instance, but it requireshappy
. I'm also not sure how to organize the test suite. I put the new test in theUnitTests
directory since it uses the existingDSL
module. Another option is to have three directories undertests
, one each for unit tests, solver quickcheck tests, and shared modules.