Ensure that test preferences are properly stacked #3061
Conversation
staticfloat
force-pushed
the
sf/preferences_stacking
branch
from
8b2685f to
e4b476c
Compare
|
I think this may require JuliaPackaging/Preferences.jl#37 |
staticfloat
force-pushed
the
sf/preferences_stacking
branch
4 times, most recently
from
94a57d0 to
c1d7564
Compare
When calling `Pkg.test("Foo")`, some information from the calling
environment is incorporated into the tests of `Foo`; for example, if the
calling environment places restrictions on the package versions of
`Bar`, a dependency of `Foo`, those restrictions will be honored when
resolving the dependencies of the test environment for `Foo`.
Similarly, we should honor preferences set within the calling
environment. The easiest way to do this is to layer the test
environment onto the end of the current `LOAD_PATH` instead of wholesale
replacing it, as we were doing when copying preferences over for the
test environment.
With this change it is now viable to test alternate configurations of a
package without dev'ing it out and modifying it directly. If a package
`Foo` uses preference to alter its behavior, previously you would need
to `dev` it out and insert an override into its `LocalPreferneces.toml`
(or `test/LocalPreferences.toml`, if it had a separate test project) in
order for the preference to be read during test time. Now, one can
simply set the preference in an environment that has `Foo` added, then
run `Pkg.test("Foo")`, and as long as the package itself doesn't define
the preference within its own testing environment, it will pick up the
default from the environment.
We allow the package to override values coming in from the environment,
as there may be rather sensitive preferences that it is best to not
allow easy overriding from. For these cases, there is of course still
the `dev`-and-edit method available for tests.
staticfloat
force-pushed
the
sf/preferences_stacking
branch
from
c1d7564 to
5e5189b
Compare
|
@vchuravy is it possible for you to test this with MPIPreferences to ensure that it's actually sufficient for your usecase? |
|
Will try to give it a go today! |
|
Doesn't seem to work for me: Edit: meh, I don't think this is using the right Edit 2: I guess it's because of https://github.com/JuliaParallel/MPI.jl/blob/e4479cfe16f1fdbbdab7423eca03c98764b0c671/test/Project.toml#L6. I'll edit this line and try again. Edit 3: either changing the UUID in the test project file wasn't enough, or I need to rebuild julia against this branch to make sure this is used everywhere (can't do it right now), but it's still not working for me. |
So that looks right to me! I built Julia with diff --git a/stdlib/Pkg.version b/stdlib/Pkg.version
index a595ed0213..c1f14c1982 100644
--- a/stdlib/Pkg.version
+++ b/stdlib/Pkg.version
@@ -1,4 +1,4 @@
PKG_BRANCH = master
-PKG_SHA1 = 7676431149332ae400a805632082b12263c20269
+PKG_SHA1 = 5e5189b84523a41c67e3ade44fe774e69b66cea5
PKG_GIT_URL := https://github.com/JuliaLang/Pkg.jl.git
PKG_TAR_URL = https://api.github.com/repos/JuliaLang/Pkg.jl/tarball/$1 |
|
Looks good Valentin! With that, I'm going to merge! |
|
@staticfloat , @vchuravy : how about having in the loadpath a default location where the default preferences are looked up - analogue to |
This can be done by having users define a default |
My suggestion is not to abuse |
|
@vchuravy , @staticfloat, what do you think about the above comment? |
|
Please add it as a comment to #3067, I don't have any strong opinions, but we need to build community consensus around it. |
5 participants
When calling
Pkg.test("Foo"), some information from the callingenvironment is incorporated into the tests of
Foo; for example, if thecalling environment places restrictions on the package versions of
Bar, a dependency ofFoo, those restrictions will be honored whenresolving the dependencies of the test environment for
Foo.Similarly, we should honor preferences set within the calling
environment. The easiest way to do this is to layer the test
environment onto the end of the current
LOAD_PATHinstead of wholesalereplacing it, as we were doing when copying preferences over for the
test environment.
With this change it is now viable to test alternate configurations of a
package without dev'ing it out and modifying it directly. If a package
Foouses preference to alter its behavior, previously you would needto
devit out and insert an override into itsLocalPreferneces.toml(or
test/LocalPreferences.toml, if it had a separate test project) inorder for the preference to be read during test time. Now, one can
simply set the preference in an environment that has
Fooadded, thenrun
Pkg.test("Foo"), and as long as the package itself doesn't definethe preference within its own testing environment, it will pick up the
default from the environment.
We allow the package to override values coming in from the environment,
as there may be rather sensitive preferences that it is best to not
allow easy overriding from. For these cases, there is of course still
the
dev-and-edit method available for tests.