better handling of runtest errors? #37809
Conversation
vtjnash
force-pushed
the
jn/runtest-errors
branch
from
97eac81
to
cc93f62
Compare
| @@ -214,6 +227,25 @@ function Base.show(io::IO, t::Broken) | |||
| end | |||
| end | |||
|
|
|||
| # Types that appear in TestSetException.errors_and_fails we convert eagerly into strings | |||
| # other types we convert lazily | |||
| function Serialization.serialize(s::Serialization.AbstractSerializer, t::Pass) | |||
There was a problem hiding this comment.
Is there any downside to stringifying everything eagerly so we don't need to add serialize methods?
There was a problem hiding this comment.
(I was sure I replied already—twice—but somehow my comments are gone?)
There's a possible performance concern (we almost never need these strings, but almost always need the other ones). More importantly though, there's a number of consumers that introspect the result of Pass to do further tests on the result.
There was a problem hiding this comment.
Hm, it's also possible that everything should be preserved as lazy until serialize (refs #24847)
vtjnash
force-pushed
the
jn/runtest-errors
branch
2 times, most recently
from
cf775ac
to
2889b88
Compare
By serializing both tests and backtraces (as strings), and trying to ensure other parts are not going to mess them up along the way. Hopefully this should improve error message reporting reliability in our Base runtests.
vtjnash
force-pushed
the
jn/runtest-errors
branch
from
2889b88
to
ee43966
Compare
|
Stringifying all fields in test results breaks some of my code. |
XUnit calls Distributed.myid through Test (i.e. it calls Test.myid). Distributed.myid is not re-exported from Test, and ergo is not part of Test's public API and cannot be relied upon. Case in point, Test's dependency on Distributed was removed in JuliaLang/julia#37809, and so Test.myid fails under Julia 1.6.
Looking at #37789, trying to interpret how to avoid these problems in the future