Too tight tolerance in one of the tests?

[KristofferC]

In the PkgEval run for the 1.2 release branch Convex.jl errored with

max atom: Test Failed at /root/.julia/packages/Convex/HwTSz/test/test_lp.jl:61
  Expression: ≈(p.optval, max(max_a, max_b), atol=10TOL)
   Evaluated: 0.9898189279146101 ≈ 1.0 (atol=0.01) 

Running the tests locally, they pass so I think the tolerance is perhaps a bit too tight?

[ericphanson]

Looking at the git history, the tolerance for this test was already pushed up by 10x in #239. From the numbers in #295 (comment), something like atol=0.011 might be enough here (whereas it's currently atol=0.01) so maybe it doesn't need that much more, but the tolerance is already pretty high so maybe there's some other underlying problem.

So one option might be to revert the tolerance to the standard atol=1e-3, set it to @test_broken and open an issue for it. (I guess @test_broken might not work if it passes in some solvers but not others, in which case we could just comment out the test). Or just bump the tolerance up again.

[odow]

Closing as stale. There are going to be tolerance issues with the tests, particularly when underlying solver versions change. The default tolerances are already quite loose. We've also fixed a lot of the discrepancies in binary distributions by using JLLs.