fix: prevent BiCGStab producing NaN on exact/near-exact intermediate convergence - #256
Open
jpbrodrick89 wants to merge 2 commits into
Open
jpbrodrick89 wants to merge 2 commits into
jpbrodrick89 wants to merge 2 commits into
Conversation
`omega_new = <s, t> / <t, t>` was an unguarded 0/0 whenever the half-step
residual `s` was already (numerically) converged -- e.g. whenever `r0` lands
in an invariant subspace of the preconditioned operator. The resulting `nan`
poisoned `y` and `r`, and evaded `breakdown_occurred` entirely, since
`nan == 0.0` is `False`. A generic non-finite-output check elsewhere in
lineax caught the corrupted result, but only by failing outright, on
problems that are perfectly well-posed (and which
`scipy.sparse.linalg.bicgstab` / `jax.scipy.sparse.linalg.bicgstab` solve
correctly, via the same guard added here).
Repro (previously nan, now solves exactly):
A = jnp.diag(jnp.array([2.0, -3.0, 5.0, 7.0]))
b = jnp.array([1.0, 0.0, 0.0, 0.0]) # exact eigenvector of A
lx.linear_solve(A_op, b, solver=lx.BiCGStab(rtol=1e-8, atol=1e-8))
…=0 forcing - Rename r_already_converged -> is_converged (it's used on s too) and t_dot_t -> t2. - Remove the forced diff=0/r_new=r overrides that were added to guarantee quiescence detection: verified (500+ randomized trials, the exact eigenvector edge case, autodiff) that the guarded natural computation already yields a sufficiently small diff on its own, so the override was an unnecessary approximation that could have slightly reduced accuracy in the has_scale case. - Trim the explanatory comments down to the essential reasoning.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
BiCGStab.compute'sbody_funcomputesomega_new = <s, t> / <t, t>, wheresis the half-step residual andt = A(M(s)). This leads to a divide by zero error whenevers(and hencet) is already (numerically) the zero vector and happens wheneverr0lands in an invariant subspace of the preconditioned operator (e.g.bis exactly an eigenvector ofA). Consequently, BiCGStab in effect solves the system in fewer steps than the recurrence "expects".This is caught by
RESULTS.nonfinite_inputmeaning we fail on problems that are perfectly well-posed.Industry standard
scipy.sparse.linalg.bicgstabguards exactly this case (if norm(s) < atol: x += alpha*phat; return x, 0), andjax.scipy.sparse.linalg.bicgstabcopies the same guard (exit_early). A second, related cascade one iteration later (alpha_new = rho_new / <r0, v_new>, both tiny onceris already converged) is also guarded here for the same reason.MWE
Fix
Mirror scipy and jax using
jnp.where:omega_new's division when the half-step residualsis already within the solver's own convergence tolerance (r_already_converged), reportingomega_new = 1(not0, so as not to spuriously tripbreakdown_occurred, which treatsomega == 0as true stagnation).alpha_new's division the same way when the incoming residualris already converged and force a genuine no-op step (diff = 0,runchanged) sonot_convergedcorrectly detects quiescence on the next loop check.