Skip to content

fix: prevent BiCGStab producing NaN on exact/near-exact intermediate convergence - #256

Open
jpbrodrick89 wants to merge 2 commits into
patrick-kidger:devfrom
jpbrodrick89:fix/bicgstab-omega-breakdown
Open

jpbrodrick89 wants to merge 2 commits into
patrick-kidger:devfrom
jpbrodrick89:fix/bicgstab-omega-breakdown

Conversation

@jpbrodrick89

Copy link
Copy Markdown
Collaborator

The bug

BiCGStab.compute's body_fun computes omega_new = <s, t> / <t, t>, where s is the half-step residual and t = A(M(s)). This leads to a divide by zero error whenever s (and hence t) is already (numerically) the zero vector and happens whenever r0 lands in an invariant subspace of the preconditioned operator (e.g. b is exactly an eigenvector of A). Consequently, BiCGStab in effect solves the system in fewer steps than the recurrence "expects".

This is caught by RESULTS.nonfinite_input meaning we fail on problems that are perfectly well-posed.

Industry standard

scipy.sparse.linalg.bicgstab guards exactly this case (if norm(s) < atol: x += alpha*phat; return x, 0), and jax.scipy.sparse.linalg.bicgstab copies the same guard (exit_early). A second, related cascade one iteration later (alpha_new = rho_new / <r0, v_new>, both tiny once r is already converged) is also guarded here for the same reason.

MWE

A = jnp.diag(jnp.array([2.0, -3.0, 5.0, 7.0]))
b = jnp.array([1.0, 0.0, 0.0, 0.0])  # exact eigenvector of A, eigenvalue 2.0
lx.linear_solve(lx.MatrixLinearOperator(A), b, solver=lx.BiCGStab(rtol=1e-8, atol=1e-8))
# before: value=[nan, nan, nan, nan], result=nonfinite output
# after:  value=[0.5, 0., 0., 0.], result=successful

Fix

Mirror scipy and jax using jnp.where:

  • Guard omega_new's division when the half-step residual s is already within the solver's own convergence tolerance (r_already_converged), reporting omega_new = 1 (not 0, so as not to spuriously trip breakdown_occurred, which treats omega == 0 as true stagnation).
  • Guard alpha_new's division the same way when the incoming residual r is already converged and force a genuine no-op step (diff = 0, r unchanged) so not_converged correctly detects quiescence on the next loop check.

claude added 2 commits August 12, 2026 00:10
`omega_new = <s, t> / <t, t>` was an unguarded 0/0 whenever the half-step
residual `s` was already (numerically) converged -- e.g. whenever `r0` lands
in an invariant subspace of the preconditioned operator. The resulting `nan`
poisoned `y` and `r`, and evaded `breakdown_occurred` entirely, since
`nan == 0.0` is `False`. A generic non-finite-output check elsewhere in
lineax caught the corrupted result, but only by failing outright, on
problems that are perfectly well-posed (and which
`scipy.sparse.linalg.bicgstab` / `jax.scipy.sparse.linalg.bicgstab` solve
correctly, via the same guard added here).

Repro (previously nan, now solves exactly):

    A = jnp.diag(jnp.array([2.0, -3.0, 5.0, 7.0]))
    b = jnp.array([1.0, 0.0, 0.0, 0.0])  # exact eigenvector of A
    lx.linear_solve(A_op, b, solver=lx.BiCGStab(rtol=1e-8, atol=1e-8))
…=0 forcing

- Rename r_already_converged -> is_converged (it's used on s too) and
  t_dot_t -> t2.
- Remove the forced diff=0/r_new=r overrides that were added to guarantee
  quiescence detection: verified (500+ randomized trials, the exact
  eigenvector edge case, autodiff) that the guarded natural computation
  already yields a sufficiently small diff on its own, so the override was
  an unnecessary approximation that could have slightly reduced accuracy in
  the has_scale case.
- Trim the explanatory comments down to the essential reasoning.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants