NonlinearLeastSquaresProblem returns stalled

[hexaeder]

Describe the bug 🐞

If it does not find a solution, a NonlinearLeastSquars problem returns Stalled, thus unsuccessful.

Expected behavior

Return successful retcode. On slack Oscar suggested to maybe create a new stalled_success retcode for that case.

Minimal Reproducible Example 👇

using NonlinearSolve
f(u,p) = [1.0]
nlf = NonlinearFunction(f; resid_prototype=zeros(1))
prob = NonlinearLeastSquaresProblem(nlf, [1.0])
sol = solve(prob)

sol.retcode == ReturnCode.Stalled
SciMLBase.successful_retcode(sol.retcode) == false

Environment (please complete the following information):

• Output of using Pkg; Pkg.status()

(jl_46LzEB) pkg> st
Status `/tmp/jl_46LzEB/Project.toml`
  [8913a72c] NonlinearSolve v3.14.0

• Output of using Pkg; Pkg.status(; mode = PKGMODE_MANIFEST)

(jl_46LzEB) pkg> st -m
Status `/tmp/jl_46LzEB/Manifest.toml`
  [47edcb42] ADTypes v1.8.1
  [7d9f7c33] Accessors v0.1.38
  [79e6a3ab] Adapt v4.0.4
  [ec485272] ArnoldiMethod v0.4.0
  [4fba245c] ArrayInterface v7.16.0
  [4c555306] ArrayLayouts v1.10.3
  [62783981] BitTwiddlingConvenienceFunctions v0.1.6
  [2a0fbf3d] CPUSummary v0.2.6
  [d360d2e6] ChainRulesCore v1.25.0
  [fb6a15b2] CloseOpenIntervals v0.1.13
  [38540f10] CommonSolve v0.2.4
  [bbf7d656] CommonSubexpressions v0.3.1
  [f70d9fcc] CommonWorldInvalidations v1.0.0
  [34da2185] Compat v4.16.0
  [a33af91c] CompositionsBase v0.1.2
  [2569d6c7] ConcreteStructs v0.2.3
  [187b0558] ConstructionBase v1.5.8
  [adafc99b] CpuId v0.3.1
  [9a962f9c] DataAPI v1.16.0
  [864edb3b] DataStructures v0.18.20
  [e2d170a0] DataValueInterfaces v1.0.0
  [2b5f629d] DiffEqBase v6.155.1
  [163ba53b] DiffResults v1.1.0
  [b552c78f] DiffRules v1.15.1
  [a0c0ee7d] DifferentiationInterface v0.5.17
  [ffbed154] DocStringExtensions v0.9.3
  [4e289a0a] EnumX v1.0.4
⌅ [f151be2c] EnzymeCore v0.7.8
  [e2ba6199] ExprTools v0.1.10
⌅ [6b7a57c9] Expronicon v0.8.5
  [7034ab61] FastBroadcast v0.3.5
  [9aa1b823] FastClosures v0.3.2
  [29a986be] FastLapackInterface v2.0.4
  [1a297f60] FillArrays v1.13.0
  [6a86dc24] FiniteDiff v2.24.0
  [f6369f11] ForwardDiff v0.10.36
  [069b7b12] FunctionWrappers v1.1.3
  [77dc65aa] FunctionWrappersWrappers v0.1.3
  [46192b85] GPUArraysCore v0.1.6
  [86223c79] Graphs v1.11.2
  [3e5b6fbb] HostCPUFeatures v0.1.17
  [615f187c] IfElse v0.1.1
  [d25df0c9] Inflate v0.1.5
⌃ [3587e190] InverseFunctions v0.1.16
  [92d709cd] IrrationalConstants v0.2.2
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.6.0
  [ef3ab10e] KLU v0.6.0
  [ba0b0d4f] Krylov v0.9.6
  [10f19ff3] LayoutPointers v0.1.17
  [5078a376] LazyArrays v2.2.1
  [d3d80556] LineSearches v7.3.0
  [7ed4a6bd] LinearSolve v2.34.0
  [2ab3a3ac] LogExpFunctions v0.3.28
  [bdcacae8] LoopVectorization v0.12.171
  [d8e11817] MLStyle v0.4.17
  [1914dd2f] MacroTools v0.5.13
  [d125e4d3] ManualMemory v0.1.8
  [bb5d69b7] MaybeInplace v0.1.4
  [46d2c3a1] MuladdMacro v0.2.4
  [d41bc354] NLSolversBase v7.8.3
  [77ba4419] NaNMath v1.0.2
  [8913a72c] NonlinearSolve v3.14.0
  [6fe1bfb0] OffsetArrays v1.14.1
  [bac558e1] OrderedCollections v1.6.3
  [65ce6f38] PackageExtensionCompat v1.0.2
  [d96e819e] Parameters v0.12.3
  [f517fe37] Polyester v0.7.16
  [1d0040c9] PolyesterWeave v0.2.2
  [d236fae5] PreallocationTools v0.4.24
  [aea7be01] PrecompileTools v1.2.1
  [21216c6a] Preferences v1.4.3
  [3cdcf5f2] RecipesBase v1.3.4
  [731186ca] RecursiveArrayTools v3.27.0
  [f2c3362d] RecursiveFactorization v0.2.23
  [189a3867] Reexport v1.2.2
  [ae029012] Requires v1.3.0
  [7e49a35a] RuntimeGeneratedFunctions v0.5.13
  [94e857df] SIMDTypes v0.1.0
  [476501e8] SLEEFPirates v0.6.43
⌃ [0bca4576] SciMLBase v2.53.1
  [c0aeaf25] SciMLOperators v0.3.10
  [53ae85a6] SciMLStructures v1.5.0
  [efcf1570] Setfield v1.1.1
  [727e6d20] SimpleNonlinearSolve v1.12.1
  [699a6c99] SimpleTraits v0.9.4
  [47a9eef4] SparseDiffTools v2.20.0
  [0a514795] SparseMatrixColorings v0.4.0
  [e56a9233] Sparspak v0.3.9
  [276daf66] SpecialFunctions v2.4.0
  [aedffcd0] Static v1.1.1
  [0d7ed370] StaticArrayInterface v1.8.0
  [90137ffa] StaticArrays v1.9.7
  [1e83bf80] StaticArraysCore v1.4.3
  [10745b16] Statistics v1.11.1
  [7792a7ef] StrideArraysCore v0.5.7
  [2efcf032] SymbolicIndexingInterface v0.3.30
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.12.0
  [8290d209] ThreadingUtilities v0.5.2
  [a759f4b9] TimerOutputs v0.5.24
  [d5829a12] TriangularSolve v0.2.1
  [410a4b4d] Tricks v0.1.9
  [781d530d] TruncatedStacktraces v1.4.0
  [3a884ed6] UnPack v1.0.2
  [3d5dd08c] VectorizationBase v0.21.70
  [19fa3120] VertexSafeGraphs v0.2.0
  [1d5cc7b8] IntelOpenMP_jll v2024.2.1+0
  [856f044c] MKL_jll v2024.2.0+0
  [efe28fd5] OpenSpecFun_jll v0.5.5+0
  [1317d2d5] oneTBB_jll v2021.12.0+0
  [0dad84c5] ArgTools v1.1.2
  [56f22d72] Artifacts v1.11.0
  [2a0f44e3] Base64 v1.11.0
  [ade2ca70] Dates v1.11.0
  [8ba89e20] Distributed v1.11.0
  [f43a241f] Downloads v1.6.0
  [7b1f6079] FileWatching v1.11.0
  [9fa8497b] Future v1.11.0
  [b77e0a4c] InteractiveUtils v1.11.0
  [4af54fe1] LazyArtifacts v1.11.0
  [b27032c2] LibCURL v0.6.4
  [76f85450] LibGit2 v1.11.0
  [8f399da3] Libdl v1.11.0
  [37e2e46d] LinearAlgebra v1.11.0
  [56ddb016] Logging v1.11.0
  [d6f4376e] Markdown v1.11.0
  [a63ad114] Mmap v1.11.0
  [ca575930] NetworkOptions v1.2.0
  [44cfe95a] Pkg v1.11.0
  [de0858da] Printf v1.11.0
  [9a3f8284] Random v1.11.0
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization v1.11.0
  [1a1011a3] SharedArrays v1.11.0
  [6462fe0b] Sockets v1.11.0
  [2f01184e] SparseArrays v1.11.0
  [fa267f1f] TOML v1.0.3
  [a4e569a6] Tar v1.10.0
  [8dfed614] Test v1.11.0
  [cf7118a7] UUIDs v1.11.0
  [4ec0a83e] Unicode v1.11.0
  [e66e0078] CompilerSupportLibraries_jll v1.1.1+0
  [deac9b47] LibCURL_jll v8.6.0+0
  [e37daf67] LibGit2_jll v1.7.2+0
  [29816b5a] LibSSH2_jll v1.11.0+1
  [c8ffd9c3] MbedTLS_jll v2.28.6+0
  [14a3606d] MozillaCACerts_jll v2023.12.12
  [4536629a] OpenBLAS_jll v0.3.27+1
  [05823500] OpenLibm_jll v0.8.1+2
  [bea87d4a] SuiteSparse_jll v7.7.0+0
  [83775a58] Zlib_jll v1.2.13+1
  [8e850b90] libblastrampoline_jll v5.11.0+0
  [8e850ede] nghttp2_jll v1.59.0+0
  [3f19e933] p7zip_jll v17.4.0+2

[avik-pal]

We can take the return code from the termination condition and convert it to success in this case

[hexaeder]

I have no idea if that would be a lot of hassle, but I quite like @oscardssmith's suggestion to introduce stalled_success. For over-constraint initialization problems, it is nice to know whether the solver stalled or actually found a root directly from the retcode.

[Ickaser]

Bump just to say that I spent a good chunk of time today trying to understand why a least-squares problem was "stalled", when I think it has simply found the minimum I am looking for.

I don't mind seeing the retcode "stalled", but what is a problem for my case was that since "stalled" is considered a failure it doesn't return the best value it found, instead giving back the starting u0. (Which seems to be in contrast to the documented behavior of the default termination conditions).

The same may also be true of a MaxIters return code in some cases, where (following the trace) it clearly finds a local minimum and I would like to simply have that local minimum out, instead it just gives back the original guess.

[Ickaser]

Since my work involves models with noticeable, expected mismatch between model and data, right now this issue prevents me from making use of these algorithms the way I would like to, so instead I have to turn to an optimization solver (which is what I was using beforehand anyway, until I got excited about this).

I would submit a PR, except that I couldn't figure out how to fix this: it looks like at present the termination conditions don't know about the problem type, so there is no way to check whether it's a NonlinearLeastSquaresProblem. Perhaps for least squares problems there should be a separate set of termination conditions, anyway, since a least squares problem may often have a persistent offset from 0 in f(u).

[ChrisRackauckas]

I guess yes the issue is that for NonlinearProblem you expect to find a zero or error. For NonlinearLeastSquaresProblem, there's no reason to believe that there should be a loss values within tolerance, since the result is a loss not a residual to zero. So the key is that it should exit when the derivative is sufficiently small. So this stalled and successfully exited. "Stalled" should mean "the gradient stopped going lower", success is "the gradient went to zero". So on NonlinearLeastSquaresProblem, the convergence criteria should be on whether the gradient does things, rather than the loss.