[Bug] Random Failures

[matekelemen]

I've been running the CI a lot lately, and noticed some random failures that occur relatively rarely, in different applications. It's possible that these failures aren't connected, but they're so similar (no exception or signal, just an untimely termination with a nonzero exit code) that makes me guess they all come from the same place.

• is it a problem with the CI (e.g.: out of memory, faulty hardware, etc)?
• is it a bug in some major app (e.g.: memory corruption or race condition in Structural/Fluid)?
• is it a bug in Core?

Here's a list of failures I ran into. Please extend this list if you find one too.

• FSIApp, PR [External] Update VexCL #13325, log.zip
• DEMApp, PR [Core] Add PMultigridBuilderAndSolver #13276, issue [DEM] Crash During Tests in CI #13289
• MPMApp, PR Updating med app doc #12848, PR [Core] Initialize MasterSlaveConstraints in Scheme #13274, issue [MPM] Memory error or race condition #12851
• FSIApp, PR [Core] Extend PMultigridBuilderAndSolver #13645, logs in comments here (ci_fsi_failure.zip)

[loumalouomega]

Huuum

[matekelemen]

Am I the only one running into these crashes or is everyone just ignoring them and rerunning the CI? Am I going insane?

[RiccardoRossi]

Puff...

@KratosMultiphysics/technical-committee this is for us