Skip to content

Numerical matrix comparison with a specified precision gives the wrong result #502

@lambrtz

Description

@lambrtz

The code below checks if the matrices matrix and identity are equal within a specified tolerance. They are supposed to be equal, but the comparison returns false.

Here is an MWE:

using AngouriMath; using AngouriMath.Extensions; using static AngouriMath.MathS; using static AngouriMath.Entity; using System; using static System.Console;
using PeterO.Numbers;

Entity.Matrix matrix = "[[1.000000000000000547458310833800074927650525249394612805610000, 0, 0, -9.55753574799863785565952768061466479249665075036612773690000E-15], [0, 1.000000000000000547458310833800074927650525249394612805610000, 0, 2.24075209004037969624007863746621061783995000E-15], [0, 0, 1, 0], [0, 0, 0, 1]]";
var identity = MathS.IdentityMatrix(4);
using var _ = MathS.Settings.PrecisionErrorCommon.Set(0.1m);
WriteLine(matrix == identity);

Output:

False

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugA serious math/runtime error

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions