Skip to content

Incorrect parenthesization in fp error bounds computation #181

Open
@mmp

Description

(Via Kostya Smolenskiy)

pages 228 and 229. In earlier section, You had mentioned that since the compiler can't rearrange floating point computation, it executes it like it's written in code. Code for transforming points is written like, for example:

 T xp = m.m[0][0] * x + m.m[0][1] * y + m.m[0][2] * z + m.m[0][3];

But in derivation of error bounds on that computation parentheses are as if it is:

 T xp = (m.m[0][0] * x + m.m[0][1] * y) + (m.m[0][2] * z + m.m[0][3]);

From which it stems that maximum gamma is gamma(3), but with former code, maximum gamma is gamma(4).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions