Skip to content

Siginificant-digits-to-precision transition made invalid by rounding #124

Open
@eyalroz

Description

If we sprintf(buffer,"%#.3g", 99.99);, we get 100.0 - with 4 significant digits - while we should be getting 100., with 3 significant digits.

This is because, initially, we find the power of 10 to be 2; then we translate significant digits to precision, setting the precision to 1. But after this happens, we perform rounding with precision 1, so that the power of 10 becomes 3...

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions