Siginificant-digits-to-precision transition made invalid by rounding #124
Open
Description
If we sprintf(buffer,"%#.3g", 99.99);
, we get 100.0
- with 4 significant digits - while we should be getting 100.
, with 3 significant digits.
This is because, initially, we find the power of 10 to be 2; then we translate significant digits to precision, setting the precision to 1. But after this happens, we perform rounding with precision 1, so that the power of 10 becomes 3...
Metadata
Assignees
Labels
No labels