-
Notifications
You must be signed in to change notification settings - Fork 20
Open
Description
The code for determining the default denominator stumbles on the following requirement: https://github.com/38/d4-format/blob/master/d4tools/src/create/main.rs#L244
The threshold of 1e-10 is too strict when dealing with high coverage (e.g. when dealing with MT coverage in WGS).
For example, a coverage value of "8284.95" ends up with a denominator of 10000 (instead of 100).
Furthermore, really large values of the denominator can also happen (example: "33262.09"), which breaks the coverage calculations completely (where value * denominator > i32::MAX).
A simple fix is to loosen the criteria of 1e-10 to a lower value. Another fix is to ensure that value * denominator < i32::MAX.
(Working on a PR)
Metadata
Metadata
Assignees
Labels
No labels