Skip to content

Runaway default denominator #105

@oyvinev

Description

@oyvinev

The code for determining the default denominator stumbles on the following requirement: https://github.com/38/d4-format/blob/master/d4tools/src/create/main.rs#L244

The threshold of 1e-10 is too strict when dealing with high coverage (e.g. when dealing with MT coverage in WGS).

For example, a coverage value of "8284.95" ends up with a denominator of 10000 (instead of 100).

Furthermore, really large values of the denominator can also happen (example: "33262.09"), which breaks the coverage calculations completely (where value * denominator > i32::MAX).

See https://play.rust-lang.org/?version=stable&mode=debug&edition=2015&gist=b69a0f4bfaffddead14fb68c088157c3

A simple fix is to loosen the criteria of 1e-10 to a lower value. Another fix is to ensure that value * denominator < i32::MAX.

(Working on a PR)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions