Filtering expressions

[bgb-ipl]

Hi,

I have a concern using some filtering expressions with bcftools view (version 1.9). Let say I have 5 individuals and one is missing : GT 0/0 ./. 0/1 0/0 0/0.

I want to filter based on missingness >=20% so the the example above should be filtered.

When I use the F_MISSING :
bcftools view -e "F_MISSING>=0.2" test.vcf : the site is not filtered
bcftools view -e "F_MISSING=>1/5" test.vcf : the site is filtered

It is the same using F_PASS :
bcftools view -e 'F_PASS(GT=="mis")>=0.2' test.vcf : the site is not filtered
bcftools view -e 'F_PASS(GT=="mis")>=1/5' test.vcf : the site is filtered

I wonder why 0.2 and 1/5 have not the same behaviour.

Best,

[pd3]

This is an artifact of floating point arithmetic. Most nice looking fractional numbers, such as 0.2, contain a small error (~1e-7) when represented on a computer. Here the program is internally working with doubles but was parsing the fractional numbers using single precision floats, thus seeing something like 1.999999 after parsing 0.2 and a more accurate value after parsing 1/5.

Why not to use the double precision always? The float parsing was used because the VCF format uses floats, not doubles, and comparisons such as QUAL=59.2 were failing because of this when parsed as double.

The commits eded7db and 44192ea change the parsing and arithmetics so that it's all done with doubles, and only for comparisons they are casted into floats. This should extend the range of expected results.

If anyone sees a better solution, please chime in.

[pd3]

Also this solution brings problems. Numeric comparisons in filtering expressions use internally the same code for both integer and floating point arithmetics. This is a problem when comparing genomic coordinates, the result become imprecise because single-precision floats can represent integers precisely only up to 16777216. The commit 0ab6642 improves this by switching to double-precision when big positive numbers are being compared.