Skip to content

Commit e12f6a0

Browse files
committed
Fix bin1d_vec to properly account for floating point precision
The tolerance for bin size `h` must be based on eps and the _involved_ numbers in calculating it, not the resulting number. The two involved numbers are of the same order of magnitude, and one if it is `a0` (the first bin edge `bin[0] == min(bins)`), for which we already calculate the tolerance (`a0_tol`). So we can use `a0_tol` also as tolerance for `h`.
1 parent 2feb914 commit e12f6a0

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

csep/utils/calc.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,7 @@ def bin1d_vec(p, bins, tol=None, right_continuous=False):
8080
h = bins[1] - bins[0]
8181

8282
a0_tol = numpy.abs(a0) * numpy.finfo(numpy.float64).eps
83-
h_tol = numpy.abs(h) * numpy.finfo(numpy.float64).eps
83+
h_tol = a0_tol # must be based on *involved* numbers
8484
p_tol = numpy.abs(p) * numpy.finfo(numpy.float64).eps
8585

8686
# absolute tolerance

0 commit comments

Comments
 (0)