[ML] Bad values for the variance scale

[tveasey]

A user data set has shown up two issues with the variance scale calculation in version 6.2.2 of the analytics:

1. it is sometimes negative(!),
2. it is sometimes infinite.

In particular, we are seeing the following error messages logged:
Error calculating joint distribution: Bad variance scale -5.75
Error calculating joint distribution: Bad variance scale inf

There is no prospect of getting hold of the data set; however the data characteristics sound benign. There were two detectors:

• detector high_mean(x) over y influencers y,z.
• detector high_median(x) over y influencers y,z

For x we have min: 0, max: 4.34571, avg: 2.0736 and cardinality of y is 430.

This issue is to investigate routes by which this problem could occur. The initial areas to investigate are CTimeSeriesDecomposition::scale and the calculation of the count variance scale, particularly for influencers.

cc @LucaWintergerst.

[LucaWintergerst]

The exceptions do not happen if influencer z is removed.
I also replaced z (which was the hostname) with z_ip and the same thing happened

[hendrikmuhs]

Update:

The root cause has been identified, counting influencer occurrences per bucket has a bug. The fix is simple (1LOC). The change affects results:

Hopefully to the better, I will analyze the diff to be sure.

I followed the code history back to version 5.5, so it's not a recent regression but likely has been introduced by PR 144 in the old repo.

As said, the fix is simple, but I plan to take some more time for related code improvements and test cases.