Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] LowessRegressor: ZeroDivisionError: Weights sum to zero, can't be normalized #707

Open
srt19170 opened this issue Oct 29, 2024 · 4 comments · May be fixed by #708
Open

[BUG] LowessRegressor: ZeroDivisionError: Weights sum to zero, can't be normalized #707

srt19170 opened this issue Oct 29, 2024 · 4 comments · May be fixed by #708
Labels
bug Something isn't working

Comments

@srt19170
Copy link

Using LowessRegressor, I received a "ZeroDivisionError: Weights sum to zero, can't be normalized" error. A little debugging revealed that the problem is in _calc_wts:

    def _calc_wts(self, x_i):
        """Calculate the weights for a single point `x_i` using the training data `self.X_` and the parameters
        `self.sigma` and `self.span`. The weights are calculated as `np.exp(-(distances**2) / self.sigma)`,
        where distances are the distances between `x_i` and all the training samples.

        If `self.span` is not None, then the weights are multiplied by
        `(distances <= np.quantile(distances, q=self.span))`.
        """
        distances = np.linalg.norm(self.X_ - x_i, axis=1)
        weights = np.exp(-(distances**2) / self.sigma)
        if self.span:
            weights = weights * (distances <= np.quantile(distances, q=self.span))
        return weights

If sigma is not sized properly, the scaled distances will be large and cause np.exp() to return zeros for the weights, resulting in an exception being raised when the weights are applied.

This seems like the wrong behavior. Perhaps the weights should be clipped to some minimum non-zero weight so that the predictor can still be used, perhaps with a warning message to explain the problem?

@srt19170 srt19170 added the bug Something isn't working label Oct 29, 2024
@FBruzzesi
Copy link
Collaborator

Hey @srt19170 , thanks for spotting and raising this issue.

This seems like the wrong behavior.

Not sure I would call it wrong 😇

perhaps with a warning message to explain the problem?

Surely a better error message can be provided.

Perhaps the weights should be clipped to some minimum non-zero weight so that the predictor can still be used

My understanding is that this issue would occur only if all the -(distances**2) / self.sigma values blow up to minus infinity, thus all the weights are zero. Clipping all elements to any constant value would lead to compute the average of y in

results = np.stack([np.average(self.y_, weights=self._calc_wts(x_i=x_i)) for x_i in X])

I am not too comfortable with doing that implicitly for the user. I would still tend to raise an error that explains the issue

@FBruzzesi
Copy link
Collaborator

RFC on the following error message:

try:
    results = np.stack([np.average(self.y_, weights=self._calc_wts(x_i=x_i)) for x_i in X])
except ZeroDivisionError:
    msg = (
        "Weights, resulting from `np.exp(-(distances**2) / self.sigma)`, are all zero. "
        "Try to increase the value of `sigma` or to normalize the input data.\n\n"
        "`distances` refer to the distance between each sample `x_i` with all the"
        "training samples."
    )
    raise ValueError(msg)

@koaning
Copy link
Owner

koaning commented Oct 29, 2024

+1 on the better error message.

@srt19170 is there anything you can share about your dataset that might help us understand the kinds of datasets for which this issue comes up?

@srt19170
Copy link
Author

My data has 75 features, and I'm using StandardScaler to condition the data. In the initial test set of data there are 2905 elements, and it looks like the distances end up being in the range 8-35. My first attempt at this I just copied the documentation and used sigma=0.1, which in this case is right at the edge of where it turns all the weights to zero.

I admit my math is weak -- is the norm being calculated more likely to yield large numbers when the number of features is large?

At any rate, thanks for considering the issue and the proposed error message would have helped me figure it out more quickly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants