Enhance TabPFNRegressor to handle constant target values

[anuragg1209]

Hi @noahho,

Fix #246.

New Tests:

• Added a new test test_constant_target to verify that the TabPFNRegressor correctly predicts a constant value when the target y is constant.

[LeoGrin]

Thanks, LGTM!

logits=torch.full((X.shape[0], 1), self.constant_value_) I think the logits shouldn't be equal to self.constant_value_, right? I'm not sure what they should be equal to though, perhaps all ones or something else uniform? (I guess self.constant_value_ could also work then, but might be confusing).

[anuragg1209]

Thanks, LGTM!

logits=torch.full((X.shape[0], 1), self.constant_value_) I think the logits shouldn't be equal to self.constant_value_, right? I'm not sure what they should be equal to though, perhaps all ones or something else uniform? (I guess self.constant_value_ could also work then, but might be confusing).

Thanks for the review @LeoGrin!
I have made a few changes. Now, the logits are set to 0, and we also have a constant_criterion for constant targets. This aligns better with the code semantics.

[LeoGrin]

Thanks @anuragg1209!
If we change the criterion like that (and I agree it's better), it might be simpler to just change self.renormalized_criterion_ in fit, to the constant criterion you've defined here. This should directly make the output mean, mode and median constant. The quantile wouldn't be exactly the same that what you have here, not sure if it's ok or if we want to hardcode it. WDYT?

[anuragg1209]

Thanks @anuragg1209! If we change the criterion like that (and I agree it's better), it might be simpler to just change self.renormalized_criterion_ in fit, to the constant criterion you've defined here. This should directly make the output mean, mode and median constant. The quantile wouldn't be exactly the same that what you have here, not sure if it's ok or if we want to hardcode it. WDYT?

Hi @LeoGrin, I like the idea of changing the self.renormalized_criterion_ in fit, to the constant criterion. It makes more sense now. Also, the mean, mode, median and quantiles all remains constant so we have everything working as intended.

[LeoGrin]

Thanks @anuragg1209! Do we still need to set the constant mean, mode etc manually now? I would have guessed that it would automatically work with the new criterion defined in fit (except for the quantiles, which would be a bit different, which is maybe fine). But I'm not sure.

[anuragg1209]

Thanks @anuragg1209! Do we still need to set the constant mean, mode etc manually now? I would have guessed that it would automatically work with the new criterion defined in fit (except for the quantiles, which would be a bit different, which is maybe fine). But I'm not sure.

The reason we are setting the mean, mode, etc. manually is because if we use a criterion-based approach, even with tight borders, numerical operations introduce tiny floating point differences from the constant value. The current approach of setting manually also works fine as we are only dealing with the edge case of constant y, but if we want to completely remove setting mean, mode, etc, manually, then we have to introduce a tolerance-based test.

[LeoGrin]

Thanks @anuragg1209! In this case maybe it actually makes more sense to put most things in predict for clarity, sorry for the change. Otherwise LGTM but I wondering if there is an issue if you call fit on a constant dataset, and then again on a new dataset. Wouldn't is_constant_target stay True for the 2nd dataset?