Add a sklearn compatible `Pipeline` class and `make_pipeline` function

[Vincent-Maladiere]

Some methods of SurvivalBoost like predict_cumulative_incidence or predict_survival_function aren't registered in sklearn Pipeline, yet we need these to run predictions.

[GaelVaroquaux]

I wouldn't try to not move too fast on that issue: any design of pipeline is a wormhole of coupled challenges (that's also one reason why I try to design predictors to stick as much as possible to the fit/transform/predict API). Maybe we'll see where the thinking in skrub goes and if we can factor in using other methods.

[Vincent-Maladiere]

Let's have it for the exp branch at the very least because otherwise, we can't even use SEER.

[GaelVaroquaux]

Let's have it for the exp branch at the very least because otherwise, we can't even use SEER.

Have you rather considered using a custom scorer?

[Vincent-Maladiere]

Not really, could you expand this idea a little bit?

It's just that subclassing sklearn Pipeline for the two missing methods seemed to be the most straightforward and minimalistic path

[GaelVaroquaux]

Not really, could you expand this idea a little bit?

The reason that we need a pipeline is for model evaluation: usage in cross-validation, in gridsearch.

It's just that subclassing sklearn Pipeline for the two missing methods seemed to be the most straightforward and minimalistic path

Short-term straightfoward, long-term unsustainable: every time someone encounters a difficulty: new method. Ecosystem ends up with many methods and incompatible packages.

[Vincent-Maladiere]

I mean, people use pipelines for deployment too. This is more versatile than scoring IMHO. Some people have put hazardous in production by extending the sklearn pipeline.

[Vincent-Maladiere]

I'll be happy not to introduce a new feature, as long as we have an alternative. Registering methods to sklearn Pipeline could be a simple way to address this, but this is not doable AFAIK

[GaelVaroquaux]

I mean, people use pipelines for deployment too. This is more versatile than scoring IMHO. Some people have put hazardous in production by extending the sklearn pipeline.

OK, but once again, we can't address every issue by a new method and a new pipeline. It's a bigger problem than hazardous, and I would prefer not seeing a hazardous pipeline, but rather to work on broader solutions. The few people who need to do deployment with hazardous can write a bit of code calling the specific methods.

[Vincent-Maladiere]

Ok, should we close this issue then?

[GaelVaroquaux]

Ok, should we close this issue then?

IMHO yes, or reframe it to talk about the problem, not the solution

[Vincent-Maladiere]

Okay, let's reframe it later