expose null_on_oob on gather.

[ritchie46]

Description

Similar to other get and list.gather, but then on Expr/Series.

Ref #15240

[cmdlineluser]

Will this issue also cover Expr.get - or does it need to be opened separately?

There have been a couple of questions in the past few days ¹² with people trying to use .list.get() as a workaround and running into confusion.

Footnotes

1. https://stackoverflow.com/questions/78878210/polars-get-nth-position-from-expression-type ↩

2. https://stackoverflow.com/questions/78868961/how-to-get-the-value-of-a-specified-index-number-from-the-sorting-of-a-column-an ↩

[ddouglas87]

Request: Have expr.get() default to null_on_oob=True. Most real world use cases would want null_on_oob=True. Always having to type that in becomes tedious. I know this breaks backwards compatibility but in this case it probably will not cause issues in preexisting code bases.

[deanm0000]

It doesn't just break backwards compatibility it goes against the fail early, fail loudly principle. It would also be inconsistent with cast strict=True default.

[ddouglas87]

Using group_by_dynamic at the end of the rolling aggregate there is always a list size of 1 so .get() will always fail, so one would have to type in null_on_oob=True for all use cases to get it to work. It will not not break preexisting code bases because you have to do null_on_oob=True or it doesn't work at all.

Right now because null_on_oob=True doesn't exist the only way to get an nth element in a rolling aggregate is to do .shift(nth).last(). Shift allows nulls by default.

An alternative solution is to allow rolling aggregates to stop ahead of time instead of unwinding at the end. When I use group_by_dynamic I always chop off the ending. imo the root cause here isn't a lack of null_on_oob=True but that rolling aggregates have no option of stopping early.

[dalejung]

I think the oob-ing on non-null indexes makes sense. The issue is with contexts like aggregations where you can get nulls.

Example:
#15163

I don't think raising OOB there makes sense.

But that might be discussion for another ticket. Prio should be to at least get an opt out with Expr.get(null_on_oob=False).

[cmdlineluser]

Found a couple of other older issues which can probably be closed in favour of this:

• Request: nth_value aggregator function #13340
• polars.Expr.take returns null if ComputeError: index out of bounds #8171