[DISCUSS] Remove panics

[hntd187]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

I would like to remove (or at least provide a non-panicking) interface for arrow-rs to have. I spoke with @alamb while at Databricks Data and AI Conference and he mentioned making an issue before I got deep into the weeds implementing something.

Describe the solution you'd like
There are a couple ways to go about this.

1. Remove the panics. This would be multiple API breaking changes across the board, so this is not nice from a compatibility standpoint. Ideally, I don't think this is the right way to go here.
2. Add try_* methods instead. For example:

pub fn as_generic_list_array<S: OffsetSizeTrait>(arr: &dyn Array) -> &GenericListArray<S> {
    arr.as_any()
        .downcast_ref::<GenericListArray<S>>()
        .expect("Unable to downcast to list array")
}

Becomes

pub fn try_as_generic_list_array<S: OffsetSizeTrait>(arr: &dyn Array) -> Option<&GenericListArray<S>> {
    arr.as_any().downcast_ref::<GenericListArray<S>>()
}

pub fn as_generic_list_array<S: OffsetSizeTrait>(arr: &dyn Array) -> &GenericListArray<S> {
    try_as_generic_list_array(arr).expect("Unable to downcast to list array")
}

This mechanism is employed in multiple places already, if this approach is preferred this would unify that approach to be consistent across all of the code base.

I'm basically looking for a preference to get started upon. I can introduce new Error types as necessary for result types, but I think #2 for options is a good place to start.

[tustvold]

Infallible methods already exist for casting, provided on https://docs.rs/arrow-array/latest/arrow_array/cast/trait.AsArray.html.

With regards to panicking interfaces, see https://github.com/apache/arrow-rs/tree/main?tab=readme-ov-file#guidelines-for-panic-vs-result and by extension #6737.

TLDR no issue with adding infallible methods where they don't exist, but converting unreachable panics (outside of a bug) to unreachable errors IMO just adds bloat and verbosity (and is also marginally slower) for no real gain.

[alamb]

I agree two different APIs for most operations make sense:

1. one that returns Result and does not panic
2. one that does not return Result and does panic

And then the caller can decide if they want to deal with very rare/bug only errors or just panic if they happen.

pub fn as_generic_list_array<S: OffsetSizeTrait>(arr: &dyn Array) -> &GenericListArray<S> {

This is actually a great example where there are already 2 APIs with the different interfaces:

• AsArray::as_list_opt (fallable, doesn't panic)
• AdArray::as_list (infallible, does panic)

(as an aside we should probably deprecate as_generic_list_array and refer people to the AsArray trait 🤔 )

We are having a similar conversation about API design on #7711 with @scovich and @friendlymatthew

1. "easy to use but panic's on bug" interface
2. "returns errors, but is more annoying to use"

[hntd187]

I would agree, removing unreachable with unreachable errors would be a waste.

[hntd187]

So I let this sit for 2 weeks, there doesn't seem to be more conversation. If helpful @alamb I can just go ahead with fallible, no panic, APIs where they do not exist to start.

[alamb]

Maybe we can start with some small / easy to review PRs and see what they look like