[Enh]: support `Map` datatype

[danielgafni]

We would like to learn about your use case. For example, if this feature is needed to adopt Narwhals in an open source project, could you please enter the link to it below?

I would really like to to natively handle Map columns in my Narwhals code in Metaxy.

Right now I had to go through a few hops [1, 2] to get it working for Polars, PyArrow and Ibis backed frames.

Please describe the purpose of the new feature or describe the problem to solve.

Currently, Narwhals does not support the Arrow Map datatype, and for a good reason: neither does Polars.

However, all the other tools built around Arrow do support it: Ibis, DuckDB, ClickHouse, and storage formats such as Parquet, DeltaLake, and Apache Iceberg.

Therefore, this is a core datatype in the Arrow ecosystem, and we should support it as well - at least in some (experimental) form.

Suggest a solution if possible.

Initially we can make a separate narwhals-map plugin and document it as experimental.

For Polars we can either use https://github.com/hafaio/polars-map, or fail at runtime (I would prefer the former).

For all the other datatypes we can use their native Map support.

If you have tried alternatives, please describe them below.

No response

Additional information that may help us understand your needs.

cc @MarcoGorelli @FBruzzesi

[MarcoGorelli]

thanks for the issue

Initially we can make a separate narwhals-map plugin and document it as experimental.

yup, on board with this (not yet sure how the details would work, but open to exploring it)

[danielgafni]

Alright, I can start working on it in my own repo? Or will you create it?

[MarcoGorelli]

sure, feel free to do so! we can then move it over to the narwhals org if/when ready

[danielgafni]

I have a draft at https://github.com/danielgafni/narwhals-map and it even works in my Metaxy project!

Very minimal for now, supports Arrow, Ibis, Polars (via polars-map, fyi @erikbrinkman) and only one operation: .get :)

@MarcoGorelli can we add some minimal hooks to Narwhals in order to expose the .map namespace and Map type?

[MarcoGorelli]

thanks!

@MarcoGorelli can we add some minimal hooks to Narwhals in order to expose the .map namespace and Map type?

discussed today, and yup, open to it! the Map type could even be fully upstreamed