Add Variant, Geometry, Geography types to implementation status page

[alamb]

• Closes Update Implementation status to include Geometry/Geography types #118

Rationale

Three new logical types have been added to the Parquet specification. Let's ensure they are listed in the implementation status matrix

Changes:

1. Add entries for Variant, Geometry, Geography types
2. Add links to relevant parts of the spec

Here is a preview:

[alamb]

this table has three new rows for Variant, Geometry, Geography, and I moved FLOAT16 up with the rest of the numeric types, to be consistent with the list in https://github.com/apache/parquet-format/blob/master/LogicalTypes.md

[zeroshade]

arrow-go implemented variant so you can add the check mark there for it 😄

[alamb]

Done in 1d23cc2 (along with arrow-rs)

[alamb]

Thank you for the review @severo 🙏

[wgtmac]

Perhaps @paleolimbot @zeroshade can provide some inputs for certain impls?

[severo]

cc @platypii for hyparquet too

[platypii]

hyparquet js:

• does not (yet) support VARIANT ❌
• does support GEOMETRY and GEOGRAPHY ✅ (edit from: (R))

[platypii]

Update: As of today, Hyparquet now fully supports reading AND writing GEOMETRY and GEOGRAPHY types. ✅

I was peer-pressured by the parquet implementation status page 😆

[alamb]

I was peer-pressured by the parquet implementation status page 😆

The plan is working!

(@julienledem and I plan to do something similar for some of the other query engines that have a reputation for not being able to read newer files)

[alamb]

hyparquet js:

• does not (yet) support VARIANT ❌
• does support GEOMETRY and GEOGRAPHY ✅ (edit from: (R))

Updated in a4ec40d

[alamb]

In arrow-rs you currently have to try pretty hard to actually get the types to be written or read but it's tested and can be done.

I hope we can make this better in the upcoming releases

Edit:

• tracking in [geometry] Wire up arrow reader/writer for GEOMETRY and GEOGRAPHY arrow-rs#8717

[alamb]

Here is what the current matrix looks like

[alamb]

I also added a check mark for variant in arrow-java based on
https://github.com/apache/parquet-java/tree/master/parquet-variant/src/main/java/org/apache/parquet/variant

I also looked arounda and think arrow-go is still working on geometry:

• https://github.com/search?q=repo%3Aapache%2Farrow-go+geometry
• https://github.com/search?q=repo%3Aapache%2Farrow-go+geography

Likewise CUDF does not seem to support geo:

• https://github.com/search?q=repo%3Arapidsai%2Fcudf+geography
• https://github.com/search?q=repo%3Arapidsai%2Fcudf+geometry

Or variant

• https://github.com/search?q=repo%3Arapidsai%2Fcudf+variant

[alamb]

@neilechao and @aihuaxu I am working to characterize the support for Variant across readers. The arrow version says:

https://github.com/apache/arrow/blob/57f626189326251c8144d4fad89c28220c68ca75/cpp/src/parquet/arrow/variant_internal.h#L33

EXPERIMENTAL: Variant is not yet fully supported.

However, looking around at the open issues in arrow https://github.com/apache/arrow/issues?q=is%3Aissue%20%20variant

It wasn't clear to me what the current status is (should arrow have a green ✅ or an ❌ or something else?)

Some relevant PRs;

• [C++][Parquet] Variant specification version should be set 1 to align with the variant spec rather than 0 arrow#47838
• GH-45937: [C++][Parquet] Variant logical type definition arrow#45375
• GH-47838: [C++][Parquet] Set Variant specification version to 1 to align with the variant spec arrow#47835

[alamb]

I think this PR is an improvement over the current version on the website so will merge it in.

If there are additional checks that were missed or need to be added in the future, let's make those changes as a follow on PR

[alamb]

Thanks again @severo