Skip to content

Add DataType::Utf8View and DataType::BinaryView #5468

@alamb

Description

@alamb

This is part of the larger project to implement StringViewArray -- see #5374

The first thing we will need is a new variant in DataType to support this type

So the basic task is to

  1. Add the appropriate variants in DataType
  2. Update the rest of arrow-rs to handle that new variant (largely would be an exercise in returning NotYetImplemented errors)

For inspiration I think you can look at #4585 (specifically types.rs https://github.com/apache/arrow-rs/pull/4585/files#diff-ff91e9fd06b025009cc1d0f9360ecdb8c3d9ea972e8f87b4419eab01e1e8fb7c)

I reviewed the Arrow spec that was approved and there does not appear to be any equivalent to LargeUtf8 (e.g there is no LargeUtf8View):

https://github.com/apache/arrow/blob/c4e088a1d6227868e020c71d596970f35bb9e4c9/format/Schema.fbs#L187-L205

Metadata

Metadata

Assignees

Labels

arrowChanges to the arrow crateparquetChanges to the parquet crate

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions