Merged
Conversation
jecsand838
force-pushed
the
avro-uuid-duration-file
branch
from
c1e3042 to
ba0ea5f
Compare
alamb
pushed a commit
to apache/arrow-rs
that referenced
this pull request
Jul 14, 2025
…oding (#7889) # Which issue does this PR close? - Part of #4886 - Related to #6965 # Rationale for this change The `arrow-avro` crate currently lacks support for the Avro `duration` type, which is a standard and commonly used type in Avro schemas. This omission prevents users from reading Avro files containing duration types, limiting the crate's utility. This change introduces support for decoding Avro duration types by mapping them to the Arrow `Interval` type. This is a logical and efficient representation. Implementing this feature brings the `arrow-avro` crate closer to full Avro specification compliance and makes it more robust for real-world use cases. # What changes are included in this PR? This PR contains: * arrow-avro decoder support for Duration types. * Minor fixes UUID decoding. UUID types now map to `utf8` type to better align with the [Avro specification](https://avro.apache.org/docs/1.11.1/specification/#uuid) * New integration test using a temporary `duration_uuid.avro` file crate using this python script: https://gist.github.com/jecsand838/cbdaaf581af78f357778bf87d2f3cf15 # Are these changes tested? Yes, this PR includes for integration and unit tests covering these modifications. # Are there any user-facing changes? N/A # Follow-Up PRs 1. PR to update `test_duration_uuid` once apache/arrow-testing#108 is merged in. --------- Co-authored-by: Matthijs Brobbel <m1brobbel@gmail.com>
Contributor
|
Can you please resolve the conflicts here? |
Contributor
|
Thanks @jecsand838 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds a new
duration_uuid.avrofile which will be used to test arrow-avro uuid and duration logical types.Here's the python script used to generate the new avro file: https://gist.github.com/jecsand838/cbdaaf581af78f357778bf87d2f3cf15
Part of apache/arrow-rs#4886