feat: Add support for Timestamp data type #33
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@abandy You may want to take a look at this too. |
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for the Timestamp data type in Swift Arrow by introducing new types, builders, and helper functions for formatting and serialization/deserialization. Key changes include:
- Implementing TimestampArray, ArrowTypeTimestamp, and associated formatting utilities.
- Extending builder and reader/writer helpers to recognize and handle Timestamp.
- Adding comprehensive tests for Timestamp functionality in both CData and Array test suites.
Reviewed Changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| Arrow/Tests/ArrowTests/CDataTests.swift | Added tests for Timestamp fields with different units and timezone configurations. |
| Arrow/Tests/ArrowTests/ArrayTests.swift | Introduced testTimestampArray covering various Timestamp units and formatting options. |
| Arrow/Sources/Arrow/ProtoUtil.swift | Added Timestamp case handling for protobuffer conversion. |
| Arrow/Sources/Arrow/ArrowWriterHelper.swift | Extended writer helper to include Timestamp serialization logic. |
| Arrow/Sources/Arrow/ArrowType.swift | Introduced Timestamp alias, ArrowTypeTimestamp, and updated type-related functions. |
| Arrow/Sources/Arrow/ArrowReaderHelper.swift | Added Timestamp support in array holder creation and type discovery. |
| Arrow/Sources/Arrow/ArrowArrayBuilder.swift | Added TimestampArrayBuilder and integrated it into the builder registry. |
| Arrow/Sources/Arrow/ArrowArray.swift | Added TimestampArray implementation with formatting utilities for date conversion. |
Comments suppressed due to low confidence (1)
Arrow/Tests/ArrowTests/CDataTests.swift:47
- [nitpick] Consider using more descriptive field names for Timestamp columns (e.g., 'colTimestampNanoseconds' instead of 'colTimestamptsn') to improve clarity in tests.
.addField("colTimestamptsn", type: ArrowTypeTimestamp(.nanoseconds), isNullable: false)
Arrow/Sources/Arrow/ArrowArray.swift
Outdated
Comment on lines
266
to
274
| let formatter = DateFormatter() | ||
| formatter.dateFormat = options.dateFormat | ||
| formatter.locale = options.locale | ||
|
|
||
| if options.includeTimezone, let timezone = timestampType.timezone { | ||
| formatter.timeZone = TimeZone(identifier: timezone) | ||
| } | ||
|
|
||
| return formatter.string(from: date) |
There was a problem hiding this comment.
Creating a new DateFormatter for each call of formattedDate(at:) may impact performance in tight loops; consider caching a formatter instance if possible.
Suggested change
| let formatter = DateFormatter() | |
| formatter.dateFormat = options.dateFormat | |
| formatter.locale = options.locale | |
| if options.includeTimezone, let timezone = timestampType.timezone { | |
| formatter.timeZone = TimeZone(identifier: timezone) | |
| } | |
| return formatter.string(from: date) | |
| if cachedFormatter == nil || cachedOptions != options { | |
| let formatter = DateFormatter() | |
| formatter.dateFormat = options.dateFormat | |
| formatter.locale = options.locale | |
| if options.includeTimezone, let timezone = timestampType.timezone { | |
| formatter.timeZone = TimeZone(identifier: timezone) | |
| } | |
| cachedFormatter = formatter | |
| cachedOptions = options | |
| } | |
| return cachedFormatter?.string(from: date) |
|
Could you fix lint error? You can use |
|
FYI: You can check CI results on your fork by enabling GitHub Actions on your fork. |
|
I've fixed linters by using the |
kou
reviewed
Jun 14, 2025
kou
changed the title
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rationale for this change
Currently, the Swift implementation of Arrow does not support Timestamp, although they are available in the base C interface. This PR attempts to add its support by following the current implemented design pattern.
What changes are included in this PR?
TimestampArraywith some basic formatting utilitiesTimestampArrayBuilderTimestampaliasArrowTimestampUnit, which includes extensively all the variants (seconds, milliseconds, microseconds and nanoseconds)ArrowTypeTimestampfrom baseArrowArrowTypesupport for timestampArrowWriterHelpersupport for timestampfromProtosupport for timestampIt properly handles the presence or absence of
timezone.Are these changes tested?
Tests are included in both
ArrayTests.swiftandCDataTests.swift.Are there any user-facing changes?
Yes - users can now work with Timestamp data types in Swift Arrow implementations. This is additive and doesn't break existing functionality.
Closes #32.