-
Notifications
You must be signed in to change notification settings - Fork 378
refactor(arrow2): Migrate utf8.right to use arrow-rs instead of arrow2 #5889
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor(arrow2): Migrate utf8.right to use arrow-rs instead of arrow2 #5889
Conversation
Greptile SummaryThis PR successfully migrates the Key changes:
The implementation correctly handles null values through the existing Confidence Score: 5/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant Caller
participant Right Function
participant parse_inputs
participant nchars Array
participant Arrow-rs
participant Result Builder
Caller->>Right Function: right_impl(arr, nchars)
Right Function->>parse_inputs: Check array lengths & nulls
parse_inputs-->>Right Function: (is_full_null, expected_size)
alt is_full_null
Right Function-->>Caller: Return full_null array
else expected_size == 0
Right Function-->>Caller: Return empty array
else Normal processing
Right Function->>nchars Array: to_arrow()
nchars Array-->>Right Function: Arrow array
Right Function->>Arrow-rs: cast to Int64Type
Arrow-rs-->>Right Function: Int64 primitive array
alt nchars.len() == 1
Right Function->>Arrow-rs: Get single value (double unwrap)
Arrow-rs-->>Right Function: Single i64 value
Right Function->>Result Builder: Collect with fixed nchar
else nchars.len() > 1
Right Function->>Result Builder: Zip and collect with variable nchar
end
Result Builder-->>Right Function: LargeStringArray
Right Function->>Right Function: from_arrow conversion
Right Function-->>Caller: Utf8Array result
end
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #5889 +/- ##
==========================================
- Coverage 72.48% 72.47% -0.01%
==========================================
Files 966 966
Lines 125838 125830 -8
==========================================
- Hits 91211 91194 -17
- Misses 34627 34636 +9
🚀 New features to boost your workflow:
|
universalmind303
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm! Thanks @huleilei.
|
Thanks. Currently, the unit test cases are in use, and there is already a PR #5878 being worked on to fix the issue. Once this PR is merged, I will proceed with a rebase and conduct CI testing. |
969993d to
6845af4
Compare
|
@universalmind303 I've made the revisions. Please review them when you have time. Thank you |
This PR migrates the implementation of
utf8.rightinsrc/daft-functions-utf8/src/right.rsfromarrow2toarrow-rs.Changes Made
#![allow(deprecated, reason = "arrow2 migration")]attribute.arrow2array usage witharrow-rsequivalents (e.g.,LargeStringArray,Int64Array).to_arrow()instead of the deprecatedas_arrow2().Related Issues
Part of the migration effort tracked in #5741.
Verification
cargo test -p daft-functions-utf8locally and all tests passed.