Backport array null handling #285
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* function: Allow more expressive array signatures This commit allows for more expressive array function signatures. Previously, `ArrayFunctionSignature` was an enum of potential argument combinations and orders. For many array functions, none of the `ArrayFunctionSignature` variants worked, so they used `TypeSignature::VariadicAny` instead. This commit will allow those functions to use more descriptive signatures which will prevent them from having to perform manual type checking in the function implementation. As an example, this commit also updates the signature of the `array_replace` family of functions to use a new expressive signature, which removes a panic that existed previously. There are still a couple of limitations with this approach. First of all, there's no way to describe a function that has multiple different arrays of different type or dimension. Additionally, there isn't support for functions with map arrays and recursive arrays that have more than one argument. Works towards resolving apache#14451 * Add mutability * Move mutability enum * fmt * Fix doctest * Add validation to array args * Remove mutability and update return types * fmt * Fix clippy * Fix imports * Add list coercion flag * Some formatting fixes * Some formatting fixes * Remove ArrayFunctionArguments struct * Simplify helper functions * Update array_and_element behavior
…#14737) * feat: arbitrary typed argument in array function * fix: array_sort null handling * fix: array_resize signature * test: add array_sort sqllogictest for null and invalid types * fix: don't match error message * chore: use string instead of data type * refactor: use new_null_array * fix: pass null to array argument should return null * fix: handle null argument for array in replace and resize * fix: mismatched error message * fix: incorrect number of rows returned * test: update null tests * fix: treat NULLs as lists directly to prevent extra handling * fix: incorrect null pushing in array_sort
* Coerce Array inner types * `coerce_list_children` macro → fn * Rearrange & simplify patterns * Add casting for `FixedSizeList` * Add sql logic test * Fix test * Expand error * Switch to `Arc::clone` * OR nullable-s * Add successful test * Switch to `type_union_resolution` * Add a note on `coerce_list_children` * Handle different list types inside `type_union_resolution_coercion` * Add a `make_array` test with different array types * Add the test value * Update datafusion/sqllogictest/test_files/array.slt Co-authored-by: Jay Zhan <jayzhan211@gmail.com> * Reorder match --------- Co-authored-by: Jay Zhan <jayzhan211@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
https://coralogix.atlassian.net/browse/FASTV2-2489
Rationale for this change
See apache#15149
What changes are included in this PR?
Are there any user-facing changes?