Port test in select.rs to sqllogic

[aprimadi]

Which issue does this PR close?

More work toward #4495

Rationale for this change

Cause I like easy task

What changes are included in this PR?

Just porting some tests to sqllogic

Are these changes tested?

Yes

Are there any user-facing changes?

No

[aprimadi]

Tests that cannot be ported (yet):

• query_get_indexed_field - unsupported SQL types ARRAY
• query_nested_get_indexed_field - unsupported SQL types ARRAY
• query_nested_get_indexed_field_on_struct - unsupported SQL types ARRAY
• query_on_string_dictionary - using DictionaryArray
• sort_on_window_null_string - using DictionaryArray
• test_prepare_statement - using partitioned csv
• parallel_query_with_filter - using partitioned csv
• boolean_literal - using partitioned csv
• unprojected_filter - this test is using dataframe interface

[alamb]

Thank you so much @aprimadi -- this looks great. I reviewed all the changes carefully

Cause I like easy task

😆 -- thank you -- this is really helpful overall and I think makes it easier, over the long term, to keep up with DataFusion and improve its test coverage

You are inspiring me to try and paralleize this work more -- I'll try and write up some tickets that can be split up for others who want to get involved with DataFusion with a useful but low overhead tasks (and it will be nice to keep track of how much we have to go)

[alamb]

👍 for testing the whole plan

[alamb]

❤️

[aprimadi]

No worries @alamb. This keeps me engaged while I'm learning through the more difficult bits of the project.

[alamb]

No worries @alamb. This keeps me engaged while I'm learning through the more difficult bits of the project.

It is much appreciated. While you have a newcomer's eyes, I would be interested in the parts you found the most difficult to pick up (really I am interested in how to make it less difficult).

[aprimadi]

The external sort come to mind and also the k-way sort preserving merge.

K-way sort preserving merge because there is a clever bit of the code that uses an array representation of a binary tree (to implement losers tree).

Like for example this code:
https://github.com/apache/arrow-datafusion/blob/main/datafusion/core/src/physical_plan/sorts/merge.rs#L252
https://github.com/apache/arrow-datafusion/blob/main/datafusion/core/src/physical_plan/sorts/merge.rs#L271
These really find the leaf node of the loser tree that this cursor index belongs to but it's not immediately apparent without realizing that the binary tree is represented this way:

   0
   1 
 2   3
4 5 6 7

i.e. the root of the tree is an element at index 1 in the vector and the first child of the root is an element at index 2 in the vector and so on...

I guess I'm just hesitant to work on larger tasks due to:

1. Whether someone else is currently working on it
2. Whether it's high priority. Don't want to block the issue for a prolonged period.
3. If the issue touch areas that I'm currently not very familiar with. Again for the same reason as 2, don't want to block the issue for a prolonged period.

Currently I prefer to work on issue that I can finish on one weekend and yeah currently #4495 is my default go to issue if I don't find issues that are interesting or that I'm sure I can finish on one weekend 😆.

[alamb]

K-way sort preserving merge because there is a clever bit of the code that uses an array representation of a binary tree (to implement losers tree).

Yes, I agree this is quite a nice piece of code that is non trivial. I think the parts of DataFusion that are the most performance critical are also the parts that are the hardest to understand. It would be great to improve the understandability.

@aprimadi I wonder if you might be willing to take a shot at adding some comments (perhaps copy/paste/modify your comments in #6158 (comment)) to help other future readers

Also, I will try and be better about triaging tickets as they come in and marking the relevant ones as "good first issues" -- https://github.com/apache/arrow-datafusion/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22

It is a little lacking now