Skip to content

Automate testing / ensuring that string functions get the same answer for String, LargeString, StringView, DictionaryString, etc  #12415

Closed
@alamb

Description

@alamb

Is your feature request related to a problem or challenge?

@2010YOUY01 found a bug where substr behaved differently for StringView than for String: #12383

As we continue to implement special versions of the string functions for different representations this drift may get worse

Describe the solution you'd like

I would like a systematic and SQL level set of tests that ensure the string functions compute the same answer for all different combinations.

Describe alternatives you've considered

I think we could do this in sqllogictest by separating the table definitions into different .slt files and then includeing the same queries file. Here is an example:

https://github.com/apache/datafusion/blob/02eab80cd62e02fcb68dee8b99d63aaac680a66c/datafusion/sqllogictest/test_files/join_disable_repartition_joins.slt#L26-L25

So this would look something like

in strings_stringview.slt

strings_stringview.slt defines the table with appropriate type. Note there would be similar files for string and largestring

statement ok
-- Populate the "t" table table with the "standard" values in StringView form
CREATE TABLE temp AS VALUES
 ('one', 'two'),
  ('three, 'four)

statement ok
CREATE TABLE t as 
SELECT 
  arrow_cast(column1, 'Utf8View') as column1,
  arrow_cast(column2, 'Utf8View') as column2,
FROM temp

-- run queries from string_queries.slt
include string_queries.slt

in strings_queries.slt

-- Note this file is intended to be run with a table 't' already defined 
-- with standard data with the appropriate data type (StringView, String, etc)

-- Test concat
query
SELECT concat(column1, column2) from t;

...

Additional context

No response

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions