Skip to content

DataType parsing no longer works correctly for old formatted timestamps #8539

@alamb

Description

@alamb

Describe the bug
DataFusion uses the DateType::try_from impl to parse Strings to DataTypes

For example, we have a function like this:

select arrow_cast('2021-01-01T00:00:00', 'Timestamp(Nanosecond, Some("-05:00"))'

That parses the string "'Timestamp(Nanosecond, Some("-05:00")" into the correct DataType::Timestamp

When testing with the latest (unreleased version) of arrow-rs, I found these query fails now with an error similar to

statement error DataFusion error: Execution error: Unsupported type 'Timestamp\(Nanosecond, None\)'\. Must be a supported arrow type name such as 'Int32' or 'Timestamp\(ns\)'\. Error expected double quoted string for Timezone, got 'None'

I found this while testing upgrading DataFusion to arrow (what will be 57) in this PR:

To Reproduce
I am pretty sure this was introduced as part of this PR

Expected behavior
The old types should be supported

Additional context

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions