Skip to content

[Java] JdbcFieldInfo is insufficient for proper type inference #35916

@lidavidm

Description

@lidavidm

Describe the enhancement requested

The APIs only let us infer the type based on the data type, precision, and scale, effectively. But in the wild, there are always exceptions; for example, the JDBC driver for PostgreSQL maps both TIMESTAMP WITHOUT TIME ZONE and TIMESTAMP WITH TIME ZONE to just Types.TIMESTAMP, so arrow-jdbc dutifully maps them both to naïve timestamps (wrong!).

  • JdbcFieldInfo should expose all the available information it gets.
  • For convenience, it should also work with DatabaseMetaData#getColumns.
  • We need a way to override how JdbcConsumers are constructed. (ArrowVectorIterator hardcodes calls to JdbcToArrowUtils#getConsumer.)
  • We should integration test with popular databases (PostgreSQL, MySQL, SQL Server, SQLite, DuckDB, others?)

Component(s)

Java

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions