Skip to content

Partitioning by day no longer works for DATE column #1162

Open
@bnaul

Description

@bnaul

It appears my fix from #1074 was undone by #1113. In 1.12.0 the following works:

import datetime
import sqlalchemy

import sqlalchemy_bigquery
from google.cloud.bigquery import TimePartitioning

!bq rm -f tmp.test_table_create
engine = sqlalchemy.create_engine(f"bigquery:///tmp", echo=True)
meta = sqlalchemy.MetaData()
table = sqlalchemy.Table(
    f"tmp.test_table_create",
    meta,
    sqlalchemy.Column("integer_c", sqlalchemy.Integer, doc="column description"),
    sqlalchemy.Column("float_c", sqlalchemy.Float),
    sqlalchemy.Column("decimal_c", sqlalchemy.DECIMAL),
    sqlalchemy.Column("string_c", sqlalchemy.String),
    sqlalchemy.Column("text_c", sqlalchemy.Text),
    sqlalchemy.Column("boolean_c", sqlalchemy.Boolean),
    sqlalchemy.Column("timestamp_c", sqlalchemy.TIMESTAMP),
    sqlalchemy.Column("datetime_c", sqlalchemy.DATETIME),
    sqlalchemy.Column("date_c", sqlalchemy.DATE),
    sqlalchemy.Column("time_c", sqlalchemy.TIME),
    sqlalchemy.Column("binary_c", sqlalchemy.BINARY),
    bigquery_description="test table description",
    bigquery_friendly_name="test table name",
    bigquery_expiration_timestamp=datetime.datetime(2183, 3, 26, 8, 30, 0),
    bigquery_time_partitioning=TimePartitioning(
        field="date_c",
        expiration_ms=1000 * 60 * 60 * 24 * 30,  # 30 days
        type_="DAY"
    ),
    bigquery_require_partition_filter=True,
    bigquery_default_rounding_mode="ROUND_HALF_EVEN",
    bigquery_clustering_fields=["integer_c", "decimal_c"],
)
table.create(engine)

whereas in 1.12.1 it fails. From the description of #1113 it seems this omission was intentional, but I don't see why it's necessary to remove support for this (extremely fundamental and ubiquitous) type of partitioning: the old special casing for DATE seems like it would apply just as well here and IMO should be restored. Same goes for the change #1116 which was similarly not addressed.

Metadata

Metadata

Assignees

Labels

api: bigqueryIssues related to the googleapis/python-bigquery-sqlalchemy API.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions