Exclude partition from BigQuery table name

[moiseenkov]

• Bugfixed BigQuery table name splitting by excluding partition name following after the "$".
• deprecated the function split_tablename which is identical to the method BigQueryHook.split_tablename.

[nathadfield]

I think I have discovered an issue with this commit that is affecting tasks that use the BigQueryToBigQueryOperator when copying a table into specific partition of another table.

Results in an error like this.

google.api_core.exceptions.BadRequest: 400 Failed to copy Non partitioned table to Column partitioned table: not supported.; reason: invalid, message: Failed to copy Non partitioned table to Column partitioned table: not supported.

I will dig into it more tomorrow and hopefully have a fix for it ASAP.

[nathadfield]

@moiseenkov Why are we excluding the partition from the table_id?

[moiseenkov]

@moiseenkov Why are we excluding the partition from the table_id?

The reason was that partitioning breaks integration of this operator with Google Cloud Dataplex when it is launched in Cloud Composer. Apparently, my fix introduced another bug, so we should probably revert it. Regarding the initial problem with integration, I think we can take care of it on the Cloud Composer's side. Sorry for troubling.

[nathadfield]

@moiseenkov No worries. It's not your fault. We should have better test coverage. Do you want to revert it?

[moiseenkov]

@moiseenkov No worries. It's not your fault. We should have better test coverage. Do you want to revert it?

This PR contains one valuable deprecation which we'd like to keep. Thus we will introduce another PR that only reverts the breaking changes.

[nathadfield]

Sounds good! I will actually look at adding a test to the BigQueryToBigQueryOperator for this scenario too.

[moiseenkov]

Thanks!