Skip to content

Comments

fix: BigQueryEngineAdapter.get_table_schema() mypy error#5153

Merged
georgesittas merged 1 commit intomainfrom
ddai/fix-get_bq_schema
Aug 14, 2025
Merged

fix: BigQueryEngineAdapter.get_table_schema() mypy error#5153
georgesittas merged 1 commit intomainfrom
ddai/fix-get_bq_schema

Conversation

@newtonapple
Copy link
Contributor

This follows the same logic from BigQueryEngineAdapter.columns() to ensure query_job is not None before using it.

@newtonapple newtonapple requested a review from izeigerman August 14, 2025 00:20
self.execute(exp.select("*").from_(table).limit(0))
return self._query_job._query_results.schema
query_job = self._query_job
assert query_job is not None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since asserts only work when running python in debug mode, it's generally better to do something like:

if query_job is None:
  raise ...

mypy will still pick up on that and it makes the check explicit

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we have quite a few asserts in the codebase. Is this something we want to be consistent about? Not sure if we've been thinking about them the way you described, but you're right:

In the current implementation, the built-in variable __debug__ is True under normal circumstances, False when optimization is requested (command line option -O). The current code generator emits no code for an assert statement when optimization is requested at compile time

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I realised that we have been using assert incorrectly outside of tests to make the type checker happy.

I learned that validating the assumptions using actual runtime checks:

  • Still keeps the type checker happy in most cases
  • Forces us to handle the case when something that theoretically shouldn't happen, happens anyway

Copy link
Contributor Author

@newtonapple newtonapple Aug 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I can change it to raise a RuntimeError or an explicit AssertionError. But like I said in the comment, I'm simply porting the original logic over. Do you prefer to change the BigQueryEngineAdapter.columns() implementation to raise an exception explicitly as well?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we can one-shot this more generally in the codebase by asking Claude to do it. Like, "change assertions aiming to satisfy mypy with raise statements" or something.

Or... just do it for the BigQuery adapter and leave the bigger refactor for later :)

@georgesittas
Copy link
Contributor

@newtonapple @erindru I'll go ahead and merge this PR for now to unblock CI in main / other PRs. Let's address the assert vs raise in a followup.

@georgesittas georgesittas merged commit e4ea4c8 into main Aug 14, 2025
27 of 28 checks passed
@georgesittas georgesittas deleted the ddai/fix-get_bq_schema branch August 14, 2025 10:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants