Skip to content

fix: BigQueryEngineAdapter.get_table_schema() mypy error #5153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
georgesittas merged 1 commit into from
Aug 14, 2025
Merged

fix: BigQueryEngineAdapter.get_table_schema() mypy error #5153

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion sqlmesh/core/engine_adapter/bigquery.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -296,7 +296,9 @@ def get_bq_schema(self, table_name: TableName) -> t.List[bigquery.SchemaField]:
table = exp.to_table(table_name)
if len(table.parts) == 3 and "." in table.name:
self.execute(exp.select("*").from_(table).limit(0))
return self._query_job._query_results.schema
query_job = self._query_job
assert query_job is not None
Copy link
Collaborator

@erindru erindru Aug 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since asserts only work when running python in debug mode, it's generally better to do something like:

if query_job is None:
  raise ...

mypy will still pick up on that and it makes the check explicit

Copy link
Contributor

@georgesittas georgesittas Aug 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we have quite a few asserts in the codebase. Is this something we want to be consistent about? Not sure if we've been thinking about them the way you described, but you're right:

In the current implementation, the built-in variable __debug__ is True under normal circumstances, False when optimization is requested (command line option -O). The current code generator emits no code for an assert statement when optimization is requested at compile time

Copy link
Collaborator

@erindru erindru Aug 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I realised that we have been using assert incorrectly outside of tests to make the type checker happy.

I learned that validating the assumptions using actual runtime checks:

  • Still keeps the type checker happy in most cases
  • Forces us to handle the case when something that theoretically shouldn't happen, happens anyway

Copy link
Contributor Author

@newtonapple newtonapple Aug 14, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I can change it to raise a RuntimeError or an explicit AssertionError. But like I said in the comment, I'm simply porting the original logic over. Do you prefer to change the BigQueryEngineAdapter.columns() implementation to raise an exception explicitly as well?

Copy link
Contributor

@georgesittas georgesittas Aug 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we can one-shot this more generally in the codebase by asking Claude to do it. Like, "change assertions aiming to satisfy mypy with raise statements" or something.

Or... just do it for the BigQuery adapter and leave the bigger refactor for later :)

return query_job._query_results.schema
return self._get_table(table).schema

def columns(
Expand Down