fix: BigQueryEngineAdapter.get_table_schema() mypy error#5153
fix: BigQueryEngineAdapter.get_table_schema() mypy error#5153georgesittas merged 1 commit intomainfrom
Conversation
| self.execute(exp.select("*").from_(table).limit(0)) | ||
| return self._query_job._query_results.schema | ||
| query_job = self._query_job | ||
| assert query_job is not None |
There was a problem hiding this comment.
Since asserts only work when running python in debug mode, it's generally better to do something like:
if query_job is None:
raise ...
mypy will still pick up on that and it makes the check explicit
There was a problem hiding this comment.
I think we have quite a few asserts in the codebase. Is this something we want to be consistent about? Not sure if we've been thinking about them the way you described, but you're right:
In the current implementation, the built-in variable
__debug__isTrueunder normal circumstances,Falsewhen optimization is requested (command line option -O). The current code generator emits no code for an assert statement when optimization is requested at compile time
There was a problem hiding this comment.
Yeah I realised that we have been using assert incorrectly outside of tests to make the type checker happy.
I learned that validating the assumptions using actual runtime checks:
- Still keeps the type checker happy in most cases
- Forces us to handle the case when something that theoretically shouldn't happen, happens anyway
There was a problem hiding this comment.
Sure, I can change it to raise a RuntimeError or an explicit AssertionError. But like I said in the comment, I'm simply porting the original logic over. Do you prefer to change the BigQueryEngineAdapter.columns() implementation to raise an exception explicitly as well?
There was a problem hiding this comment.
Perhaps we can one-shot this more generally in the codebase by asking Claude to do it. Like, "change assertions aiming to satisfy mypy with raise statements" or something.
Or... just do it for the BigQuery adapter and leave the bigger refactor for later :)
|
@newtonapple @erindru I'll go ahead and merge this PR for now to unblock CI in main / other PRs. Let's address the |
This follows the same logic from BigQueryEngineAdapter.columns() to ensure
query_jobis not None before using it.