-
Notifications
You must be signed in to change notification settings - Fork 16.4k
Prevent OpenLineage failure when BigQuery table metadata is missing #60380
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prevent OpenLineage failure when BigQuery table metadata is missing #60380
Conversation
mobuchowski
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@KamranImaaz can we leave the conditional imports? OpenLineage isn't a required provider; the operator should work even if it's not installed.
@mobuchowski Got it.I will update the implementation to keep the imports conditional. |
c170d03 to
3b887d6
Compare
|
@mobuchowski I have Modified it. Looks good to me. Could you pls look into it. Thanks :) |
|
Thanks @mobuchowski :) |
|
Hi @eladkal All Checks have passed, PR approved and the Changes can be merged cleanly :) |
|
Thanks @KamranImaaz. |
…pache#60380) * modified * replaced back the imports
Fixes #60096
What
Adds a safety condition to OpenLineage extraction to prevent errors when
BigQueryUpdateTableSchemaOperatordo not return a table resource.Also removes duplicate and unused imports
Why
OpenLineage callbacks run after task execution and should not cause
task failures when metadata is unavailable.