cleanup: Upgrade Composer samples to Airflow 2.0 #5782
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
product-auto-label
bot
added
the
samples
You are about to delete the following sample browser pages.
Here is the summary of changes. You are about to add 1 region tag.
You are about to delete 3 region tags.
This comment is generated by snippet-bot.
|
google-cla
bot
added
the
cla: yes
8 tasks
leahecole
force-pushed
the
airflow_migration
branch
from
78a9ae2 to
7c60e1c
Compare
|
YAY #5806 made the tests pass!! This is only red because I'm bad at remembering to run lint locally!!!!!!!!!!!!! |
8 tasks
leahecole
added
the
status: blocked
8 tasks
leahecole
force-pushed
the
airflow_migration
branch
from
05cb6f9 to
f87bcac
Compare
leahecole
changed the title
leahecole
removed
the
status: blocked
|
Just looked at the dataflow/dataproc tutorials, LGTM! |
sofisl
approved these changes
May 24, 2021
rachael-ds
reviewed
May 26, 2021
1 task
|
omg I have no idea how I made this mess with adding these requirements files. I'll fix later today 🤦🏻♀️ |
leahecole
force-pushed
the
airflow_migration
branch
2 times, most recently
from
affa66f to
207371f
Compare
leahecole
force-pushed
the
airflow_migration
branch
from
207371f to
f9447fa
Compare
leahecole
added
the
do not merge
|
I have been talking to TWs and need to move the Airflow 1 versions to a "legacy samples" folder |
leahecole
removed
the
do not merge
|
Actually, I'll do the legacy samples thing in a separate PR. I'm ready for you, @dinagraves ! :) |
dinagraves
approved these changes
May 27, 2021
jacek-izykowski
pushed a commit
to jacek-izykowski/python-docs-samples
that referenced
this pull request
Jul 13, 2021
…#5782) * modify dag_test_utils to point to branch; add airflow 2 changes * upgrade airflow_db_cleanup to Airflow 2.0 * upgrade hadoop tutorial to Airflow 2.0 * upgrade kubernetespodoperator to Airflow 2.0 * upgrade dataproc workflow template to Airflow 2.0 * remove bashoperator python2 example * upgrade DataflowTemplateOperator tutorial to 2.0 * WIP: upgrade bq_notify to Airflow 2 * upgrade bq_copy_across_locations to Airflow 2.0 * upgrade connections to Airflow 2 * more airflow 2.0 migration * db reset update * copy constraints from Airflow * update noxfile config * update constraints to py-3.8 * add pip version override * upgrade simple to airflow 2 * fix lint * bqnotify cleanup * add blog/ conftest * fix license header * more license header fixes * license again * remove unused import from blog conftest * remove unused requirements * remove script used by dag deleted in this PR * Add clarifying comment about email operator * add newline back to csv * fix presumed typo Co-authored-by: Dina Graves Portman <dinagraves@google.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR follows best practices for upgrading DAGs from Airflow 1 to Airflow 2. Changes included:
conftest.pyto theblogsfolder. For now, that DAG will stay at using Airflow 1.0, and we need to initialize a compatible DB for those tests separate from theworkflowsDB.workflows/requirements-test.txtcontribtoprovidersbash_operator->bash)If you would like access to my Composer environment to see the DAGs deployed, please let me know. Otherwise, here is a screenshot showing successful dag runs across the board
Fixes #5171
DataflowTemplateOperator tutorial update in CL/371792521 - removed instructions for setting
gce_regionAirflow variable (@sofisl )CHANGE NOT ADDRESSED - I need to look further at the DAG example we have triggered by GCF. That will be addressed in an upcoming PR.
Note: It's a good idea to open an issue first for discussion.
Checklist
nox -s py-3.6(see Test Environment Setup)nox -s lint(see Test Environment Setup)