Sync v3-0-stable with v3-0-test to release 3.0.3
#52732
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…pache#51401) (apache#51425) * Attach downstream assets to task * Adjust tests (cherry picked from commit 14ee1f4) Co-authored-by: Pierre Jeambrun <pierrejbrun@gmail.com>
…pache#51476) (cherry picked from commit 3fb3fb3)
(cherry picked from commit 414407b) Co-authored-by: Pierre Jeambrun <pierrejbrun@gmail.com>
(cherry picked from commit a08ea0a) Co-authored-by: Dheeraj Turaga <dheerajturaga@gmail.com>
These were outdated. Still need to automate this process! (cherry picked from commit 687decf) Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
* Sync python client readme Sync python client readme with https://github.com/apache/airflow-client-python/pull/135/files * Revert bad change from airflow-client-python * pre-commit pass (cherry picked from commit 1ccca81) Co-authored-by: r-richmond <rrichmond.gh@gmail.com>
…che#51584) (cherry picked from commit 6ab6d53) Co-authored-by: Karthikeyan Singaravelan <tir.karthi@gmail.com>
(cherry picked from commit baf5fae)
The command that was generated before: ``` docker run --rm -u 501:20 -v /Users/kaxilnaik/Documents/GitHub/astronomer/airflow/clients/python/v2.yaml:/spec.yaml -v /Users/kaxilnaik/Documents/GitHub/astronomer/airflow/clients/python/tmp:/output openapitools/openapi-generator-cli:v7.13.0 generate --input-spec /spec.yaml --generator-name python --git-user-id None --output /output --package-name airflow_client.client --git-repo-id airflow-client-python --additional-properties 'packageVersion="3.0.0"' ``` which caused things like `__version__ = ""2.10.0""` ( [here](https://github.com/apache/airflow-client-python/blob/4bd5b2544e30b05bd8cd03502dc99bd6784a20d6/airflow_client/client/__init__.py) ) or `__version__ = ""3.0.0""` in [here](https://github.com/apache/airflow-client-python/blob/38367b1158b777c87f51fe4d6283273031a4fb60/airflow_client/client/__init__.py#L17) While this does not cause error the user-agent and things are weird: ```py self.user_agent = 'OpenAPI-Generator/"3.0.0"/python' ``` vs ```py self.user_agent = 'OpenAPI-Generator/3.0.0/python' ``` (cherry picked from commit e1f4290)
(cherry picked from commit e6430c2)
) (apache#51579) * Structure endpoint task level resolved aliases * Address PR comments (cherry picked from commit acf1e77) Co-authored-by: Pierre Jeambrun <pierrejbrun@gmail.com>
…ache#51617) * feat: add a config setting to expose stacktraces * fix: remove the starting newline to simplify the traceback info * test: update tests to align with the config change * feat: add exception id for better correlation between ui messages and log entries * test: update tests * fix: use random string as exception id instead of python object id * fix: update tests with patched random strings * test: use patch fixtures in tests to prevent side effects (cherry picked from commit c4cd58c) Co-authored-by: Zhen-Lun (Kevin) Hong <zhenlun.hong01@gmail.com>
…ion IDs (apache#51599) (apache#51613) (cherry picked from commit f41f5f2) Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>
…) (apache#51626) (cherry picked from commit b6db7a7) Co-authored-by: Guan Ming(Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
…pache#51642) Not sure how this ended up with the wrong number... (cherry picked from commit 61e9b08) Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>
* Move asset events to its own tab * Address PR comments (cherry picked from commit e8505aa)
…ction (apache#51627) (apache#51654) We didn't close log file descriptor properly hence leading to too many open files error from the operating system. (cherry picked from commit 4a0a89b) Co-authored-by: Adylzhan Khashtamov <adil.khashtamov@gmail.com>
…pache#51611) (apache#51612) (cherry picked from commit db435f7) Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
…ache#51670) (apache#51673) The `dag.test` method currently (in 3.0.2) clears dag runs before running ( [code](https://github.com/apache/airflow/blob/3.0.2/task-sdk/src/airflow/sdk/definitions/dag.py#L1092-L1097) ) the dag. For that it uses `SchedulerDAG.clear_dags`, which create `SchedulerDAG` from Task SDK DAG -- to do that it uses [`deepcopy.copy`](https://github.com/apache/airflow/blob/3.0.2/airflow-core/src/airflow/models/dag.py#L1797). Now that works for most cases, but not where `jinja2.Template` class is passed as a Task/Operator argument because of [this issue](pallets/jinja#758). ```python In [1]: from copy import deepcopy ...: ...: from jinja2 import Template ...: ...: deepcopy(Template('')) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) Cell In[1], line 5 1 from copy import deepcopy 3 from jinja2 import Template ----> 5 deepcopy(Template('')) File ~/.local/share/uv/python/cpython-3.12.8-macos-aarch64-none/lib/python3.12/copy.py:162, in deepcopy(x, memo, _nil) 160 y = x 161 else: --> 162 y = _reconstruct(x, memo, *rv) 164 # If is its own copy, don't memoize. 165 if y is not x: File ~/.local/share/uv/python/cpython-3.12.8-macos-aarch64-none/lib/python3.12/copy.py:253, in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy) 251 if deep and args: 252 args = (deepcopy(arg, memo) for arg in args) --> 253 y = func(*args) 254 if deep: 255 memo[id(x)] = y File ~/.local/share/uv/python/cpython-3.12.8-macos-aarch64-none/lib/python3.12/copyreg.py:99, in __newobj__(cls, *args) 98 def __newobj__(cls, *args): ---> 99 return cls.__new__(cls, *args) TypeError: Template.__new__() missing 1 required positional argument: 'source' ``` This is a general issue that can affect any DAG using custom arguments that store or cache Jinja2 Template objects or similar. Clearly, the longer term solution for dag.test is to not use SchedulerDAG like that and instead call some task Execution endpoint to clear dagruns -- but this works for 3.0.x. (cherry picked from commit 75f8aa9) Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
Running pre-commits for typescript compilation, linting formatting is currently very slow. The pre-commit that is doing it took more than 2.5 minutes - always, independently which files changed. Also, no matter whether UI or simple auth manager UI changed, both were compiled. This is the first iteration of speeding up the pre-commit especially for incremental changes. The following changes were implemented: * the scripts were split into ui and simple-auth-manager-ui * timing information and verbose commands displayed so that we can measure which commands take how much time * the pre-commmit now passes all the files as input (and pre-commit is serialized) - which means that some commands can only be run on the subset of files that changed * tsc step is modified to only run on a subset of .ts and .tsx files that were changed (via dynamic extension of the project) Overall when just few files change, the pre-commit should now complete in less than 10-20 seconds Still some optimisations are possible in the future. (cherry picked from commit 52be0dd)
…pache#51755) (apache#51757) The openeop-gen files do not follow our formatting and whitespace expectations - which means that our linting rules and whitespace pre-commit might complain and mofify them after they are generated. But we definitely do not want it and, hey - they are generated so we should just skip them whenever we do linting or whitespace correction. (cherry picked from commit 2c92125)
…pache#51732) (apache#51768) (cherry picked from commit 69da3f7) Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
) (apache#51798) (cherry picked from commit 0df91d9) Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
…che#51822) (apache#51824) We changed the rule last year and now "the highest" version of Python is default version for docker, but the docs were not updated in the table to reflect that. (cherry picked from commit c0f5328) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
boring-cyborg
bot
added
the
backport-to-v3-1-test
jedcunningham
approved these changes
Jul 2, 2025
kaxil
force-pushed
the
3.0.3-rel-v3-0-test
branch
from
dafcb6e to
502f988
Compare
kaxil
requested review from
XD-DENG,
amoghrajesh,
ashb,
bbovenzi,
bolkedebruin,
bugraoz93,
dstandish,
eladkal,
ephraimbuddy,
gopidesupavan,
hussein-awala,
jason810496,
josh-fell,
jscheffl,
mik-laj,
o-nikolas,
pierrejeambrun,
potiuk,
rawwar,
ryanahamilton,
uranusjr and
vincbeck
as code owners
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
16 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR Syncs
v3-0-stablewithv3-0-testto release 3.0.3.Release notes and version bumps added in
Changes between Task SDK 1.0.2 and 1.0.3
run_as_userto avoid "double" logging/plain-text-over-stdout #51934)run_as_user) support for task execution #51780)xcom_pullto cover different scenarios for mapped tasks #51568)EOFis missed (#51180) #51970)lenin GetXComCount processing toxcom_count(#51611) #51612)dag.testconsistent withairflow dags testCLI command #51476)^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.