Airbyte Provider: api_version on on_kill method in AirbytetriggeSyCoperator #46773
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
|
dantonbertuol
changed the title
Member
|
I am always suspicious of changes that change the code but have no "test" modified., This usually means the test coverage was not enough (if tests pass) or that the tests will fail. Both cases scream for tests adding/modification. Can you please add tests to cover it @dantonbertuol ? |
dantonbertuol
force-pushed
the
airbyte-on-kill-operator
branch
from
17631c9 to
b89e40f
Compare
* Add run_after to grid data * Update grid tests
* Add Pool UI * Text to display correct limit
* Align layout in trigger form with flexible form * Align helper text for Run ID as well * Take out the editable markdown * Revert extraction of JavaScript * Fix layout inconsistency after merge with Logical Date upstream --------- Co-authored-by: shubhamraj-git <shubhamraj.osc@gmail.com> Co-authored-by: Shubham Raj <48172486+shubhamraj-git@users.noreply.github.com>
* Add rendered templates tab. * Wrap value inside code block.
Co-authored-by: raphaelauv <raphaelauv@users.noreply.github.com>
This is a set of cleanup steps (first stage) that allow us to remove the "intermediate" provider's distribution from Airlfow code and replace it fully with individual provider's distributions - already with own `pyproject.toml` files and basically being (when we complete) a completely separate distributions from Airflow and without implicit dependencies between unrelated distributions. There are a number of other changes needed but that one is only focusing on removing all references to the "umbrella" `providers` distribution and consequences of removing it. Those are the changes implemented in this PR: * There are no separate "providers" system tests - each provider has own system tests and there are no common "generic" providers empty system test * Integration tests are moved to respective providers under the `integration` package inside `tests` directory * (nearly) empty __init__.py files are added in `tests` directories of providers - this way "tests" becomes just a directory and root for all tests per provider, rather than a Python package on its own. That allows to use "from integration.PROVIDER import" and "from system.PROVIDER" rather than importing them from the root of the whole airflow project. The (nearly) is because we need to handle multiple "system", "system.apache" and other import locations. * Removed references to "providers/" generic package which were scheduled for removal after all providers are moved to the new structure * Few remaining references / links referring to old "providers/src" and "providers/test" have been fixed. * The "conftest.py" files in all providers are trimmed down - the code to store ignored deprecation warnings have been moved to the test_common pytest_plugin. That allows to remove 90+ duplicated snippets of deprecation_warnings retrieval while keeping the warnings per-provider in the provider's distribution. * The "moving_providers" scripts are removed. They've done their job and are not needed any more - we keep them in history * The __init__.py files are automatically checked and properly updated in provider folders - in order to properly handle path extension mechanisms * The www tests that were using FAB permisssion model are moved to the FAB provider tests.
* Updated start.rst to avoid errors * Modified the quick start guide from pip to uv
…xt manager from SFTPHook (apache#46716) * fix: Make sure the SSHClient is also closed when getting a SFTPClient connection * refactor: Changed call of super get_conn in SFTPHook * refactor: Only open connection when call on connection has to be done * refactor: Added types for files and dirs --------- Co-authored-by: David Blain <david.blain@infrabel.be> Co-authored-by: David Blain <david.blain@b-holding.be>
This was added when we removed TaskContextLogger and replaced it with just adding events to the log table. Because pre-Airflow-3.0, executors did not have to inherit from BaseExecutor, we had to add this check. In 3.0, all executors should inherit from BaseExecutor so it should not be a problem.
* Using quotes for file path in find command * preventing quoted params
This is done by triggering the DAG that has the given asset as a task outlet. 409 is returned if more than one DAG uses the asset as outlet.
…to `None` (apache#46766) * Fix TypeError when deserializing task with None execution_timeout * Fix pytest-extraneous-scope-function (PT003)
* Do not run initial_db_init on module top level * Set default in tests * Move google provider `test_dataform.py` from tests/utils/ to provider/google/tests/unit/... * Fix test depending on parametrized params execution order
…e#46776) * refactor: Update DAG response model aligning with logical_date * fixup! refactor: Update DAG response model aligning with logical_date
The check-default-configuration pre-commit added in apache#46622 is slow and it's not necessary to be run always - only when configuration definition changes. Also it does not really take any files from changed PR as input, so it should be configured to not pass the files to it and to not run parallel instances - because when run with `--all-files` it will run as many parallel copies of it as many processors you have and they will essentially run the same check. Also this pre-commit requires breeze image to be present so it should be addded at the end of pre-commit files, so that is not run when breeze image is not built. All this behaviours have been fixed in this PR. After this change: * only one copy of the check is run when this pre-commit runs * in local pre-commit will only be run if config.yml changes * in canary runs it will always run (with --all-files) * it is marked as "breeze image" tests by placing it at the end of pre-commit configuration file
…pache#46810) There was a lot of code and references to old provider ways of handling old structure of providers. Once all providers have been moved, we can now remove that old code and rename old the "new_providers" references to just "providers"
* Add run_after to task_instances endpoints in fastapi * Fix CI and tests
…e#46798) Bumps [dompurify](https://github.com/cure53/DOMPurify) from 3.2.2 to 3.2.4. - [Release notes](https://github.com/cure53/DOMPurify/releases) - [Commits](cure53/DOMPurify@3.2.2...3.2.4) --- updated-dependencies: - dependency-name: dompurify dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
dantonbertuol
requested review from
Fokko,
XD-DENG,
ashb,
bbovenzi,
bolkedebruin,
eladkal,
ephraimbuddy,
hussein-awala,
jedcunningham,
josh-fell,
jscheffl,
mik-laj,
mobuchowski,
o-nikolas,
pierrejeambrun,
potiuk and
ryanahamilton
as code owners
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixed AirbytetriggeSyCoperator on_kill method, changing the instantiation of api_type=self.api_type to api_version = self.api_version, as the variable self.api_type no longer exists in the class.