Add on_kill equivalent to Databricks SQL Hook to cancel timed out queries#42668

Merged

uranusjr merged 212 commits intoapache:mainfrom

R7L208:lorin/databricks-sql-operator-on_kill-equivalent

Nov 13, 2024

Commits on Oct 2, 2024

add on_kill override to databricks workflow operator

Lorin
authored and
R7L208
committed
on_kill equivalent for DatabricksSqlOperator

Lorin
authored and
R7L208
committed
add tests for create_timeout_thread

Lorin
authored and
R7L208
committed
add note for on_kill in DatabricksCopyIntoOperator

Lorin
authored and
R7L208
committed
chore: static checks

Lorin
authored and
R7L208
committed
remove changes for databricks_workflow.py for PR isolated to databricks_sql

Lorin
authored and
R7L208
committed
remove file accidently picked up rebasing with main

Lorin
authored and
R7L208
committed
Revert "remove file accidently picked up rebasing with main"

Lorin
authored and
R7L208
committed
remove file from tracking

Lorin
authored and
R7L208
committed

Commits on Oct 3, 2024

Commits on Oct 4, 2024

Update airflow/providers/databricks/hooks/databricks_sql.py

R7L208
and
Lee-W
authored
add new exceptions
Lorin
committed
add new exceptions
Lorin
committed
fix typing warning
Lorin
committed
fix ruff error
Lorin
committed
fix ruff error
Lorin
committed

Commits on Oct 5, 2024

Merge branch 'main' into lorin/databricks-sql-operator-on_kill-equivalent
R7L208
authored

Commits on Oct 7, 2024

move new exceptions to Databricks provider
Lorin
committed
ruff formatting
Lorin
committed
add tests for new exceptions
Lorin
committed
chore - static checks
Lorin
committed

Commits on Oct 8, 2024

Merge branch 'main' into lorin/databricks-sql-operator-on_kill-equivalent
R7L208
authored

Commits on Oct 10, 2024

Update airflow/providers/databricks/hooks/databricks_sql.py

R7L208
and
Lee-W
authored

Commits on Oct 17, 2024

make new exceptions databricks specific
Lorin
committed
Simplify code for recent dbt provider change (#42840)

kaxil
authored and
Lorin
committed
- Fixes #42432 (#42770)

harjeevanmaan
authored and
Lorin
committed
AIP-84 Migrate views /object/historical_metrics_data to Fast API (#42629)

bugraoz93
authored and
Lorin
committed
Add support to filter by last dagrun state in UI. (#42779)

tirkarthi
authored and
Lorin
committed
feat(datasets): make strict_dataset_uri_validation default to True (#41814)

authored and
Lorin
committed
Standard provider bash operator (#42252)

gopidesupavan
authored and
Lorin
committed
Fix mark as success when pod fails while fetching log (#42815)

authored and
Lorin
committed
Fix spelling; `Airlfow` -> `Airflow` (#42855)

jbampton
authored and
Lorin
committed
Docs: Add Template field related info for python operators (#42847)

rawwar
authored and
Lorin
committed
AIP-84 Get Variable (#42834)

pierrejeambrun
authored and
Lorin
committed
Move test of DagRun.update_state to better place (#42845)

dstandish
authored and
Lorin
committed
Revert "Remove `sqlalchemy-redshift` dependency from Amazon provider (#42830)" (#42864)

mobuchowski
authored and
Lorin
committed
Prepare docs for Oct 1st adhoc wave of providers (#42862)

eladkal
authored and
Lorin
committed
fix more PT004 PyDocStyle checks (#42841)

dannyl1u
authored and
Lorin
committed
Revert Asset to Dataset for Core Extension Doc (#42867)

kaxil
authored and
Lorin
committed
Split providers out of the main "airflow/" tree into a UV workspace project (#42505)

ashb
authored and
Lorin
committed
Improving validation of task retries to handle None values (#42532)

authored and
Lorin
committed
Fix deprecated stage names for Pre-commit (#42872)

kaxil
authored and
Lorin
committed
Fix ui lint pre-commit interactive mode (#42854)

pierrejeambrun
authored and
Lorin
committed
update k8s tests urllib3 retry config status_forcelist and allowed_methods (#42871)

gopidesupavan
authored and
Lorin
committed
Bump `blacken-docs` pre-commit (#42880)

kaxil
authored and
Lorin
committed
Exclude "not-ready" providers when building docs (#42873)

kaxil
authored and
Lorin
committed
Move Hooks to Standard provider (#42794)

gopidesupavan
authored and
Lorin
committed
Fix dag warning documentation (#42858)

pierrejeambrun
authored and
Lorin
committed
Fix issue generation for provider after folder restructure (#42883)

eladkal
authored and
Lorin
committed
AIP-84 Migrate the public endpoint Get DAG to FastAPI (#42848)

omkar-foss
authored and
Lorin
committed
AIP-84 Migrate GET Dag Run endpoint to FastAPI (#42725)

authored and
Lorin
committed
add majorosdonat as bosch user (#42890)

authored and
Lorin
committed
Move the session auth backend to FAB auth manager (#42878)

vincbeck
authored and
Lorin
committed
Move user and roles schemas to fab provider (#42869)

vincbeck
authored and
Lorin
committed
Add support for run conf to backfill (#42865)

dstandish
authored and
Lorin
committed
Fix main (#42903)

pierrejeambrun
authored and
Lorin
committed
Fix main: js/types/api-generated.ts (#42906)

dstandish
authored and
Lorin
committed
mark TestGKEStartKueueInsideClusterOperator tests with flaky decorator (#42916)

gopidesupavan
authored and
Lorin
committed
Remove dag.run() method (#42761)

dstandish
authored and
Lorin
committed
Add test for behavior for paused backfill (#42837)

authored and
Lorin
committed
Use url_from_endpoint inside HttpHook. (#42785)

simi
authored and
Lorin
committed
Fix typo in Breeze (#42919)

kaxil
authored and
Lorin
committed
Docs: Add templating info to TaskFlow tutorial (#42887)

infused-kim
authored and
Lorin
committed
Add possibility to override the conn type for Druid (#42793)

authored and
Lorin
committed
fix: HttpSensorTrigger to include `method` when serializing (#42925)

rawwar
authored and
Lorin
committed
AIP-84 Fix dag display name search (#42863)

pierrejeambrun
authored and
Lorin
committed
Check _is_canary_run/pr condition in is_legacy_ui_api_labeled method (#42921)

gopidesupavan
authored and
Lorin
committed
Allow python 3.12 for the breeze release commands (#42936)

ashb
authored and
Lorin
committed
Render errors when getting a list of dags (#42897)

bbovenzi
authored and
Lorin
committed
Remove the referrer from Webserver to Scarf (#42901)

kaxil
authored and
Lorin
committed
Split providers out of the main "airflow/" tree into a UV workspace project (#42505) (#42624)

authored and
Lorin
committed
increase backoff_factor and add try/catch in k8s tests (#42940)

gopidesupavan
authored and
Lorin
committed
Consistent python version checks and troubleshooting (#42944)

potiuk
authored and
Lorin
committed
Update README.rst (#42941)

Andre Miranda
authored and
Lorin
committed
Add a "backfill create" command (#42922)

dstandish
authored and
Lorin
committed
Fix SNOWFLAKE_CONN_ID and DAG_ID in Snowpark system tests (#42952)

sfc-gh-jdu
authored and
Lorin
committed
Fix broken links in Release Management docs (#42958)

kaxil
authored and
Lorin
committed
Restrict looker-sdk version 24.18.0 and microsoft-kiota-http 1.3.4 (#42954)

gopidesupavan
authored and
Lorin
committed
upgrade trove-classifiers (#42950)

gopidesupavan
authored and
Lorin
committed
Chart: fix VCT for scheduler in local and persistent mode (#42946)

Aakcht
authored and
Lorin
committed
fix: use instance base_container_name to fetch logs on trigger_reentry (#42960)

peloyeje
authored and
Lorin
committed
fix: 03_contributors_quick_start_docs (#42927)

kgw7401
authored and
Lorin
committed
Add skeleton project for task-sdk (#42904)

kaxil
authored and
Lorin
committed
Upgrade Trove classifier to `2024.10.12` (#42961)

kaxil
authored and
Lorin
committed
Add missed brackets for our dev script for Spell checks (#42965)

kaxil
authored and
Lorin
committed
FIX: Don't raise a warning in ExecutorSafeguard when execute is called from an extended operator (#42849)

authored and
Lorin
committed
Upgrade Helm Chart dependencies to latest released (#42816)

potiuk
authored and
Lorin
committed
Fix PythonOperator DAG error when DAG has hyphen in name (#42902)

jason810496
authored and
Lorin
committed
fIX handling removal of dependencies (#42967)

potiuk
authored and
Lorin
committed
mark test_task_workflow_trigger_success as flaky (#42972)

gopidesupavan
authored and
Lorin
committed
Create a User Settings button with light/dark mode toggle as a menu item (#42964)

AryanK1511
authored and
Lorin
committed
AIP-84 Migrate delete Dag Run endpoint to FastAPI (#42910)

rawwar
authored and
Lorin
committed
Make datStats endpoint dag_ids parameter optional (#42955)

authored and
Lorin
committed
trove classifier upgrade (#42979)

gopidesupavan
authored and
Lorin
committed
Change directory used by simple auth manager to store generated passwords (#42860)

authored and
Lorin
committed
Disable flaky mssql based integration tests (#42811)

gopidesupavan
authored and
Lorin
committed
Remove BackfillJobRunner class (#42943)

dstandish
authored and
Lorin
committed
Add early job_id xcom_push for google provider Beam Pipeline operators (#42982)

authored and
Lorin
committed
Add logic to mark backfills as complete (#42683)

dstandish
authored and
Lorin
committed
fix mypy check failure on main (#42976)

authored and
Lorin
committed
mark test_setup_constraint_mapped_task_upstream_removed_and_success as flaky (#42997)

gopidesupavan
authored and
Lorin
committed
Update providers metadata 2024-10-14 (#42995)

eladkal
authored and
Lorin
committed
Update json schema pre-commit to have draft7 schema in file (#43005)

ephraimbuddy
authored and
Lorin
committed
AIP-84 Patch Variable (#42929)

pierrejeambrun
authored and
Lorin
committed
Commit the session between writing and deletion of RTIF (#42928)

ephraimbuddy
authored and
Lorin
committed
Refactor FastApi Dag and DagRun endpoints tests (#42949)

rawwar
authored and
Lorin
committed
Minor updates in UI contributing docs (#43013)

kaxil
authored and
Lorin
committed
AIP-84 post variable (#42948)

pierrejeambrun
authored and
Lorin
committed
Feature: Added event_handler parameter in MSGraphAsyncOperator (#42539)

authored and
Lorin
committed
uv version bump to 0.4.20 (#42905)

dirrao
authored and
Lorin
committed
Removed deprecated Chainable type from BaseOperator (#42776)

dirrao
authored and
Lorin
committed
pin in providers (#43001)

rawwar
authored and
Lorin
committed
Move tests_common from "dev" to top-level. (#42985)

potiuk
authored and
Lorin
committed
Add AssetActive model (#42612)

uranusjr
authored and
Lorin
committed
Migrate the public endpoint Delete DAG to FastAPI (#42914)

omkar-foss
authored and
Lorin
committed
Clarifying PLUGINS_FOLDER permissions by DAG authors (#43022)

amoghrajesh
authored and
Lorin
committed
kubernetes package version bump to 31.0.0 (#42907)

dirrao
authored and
Lorin
committed
Make google provider pyarrow dependency explicit (#42996)

saucoide
authored and
Lorin
committed
Add search by dag_display_name_pattern on dag list page with rebase (#42896)

authored and
Lorin
committed
Deprecate session auth backend (#42909)

vincbeck
authored and
Lorin
committed
fix: Change CustomSecurityManager method name (#43034)

kgw7401
authored and
Lorin
committed
Update to 0.4.21 of UV (#43032)

potiuk
authored and
Lorin
committed
Add retry on error 502 and 504 (#42994)

authored and
Lorin
committed
AIP-84 Migrate get connections to FastAPI API #42571 (#42782)

authored and
Lorin
committed
Make `RedshiftDataOperator` handle multiple queries (#42900)

jroachgolf84
authored and
Lorin
committed
Bump `uv` to `0.4.21` in other places (#43033)

kaxil
authored and
Lorin
committed
require 1.2.1 common.compat for openlineage provider (#43039)

mobuchowski
authored and
Lorin
committed
Detect system color theme and add active state to nav button (#43041)

bbovenzi
authored and
Lorin
committed
Migrate health info to fastapi (#42938)

bbovenzi
authored and
Lorin
committed
Add upperbound to microsoft-kiota-abstractions (#43021)

rawwar
authored and
Lorin
committed
fix path to providers dir for mount sources breeze flag (#43042)

mobuchowski
authored and
Lorin
committed
Create Operators for Google Cloud Vertex AI Context Caching (#43008)

CYarros10
authored and
Lorin
committed
add lowerbount to requests-toolbelt and replace requests_toolbelt with requests-toolbelt (#43020)

rawwar
authored and
Lorin
committed
✨ Allow node_selector templating in KPO (#43051)

bdsoha
authored and
Lorin
committed
Rename views to routes for FastAPI apps (#43057)

kaxil
authored and
Lorin
committed
Follow-up SLA purge (#42808)

authored and
Lorin
committed
Bump `uv` to `0.4.22` (#43056)

kaxil
authored and
Lorin
committed
Add 'name' and 'group' to public Asset class (#42812)

uranusjr
authored and
Lorin
committed
Remove zombie ti from its executor (#42932)

uranusjr
authored and
Lorin
committed
Drop unneeded unique() call on SQL (#43064)

uranusjr
authored and
Lorin
committed
Allow building docs for not-ready providers (#43071)

kaxil
authored and
Lorin
committed
Typo correction in 07_local_virtualenv.rst - "way" (#43072)

biswa-b
authored and
Lorin
committed
Enable explicit namespaces (#42951)

authored and
Lorin
committed
Replace Mateusz with Kalyan in the triage team (#43074)

potiuk
authored and
Lorin
committed
Reorganize ``api_fastapi`` folder into apps (#43062)

kaxil
authored and
Lorin
committed
Tweak AssetAlias to match Asset for AIP-74 additions (#42814)

uranusjr
authored and
Lorin
committed
Fix pytest from working outside breeze (#43082)

kaxil
authored and
Lorin
committed
Fix typo in ``pyproject.toml`` (#43077)

shahar1
authored and
Lorin
committed
Remove DAG.get_num_active_runs (#43067)

dstandish
authored and
Lorin
committed
Fix UI lint pre-commit hook (#43086)

pierrejeambrun
authored and
Lorin
committed
Fixed failing static checks (#43087)

kaxil
authored and
Lorin
committed
Added task_instance_mutation_hook for mapped operator index 0 (#42661)

authored and
Lorin
committed
Fix selective checks when only pyproject.toml changes (#43088)

potiuk
authored and
Lorin
committed
fix(providers/mongo): prevent applying lower method on boolean field (#43024)

josix
authored and
Lorin
committed
add kubernetes_conn_id to templated fields (#42786)

gopidesupavan
authored and
Lorin
committed
Add min version to cloudpickle (#43066)

rawwar
authored and
Lorin
committed
Initialize dashboard page with health (#43090)

bbovenzi
authored and
Lorin
committed
Update constraints to broken canary (#43095)

jscheffl
authored and
Lorin
committed
Fix flaky `test_get_dags` in FastAPI routes (#43100)

kaxil
authored and
Lorin
committed
SSHHook expose auth_timeout parameter (#43048)

gyandeeps
authored and
Lorin
committed
vertex ai training operators: add display_name to rendered fields (#43028)

authored and
Lorin
committed
Print the key name when max_length is exceeded (#43061)

jabbera
authored and
Lorin
committed
Revert "Fix flaky `test_get_dags` in FastAPI routes (#43100)" (#43108)

pierrejeambrun
authored and
Lorin
committed
Ensure stable secondary ordering (#43085)

pierrejeambrun
authored and
Lorin
committed
Add simple "Task Execution" API server (#43015)

kaxil
authored and
Lorin
committed
Add ClientConnectorError to be a retryable error in databricks provider (#43091)

rawwar
authored and
Lorin
committed
passing the filetype for SlackAPIFileOperator (#43069)

Bowrna
authored and
Lorin
committed
chore(docs): add required import of BranchDayOfWeekOperator (#43053)

authored and
Lorin
committed
Update unit_tests.rst links error (#43114)

yangyulely
authored and
Lorin
committed
added MultipleFilesWebHdfsSensor (#43045)

eilon246810
authored and
Lorin
committed
Bump to `0.4.23` to fix a uv bug (#43121)

kaxil
authored and
Lorin
committed
Clean up some eslint rules to better fit what we do (#43093)

bbovenzi
authored and
Lorin
committed
Fixed failing static checks & provider tests (#43122)

kaxil
authored and
Lorin
committed
Add min version to python-ldap (#43104)

rawwar
authored and
Lorin
committed
feat(providers/fab): Use asset in common provider (#43112)

Lee-W
authored and
Lorin
committed
Add `bugraoz93` in the triage team (#43124)

kaxil
authored and
Lorin
committed
add on_kill override to databricks workflow operator
Lorin
committed
on_kill equivalent for DatabricksSqlOperator
Lorin
committed
add tests for create_timeout_thread
Lorin
committed
add note for on_kill in DatabricksCopyIntoOperator
Lorin
committed
chore: static checks
Lorin
committed
remove changes for databricks_workflow.py for PR isolated to databricks_sql
Lorin
committed
add indent to execution_timeout param in docstring of DatabricksSqlHook.run
Lorin
committed
Update airflow/providers/databricks/hooks/databricks_sql.py

authored and
Lorin
committed
add new exceptions
Lorin
committed
add new exceptions
Lorin
committed
fix typing warning
Lorin
committed
fix ruff error
Lorin
committed
fix ruff error
Lorin
committed
move new exceptions to Databricks provider
Lorin
committed
ruff formatting
Lorin
committed
add tests for new exceptions
Lorin
committed
chore - static checks
Lorin
committed
Update airflow/providers/databricks/hooks/databricks_sql.py

authored and
Lorin
committed
make new exceptions databricks specific
Lorin
committed
accidentally removed necessary changes when resolving conflicts
Lorin
committed
remove redefinition of mock_workflow_run_metadata
Lorin
committed
merge main into this branch bc rebase failed
Lorin
committed
Merge branch 'main' into lorin/databricks-sql-operator-on_kill-equivalent
R7L208
authored

Commits on Oct 21, 2024

Merge branch 'main' into lorin/databricks-sql-operator-on_kill-equivalent
R7L208
authored

Commits on Oct 23, 2024

Merge branch 'main' into lorin/databricks-sql-operator-on_kill-equivalent
R7L208
authored

Commits on Oct 24, 2024

Merge branch 'main' into lorin/databricks-sql-operator-on_kill-equivalent
R7L208
authored

Commits on Oct 28, 2024

Merge branch 'main' into lorin/databricks-sql-operator-on_kill-equivalent
R7L208
authored

Commits on Nov 12, 2024

Merge branch 'main' into lorin/databricks-sql-operator-on_kill-equivalent
R7L208
authored