-
Notifications
You must be signed in to change notification settings - Fork 16.6k
Open
Labels
area:providerskind:metaHigh-level information important to the communityHigh-level information important to the community
Description
Body
I have a kind request for all the contributors to the latest provider distributions release.
Could you please help us to test the RC versions of the providers?
The guidelines on how to test providers can be found in
Verify providers by contributors
Let us know in the comments, whether the issue is addressed.
These are providers that require testing as there were some substantial changes introduced:
Provider amazon: 9.22.0rc2
- Support custom KMS keys in S3CopyObjectOperator (#60597): @dominikhei
Linked issues: - Add resume_glue_job_on_retry to GlueJobOperator (#59392): @henry3260
Linked issues: - Add best-effort cleanup to EmrCreateJobFlowOperator on post-creation failure (#61010): @SameerMesiah97
Linked issues: - Add best-effort cleanup to EksCreateNodegroupOperator on post-create failure (#61145): @SameerMesiah97
- Add best-effort cleanup to EcsRunTaskOperator on post-start failure (#61051): @SameerMesiah97
- Restrict EC2CreateInstanceOperator cleanup to waiter failures and add guard flag (#61272): @SameerMesiah97
Linked issues: - Add cancel_on_kill support for EMR Serverless deferrable operator (#60440): @cruseakshay
- Add delay in
example_emr_eksafter updating execution role (#62415): @vincbeck - AIP-67 - Multi Team: Update AwsLambdaExecutor to support multi team (#61321): @henry3260
- Add SageMaker Unified Studio domain_id, project_id, domain_region as new parameters to SageMakerNotebookOperator (#62147): @Shnekit
Linked issues: - feat: Add Hook Level Lineage to SQL hooks (#61535): @kacpermuda
- EKS hook: Update auth. API version to v1 and replace non-POSIX 'source' with '.' (#61441): @idrisakorede
Linked issues: - add wait_policy in EmrCreateJobFlorOperator (#61195): @henry3260
Linked issues: - Fix
hostin AWS auth manager (#61368): @vincbeck - Fix variable name in
exemple_emr_eks(#62480): @vincbeck - fix S3RemoteLogIO using incorrect transfer config (#62423): @stephen-bracken
- Fix EKSPodOperator credential refresh errors and improve error handling (#57585) (#58743): @aviralgarg05
Linked issues: - Fix ECS Executor compatibility with Airflow 3.x in
try_adopt_task_instances(#62192): @vincbeck
Linked issues: - fix(amazon): flush file buffer in S3Hook.download_file() before returning path (#62078): @pippo995
Linked issues: - Handle CrawlerRunningException gracefully in GlueCrawlerOperator (#62016): @bahram-cdt
- RedshiftCreateClusterOperator could leave clusters running after failure (#61333): @SameerMesiah97
- Fix EKS operator failures by consolidating API version references into a shared constant (#61891): @ferruzzi
- Increase the redshift cluster deletion to 25 minutes in system tests (#62413): @vincbeck
- Bump min sqlalchemy version to 1.4.54 (#62299): @potiuk
- Update trigger rule in
example_emr_eks.py(#62228): @vincbeck - Fix EKS system test xcom issue and standardize naming between tests (#61893): @ferruzzi
- Add warning about AWS auth manager being in alpha stage in identity-c… (#61745): @imrichardwu
- fix(amazon): Filter kwargs in AthenaSQLHook to prevent TypeError (#62227): @shivaam
Linked issues:
Provider apache.beam: 6.2.3rc1
- Support Python 3.13 in apache beam provider (#61978): @cruseakshay
Linked issues:
Provider apache.drill: 3.3.0rc1
Provider apache.druid: 4.5.0rc1
Provider apache.hive: 9.3.0rc1
- feat: Add Hook Level Lineage to SQL hooks (#61535): @kacpermuda
- Fix HiveServer2Hook password handling for PLAIN auth (#62338) (#62364): @shashbha14
Linked issues: - Bump min sqlalchemy version to 1.4.54 (#62299): @potiuk
Provider apache.impala: 1.9.0rc1
- feat: Add Hook Level Lineage to SQL hooks (#61535): @kacpermuda
- Bump min sqlalchemy version to 1.4.54 (#62299): @potiuk
Provider apache.pinot: 4.10.0rc1
Provider apache.spark: 5.5.1rc1
Provider celery: 3.17.0rc1
- Celery: explicitly specify
psycopg2in driverless postgres URLs (#62159): @Dev-iL
Linked issues: - docs(celery): fix wrong link in cli (#62460): @Lee-W
- Insert a team_name similar to team conf for 2.X compat (#61923): @o-nikolas
Linked issues:
Provider cncf.kubernetes: 10.13.0rc1
- Add Kubernetes Secrets Backend to cncf.kubernetes provider (#61527): @piotrlinski
- Fix EKSPodOperator credential refresh errors and improve error handling (#57585) (#58743): @aviralgarg05
Linked issues: - fix: pod_override existing init_containers (#62284): @bintocher
Linked issues: - fix: Transient error state caused by rate limits from the container registry in
KubernetesPodOperator(#62215): @johnhoran
Linked issues:
Provider common.compat: 1.14.0rc1
Provider common.sql: 1.32.0rc2
- feat: Add Hook Level Lineage to SQL hooks (#61535): @kacpermuda
-
task-sql-decorator: Introducing the@task.sqldecorator (#60851): @jroachgolf84 - Bump min sqlalchemy version to 1.4.54 (#62299): @potiuk
Provider databricks: 7.10.0rc1
- feat: Add OpenLineage methods to DatabricksHook (#62179): @kacpermuda
Linked issues: - Databricks OIDC token federation for Kubernetes deployment (#61458): @mwojtyczka
Linked issues: - feat: Add Hook Level Lineage to SQL hooks (#61535): @kacpermuda
- Add missing template fields to DatabricksWorkflowTaskGroup (#61865): @pgagnon
Provider dbt.cloud: 4.6.5rc1
- Fix DbtCloudRunJobTrigger timeout error message and add final status check (#62306): @Vamsi-klu
Linked issues:
Provider edge3: 3.1.0rc1
- Introduce EdgeDBManager: Independent Provider Specific Database Schema Management (#61155): @dheerajturaga
Linked issues: - Add Filtering to Edge UI Plugin Jobs Page (#61934): @Eason09053360
Linked issues: - Route providers to consume Stats from common compat provider (#61812): @amoghrajesh
- Bump minimum prek version to 0.3.2 in edge3 provider (#62194): @dheerajturaga
- Auto-discover DB managers from provider.yaml (#62308): @dheerajturaga
- Upgrade dependency to minimatch (#62303): @jscheffl
- Warn when EdgeDBManager is missing from external_db_managers in config (#62204): @dheerajturaga
- Replace Base.metadata.remove() hack with dedicated Edge3 declarative base (#62202): @dheerajturaga
Linked issues: - Enable dynamic generation of assets for Edge UI Plugin (#56456): @jscheffl
- docs: Add external_db_managers configuration requirement (#62203): @dheerajturaga
Provider elasticsearch: 6.5.0rc2
- feat: Add Hook Level Lineage to SQL hooks (#61535): @kacpermuda
- Fix the write-to-es feature (#53821): @Owen-CH-Leung
Linked issues: - add max_lines_per_page setting to ElasticsearchTaskHandler (#61492): @stephen-bracken
Provider exasol: 4.10.0rc1
- feat: Add Hook Level Lineage to SQL hooks (#61535): @kacpermuda
- Bump min sqlalchemy version to 1.4.54 (#62299): @potiuk
Provider fab: 3.4.0rc1
- Auto-discover DB managers from provider.yaml (#62308): @dheerajturaga
- Feature/enable dynamic build of fab UI assets (#61959): @jscheffl
Linked issues: - feat(fab): Add configurable role key for Azure OAuth groups (#61585): @Eason09053360
Linked issues: - providers-fab: Handle database errors in cleanup_session_middleware Session.remove() (#62336): @8silvergun
Linked issues:- Linked Issue #61480: @yulit0738
- Linked Issue #57859: @FazalUrrehmanKhan
- Linked Issue #59349: @berglh
- Linked Issue #57470: @rebel-wansoo
- Fix 500 Error in FAB API Users endpoint by allowing naive datetimes i… (#62327): @anyeole
Linked issues:- Linked Issue #62325: @anyeole
- fix(providers/fab): restore OAuth callback route exposure (#62151): @tschroeder-zendesk
Linked issues:- Linked Issue #62028: @csp33
- fix: fab deserialize issue (#62153): @cruseakshay
Linked issues:- Linked Issue #61480: @yulit0738
- Linked Issue #61761: @inkerinmaa
- Linked Issue #61518: @anavrotski
- Linked Issue #61943: @MonsterChenzhuo
- Fix FAB auth_manager load_user causing PendingRollbackError and Inter… (#61943): @MonsterChenzhuo
- Unify get_permissions pagination with other list endpoints (#61726): @henry3260
- Remove unused dependencies from FAB provider (#62382): @vincbeck
- fab: centralize FastAPI auth manager routing (#61647): @henry3260
- Change FAB asset compilation from yarn to pnpm (#61958): @jscheffl
Linked issues:
Provider git: 0.2.4rc1
Provider google: 20.0.0rc2
- Migrate ADLSListOperator from ADLS Gen1 to Gen2 (#61188): @nailo2c
Linked issues:- Linked Issue #44228: @vince-vanh
- Add Google Cloud Ray Job integration (#59558): @VladaZakharova
- Return list of destination URIs from GCSToGCSOperator (#61320): @yuseok89
Linked issues:- Linked Issue #11323: @FHoffmannCode
- Return list of destination URIs from LocalFilesystemToGCSOperator (#61355): @yuseok89
- Return list of GCS URIs from Azure*ToGCS operators (#61048): @nailo2c
Linked issues:- Linked Issue #11323: @FHoffmannCode
- Add
run_idparameter to stop a specific execution of GCP Data Fusion pipelines (#61290): @chirodip98 - Add support for
parquetOptionsin GCSToBigQueryOperator (#60876): @mlauter - feat: Improve Hook Level Lineage for BigQueryHook (#62231): @kacpermuda
Linked issues: - Add unit tests for Gen AI operator exception handling. (#61790): @ytoprakc
Linked issues: - Add missing conn-fields for providers migrated to yaml (#62116): @amoghrajesh
Linked issues: - Return list of GCS URIs from FacebookAdsReportToGcsOperator (#61349): @Abhishekmishra2808
- feat: Add Hook Level Lineage to SQL hooks (#61535): @kacpermuda
- Return list of destination URIs from HttpToGCSOperator (#61306): @Srabasti
- Return files destination uris in
GoogleDriverToGCSOperatorandSheetsToGCSOperator(#61347): @Ajay9704
Linked issues:- Linked Issue #11323: @FHoffmannCode
- Fix deferrable mode in CloudRunExecuteJobOperator (#61546): @shahar1
Linked issues:- Linked Issue #60394: @arjav1528
- Fix CloudDataTransferServiceUpdateJobOperator AWS credential injection for S3 sources (#61611): @shahar1
Linked issues:- Linked Issue #22021: @abhinavraj23
- Change serialization inside GenAIGeminiCreateBatchJobOperator (#61253): @Crowiant
- In GCP Datafusion hook handle start_pipeline() when response is None / Invalid (#60688): @chirodip98
Linked issues:- Linked Issue #50387: @bhardwaj-priyanshu
- Pass credentials object to GCSFileSystem for automatic token refresh (#62013): @simi
Linked issues:- Linked Issue #37834: @pankajastro
- Linked Issue #38102: @pankajastro
- Fix serialization for return value in GenAIGeminiGetBatchJobOperator (#61842): @MaksYermak
- GoogleCalendarToGCSOperator: add unwrap_single flag and return full GCS URIs (#61284): @Ajay9704
Linked issues:- Linked Issue #11323: @FHoffmannCode
- Move exception handling from Google Bigtable operators to hooks (#61124): @KamranImaaz
Linked issues:- Linked Issue #60687: @KamranImaaz
- Replace
apache-beamwithapache-airflow-providers-apache-beam(#62071): @eladkal - Switch Cloud Run operators to use regional endpoints (#61857): @VladaZakharova
- Support Python 3.13 in apache beam provider (#61978): @cruseakshay
Linked issues: - GoogleDisplayVideo360SDFtoGCSOperator: add unwrap_single flag (#61709): @Ajay9704
Linked issues:- Linked Issue #61284: @Ajay9704
- Fix Ray Job system test in Google provider (#61848): @VladaZakharova
- Align GCS operator documentation headings with RST style (#61578): @KamranImaaz
- Align Google Cloud Compute operator documentation headings with RST style (#61574): @KamranImaaz
- Align Google Cloud Functions operator documentation headings with RST style (#61576): @KamranImaaz
- Align Google Cloud Vision operator documentation headings with RST style (#61577): @KamranImaaz
- Fix RST heading hierarchy in Google Cloud Build docs (#61365): @KamranImaaz
- Improve Bigtable operator documentation (#61277): @sawantbandita
Linked issues:- Linked Issue #61269: @KamranImaaz
- Update the heading for BigtableCreateTableOperator in docs (#61194): @KamranImaaz
- Update Stackdriver docstrings to Cloud Monitoring (#61635): @KamranImaaz
Linked issues:- Linked Issue #61032: @Jasminekaur2005
- Linked Issue #61009: @shahar1
Provider http: 6.0.0rc1
Provider jdbc: 5.4.0rc1
- feat: Add Hook Level Lineage to SQL hooks (#61535): @kacpermuda
- Providers: JDBC provider updated for conf import migration (#61413): @Aditi102005
Linked issues:- Linked Issue #60050: @Aditi102005
Provider keycloak: 0.5.3rc1
- Keycloak CLI: provision multi‑team resources for auth manager (AIP‑67) (#61256): @stegololz
Linked issues: - Keycloak auth manager: enforce team‑scoped authorization (AIP‑67) (#61351): @stegololz
Linked issues: - Fix HTTP 500 on /ui/teams endpoint when using Keycloak auth manager (#62471): @stegololz
Linked issues: - Encode id_token to avoid special characters. (#62429): @potiuk
Provider microsoft.azure: 13.0.0rc2
- Migrate ADLSListOperator from ADLS Gen1 to Gen2 (#61188): @nailo2c
Linked issues:- Linked Issue #44228: @vince-vanh
- Add wait_for_termination parameter and fix double-deferral in PowerBIDatasetRefreshOperator (#60369): @shreyas-dev
- Azure Message Bus - CommonMessageQueue Interface (#52712) (#61924): @nailo2c
Linked issues:- Linked Issue #52712: @jason810496
- Linked Issue #56445: @dejii
- Always apply proxy configuration with MSAL for KiotaRequestAdapterHook in msgraph module (#61199): @morelgeorge
Linked issues:- Linked Issue #61197: @morelgeorge
- fix: Add protocol validation to KiotaRequestAdapterHook (#61103): @Fury0508
Linked issues:- Linked Issue #61081: @ryan-dyer-sp
- Linked Issue #45006: @dabla
- Add missing conn-fields for providers migrated to yaml (#62116): @amoghrajesh
Linked issues:
Provider microsoft.mssql: 4.5.0rc1
Provider microsoft.winrm: 3.14.0rc1
Provider mysql: 6.5.0rc1
Provider odbc: 4.12.0rc1
Provider openlineage: 2.11.0rc1
- feat: Consume SQL hook lineage in OpenLineage (#62171): @kacpermuda
Linked issues: - feat: Add Dag Edge Labels to OpenLineage event (#62018): @kacpermuda
- feat: Add OpenLineage support for HITL operators (#61867): @kacpermuda
- Bump min sqlalchemy version to 1.4.54 (#62299): @potiuk
- Route providers to consume Stats from common compat provider (#61812): @amoghrajesh
Provider oracle: 4.5.0rc1
Provider pgvector: 1.7.0rc1
Provider postgres: 6.6.0rc1
- feat: Add Hook Level Lineage to SQL hooks (#61535): @kacpermuda
- Added validation for missing redshift connection host in postgres hook (#60918): @justinpakzad
Linked issues:- Linked Issue #58299: @rich7420
- Bump min sqlalchemy version to 1.4.54 (#62299): @potiuk
Provider presto: 5.11.0rc1
- feat: Add Hook Level Lineage to SQL hooks (#61535): @kacpermuda
- Bump min sqlalchemy version to 1.4.54 (#62299): @potiuk
Provider qdrant: 1.5.3rc1
- Cap qdrant_client version (#62193): @bugraoz93
Provider slack: 9.7.0rc1
- Add display_filename parameter to SlackAPIFileOperator (#62112): @yuseok89
Linked issues:- Linked Issue #61952: @yuseok89
Provider snowflake: 6.10.0rc1
- Add missing conn-fields for providers migrated to yaml (#62116): @amoghrajesh
Linked issues: - feat: Add Hook Level Lineage to SQL hooks (#61535): @kacpermuda
- Add retry logic for OAuth token requests in SnowflakeHook (#61796): @SameerMesiah97
Provider sqlite: 4.3.0rc1
Provider standard: 1.12.0rc1
- feat: Add OpenLineage support for HITL operators (#61867): @kacpermuda
- Add note support to TriggerDagRunOperator (#60810): @arnoldmr01
Linked issues:- Linked Issue #56543: @Urahara
Provider teradata: 3.5.0rc1
- feat: Add Hook Level Lineage to SQL hooks (#61535): @kacpermuda
- Bump min sqlalchemy version to 1.4.54 (#62299): @potiuk
Provider trino: 6.5.0rc1
Provider vertica: 4.3.0rc1
- feat: Add Hook Level Lineage to SQL hooks (#61535): @kacpermuda
- Bump min sqlalchemy version to 1.4.54 (#62299): @potiuk
Provider ydb: 2.5.0rc1
All users involved in the PRs:
@shivaam @arnoldmr01 @o-nikolas @Owen-CH-Leung @blcksrx @shreyas-dev @Abhishekmishra2808 @MonsterChenzhuo @pgagnon @eladkal @justinpakzad @piotrlinski @idrisakorede @kacpermuda @ferruzzi @nailo2c @Eas
Committer
- I acknowledge that I am a maintainer/committer of the Apache Airflow project.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
area:providerskind:metaHigh-level information important to the communityHigh-level information important to the community