Status of testing Providers that were prepared on April 21, 2023

[eladkal]

I have a kind request for all the contributors to the latest provider packages release.
Could you please help us to test the RC versions of the providers?

Let us know in the comment, whether the issue is addressed.

Those are providers that require testing as there were some substantial changes introduced:

Provider amazon: 8.0.0rc1

• Remove deprecated "delegate_to" from GCP operators and hooks (#30748): @shahar1
• take advantage of upcoming major version release to remove deprecated things (#30755): @vandonr-amz
• add a stop operator to emr serverless (#30720): @vandonr-amz
• SqlToS3Operator - Add feature to partition SQL table (#30460): @utkarsharma2
• New AWS sensor — DynamoDBValueSensor (#28338): @mrichman
• Fixed logging issue in DynamoDBValueSensor (#30703): @mrichman
• DynamoDBHook - waiter_path() to consider resource_type or client_type (#30595): @utkarsharma2
• Add ability to override waiter delay in EcsRunTaskOperator (#30586): @vincbeck
• Add support in AWS Batch Operator for multinode jobs (#29522): @vandonr-amz
• AWS logs. Exit fast when 3 consecutive responses are returned from AWS Cloudwatch logs (#30756): @vincbeck
• Remove @poke_mode_only from EmrStepSensor (#30774): @vincbeck
• Organize Amazon provider docs index (#30541): @eladkal
• Remove duplicate param docstring in EksPodOperator (#30634): @jlaneve

Provider apache.beam: 5.0.0rc1

• Remove deprecated "delegate_to" from GCP operators and hooks (#30748): @shahar1
• Add mechanism to suspend providers (#30422): @potiuk

Provider apache.kafka: 1.0.0rc1

• Add provider for Apache Kafka (#30175): @dylanbstorey

Provider cncf.kubernetes: 6.1.0rc1

• Add multiple exit code handling in skip logic for DockerOperator and KubernetesPodOperator (#30769): @eladkal
• Skip KubernetesPodOperator task when it returns a provided exit code (#29000): @hussein-awala

Provider databricks: 4.1.0rc1

• Add delete inactive run functionality to databricks provider (#30646): @phanikumv
• Databricks SQL sensor (#30477): @harishkesavarao

Provider dbt.cloud: 3.1.1rc1

• Merge DbtCloudJobRunAsyncSensor logic to DbtCloudJobRunSensor (#30227): @Lee-W
• Move typing imports behind TYPE_CHECKING in DbtCloudHook (#29989): @josh-fell

Provider docker: 3.6.0rc1

• Add multiple exit code handling in skip logic for DockerOperator and KubernetesPodOperator (#30769): @eladkal
• In DockerOperator, adding an attribute tls_verify to choose whether to validate certificate (#30309) (#30310): @oboki
• Deprecate skip_exit_code in DockerOperator and KubernetesPodOperator (#30733): @eladkal

Provider google: 10.0.0rc1

• Remove deprecated "delegate_to" from GCP operators and hooks (#30748): @shahar1
• Update Google Campaign Manager360 operators to use API v4 (#30598): @VladaZakharova
• Update DataprocCreateCluster operator to use 'label' parameter properly (#30741): @VladaZakharova
• BigQueryGetDataOperator should use project_id (#30651): @ying-w
• Display Video 360 cleanup v1 API usage (#30577): @lwyszomi

Provider microsoft.azure: 6.0.0rc1

• Remove deprecated "delegate_to" from GCP operators and hooks (#30748): @shahar1
• Add deferrable mode to WasbBlobSensor (#30488): @phanikumv

Provider mysql: 5.0.0rc1

• Remove mysql-connector-python (#30487): @moiseenkov

Provider presto: 5.0.0rc1

• Remove deprecated "delegate_to" from GCP operators and hooks (#30748): @shahar1
• Add mechanism to suspend providers (#30422): @potiuk

Provider snowflake: 4.0.5rc1

• Update documentation for snowflake provider 4.0 breaking change (#30020): @potiuk

Provider sqlite: 3.3.2rc1

• Use connection URI in SqliteHook (#28721): @feluelle

Provider trino: 5.0.0rc1

• Remove deprecated "delegate_to" from GCP operators and hooks (#30748): @shahar1
• Add mechanism to suspend providers (#30422): @potiuk

The guidelines on how to test providers can be found in

Verify providers by contributors

All users involved in the PRs:
@feluelle @ying-w @vandonr-amz @eladkal @mrichman @hussein-awala @lwyszomi @utkarsharma2 @jlaneve @josh-fell @vincbeck @shahar1 @Lee-W @harishkesavarao @oboki @phanikumv @moiseenkov @potiuk
@VladaZakharova @dylanbstorey

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[potiuk]

Looked at my changes - they were doc or "cI" only.

[vandonr-amz]

System tests revealed that RdsCreateDbSnapshotOperator had a bad default value for aws_conn_id (something that was hidden under a deprecation warning before). I don't know what percentage of users rely on the default value for this, but it should be fixed before we package this I think.
There is a PR for that: #30804

[eladkal]

amazon 8.0.0rc1 will be excluded from the release and will cut rc2 due to #30806
this will happen after I release the other providers.
Please continue to test rc1 as usual.

[vandonr-amz]

ok then if #30806 gets integrated in rc2, ignore my comment above.

[dylanbstorey]

Test DAGs worked as expected for apache.kafka 1.0.0rc1 (after fixing some errant strings, will submit PR to update those dags in a bit). #30818, submitted. Not required for release though.

[harishkesavarao]

Test DAG ran successfully with databricks: 4.1.0rc1.

[ying-w]

I tested my change locally and it works

[mrichman]

Tested my changes successfully:

#28338
#30703

[eladkal]

Thank you everyone.
amazon 8.0.0rc1 is excluded from the release and rc2 will follow shortly the rest of providers are released.

I invite everyone to help improve providers for the next release, a list of open issues can be found here.