-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bump testcontainers version to latest #30493
bump testcontainers version to latest #30493
Conversation
Before Merging a Connector Pull RequestWow! What a great pull request you have here! 🎉 To merge this PR, ensure the following has been done/considered for each connector added or updated:
If the checklist is complete, but the CI check is failing,
|
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build destination-cassandra docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ❌ |
Validate airbyte-integrations/connectors/destination-cassandra/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-cassandra test
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ✅ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build destination-elasticsearch-strict-encrypt docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ❌ |
Validate airbyte-integrations/connectors/destination-elasticsearch-strict-encrypt/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-elasticsearch-strict-encrypt test
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build destination-mongodb-strict-encrypt docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ✅ |
Validate airbyte-integrations/connectors/destination-mongodb-strict-encrypt/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-mongodb-strict-encrypt test
Coverage report for source-postgres
|
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build destination-redis docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ❌ |
Validate airbyte-integrations/connectors/destination-redis/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-redis test
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense! If you do note any new failures, and if you can document those for reference, that is much appreciated. That said, I now it is tricky to identify 'new' failures in cases where tests are failing already, so I wouldn't block on it either way.
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-clickhouse-strict-encrypt docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ❌ |
Validate airbyte-integrations/connectors/source-clickhouse-strict-encrypt/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-clickhouse-strict-encrypt test
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-postgres docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ✅ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-postgres/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-postgres test
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-mssql-strict-encrypt docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ❌ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ❌ |
Validate airbyte-integrations/connectors/source-mssql-strict-encrypt/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ❌ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mssql-strict-encrypt test
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build destination-postgres-strict-encrypt docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Build airbyte/normalization:dev | ✅ |
Java Connector Integration Tests | ❌ |
Validate airbyte-integrations/connectors/destination-postgres-strict-encrypt/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-postgres-strict-encrypt test
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build destination-pulsar docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ❌ |
Java Connector Integration Tests | ✅ |
Validate airbyte-integrations/connectors/destination-pulsar/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-pulsar test
|
Step | Result |
---|---|
Build connector tar | ❌ |
Validate airbyte-integrations/connectors/source-cockroachdb-strict-encrypt/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-cockroachdb-strict-encrypt test
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build destination-kafka docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ❌ |
Validate airbyte-integrations/connectors/destination-kafka/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-kafka test
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build destination-mssql-strict-encrypt docker image for platform linux/x86_64 | ❌ |
Validate airbyte-integrations/connectors/destination-mssql-strict-encrypt/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-mssql-strict-encrypt test
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-db2 docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ❌ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-db2/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-db2 test
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build destination-mongodb docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ❌ |
Validate airbyte-integrations/connectors/destination-mongodb/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-mongodb test
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build destination-oracle docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Build airbyte/normalization-oracle:dev | ✅ |
Java Connector Integration Tests | ❌ |
Validate airbyte-integrations/connectors/destination-oracle/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-oracle test
FYI @airbytehq/connector-operations the CI hit some github rate limiter:
I can't imagine any scenario where this is desirable. It's bad enough that the whole thing takes several days, frankly. I'm going to merge this PR now and deal with the consequences later. |
/approve-and-merge reason="need to unblock source-mysql" |
This PR bumps all testcontainers package versions to the latest across the board, following up on work started by @alafanechere. We need to do this to have access to images for the architectures that we use (arm machines on aws, apple silicon locally...). The need for this was especially pressing for source-mysql apparently, airbyte-ci just doesn't work for that connector right now.
Notice that I tie all testcontainers dependencies to one version number. This is deliberate, any testcontainers package will transitively depend on the testcontainers base package of the same version. To use a specific, different version in a connector for whatever reason, let's import it using a hardcoded string instead of a reference to deps.toml. @aaronsteers perhaps the CDK will formalize things in this respect? It seems sensible that testcontainers version pinning could happen in there.
I made a reasonable effort to test that this doesn't break anything that I think that we care about. This change actually breaks many connector's tests, but I haven't worried about those which were already routinely broken according to https://connectors.airbyte.com/files/generated_reports/connector_registry_report.html. The fixes, in any case, are fairly straightforward.
This PR will trigger a bunch of CI runs, which is actually desirable for a change. I'll go over the reports after the week-end, once they've had a chance to run. I did some manual tests before publishing this that verified that this fixes the source-mysql + strict-encrypt builds so this is ready for review. If any other connectors need fixing the changes are along the same lines so that shouldn't block the review.
Informs #29463