Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[certified db sources] adopt latest CDK to use error translation bug fixes #44841

Merged
merged 11 commits into from
Aug 28, 2024
Merged

[certified db sources] adopt latest CDK to use error translation bug fixes #44841

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
35e2271
always use lower cases for error message regex matching in error tran…
theyueli Aug 27, 2024
22559b6
readme
theyueli Aug 27, 2024
b9eb13a
formatting
theyueli Aug 27, 2024
a195b2b
code review
theyueli Aug 27, 2024
2ed8e7f
code review
theyueli Aug 27, 2024
de2f068
adopt latest CDK for ceritified db sources
theyueli Aug 28, 2024
8854dcc
Merge branch 'master' into yue/bump-certified-connectors
theyueli Aug 28, 2024
e46008b
bump up versions
theyueli Aug 28, 2024
6048df3
Merge branch 'master' into yue/bump-certified-connectors
theyueli Aug 28, 2024
93f5fa1
Merge branch 'master' into yue/bump-certified-connectors
theyueli Aug 28, 2024
8b90695
update mongodb version
theyueli Aug 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-mongodb-v2/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ plugins {
}

airbyteJavaConnector {
cdkVersionRequired = '0.44.4'
cdkVersionRequired = '0.44.17'
features = ['db-sources', 'datastore-mongo']
useLocalCdk = false
}
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-mongodb-v2/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ data:
connectorSubtype: database
connectorType: source
definitionId: b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e
dockerImageTag: 1.5.6
dockerImageTag: 1.5.7
dockerRepository: airbyte/source-mongodb-v2
documentationUrl: https://docs.airbyte.com/integrations/sources/mongodb-v2
githubIssueLabel: source-mongodb-v2
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-mssql/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ plugins {
}

airbyteJavaConnector {
cdkVersionRequired = '0.44.4'
cdkVersionRequired = '0.44.17'
features = ['db-sources']
useLocalCdk = false
}
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-mssql/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ data:
connectorSubtype: database
connectorType: source
definitionId: b5ea17b1-f170-46dc-bc31-cc744ca984c1
dockerImageTag: 4.1.8
dockerImageTag: 4.1.9
dockerRepository: airbyte/source-mssql
documentationUrl: https://docs.airbyte.com/integrations/sources/mssql
githubIssueLabel: source-mssql
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-mysql/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ plugins {
}

airbyteJavaConnector {
cdkVersionRequired = '0.44.4'
cdkVersionRequired = '0.44.17'
features = ['db-sources']
useLocalCdk = false
}
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-mysql/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ data:
connectorSubtype: database
connectorType: source
definitionId: 435bb9a5-7887-4809-aa58-28c27df0d7ad
dockerImageTag: 3.7.0
dockerImageTag: 3.7.1
dockerRepository: airbyte/source-mysql
documentationUrl: https://docs.airbyte.com/integrations/sources/mysql
githubIssueLabel: source-mysql
Expand Down
3 changes: 3 additions & 0 deletions ...e-integrations/connectors/source-mysql/src/test/kotlin/MySqlSourceExceptionHandlerTest.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
* Copyright (c) 2024 Airbyte, Inc., all rights reserved.
*/

import io.airbyte.cdk.integrations.util.FailureType
import io.airbyte.integrations.source.mysql.MySqlSourceExceptionHandler
import java.io.EOFException
import java.sql.SQLSyntaxErrorException
Expand All @@ -21,6 +22,7 @@ class MySqlSourceExceptionHandlerTest {
fun testTranslateMySQLSyntaxException() {
val exception = SQLSyntaxErrorException("Unknown column 'xmin' in 'field list'")
val externalMessage = exceptionHandler!!.getExternalMessage(exception)
Assertions.assertTrue(exceptionHandler!!.checkErrorType(exception, FailureType.CONFIG))
Assertions.assertEquals(
"A column needed by MySQL source connector is missing in the database",
externalMessage
Expand All @@ -34,6 +36,7 @@ class MySqlSourceExceptionHandlerTest {
"Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost."
)
val externalMessage = exceptionHandler!!.getExternalMessage(exception)
Assertions.assertTrue(exceptionHandler!!.checkErrorType(exception, FailureType.TRANSIENT))
Assertions.assertEquals("Can not read data from MySQL server", externalMessage)
}
}
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-postgres/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ java {
}

airbyteJavaConnector {
cdkVersionRequired = '0.44.4'
cdkVersionRequired = '0.44.17'
features = ['db-sources', 'datastore-postgres']
useLocalCdk = false
}
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-postgres/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ data:
connectorSubtype: database
connectorType: source
definitionId: decd338e-5647-4c0b-adf4-da0e75f5a750
dockerImageTag: 3.6.16
dockerImageTag: 3.6.17
dockerRepository: airbyte/source-postgres
documentationUrl: https://docs.airbyte.com/integrations/sources/postgres
githubIssueLabel: source-postgres
Expand Down
2 changes: 2 additions & 0 deletions ...grations/connectors/source-postgres/src/test/kotlin/PostgresSourceExceptionHandlerTest.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
* Copyright (c) 2024 Airbyte, Inc., all rights reserved.
*/

import io.airbyte.cdk.integrations.util.FailureType
import io.airbyte.integrations.source.postgres.PostgresSourceExceptionHandler
import org.junit.jupiter.api.Assertions
import org.junit.jupiter.api.BeforeEach
Expand All @@ -21,6 +22,7 @@ class PostgresSourceExceptionHandlerTest {
val exception =
PSQLException("ERROR: temporary file size exceeds temp_file_limit (500kB)", null)
val externalMessage = exceptionHandler!!.getExternalMessage(exception)
Assertions.assertTrue(exceptionHandler!!.checkErrorType(exception, FailureType.TRANSIENT))
Assertions.assertEquals("Encountered an error while reading the database", externalMessage)
}
}
1 change: 1 addition & 0 deletions docs/integrations/sources/mongodb-v2.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,7 @@ For more information regarding configuration parameters, please see [MongoDb Doc

| Version | Date | Pull Request | Subject |
|:--------|:-----------| :------------------------------------------------------- |:----------------------------------------------------------------------------------------------------------|
| 1.5.7 | 2024-08-27 | [44841](https://github.com/airbytehq/airbyte/pull/44841) | Adopt latest CDK. |
| 1.5.6 | 2024-08-27 | [44839](https://github.com/airbytehq/airbyte/pull/44839) | DBZ filters in related streams only. |
| 1.5.5 | 2024-08-26 | [44779](https://github.com/airbytehq/airbyte/pull/44779) | Revert permission check on oplog.rs. |
| 1.5.4 | 2024-08-20 | [44490](https://github.com/airbytehq/airbyte/pull/44490) | Add read permission check on oplog.rs collection used by CDC. |
Expand Down
1 change: 1 addition & 0 deletions docs/integrations/sources/mssql.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -422,6 +422,7 @@ WHERE actor_definition_id ='b5ea17b1-f170-46dc-bc31-cc744ca984c1' AND (configura

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------|
| 4.1.9 | 2024-08-27 | [44841](https://github.com/airbytehq/airbyte/pull/44841) | Adopt latest CDK. |
| 4.1.8 | 2024-08-08 | [43410](https://github.com/airbytehq/airbyte/pull/43410) | Adopt latest CDK. |
| 4.1.7 | 2024-08-06 | [42869](https://github.com/airbytehq/airbyte/pull/42869) | Adopt latest CDK. |
| 4.1.6 | 2024-07-30 | [42550](https://github.com/airbytehq/airbyte/pull/42550) | Correctly report stream states. |
Expand Down
1 change: 1 addition & 0 deletions docs/integrations/sources/mysql.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -233,6 +233,7 @@ Any database or table encoding combination of charset and collation is supported

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:-----------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------|
| 3.7.1 | 2024-08-27 | [44841](https://github.com/airbytehq/airbyte/pull/44841) | Adopt latest CDK. |
| 3.7.0 | 2024-08-13 | [44013](https://github.com/airbytehq/airbyte/pull/44013) | Upgrading to Debezium 2.7.1.Final |
| 3.6.9 | 2024-08-08 | [43410](https://github.com/airbytehq/airbyte/pull/43410) | Adopt latest CDK. |
| 3.6.8 | 2024-07-30 | [42869](https://github.com/airbytehq/airbyte/pull/42869) | Adopt latest CDK. |
Expand Down
1 change: 1 addition & 0 deletions docs/integrations/sources/postgres.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,7 @@ According to Postgres [documentation](https://www.postgresql.org/docs/14/datatyp

| Version | Date | Pull Request | Subject |
|---------|------------|----------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 3.6.17 | 2024-08-27 | [44841](https://github.com/airbytehq/airbyte/pull/44841) | Adopt latest CDK. |
| 3.6.16 | 2024-08-15 | [44119](https://github.com/airbytehq/airbyte/pull/44119) | Fix incorrect final state on initial read in CDC mode. |
| 3.6.15 | 2024-08-12 | [43945](https://github.com/airbytehq/airbyte/pull/43945) | Add missing replication slot config error. |
| 3.6.14 | 2024-08-08 | [43418](https://github.com/airbytehq/airbyte/pull/43418) | Adopt latest CDK. |
Expand Down
Loading