Destination Redshift: deprecate old migration normalization code #25771
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
octavia-squidington-iii
added
area/documentation
This comment was marked as outdated.
This comment was marked as outdated.
Affected Connector ReportNOTE
|
| Connector | Version | Changelog | Publish |
|---|
- See "Actionable Items" below for how to resolve warnings and errors.
❌ Destinations (15)
| Connector | Version | Changelog | Publish |
|---|---|---|---|
destination-bigquery |
1.3.4 |
✅ | ✅ |
destination-bigquery-denormalized |
1.3.3 |
✅ | ✅ |
destination-clickhouse |
0.2.3 |
✅ | ✅ |
destination-clickhouse-strict-encrypt |
0.2.3 |
🔵 (ignored) |
🔵 (ignored) |
destination-mssql |
0.1.23 |
✅ | ✅ |
destination-mssql-strict-encrypt |
0.1.23 |
🔵 (ignored) |
🔵 (ignored) |
destination-mysql |
0.1.20 |
✅ | ✅ |
destination-mysql-strict-encrypt |
❌ 0.1.21(mismatch: 0.1.20) |
🔵 (ignored) |
🔵 (ignored) |
destination-oracle |
0.1.19 |
✅ | ✅ |
destination-oracle-strict-encrypt |
0.1.19 |
🔵 (ignored) |
🔵 (ignored) |
destination-postgres |
0.3.27 |
✅ | ✅ |
destination-postgres-strict-encrypt |
0.3.27 |
🔵 (ignored) |
🔵 (ignored) |
destination-redshift |
0.4.7 |
✅ | ✅ |
destination-snowflake |
1.0.0 |
✅ | ✅ |
destination-tidb |
0.1.1 |
✅ | ✅ |
- See "Actionable Items" below for how to resolve warnings and errors.
👀 Other Modules (1)
- base-normalization
Actionable Items
(click to expand)
| Category | Status | Actionable Item |
|---|---|---|
| Version | ❌ mismatch |
The version of the connector is different from its normal variant. Please bump the version of the connector. |
| ⚠ doc not found |
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug. |
|
| Changelog | ⚠ doc not found |
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug. |
| ❌ changelog missing |
There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog. | |
| Publish | ⚠ not in seed |
The connector is not in the seed file (e.g. source_definitions.yaml), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug. |
| ❌ diff seed version |
The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the /publish command to publish the latest version. |
edgao
approved these changes
May 2, 2023
There was a problem hiding this comment.
lgtm assuming tests pass
but you'll probably need to remove this test case https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/bases/base-normalization/integration_tests/test_normalization.py#L486
|
No idea why the so yeah, the only failure is the one that's expected to fail now. |
cynthiaxyin
force-pushed
the
cynthia/destination_redshift_migration_deprecate_normalization
branch
from
397d6f5 to
da70f49
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
cynthiaxyin
force-pushed
the
cynthia/destination_redshift_migration_deprecate_normalization
branch
from
da70f49 to
1bad657
Compare
This comment was marked as outdated.
This comment was marked as outdated.
|
/test connector=bases/base-normalization
Build PassedTest summary info: |
|
/publish connector=bases/base-normalization run-tests=false
if you have connectors that successfully published but failed definition generation, follow step 4 here |
cynthiaxyin
deleted the
cynthia/destination_redshift_migration_deprecate_normalization
branch
marcosmarxm
pushed a commit
to natalia-miinto/airbyte
that referenced
this pull request
Jun 8, 2023
…bytehq#25771) * first pass normalization * add pr link * remove python test & resources * linting
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
closes #25519
Functionality was added over a year ago (#12064) to migrate Redshift tables from
VARCHARtype toSUPERtype. Tables have now beenSUPERtype by default for awhile, so we are removing this old migration code that is no longer needed.How
Delete migration-related normalization code. Java code was deleted in #25698.
Recommended reading order
configuration.sqlandcolumns.sql(remove macros)*.sql(remove usages of macros)🚨 User Impact 🚨
The set of impacted users is expected to be very small: anyone who has been running old versions of Redshift and Airbyte for over a year and is still using
VARCHARtables with likely stale data. Anyone who is impacted can manually migrate their Redshift tables if needed. The team voted to remove this functionality as it was implemented over a year ago.