Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Splits bases and updates build.gradle files #25649

Merged
merged 6 commits into from
May 6, 2023
Merged

Conversation

ryankfu
Copy link
Contributor

@ryankfu ryankfu commented Apr 27, 2023

What

Closes #25225

Splits base-java and bases-destination-jdbc such that it unblocks Q2a projects for normalization and Snowflake performance

How

  1. Copies base-java and renames to base-java-async
  2. Copies bases-destination-jdbc to bases-destination-jdbc-aync
  3. Updates build.gradle for bases-destination-jdbc-aync and airbyte to point to aync variants
  4. Updates settings.gradle to include base-java-async and bases-destination-jdbc-async into build process

Recommended reading order

  1. build.gradle
  2. settings.gradle
  3. Draw the rest of the 🦉

🚨 User Impact 🚨

Are there any breaking changes? What is the end result perceived by the user?

For connector PRs, use this section to explain which type of semantic versioning bump occurs as a result of the changes. Refer to our Semantic Versioning for Connectors guidelines for more information. Breaking changes to connectors must be documented by an Airbyte engineer (PR author, or reviewer for community PRs) by using the Breaking Change Release Playbook.

If there are breaking changes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.

@ryankfu

This comment was marked as outdated.

@ryankfu

This comment was marked as outdated.

@github-actions
Copy link
Contributor

github-actions bot commented Apr 27, 2023

Affected Connector Report

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to do the following as needed:

  • Run integration tests
  • Bump connector or module version
  • Add changelog
  • Publish the new version

✅ Sources (0)

Connector Version Changelog Publish
  • See "Actionable Items" below for how to resolve warnings and errors.

❌ Destinations (23)

Connector Version Changelog Publish
destination-azure-blob-storage 0.2.0
destination-clickhouse 0.2.3
destination-clickhouse-strict-encrypt 0.2.3 🔵
(ignored)
🔵
(ignored)
destination-databricks 1.0.2
destination-dynamodb 0.1.7
destination-exasol 0.1.1
destination-gcs 0.3.0
destination-mariadb-columnstore 0.1.7
destination-mssql 0.1.23
destination-mssql-strict-encrypt 0.1.23 🔵
(ignored)
🔵
(ignored)
destination-mysql 0.1.20
destination-mysql-strict-encrypt 0.1.21
(mismatch: 0.1.20)
🔵
(ignored)
🔵
(ignored)
destination-oracle 0.1.19
destination-oracle-strict-encrypt 0.1.19 🔵
(ignored)
🔵
(ignored)
destination-postgres 0.3.27
destination-postgres-strict-encrypt 0.3.27 🔵
(ignored)
🔵
(ignored)
destination-redshift 0.4.7
destination-rockset 0.1.4
destination-snowflake 1.0.2
destination-starburst-galaxy 0.0.1
destination-teradata 0.1.1
destination-tidb 0.1.1
destination-yugabytedb 0.1.1
  • See "Actionable Items" below for how to resolve warnings and errors.

👀 Other Modules (1)

  • base-normalization

Actionable Items

(click to expand)

Category Status Actionable Item
Version
mismatch
The version of the connector is different from its normal variant. Please bump the version of the connector.

doc not found
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.
Changelog
doc not found
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.

changelog missing
There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog.
Publish
not in seed
The connector is not in the seed file (e.g. source_definitions.yaml), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug.

diff seed version
The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the /publish command to publish the latest version.

@ryankfu ryankfu force-pushed the ryan/split-bases-refactor branch from fe03fba to 133ab18 Compare April 27, 2023 21:36
@ryankfu

This comment was marked as outdated.

@ryankfu

This comment was marked as outdated.

@ryankfu ryankfu force-pushed the ryan/split-bases-refactor branch from 133ab18 to 1363def Compare April 27, 2023 22:25
@ryankfu

This comment was marked as outdated.

@ryankfu

This comment was marked as outdated.

@octavia-squidington-iii octavia-squidington-iii added the area/documentation Improvements or additions to documentation label Apr 27, 2023
@ryankfu

This comment was marked as outdated.

@ryankfu ryankfu marked this pull request as ready for review April 27, 2023 23:26
@ryankfu ryankfu requested a review from a team as a code owner April 27, 2023 23:26
@bnchrch
Copy link
Contributor

bnchrch commented Apr 28, 2023

@ryankfu This is was something I did not consider with the new checklist github action.

Let me push a fix here, one sec

Test

  • a

@ryankfu

This comment was marked as outdated.

@ryankfu ryankfu force-pushed the ryan/split-bases-refactor branch from 830ecd7 to 7677a93 Compare April 28, 2023 18:18
@ryankfu

This comment was marked as outdated.

@ryankfu

This comment was marked as outdated.

Copy link
Contributor

@edgao edgao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blind approval

@ryankfu

This comment was marked as outdated.

@ryankfu

This comment was marked as outdated.

@ryankfu
Copy link
Contributor Author

ryankfu commented May 5, 2023

/test connector=connectors/destination-bigquery

🕑 connectors/destination-bigquery https://github.com/airbytehq/airbyte/actions/runs/4898189517
✅ connectors/destination-bigquery https://github.com/airbytehq/airbyte/actions/runs/4898189517
Python tests coverage:

Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
normalization/transform_config/__init__.py                            2      0   100%
normalization/transform_catalog/reserved_keywords.py                 15      0   100%
normalization/transform_catalog/__init__.py                           2      0   100%
normalization/destination_type.py                                    18      0   100%
normalization/__init__.py                                             4      0   100%
normalization/transform_catalog/destination_name_transformer.py     171     10    94%
normalization/transform_catalog/table_name_registry.py              174     34    80%
normalization/transform_config/transform.py                         195     48    75%
normalization/transform_catalog/utils.py                             51     14    73%
normalization/transform_catalog/dbt_macro.py                         22      7    68%
normalization/transform_catalog/catalog_processor.py                147     80    46%
normalization/transform_catalog/transform.py                         65     39    40%
normalization/transform_catalog/stream_processor.py                 595    400    33%
-------------------------------------------------------------------------------------
TOTAL                                                              1461    632    57%

Build Passed

Test summary info:

All Passed

@ryankfu
Copy link
Contributor Author

ryankfu commented May 5, 2023

/test connector=connectors/destination-snowflake

🕑 connectors/destination-snowflake https://github.com/airbytehq/airbyte/actions/runs/4898190262
✅ connectors/destination-snowflake https://github.com/airbytehq/airbyte/actions/runs/4898190262
Python tests coverage:

Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
normalization/transform_config/__init__.py                            2      0   100%
normalization/transform_catalog/reserved_keywords.py                 15      0   100%
normalization/transform_catalog/__init__.py                           2      0   100%
normalization/destination_type.py                                    18      0   100%
normalization/__init__.py                                             4      0   100%
normalization/transform_catalog/destination_name_transformer.py     171     10    94%
normalization/transform_catalog/table_name_registry.py              174     34    80%
normalization/transform_config/transform.py                         195     48    75%
normalization/transform_catalog/utils.py                             51     14    73%
normalization/transform_catalog/dbt_macro.py                         22      7    68%
normalization/transform_catalog/catalog_processor.py                147     80    46%
normalization/transform_catalog/transform.py                         65     39    40%
normalization/transform_catalog/stream_processor.py                 595    400    33%
-------------------------------------------------------------------------------------
TOTAL                                                              1461    632    57%

Build Passed

Test summary info:

All Passed

@ryankfu

This comment was marked as outdated.

@ryankfu
Copy link
Contributor Author

ryankfu commented May 5, 2023

/publish connector=connectors/destination-snowflake run-tests=false

🕑 Publishing the following connectors:
connectors/destination-snowflake
https://github.com/airbytehq/airbyte/actions/runs/4898437409


Connector Version Did it publish? Were definitions generated?
connectors/destination-snowflake 1.0.2

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@ryankfu ryankfu enabled auto-merge (squash) May 6, 2023 00:03
@ryankfu ryankfu merged commit c673b0a into master May 6, 2023
@ryankfu ryankfu deleted the ryan/split-bases-refactor branch May 6, 2023 00:24
ryankfu added a commit that referenced this pull request May 15, 2023
ryankfu added a commit that referenced this pull request May 15, 2023
* Revert "Splits bases and updates build.gradle files (#25649)"

This reverts commit c673b0a.

* Bumps branch to prevent a conflict with publishing

* Forward fixes Snowflake to use singular base-java and develop within a new package within the same module

* Forcing automated change to merge changes
marcosmarxm pushed a commit to natalia-miinto/airbyte that referenced this pull request Jun 8, 2023
* Splits bases and updates build.gradle files

* Fixed changelog out of sync

* Bumps version number and metadata files

* auto-bump connector version

* Downgraded untouched connector bumps

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
marcosmarxm pushed a commit to natalia-miinto/airbyte that referenced this pull request Jun 8, 2023
airbytehq#26081)

* Revert "Splits bases and updates build.gradle files (airbytehq#25649)"

This reverts commit c673b0a.

* Bumps branch to prevent a conflict with publishing

* Forward fixes Snowflake to use singular base-java and develop within a new package within the same module

* Forcing automated change to merge changes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

One Java One Publish: Split bases-destination-jdbc and base-java into async and sequential modules
4 participants