[Sourcedb-to-spanner] Bulk migration Mysql to spanner 1tb Load test #2063
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2063 +/- ##
============================================
+ Coverage 46.16% 46.36% +0.19%
- Complexity 3843 4256 +413
============================================
Files 856 862 +6
Lines 51010 51340 +330
Branches 5365 5394 +29
============================================
+ Hits 23548 23803 +255
- Misses 25760 25823 +63
- Partials 1702 1714 +12
|
v2/sourcedb-to-spanner/src/test/resources/SourceDbToSpannerLT/mysql-spanner-schema.sql
Show resolved
Hide resolved
darshan-sj
force-pushed
the
import-export-it-2
branch
from
3fc39a8 to
4f87fa3
Compare
darshan-sj
requested review from
VardhanThigle,
asthamohta and
shreyakhajanchi
it/google-cloud-platform/src/main/java/org/apache/beam/it/jdbc/StaticJDBCResource.java
Show resolved
Hide resolved
it/google-cloud-platform/src/main/java/org/apache/beam/it/jdbc/StaticJDBCResource.java
Show resolved
Hide resolved
VardhanThigle
previously approved these changes
Dec 30, 2024
VardhanThigle
approved these changes
Dec 30, 2024
VardhanThigle
added
the
integration testing
taherkl
pushed a commit
to ollionorg/DataflowTemplates-fork
that referenced
this pull request
Jan 2, 2025
…oogleCloudPlatform#2063) * [Sourcedb-to-spanner] Bulk migration Mysql to spanner 1tb Load test * Updating row counts, added static sql resources * Renaming the test and addressing comments
taherkl
pushed a commit
to ollionorg/DataflowTemplates-fork
that referenced
this pull request
Jan 2, 2025
…oogleCloudPlatform#2063) * [Sourcedb-to-spanner] Bulk migration Mysql to spanner 1tb Load test * Updating row counts, added static sql resources * Renaming the test and addressing comments
shreyakhajanchi
pushed a commit
that referenced
this pull request
Jan 7, 2025
* Added Connection Helper * Added Formatter * Added File Reader * Refectored * Added Changes for Schema and Spanner Schema * Added Schema Changes to read Spanner Table in Schema object * Added Schema Changes * Added Changes for Schma * Added Source Writn Fn Changes * Added Source Factory Changes * Added Fixed for the Source factory and Casssandra Connection helper * Added Cassandra Schema Reader * Added Pipeline Process * Removed Unwanted Validation * Added Access validator * removed unwanted Return * Added Thread safe optimization in cassandra Connection helper * Applied spotless:apply * spotless:apply * Added Constructor for Test case * Added DUMMY Generator For UT * Fixed UT for Metadata config (#30) Co-authored-by: Narendra Rajput <narendra.rajput@ollion.com> Cassandra metadata PR To accomodate Driver Loader Class (#33) * Removed * * Create README.md for UDF samples (#2083) This commit adds a README.md file to the directory. The README file provides descriptions for each of the sample Javascript UDF files in the directory, including their purpose and usage examples. Co-authored-by: labs-code-app[bot] <161369871+labs-code-app[bot]@users.noreply.github.com> * CassandraDriverConfigLoader from GCS (#2077) * Added Config File Path * Added Fix for Loading Driver Options * Added Dependecy Fixes * Fix UT --------- Co-authored-by: labs-code-app[bot] <161369871+labs-code-app[bot]@users.noreply.github.com> Co-authored-by: Vardhan Vinay Thigle <39047439+VardhanThigle@users.noreply.github.com> * PR Review Comments (#35) * Convert it to builder Pattern * Convert Waring to Error * remove the unwanted comments * Removed Unwanted options * Address the PR * Address the PR Comments * Added Missing Getter for Configuration * Address to removed configuration changes * removed unwanted getters * Removal of getter from cassandra dao and test case fixes * Handle Optionmap in cassandra DAO for serialization * removed dependecy of getKeyOrder * Removed And Update Exception * Fix Checkstyle Voilation * Missing UT Added (#37) * Added Dummy Test case * Removed * from import * removed unwanted * Added Fixes * Added test case and fixes * Added Some more PR comments * Added Test case for Cassandra Reader * Added New testcase * Added Dependecny * Update test case and remove Dependency of Jupiter (#39) * Update test case and remove Dependecy of jupiter * Added UT fixes * Added Missing Getter for Configuration * [Sourcedb-to-spanner] Bulk migration Mysql to spanner 1tb Load test (#2063) * [Sourcedb-to-spanner] Bulk migration Mysql to spanner 1tb Load test * Updating row counts, added static sql resources * Renaming the test and addressing comments * PipelineController Changes for Cassandra (#2086) * Fix Spanner Load tests and add display test report (#2092) * correcting lt failures * Adding test report * Meta code coverage (#41) Meta code coverage * Pr bug fixes (#42) * Schema Reader Optimization (#43) * Added Pr Fix related to Changes Year in Javadoc * Address the revert of 2025 and removal of extra constructor * Added Extra Testcase to reach target patch --------- Co-authored-by: taherkl <taher.lakdawala@ollion.com> Co-authored-by: Taher Lakdawala <78196491+taherkl@users.noreply.github.com> Co-authored-by: pawankashyapollion <v-pawan.kumar@ollion.com> Co-authored-by: labs-code-app[bot] <161369871+labs-code-app[bot]@users.noreply.github.com> Co-authored-by: Vardhan Vinay Thigle <39047439+VardhanThigle@users.noreply.github.com> Co-authored-by: darshan-sj <darshan-sj@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.