-
Notifications
You must be signed in to change notification settings - Fork 1.1k
data-1172-dataflow #35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
data-1172-dataflow #35
Conversation
…sub topic, transforms the event via customizable JavaScript, and then emits the transformed event on a separate topic
… pubsub events and determines if any currently running strategies have triggers that care. If so, the event is re-emitted on a separate topic. This saves us thousands of dollars a month as we handle the firehose of all events in dataflow and then handle only the interesting events in strategy-engine-api.
|
Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). 📝 Please visit https://cla.developers.google.com/ to sign. Once you've signed (or fixed any issues), please reply here (e.g. What to do if you already signed the CLAIndividual signers
Corporate signers
ℹ️ Googlers: Go here for more info. |
|
Sorry, I put the pull request on the base branch when I meant to put it on our fork. 😊 I closed the pull request and deleted that branch.
Thanks!
Mike Daniels
Developed the Infusionsoft Campaign Builder
Software Engineer V
…________________________________
From: googlebot <notifications@github.com>
Sent: Tuesday, April 30, 2019 4:55:50 PM
To: GoogleCloudPlatform/DataflowTemplates
Cc: Mike Daniels; Author
Subject: Re: [GoogleCloudPlatform/DataflowTemplates] data-1172-dataflow (#35)
Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).
📝 Please visit https://cla.developers.google.com/ to sign.
Once you've signed (or fixed any issues), please reply here (e.g. I signed it!) and we'll verify it.
________________________________
What to do if you already signed the CLA
Individual signers
* It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data<https://cla.developers.google.com/clas> and verify that your email is set on your git commits<https://help.github.com/articles/setting-your-email-in-git/>.
Corporate signers
* Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot<http://go/cla#troubleshoot> (Public version<https://opensource.google.com/docs/cla/#troubleshoot>).
* The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data<https://cla.developers.google.com/clas> and verify that your email is set on your git commits<https://help.github.com/articles/setting-your-email-in-git/>.
* The email used to register you as an authorized contributor must also be attached to your GitHub account<https://github.com/settings/emails>.
ℹ️ Googlers: Go here<https://goto.google.com/prinfo/https%3A%2F%2Fgithub.com%2FGoogleCloudPlatform%2FDataflowTemplates%2Fpull%2F35> for more info.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub<#35 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ABZP5BK6YPUYFQM7W5PEUELPTDMANANCNFSM4HJQ73XQ>.
|
* Added Connection Helper * Added Formatter * Added File Reader * Refectored * Added Changes for Schema and Spanner Schema * Added Schema Changes to read Spanner Table in Schema object * Added Schema Changes * Added Changes for Schma * Added Source Writn Fn Changes * Added Source Factory Changes * Added Fixed for the Source factory and Casssandra Connection helper * Added Cassandra Schema Reader * Added Pipeline Process * Removed Unwanted Validation * Added Access validator * removed unwanted Return * Added Thread safe optimization in cassandra Connection helper * Applied spotless:apply * spotless:apply * Added Constructor for Test case * Added DUMMY Generator For UT * Fixed UT for Metadata config (#30) Co-authored-by: Narendra Rajput <narendra.rajput@ollion.com> Cassandra metadata PR To accomodate Driver Loader Class (#33) * Removed * * Create README.md for UDF samples (#2083) This commit adds a README.md file to the directory. The README file provides descriptions for each of the sample Javascript UDF files in the directory, including their purpose and usage examples. Co-authored-by: labs-code-app[bot] <161369871+labs-code-app[bot]@users.noreply.github.com> * CassandraDriverConfigLoader from GCS (#2077) * Added Config File Path * Added Fix for Loading Driver Options * Added Dependecy Fixes * Fix UT --------- Co-authored-by: labs-code-app[bot] <161369871+labs-code-app[bot]@users.noreply.github.com> Co-authored-by: Vardhan Vinay Thigle <39047439+VardhanThigle@users.noreply.github.com> * PR Review Comments (#35) * Convert it to builder Pattern * Convert Waring to Error * remove the unwanted comments * Removed Unwanted options * Address the PR * Address the PR Comments * Added Missing Getter for Configuration * Address to removed configuration changes * removed unwanted getters * Removal of getter from cassandra dao and test case fixes * Handle Optionmap in cassandra DAO for serialization * removed dependecy of getKeyOrder * Removed And Update Exception * Fix Checkstyle Voilation * Missing UT Added (#37) * Added Dummy Test case * Removed * from import * removed unwanted * Added Fixes * Added test case and fixes * Added Some more PR comments * Added Test case for Cassandra Reader * Added New testcase * Added Dependecny * Update test case and remove Dependency of Jupiter (#39) * Update test case and remove Dependecy of jupiter * Added UT fixes * Added Missing Getter for Configuration * [Sourcedb-to-spanner] Bulk migration Mysql to spanner 1tb Load test (#2063) * [Sourcedb-to-spanner] Bulk migration Mysql to spanner 1tb Load test * Updating row counts, added static sql resources * Renaming the test and addressing comments * PipelineController Changes for Cassandra (#2086) * Fix Spanner Load tests and add display test report (#2092) * correcting lt failures * Adding test report * Meta code coverage (#41) Meta code coverage * Pr bug fixes (#42) * Schema Reader Optimization (#43) * Added Pr Fix related to Changes Year in Javadoc * Address the revert of 2025 and removal of extra constructor * Added Extra Testcase to reach target patch --------- Co-authored-by: taherkl <taher.lakdawala@ollion.com> Co-authored-by: Taher Lakdawala <78196491+taherkl@users.noreply.github.com> Co-authored-by: pawankashyapollion <v-pawan.kumar@ollion.com> Co-authored-by: labs-code-app[bot] <161369871+labs-code-app[bot]@users.noreply.github.com> Co-authored-by: Vardhan Vinay Thigle <39047439+VardhanThigle@users.noreply.github.com> Co-authored-by: darshan-sj <darshan-sj@users.noreply.github.com>
Dataflow templates