Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

An example illustrating Dataflow Custom templates provisioned within a terraform context #873

Merged
merged 5 commits into from
Aug 23, 2022
Merged

An example illustrating Dataflow Custom templates provisioned within a terraform context #873

merged 5 commits into from
Aug 23, 2022

Conversation

damondouglas
Copy link
Contributor

@damondouglas damondouglas commented Aug 16, 2022 •
edited
Loading

This pull request provides an example for building Dataflow Custom templates within an Infrastructure-as-Code context. To complete this example this PR includes the following:

  • an Open in Cloud Shell step-by-step walk-through tutorial
  • terraform module to provision a Cloud Build trigger that builds the Dataflow Custom templates
  • terraform modules to provision worker service accounts, IAM permissions, and a custom network
  • Google Cloud storage bucket and objects to run the word count example pipeline

Co-authored-by: Timothy Itodo titodo@google.com

damondouglas and others added 3 commits August 15, 2022 22:20
@damondouglas @itodotimothy6
Create Custom Dataflow Template example
db44e51 
Co-authored-by: Timothy Itodo <titodo@google.com>
@damondouglas @itodotimothy6
Patch README,variables
d2e4f2e 
Co-authored-by: Timothy Itodo <titodo@google.com>
@damondouglas @itodotimothy6
Patch additional variables and build configuration
7fcc3c7 
Co-authored-by: Timothy Itodo <titodo@google.com>
@pull-request-size pull-request-size bot added the size/XXL Denotes a PR that changes 1000+ lines. label Aug 16, 2022
@prathapreddy123 prathapreddy123 self-requested a review August 22, 2022 18:11
prathapreddy123
prathapreddy123 approved these changes Aug 22, 2022
View reviewed changes
Copy link
Contributor

@prathapreddy123 prathapreddy123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great and thanks for contribution. Left few minor comments

damondouglas and others added 2 commits August 22, 2022 18:38
@prathapreddy123 prathapreddy123 merged commit 4d10ac8 into GoogleCloudPlatform:main Aug 23, 2022
@damondouglas damondouglas deleted the dataflow-custom-templates branch October 6, 2022 20:46
SMaktub pushed a commit to SMaktub/professional-services that referenced this pull request Dec 13, 2022
@damondouglas @itodotimothy6 @SMaktub
An example illustrating Dataflow Custom templates provisioned within …
968c16e 
…a terraform context (GoogleCloudPlatform#873)

* Create Custom Dataflow Template example

Co-authored-by: Timothy Itodo <titodo@google.com>

* Patch README,variables

Co-authored-by: Timothy Itodo <titodo@google.com>

* Patch additional variables and build configuration

Co-authored-by: Timothy Itodo <titodo@google.com>

* Refactor per PR review

Co-authored-by: Timothy Itodo <titodo@google.com>

Co-authored-by: Timothy Itodo <titodo@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@prathapreddy123 prathapreddy123 prathapreddy123 approved these changes

Assignees
No one assigned
Labels
size/XXL Denotes a PR that changes 1000+ lines.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@damondouglas @prathapreddy123