Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding terraform and tests #371

Merged
merged 36 commits into from
Jul 12, 2023
Merged
Changes from 1 commit
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
d8bc049
adding terraform and tests
franklinWhaite Jul 6, 2023
2e14dd1
Merge branch 'master' into master
franklinWhaite Jul 11, 2023
f98521d
Update tools/cloud_functions/bq_table_snapshots/terraform/backend.tf
franklinWhaite Jul 11, 2023
8d9234e
Update tools/cloud_functions/bq_table_snapshots/terraform/main.tf
franklinWhaite Jul 11, 2023
c7a5b54
Update tools/cloud_functions/bq_table_snapshots/terraform/main.tf
franklinWhaite Jul 11, 2023
e8ec829
Update tools/cloud_functions/bq_table_snapshots/terraform/backend.tf
franklinWhaite Jul 11, 2023
f88db34
Update tools/cloud_functions/bq_table_snapshots/tests/bq_backup_fetch…
franklinWhaite Jul 11, 2023
c0a930d
Update tools/cloud_functions/bq_table_snapshots/tests/bq_backup_creat…
franklinWhaite Jul 11, 2023
4eb7585
Update tools/cloud_functions/bq_table_snapshots/terraform/variables.tf
franklinWhaite Jul 11, 2023
8501dfc
Update tools/cloud_functions/bq_table_snapshots/tests/bq_backup_creat…
franklinWhaite Jul 11, 2023
fd48691
Update tools/cloud_functions/bq_table_snapshots/terraform/function.tf
franklinWhaite Jul 11, 2023
856c8a3
Update tools/cloud_functions/bq_table_snapshots/terraform/variables.tf
franklinWhaite Jul 11, 2023
b172b36
Update tools/cloud_functions/bq_table_snapshots/tests/bq_backup_creat…
franklinWhaite Jul 11, 2023
edfd098
Update tools/cloud_functions/bq_table_snapshots/tests/bq_backup_fetch…
franklinWhaite Jul 11, 2023
6525b7a
using .tfvars and adding gcs backend
franklinWhaite Jul 12, 2023
7a5018e
updating readme, using .tfvars, adding gcs backend
franklinWhaite Jul 12, 2023
cf01080
adding descriptions for variables
franklinWhaite Jul 12, 2023
f38219d
removing tf state files
franklinWhaite Jul 12, 2023
b178a40
Update tools/cloud_functions/bq_table_snapshots/terraform/deployment_…
franklinWhaite Jul 12, 2023
3f8f8bc
Update tools/cloud_functions/bq_table_snapshots/terraform/deployment_…
franklinWhaite Jul 12, 2023
ec95133
Update tools/cloud_functions/bq_table_snapshots/README.md
franklinWhaite Jul 12, 2023
18c178c
Update tools/cloud_functions/bq_table_snapshots/README.md
franklinWhaite Jul 12, 2023
67b2671
Update tools/cloud_functions/bq_table_snapshots/terraform/variables.tf
franklinWhaite Jul 12, 2023
086428a
Update tools/cloud_functions/bq_table_snapshots/terraform/variables.tf
franklinWhaite Jul 12, 2023
7d45fda
Update tools/cloud_functions/bq_table_snapshots/terraform/variables.tf
franklinWhaite Jul 12, 2023
fa9bd01
fixing typos
franklinWhaite Jul 12, 2023
f3c5e20
renaiming .tfvars file
franklinWhaite Jul 12, 2023
7308c3c
Update tools/cloud_functions/bq_table_snapshots/terraform/variables.tf
franklinWhaite Jul 12, 2023
042d060
Update tools/cloud_functions/bq_table_snapshots/terraform/variables.tf
franklinWhaite Jul 12, 2023
4be091d
Update tools/cloud_functions/bq_table_snapshots/terraform/variables.tf
franklinWhaite Jul 12, 2023
9c665ae
Update tools/cloud_functions/bq_table_snapshots/terraform/variables.tf
franklinWhaite Jul 12, 2023
afe5e2e
Update tools/cloud_functions/bq_table_snapshots/terraform/variables.tf
franklinWhaite Jul 12, 2023
232f7c0
Update tools/cloud_functions/bq_table_snapshots/terraform/variables.tf
franklinWhaite Jul 12, 2023
2bf70ea
reafactoring readme
franklinWhaite Jul 12, 2023
067b4dc
Update tools/cloud_functions/bq_table_snapshots/terraform/variables.tf
franklinWhaite Jul 12, 2023
243e9d4
Update tools/cloud_functions/bq_table_snapshots/terraform/backend.tf
franklinWhaite Jul 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update tools/cloud_functions/bq_table_snapshots/README.md
Co-authored-by: Daniel De Leo <danieldeleo@users.noreply.github.com>
  • Loading branch information
franklinWhaite and danieldeleo authored Jul 12, 2023
commit 18c178cae5932301f5f08ef3931018a672027538
2 changes: 1 addition & 1 deletion tools/cloud_functions/bq_table_snapshots/README.md
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line 16 replace Scheduloer with Scheduler

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line 35 has typo: ame
Fix: name

Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ The following environemnt variables must be set:
* `TABLE_NAME_PUBSUB_TOPIC_ID` ame of P/S topic where this code will publish to

## bq_backup_create_snapshots
The **bq_backup_create_snapshots** cloud function will submit a BigQuery job to create a snapshot for each table in scope. This cloud function will suffix the snapshot name with the snapshot datetime to guarantee a unique name. It will also calculate and set the expiration time of the snapshot based on seconds_before_expiration. Finally, it will determine the snapshot time based on crontab_format.
The **bq_backup_create_snapshots** Cloud Function will submit a BigQuery job to create a snapshot for each table in scope. This Cloud Function will suffix the snapshot name with the snapshot datetime to guarantee a unique name. It will also calculate and set the expiration time of the snapshot based on seconds_before_expiration. Finally, it will determine the snapshot time based on crontab_format.
The following environemnt variables must be set:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reword this and line 32 as well since it instructs the user to set an environment variable but the Terraform will actually do this.

The following could be an appropriate rewording:
"The Cloud Function uses the following environment variables to determine which BigQuery project to use for compute and which to use for storing your BigQuery snapshots:"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also lines 32 & 39 have typo: environemnt
Fix: environment

* `BQ_DATA_PROJECT_ID` id of project used for BQ storage
* `BQ_JOBS_PROJECT_ID` id of project used for BQ compute
Expand Down