Skip to content

Latest commit

 

History

History
132 lines (89 loc) · 2.57 KB

07-cleanup.md

File metadata and controls

132 lines (89 loc) · 2.57 KB
Raw

About

This module includes the cleanup of resources created for the lab.

1. Declare variables
2. Delete Buckets
3. Delete Spark Persistent History Server
4. Delete BQ Dataset
5. Delete Sessions
5. Delete Managed Notebook

0. Prerequisites

1. GCP Project Details

Note the project number and project ID.
We will need this for the rest fo the lab

2. IAM Roles needed to create Persistent History Server

Grant the following permissions

  • Viewer
  • Dataproc Editor
  • Storage Admin
  • BigQuery DataEditor
  • Composer Administrator
  • Service Usage Admin

3. Attach cloud shell to your project.

Open Cloud shell or navigate to shell.cloud.google.com
Run the below command to set the project in the cloud shell terminal:

gcloud config set project $PROJECT_ID

1. Declare variables

We will use these throughout the lab.
Run the below in cloud shells coped to the project you selected-

PROJECT_ID= #Project ID
REGION= #Region to be used
BUCKET_PHS= #Bucket name for Persistent History Server
BUCKET_CODE=
BQ_DATASET_NAME=
PHS_NAME = Name of your PHS cluster in the dataproc.

2. Delete buckets

Follow the commands to delete the following buckets

  1. Bucket attached to spark history server
  2. Bucket with code files
gcloud alpha storage rm --recursive gs://$BUCKET_PHS
gcloud alpha storage rm --recursive gs://$BUCKET_CODE

3. Delete Spark Persistent History Server

Run the below command to delete Spark PHS

gcloud dataproc clusters delete ${PHS_NAME} \
	--region=${REGION}

4. Delete BQ Dataset

Run the below command to delete BQ dataset and all the tables within the dataset

gcloud alpha bq datasets delete $BQ_DATASET_NAME \
	--remove-tables

5. Delete Sessions

Select the sessions created as part of this lab and click on Delete


6. Delete Managed Notebook

Select the notebook created and click on delete.


7. Delete Artifact Registry

Select your artifact registry and click on delete


8. Delete VM

Select your VM and click on delete