Skip to content

Satyaraj28/GCP-1

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

The Google Cloud Developer's Cheat Sheet

Google Cloud Developer's Cheat Sheet Poster Image
White background: Poster PDF | Brochure PDF | High-resolution PNG | Medium-res PNG | Low-res PNG
Dark background: Poster PDF | Brochure PDF | High-resolution PNG | Medium-res PNG | Low-res PNG

Desktop Wallpapers: 16:9 (iMac, etc.) | 16:10 (Macbook Pro) | 4:3 (Older computers) | 3:2 (Pixelbook)

Text descriptions with links are below

Suggestions for better descriptions? Missing products? Pull requests are welcome on README.md (this file).

Feedback? @aslamlatheef


All Products

🔗-Product page 📄-Documentation

Compute

  • App Engine: Managed app platform 🔗 📄
  • Cloud Functions: Event-driven serverless functions 🔗 📄
  • Cloud Run 🆕: Serverless for containerized applications 🔗 📄
  • Compute Engine: VMs, GPUs, TPUs, Disks 🔗 📄
  • Kubernetes Engine (GKE): Managed Kubernetes/containers 🔗 📄
  • Anthos 🆕: Enterprise hybrid/multi-cloud platform 🔗 📄

Storage

  • Cloud Storage: Object storage and serving 🔗 📄
  • Nearline: Archival occasional access storage 🔗 📄
  • Coldline: Archival rare access storage 🔗 📄
  • Persistent Disk: VM-attached disks 🔗 📄
  • Cloud Filestore: Managed NFS server 🔗 📄

Database

  • Cloud Bigtable: Petabyte-scale, low-latency, non-relational 🔗 📄
  • Cloud Datastore: Horizontally scalable document DB 🔗 📄
  • Cloud Firestore: Strongly-consistent serverless document DB 🔗 📄
  • Cloud Memorystore: Managed Redis 🔗 📄
  • Cloud Spanner: Horizontally scalable relational DB 🔗 📄
  • Cloud SQL: Managed MySQL and PostgreSQL 🔗 📄

Data and Analytics

  • BigQuery: Data warehouse/analytics 🔗 📄
  • BigQuery BI Engine 🆕: In-memory analytics engine 📄
  • BigQuery ML 🆕: BigQuery model training/serving 📄
  • Cloud Composer: Managed workflow orchestration service 🔗 📄
  • Cloud Data Fusion 🆕: Graphically manage data pipelines 🔗 📄
  • Cloud Dataflow: Stream/batch data processing 🔗 📄
  • Cloud Datalab: Managed Jupyter notebook 🔗 📄
  • Cloud Dataprep: Visual data wrangling 🔗 📄
  • Cloud Dataproc: Managed Spark and Hadoop 🔗 📄
  • Cloud Pub/Sub: Global real-time messaging 🔗 📄
  • Data Catalog 🆕: Metadata management service 🔗 📄
  • Data Studio: Collaborative data exploration/dashboarding 🔗 📄
  • Genomics: Managed genomics platform 🔗 📄

AI/ML

  • AI Hub 🆕: Hosted AI component sharing 🔗 📄
  • AI Platform: Managed platform for ML 🔗 📄
  • AI Platform Data Labeling 🆕: Data labeling by humans 📄
  • AI Platform Deep Learning VMs: Preconfigured VMs for deep learning 🔗 📄
  • AI Platform Notebooks 🆕: Managed JupyterLab notebook instances 🔗📄
  • AI Platform Training 🆕: Parallel and distributed training 📄
  • AI Platform Predictions 🆕: Autoscaled model serving 📄
  • AutoML Natural Language: Custom text models 📄
  • AutoML Tables 🆕: Custom structured data models 🔗 📄
  • AutoML Translation: Custom domain-specific translation 🔗 📄
  • AutoML Video Intelligence 🆕: Custom video annotation models 🔗 📄
  • AutoML Vision: Custom image models 🔗 📄
  • Cloud AI Building Blocks 🆕: Hosted AI component repository 🔗 📄
  • Cloud Natural Language API: Text parsing and analysis 🔗 📄
  • Cloud Speech-To-Text API: Convert audio to text 🔗 📄
  • Cloud Talent Solutions API: Job search with ML 🔗 📄
  • Cloud Text-To-Speech API: Convert text to audio 🔗 📄
  • Cloud Translation API: Language detection and translation 🔗 📄
  • Cloud Video Intelligence API: Scene-level video annotation 🔗 📄
  • Cloud Vision API: Image recognition and classification 🔗 📄
  • Cloud TPU: Hardware acceleration for ML 🔗 📄
  • Dialogflow Enterprise Edition: Create conversational interfaces 🔗 📄
  • Document Understanding AI 🆕: Analyze, classify, search documents 🔗 📄
  • Recommendations AI 🆕: Create custom recommendations 🔗 📄
  • Vision Product Search 🆕: Visual search for products 📄

Networking

  • Carrier Peering: Peer through a carrier 📄
  • Direct Peering: Peer with GCP 📄
  • Dedicated Interconnect: Dedicated private network connection 📄
  • Partner Interconnect: Connect on-prem network to VPC 📄
  • Cloud Armor: DDoS protection and WAF 🔗 📄
  • Cloud CDN: Content delivery network 🔗 📄
  • Cloud DNS: Programmable DNS serving 🔗 📄
  • Cloud Load Balancing: Multi-region load distribution 🔗 📄
  • Cloud NAT: Network address translation service 📄
  • Cloud Router 🆕: VPC/on-prem network route exchange (BGP) 📄
  • IPsec VPN: Virtual private network connection 📄
  • Network Service Tiers: Price vs performance tiering 🔗 📄
  • Network Telemetry: Network telemetry service 🔗 📄
  • Traffic Director 🆕: Service mesh traffic management 🔗 📄
  • Google Cloud Service Mesh 🆕: Service-aware network management 🔗 📄
  • Virtual Private Cloud: Software defined networking 🔗 📄

Internet of Things (IoT)

  • Cloud IoT Core: Device management and ingest data 🔗 📄

Identity and Security

  • Access Transparency: Audit cloud provider access 🔗 📄
  • Binary Authorization: Kubernetes deploy-time security 🔗 📄
  • Cloud Audit Logs 🆕: Audit trails for GCP 🔗 📄
  • Cloud Data Loss Prevention API: Classify and redact sensitive data 🔗 📄
  • Cloud HSM: Hardware security module service 🔗 📄
  • Cloud IAM: Resource access control 🔗 📄
  • Cloud Identity: Manage users, devices & apps 🔗 📄
  • Cloud Identity-Aware Proxy: Identity-based app sign in 🔗 📄
  • Cloud Key Management Service: Hosted key management service 🔗 📄
  • Cloud Resource Manager: Cloud project metadata management 🔗 📄
  • Cloud Security Scanner: App engine security scanner 🔗 📄
  • Cloud Security Command Center: Asset inventory, discovery, search, management 🔗 📄
  • Context-aware Access 🆕: End-user attribute-based access control 🔗 📄
  • Event Threat Detection 🆕: Scans for suspicious activity 🔗
  • Managed Service for Microsoft Active Directory 🆕: Managed Microsoft Active Directory 🔗 📄
  • Security Key Enforcement: Two-step key verification 🔗
  • Shielded VMs: Hardened VMs 🔗 📄
  • Titan Security Key: Two-factor authentication (2FA) device 🔗
  • VPC Service Controls 🆕: VPC constrain data 🔗 📄

Management Tools

  • Cloud APIs: APIs for cloud services 🔗 📄
  • Cloud Billing: Billing and cost management tools 🔗 📄
  • Cloud Billing API: Programmatically manage GCP billing 📄
  • Cloud Console: Web-based management console 🔗
  • Cloud Deployment Manager: Templated infrastructure deployment 🔗 📄
  • Cloud Mobile App: iOS/Android GCP manager app 🔗
  • Cloud Shell: Browser-based terminal/CLI 🔗 📄
  • Stackdriver Debugger: Live production debugging 🔗 📄
  • Stackdriver Error Reporting: App error reporting 🔗 📄
  • Stackdriver Logging: Centralized logging 🔗 📄
  • Stackdriver Monitoring: Infrastructure and application monitoring 🔗 📄
  • Stackdriver Profiler: CPU and heap profiling 🔗 📄
  • Stackdriver Transparent SLIs: Monitor GCP services 🔗 📄
  • Stackdriver Trace: App performance insights 🔗 📄

Developer Tools

  • Cloud SDK: CLI for GCP 🔗 📄
  • Cloud Build: Continuous integration/delivery platform 🔗 📄
  • Cloud Code 🆕: Cloud native IDE extensions 🔗 📄
  • Cloud Source Repositories: Hosted private git repos 🔗 📄
  • Cloud Scheduler: Managed cron job service 🔗 📄
  • Cloud Tasks 🆕: Asynchronous task execution 🔗 📄
  • Cloud Tools for IntelliJ: IntelliJ GCP tools 🔗
  • Cloud Tools for PowerShell: PowerShell GCP tools 🔗
  • Cloud Tools for Visual Studio: Visual Studio GCP tools 🔗
  • Cloud Tools for Eclipse: Eclipse GCP tools 🔗
  • Container Registry: Private container registry/storage 🔗 📄
  • Gradle App Engine Plugin: Gradle App Engine plugin 🔗
  • Maven App Engine Plugin: Maven App Engine plugin 🔗

Migration to GCP

  • Cloud Data Transfer: Data migration tools/CLI 🔗
  • Google Transfer Appliance: Rentable data transport box 🔗 📄
  • Cloud Storage Transfer Service: Cloud to cloud transfers 📄
  • BigQuery Data Transfer Service: Bulk import analytics data 🔗 📄
  • Migrate from Amazon Redshift 🆕: Migrate from Redshift to BigQuery 🔗
  • Migrate from Teradata 🆕: Migrate from Teradata to BigQuery 🔗
  • Migrate for Anthos 🆕: Migrate VMs to GKE containers 🔗 📄
  • Migrate for Compute Engine: Compute Engine migration tools 🔗 📄
  • VM Migration: VM migration tools 🔗

API Platform and Ecosystems

  • API Analytics: API metrics 🔗
  • API Monetization: Monetize APIs 🔗
  • Apigee API Platform: Develop, secure, monitor APIs 🔗
  • Apigee Sense: API protection from attacks 🔗
  • Apigee Hybrid 🆕: Manage hybrid/multi-cloud API environments 🔗📄
  • Cloud Endpoints: Cloud API gateway 🔗
  • Cloud Healthcare API: Healthcare system GCP interoperability 🔗
  • Developer Portal: API management portal 🔗
  • GCP Marketplace: Partner & open source marketplace 🔗

Google Maps Platform

  • Directions API: Get directions between locations 🔗
  • Distance Matrix API: Calculate travel times 🔗
  • Geocoding API: Convert address to/from coordinates 🔗
  • Geolocation API: Derive location without GPS 🔗
  • Maps Embed API: Web embedded maps 🔗
  • Maps JavaScript API: Dynamic web maps 🔗
  • Maps SDK for Android: Maps SDK for Android 🔗
  • Maps SDK for iOS: Maps SDK for iOS 🔗
  • Maps Static API: Web static maps 🔗
  • Maps Unity SDK: Unity SDK for games 🔗
  • Maps URLs: URL scheme for maps 🔗
  • Places API: Metadata about places (REST) 🔗
  • Places Library, Maps JS API: Metadata about places (JavaScript) 🔗
  • Places SDK for Android: Places SDK for Android 🔗
  • Places SDK for iOS: Places SDK for iOS 🔗
  • Roads API: Metadata about roads 🔗
  • Street View Static API: Static street view images 🔗
  • Street View Service: Interactive street view images" 🔗
  • Time Zone API: Convert coordinates to timezone 🔗

G Suite Platform

  • App Maker: Assistive app building 🔗
  • Apps Script: Extend and automate everything 🔗
  • Editor Add-ons: Extend Docs, Sheets, Slides 🔗
  • Gmail Add-ons: Contextual apps in Gmail 🔗
  • Hangouts Chat Bots: Conversational bots in chat 🔗
  • Calendar API: Create and manage calendars 🔗
  • Classroom API: Provision and manage classrooms 🔗
  • Docs API: Create and edit documents 🔗
  • Drive API: Read and write files 🔗
  • Gmail API: Enhance Gmail 🔗
  • Sheets API: Read and write spreadsheets 🔗
  • Slides API: Create and edit presentations 🔗
  • Drive Picker: Drive file selection widget 🔗
  • Cloud Search: Unified search for enterprise 🔗
  • Admin SDK: Manage G Suite resources 🔗
  • Email Markup: Interactive email using schema.org 🔗
  • G Suite Marketplace: Storefront for integrated applications 🔗
  • Other G Suite APIs/SDKs: Contacts, Google+, Tasks, Vault... 🔗

Mobile (Firebase)

  • Cloud Firestore: Document store and sync 🔗
  • Cloud Functions for Firebase: Event-driven serverless applications 🔗
  • Cloud Storage for Firebase: Object storage and serving 🔗
  • Crashlytics: Crash reporting and analytics 🔗
  • Firebase A/B Testing: Create A/B test experiments 🔗
  • Firebase App Indexing: App/Google search integration 🔗
  • Firebase Authentication: Drop-in authentication 🔗
  • Firebase Cloud Messaging: Send device notifications 🔗
  • Firebase Dynamic Links: Link to app content 🔗
  • Firebase Hosting: Web hosting with CDN/SSL 🔗
  • Firebase In-App Messaging: Send in-app contextual messages 🔗
  • Firebase Performance Monitoring: App performance monitoring 🔗
  • Firebase Predictions: Predict user targeting 🔗
  • Firebase Realtime Database: Real-time data synchronization 🔗
  • Firebase Remote Config: Remotely configure installed apps 🔗
  • Firebase Test Lab: Mobile testing device farm 🔗
  • Google Analytics for Firebase: Mobile app analytics 🔗
  • ML Kit for Firebase: ML APIs for mobile 🔗

GCP Foundational Open Source Projects

  • Apache Beam: Batch/streaming data processing 🔗
  • gRPC: RPC framework 🔗
  • gVisor: Secure container runtime 🔗
  • Istio: Connect and secure services 🔗
  • Knative: Serverless framework for Kubernetes 🔗
  • Kubeflow: ML toolkit for Kubernetes 🔗
  • Kubernetes: Management of containerized applications 🔗
  • OpenCensus: Cloud native observability framework 🔗
  • TensorFlow: ML framework 🔗

Additional Resources

  • Google Cloud Home Page: 🔗
  • Google Cloud Blog: 🔗
  • GCP Medium Publication: 🔗
  • Apigee Blog: 🔗
  • Firebase Blog: 🔗
  • G Suite Developers Blog: 🔗
  • Google Cloud Certifications: 🔗
  • Google Cloud System Status: 🔗
  • Google Cloud Training: 🔗
  • Google Developers Blog: 🔗
  • Google Maps Platform Blog: 🔗
  • Google Open Source Blog: 🔗
  • Google Security Blog: 🔗
  • Kaggle Home Page: 🔗
  • Kubernetes Blog: 🔗
  • Regions and Network Map: 🔗

Everything below is not included in the PDFs/PNGs due to space limitations


Additional Resources

  • Google Cloud Solutions Library:🔗
  • Google Cloud Support Hub:🔗
  • GCP Pricing: 🔗
  • GCP Pricing Calculator: 🔗
  • Qwiklabs Home Page:🔗
  • Codelabs Home Page:🔗
  • YouTube Channels:
    • Google Cloud YouTube Channel:🔗
    • GCP YouTube Channel:🔗
    • Google Developer's YouTube Channel:🔗
    • Firebase YouTube Channel:🔗
  • Podcasts:
    • GCP Podcast:🔗
    • Kubernetes Podcast:🔗
  • Reddit:

Platform Comparisons

  • Google Cloud Platform for AWS Professionals:📄
  • Google Cloud Platform for Azure Professionals:📄
  • Google Cloud Platform for Data Center Professionals:📄
  • Google Cloud Platform for OpenStack Users:📄

Language Specific Documentation

GKE

  Set default region/zone

  gcloud config set compute/region us-central1

  gcloud config set compute/zone us-central1-a

Enable API's

  gcloud services enable container.googleapis.com

  gcloud services enable containerregistry.googleapis.com

Clone github for sample application and browse to directory

git clone https://github.com/linuxacademy/content-gc-essentials

cd content-gc-essentials/gke-lab-01

Package code into Docker container - tag as version 1 ("build it")

docker build -t gcr.io/$DEVSHELL_PROJECT_ID/hello-la:v1 .

Note: the period at the end is required to build it at this location

Check status of images to ensure success

docker images

Authenticate gcloud as a Docker credential helper

gcloud auth configure-docker

Push Docker container into Container Registry

docker push gcr.io/$DEVSHELL_PROJECT_ID/hello-la:v1

Create Kubernetes Engine Cluster (Nodes) named 'hello-cluster'

gcloud container clusters create hello-cluster --num-nodes=2

Authenticate kubectl to point to the cluster we just made (already done for last created cluster)

  gcloud container clusters get-credentials hello-cluster

  Deploy your app ("deploy and run it") - listen on port 80

  kubectl run hello-la --image=gcr.io/$DEVSHELL_PROJECT_ID/hello-la:v1 --port 80

Check out our pods on the nodes

  kubectl get pods

  Create load balancer and expose application to the Internet on port 80

  kubectl expose deployment hello-la --type=LoadBalancer --port 80 --target-port 80

Find our load balancer frontend IP address

  kubectl get service

Scale up deployment - add static number of replicas (pods)

  kubectl scale deployment hello-la --replicas=3

On second thought, let's just autoscale our application instead....

Horizontal Pod Autoscaler

  kubectl autoscale deployment hello-la --max 6 --min 4 --cpu-percent 50

Maybe we should statically resize the node pool/cluster as well?

  gcloud container clusters resize hello-cluster --size 3

If more than one pool per cluster, specify pool with --node-pool (pool_name)

On second thought (again), let's also enable autoscaling for our cluster

  gcloud container clusters update hello-cluster --enable-autoscaling --min-nodes 2 --max-nodes 8

Make changes to source code, then build as Docker file as VERSION 2

  docker build -t gcr.io/$DEVSHELL_PROJECT_ID/hello-la:v2 .

Push to Container Registry, also as version 2

  docker push gcr.io/$DEVSHELL_PROJECT_ID/hello-la:v2

Update our website - Apply rolling update to deployment with image update

  kubectl set image deployment/hello-la hello-la=gcr.io/$DEVSHELL_PROJECT_ID/hello-la:v2

Get log info

Logs are written to pods, by default also written to Stackdriver Logging

View log on pod

  kubectl logs (POD_ID)

Other commands/scenarios

Upgrade version of Kubernetes on cluster

  gcloud containers clusters upgrade (cluster_name)

BILLING >>>> Storing on BIGQUERY

You can capture and save your date on S3 and also Bigquery where you can also do some analytics if you want.

CAPTURE BILLING DATA FROM BIGQUERY and Few more examples for Billing Report.

SELECT *  
FROM `cloud-training-prod-bucket.arch_infra.billing_data`

Highest cost First

SELECT *  
FROM `cloud-training-prod-bucket.arch_infra.billing_data`
ORDER BY cost DESC

Specific COST

 SELECT product, resource_type, start_time, end_time,  
 cost, project_id, project_name, project_labels_key, currency, currency_conversion_rate,
 usage_amount, usage_unit
 FROM `cloud-training-prod-bucket.arch_infra.billing_data`
 WHERE (cost > 3)  

note here "3" is cost

Next let’s find which product had the highest total number of records:

SELECT product, COUNT(*)
FROM `cloud-training-prod-bucket.arch_infra.billing_data`
GROUP BY product
LIMIT 200

let’s see which product most frequently cost more than a dollar:

SELECT product, cost, COUNT(*)
FROM `cloud-training-prod-bucket.arch_infra.billing_data`
WHERE (cost > 1)
GROUP BY cost, product
LIMIT 200

CLOUD STORAGE >>>>>>>>>>>>>>>>>>>

Assiging Access

Assign IAM roles to buckets:

    gsutil iam ch user:(user_email):(role1,role2) gs://(BUCKET)

Remove IAM role from bucket:

    gsutil iam ch -d user:(user_email):(role1,role2) gs://(BUCKET)

Remove all roles from bucket for given user:

    gsutil iam ch -d user:(user_email) gs://(BUCKET)

Assign ACL roles to buckets and objects:

    gsutil acl ch -u (user_email):(O/R/W) gs://(BUCKET)

Delete all ACLs:

    gsutil acl ch -d (user_email) gs://(BUCKET)

Signed URLs

Create service account with key

    Upload to cloud shell (or add to current CLI environment)
    gsutil signurl -d (time_period (10m)) (keyfile.json) gs://(BUCKET)/(object)

Check current versioning policy:

    gsutil versioning get gs://<BUCKET>

Enable Object Versioning:

    gsutil versioning set on gs://<BUCKET>

Check full object details in bucket:

    gsutil ls -a gs://<BUCKET>

Download current lifecycle policy to local machine to edit:

    gsutil lifecycle get gs://<BUCKET> > filename.json

Set new lifecycle policy after making above edits:

    gsutil lifecycle set filename.json gs://<BUCKET>

DISK MANAGEMENT >>>>>>>>>>>

Create disk:

gcloud compute disks create <DISK_NAME> --type=<DISK_TYPE> --size=<SIZE> --zone=<ZONE>

Resize disk:

gcloud compute disks resize <disk_name> --size=<size> --zone=<zone>

Attach disk:

gcloud compute instances attach-disk <instance> --disk=<disk_name> --zone=<zone>

Formatting and Mounting DISK for LINUX in GCP >>>>>>

View available disks:

sudo lsblk

Format attached disk:

sudo mkfs.ext4 -m 0 -F -E lazy_itable_init=0,lazy_journal_init=0,discard /dev/sdb

Create mount directory:

sudo mkdir -p /mnt/disks/disk2

Mount disk:

sudo mount -o discard,defaults /dev/sdb /mnt/disks/disk2

Set read/write permissions:

sudo chmod a+w /mnt/disks/
Resize existing Linux disk

Identify the disk to resize:

sudo lsblk

Resize (grow) the partition:

sudo growpart /dev/sda 1

Extend file system to use added space:

sudo resize2fs /dev/sda1

Verify file system is resized:

df -h 

VIRTUAL MACHINE IMAGE MANAGEMENT ->>>>>>>>>

Create image via command line:

gcloud compute images create <image_name> --source-disk <disk_name> --source-disk-zone <zone> --family <image_family>

View image family info:

gcloud compute images describe-from-family <image_family>

Deprecate/set active image version:

gcloud compute images deprecate <image_name> --state <STATE>

Delete image:

gcloud compute images delete <image_name>

STACK DRIVER >>>>>>>>>>>>>>>>>>>>

About

Google Clood Cook BOOK

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published