Skip to content
This repository has been archived by the owner on Nov 14, 2024. It is now read-only.

Latest commit

 

History

History
405 lines (327 loc) · 22.5 KB

README.md

File metadata and controls

405 lines (327 loc) · 22.5 KB
Raw

InSpec GCP (Google Cloud Platform) Resource Pack

  • Project State: Maintained

For more information on project states and SLAs, see this documentation.

This InSpec resource pack uses the native Google Cloud Platform (GCP) support in InSpec and provides the required resources to write tests for GCP.

This implementation was inspired on the ideas by Martez Reed.

Prerequisites

Install and configure the Google cloud SDK*

Download the SDK and run the installation:

./google-cloud-sdk/install.sh

Create credentials file via:

$ gcloud auth application-default login

If successful, this should be similar to:

$ cat ~/.config/gcloud/application_default_credentials.json
{
  "client_id": "764086051850-6qr4p6gpi6hn50asdr.apps.googleusercontent.com",
  "client_secret": "d-fasdfasdfasdfaweroi23jknrmfs;f8sh",
  "refresh_token": "1/asdfjlklwna;ldkna'dfmk-lCkju3-yQmjr20xVZonrfkE48L",
  "type": "authorized_user"
}

While InSpec can use user accounts for authentication, Google Cloud documentation recommends using service accounts.

The json credential file for a service account looks like this:

$ cat /Users/john/.config/gcloud/myproject-1-feb7993e8660.json
{
  "type": "service_account",
  "project_id": "myproject-1",
  "private_key_id": "eb45b2fc0c33ea9b6fa212aaa08b1ed814bf8660",
  "private_key": "-----BEGIN PRIVATE KEY-----\nMIIEvwIBADAN3662...fke9n6LAf268E/4EWhIzg==\n-----END PRIVATE KEY-----\n",
  "client_email": "auto-testing@myproject-1.iam.gserviceaccount.com",
  "client_id": "112144174133171863632",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/auto-testing%40myproject-1.iam.gserviceaccount.com"
}

And InSpec can be instructed to use it by setting this ENV variable prior to running inspec exec:

$ export GOOGLE_APPLICATION_CREDENTIALS='/Users/john/.config/gcloud/myproject-1-feb7993e8660.json'

Enable the appropriate APIs that you want to use:

Use the resources

Since this is an InSpec resource pack, it only defines InSpec resources. It includes example tests only. To easily use the GCP resources in your tests do the following:

Create a new profile for GCP

$ inspec init profile --platform gcp my-profile
Create new profile at /Users/spaterson/my-profile
 * Create directory libraries
 * Create file README.md
 * Create directory controls
 * Create file controls/example.rb
 * Create file inspec.yml
 * Create file attributes.yml
 * Create file libraries/.gitkeep

Update attributes.yml to point to your project

gcp_project_id: 'my-gcp-project'

Run the tests

$ cd my-profile/
$ inspec exec . -t gcp:// --attrs attributes.yml

Profile: GCP InSpec Profile (my-profile)
Version: 0.1.0
Target:  gcp://local-service-account@my-gcp-project.iam.gserviceaccount.com

  ✔  gcp-single-region-1.0: Ensure single region has the correct properties.
     ✔  Region europe-west2 zone_names should include "europe-west2-a"
  ✔  gcp-regions-loop-1.0: Ensure regions have the correct properties in bulk.
     ✔  Region asia-east1 should be up
     ✔  Region asia-northeast1 should be up
     ✔  Region asia-south1 should be up
     ✔  Region asia-southeast1 should be up
     ✔  Region australia-southeast1 should be up
     ✔  Region europe-north1 should be up
     ✔  Region europe-west1 should be up
     ✔  Region europe-west2 should be up
     ✔  Region europe-west3 should be up
     ✔  Region europe-west4 should be up
     ✔  Region northamerica-northeast1 should be up
     ✔  Region southamerica-east1 should be up
     ✔  Region us-central1 should be up
     ✔  Region us-east1 should be up
     ✔  Region us-east4 should be up
     ✔  Region us-west1 should be up
     ✔  Region us-west2 should be up


Profile: Google Cloud Platform Resource Pack (inspec-gcp)
Version: 0.5.0
Target:  gcp://local-service-account@my-gcp-project.iam.gserviceaccount.com

     No tests executed.

Profile Summary: 2 successful controls, 0 control failures, 0 controls skipped
Test Summary: 18 successful, 0 failures, 0 skipped

The generated inspec.yml file automatically points to the InSpec GCP resource pack:

name: my-profile
title: My GCP InSpec Profile
version: 0.1.0
inspec_version: '>= 4.6.9'
depends:
  - name: inspec-gcp
    url: https://github.com/inspec/inspec-gcp/archive/x.tar.gz
supports:
  - platform: gcp

(For available inspec-gcp versions, see this list of inspec-gcp versions.)

Resource documentation

The following resources are available in the InSpec GCP Profile

InSpec GCP Supported Resources https://www.inspec.io/docs/reference/resources/#gcp-resources
google_bigquery_dataset google_bigquery_datasets
google_bigquery_table google_bigquery_tables
google_cloudbuild_trigger google_cloudbuild_triggers
google_cloudfunctions_cloud_function google_cloudfunctions_cloud_functions
google_compute_address google_compute_autoscaler
google_compute_autoscalers google_compute_backend_service
google_compute_backend_services google_compute_disk
google_compute_disks google_compute_firewall
google_compute_firewalls google_compute_forwarding_rule
google_compute_forwarding_rules google_compute_global_address
google_compute_global_addresses google_compute_global_forwarding_rule
google_compute_global_forwarding_rules google_compute_health_check
google_compute_health_checks google_compute_http_health_check
google_compute_http_health_checks google_compute_https_health_check
google_compute_https_health_checks google_compute_image
google_compute_instance google_compute_instance_group
google_compute_instance_group_manager google_compute_instance_group_managers
google_compute_instance_groups google_compute_instance_template
google_compute_instance_templates google_compute_instances
google_compute_network google_compute_networks
google_compute_project_info google_compute_region
google_compute_region_instance_group_manager google_compute_region_instance_group_managers
google_compute_regional_disk google_compute_regions
google_compute_route google_compute_router
google_compute_routers google_compute_routes
google_compute_snapshot google_compute_snapshots
google_compute_ssl_certificate google_compute_ssl_certificates
google_compute_ssl_policies google_compute_ssl_policy
google_compute_subnetwork google_compute_subnetworks
google_compute_target_http_proxies google_compute_target_http_proxy
google_compute_target_https_proxies google_compute_target_https_proxy
google_compute_target_pool google_compute_target_pools
google_compute_target_tcp_proxies google_compute_target_tcp_proxy
google_compute_url_map google_compute_url_maps
google_compute_vpn_tunnel google_compute_vpn_tunnels
google_compute_zone google_compute_zones
google_container_cluster google_container_clusters
google_container_node_pool google_container_node_pools
google_container_regional_cluster google_container_regional_clusters
google_dns_managed_zone google_dns_managed_zones
google_dns_resource_record_set google_dns_resource_record_sets
google_kms_crypto_key google_kms_crypto_key_iam_binding
google_kms_crypto_key_iam_bindings google_kms_crypto_keys
google_kms_key_ring google_kms_key_ring_iam_binding
google_kms_key_ring_iam_bindings google_kms_key_rings
google_logging_project_exclusion google_logging_project_sink
google_logging_project_sinks google_organization
google_organizations google_project
google_project_alert_policies google_project_alert_policy
google_project_alert_policy_condition google_project_iam_binding
google_project_iam_bindings google_project_iam_custom_role
google_project_logging_audit_config google_project_metric
google_project_metrics google_projects
google_pubsub_subscription google_pubsub_subscriptions
google_pubsub_topic google_pubsub_topics
google_service_account google_service_account_key
google_service_account_keys google_service_accounts
google_sourcerepo_repositories google_sourcerepo_repository
google_sql_database_instance google_sql_database_instances
google_sql_users google_storage_bucket
google_storage_bucket_acl google_storage_bucket_iam_binding
google_storage_bucket_iam_bindings google_storage_bucket_object
google_storage_bucket_objects google_storage_buckets
google_storage_default_object_acl google_storage_object_acl
google_user google_users

Examples

Check SSH is Disabled for INGRESS across all GCP Projects

We use several plural resources for this example that loops across all projects and firewall rules. Making use of a plural resource property, we filter firewall rules for direction 'INGRESS' :

title 'Loop over all GCP projects and look at firewalls in INGRESS direction'

control 'gcp-projects-firewalls-loop-1.0' do

  impact 1.0
  title 'Ensure INGRESS firewalls in all projects have the correct properties using google_compute_firewall for detail.'

  google_projects.project_names.each do |project_name|
    google_compute_firewalls(project: project_name).where(firewall_direction: 'INGRESS').firewall_names.each do |firewall_name|
      describe google_compute_firewall(project: project_name, name: firewall_name) do
        its('allowed_ssh?')  { should be false }
      end
    end
  end
end

This example assumes there are sufficient privileges to list all GCP projects.

Check that a particular label exists on all VMs across all projects and zones

This check ensures that VMs have label must_be_there for each project:

title 'Loop over all GCP projects and ensure all VMs have a particular label'

control 'gcp-projects-zones-vm-label-loop-1.0' do

  impact 1.0
  title 'Ensure all VMs have must_be_there label key set'

  google_projects.project_names.each do |project_name|
    google_compute_zones(project: project_name).zone_names.each do |zone_name|
      google_compute_instances(project: project_name, zone: zone_name).instance_names.each do |instance_name|
        describe google_compute_instance(project: project_name, zone: zone_name, name: instance_name) do
          its('labels_keys') { should include 'must_be_there' }
        end
      end
    end
  end
end

This example assumes there are sufficient privileges to list all GCP projects.

Test inspec-gcp resources

  1. Create a new GCP project
  2. Ensure this is currently set following: https://cloud.google.com/shell/docs/examples
$ gcloud config set project <project-name>
$ gcloud config list project
  1. Enable billing for your new project
  2. Enable various services necessary to run the tests:
$ gcloud services enable compute.googleapis.com \
    sourcerepo.googleapis.com \
    dns.googleapis.com \
    container.googleapis.com \
    cloudkms.googleapis.com \
    cloudbuild.googleapis.com
  1. Environment variables can be used to specify project details e.g.
export GCP_PROJECT_NAME=<project-name>
export GCP_PROJECT_NUMBER=<project-number>
export GCP_PROJECT_ID=<project-id>

Some resources require elevated privileges to create in GCP. These are disabled by default but can be activated via:

export GCP_ENABLE_PRIVILEGED_RESOURCES=1

This takes effect during the "plan" task as described in the next section. Affected terraform resources are included/excluded and associated inspec tests enabled/disabled accordingly.

The resource pack defaults to region europe-west2. If a different region is desired e.g. us-central1, the following variables should be set:

export GCP_LOCATION="us-central1"
export GCP_ZONE="us-central1-a"
export GCP_LB_REGION="us-central1"
export GCP_LB_ZONE="us-central1-a"
export GCP_LB_ZONE_MIG2="us-central1-b"
export GCP_LB_ZONE_MIG3="us-central1-c"
export GCP_KUBE_CLUSTER_ZONE="us-central1-a"
export GCP_KUBE_CLUSTER_ZONE_EXTRA1="us-central1-b"
export GCP_KUBE_CLUSTER_ZONE_EXTRA2="us-central1-c"
export GCP_CLOUD_FUNCTION_REGION="us-central1"

Other regions can be targeted by updating the above. For example, see https://cloud.google.com/compute/docs/regions-zones/ for suitable values.

  1. Ensure the In-use IP addresses quota is set to 20 or above for chosen region

  2. Run the integration tests via:

$ bundle install
$ bundle exec rake test:integration

Alternatively, finer grained rake tasks are also available. Executing these in order is the same as the above command:

  • Initialize local workspace (terraform init)
$ bundle exec rake test:init_workspace
  • Plan integration tests - ensures variables are set for Inspec and Terraform, runs "terraform plan"
$ bundle exec rake test:plan_integration_tests
  • Set up integration tests - actually creates the resources in GCP (terraform apply)
$ bundle exec rake test:setup_integration_tests
  • Run integration tests - runs the tests (inspec exec)
$ bundle exec rake test:run_integration_tests
  • Clean up integration tests - removes GCP resources (terraform destroy)
$ bundle exec rake test:cleanup_integration_tests

Further Reading

FAQ

Failure running "inspec exec" on my GCP profile

If an error such as the below occurs when running "inspec exec" on a newly created GCP profile:

libraries/google_compute_instance.rb:26:in `block in initialize': undefined method `gcp_compute_client' for #<Train::Transports::Local::Connection:0x00007fcasdf1a532d0> (NoMethodError)

Check that the GCP transport is being specified as below:

$ inspec exec . -t gcp://

This tells the underlying transport layer (train) to use GCP.

access not configured error

InSpec relies on the GCP API's to verify the settings. Therefore, it requires access to the API. If you try to access an API via an InSpec resource that is not enabled in your account, then you see an error like:

googleapi: Error 403: Access Not Configured. Compute Engine API has not been used in project 41111111111 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/compute.googleapis.com/overview then retry.

Quota increase

The terraform templates generate sufficient resources to require an increase to default in_use IP addresses. Normally new projects have 10, increasing this to 20 or higher should be sufficient.

To find this setting, log in to the GCP web interface and go to IAM and admin->Quotas and look for "Compute Engine API In-use IP addresses". From here you can "Edit quotas" to request more.

Changed Quota:
+----------------------+------------------+
| Region: europe-west2 | IN_USE_ADDRESSES |
+----------------------+------------------+
|       Changes        |     8 -> 64      |
+----------------------+------------------+

Errors on terraform destroy

Sometimes there can be occasional errors when performing the cleanup rake task. This happens when resources are already deleted and can be ignored.

Upgrading to version 1.0

A guide on upgrading to version 1.0 can be found here

Support

The InSpec GCP resources are community supported. For bugs and features, please open a github issue and label it appropriately.

Kudos

This implementation is inspired by inspec-azure and inspec-gcp