Note: This is not officially maintained right now and mostly serves as an example. If you are looking for an official k8s helm chart take a look here.

CloudQuery AWS Module

This folder contains a Terraform module to deploy a CloudQuery cluster in AWS on top of EKS.

Usage

Examples are included in the example folder, but simple usage is as follows:

module "cloudquery" {
  source = "cloudquery/cloudquery/aws"
  version = "~> 0.5"

  name = "cloudquery"

  cidr = "10.20.0.0/16"
  azs  = ["us-east-1a", "us-east-1b", "us-east-1c"]
  public_subnets  = ["10.20.1.0/24", "10.20.2.0/24", "10.20.3.0/24"]
  private_subnets = ["10.20.101.0/24", "10.20.102.0/24", "10.20.103.0/24"]
  database_subnets = ["10.10.21.0/24", "10.10.22.0/24"]

  # path to your cloudquery config
  config_file = "config.yml"

}

Existing VPC

This way allows integration with your existing AWS resources - VPC, public and private subnets. Specify the following arguments (see methods described above):

If vpc_id is specified it will take precedence over cidr and existing VPC will be used.

Make sure that both private and public subnets were created in the same set of availability zones.

Run Helm Separately

Examples

Requirements

Name	Version
terraform	>= 1.0
aws	~> 4.15
helm	~> 2.5
random	~> 3.2

Providers

Name	Version
aws	~> 4.15
helm	~> 2.5
random	~> 3.2

Modules

Name	Source	Version
cluster_irsa	terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks	~> 4.20
eks	terraform-aws-modules/eks/aws	~> 18.31.0
iam_policy	terraform-aws-modules/iam/aws//modules/iam-policy	~> 4
rds	terraform-aws-modules/rds-aurora/aws	~> 7.7.0
security_group	terraform-aws-modules/security-group/aws	~> 4.2
vpc	terraform-aws-modules/vpc/aws	~> 3.0

Resources

Name	Type
aws_db_parameter_group.cloudquery	resource
aws_iam_role_policy_attachment.irsa	resource
aws_rds_cluster_parameter_group.cloudquery	resource
aws_secretsmanager_secret.cloudquery_secret	resource
aws_secretsmanager_secret_version.cloudquery_secret_version	resource
helm_release.cloudquery	resource
random_password.rds	resource
aws_availability_zones.available	data source
aws_eks_cluster_auth.cluster	data source
aws_secretsmanager_secret_version.cloudquery_secret_version	data source
aws_vpc.cq_vpc	data source

Inputs

Name	Description	Type	Default	Required
allowed_cidr_blocks	If RDS is publicly accessible it is highly advised to specify allowed cidrs from where you are planning to connect	list(string)	[]	no
chart_values	Variables to pass to the helm chart	string	""	no
chart_version	The version of CloudQuery helm chart	string	"17.0.1"	no
config_file	Path to the CloudQuery config.yml	string	""	no
database_subnet_group	If vpc_id is specified, path the subnet_group name where the RDS should reside	string	""	no
install_helm_chart	Enable/Disable helm chart installation	bool	true	no
name	Name to use on all resources created (VPC, RDS, etc)	string	"cloudquery"	no
postgres_engine_version	Version of Amazon RDS Postgres engine to use	string	"13.6"	no
postgres_family	Family of Amazon RDS Postgres engine to use	string	"aurora-postgresql13"	no
postgres_instance_class	Postgresql Instance Class	string	"db.t3.medium"	no
public_subnet_ids	A list of IDs of existing public subnets inside the VPC	list(string)	[]	no
publicly_accessible	Make RDS publicly accessible (might be needed if you want to connect to it from Grafana or other tools).	bool	false	no
role_policy_arns	Policies for the role to use for the EKS service account	list(string)	[ "arn:aws:iam::aws:policy/ReadOnlyAccess" ]	no
tags	A map of tags to use on all resources	map(string)	{}	no
vpc_id	ID of an existing VPC where resources will be created	string	null	no

Outputs

Name	Description
helm_values	Helm values for the cluster
irsa_arn	ARN of IRSA - (IAM Role for service account)
irsa_name	Name of IRSA - (IAM Role for service account)
rds_cluster_master_password	Master password for cloudquery rds database
vpc_id	ID of the VPC that was created or passed in

Troubleshooting

If helm installtion is stuck in some hanging state you can run the following commands:

# check if helm is installed in cloudquery namespace
helm ls -n cloudquery
# If yes uninstall with the your release name
helm uninstall YOUR_RELEASE_NAME -n cloudquery

Authors

Module is maintained by Anton Babenko and CloudQuery Team.

License

Apache 2 Licensed. See LICENSE for full details.