From 1b938f975d294020f089048fd065be007d953f9a Mon Sep 17 00:00:00 2001 From: "Byungjin Park (Claud)" Date: Sun, 17 Jul 2022 02:17:33 +0900 Subject: [PATCH 01/33] Support SASL/SCRAM for msk-cluster module (#24) --- modules/msk-cluster/README.md | 11 +++++- modules/msk-cluster/cluster.tf | 11 +++++- modules/msk-cluster/outputs.tf | 10 ++++- modules/msk-cluster/scram-secrets.tf | 55 ++++++++++++++++++++++++++++ modules/msk-cluster/variables.tf | 18 ++++++++- modules/msk-cluster/versions.tf | 4 ++ 6 files changed, 102 insertions(+), 7 deletions(-) create mode 100644 modules/msk-cluster/scram-secrets.tf diff --git a/modules/msk-cluster/README.md b/modules/msk-cluster/README.md index 77963a4..37deaa0 100644 --- a/modules/msk-cluster/README.md +++ b/modules/msk-cluster/README.md @@ -4,8 +4,10 @@ This module creates following resources. - `aws_msk_cluster` - `aws_msk_configuration` +- `aws_msk_scram_secret_association` (optional) - `aws_security_group` (optional) - `aws_security_group_rule` (optional) +- `aws_secretsmanager_secret` (optional) ## Requirements @@ -14,17 +16,20 @@ This module creates following resources. |------|---------| | [terraform](#requirement\_terraform) | >= 1.2 | | [aws](#requirement\_aws) | >= 4.22 | +| [random](#requirement\_random) | >= 3.3 | ## Providers | Name | Version | |------|---------| | [aws](#provider\_aws) | 4.22.0 | +| [random](#provider\_random) | 3.3.2 | ## Modules | Name | Source | Version | |------|--------|---------| +| [secret](#module\_secret) | tedilabs/secret/aws//modules/secrets-manager-secret | ~> 0.2.0 | | [security\_group](#module\_security\_group) | tedilabs/network/aws//modules/security-group | 0.26.0 | ## Resources @@ -33,7 +38,9 @@ This module creates following resources. |------|------| | [aws_msk_cluster.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/msk_cluster) | resource | | [aws_msk_configuration.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/msk_configuration) | resource | +| [aws_msk_scram_secret_association.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/msk_scram_secret_association) | resource | | [aws_resourcegroups_group.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/resourcegroups_group) | resource | +| [random_password.this](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) | resource | | [aws_msk_broker_nodes.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/msk_broker_nodes) | data source | | [aws_subnet.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnet) | data source | @@ -46,6 +53,8 @@ This module creates following resources. | [name](#input\_name) | (Required) Name of the MSK cluster. | `string` | n/a | yes | | [auth\_sasl\_iam\_enabled](#input\_auth\_sasl\_iam\_enabled) | (Optional) Enables IAM client authentication. | `bool` | `false` | no | | [auth\_sasl\_scram\_enabled](#input\_auth\_sasl\_scram\_enabled) | (Optional) Enables SCRAM client authentication via AWS Secrets Manager. | `bool` | `false` | no | +| [auth\_sasl\_scram\_kms\_key](#input\_auth\_sasl\_scram\_kms\_key) | (Optional) The ARN of a KMS key to encrypt AWS SeecretsManager Secret resources for storing SASL/SCRAM authentication data. Only required when the MSK cluster has SASL/SCRAM authentication enabled. The Username/Password Authentication based on SASL/SCRAM needs to create a Secret resource in AWS SecretsManager with a custom AWS KMS Key. A secret created with the default AWS KMS key cannot be used with an Amazon MSK cluster. | `string` | `null` | no | +| [auth\_sasl\_scram\_users](#input\_auth\_sasl\_scram\_users) | (Optional) A list of usernames to be allowed for SASL/SCRAM authentication to the MSK cluster. The password for each username is randomly generated and stored in AWS SecretsManager secret. | `set(string)` | `[]` | no | | [auth\_tls\_acm\_ca\_arns](#input\_auth\_tls\_acm\_ca\_arns) | (Optional) List of ACM Certificate Authority Amazon Resource Names (ARNs). | `list(string)` | `[]` | no | | [auth\_tls\_enabled](#input\_auth\_tls\_enabled) | (Optional) Enables TLS client authentication. | `bool` | `false` | no | | [auth\_unauthenticated\_access\_enabled](#input\_auth\_unauthenticated\_access\_enabled) | (Optional) Enables unauthenticated access. Defaults to `true`. | `bool` | `true` | no | @@ -59,7 +68,7 @@ This module creates following resources. | [encryption\_at\_rest\_kms\_key](#input\_encryption\_at\_rest\_kms\_key) | (Optional) Specify a KMS key short ID or ARN (it will always output an ARN) to use for encrypting your data at rest. If no key is specified, an AWS managed KMS ('aws/msk' managed service) key will be used for encrypting the data at rest. | `string` | `""` | no | | [encryption\_in\_transit\_client\_mode](#input\_encryption\_in\_transit\_client\_mode) | (Optional) Encryption setting for data in transit between clients and brokers. `TLS`, `TLS_PLAINTEXT`, `PLAINTEXT` are available. | `string` | `"TLS_PLAINTEXT"` | no | | [encryption\_in\_transit\_in\_cluster\_enabled](#input\_encryption\_in\_transit\_in\_cluster\_enabled) | (Optional) Whether data communication among broker nodes is encrypted. | `bool` | `true` | no | -| [kafka\_server\_properties](#input\_kafka\_server\_properties) | (Optional) Contents of the `server.properties` file for configuration of Kafka. | `string` | `""` | no | +| [kafka\_server\_properties](#input\_kafka\_server\_properties) | (Optional) Contents of the `server.properties` file for configuration of Kafka. | `map(string)` | `{}` | no | | [kafka\_version](#input\_kafka\_version) | (Optional) Kafka version to use for the MSK cluster. | `string` | `"2.8.0"` | no | | [logging\_cloudwatch\_enabled](#input\_logging\_cloudwatch\_enabled) | (Optional) Indicates whether you want to enable or disable streaming broker logs to Cloudwatch Logs. | `bool` | `false` | no | | [logging\_cloudwatch\_log\_group](#input\_logging\_cloudwatch\_log\_group) | (Optional) The name of log group on CloudWatch Logs to deliver logs to. | `string` | `""` | no | diff --git a/modules/msk-cluster/cluster.tf b/modules/msk-cluster/cluster.tf index 7b6b390..6ec4b84 100644 --- a/modules/msk-cluster/cluster.tf +++ b/modules/msk-cluster/cluster.tf @@ -19,12 +19,20 @@ locals { # Configuration for MSK Cluster ################################################### +locals { + server_properties = < 0 ? 1 : 0 + + cluster_arn = aws_msk_cluster.this.arn + secret_arn_list = values(module.secret).*.arn +} diff --git a/modules/msk-cluster/variables.tf b/modules/msk-cluster/variables.tf index 4c633fa..634de59 100644 --- a/modules/msk-cluster/variables.tf +++ b/modules/msk-cluster/variables.tf @@ -12,8 +12,9 @@ variable "kafka_version" { variable "kafka_server_properties" { description = "(Optional) Contents of the `server.properties` file for configuration of Kafka." - type = string - default = "" + type = map(string) + default = {} + nullable = false } variable "broker_size" { @@ -101,6 +102,19 @@ variable "auth_sasl_scram_enabled" { nullable = false } +variable "auth_sasl_scram_kms_key" { + description = "(Optional) The ARN of a KMS key to encrypt AWS SeecretsManager Secret resources for storing SASL/SCRAM authentication data. Only required when the MSK cluster has SASL/SCRAM authentication enabled. The Username/Password Authentication based on SASL/SCRAM needs to create a Secret resource in AWS SecretsManager with a custom AWS KMS Key. A secret created with the default AWS KMS key cannot be used with an Amazon MSK cluster." + type = string + default = null +} + +variable "auth_sasl_scram_users" { + description = "(Optional) A list of usernames to be allowed for SASL/SCRAM authentication to the MSK cluster. The password for each username is randomly generated and stored in AWS SecretsManager secret." + type = set(string) + default = [] + nullable = false +} + variable "auth_tls_enabled" { description = "(Optional) Enables TLS client authentication." type = bool diff --git a/modules/msk-cluster/versions.tf b/modules/msk-cluster/versions.tf index 341c2f1..583387c 100644 --- a/modules/msk-cluster/versions.tf +++ b/modules/msk-cluster/versions.tf @@ -6,5 +6,9 @@ terraform { source = "hashicorp/aws" version = ">= 4.22" } + random = { + source = "hashicorp/random" + version = ">= 3.3" + } } } From e211a579bcf8fa7bdcd0205ae771cabf212ab329 Mon Sep 17 00:00:00 2001 From: Byungjin Park Date: Sun, 17 Jul 2022 02:18:19 +0900 Subject: [PATCH 02/33] Bump to v0.9.0 --- VERSION | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/VERSION b/VERSION index a3df0a6..ac39a10 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.8.0 +0.9.0 From 10f464a991af3cc4567dc288a865238a3e442d86 Mon Sep 17 00:00:00 2001 From: "Byungjin Park (Claud)" Date: Wed, 20 Jul 2022 21:06:03 +0900 Subject: [PATCH 03/33] Add cloudwatch-log-group module (#25) --- modules/cloudwatch-log-group/README.md | 58 ++++++++++++++++++ modules/cloudwatch-log-group/main.tf | 47 +++++++++++++++ modules/cloudwatch-log-group/outputs.tf | 35 +++++++++++ .../cloudwatch-log-group/resource-group.tf | 44 ++++++++++++++ modules/cloudwatch-log-group/variables.tf | 60 +++++++++++++++++++ modules/cloudwatch-log-group/versions.tf | 10 ++++ 6 files changed, 254 insertions(+) create mode 100644 modules/cloudwatch-log-group/README.md create mode 100644 modules/cloudwatch-log-group/main.tf create mode 100644 modules/cloudwatch-log-group/outputs.tf create mode 100644 modules/cloudwatch-log-group/resource-group.tf create mode 100644 modules/cloudwatch-log-group/variables.tf create mode 100644 modules/cloudwatch-log-group/versions.tf diff --git a/modules/cloudwatch-log-group/README.md b/modules/cloudwatch-log-group/README.md new file mode 100644 index 0000000..2150356 --- /dev/null +++ b/modules/cloudwatch-log-group/README.md @@ -0,0 +1,58 @@ +# cloudwatch-log-group + +This module creates following resources. + +- `aws_cloudwatch_log_group` +- `aws_cloudwatch_log_stream` (optional) + + +## Requirements + +| Name | Version | +|------|---------| +| [terraform](#requirement\_terraform) | >= 1.2 | +| [aws](#requirement\_aws) | >= 4.22 | + +## Providers + +| Name | Version | +|------|---------| +| [aws](#provider\_aws) | 4.22.0 | + +## Modules + +No modules. + +## Resources + +| Name | Type | +|------|------| +| [aws_cloudwatch_log_group.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) | resource | +| [aws_cloudwatch_log_stream.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_stream) | resource | +| [aws_resourcegroups_group.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/resourcegroups_group) | resource | + +## Inputs + +| Name | Description | Type | Default | Required | +|------|-------------|------|---------|:--------:| +| [name](#input\_name) | (Required) The name of the CloudWatch log group. | `string` | n/a | yes | +| [encryption\_kms\_key](#input\_encryption\_kms\_key) | (Optional) The ARN of the KMS Key to use when encrypting log data. Please note, after the AWS KMS CMK is disassociated from the log group, AWS CloudWatch Logs stops encrypting newly ingested data for the log group. All previously ingested data remains encrypted, and AWS CloudWatch Logs requires permissions for the CMK whenever the encrypted data is requested. | `string` | `null` | no | +| [module\_tags\_enabled](#input\_module\_tags\_enabled) | (Optional) Whether to create AWS Resource Tags for the module informations. | `bool` | `true` | no | +| [resource\_group\_description](#input\_resource\_group\_description) | (Optional) The description of Resource Group. | `string` | `"Managed by Terraform."` | no | +| [resource\_group\_enabled](#input\_resource\_group\_enabled) | (Optional) Whether to create Resource Group to find and group AWS resources which are created by this module. | `bool` | `true` | no | +| [resource\_group\_name](#input\_resource\_group\_name) | (Optional) The name of Resource Group. A Resource Group name can have a maximum of 127 characters, including letters, numbers, hyphens, dots, and underscores. The name cannot start with `AWS` or `aws`. | `string` | `""` | no | +| [retention\_in\_days](#input\_retention\_in\_days) | (Optional) Specify the number of days to retain log events in the log group. Possible values are: 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 3653, and 0. If you select 0, the events in the log group are always retained and never expire. Default to `90` days. | `number` | `90` | no | +| [streams](#input\_streams) | (Optional) A list of log streams for the CloudWatch log group. | `set(string)` | `[]` | no | +| [tags](#input\_tags) | (Optional) A map of tags to add to all resources. | `map(string)` | `{}` | no | + +## Outputs + +| Name | Description | +|------|-------------| +| [arn](#output\_arn) | The ARN of the CloudWatch log group. | +| [encryption\_kms\_key](#output\_encryption\_kms\_key) | The ARN of the KMS Key for log data encryption. | +| [id](#output\_id) | The ID of the CloudWatch log group. | +| [name](#output\_name) | The name of CloudWatch log group. | +| [retention\_in\_days](#output\_retention\_in\_days) | The number of days to retain log events in the log group. | +| [streams](#output\_streams) | The list of log streams for the log group. | + diff --git a/modules/cloudwatch-log-group/main.tf b/modules/cloudwatch-log-group/main.tf new file mode 100644 index 0000000..1756c73 --- /dev/null +++ b/modules/cloudwatch-log-group/main.tf @@ -0,0 +1,47 @@ +locals { + metadata = { + package = "terraform-aws-misc" + version = trimspace(file("${path.module}/../../VERSION")) + module = basename(path.module) + name = var.name + } + module_tags = var.module_tags_enabled ? { + "module.terraform.io/package" = local.metadata.package + "module.terraform.io/version" = local.metadata.version + "module.terraform.io/name" = local.metadata.module + "module.terraform.io/full-name" = "${local.metadata.package}/${local.metadata.module}" + "module.terraform.io/instance" = local.metadata.name + } : {} +} + + +################################################### +# CloudWatch Log Group +################################################### + +resource "aws_cloudwatch_log_group" "this" { + name = var.name + + retention_in_days = var.retention_in_days + kms_key_id = var.encryption_kms_key + + tags = merge( + { + "Name" = local.metadata.name + }, + local.module_tags, + var.tags, + ) +} + + +################################################### +# Log Streams for CloudWatch Log Group +################################################### + +resource "aws_cloudwatch_log_stream" "this" { + for_each = var.streams + + name = each.key + log_group_name = aws_cloudwatch_log_group.this.name +} diff --git a/modules/cloudwatch-log-group/outputs.tf b/modules/cloudwatch-log-group/outputs.tf new file mode 100644 index 0000000..b0a59b3 --- /dev/null +++ b/modules/cloudwatch-log-group/outputs.tf @@ -0,0 +1,35 @@ +output "arn" { + description = "The ARN of the CloudWatch log group." + value = aws_cloudwatch_log_group.this.arn +} + +output "id" { + description = "The ID of the CloudWatch log group." + value = aws_cloudwatch_log_group.this.id +} + +output "name" { + description = "The name of CloudWatch log group." + value = aws_cloudwatch_log_group.this.name +} + +output "retention_in_days" { + description = "The number of days to retain log events in the log group." + value = aws_cloudwatch_log_group.this.retention_in_days +} + +output "encryption_kms_key" { + description = "The ARN of the KMS Key for log data encryption." + value = aws_cloudwatch_log_group.this.kms_key_id +} + +output "streams" { + description = "The list of log streams for the log group." + value = { + for name, stream in aws_cloudwatch_log_stream.this : + name => { + arn = stream.arn + name = stream.name + } + } +} diff --git a/modules/cloudwatch-log-group/resource-group.tf b/modules/cloudwatch-log-group/resource-group.tf new file mode 100644 index 0000000..af108f9 --- /dev/null +++ b/modules/cloudwatch-log-group/resource-group.tf @@ -0,0 +1,44 @@ +locals { + resource_group_name = (var.resource_group_name != "" + ? var.resource_group_name + : join(".", [ + local.metadata.package, + local.metadata.module, + replace(local.metadata.name, "/[^a-zA-Z0-9_\\.-]/", "-"), + ]) + ) + resource_group_filters = [ + for key, value in local.module_tags : { + "Key" = key + "Values" = [value] + } + ] + resource_group_query = <<-JSON + { + "ResourceTypeFilters": [ + "AWS::AllSupported" + ], + "TagFilters": ${jsonencode(local.resource_group_filters)} + } + JSON +} + +resource "aws_resourcegroups_group" "this" { + count = (var.resource_group_enabled && var.module_tags_enabled) ? 1 : 0 + + name = local.resource_group_name + description = var.resource_group_description + + resource_query { + type = "TAG_FILTERS_1_0" + query = local.resource_group_query + } + + tags = merge( + { + "Name" = local.resource_group_name + }, + local.module_tags, + var.tags, + ) +} diff --git a/modules/cloudwatch-log-group/variables.tf b/modules/cloudwatch-log-group/variables.tf new file mode 100644 index 0000000..4e0ac77 --- /dev/null +++ b/modules/cloudwatch-log-group/variables.tf @@ -0,0 +1,60 @@ +variable "name" { + description = "(Required) The name of the CloudWatch log group." + type = string +} + +variable "retention_in_days" { + description = "(Optional) Specify the number of days to retain log events in the log group. Possible values are: 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 3653, and 0. If you select 0, the events in the log group are always retained and never expire. Default to `90` days." + type = number + default = 90 +} + +variable "encryption_kms_key" { + description = "(Optional) The ARN of the KMS Key to use when encrypting log data. Please note, after the AWS KMS CMK is disassociated from the log group, AWS CloudWatch Logs stops encrypting newly ingested data for the log group. All previously ingested data remains encrypted, and AWS CloudWatch Logs requires permissions for the CMK whenever the encrypted data is requested." + type = string + default = null +} + +variable "streams" { + description = "(Optional) A list of log streams for the CloudWatch log group." + type = set(string) + default = [] + nullable = false +} + +variable "tags" { + description = "(Optional) A map of tags to add to all resources." + type = map(string) + default = {} + nullable = false +} + +variable "module_tags_enabled" { + description = "(Optional) Whether to create AWS Resource Tags for the module informations." + type = bool + default = true + nullable = false +} + + +################################################### +# Resource Group +################################################### + +variable "resource_group_enabled" { + description = "(Optional) Whether to create Resource Group to find and group AWS resources which are created by this module." + type = bool + default = true +} + +variable "resource_group_name" { + description = "(Optional) The name of Resource Group. A Resource Group name can have a maximum of 127 characters, including letters, numbers, hyphens, dots, and underscores. The name cannot start with `AWS` or `aws`." + type = string + default = "" +} + +variable "resource_group_description" { + description = "(Optional) The description of Resource Group." + type = string + default = "Managed by Terraform." +} diff --git a/modules/cloudwatch-log-group/versions.tf b/modules/cloudwatch-log-group/versions.tf new file mode 100644 index 0000000..341c2f1 --- /dev/null +++ b/modules/cloudwatch-log-group/versions.tf @@ -0,0 +1,10 @@ +terraform { + required_version = ">= 1.2" + + required_providers { + aws = { + source = "hashicorp/aws" + version = ">= 4.22" + } + } +} From ae448a0d3caac8683e0b53ff506df608f85dcd1f Mon Sep 17 00:00:00 2001 From: Byungjin Park Date: Wed, 20 Jul 2022 21:06:53 +0900 Subject: [PATCH 04/33] Bump to v0.10.0 --- VERSION | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/VERSION b/VERSION index ac39a10..78bc1ab 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.9.0 +0.10.0 From 4a95497c62aea3ea9a8768da49c7cfaf3946fd7e Mon Sep 17 00:00:00 2001 From: "Byungjin Park (Claud)" Date: Tue, 26 Jul 2022 01:27:51 +0900 Subject: [PATCH 05/33] Remove cloudwatch-log-group module (#26) --- modules/cloudwatch-log-group/README.md | 58 ------------------ modules/cloudwatch-log-group/main.tf | 47 --------------- modules/cloudwatch-log-group/outputs.tf | 35 ----------- .../cloudwatch-log-group/resource-group.tf | 44 -------------- modules/cloudwatch-log-group/variables.tf | 60 ------------------- modules/cloudwatch-log-group/versions.tf | 10 ---- 6 files changed, 254 deletions(-) delete mode 100644 modules/cloudwatch-log-group/README.md delete mode 100644 modules/cloudwatch-log-group/main.tf delete mode 100644 modules/cloudwatch-log-group/outputs.tf delete mode 100644 modules/cloudwatch-log-group/resource-group.tf delete mode 100644 modules/cloudwatch-log-group/variables.tf delete mode 100644 modules/cloudwatch-log-group/versions.tf diff --git a/modules/cloudwatch-log-group/README.md b/modules/cloudwatch-log-group/README.md deleted file mode 100644 index 2150356..0000000 --- a/modules/cloudwatch-log-group/README.md +++ /dev/null @@ -1,58 +0,0 @@ -# cloudwatch-log-group - -This module creates following resources. - -- `aws_cloudwatch_log_group` -- `aws_cloudwatch_log_stream` (optional) - - -## Requirements - -| Name | Version | -|------|---------| -| [terraform](#requirement\_terraform) | >= 1.2 | -| [aws](#requirement\_aws) | >= 4.22 | - -## Providers - -| Name | Version | -|------|---------| -| [aws](#provider\_aws) | 4.22.0 | - -## Modules - -No modules. - -## Resources - -| Name | Type | -|------|------| -| [aws_cloudwatch_log_group.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) | resource | -| [aws_cloudwatch_log_stream.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_stream) | resource | -| [aws_resourcegroups_group.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/resourcegroups_group) | resource | - -## Inputs - -| Name | Description | Type | Default | Required | -|------|-------------|------|---------|:--------:| -| [name](#input\_name) | (Required) The name of the CloudWatch log group. | `string` | n/a | yes | -| [encryption\_kms\_key](#input\_encryption\_kms\_key) | (Optional) The ARN of the KMS Key to use when encrypting log data. Please note, after the AWS KMS CMK is disassociated from the log group, AWS CloudWatch Logs stops encrypting newly ingested data for the log group. All previously ingested data remains encrypted, and AWS CloudWatch Logs requires permissions for the CMK whenever the encrypted data is requested. | `string` | `null` | no | -| [module\_tags\_enabled](#input\_module\_tags\_enabled) | (Optional) Whether to create AWS Resource Tags for the module informations. | `bool` | `true` | no | -| [resource\_group\_description](#input\_resource\_group\_description) | (Optional) The description of Resource Group. | `string` | `"Managed by Terraform."` | no | -| [resource\_group\_enabled](#input\_resource\_group\_enabled) | (Optional) Whether to create Resource Group to find and group AWS resources which are created by this module. | `bool` | `true` | no | -| [resource\_group\_name](#input\_resource\_group\_name) | (Optional) The name of Resource Group. A Resource Group name can have a maximum of 127 characters, including letters, numbers, hyphens, dots, and underscores. The name cannot start with `AWS` or `aws`. | `string` | `""` | no | -| [retention\_in\_days](#input\_retention\_in\_days) | (Optional) Specify the number of days to retain log events in the log group. Possible values are: 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 3653, and 0. If you select 0, the events in the log group are always retained and never expire. Default to `90` days. | `number` | `90` | no | -| [streams](#input\_streams) | (Optional) A list of log streams for the CloudWatch log group. | `set(string)` | `[]` | no | -| [tags](#input\_tags) | (Optional) A map of tags to add to all resources. | `map(string)` | `{}` | no | - -## Outputs - -| Name | Description | -|------|-------------| -| [arn](#output\_arn) | The ARN of the CloudWatch log group. | -| [encryption\_kms\_key](#output\_encryption\_kms\_key) | The ARN of the KMS Key for log data encryption. | -| [id](#output\_id) | The ID of the CloudWatch log group. | -| [name](#output\_name) | The name of CloudWatch log group. | -| [retention\_in\_days](#output\_retention\_in\_days) | The number of days to retain log events in the log group. | -| [streams](#output\_streams) | The list of log streams for the log group. | - diff --git a/modules/cloudwatch-log-group/main.tf b/modules/cloudwatch-log-group/main.tf deleted file mode 100644 index 1756c73..0000000 --- a/modules/cloudwatch-log-group/main.tf +++ /dev/null @@ -1,47 +0,0 @@ -locals { - metadata = { - package = "terraform-aws-misc" - version = trimspace(file("${path.module}/../../VERSION")) - module = basename(path.module) - name = var.name - } - module_tags = var.module_tags_enabled ? { - "module.terraform.io/package" = local.metadata.package - "module.terraform.io/version" = local.metadata.version - "module.terraform.io/name" = local.metadata.module - "module.terraform.io/full-name" = "${local.metadata.package}/${local.metadata.module}" - "module.terraform.io/instance" = local.metadata.name - } : {} -} - - -################################################### -# CloudWatch Log Group -################################################### - -resource "aws_cloudwatch_log_group" "this" { - name = var.name - - retention_in_days = var.retention_in_days - kms_key_id = var.encryption_kms_key - - tags = merge( - { - "Name" = local.metadata.name - }, - local.module_tags, - var.tags, - ) -} - - -################################################### -# Log Streams for CloudWatch Log Group -################################################### - -resource "aws_cloudwatch_log_stream" "this" { - for_each = var.streams - - name = each.key - log_group_name = aws_cloudwatch_log_group.this.name -} diff --git a/modules/cloudwatch-log-group/outputs.tf b/modules/cloudwatch-log-group/outputs.tf deleted file mode 100644 index b0a59b3..0000000 --- a/modules/cloudwatch-log-group/outputs.tf +++ /dev/null @@ -1,35 +0,0 @@ -output "arn" { - description = "The ARN of the CloudWatch log group." - value = aws_cloudwatch_log_group.this.arn -} - -output "id" { - description = "The ID of the CloudWatch log group." - value = aws_cloudwatch_log_group.this.id -} - -output "name" { - description = "The name of CloudWatch log group." - value = aws_cloudwatch_log_group.this.name -} - -output "retention_in_days" { - description = "The number of days to retain log events in the log group." - value = aws_cloudwatch_log_group.this.retention_in_days -} - -output "encryption_kms_key" { - description = "The ARN of the KMS Key for log data encryption." - value = aws_cloudwatch_log_group.this.kms_key_id -} - -output "streams" { - description = "The list of log streams for the log group." - value = { - for name, stream in aws_cloudwatch_log_stream.this : - name => { - arn = stream.arn - name = stream.name - } - } -} diff --git a/modules/cloudwatch-log-group/resource-group.tf b/modules/cloudwatch-log-group/resource-group.tf deleted file mode 100644 index af108f9..0000000 --- a/modules/cloudwatch-log-group/resource-group.tf +++ /dev/null @@ -1,44 +0,0 @@ -locals { - resource_group_name = (var.resource_group_name != "" - ? var.resource_group_name - : join(".", [ - local.metadata.package, - local.metadata.module, - replace(local.metadata.name, "/[^a-zA-Z0-9_\\.-]/", "-"), - ]) - ) - resource_group_filters = [ - for key, value in local.module_tags : { - "Key" = key - "Values" = [value] - } - ] - resource_group_query = <<-JSON - { - "ResourceTypeFilters": [ - "AWS::AllSupported" - ], - "TagFilters": ${jsonencode(local.resource_group_filters)} - } - JSON -} - -resource "aws_resourcegroups_group" "this" { - count = (var.resource_group_enabled && var.module_tags_enabled) ? 1 : 0 - - name = local.resource_group_name - description = var.resource_group_description - - resource_query { - type = "TAG_FILTERS_1_0" - query = local.resource_group_query - } - - tags = merge( - { - "Name" = local.resource_group_name - }, - local.module_tags, - var.tags, - ) -} diff --git a/modules/cloudwatch-log-group/variables.tf b/modules/cloudwatch-log-group/variables.tf deleted file mode 100644 index 4e0ac77..0000000 --- a/modules/cloudwatch-log-group/variables.tf +++ /dev/null @@ -1,60 +0,0 @@ -variable "name" { - description = "(Required) The name of the CloudWatch log group." - type = string -} - -variable "retention_in_days" { - description = "(Optional) Specify the number of days to retain log events in the log group. Possible values are: 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 3653, and 0. If you select 0, the events in the log group are always retained and never expire. Default to `90` days." - type = number - default = 90 -} - -variable "encryption_kms_key" { - description = "(Optional) The ARN of the KMS Key to use when encrypting log data. Please note, after the AWS KMS CMK is disassociated from the log group, AWS CloudWatch Logs stops encrypting newly ingested data for the log group. All previously ingested data remains encrypted, and AWS CloudWatch Logs requires permissions for the CMK whenever the encrypted data is requested." - type = string - default = null -} - -variable "streams" { - description = "(Optional) A list of log streams for the CloudWatch log group." - type = set(string) - default = [] - nullable = false -} - -variable "tags" { - description = "(Optional) A map of tags to add to all resources." - type = map(string) - default = {} - nullable = false -} - -variable "module_tags_enabled" { - description = "(Optional) Whether to create AWS Resource Tags for the module informations." - type = bool - default = true - nullable = false -} - - -################################################### -# Resource Group -################################################### - -variable "resource_group_enabled" { - description = "(Optional) Whether to create Resource Group to find and group AWS resources which are created by this module." - type = bool - default = true -} - -variable "resource_group_name" { - description = "(Optional) The name of Resource Group. A Resource Group name can have a maximum of 127 characters, including letters, numbers, hyphens, dots, and underscores. The name cannot start with `AWS` or `aws`." - type = string - default = "" -} - -variable "resource_group_description" { - description = "(Optional) The description of Resource Group." - type = string - default = "Managed by Terraform." -} diff --git a/modules/cloudwatch-log-group/versions.tf b/modules/cloudwatch-log-group/versions.tf deleted file mode 100644 index 341c2f1..0000000 --- a/modules/cloudwatch-log-group/versions.tf +++ /dev/null @@ -1,10 +0,0 @@ -terraform { - required_version = ">= 1.2" - - required_providers { - aws = { - source = "hashicorp/aws" - version = ">= 4.22" - } - } -} From 6f25e686eac2a65cc6f2f9dfae64bbe73cbd4547 Mon Sep 17 00:00:00 2001 From: "Byungjin Park (Claud)" Date: Wed, 27 Jul 2022 01:43:40 +0900 Subject: [PATCH 06/33] Fix error on outputs of msk-cluster module when provisioned_throughput is disabled (#27) --- modules/msk-cluster/cluster.tf | 10 +++++++--- modules/msk-cluster/outputs.tf | 4 ++-- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/modules/msk-cluster/cluster.tf b/modules/msk-cluster/cluster.tf index 6ec4b84..c174a1b 100644 --- a/modules/msk-cluster/cluster.tf +++ b/modules/msk-cluster/cluster.tf @@ -69,9 +69,13 @@ resource "aws_msk_cluster" "this" { ebs_storage_info { volume_size = var.broker_volume_size - provisioned_throughput { - enabled = var.broker_volume_provisioned_throughput_enabled - volume_throughput = var.broker_volume_provisioned_throughput + dynamic "provisioned_throughput" { + for_each = var.broker_volume_provisioned_throughput_enabled ? ["go"] : [] + + content { + enabled = true + volume_throughput = var.broker_volume_provisioned_throughput + } } } } diff --git a/modules/msk-cluster/outputs.tf b/modules/msk-cluster/outputs.tf index 2dae55d..fd44387 100644 --- a/modules/msk-cluster/outputs.tf +++ b/modules/msk-cluster/outputs.tf @@ -61,8 +61,8 @@ output "broker" { volume = { size = aws_msk_cluster.this.broker_node_group_info.0.storage_info.0.ebs_storage_info.0.volume_size provisioned_throughput = { - enabled = aws_msk_cluster.this.broker_node_group_info.0.storage_info.0.ebs_storage_info.0.provisioned_throughput.0.enabled - throughput = aws_msk_cluster.this.broker_node_group_info.0.storage_info.0.ebs_storage_info.0.provisioned_throughput.0.volume_throughput + enabled = try(aws_msk_cluster.this.broker_node_group_info.0.storage_info.0.ebs_storage_info.0.provisioned_throughput.0.enabled, false) + throughput = try(aws_msk_cluster.this.broker_node_group_info.0.storage_info.0.ebs_storage_info.0.provisioned_throughput.0.volume_throughput, null) } } } From 5555bc9dc621f1d51630ceaa3e7503355e7df7a7 Mon Sep 17 00:00:00 2001 From: Byungjin Park Date: Wed, 27 Jul 2022 01:45:22 +0900 Subject: [PATCH 07/33] Bump to v0.10.1 --- VERSION | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/VERSION b/VERSION index 78bc1ab..5712157 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.10.0 +0.10.1 From f2b4bfe461c59d25c2aac17d1bdbab6a71e8e7a7 Mon Sep 17 00:00:00 2001 From: "Byungjin Park (Claud)" Date: Thu, 2 Feb 2023 20:33:02 +0900 Subject: [PATCH 08/33] Use module instead of resource for resource group (#29) --- modules/msk-cluster/README.md | 6 ++-- modules/msk-cluster/migrations.tf | 5 ++++ modules/msk-cluster/resource-group.tf | 29 +++++-------------- modules/s3-archive-bucket/README.md | 9 +++--- modules/s3-archive-bucket/migrations.tf | 5 ++++ modules/s3-archive-bucket/resource-group.tf | 29 +++++-------------- modules/s3-archive-bucket/versions.tf | 2 +- modules/sqs-aws-event-queue/README.md | 9 +++--- modules/sqs-aws-event-queue/resource-group.tf | 29 +++++-------------- modules/sqs-aws-event-queue/versions.tf | 2 +- 10 files changed, 49 insertions(+), 76 deletions(-) create mode 100644 modules/msk-cluster/migrations.tf create mode 100644 modules/s3-archive-bucket/migrations.tf diff --git a/modules/msk-cluster/README.md b/modules/msk-cluster/README.md index 37deaa0..55e1239 100644 --- a/modules/msk-cluster/README.md +++ b/modules/msk-cluster/README.md @@ -22,13 +22,14 @@ This module creates following resources. | Name | Version | |------|---------| -| [aws](#provider\_aws) | 4.22.0 | -| [random](#provider\_random) | 3.3.2 | +| [aws](#provider\_aws) | 4.52.0 | +| [random](#provider\_random) | 3.4.3 | ## Modules | Name | Source | Version | |------|--------|---------| +| [resource\_group](#module\_resource\_group) | tedilabs/misc/aws//modules/resource-group | ~> 0.10.0 | | [secret](#module\_secret) | tedilabs/secret/aws//modules/secrets-manager-secret | ~> 0.2.0 | | [security\_group](#module\_security\_group) | tedilabs/network/aws//modules/security-group | 0.26.0 | @@ -39,7 +40,6 @@ This module creates following resources. | [aws_msk_cluster.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/msk_cluster) | resource | | [aws_msk_configuration.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/msk_configuration) | resource | | [aws_msk_scram_secret_association.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/msk_scram_secret_association) | resource | -| [aws_resourcegroups_group.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/resourcegroups_group) | resource | | [random_password.this](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) | resource | | [aws_msk_broker_nodes.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/msk_broker_nodes) | data source | | [aws_subnet.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnet) | data source | diff --git a/modules/msk-cluster/migrations.tf b/modules/msk-cluster/migrations.tf new file mode 100644 index 0000000..83a0f7e --- /dev/null +++ b/modules/msk-cluster/migrations.tf @@ -0,0 +1,5 @@ +# 2023-02-01 +moved { + from = aws_resourcegroups_group.this[0] + to = module.resource_group[0].aws_resourcegroups_group.this +} diff --git a/modules/msk-cluster/resource-group.tf b/modules/msk-cluster/resource-group.tf index af108f9..7487ba0 100644 --- a/modules/msk-cluster/resource-group.tf +++ b/modules/msk-cluster/resource-group.tf @@ -7,37 +7,24 @@ locals { replace(local.metadata.name, "/[^a-zA-Z0-9_\\.-]/", "-"), ]) ) - resource_group_filters = [ - for key, value in local.module_tags : { - "Key" = key - "Values" = [value] - } - ] - resource_group_query = <<-JSON - { - "ResourceTypeFilters": [ - "AWS::AllSupported" - ], - "TagFilters": ${jsonencode(local.resource_group_filters)} - } - JSON } -resource "aws_resourcegroups_group" "this" { + +module "resource_group" { + source = "tedilabs/misc/aws//modules/resource-group" + version = "~> 0.10.0" + count = (var.resource_group_enabled && var.module_tags_enabled) ? 1 : 0 name = local.resource_group_name description = var.resource_group_description - resource_query { - type = "TAG_FILTERS_1_0" - query = local.resource_group_query + query = { + resource_tags = local.module_tags } + module_tags_enabled = false tags = merge( - { - "Name" = local.resource_group_name - }, local.module_tags, var.tags, ) diff --git a/modules/s3-archive-bucket/README.md b/modules/s3-archive-bucket/README.md index cbbae13..53eacb9 100644 --- a/modules/s3-archive-bucket/README.md +++ b/modules/s3-archive-bucket/README.md @@ -18,24 +18,25 @@ This module creates following resources. | Name | Version | |------|---------| -| [terraform](#requirement\_terraform) | >= 1.1 | +| [terraform](#requirement\_terraform) | >= 1.2 | | [aws](#requirement\_aws) | >= 4.16 | ## Providers | Name | Version | |------|---------| -| [aws](#provider\_aws) | 4.16.0 | +| [aws](#provider\_aws) | 4.52.0 | ## Modules -No modules. +| Name | Source | Version | +|------|--------|---------| +| [resource\_group](#module\_resource\_group) | tedilabs/misc/aws//modules/resource-group | ~> 0.10.0 | ## Resources | Name | Type | |------|------| -| [aws_resourcegroups_group.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/resourcegroups_group) | resource | | [aws_s3_bucket.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) | resource | | [aws_s3_bucket_accelerate_configuration.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_accelerate_configuration) | resource | | [aws_s3_bucket_acl.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_acl) | resource | diff --git a/modules/s3-archive-bucket/migrations.tf b/modules/s3-archive-bucket/migrations.tf new file mode 100644 index 0000000..83a0f7e --- /dev/null +++ b/modules/s3-archive-bucket/migrations.tf @@ -0,0 +1,5 @@ +# 2023-02-01 +moved { + from = aws_resourcegroups_group.this[0] + to = module.resource_group[0].aws_resourcegroups_group.this +} diff --git a/modules/s3-archive-bucket/resource-group.tf b/modules/s3-archive-bucket/resource-group.tf index af108f9..7487ba0 100644 --- a/modules/s3-archive-bucket/resource-group.tf +++ b/modules/s3-archive-bucket/resource-group.tf @@ -7,37 +7,24 @@ locals { replace(local.metadata.name, "/[^a-zA-Z0-9_\\.-]/", "-"), ]) ) - resource_group_filters = [ - for key, value in local.module_tags : { - "Key" = key - "Values" = [value] - } - ] - resource_group_query = <<-JSON - { - "ResourceTypeFilters": [ - "AWS::AllSupported" - ], - "TagFilters": ${jsonencode(local.resource_group_filters)} - } - JSON } -resource "aws_resourcegroups_group" "this" { + +module "resource_group" { + source = "tedilabs/misc/aws//modules/resource-group" + version = "~> 0.10.0" + count = (var.resource_group_enabled && var.module_tags_enabled) ? 1 : 0 name = local.resource_group_name description = var.resource_group_description - resource_query { - type = "TAG_FILTERS_1_0" - query = local.resource_group_query + query = { + resource_tags = local.module_tags } + module_tags_enabled = false tags = merge( - { - "Name" = local.resource_group_name - }, local.module_tags, var.tags, ) diff --git a/modules/s3-archive-bucket/versions.tf b/modules/s3-archive-bucket/versions.tf index 6235d27..5242434 100644 --- a/modules/s3-archive-bucket/versions.tf +++ b/modules/s3-archive-bucket/versions.tf @@ -1,5 +1,5 @@ terraform { - required_version = ">= 1.1" + required_version = ">= 1.2" required_providers { aws = { diff --git a/modules/sqs-aws-event-queue/README.md b/modules/sqs-aws-event-queue/README.md index d5f7817..954be30 100644 --- a/modules/sqs-aws-event-queue/README.md +++ b/modules/sqs-aws-event-queue/README.md @@ -10,24 +10,25 @@ This module creates following resources. | Name | Version | |------|---------| -| [terraform](#requirement\_terraform) | >= 0.15 | +| [terraform](#requirement\_terraform) | >= 1.2 | | [aws](#requirement\_aws) | >= 3.45 | ## Providers | Name | Version | |------|---------| -| [aws](#provider\_aws) | 3.69.0 | +| [aws](#provider\_aws) | 4.52.0 | ## Modules -No modules. +| Name | Source | Version | +|------|--------|---------| +| [resource\_group](#module\_resource\_group) | tedilabs/misc/aws//modules/resource-group | ~> 0.10.0 | ## Resources | Name | Type | |------|------| -| [aws_resourcegroups_group.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/resourcegroups_group) | resource | | [aws_sqs_queue.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sqs_queue) | resource | | [aws_sqs_queue_policy.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sqs_queue_policy) | resource | | [aws_iam_policy_document.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source | diff --git a/modules/sqs-aws-event-queue/resource-group.tf b/modules/sqs-aws-event-queue/resource-group.tf index af108f9..7487ba0 100644 --- a/modules/sqs-aws-event-queue/resource-group.tf +++ b/modules/sqs-aws-event-queue/resource-group.tf @@ -7,37 +7,24 @@ locals { replace(local.metadata.name, "/[^a-zA-Z0-9_\\.-]/", "-"), ]) ) - resource_group_filters = [ - for key, value in local.module_tags : { - "Key" = key - "Values" = [value] - } - ] - resource_group_query = <<-JSON - { - "ResourceTypeFilters": [ - "AWS::AllSupported" - ], - "TagFilters": ${jsonencode(local.resource_group_filters)} - } - JSON } -resource "aws_resourcegroups_group" "this" { + +module "resource_group" { + source = "tedilabs/misc/aws//modules/resource-group" + version = "~> 0.10.0" + count = (var.resource_group_enabled && var.module_tags_enabled) ? 1 : 0 name = local.resource_group_name description = var.resource_group_description - resource_query { - type = "TAG_FILTERS_1_0" - query = local.resource_group_query + query = { + resource_tags = local.module_tags } + module_tags_enabled = false tags = merge( - { - "Name" = local.resource_group_name - }, local.module_tags, var.tags, ) diff --git a/modules/sqs-aws-event-queue/versions.tf b/modules/sqs-aws-event-queue/versions.tf index 6078ceb..dd959fc 100644 --- a/modules/sqs-aws-event-queue/versions.tf +++ b/modules/sqs-aws-event-queue/versions.tf @@ -1,5 +1,5 @@ terraform { - required_version = ">= 0.15" + required_version = ">= 1.2" required_providers { aws = { From dd9d222168bd7726f6ad4d52d85f19a9fafa9a1c Mon Sep 17 00:00:00 2001 From: Byungjin Park Date: Fri, 24 Feb 2023 02:15:03 +0900 Subject: [PATCH 09/33] Hello, 2023 --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 7ab0281..980fdc3 100644 --- a/README.md +++ b/README.md @@ -16,4 +16,4 @@ Like this project? Follow the repository on [GitHub](https://github.com/tedilabs Provided under the terms of the [Apache License](LICENSE). -Copyright © 2021-2022, [Byungjin Park](https://www.posquit0.com). +Copyright © 2021-2023, [Byungjin Park](https://www.posquit0.com). From 4c77ccf7605c1cac8255e5d90ad1938a9feebeb3 Mon Sep 17 00:00:00 2001 From: "Byungjin Park (Claud)" Date: Fri, 24 Feb 2023 02:56:17 +0900 Subject: [PATCH 10/33] Introduce tflint (#30) --- .github/workflows/integration.yaml | 101 ------------------ .github/workflows/terraform.integration.yaml | 104 +++++++++++++++++++ .github/workflows/yaml.integration.yaml | 58 +++++++++++ .pre-commit-config.yaml | 14 ++- .tflint.hcl | 77 ++++++++++++++ 5 files changed, 252 insertions(+), 102 deletions(-) delete mode 100644 .github/workflows/integration.yaml create mode 100644 .github/workflows/terraform.integration.yaml create mode 100644 .github/workflows/yaml.integration.yaml create mode 100644 .tflint.hcl diff --git a/.github/workflows/integration.yaml b/.github/workflows/integration.yaml deleted file mode 100644 index e915d59..0000000 --- a/.github/workflows/integration.yaml +++ /dev/null @@ -1,101 +0,0 @@ -name: Integration - -on: - push: - branches: - - main - pull_request: {} - -concurrency: - group: integration-${{ github.ref }} - cancel-in-progress: true - -jobs: - changed: - runs-on: ubuntu-latest - - outputs: - terraform_modules_changed: ${{ steps.filter-terraform-modules.outputs.changed }} - terraform_modules_files: ${{ steps.filter-terraform-modules.outputs.files }} - terraform_modules_dirs: ${{ steps.filter-terraform-modules.outputs.dirs }} - yaml_changed: ${{ steps.filter-yaml.outputs.changed }} - yaml_files: ${{ steps.filter-yaml.outputs.files }} - - steps: - - name: Checkout - uses: actions/checkout@v3 - with: - fetch-depth: 0 - - - name: Get Changed Files - id: changed-files - uses: dorny/paths-filter@v2 - with: - list-files: json - filters: | - modules: - - 'modules/**' - yaml: - - '**/*.yaml' - - '**/*.yml' - - - name: Filter changed Terraform Modules files to outputs - id: filter-terraform-modules - run: | - dirs=$(echo '${{ steps.changed-files.outputs.modules_files }}' | jq '[.[] | match("modules/[^/]+").string] | unique') - echo ::set-output name=changed::${{ steps.changed-files.outputs.modules }} - echo ::set-output name=files::${{ steps.changed-files.outputs.modules_files }} - echo ::set-output name=dirs::$dirs - - - name: Filter changed YAML files to outputs - id: filter-yaml - run: | - echo ::set-output name=changed::${{ steps.changed-files.outputs.yaml }} - echo ::set-output name=files::${{ steps.changed-files.outputs.yaml_files }} - - - terraform: - needs: - - changed - if: ${{ needs.changed.outputs.terraform_modules_changed != 'false' }} - runs-on: ubuntu-latest - - strategy: - matrix: - path: ${{ fromJson(needs.changed.outputs.terraform_modules_dirs) }} - - steps: - - name: Checkout - uses: actions/checkout@v3 - - - name: Set-up terraform - uses: hashicorp/setup-terraform@v2 - - - name: Terraform fmt - id: terraform-fmt - working-directory: ${{ matrix.path }} - run: terraform fmt -check - continue-on-error: true - - - name: Terraform Validate - id: terraform-validate - working-directory: ${{ matrix.path }} - run: | - terraform init -backend=false - terraform validate -no-color - - - yaml: - needs: - - changed - if: ${{ needs.changed.outputs.yaml_changed != 'false' }} - runs-on: ubuntu-latest - - steps: - - name: Checkout - uses: actions/checkout@v3 - - - name: Lint YAML Files - id: yaml-lint - run: | - yamllint . diff --git a/.github/workflows/terraform.integration.yaml b/.github/workflows/terraform.integration.yaml new file mode 100644 index 0000000..b4cfa94 --- /dev/null +++ b/.github/workflows/terraform.integration.yaml @@ -0,0 +1,104 @@ +name: Integration (Terraform) + +on: + push: + branches: + - main + pull_request: {} + +concurrency: + group: terraform-integration-${{ github.ref }} + cancel-in-progress: true + +jobs: + changed: + name: Filter Changed Files and Directories + runs-on: ubuntu-latest + + outputs: + changed: ${{ steps.set-outputs.outputs.changed }} + modified: ${{ steps.set-outputs.outputs.modified }} + changed_files: ${{ steps.set-outputs.outputs.changed_files }} + modified_files: ${{ steps.set-outputs.outputs.modified_files }} + changed_directories: ${{ steps.set-outputs.outputs.changed_directories }} + modified_directories: ${{ steps.set-outputs.outputs.modified_directories }} + + steps: + - name: Checkout + uses: actions/checkout@v3 + with: + fetch-depth: 0 + + - name: Get Changed Files + id: changed-files + uses: tj-actions/changed-files@v35 + with: + files: | + modules/** + json: true + + - name: Get Changed Directories + id: changed-directories + uses: tj-actions/changed-files@v35 + with: + files: | + modules/** + examples/** + dir_names: "true" + dir_names_max_depth: 2 + json: true + + - name: Set outputs + id: set-outputs + run: | + echo "changed=${{ steps.changed-directories.outputs.any_changed }}" >> $GITHUB_OUTPUT + echo "modified=${{ steps.changed-directories.outputs.any_modified }}" >> $GITHUB_OUTPUT + + echo "changed_files=${{ steps.changed-files.outputs.all_changed_files }}" >> $GITHUB_OUTPUT + echo "modified_files=${{ steps.changed-files.outputs.all_modified_files }}" >> $GITHUB_OUTPUT + + echo "changed_directories=${{ steps.changed-directories.outputs.all_changed_files }}" >> $GITHUB_OUTPUT + echo "modified_directories=${{ steps.changed-directories.outputs.all_modified_files }}" >> $GITHUB_OUTPUT + + + terraform: + name: Lint (terraform) + needs: + - changed + if: ${{ needs.changed.outputs.modified == 'true' }} + uses: tedilabs/.github/.github/workflows/terraform.terraform.yaml@main + + strategy: + matrix: + path: ${{ fromJson(needs.changed.outputs.modified_directories) }} + + with: + terraform_target_dir: ${{ matrix.path }} + terraform_version: latest + terraform_host: app.terraform.io + secrets: + terraform_token: ${{ secrets.TERRAFORM_TOKEN }} + + + tflint: + name: Lint (tflint) + needs: + - changed + if: ${{ needs.changed.outputs.modified == 'true' }} + uses: tedilabs/.github/.github/workflows/terraform.tflint.yaml@main + + strategy: + matrix: + path: ${{ fromJson(needs.changed.outputs.modified_directories) }} + + with: + tflint_version: latest + tflint_config_file: .tflint.hcl + tflint_target_dir: ${{ matrix.path }} + tflint_recursive_enabled: false + tflint_terraform_init_enabled: true + terraform_version: latest + terraform_host: app.terraform.io + secrets: + token: ${{ secrets.GITHUB_TOKEN }} + terraform_token: ${{ secrets.TERRAFORM_TOKEN }} diff --git a/.github/workflows/yaml.integration.yaml b/.github/workflows/yaml.integration.yaml new file mode 100644 index 0000000..20342a1 --- /dev/null +++ b/.github/workflows/yaml.integration.yaml @@ -0,0 +1,58 @@ +name: Integration (YAML) + +on: + push: + branches: + - main + pull_request: {} + +concurrency: + group: yaml-integration-${{ github.ref }} + cancel-in-progress: true + +jobs: + changed: + name: Filter Changed Files and Directories + runs-on: ubuntu-latest + + outputs: + changed: ${{ steps.set-outputs.outputs.changed }} + modified: ${{ steps.set-outputs.outputs.modified }} + changed_files: ${{ steps.set-outputs.outputs.changed_files }} + modified_files: ${{ steps.set-outputs.outputs.modified_files }} + + steps: + - name: Checkout + uses: actions/checkout@v3 + with: + fetch-depth: 0 + + - name: Get Changed Files + id: changed-files + uses: tj-actions/changed-files@v35 + with: + files: | + **/*.yaml + **/*.yml + json: true + + - name: Set outputs + id: set-outputs + run: | + echo "changed=${{ steps.changed-files.outputs.any_changed }}" >> $GITHUB_OUTPUT + echo "modified=${{ steps.changed-files.outputs.any_modified }}" >> $GITHUB_OUTPUT + + echo "changed_files=${{ steps.changed-files.outputs.all_changed_files }}" >> $GITHUB_OUTPUT + echo "modified_files=${{ steps.changed-files.outputs.all_modified_files }}" >> $GITHUB_OUTPUT + + lint: + name: Lint (yamllint) + needs: + - changed + if: ${{ needs.changed.outputs.modified == 'true' }} + uses: tedilabs/.github/.github/workflows/yaml.yamllint.yaml@main + + with: + yamllint_version: latest + yamllint_config_file: .yamllint.yaml + yamllint_target_dir: ./ diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index aa33d01..d0b1fea 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,8 +1,20 @@ repos: - repo: https://github.com/antonbabenko/pre-commit-terraform - rev: v1.70.1 + rev: v1.77.1 hooks: - id: terraform_fmt + args: + - --args=-diff - id: terraform_validate + args: + - --hook-config=--retry-once-with-cleanup=true + - id: terraform_tflint + args: + - --args=--config=__GIT_WORKING_DIR__/.tflint.hcl + files: ^modules/ - id: terraform_docs args: ["--args=--sort-by required"] +- repo: https://github.com/adrienverge/yamllint + rev: v1.29.0 + hooks: + - id: yamllint diff --git a/.tflint.hcl b/.tflint.hcl new file mode 100644 index 0000000..0788789 --- /dev/null +++ b/.tflint.hcl @@ -0,0 +1,77 @@ +config { + plugin_dir = "~/.tflint.d/plugins" + + format = "compact" + module = true + force = false + disabled_by_default = false + + ignore_module = {} +} + + +################################################### +# Rule Sets - Terraform +################################################### + +plugin "terraform" { + enabled = true + preset = "recommended" +} + +rule "terraform_comment_syntax" { + enabled = true +} + +rule "terraform_documented_variables" { + enabled = true +} + +rule "terraform_documented_outputs" { + enabled = true +} + +rule "terraform_naming_convention" { + enabled = true + format = "snake_case" + + custom_formats = { + extended_snake_case = { + description = "Extended snake_case Format which allows double underscore like `a__b`." + regex = "^[a-z][a-z0-9]+([_]{1,2}[a-z0-9]+)*$" + } + } + + module { + format = "extended_snake_case" + } + + resource { + format = "extended_snake_case" + } + + data { + format = "extended_snake_case" + } +} + +rule "terraform_unused_declarations" { + enabled = false +} + +rule "terraform_unused_required_providers" { + enabled = true +} + + +################################################### +# Rule Sets - AWS +################################################### + +plugin "aws" { + source = "github.com/terraform-linters/tflint-ruleset-aws" + version = "0.21.1" + + enabled = true + deep_check = false +} From 308272807e71be9b5a26dd76dba1bc54f9543ecc Mon Sep 17 00:00:00 2001 From: "Byungjin Park (Claud)" Date: Fri, 24 Feb 2023 03:05:24 +0900 Subject: [PATCH 11/33] Fix tflint failed code (#31) --- modules/msk-cluster/cluster.tf | 2 +- modules/msk-cluster/outputs.tf | 48 +++++++++++++++++----------------- 2 files changed, 25 insertions(+), 25 deletions(-) diff --git a/modules/msk-cluster/cluster.tf b/modules/msk-cluster/cluster.tf index c174a1b..0d83526 100644 --- a/modules/msk-cluster/cluster.tf +++ b/modules/msk-cluster/cluster.tf @@ -55,7 +55,7 @@ resource "aws_msk_cluster" "this" { az_distribution = "DEFAULT" client_subnets = var.broker_subnets security_groups = concat( - module.security_group.*.id, + module.security_group[*].id, var.broker_additional_security_groups ) diff --git a/modules/msk-cluster/outputs.tf b/modules/msk-cluster/outputs.tf index fd44387..844179e 100644 --- a/modules/msk-cluster/outputs.tf +++ b/modules/msk-cluster/outputs.tf @@ -30,7 +30,7 @@ output "kafka_config" { output "broker_security_group_id" { description = "The id of security group that were created for the MSK cluster." - value = try(module.security_group.*.id[0], null) + value = try(module.security_group[*].id[0], null) } output "broker_nodes" { @@ -51,18 +51,18 @@ output "broker" { EOF value = { size = aws_msk_cluster.this.number_of_broker_nodes - instance_type = aws_msk_cluster.this.broker_node_group_info.0.instance_type + instance_type = aws_msk_cluster.this.broker_node_group_info[0].instance_type - subnets = aws_msk_cluster.this.broker_node_group_info.0.client_subnets + subnets = aws_msk_cluster.this.broker_node_group_info[0].client_subnets public_access_enabled = var.broker_public_access_enabled - security_groups = aws_msk_cluster.this.broker_node_group_info.0.security_groups - default_security_group_id = try(module.security_group.*.id[0], null) + security_groups = aws_msk_cluster.this.broker_node_group_info[0].security_groups + default_security_group_id = try(module.security_group[*].id[0], null) volume = { - size = aws_msk_cluster.this.broker_node_group_info.0.storage_info.0.ebs_storage_info.0.volume_size + size = aws_msk_cluster.this.broker_node_group_info[0].storage_info[0].ebs_storage_info[0].volume_size provisioned_throughput = { - enabled = try(aws_msk_cluster.this.broker_node_group_info.0.storage_info.0.ebs_storage_info.0.provisioned_throughput.0.enabled, false) - throughput = try(aws_msk_cluster.this.broker_node_group_info.0.storage_info.0.ebs_storage_info.0.provisioned_throughput.0.volume_throughput, null) + enabled = try(aws_msk_cluster.this.broker_node_group_info[0].storage_info[0].ebs_storage_info[0].provisioned_throughput[0].enabled, false) + throughput = try(aws_msk_cluster.this.broker_node_group_info[0].storage_info[0].ebs_storage_info[0].provisioned_throughput[0].volume_throughput, null) } } } @@ -72,21 +72,21 @@ output "auth" { description = "A configuration for authentication of the Kafka cluster." value = { unauthenticated_access = { - enabled = aws_msk_cluster.this.client_authentication.0.unauthenticated + enabled = aws_msk_cluster.this.client_authentication[0].unauthenticated } sasl = { iam = { - enabled = aws_msk_cluster.this.client_authentication.0.sasl.0.iam + enabled = aws_msk_cluster.this.client_authentication[0].sasl[0].iam } scram = { - enabled = aws_msk_cluster.this.client_authentication.0.sasl.0.scram + enabled = aws_msk_cluster.this.client_authentication[0].sasl[0].scram kms_key = var.auth_sasl_scram_kms_key users = var.auth_sasl_scram_users } } tls = { enabled = var.auth_tls_enabled - acm_ca_arns = try(aws_msk_cluster.this.client_authentication.0.tls.0.certificate_authority_arns, []) + acm_ca_arns = try(aws_msk_cluster.this.client_authentication[0].tls[0].certificate_authority_arns, []) } } } @@ -99,11 +99,11 @@ output "encryption" { EOF value = { at_rest = { - kms_key = aws_msk_cluster.this.encryption_info.0.encryption_at_rest_kms_key_arn + kms_key = aws_msk_cluster.this.encryption_info[0].encryption_at_rest_kms_key_arn } in_transit = { - in_cluster_enabled = aws_msk_cluster.this.encryption_info.0.encryption_in_transit.0.in_cluster - client_mode = aws_msk_cluster.this.encryption_info.0.encryption_in_transit.0.client_broker + in_cluster_enabled = aws_msk_cluster.this.encryption_info[0].encryption_in_transit[0].in_cluster + client_mode = aws_msk_cluster.this.encryption_info[0].encryption_in_transit[0].client_broker } } } @@ -117,17 +117,17 @@ output "logging" { EOF value = { cloudwatch = { - enabled = aws_msk_cluster.this.logging_info.0.broker_logs.0.cloudwatch_logs.0.enabled - log_group = aws_msk_cluster.this.logging_info.0.broker_logs.0.cloudwatch_logs.0.log_group + enabled = aws_msk_cluster.this.logging_info[0].broker_logs[0].cloudwatch_logs[0].enabled + log_group = aws_msk_cluster.this.logging_info[0].broker_logs[0].cloudwatch_logs[0].log_group } firehose = { - enabled = aws_msk_cluster.this.logging_info.0.broker_logs.0.firehose.0.enabled - delivery_stream = aws_msk_cluster.this.logging_info.0.broker_logs.0.firehose.0.delivery_stream + enabled = aws_msk_cluster.this.logging_info[0].broker_logs[0].firehose[0].enabled + delivery_stream = aws_msk_cluster.this.logging_info[0].broker_logs[0].firehose[0].delivery_stream } s3 = { - enabled = aws_msk_cluster.this.logging_info.0.broker_logs.0.s3.0.enabled - bucket = aws_msk_cluster.this.logging_info.0.broker_logs.0.s3.0.bucket - prefix = aws_msk_cluster.this.logging_info.0.broker_logs.0.s3.0.prefix + enabled = aws_msk_cluster.this.logging_info[0].broker_logs[0].s3[0].enabled + bucket = aws_msk_cluster.this.logging_info[0].broker_logs[0].s3[0].bucket + prefix = aws_msk_cluster.this.logging_info[0].broker_logs[0].s3[0].prefix } } } @@ -143,8 +143,8 @@ output "monitoring" { level = aws_msk_cluster.this.enhanced_monitoring } prometheus = { - jmx_exporter_enabled = aws_msk_cluster.this.open_monitoring.0.prometheus.0.jmx_exporter.0.enabled_in_broker - node_exporter_enabled = aws_msk_cluster.this.open_monitoring.0.prometheus.0.node_exporter.0.enabled_in_broker + jmx_exporter_enabled = aws_msk_cluster.this.open_monitoring[0].prometheus[0].jmx_exporter[0].enabled_in_broker + node_exporter_enabled = aws_msk_cluster.this.open_monitoring[0].prometheus[0].node_exporter[0].enabled_in_broker } } } From c25ca6414ec26016442b3c9373a3c4ef3da72143 Mon Sep 17 00:00:00 2001 From: Byungjin Park Date: Sun, 26 Feb 2023 22:45:44 +0900 Subject: [PATCH 12/33] Update workflow --- .github/workflows/terraform.integration.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/terraform.integration.yaml b/.github/workflows/terraform.integration.yaml index b4cfa94..2fc3c46 100644 --- a/.github/workflows/terraform.integration.yaml +++ b/.github/workflows/terraform.integration.yaml @@ -35,6 +35,7 @@ jobs: with: files: | modules/** + examples/** json: true - name: Get Changed Directories From af059ce2011383570d6420970e701438ae5e3b37 Mon Sep 17 00:00:00 2001 From: Byungjin Park Date: Mon, 6 Mar 2023 02:39:18 +0900 Subject: [PATCH 13/33] Update github workflows --- .github/workflows/terraform.integration.yaml | 3 +++ .github/workflows/yaml.integration.yaml | 2 ++ 2 files changed, 5 insertions(+) diff --git a/.github/workflows/terraform.integration.yaml b/.github/workflows/terraform.integration.yaml index 2fc3c46..a1ad44f 100644 --- a/.github/workflows/terraform.integration.yaml +++ b/.github/workflows/terraform.integration.yaml @@ -78,6 +78,8 @@ jobs: terraform_version: latest terraform_host: app.terraform.io secrets: + gh_token: ${{ secrets.GITHUB_TOKEN }} + token: ${{ secrets.GITHUB_TOKEN }} terraform_token: ${{ secrets.TERRAFORM_TOKEN }} @@ -101,5 +103,6 @@ jobs: terraform_version: latest terraform_host: app.terraform.io secrets: + gh_token: ${{ secrets.GITHUB_TOKEN }} token: ${{ secrets.GITHUB_TOKEN }} terraform_token: ${{ secrets.TERRAFORM_TOKEN }} diff --git a/.github/workflows/yaml.integration.yaml b/.github/workflows/yaml.integration.yaml index 20342a1..751ecf9 100644 --- a/.github/workflows/yaml.integration.yaml +++ b/.github/workflows/yaml.integration.yaml @@ -56,3 +56,5 @@ jobs: yamllint_version: latest yamllint_config_file: .yamllint.yaml yamllint_target_dir: ./ + secrets: + token: ${{ secrets.GITHUB_TOKEN }} From 00516d034aa98453086deb894d82b188b8e7505f Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Sun, 25 Jun 2023 23:59:02 +0900 Subject: [PATCH 14/33] Bump tj-actions/changed-files from 35 to 37 (#33) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/terraform.integration.yaml | 4 ++-- .github/workflows/yaml.integration.yaml | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/terraform.integration.yaml b/.github/workflows/terraform.integration.yaml index a1ad44f..1572c19 100644 --- a/.github/workflows/terraform.integration.yaml +++ b/.github/workflows/terraform.integration.yaml @@ -31,7 +31,7 @@ jobs: - name: Get Changed Files id: changed-files - uses: tj-actions/changed-files@v35 + uses: tj-actions/changed-files@v37 with: files: | modules/** @@ -40,7 +40,7 @@ jobs: - name: Get Changed Directories id: changed-directories - uses: tj-actions/changed-files@v35 + uses: tj-actions/changed-files@v37 with: files: | modules/** diff --git a/.github/workflows/yaml.integration.yaml b/.github/workflows/yaml.integration.yaml index 751ecf9..a81b49e 100644 --- a/.github/workflows/yaml.integration.yaml +++ b/.github/workflows/yaml.integration.yaml @@ -29,7 +29,7 @@ jobs: - name: Get Changed Files id: changed-files - uses: tj-actions/changed-files@v35 + uses: tj-actions/changed-files@v37 with: files: | **/*.yaml From 905fd3bbac10239c89bee6328c53bc808f18821c Mon Sep 17 00:00:00 2001 From: Byungjin Park Date: Tue, 27 Jun 2023 23:19:01 +0900 Subject: [PATCH 15/33] Update pre-commit config --- .pre-commit-config.yaml | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index d0b1fea..10bde48 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,12 +1,13 @@ repos: - repo: https://github.com/antonbabenko/pre-commit-terraform - rev: v1.77.1 + rev: v1.81.0 hooks: - id: terraform_fmt args: - --args=-diff - id: terraform_validate args: + - --tf-init-args=-upgrade - --hook-config=--retry-once-with-cleanup=true - id: terraform_tflint args: @@ -14,7 +15,8 @@ repos: files: ^modules/ - id: terraform_docs args: ["--args=--sort-by required"] + - repo: https://github.com/adrienverge/yamllint - rev: v1.29.0 + rev: v1.32.0 hooks: - id: yamllint From 514e9de5113440793b8b0fa957a66a9f615061d5 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Sat, 26 Aug 2023 19:23:22 +0900 Subject: [PATCH 16/33] Bump tj-actions/changed-files from 37 to 38 (#34) Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 37 to 38. - [Release notes](https://github.com/tj-actions/changed-files/releases) - [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md) - [Commits](https://github.com/tj-actions/changed-files/compare/v37...v38) --- updated-dependencies: - dependency-name: tj-actions/changed-files dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Byungjin Park (Claud) --- .github/workflows/terraform.integration.yaml | 4 ++-- .github/workflows/yaml.integration.yaml | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/terraform.integration.yaml b/.github/workflows/terraform.integration.yaml index 1572c19..8cfe55e 100644 --- a/.github/workflows/terraform.integration.yaml +++ b/.github/workflows/terraform.integration.yaml @@ -31,7 +31,7 @@ jobs: - name: Get Changed Files id: changed-files - uses: tj-actions/changed-files@v37 + uses: tj-actions/changed-files@v38 with: files: | modules/** @@ -40,7 +40,7 @@ jobs: - name: Get Changed Directories id: changed-directories - uses: tj-actions/changed-files@v37 + uses: tj-actions/changed-files@v38 with: files: | modules/** diff --git a/.github/workflows/yaml.integration.yaml b/.github/workflows/yaml.integration.yaml index a81b49e..5149a2d 100644 --- a/.github/workflows/yaml.integration.yaml +++ b/.github/workflows/yaml.integration.yaml @@ -29,7 +29,7 @@ jobs: - name: Get Changed Files id: changed-files - uses: tj-actions/changed-files@v37 + uses: tj-actions/changed-files@v38 with: files: | **/*.yaml From d16060339e8831267ff7cb5a9d2550419eb609f3 Mon Sep 17 00:00:00 2001 From: Byungjin Park Date: Sat, 2 Sep 2023 23:26:58 +0900 Subject: [PATCH 17/33] Migrate msk-cluster to terraform-aws-messaging --- .github/labeler.yaml | 3 - .github/labels.yaml | 3 - modules/msk-cluster/README.md | 108 ---------- modules/msk-cluster/cluster.tf | 172 ---------------- modules/msk-cluster/migrations.tf | 5 - modules/msk-cluster/outputs.tf | 185 ----------------- modules/msk-cluster/resource-group.tf | 31 --- modules/msk-cluster/scram-secrets.tf | 55 ----- modules/msk-cluster/security-group.tf | 124 ------------ modules/msk-cluster/variables.tf | 278 -------------------------- modules/msk-cluster/versions.tf | 14 -- 11 files changed, 978 deletions(-) delete mode 100644 modules/msk-cluster/README.md delete mode 100644 modules/msk-cluster/cluster.tf delete mode 100644 modules/msk-cluster/migrations.tf delete mode 100644 modules/msk-cluster/outputs.tf delete mode 100644 modules/msk-cluster/resource-group.tf delete mode 100644 modules/msk-cluster/scram-secrets.tf delete mode 100644 modules/msk-cluster/security-group.tf delete mode 100644 modules/msk-cluster/variables.tf delete mode 100644 modules/msk-cluster/versions.tf diff --git a/.github/labeler.yaml b/.github/labeler.yaml index 30da008..3977b4a 100644 --- a/.github/labeler.yaml +++ b/.github/labeler.yaml @@ -1,7 +1,4 @@ # Modules -":floppy_disk: msk-cluster": -- modules/msk-cluster/**/* - ":floppy_disk: resource-group": - modules/resource-group/**/* diff --git a/.github/labels.yaml b/.github/labels.yaml index b2de133..1272df1 100644 --- a/.github/labels.yaml +++ b/.github/labels.yaml @@ -40,9 +40,6 @@ name: "size/XL" # Modules -- color: "fbca04" - description: "This issue or pull request is related to msk-cluster module." - name: ":floppy_disk: msk-cluster" - color: "fbca04" description: "This issue or pull request is related to resource-group module." name: ":floppy_disk: resource-group" diff --git a/modules/msk-cluster/README.md b/modules/msk-cluster/README.md deleted file mode 100644 index 55e1239..0000000 --- a/modules/msk-cluster/README.md +++ /dev/null @@ -1,108 +0,0 @@ -# msk-cluster - -This module creates following resources. - -- `aws_msk_cluster` -- `aws_msk_configuration` -- `aws_msk_scram_secret_association` (optional) -- `aws_security_group` (optional) -- `aws_security_group_rule` (optional) -- `aws_secretsmanager_secret` (optional) - - -## Requirements - -| Name | Version | -|------|---------| -| [terraform](#requirement\_terraform) | >= 1.2 | -| [aws](#requirement\_aws) | >= 4.22 | -| [random](#requirement\_random) | >= 3.3 | - -## Providers - -| Name | Version | -|------|---------| -| [aws](#provider\_aws) | 4.52.0 | -| [random](#provider\_random) | 3.4.3 | - -## Modules - -| Name | Source | Version | -|------|--------|---------| -| [resource\_group](#module\_resource\_group) | tedilabs/misc/aws//modules/resource-group | ~> 0.10.0 | -| [secret](#module\_secret) | tedilabs/secret/aws//modules/secrets-manager-secret | ~> 0.2.0 | -| [security\_group](#module\_security\_group) | tedilabs/network/aws//modules/security-group | 0.26.0 | - -## Resources - -| Name | Type | -|------|------| -| [aws_msk_cluster.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/msk_cluster) | resource | -| [aws_msk_configuration.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/msk_configuration) | resource | -| [aws_msk_scram_secret_association.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/msk_scram_secret_association) | resource | -| [random_password.this](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) | resource | -| [aws_msk_broker_nodes.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/msk_broker_nodes) | data source | -| [aws_subnet.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnet) | data source | - -## Inputs - -| Name | Description | Type | Default | Required | -|------|-------------|------|---------|:--------:| -| [broker\_size](#input\_broker\_size) | (Required) The desired total number of broker nodes in the kafka cluster. It must be a multiple of the number of specified client subnets. | `number` | n/a | yes | -| [broker\_subnets](#input\_broker\_subnets) | (Required) A list of subnet IDs to place ENIs of the MSK cluster broker nodes within. | `list(string)` | n/a | yes | -| [name](#input\_name) | (Required) Name of the MSK cluster. | `string` | n/a | yes | -| [auth\_sasl\_iam\_enabled](#input\_auth\_sasl\_iam\_enabled) | (Optional) Enables IAM client authentication. | `bool` | `false` | no | -| [auth\_sasl\_scram\_enabled](#input\_auth\_sasl\_scram\_enabled) | (Optional) Enables SCRAM client authentication via AWS Secrets Manager. | `bool` | `false` | no | -| [auth\_sasl\_scram\_kms\_key](#input\_auth\_sasl\_scram\_kms\_key) | (Optional) The ARN of a KMS key to encrypt AWS SeecretsManager Secret resources for storing SASL/SCRAM authentication data. Only required when the MSK cluster has SASL/SCRAM authentication enabled. The Username/Password Authentication based on SASL/SCRAM needs to create a Secret resource in AWS SecretsManager with a custom AWS KMS Key. A secret created with the default AWS KMS key cannot be used with an Amazon MSK cluster. | `string` | `null` | no | -| [auth\_sasl\_scram\_users](#input\_auth\_sasl\_scram\_users) | (Optional) A list of usernames to be allowed for SASL/SCRAM authentication to the MSK cluster. The password for each username is randomly generated and stored in AWS SecretsManager secret. | `set(string)` | `[]` | no | -| [auth\_tls\_acm\_ca\_arns](#input\_auth\_tls\_acm\_ca\_arns) | (Optional) List of ACM Certificate Authority Amazon Resource Names (ARNs). | `list(string)` | `[]` | no | -| [auth\_tls\_enabled](#input\_auth\_tls\_enabled) | (Optional) Enables TLS client authentication. | `bool` | `false` | no | -| [auth\_unauthenticated\_access\_enabled](#input\_auth\_unauthenticated\_access\_enabled) | (Optional) Enables unauthenticated access. Defaults to `true`. | `bool` | `true` | no | -| [broker\_additional\_security\_groups](#input\_broker\_additional\_security\_groups) | (Optional) A list of security group IDs to associate with ENIs to control who can communicate with the cluster. | `list(string)` | `[]` | no | -| [broker\_allowed\_ingress\_cidrs](#input\_broker\_allowed\_ingress\_cidrs) | (Optional) A list of CIDR for MSK ingress access. | `list(string)` | `[]` | no | -| [broker\_instance\_type](#input\_broker\_instance\_type) | (Optional) The instance type to use for the kafka brokers. | `string` | `"kafka.m5.large"` | no | -| [broker\_public\_access\_enabled](#input\_broker\_public\_access\_enabled) | (Optional) Whether to allow public access to MSK brokers. | `bool` | `false` | no | -| [broker\_volume\_provisioned\_throughput](#input\_broker\_volume\_provisioned\_throughput) | (Optional) Throughput value of the EBS volumes for the data drive on each kafka broker node in MiB per second. The minimum value is `250`. The maximum value varies between broker type. | `number` | `null` | no | -| [broker\_volume\_provisioned\_throughput\_enabled](#input\_broker\_volume\_provisioned\_throughput\_enabled) | (Optional) Whether provisioned throughput is enabled or not. You can specify the provisioned throughput rate in MiB per second for clusters whose brokers are of type `kafka.m5.4xlarge` or larger and if the storage volume is 10 GiB or greater. Defaults to `false`. | `bool` | `false` | no | -| [broker\_volume\_size](#input\_broker\_volume\_size) | (Optional) The size in GiB of the EBS volume for the data drive on each broker node. Minimum value of `1` and maximum value of `16384`. Defaults to `1000`. | `number` | `1000` | no | -| [encryption\_at\_rest\_kms\_key](#input\_encryption\_at\_rest\_kms\_key) | (Optional) Specify a KMS key short ID or ARN (it will always output an ARN) to use for encrypting your data at rest. If no key is specified, an AWS managed KMS ('aws/msk' managed service) key will be used for encrypting the data at rest. | `string` | `""` | no | -| [encryption\_in\_transit\_client\_mode](#input\_encryption\_in\_transit\_client\_mode) | (Optional) Encryption setting for data in transit between clients and brokers. `TLS`, `TLS_PLAINTEXT`, `PLAINTEXT` are available. | `string` | `"TLS_PLAINTEXT"` | no | -| [encryption\_in\_transit\_in\_cluster\_enabled](#input\_encryption\_in\_transit\_in\_cluster\_enabled) | (Optional) Whether data communication among broker nodes is encrypted. | `bool` | `true` | no | -| [kafka\_server\_properties](#input\_kafka\_server\_properties) | (Optional) Contents of the `server.properties` file for configuration of Kafka. | `map(string)` | `{}` | no | -| [kafka\_version](#input\_kafka\_version) | (Optional) Kafka version to use for the MSK cluster. | `string` | `"2.8.0"` | no | -| [logging\_cloudwatch\_enabled](#input\_logging\_cloudwatch\_enabled) | (Optional) Indicates whether you want to enable or disable streaming broker logs to Cloudwatch Logs. | `bool` | `false` | no | -| [logging\_cloudwatch\_log\_group](#input\_logging\_cloudwatch\_log\_group) | (Optional) The name of log group on CloudWatch Logs to deliver logs to. | `string` | `""` | no | -| [logging\_firehose\_delivery\_stream](#input\_logging\_firehose\_delivery\_stream) | (Optional) Name of the Kinesis Data Firehose delivery stream to deliver logs to. | `string` | `""` | no | -| [logging\_firehose\_enabled](#input\_logging\_firehose\_enabled) | (Optional) Indicates whether you want to enable or disable streaming broker logs to Kinesis Data Firehose. | `bool` | `false` | no | -| [logging\_s3\_bucket](#input\_logging\_s3\_bucket) | (Optional) The name of the S3 bucket to deliver logs to. | `string` | `""` | no | -| [logging\_s3\_enabled](#input\_logging\_s3\_enabled) | (Optional) Indicates whether you want to enable or disable streaming broker logs to S3. | `bool` | `false` | no | -| [logging\_s3\_prefix](#input\_logging\_s3\_prefix) | (Optional) The prefix to append to the folder name. | `string` | `""` | no | -| [module\_tags\_enabled](#input\_module\_tags\_enabled) | (Optional) Whether to create AWS Resource Tags for the module informations. | `bool` | `true` | no | -| [monitoring\_cloudwatch\_level](#input\_monitoring\_cloudwatch\_level) | (Optional) The desired enhanced MSK CloudWatch monitoring level. `DEFAULT`, `PER_BROKER`, `PER_TOPIC_PER_BROKER`, `PER_TOPIC_PER_PARTITION` are available. | `string` | `"DEFAULT"` | no | -| [monitoring\_prometheus\_jmx\_exporter\_enabled](#input\_monitoring\_prometheus\_jmx\_exporter\_enabled) | (Optional) Indicates whether you want to enable or disable the JMX Exporter. | `bool` | `false` | no | -| [monitoring\_prometheus\_node\_exporter\_enabled](#input\_monitoring\_prometheus\_node\_exporter\_enabled) | (Optional) Indicates whether you want to enable or disable the Node Exporter. | `bool` | `false` | no | -| [resource\_group\_description](#input\_resource\_group\_description) | (Optional) The description of Resource Group. | `string` | `"Managed by Terraform."` | no | -| [resource\_group\_enabled](#input\_resource\_group\_enabled) | (Optional) Whether to create Resource Group to find and group AWS resources which are created by this module. | `bool` | `true` | no | -| [resource\_group\_name](#input\_resource\_group\_name) | (Optional) The name of Resource Group. A Resource Group name can have a maximum of 127 characters, including letters, numbers, hyphens, dots, and underscores. The name cannot start with `AWS` or `aws`. | `string` | `""` | no | -| [tags](#input\_tags) | (Optional) A map of tags to add to all resources. | `map(string)` | `{}` | no | -| [timeouts](#input\_timeouts) | (Optional) How long to wait for the MSK cluster to be created/updated/deleted. | `map(string)` | 
{
  "create": "120m",
  "delete": "120m",
  "update": "120m"
}
| no | - -## Outputs - -| Name | Description | -|------|-------------| -| [arn](#output\_arn) | The ARN of the MSK cluster. | -| [auth](#output\_auth) | A configuration for authentication of the Kafka cluster. | -| [bootstrap\_brokers](#output\_bootstrap\_brokers) | A configuration for connecting to the Kafka cluster.
`plaintext` - A comma separated list of one or more hostname:port pairs of kafka brokers suitable to boostrap connectivity to the kafka cluster. Only contains value if `client_encryption_in_transit_mode` is set to PLAINTEXT or TLS\_PLAINTEXT. AWS may not always return all endpoints so the values may not be stable across applies.
`sasl_iam` - A comma separated list of one or more DNS names (or IPs) and SASL IAM port pairs. Only contains value if `client_encryption_in_transit_mode` is set to TLS\_PLAINTEXT or TLS. AWS may not always return all endpoints so the values may not be stable across applies.
`sasl_scram` - A comma separated list of one or more DNS names (or IPs) and SASL SCRAM port pairs. Only contains value if `client_encryption_in_transit_mode` is set to TLS\_PLAINTEXT or TLS. AWS may not always return all endpoints so the values may not be stable across applies.
`tls` - A comma separated list of one or more DNS names (or IPs) and TLS port pairs kafka brokers suitable to boostrap connectivity to the kafka cluster. Only contains value if `client_encryption_in_transit_mode is set to TLS_PLAINTEXT or TLS. AWS may not always return all endpoints so the values may not be stable across applies.
`public\_sasl\_iam` - A comma separated list of one or more DNS names (or IPs) and SASL IAM port pairs. Only contains value if `client\_encryption\_in\_transit\_mode` is set to TLS_PLAINTEXT or TLS and `auth\_sasl\_iam\_enabled` is `true` and `broker\_public\_access\_enabled` is `true`. AWS may not always return all endpoints so the values may not be stable across applies.
`public\_sasl\_scram` - A comma separated list of one or more DNS names (or IPs) and SASL SCRAM port pairs. Only contains value if `client\_encryption\_in\_transit\_mode` is set to TLS_PLAINTEXT or TLS and `auth\_sasl\_scram\_enabled` is `true` and `broker\_public\_access\_enabled` is `true`. AWS may not always return all endpoints so the values may not be stable across applies.
`public\_tls` - A comma separated list of one or more DNS names (or IPs) and TLS port pairs. Only contains value if `client\_encryption\_in\_transit\_mode` is set to TLS_PLAINTEXT or TLS and `broker\_public\_access\_enabled` is `true`. AWS may not always return all endpoints so the values may not be stable across applies.
` | -| [broker](#output\_broker) | A configuration for brokers of the Kafka cluster.
`size` - The number of broker nodes in the kafka cluster.
`instance_type` - The instance type used by the kafka brokers.

`public_access_enabled` - Whether public access to MSK brokers is enabled.
`security_groups` - A list of the security groups associated with the MSK cluster.

`volume` - A EBS volume information for MSK brokers. | -| [broker\_nodes](#output\_broker\_nodes) | The information of broker nodes in the kafka cluster. | -| [broker\_security\_group\_id](#output\_broker\_security\_group\_id) | The id of security group that were created for the MSK cluster. | -| [encryption](#output\_encryption) | A configuration for encryption of the Kafka cluster.
`at_rest` - The configuration for encryption at rest.
`in_transit` - The configuration for encryption in transit. | -| [kafka\_config](#output\_kafka\_config) | The MSK configuration. | -| [kafka\_version](#output\_kafka\_version) | The MSK cluster version. | -| [logging](#output\_logging) | A configuration for logging of the Kafka cluster.
`cloudwatch` - The configuration for MSK broker logs to CloudWatch Logs.
`firehose` - The configuration for MSK broker logs to Kinesis Firehose.
`s3` - The configuration for MSK broker logs to S3 Bucket. | -| [monitoring](#output\_monitoring) | A configuration for monitoring of the Kafka cluster.
`cloudwatch` - The configuration for MSK CloudWatch Metrics.
`prometheus` - The configuration for Prometheus open monitoring. | -| [name](#output\_name) | The MSK cluster name. | -| [version](#output\_version) | Current version of the MSK Cluster used for updates. | -| [zookeeper\_connections](#output\_zookeeper\_connections) | A configuration for connecting to the Apache Zookeeper cluster.
`tcp` - A comma separated list of one or more IP:port pairs to use to connect to the Apache Zookeeper cluster.
`tls` - A comma separated list of one or more IP:port pairs to use to connect to the Apache Zookeeper cluster via TLS. | - diff --git a/modules/msk-cluster/cluster.tf b/modules/msk-cluster/cluster.tf deleted file mode 100644 index 0d83526..0000000 --- a/modules/msk-cluster/cluster.tf +++ /dev/null @@ -1,172 +0,0 @@ -locals { - metadata = { - package = "terraform-aws-misc" - version = trimspace(file("${path.module}/../../VERSION")) - module = basename(path.module) - name = var.name - } - module_tags = var.module_tags_enabled ? { - "module.terraform.io/package" = local.metadata.package - "module.terraform.io/version" = local.metadata.version - "module.terraform.io/name" = local.metadata.module - "module.terraform.io/full-name" = "${local.metadata.package}/${local.metadata.module}" - "module.terraform.io/instance" = local.metadata.name - } : {} -} - - -################################################### -# Configuration for MSK Cluster -################################################### - -locals { - server_properties = < 0 ? 1 : 0 - - cluster_arn = aws_msk_cluster.this.arn - secret_arn_list = values(module.secret).*.arn -} diff --git a/modules/msk-cluster/security-group.tf b/modules/msk-cluster/security-group.tf deleted file mode 100644 index be49bf1..0000000 --- a/modules/msk-cluster/security-group.tf +++ /dev/null @@ -1,124 +0,0 @@ -data "aws_subnet" "this" { - id = var.broker_subnets[0] -} - -locals { - vpc_id = data.aws_subnet.this.vpc_id -} - - -################################################### -# Security Group -################################################### - -module "security_group" { - source = "tedilabs/network/aws//modules/security-group" - version = "0.26.0" - - count = length(var.broker_allowed_ingress_cidrs) > 0 ? 1 : 0 - - name = var.name - description = "Security group for MSK Cluster." - vpc_id = local.vpc_id - - ingress_rules = [ - { - id = "broker-plaintext/cidrs" - description = "Allow CIDRs to communicate with Kafka brokers in plaintext." - protocol = "tcp" - from_port = 9092 - to_port = 9092 - - cidr_blocks = var.broker_allowed_ingress_cidrs - }, - { - id = "broker-tls/cidrs" - description = "Allow CIDRs to communicate with Kafka brokers in tls." - protocol = "tcp" - from_port = 9094 - to_port = 9094 - - cidr_blocks = var.broker_allowed_ingress_cidrs - }, - { - id = "broker-sasl-scram/cidrs" - description = "Allow CIDRs to communicate with Kafka brokers in SASL SCRAM." - protocol = "tcp" - from_port = 9096 - to_port = 9096 - - cidr_blocks = var.broker_allowed_ingress_cidrs - }, - { - id = "broker-sasl-iam/cidrs" - description = "Allow CIDRs to communicate with Kafka brokers in SASL IAM." - protocol = "tcp" - from_port = 9098 - to_port = 9098 - - cidr_blocks = var.broker_allowed_ingress_cidrs - }, - { - id = "broker-public-tls/cidrs" - description = "Allow CIDRs to communicate with Kafka brokers in tls (public)." - protocol = "tcp" - from_port = 9194 - to_port = 9194 - - cidr_blocks = var.broker_allowed_ingress_cidrs - }, - { - id = "broker-public-sasl-scram/cidrs" - description = "Allow CIDRs to communicate with Kafka brokers in SASL SCRAM (public)." - protocol = "tcp" - from_port = 9196 - to_port = 9196 - - cidr_blocks = var.broker_allowed_ingress_cidrs - }, - { - id = "broker-public-sasl-iam/cidrs" - description = "Allow CIDRs to communicate with Kafka brokers in SASL IAM (public)." - protocol = "tcp" - from_port = 9198 - to_port = 9198 - - cidr_blocks = var.broker_allowed_ingress_cidrs - }, - { - id = "zookeeper/cidrs" - description = "Allow CIDRs to communicate with Kafka zookeepers." - protocol = "tcp" - from_port = 2181 - to_port = 2181 - - cidr_blocks = var.broker_allowed_ingress_cidrs - }, - { - id = "prometheus-jmx-exporter/cidrs" - description = "Allow CIDRs to communicate with Prometheus JMX Exporter." - protocol = "tcp" - from_port = 11001 - to_port = 11001 - - cidr_blocks = var.broker_allowed_ingress_cidrs - }, - { - id = "prometheus-node-exporter/cidrs" - description = "Allow CIDRs to communicate with Prometheus Node Exporter." - protocol = "tcp" - from_port = 11002 - to_port = 11002 - - cidr_blocks = var.broker_allowed_ingress_cidrs - }, - ] - - resource_group_enabled = false - module_tags_enabled = false - - tags = merge( - local.module_tags, - var.tags, - ) -} diff --git a/modules/msk-cluster/variables.tf b/modules/msk-cluster/variables.tf deleted file mode 100644 index 634de59..0000000 --- a/modules/msk-cluster/variables.tf +++ /dev/null @@ -1,278 +0,0 @@ -variable "name" { - description = "(Required) Name of the MSK cluster." - type = string -} - -variable "kafka_version" { - description = "(Optional) Kafka version to use for the MSK cluster." - type = string - default = "2.8.0" - nullable = false -} - -variable "kafka_server_properties" { - description = "(Optional) Contents of the `server.properties` file for configuration of Kafka." - type = map(string) - default = {} - nullable = false -} - -variable "broker_size" { - description = "(Required) The desired total number of broker nodes in the kafka cluster. It must be a multiple of the number of specified client subnets." - type = number -} - -variable "broker_instance_type" { - description = "(Optional) The instance type to use for the kafka brokers." - type = string - default = "kafka.m5.large" - nullable = false -} - -variable "broker_volume_size" { - description = "(Optional) The size in GiB of the EBS volume for the data drive on each broker node. Minimum value of `1` and maximum value of `16384`. Defaults to `1000`." - type = number - default = 1000 - nullable = false - - validation { - condition = alltrue([ - var.broker_volume_size >= 1, - var.broker_volume_size <= 16384, - ]) - error_message = "Valid value for `broker_volume_size` is between `1` and `16384`." - } -} - -variable "broker_volume_provisioned_throughput_enabled" { - description = "(Optional) Whether provisioned throughput is enabled or not. You can specify the provisioned throughput rate in MiB per second for clusters whose brokers are of type `kafka.m5.4xlarge` or larger and if the storage volume is 10 GiB or greater. Defaults to `false`." - type = bool - default = false - nullable = false -} - -variable "broker_volume_provisioned_throughput" { - description = "(Optional) Throughput value of the EBS volumes for the data drive on each kafka broker node in MiB per second. The minimum value is `250`. The maximum value varies between broker type." - type = number - default = null -} - -variable "broker_subnets" { - description = "(Required) A list of subnet IDs to place ENIs of the MSK cluster broker nodes within." - type = list(string) -} - -variable "broker_public_access_enabled" { - description = "(Optional) Whether to allow public access to MSK brokers." - type = bool - default = false - nullable = false -} - -variable "broker_allowed_ingress_cidrs" { - description = "(Optional) A list of CIDR for MSK ingress access." - type = list(string) - default = [] -} - -variable "broker_additional_security_groups" { - description = "(Optional) A list of security group IDs to associate with ENIs to control who can communicate with the cluster." - type = list(string) - default = [] -} - -variable "auth_unauthenticated_access_enabled" { - description = "(Optional) Enables unauthenticated access. Defaults to `true`." - type = bool - default = true - nullable = false -} - -variable "auth_sasl_iam_enabled" { - description = "(Optional) Enables IAM client authentication." - type = bool - default = false - nullable = false -} - -variable "auth_sasl_scram_enabled" { - description = "(Optional) Enables SCRAM client authentication via AWS Secrets Manager." - type = bool - default = false - nullable = false -} - -variable "auth_sasl_scram_kms_key" { - description = "(Optional) The ARN of a KMS key to encrypt AWS SeecretsManager Secret resources for storing SASL/SCRAM authentication data. Only required when the MSK cluster has SASL/SCRAM authentication enabled. The Username/Password Authentication based on SASL/SCRAM needs to create a Secret resource in AWS SecretsManager with a custom AWS KMS Key. A secret created with the default AWS KMS key cannot be used with an Amazon MSK cluster." - type = string - default = null -} - -variable "auth_sasl_scram_users" { - description = "(Optional) A list of usernames to be allowed for SASL/SCRAM authentication to the MSK cluster. The password for each username is randomly generated and stored in AWS SecretsManager secret." - type = set(string) - default = [] - nullable = false -} - -variable "auth_tls_enabled" { - description = "(Optional) Enables TLS client authentication." - type = bool - default = false - nullable = false -} - -variable "auth_tls_acm_ca_arns" { - description = "(Optional) List of ACM Certificate Authority Amazon Resource Names (ARNs)." - type = list(string) - default = [] - nullable = false -} - -variable "encryption_at_rest_kms_key" { - description = "(Optional) Specify a KMS key short ID or ARN (it will always output an ARN) to use for encrypting your data at rest. If no key is specified, an AWS managed KMS ('aws/msk' managed service) key will be used for encrypting the data at rest." - type = string - default = "" -} - -variable "encryption_in_transit_in_cluster_enabled" { - description = "(Optional) Whether data communication among broker nodes is encrypted." - type = bool - default = true - nullable = false -} - -variable "encryption_in_transit_client_mode" { - description = "(Optional) Encryption setting for data in transit between clients and brokers. `TLS`, `TLS_PLAINTEXT`, `PLAINTEXT` are available." - type = string - default = "TLS_PLAINTEXT" - nullable = false - - validation { - condition = contains(["TLS", "TLS_PLAINTEXT", "PLAINTEXT"], var.encryption_in_transit_client_mode) - error_message = "Valid values are `TLS`, `TLS_PLAINTEXT`, `PLAINTEXT`." - } -} - -variable "logging_cloudwatch_enabled" { - description = "(Optional) Indicates whether you want to enable or disable streaming broker logs to Cloudwatch Logs." - type = bool - default = false - nullable = false -} - -variable "logging_cloudwatch_log_group" { - description = "(Optional) The name of log group on CloudWatch Logs to deliver logs to." - type = string - default = "" - nullable = false -} - -variable "logging_firehose_enabled" { - description = "(Optional) Indicates whether you want to enable or disable streaming broker logs to Kinesis Data Firehose." - type = bool - default = false - nullable = false -} - -variable "logging_firehose_delivery_stream" { - description = "(Optional) Name of the Kinesis Data Firehose delivery stream to deliver logs to." - type = string - default = "" - nullable = false -} - -variable "logging_s3_enabled" { - description = "(Optional) Indicates whether you want to enable or disable streaming broker logs to S3." - type = bool - default = false - nullable = false -} - -variable "logging_s3_bucket" { - description = "(Optional) The name of the S3 bucket to deliver logs to." - type = string - default = "" - nullable = false -} - -variable "logging_s3_prefix" { - description = "(Optional) The prefix to append to the folder name." - type = string - default = "" - nullable = false -} - -variable "monitoring_cloudwatch_level" { - description = "(Optional) The desired enhanced MSK CloudWatch monitoring level. `DEFAULT`, `PER_BROKER`, `PER_TOPIC_PER_BROKER`, `PER_TOPIC_PER_PARTITION` are available." - type = string - default = "DEFAULT" - nullable = false - - validation { - condition = contains(["DEFAULT", "PER_BROKER", "PER_TOPIC_PER_BROKER", "PER_TOPIC_PER_PARTITION"], var.monitoring_cloudwatch_level) - error_message = "Valid values are `DEFAULT`, `PER_BROKER`, `PER_TOPIC_PER_BROKER`, `PER_TOPIC_PER_PARTITION`." - } -} - -variable "monitoring_prometheus_jmx_exporter_enabled" { - description = "(Optional) Indicates whether you want to enable or disable the JMX Exporter." - type = bool - default = false - nullable = false -} - -variable "monitoring_prometheus_node_exporter_enabled" { - description = "(Optional) Indicates whether you want to enable or disable the Node Exporter." - type = bool - default = false - nullable = false -} - -variable "timeouts" { - description = "(Optional) How long to wait for the MSK cluster to be created/updated/deleted." - type = map(string) - default = { - create = "120m" - update = "120m" - delete = "120m" - } - nullable = false -} - -variable "tags" { - description = "(Optional) A map of tags to add to all resources." - type = map(string) - default = {} - nullable = false -} - -variable "module_tags_enabled" { - description = "(Optional) Whether to create AWS Resource Tags for the module informations." - type = bool - default = true - nullable = false -} - - -################################################### -# Resource Group -################################################### - -variable "resource_group_enabled" { - description = "(Optional) Whether to create Resource Group to find and group AWS resources which are created by this module." - type = bool - default = true -} - -variable "resource_group_name" { - description = "(Optional) The name of Resource Group. A Resource Group name can have a maximum of 127 characters, including letters, numbers, hyphens, dots, and underscores. The name cannot start with `AWS` or `aws`." - type = string - default = "" -} - -variable "resource_group_description" { - description = "(Optional) The description of Resource Group." - type = string - default = "Managed by Terraform." -} diff --git a/modules/msk-cluster/versions.tf b/modules/msk-cluster/versions.tf deleted file mode 100644 index 583387c..0000000 --- a/modules/msk-cluster/versions.tf +++ /dev/null @@ -1,14 +0,0 @@ -terraform { - required_version = ">= 1.2" - - required_providers { - aws = { - source = "hashicorp/aws" - version = ">= 4.22" - } - random = { - source = "hashicorp/random" - version = ">= 3.3" - } - } -} From 99a48966be0866dc898a950663b70df427aabc5f Mon Sep 17 00:00:00 2001 From: "Byungjin Park (Claud)" Date: Sun, 3 Sep 2023 00:25:06 +0900 Subject: [PATCH 18/33] Update resource-group module to support optional attributes (#35) --- modules/resource-group/README.md | 6 +++--- modules/resource-group/main.tf | 4 ++-- modules/resource-group/outputs.tf | 4 ++-- modules/resource-group/variables.tf | 14 +++++++++++--- modules/resource-group/versions.tf | 2 +- 5 files changed, 19 insertions(+), 11 deletions(-) diff --git a/modules/resource-group/README.md b/modules/resource-group/README.md index 348fae5..5cb4712 100644 --- a/modules/resource-group/README.md +++ b/modules/resource-group/README.md @@ -9,14 +9,14 @@ This module creates following resources. | Name | Version | |------|---------| -| [terraform](#requirement\_terraform) | >= 1.1 | +| [terraform](#requirement\_terraform) | >= 1.5 | | [aws](#requirement\_aws) | >= 4.14 | ## Providers | Name | Version | |------|---------| -| [aws](#provider\_aws) | 4.16.0 | +| [aws](#provider\_aws) | 5.15.0 | ## Modules @@ -35,7 +35,7 @@ No modules. | [name](#input\_name) | (Required) A name to identify the resource group. A resource group name can have a maximum of 127 characters, including letters, numbers, hyphens, dots, and underscores. The name cannot start with `AWS` or `aws`. | `string` | n/a | yes | | [description](#input\_description) | (Optional) The description of the resource group. | `string` | `"Managed by Terraform."` | no | | [module\_tags\_enabled](#input\_module\_tags\_enabled) | (Optional) Whether to create AWS Resource Tags for the module informations. | `bool` | `true` | no | -| [query](#input\_query) | (Optional) A configuration for the actual query used to match against resources. It supports `resource_types` and `resource_tags`. `query` block as defined below.
(Required) `resource_tags` - A map of key/value pairs that are compared to the tags attached to resources.
(Optional) `resource_types` - A list of resource-type specification strings with `AWS::service-id::resource-type` format. Limit the results to only those resource types that match the filter. Specify `AWS::AllSupported` to include resources of any resources that are currently supported by Resource Group. | `any` | `{}` | no | +| [query](#input\_query) | (Optional) A configuration for the actual query used to match against resources. It supports `resource_types` and `resource_tags`. `query` block as defined below.
(Optional) `resource_tags` - A map of key/value pairs that are compared to the tags attached to resources.
(Optional) `resource_types` - A list of resource-type specification strings with `AWS::service-id::resource-type` format. Limit the results to only those resource types that match the filter. Specify `AWS::AllSupported` to include resources of any resources that are currently supported by Resource Group. | 
object({
    resource_tags  = optional(map(string), {})
    resource_types = optional(list(string), ["AWS::AllSupported"])
  })
| `{}` | no | | [tags](#input\_tags) | (Optional) A map of tags to add to all resources. | `map(string)` | `{}` | no | ## Outputs diff --git a/modules/resource-group/main.tf b/modules/resource-group/main.tf index 1efd8b3..fe9e009 100644 --- a/modules/resource-group/main.tf +++ b/modules/resource-group/main.tf @@ -21,14 +21,14 @@ locals { locals { filters = [ - for key, value in try(var.query.resource_tags, {}) : { + for key, value in var.query.resource_tags : { "Key" = key "Values" = flatten([value]) } ] query = <<-JSON { - "ResourceTypeFilters": ${jsonencode(try(var.query.resource_types, ["AWS::AllSupported"]))}, + "ResourceTypeFilters": ${jsonencode(var.query.resource_types)}, "TagFilters": ${jsonencode(local.filters)} } JSON diff --git a/modules/resource-group/outputs.tf b/modules/resource-group/outputs.tf index 1e0f6ea..4ddf8a8 100644 --- a/modules/resource-group/outputs.tf +++ b/modules/resource-group/outputs.tf @@ -15,10 +15,10 @@ output "description" { output "resource_types" { description = "The resource types used by the resource group to query resources." - value = try(var.query.resource_types, ["AWS::AllSupported"]) + value = var.query.resource_types } output "resource_tags" { description = "The resource tags used by the resource group to query resources." - value = try(var.query.resource_tags, {}) + value = var.query.resource_tags } diff --git a/modules/resource-group/variables.tf b/modules/resource-group/variables.tf index 6ae4201..70b82c2 100644 --- a/modules/resource-group/variables.tf +++ b/modules/resource-group/variables.tf @@ -1,32 +1,40 @@ variable "name" { description = "(Required) A name to identify the resource group. A resource group name can have a maximum of 127 characters, including letters, numbers, hyphens, dots, and underscores. The name cannot start with `AWS` or `aws`." type = string + nullable = false } variable "description" { description = "(Optional) The description of the resource group." type = string default = "Managed by Terraform." + nullable = false } variable "query" { description = < Date: Sun, 3 Sep 2023 00:25:30 +0900 Subject: [PATCH 19/33] Bump to v0.11.0 --- VERSION | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/VERSION b/VERSION index 5712157..d9df1bb 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.10.1 +0.11.0 From c09543d3647773920869fac872fe8b0b68cd093f Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 3 Oct 2023 00:11:06 +0900 Subject: [PATCH 20/33] Bump crazy-max/ghaction-github-labeler from 4 to 5 (#38) Bumps [crazy-max/ghaction-github-labeler](https://github.com/crazy-max/ghaction-github-labeler) from 4 to 5. - [Release notes](https://github.com/crazy-max/ghaction-github-labeler/releases) - [Commits](https://github.com/crazy-max/ghaction-github-labeler/compare/v4...v5) --- updated-dependencies: - dependency-name: crazy-max/ghaction-github-labeler dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/sync-labels.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/sync-labels.yaml b/.github/workflows/sync-labels.yaml index 69a5674..82a9718 100644 --- a/.github/workflows/sync-labels.yaml +++ b/.github/workflows/sync-labels.yaml @@ -17,7 +17,7 @@ jobs: uses: actions/checkout@v3 - name: Sync labels - uses: crazy-max/ghaction-github-labeler@v4 + uses: crazy-max/ghaction-github-labeler@v5 with: github-token: ${{ secrets.GITHUB_TOKEN }} yaml-file: .github/labels.yaml From c77ae9ef377ecc38420365c644a9d7d2a36a3813 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 2 Oct 2023 15:13:04 +0000 Subject: [PATCH 21/33] Bump tj-actions/changed-files from 38 to 39 (#37) --- .github/workflows/terraform.integration.yaml | 4 ++-- .github/workflows/yaml.integration.yaml | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/terraform.integration.yaml b/.github/workflows/terraform.integration.yaml index 8cfe55e..5c447f0 100644 --- a/.github/workflows/terraform.integration.yaml +++ b/.github/workflows/terraform.integration.yaml @@ -31,7 +31,7 @@ jobs: - name: Get Changed Files id: changed-files - uses: tj-actions/changed-files@v38 + uses: tj-actions/changed-files@v39 with: files: | modules/** @@ -40,7 +40,7 @@ jobs: - name: Get Changed Directories id: changed-directories - uses: tj-actions/changed-files@v38 + uses: tj-actions/changed-files@v39 with: files: | modules/** diff --git a/.github/workflows/yaml.integration.yaml b/.github/workflows/yaml.integration.yaml index 5149a2d..9d935b3 100644 --- a/.github/workflows/yaml.integration.yaml +++ b/.github/workflows/yaml.integration.yaml @@ -29,7 +29,7 @@ jobs: - name: Get Changed Files id: changed-files - uses: tj-actions/changed-files@v38 + uses: tj-actions/changed-files@v39 with: files: | **/*.yaml From 73beab0823b9717a37794329a2973e4db4451e6e Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 2 Oct 2023 15:15:35 +0000 Subject: [PATCH 22/33] Bump actions/checkout from 3 to 4 (#36) --- .github/workflows/sync-labels.yaml | 2 +- .github/workflows/terraform.integration.yaml | 2 +- .github/workflows/yaml.integration.yaml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/sync-labels.yaml b/.github/workflows/sync-labels.yaml index 82a9718..40825ce 100644 --- a/.github/workflows/sync-labels.yaml +++ b/.github/workflows/sync-labels.yaml @@ -14,7 +14,7 @@ jobs: steps: - name: Checkout - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Sync labels uses: crazy-max/ghaction-github-labeler@v5 diff --git a/.github/workflows/terraform.integration.yaml b/.github/workflows/terraform.integration.yaml index 5c447f0..ef58145 100644 --- a/.github/workflows/terraform.integration.yaml +++ b/.github/workflows/terraform.integration.yaml @@ -25,7 +25,7 @@ jobs: steps: - name: Checkout - uses: actions/checkout@v3 + uses: actions/checkout@v4 with: fetch-depth: 0 diff --git a/.github/workflows/yaml.integration.yaml b/.github/workflows/yaml.integration.yaml index 9d935b3..c2233c0 100644 --- a/.github/workflows/yaml.integration.yaml +++ b/.github/workflows/yaml.integration.yaml @@ -23,7 +23,7 @@ jobs: steps: - name: Checkout - uses: actions/checkout@v3 + uses: actions/checkout@v4 with: fetch-depth: 0 From 65b3b5e982abba7ace093d2e604e55b01c24c43e Mon Sep 17 00:00:00 2001 From: "Byungjin Park (Claud)" Date: Tue, 3 Oct 2023 19:08:10 +0900 Subject: [PATCH 23/33] Update terraform version to v1.5 (#39) --- .pre-commit-config.yaml | 5 ++++- modules/s3-archive-bucket/README.md | 4 ++-- modules/s3-archive-bucket/versions.tf | 2 +- modules/sqs-aws-event-queue/README.md | 4 ++-- modules/sqs-aws-event-queue/versions.tf | 2 +- 5 files changed, 10 insertions(+), 7 deletions(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 10bde48..8f8a374 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,6 +1,6 @@ repos: - repo: https://github.com/antonbabenko/pre-commit-terraform - rev: v1.81.0 + rev: v1.83.4 hooks: - id: terraform_fmt args: @@ -9,6 +9,9 @@ repos: args: - --tf-init-args=-upgrade - --hook-config=--retry-once-with-cleanup=true + - id: terraform_providers_lock + args: + - --hook-config=--mode=always-regenerate-lockfile - id: terraform_tflint args: - --args=--config=__GIT_WORKING_DIR__/.tflint.hcl diff --git a/modules/s3-archive-bucket/README.md b/modules/s3-archive-bucket/README.md index 53eacb9..72334ae 100644 --- a/modules/s3-archive-bucket/README.md +++ b/modules/s3-archive-bucket/README.md @@ -18,14 +18,14 @@ This module creates following resources. | Name | Version | |------|---------| -| [terraform](#requirement\_terraform) | >= 1.2 | +| [terraform](#requirement\_terraform) | >= 1.5 | | [aws](#requirement\_aws) | >= 4.16 | ## Providers | Name | Version | |------|---------| -| [aws](#provider\_aws) | 4.52.0 | +| [aws](#provider\_aws) | 5.19.0 | ## Modules diff --git a/modules/s3-archive-bucket/versions.tf b/modules/s3-archive-bucket/versions.tf index 5242434..951ec4f 100644 --- a/modules/s3-archive-bucket/versions.tf +++ b/modules/s3-archive-bucket/versions.tf @@ -1,5 +1,5 @@ terraform { - required_version = ">= 1.2" + required_version = ">= 1.5" required_providers { aws = { diff --git a/modules/sqs-aws-event-queue/README.md b/modules/sqs-aws-event-queue/README.md index 954be30..bc97421 100644 --- a/modules/sqs-aws-event-queue/README.md +++ b/modules/sqs-aws-event-queue/README.md @@ -10,14 +10,14 @@ This module creates following resources. | Name | Version | |------|---------| -| [terraform](#requirement\_terraform) | >= 1.2 | +| [terraform](#requirement\_terraform) | >= 1.5 | | [aws](#requirement\_aws) | >= 3.45 | ## Providers | Name | Version | |------|---------| -| [aws](#provider\_aws) | 4.52.0 | +| [aws](#provider\_aws) | 5.19.0 | ## Modules diff --git a/modules/sqs-aws-event-queue/versions.tf b/modules/sqs-aws-event-queue/versions.tf index dd959fc..9ae3d78 100644 --- a/modules/sqs-aws-event-queue/versions.tf +++ b/modules/sqs-aws-event-queue/versions.tf @@ -1,5 +1,5 @@ terraform { - required_version = ">= 1.2" + required_version = ">= 1.5" required_providers { aws = { From 6568b8e0ae729148a0587d5227d98add81635ab3 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 1 Nov 2023 02:04:34 +0900 Subject: [PATCH 24/33] Bump tj-actions/changed-files from 39 to 40 (#40) Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 39 to 40. - [Release notes](https://github.com/tj-actions/changed-files/releases) - [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md) - [Commits](https://github.com/tj-actions/changed-files/compare/v39...v40) --- updated-dependencies: - dependency-name: tj-actions/changed-files dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/terraform.integration.yaml | 4 ++-- .github/workflows/yaml.integration.yaml | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/terraform.integration.yaml b/.github/workflows/terraform.integration.yaml index ef58145..92b2077 100644 --- a/.github/workflows/terraform.integration.yaml +++ b/.github/workflows/terraform.integration.yaml @@ -31,7 +31,7 @@ jobs: - name: Get Changed Files id: changed-files - uses: tj-actions/changed-files@v39 + uses: tj-actions/changed-files@v40 with: files: | modules/** @@ -40,7 +40,7 @@ jobs: - name: Get Changed Directories id: changed-directories - uses: tj-actions/changed-files@v39 + uses: tj-actions/changed-files@v40 with: files: | modules/** diff --git a/.github/workflows/yaml.integration.yaml b/.github/workflows/yaml.integration.yaml index c2233c0..884f3ee 100644 --- a/.github/workflows/yaml.integration.yaml +++ b/.github/workflows/yaml.integration.yaml @@ -29,7 +29,7 @@ jobs: - name: Get Changed Files id: changed-files - uses: tj-actions/changed-files@v39 + uses: tj-actions/changed-files@v40 with: files: | **/*.yaml From a0045d2d44e017d3c8e2ad77dcc3dedffb80ac21 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 26 Dec 2023 02:33:10 +0000 Subject: [PATCH 25/33] Bump tj-actions/changed-files from 40 to 41 (#42) --- .github/workflows/terraform.integration.yaml | 4 ++-- .github/workflows/yaml.integration.yaml | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/terraform.integration.yaml b/.github/workflows/terraform.integration.yaml index 92b2077..fd6ca9f 100644 --- a/.github/workflows/terraform.integration.yaml +++ b/.github/workflows/terraform.integration.yaml @@ -31,7 +31,7 @@ jobs: - name: Get Changed Files id: changed-files - uses: tj-actions/changed-files@v40 + uses: tj-actions/changed-files@v41 with: files: | modules/** @@ -40,7 +40,7 @@ jobs: - name: Get Changed Directories id: changed-directories - uses: tj-actions/changed-files@v40 + uses: tj-actions/changed-files@v41 with: files: | modules/** diff --git a/.github/workflows/yaml.integration.yaml b/.github/workflows/yaml.integration.yaml index 884f3ee..52c19f9 100644 --- a/.github/workflows/yaml.integration.yaml +++ b/.github/workflows/yaml.integration.yaml @@ -29,7 +29,7 @@ jobs: - name: Get Changed Files id: changed-files - uses: tj-actions/changed-files@v40 + uses: tj-actions/changed-files@v41 with: files: | **/*.yaml From 902138b8c1f9ac9e78e32fb9bd1845f0c15963e2 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Sun, 17 Mar 2024 13:11:40 +0000 Subject: [PATCH 26/33] Bump tj-actions/changed-files from 41 to 43 (#44) --- .github/workflows/terraform.integration.yaml | 4 ++-- .github/workflows/yaml.integration.yaml | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/terraform.integration.yaml b/.github/workflows/terraform.integration.yaml index fd6ca9f..20f5bd6 100644 --- a/.github/workflows/terraform.integration.yaml +++ b/.github/workflows/terraform.integration.yaml @@ -31,7 +31,7 @@ jobs: - name: Get Changed Files id: changed-files - uses: tj-actions/changed-files@v41 + uses: tj-actions/changed-files@v43 with: files: | modules/** @@ -40,7 +40,7 @@ jobs: - name: Get Changed Directories id: changed-directories - uses: tj-actions/changed-files@v41 + uses: tj-actions/changed-files@v43 with: files: | modules/** diff --git a/.github/workflows/yaml.integration.yaml b/.github/workflows/yaml.integration.yaml index 52c19f9..b1f80a3 100644 --- a/.github/workflows/yaml.integration.yaml +++ b/.github/workflows/yaml.integration.yaml @@ -29,7 +29,7 @@ jobs: - name: Get Changed Files id: changed-files - uses: tj-actions/changed-files@v41 + uses: tj-actions/changed-files@v43 with: files: | **/*.yaml From 61e1197f501aca6d4bcf8151acd1d36a7f3c3895 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Sun, 31 Mar 2024 10:56:27 +0000 Subject: [PATCH 27/33] Bump tj-actions/changed-files from 43 to 44 (#45) --- .github/workflows/terraform.integration.yaml | 4 ++-- .github/workflows/yaml.integration.yaml | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/terraform.integration.yaml b/.github/workflows/terraform.integration.yaml index 20f5bd6..daca2fa 100644 --- a/.github/workflows/terraform.integration.yaml +++ b/.github/workflows/terraform.integration.yaml @@ -31,7 +31,7 @@ jobs: - name: Get Changed Files id: changed-files - uses: tj-actions/changed-files@v43 + uses: tj-actions/changed-files@v44 with: files: | modules/** @@ -40,7 +40,7 @@ jobs: - name: Get Changed Directories id: changed-directories - uses: tj-actions/changed-files@v43 + uses: tj-actions/changed-files@v44 with: files: | modules/** diff --git a/.github/workflows/yaml.integration.yaml b/.github/workflows/yaml.integration.yaml index b1f80a3..fc933f2 100644 --- a/.github/workflows/yaml.integration.yaml +++ b/.github/workflows/yaml.integration.yaml @@ -29,7 +29,7 @@ jobs: - name: Get Changed Files id: changed-files - uses: tj-actions/changed-files@v43 + uses: tj-actions/changed-files@v44 with: files: | **/*.yaml From f263c97ec0d067baa948f993e6a20f76614c1566 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 12 Apr 2024 00:15:14 +0900 Subject: [PATCH 28/33] Bump actions/labeler from 4 to 5 (#41) Bumps [actions/labeler](https://github.com/actions/labeler) from 4 to 5. - [Release notes](https://github.com/actions/labeler/releases) - [Commits](https://github.com/actions/labeler/compare/v4...v5) --- updated-dependencies: - dependency-name: actions/labeler dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/pull-request-labeler.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/pull-request-labeler.yaml b/.github/workflows/pull-request-labeler.yaml index 1784c72..d5ac851 100644 --- a/.github/workflows/pull-request-labeler.yaml +++ b/.github/workflows/pull-request-labeler.yaml @@ -9,7 +9,7 @@ jobs: steps: - name: Add Labels for PR - uses: actions/labeler@v4 + uses: actions/labeler@v5 with: repo-token: "${{ secrets.GITHUB_TOKEN }}" configuration-path: .github/labeler.yaml From dc0c22e496811493d966def45895e51cd05ab28d Mon Sep 17 00:00:00 2001 From: Byungjin Park Date: Fri, 3 May 2024 02:18:51 +0900 Subject: [PATCH 29/33] Update dependabot --- .github/dependabot.yml | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 759819d..a08a33c 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -1,6 +1,14 @@ version: 2 updates: -- package-ecosystem: "github-actions" - directory: "/" +- package-ecosystem: github-actions + directory: / schedule: - interval: "daily" + interval: daily + +- package-ecosystem: terraform + directories: + - /modules/resource-group + - /modules/s3-archive-bucket + - /modules/sqs-aws-event-queue + schedule: + interval: weekly From 0aba88403cb71595a97b32f75de17b1edc21dd58 Mon Sep 17 00:00:00 2001 From: Byungjin Park Date: Mon, 27 May 2024 02:02:34 +0900 Subject: [PATCH 30/33] Fix labeler action --- .github/labeler.yaml | 12 +++++++++--- .github/workflows/pull-request-labeler.yaml | 5 +++++ 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/.github/labeler.yaml b/.github/labeler.yaml index 3977b4a..7fec752 100644 --- a/.github/labeler.yaml +++ b/.github/labeler.yaml @@ -1,9 +1,15 @@ # Modules ":floppy_disk: resource-group": -- modules/resource-group/**/* +- changed-files: + - any-glob-to-any-file: + - modules/resource-group/**/* ":floppy_disk: s3-archive-bucket": -- modules/s3-archive-bucket/**/* +- changed-files: + - any-glob-to-any-file: + - modules/s3-archive-bucket/**/* ":floppy_disk: sqs-aws-event-queue": -- modules/sqs-aws-event-queue/**/* +- changed-files: + - any-glob-to-any-file: + - modules/sqs-aws-event-queue/**/* diff --git a/.github/workflows/pull-request-labeler.yaml b/.github/workflows/pull-request-labeler.yaml index d5ac851..aac2537 100644 --- a/.github/workflows/pull-request-labeler.yaml +++ b/.github/workflows/pull-request-labeler.yaml @@ -7,12 +7,17 @@ jobs: label-pr: runs-on: ubuntu-latest + permissions: + contents: read + pull-requests: write + steps: - name: Add Labels for PR uses: actions/labeler@v5 with: repo-token: "${{ secrets.GITHUB_TOKEN }}" configuration-path: .github/labeler.yaml + dot: true sync-labels: true - name: Add PR Size Labels for PR From 93b66cdcfbb19a1a65c2211aac178c4f99c616e5 Mon Sep 17 00:00:00 2001 From: Byungjin Park Date: Fri, 28 Jun 2024 14:43:08 +0900 Subject: [PATCH 31/33] Update dependabot --- .github/dependabot.yml | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/.github/dependabot.yml b/.github/dependabot.yml index a08a33c..2f2d7cb 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -7,8 +7,6 @@ updates: - package-ecosystem: terraform directories: - - /modules/resource-group - - /modules/s3-archive-bucket - - /modules/sqs-aws-event-queue + - /modules/* schedule: interval: weekly From 0a7cdaef423c77cb417edae9833f51992e63461e Mon Sep 17 00:00:00 2001 From: Byungjin Park Date: Wed, 19 Mar 2025 21:26:27 +0900 Subject: [PATCH 32/33] Update lint configurations --- .editorconfig | 2 +- .pre-commit-config.yaml | 4 ++-- .tflint.hcl | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.editorconfig b/.editorconfig index 8289548..d7d4a5f 100644 --- a/.editorconfig +++ b/.editorconfig @@ -8,7 +8,7 @@ indent_style = space indent_size = 2 trim_trailing_whitespace = true insert_final_newline = true -max_line_length = 100 +max_line_length = 150 [*.{tf,tfvars}] indent_size = 2 diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 8f8a374..1ec2e42 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,6 +1,6 @@ repos: - repo: https://github.com/antonbabenko/pre-commit-terraform - rev: v1.83.4 + rev: v1.97.4 hooks: - id: terraform_fmt args: @@ -20,6 +20,6 @@ repos: args: ["--args=--sort-by required"] - repo: https://github.com/adrienverge/yamllint - rev: v1.32.0 + rev: v1.36.2 hooks: - id: yamllint diff --git a/.tflint.hcl b/.tflint.hcl index 0788789..6d81dbe 100644 --- a/.tflint.hcl +++ b/.tflint.hcl @@ -2,7 +2,7 @@ config { plugin_dir = "~/.tflint.d/plugins" format = "compact" - module = true + call_module_type = "local" force = false disabled_by_default = false From 4939b3c0bb64bfab1c13e874fc19bc3b25fc7671 Mon Sep 17 00:00:00 2001 From: Byungjin Park Date: Fri, 21 Mar 2025 03:09:17 +0900 Subject: [PATCH 33/33] chore: update pre-commit configurations --- .pre-commit-config.yaml | 22 ++++++++++++++++++---- .tflint.hcl | 2 +- modules/resource-group/README.md | 4 ++-- modules/s3-archive-bucket/README.md | 4 ++-- modules/sqs-aws-event-queue/README.md | 4 ++-- 5 files changed, 25 insertions(+), 11 deletions(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 1ec2e42..85048fd 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,25 +1,39 @@ +default_install_hook_types: +- pre-commit +- commit-msg + repos: - repo: https://github.com/antonbabenko/pre-commit-terraform rev: v1.97.4 hooks: - id: terraform_fmt + name: (terraform) Format .tf files with `terraform fmt` args: - --args=-diff - id: terraform_validate + name: (terraform) Check with `terraform validate` args: - - --tf-init-args=-upgrade - --hook-config=--retry-once-with-cleanup=true - - id: terraform_providers_lock - args: - - --hook-config=--mode=always-regenerate-lockfile + - --tf-init-args=-upgrade - id: terraform_tflint + name: (terraform) Check with `tflint` args: - --args=--config=__GIT_WORKING_DIR__/.tflint.hcl files: ^modules/ - id: terraform_docs + name: (terraform) Generate docs with `terraform-docs` args: ["--args=--sort-by required"] - repo: https://github.com/adrienverge/yamllint rev: v1.36.2 hooks: - id: yamllint + name: (yaml) Check with `yamllint` + +- repo: https://github.com/compilerla/conventional-pre-commit + rev: v4.0.0 + hooks: + - id: conventional-pre-commit + name: (commit-message) Check conventional commit + stages: [commit-msg] + args: [] diff --git a/.tflint.hcl b/.tflint.hcl index 6d81dbe..1432c6f 100644 --- a/.tflint.hcl +++ b/.tflint.hcl @@ -70,7 +70,7 @@ rule "terraform_unused_required_providers" { plugin "aws" { source = "github.com/terraform-linters/tflint-ruleset-aws" - version = "0.21.1" + version = "0.38.0" enabled = true deep_check = false diff --git a/modules/resource-group/README.md b/modules/resource-group/README.md index 5cb4712..0a0536d 100644 --- a/modules/resource-group/README.md +++ b/modules/resource-group/README.md @@ -4,7 +4,7 @@ This module creates following resources. - `aws_resourcegroups_group` - + ## Requirements | Name | Version | @@ -47,4 +47,4 @@ No modules. | [name](#output\_name) | The name of the resource group. | | [resource\_tags](#output\_resource\_tags) | The resource tags used by the resource group to query resources. | | [resource\_types](#output\_resource\_types) | The resource types used by the resource group to query resources. | - + diff --git a/modules/s3-archive-bucket/README.md b/modules/s3-archive-bucket/README.md index 72334ae..918b321 100644 --- a/modules/s3-archive-bucket/README.md +++ b/modules/s3-archive-bucket/README.md @@ -13,7 +13,7 @@ This module creates following resources. - `aws_s3_bucket_server_side_encryption_configuration` - `aws_s3_bucket_versioning` - + ## Requirements | Name | Version | @@ -106,4 +106,4 @@ This module creates following resources. | [server\_side\_encryption](#output\_server\_side\_encryption) | The configuration for the S3 bucket server-side encryption. | | [transfer\_acceleration](#output\_transfer\_acceleration) | The configuration for the S3 Transfer Acceleration of the bucket. | | [versioning](#output\_versioning) | The versioning configuration for the bucket. | - + diff --git a/modules/sqs-aws-event-queue/README.md b/modules/sqs-aws-event-queue/README.md index bc97421..c63b519 100644 --- a/modules/sqs-aws-event-queue/README.md +++ b/modules/sqs-aws-event-queue/README.md @@ -5,7 +5,7 @@ This module creates following resources. - `aws_sqs_queue` - `aws_sqs_queue_policy` - + ## Requirements | Name | Version | @@ -58,4 +58,4 @@ This module creates following resources. | [arn](#output\_arn) | The ARN of the SQS queue. | | [id](#output\_id) | The URL for the created Amazon SQS queue. | | [name](#output\_name) | The name of the SQS queue. | - +