Skip to content

Commit

Permalink
feat!: Add support for creating an associated dead-letter queue and q…
Browse files Browse the repository at this point in the history
…ueue policies (terraform-aws-modules#46)

Co-authored-by: Anton Babenko <anton@antonbabenko.com>
Resolves undefined
  • Loading branch information
bryantbiggs authored Jan 3, 2023
1 parent 1b622ac commit f49fe2b
Show file tree
Hide file tree
Showing 12 changed files with 1,338 additions and 194 deletions.
217 changes: 187 additions & 30 deletions README.md

Large diffs are not rendered by default.

140 changes: 140 additions & 0 deletions UPGRADE-4.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
# Upgrade from v3.x to v4.x

If you have any questions regarding this upgrade process, please consult the [`examples`](https://github.com/terraform-aws-modules/terraform-aws-sns/tree/master/examples/complete) directory:

If you find a bug, please open an issue with supporting configuration to reproduce.

## List of backwards incompatible changes

- `var.redrive_policy`, and `var.redrive_allow_policy` data types have changed from `string` to `any` which is a map of values. The conversion from a map to a jsonencoded string is now handled by the module
- `var.name_prefix` has been replaced with `var.use_name_prefix` which is a boolean that will use `var.name` as a prefix
- `var.policy` has been removed; users can create a policy via the queue policy or dead-letter queue policy which by default uses the associated queue ARN as the `resource` (avoids the chicken vs the egg scenario)

## Additional changes

### Added

- When creating a FIFO queue, the `.fifo` postfix will now be automatically added to the queue name
- Added support for creating:
- Queue policy
- Dead letter queue
- Dead letter queue policy
- Redrive and redrive allow policies have been converted to their separate resources to avoid lifecycle conflicts; now you can create both the source queue and dead-letter queue in the same `terraform apply` without conflict
- The queue data source previously used to extract the queue name has been replaced since this is natively supported in the current AWS provider queue resource

### Modified

- `visibility_timeout_seconds` default value has been changed from `30` to `null`
- `message_retention_seconds` default value has been changed from `345600` to `null`
- `max_message_size` default value has been changed from `262144` to `null`
- `delay_seconds` default value has been changed from `0` to `null`
- `receive_wait_time_seconds` default value has been changed from `0` to `null`
- `content_based_deduplication` default value has been changed from `false` to `null`
- `sqs_managed_sse_enabled` default value has been changed from `false` to `true` (matches current default behavior but value is needed for internal logic evaluation)
- `kms_data_key_reuse_period_seconds` default value has been changed from `300` to `null`

### Variable and output changes

1. Removed variables:

- `name_prefix` has been replaced with `use_name_prefix` which is a boolean that will use `name` as a prefix
- `policy` has been removed; users can create a policy via the queue policy or dead-letter queue policy which by default uses the associated queue ARN as the `resource` (avoids the chicken vs the egg scenario)

2. Renamed variables:

- None

3. Added variables:

- `use_name_prefix`
- `create_queue_policy`
- `source_queue_policy_documents`
- `override_queue_policy_documents`
- `queue_policy_statements`
- `create_dlq`
- `dlq_content_based_deduplication`
- `dlq_deduplication_scope`
- `dlq_delay_seconds`
- `dlq_kms_data_key_reuse_period_seconds`
- `dlq_kms_master_key_id`
- `dlq_message_retention_seconds`
- `dlq_name`
- `dlq_receive_wait_time_seconds`
- `dlq_redrive_allow_policy`
- `dlq_sqs_managed_sse_enabled`
- `dlq_visibility_timeout_seconds`
- `dlq_tags`
- `create_dlq_queue_policy`
- `source_dlq_queue_policy_documents`
- `override_dlq_queue_policy_documents`
- `dlq_queue_policy_statements`

4. Removed outputs:

- None

5. Renamed outputs:

- All output names have had the `sqs_` prefix removed

6. Added outputs:

- `queue_url`
- `dead_letter_queue_id`
- `dead_letter_queue_arn`
- `dead_letter_queue_url`
- `dead_letter_queue_name`

## Upgrade Migrations

Note: Only the affected attributes are shown below for brevity.

### Before 3.x Example

```hcl
module "sqs" {
source = "terraform-aws-modules/sqs/aws"
version = "~> 3.0"
name_prefix = "example-"
redrive_policy = jsonencoded({
redrivePermission = "byQueue",
sourceQueueArns = [aws_sqs_queue.example.arn]
})
redrive_allow_policy = jsonencoded({
deadLetterTargetArn = aws_sqs_queue.example.arn
maxReceiveCount = 4
})
policy = "..."
}
```

### After 4.x Example

```hcl
module "sqs" {
source = "terraform-aws-modules/sns/aws"
version = "~> 4.0"
name = "example"
use_name_prefix = true
redrive_policy = {
redrivePermission = "byQueue",
sourceQueueArns = [aws_sqs_queue.example.arn]
}
redrive_allow_policy = {
deadLetterTargetArn = aws_sqs_queue.example.arn
maxReceiveCount = 4
}
# Can be used to utilize v3.x `var.policy` value without modification
# source_queue_policy_documents = ["..."]
}
```

### State Changes

No state changes required.
8 changes: 8 additions & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Examples

Please note - the examples provided serve two primary means:

1. Show users working examples of the various ways in which the module can be configured and features supported
2. A means of testing/validating module changes

Please do not mistake the examples provided as "best practices". It is up to users to consult the AWS service documentation for best practices, usage recommendations, etc.
101 changes: 86 additions & 15 deletions examples/complete/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,14 @@
# Complete SQS queues example

Configuration in this directory creates 2 SQS queues - with server-side encryption (SSE) using specified KMS key and without SSE.
# Complete SQS Queue Example

Configuration in this directory creates:
- Queue using module default settings
- FIFO (first-in, first-out) queue
- Unencrypted queue (encryption disabled)
- Queue encrypted with customer managed KMS key
- Queue encrypted with default SQS SSE (server-side encryption) w/ separate dead-letter queue
- Dead letter queue created in separate module definition
- Queue with dead-letter queue created in the same module defintion w/ queue policies for both the source queue and dead-letter queue
- Disabled queue (no resources created)

## Usage

Expand All @@ -19,30 +27,33 @@ Note that this example may create resources which cost money. Run `terraform des

| Name | Version |
|------|---------|
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.13.1 |
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 3.63 |
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0 |
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.36 |

## Providers

| Name | Version |
|------|---------|
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 3.63 |
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.36 |

## Modules

| Name | Source | Version |
|------|--------|---------|
| <a name="module_sqs_dlq_allow_redrive_policy"></a> [sqs\_dlq\_allow\_redrive\_policy](#module\_sqs\_dlq\_allow\_redrive\_policy) | ../../ | n/a |
| <a name="module_users_encrypted"></a> [users\_encrypted](#module\_users\_encrypted) | ../../ | n/a |
| <a name="module_users_encrypted_with_sse"></a> [users\_encrypted\_with\_sse](#module\_users\_encrypted\_with\_sse) | ../../ | n/a |
| <a name="module_users_unencrypted"></a> [users\_unencrypted](#module\_users\_unencrypted) | ../../ | n/a |
| <a name="module_cmk_encrypted_sqs"></a> [cmk\_encrypted\_sqs](#module\_cmk\_encrypted\_sqs) | ../../ | n/a |
| <a name="module_default_sqs"></a> [default\_sqs](#module\_default\_sqs) | ../../ | n/a |
| <a name="module_disabled_sqs"></a> [disabled\_sqs](#module\_disabled\_sqs) | ../../ | n/a |
| <a name="module_fifo_sqs"></a> [fifo\_sqs](#module\_fifo\_sqs) | ../../ | n/a |
| <a name="module_sqs_with_dlq"></a> [sqs\_with\_dlq](#module\_sqs\_with\_dlq) | ../../ | n/a |
| <a name="module_sse_encrypted_dlq_sqs"></a> [sse\_encrypted\_dlq\_sqs](#module\_sse\_encrypted\_dlq\_sqs) | ../../ | n/a |
| <a name="module_sse_encrypted_sqs"></a> [sse\_encrypted\_sqs](#module\_sse\_encrypted\_sqs) | ../../ | n/a |
| <a name="module_unencrypted_sqs"></a> [unencrypted\_sqs](#module\_unencrypted\_sqs) | ../../ | n/a |

## Resources

| Name | Type |
|------|------|
| [aws_kms_key.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_key) | resource |
| [aws_sqs_queue_policy.users_unencrypted_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sqs_queue_policy) | resource |
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |

## Inputs
Expand All @@ -53,8 +64,68 @@ No inputs.

| Name | Description |
|------|-------------|
| <a name="output_users_encrypted_sqs_queue_arn"></a> [users\_encrypted\_sqs\_queue\_arn](#output\_users\_encrypted\_sqs\_queue\_arn) | The ARN of the SQS queue |
| <a name="output_users_encrypted_sqs_queue_id"></a> [users\_encrypted\_sqs\_queue\_id](#output\_users\_encrypted\_sqs\_queue\_id) | The URL for the created Amazon SQS queue |
| <a name="output_users_unencrypted_sqs_queue_arn"></a> [users\_unencrypted\_sqs\_queue\_arn](#output\_users\_unencrypted\_sqs\_queue\_arn) | The ARN of the SQS queue |
| <a name="output_users_unencrypted_sqs_queue_id"></a> [users\_unencrypted\_sqs\_queue\_id](#output\_users\_unencrypted\_sqs\_queue\_id) | The URL for the created Amazon SQS queue |
| <a name="output_cmk_encrypted_sqs_dlq_arn"></a> [cmk\_encrypted\_sqs\_dlq\_arn](#output\_cmk\_encrypted\_sqs\_dlq\_arn) | The ARN of the SQS queue |
| <a name="output_cmk_encrypted_sqs_dlq_id"></a> [cmk\_encrypted\_sqs\_dlq\_id](#output\_cmk\_encrypted\_sqs\_dlq\_id) | The URL for the created Amazon SQS queue |
| <a name="output_cmk_encrypted_sqs_dlq_name"></a> [cmk\_encrypted\_sqs\_dlq\_name](#output\_cmk\_encrypted\_sqs\_dlq\_name) | The name of the SQS queue |
| <a name="output_cmk_encrypted_sqs_dlq_url"></a> [cmk\_encrypted\_sqs\_dlq\_url](#output\_cmk\_encrypted\_sqs\_dlq\_url) | Same as `dead_letter_queue_id`: The URL for the created Amazon SQS queue |
| <a name="output_cmk_encrypted_sqs_queue_arn"></a> [cmk\_encrypted\_sqs\_queue\_arn](#output\_cmk\_encrypted\_sqs\_queue\_arn) | The ARN of the SQS queue |
| <a name="output_cmk_encrypted_sqs_queue_id"></a> [cmk\_encrypted\_sqs\_queue\_id](#output\_cmk\_encrypted\_sqs\_queue\_id) | The URL for the created Amazon SQS queue |
| <a name="output_cmk_encrypted_sqs_queue_name"></a> [cmk\_encrypted\_sqs\_queue\_name](#output\_cmk\_encrypted\_sqs\_queue\_name) | The name of the SQS queue |
| <a name="output_cmk_encrypted_sqs_queue_url"></a> [cmk\_encrypted\_sqs\_queue\_url](#output\_cmk\_encrypted\_sqs\_queue\_url) | Same as `queue_id`: The URL for the created Amazon SQS queue |
| <a name="output_default_sqs_dlq_arn"></a> [default\_sqs\_dlq\_arn](#output\_default\_sqs\_dlq\_arn) | The ARN of the SQS queue |
| <a name="output_default_sqs_dlq_id"></a> [default\_sqs\_dlq\_id](#output\_default\_sqs\_dlq\_id) | The URL for the created Amazon SQS queue |
| <a name="output_default_sqs_dlq_name"></a> [default\_sqs\_dlq\_name](#output\_default\_sqs\_dlq\_name) | The name of the SQS queue |
| <a name="output_default_sqs_dlq_url"></a> [default\_sqs\_dlq\_url](#output\_default\_sqs\_dlq\_url) | Same as `dead_letter_queue_id`: The URL for the created Amazon SQS queue |
| <a name="output_default_sqs_queue_arn"></a> [default\_sqs\_queue\_arn](#output\_default\_sqs\_queue\_arn) | The ARN of the SQS queue |
| <a name="output_default_sqs_queue_id"></a> [default\_sqs\_queue\_id](#output\_default\_sqs\_queue\_id) | The URL for the created Amazon SQS queue |
| <a name="output_default_sqs_queue_name"></a> [default\_sqs\_queue\_name](#output\_default\_sqs\_queue\_name) | The name of the SQS queue |
| <a name="output_default_sqs_queue_url"></a> [default\_sqs\_queue\_url](#output\_default\_sqs\_queue\_url) | Same as `queue_id`: The URL for the created Amazon SQS queue |
| <a name="output_disabled_sqs_dlq_arn"></a> [disabled\_sqs\_dlq\_arn](#output\_disabled\_sqs\_dlq\_arn) | The ARN of the SQS queue |
| <a name="output_disabled_sqs_dlq_id"></a> [disabled\_sqs\_dlq\_id](#output\_disabled\_sqs\_dlq\_id) | The URL for the created Amazon SQS queue |
| <a name="output_disabled_sqs_dlq_name"></a> [disabled\_sqs\_dlq\_name](#output\_disabled\_sqs\_dlq\_name) | The name of the SQS queue |
| <a name="output_disabled_sqs_dlq_url"></a> [disabled\_sqs\_dlq\_url](#output\_disabled\_sqs\_dlq\_url) | Same as `dead_letter_queue_id`: The URL for the created Amazon SQS queue |
| <a name="output_disabled_sqs_queue_arn"></a> [disabled\_sqs\_queue\_arn](#output\_disabled\_sqs\_queue\_arn) | The ARN of the SQS queue |
| <a name="output_disabled_sqs_queue_id"></a> [disabled\_sqs\_queue\_id](#output\_disabled\_sqs\_queue\_id) | The URL for the created Amazon SQS queue |
| <a name="output_disabled_sqs_queue_name"></a> [disabled\_sqs\_queue\_name](#output\_disabled\_sqs\_queue\_name) | The name of the SQS queue |
| <a name="output_disabled_sqs_queue_url"></a> [disabled\_sqs\_queue\_url](#output\_disabled\_sqs\_queue\_url) | Same as `queue_id`: The URL for the created Amazon SQS queue |
| <a name="output_fifo_sqs_dlq_arn"></a> [fifo\_sqs\_dlq\_arn](#output\_fifo\_sqs\_dlq\_arn) | The ARN of the SQS queue |
| <a name="output_fifo_sqs_dlq_id"></a> [fifo\_sqs\_dlq\_id](#output\_fifo\_sqs\_dlq\_id) | The URL for the created Amazon SQS queue |
| <a name="output_fifo_sqs_dlq_name"></a> [fifo\_sqs\_dlq\_name](#output\_fifo\_sqs\_dlq\_name) | The name of the SQS queue |
| <a name="output_fifo_sqs_dlq_url"></a> [fifo\_sqs\_dlq\_url](#output\_fifo\_sqs\_dlq\_url) | Same as `dead_letter_queue_id`: The URL for the created Amazon SQS queue |
| <a name="output_fifo_sqs_queue_arn"></a> [fifo\_sqs\_queue\_arn](#output\_fifo\_sqs\_queue\_arn) | The ARN of the SQS queue |
| <a name="output_fifo_sqs_queue_id"></a> [fifo\_sqs\_queue\_id](#output\_fifo\_sqs\_queue\_id) | The URL for the created Amazon SQS queue |
| <a name="output_fifo_sqs_queue_name"></a> [fifo\_sqs\_queue\_name](#output\_fifo\_sqs\_queue\_name) | The name of the SQS queue |
| <a name="output_fifo_sqs_queue_url"></a> [fifo\_sqs\_queue\_url](#output\_fifo\_sqs\_queue\_url) | Same as `queue_id`: The URL for the created Amazon SQS queue |
| <a name="output_sqs_with_dlq_dlq_arn"></a> [sqs\_with\_dlq\_dlq\_arn](#output\_sqs\_with\_dlq\_dlq\_arn) | The ARN of the SQS queue |
| <a name="output_sqs_with_dlq_dlq_id"></a> [sqs\_with\_dlq\_dlq\_id](#output\_sqs\_with\_dlq\_dlq\_id) | The URL for the created Amazon SQS queue |
| <a name="output_sqs_with_dlq_dlq_name"></a> [sqs\_with\_dlq\_dlq\_name](#output\_sqs\_with\_dlq\_dlq\_name) | The name of the SQS queue |
| <a name="output_sqs_with_dlq_dlq_url"></a> [sqs\_with\_dlq\_dlq\_url](#output\_sqs\_with\_dlq\_dlq\_url) | Same as `dead_letter_queue_id`: The URL for the created Amazon SQS queue |
| <a name="output_sqs_with_dlq_queue_arn"></a> [sqs\_with\_dlq\_queue\_arn](#output\_sqs\_with\_dlq\_queue\_arn) | The ARN of the SQS queue |
| <a name="output_sqs_with_dlq_queue_id"></a> [sqs\_with\_dlq\_queue\_id](#output\_sqs\_with\_dlq\_queue\_id) | The URL for the created Amazon SQS queue |
| <a name="output_sqs_with_dlq_queue_name"></a> [sqs\_with\_dlq\_queue\_name](#output\_sqs\_with\_dlq\_queue\_name) | The name of the SQS queue |
| <a name="output_sqs_with_dlq_queue_url"></a> [sqs\_with\_dlq\_queue\_url](#output\_sqs\_with\_dlq\_queue\_url) | Same as `queue_id`: The URL for the created Amazon SQS queue |
| <a name="output_sse_encrypted_dlq_sqs_dlq_arn"></a> [sse\_encrypted\_dlq\_sqs\_dlq\_arn](#output\_sse\_encrypted\_dlq\_sqs\_dlq\_arn) | The ARN of the SQS queue |
| <a name="output_sse_encrypted_dlq_sqs_dlq_id"></a> [sse\_encrypted\_dlq\_sqs\_dlq\_id](#output\_sse\_encrypted\_dlq\_sqs\_dlq\_id) | The URL for the created Amazon SQS queue |
| <a name="output_sse_encrypted_dlq_sqs_dlq_name"></a> [sse\_encrypted\_dlq\_sqs\_dlq\_name](#output\_sse\_encrypted\_dlq\_sqs\_dlq\_name) | The name of the SQS queue |
| <a name="output_sse_encrypted_dlq_sqs_dlq_url"></a> [sse\_encrypted\_dlq\_sqs\_dlq\_url](#output\_sse\_encrypted\_dlq\_sqs\_dlq\_url) | Same as `dead_letter_queue_id`: The URL for the created Amazon SQS queue |
| <a name="output_sse_encrypted_dlq_sqs_queue_arn"></a> [sse\_encrypted\_dlq\_sqs\_queue\_arn](#output\_sse\_encrypted\_dlq\_sqs\_queue\_arn) | The ARN of the SQS queue |
| <a name="output_sse_encrypted_dlq_sqs_queue_id"></a> [sse\_encrypted\_dlq\_sqs\_queue\_id](#output\_sse\_encrypted\_dlq\_sqs\_queue\_id) | The URL for the created Amazon SQS queue |
| <a name="output_sse_encrypted_dlq_sqs_queue_name"></a> [sse\_encrypted\_dlq\_sqs\_queue\_name](#output\_sse\_encrypted\_dlq\_sqs\_queue\_name) | The name of the SQS queue |
| <a name="output_sse_encrypted_dlq_sqs_queue_url"></a> [sse\_encrypted\_dlq\_sqs\_queue\_url](#output\_sse\_encrypted\_dlq\_sqs\_queue\_url) | Same as `queue_id`: The URL for the created Amazon SQS queue |
| <a name="output_sse_encrypted_sqs_dlq_arn"></a> [sse\_encrypted\_sqs\_dlq\_arn](#output\_sse\_encrypted\_sqs\_dlq\_arn) | The ARN of the SQS queue |
| <a name="output_sse_encrypted_sqs_dlq_id"></a> [sse\_encrypted\_sqs\_dlq\_id](#output\_sse\_encrypted\_sqs\_dlq\_id) | The URL for the created Amazon SQS queue |
| <a name="output_sse_encrypted_sqs_dlq_name"></a> [sse\_encrypted\_sqs\_dlq\_name](#output\_sse\_encrypted\_sqs\_dlq\_name) | The name of the SQS queue |
| <a name="output_sse_encrypted_sqs_dlq_url"></a> [sse\_encrypted\_sqs\_dlq\_url](#output\_sse\_encrypted\_sqs\_dlq\_url) | Same as `dead_letter_queue_id`: The URL for the created Amazon SQS queue |
| <a name="output_sse_encrypted_sqs_queue_arn"></a> [sse\_encrypted\_sqs\_queue\_arn](#output\_sse\_encrypted\_sqs\_queue\_arn) | The ARN of the SQS queue |
| <a name="output_sse_encrypted_sqs_queue_id"></a> [sse\_encrypted\_sqs\_queue\_id](#output\_sse\_encrypted\_sqs\_queue\_id) | The URL for the created Amazon SQS queue |
| <a name="output_sse_encrypted_sqs_queue_name"></a> [sse\_encrypted\_sqs\_queue\_name](#output\_sse\_encrypted\_sqs\_queue\_name) | The name of the SQS queue |
| <a name="output_sse_encrypted_sqs_queue_url"></a> [sse\_encrypted\_sqs\_queue\_url](#output\_sse\_encrypted\_sqs\_queue\_url) | Same as `queue_id`: The URL for the created Amazon SQS queue |
| <a name="output_unencrypted_sqs_dlq_arn"></a> [unencrypted\_sqs\_dlq\_arn](#output\_unencrypted\_sqs\_dlq\_arn) | The ARN of the SQS queue |
| <a name="output_unencrypted_sqs_dlq_id"></a> [unencrypted\_sqs\_dlq\_id](#output\_unencrypted\_sqs\_dlq\_id) | The URL for the created Amazon SQS queue |
| <a name="output_unencrypted_sqs_dlq_name"></a> [unencrypted\_sqs\_dlq\_name](#output\_unencrypted\_sqs\_dlq\_name) | The name of the SQS queue |
| <a name="output_unencrypted_sqs_dlq_url"></a> [unencrypted\_sqs\_dlq\_url](#output\_unencrypted\_sqs\_dlq\_url) | Same as `dead_letter_queue_id`: The URL for the created Amazon SQS queue |
| <a name="output_unencrypted_sqs_queue_arn"></a> [unencrypted\_sqs\_queue\_arn](#output\_unencrypted\_sqs\_queue\_arn) | The ARN of the SQS queue |
| <a name="output_unencrypted_sqs_queue_id"></a> [unencrypted\_sqs\_queue\_id](#output\_unencrypted\_sqs\_queue\_id) | The URL for the created Amazon SQS queue |
| <a name="output_unencrypted_sqs_queue_name"></a> [unencrypted\_sqs\_queue\_name](#output\_unencrypted\_sqs\_queue\_name) | The name of the SQS queue |
| <a name="output_unencrypted_sqs_queue_url"></a> [unencrypted\_sqs\_queue\_url](#output\_unencrypted\_sqs\_queue\_url) | Same as `queue_id`: The URL for the created Amazon SQS queue |
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
Loading

0 comments on commit f49fe2b

Please sign in to comment.