RDS parameter group creation can fail when parameters get "chunked" and charset, collation get separated

[jhriggs]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v1.1.5
on darwin_amd64
+ provider registry.terraform.io/hashicorp/aws v4.8.0

Affected Resource(s)

• aws_rds_cluster_parameter_group
• likely aws_db_parameter_group too, but untested

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "aws_rds_cluster_parameter_group" "this" {
  name        = "jhrtest"
  description = "jhrtest"
  family      = "aurora-mysql5.7"

  parameter {
    apply_method = "immediate"
    name         = "auto_increment_increment"
    value        = "10"
  }
  parameter {
    apply_method = "immediate"
    name         = "auto_increment_offset"
    value        = "1"
  }
  parameter {
    apply_method = "immediate"
    name         = "binlog_cache_size"
    value        = "262144"
  }
  parameter {
    apply_method = "immediate"
    name         = "character_set_client"
    value        = "utf8"
  }
  parameter {
    apply_method = "immediate"
    name         = "character_set_connection"
    value        = "utf8"
  }
  parameter {
    apply_method = "immediate"
    name         = "character_set_database"
    value        = "utf8"
  }
  parameter {
    apply_method = "immediate"
    name         = "character_set_results"
    value        = "utf8"
  }
  parameter {
    apply_method = "immediate"
    name         = "character_set_server"
    value        = "utf8"
  }
  parameter {
    apply_method = "immediate"
    name         = "collation_server"
    value        = "utf8_unicode_ci"
  }
  parameter {
    apply_method = "immediate"
    name         = "group_concat_max_len"
    value        = "5120"
  }
  parameter {
    apply_method = "immediate"
    name         = "init_connect"
    value        = "set names utf8"
  }
  parameter {
    apply_method = "immediate"
    name         = "innodb_adaptive_hash_index"
    value        = "1"
  }
  parameter {
    apply_method = "immediate"
    name         = "innodb_flush_log_at_trx_commit"
    value        = "2"
  }
  parameter {
    apply_method = "immediate"
    name         = "innodb_monitor_enable"
    value        = "all"
  }
  parameter {
    apply_method = "immediate"
    name         = "key_buffer_size"
    value        = "33554432"
  }
  parameter {
    apply_method = "immediate"
    name         = "log_error_verbosity"
    value        = "2"
  }
  parameter {
    apply_method = "immediate"
    name         = "log_warnings"
    value        = "1"
  }
  parameter {
    apply_method = "immediate"
    name         = "long_query_time"
    value        = "1"
  }
  parameter {
    apply_method = "immediate"
    name         = "max_allowed_packet"
    value        = "167772160"
  }
  parameter {
    apply_method = "immediate"
    name         = "max_connect_errors"
    value        = "10"
  }
  parameter {
    apply_method = "immediate"
    name         = "max_connections"
    value        = "1000"
  }
  parameter {
    apply_method = "immediate"
    name         = "max_heap_table_size"
    value        = "67108864"
  }
  parameter {
    apply_method = "immediate"
    name         = "optimizer_switch"
    value        = "index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,engine_condition_pushdown=on,index_condition_pushdown=on,mrr=on,mrr_cost_based=on,block_nested_loop=on,batched_key_access=off,materialization=on,semijoin=on,loosescan=on,firstmatch=on,duplicateweedout=on,subquery_materialization_cost_based=on,use_index_extensions=on,condition_fanout_filter=on,derived_merge=on"
  }
  parameter {
    apply_method = "immediate"
    name         = "query_cache_size"
    value        = "0"
  }
  parameter {
    apply_method = "immediate"
    name         = "read_rnd_buffer_size"
    value        = "4194304"
  }
  parameter {
    apply_method = "immediate"
    name         = "slow_query_log"
    value        = "1"
  }
  parameter {
    apply_method = "immediate"
    name         = "sort_buffer_size"
    value        = "4194304"
  }
  parameter {
    apply_method = "immediate"
    name         = "thread_cache_size"
    value        = "64"
  }
  parameter {
    apply_method = "immediate"
    name         = "time_zone"
    value        = "US/Pacific"
  }
  parameter {
    apply_method = "immediate"
    name         = "tmp_table_size"
    value        = "67108864"
  }
  parameter {
    apply_method = "immediate"
    name         = "tx_isolation"
    value        = "read-committed"
  }
  parameter {
    apply_method = "pending-reboot"
    name         = "back_log"
    value        = "500"
  }
  parameter {
    apply_method = "pending-reboot"
    name         = "binlog_format"
    value        = "row"
  }
  parameter {
    apply_method = "pending-reboot"
    name         = "explicit_defaults_for_timestamp"
    value        = "0"
  }
  parameter {
    apply_method = "pending-reboot"
    name         = "innodb_rollback_on_timeout"
    value        = "1"
  }
  parameter {
    apply_method = "pending-reboot"
    name         = "performance_schema"
    value        = "1"
  }
  parameter {
    apply_method = "pending-reboot"
    name         = "query_cache_type"
    value        = "0"
  }
  parameter {
    apply_method = "pending-reboot"
    name         = "slave-skip-errors"
    value        = "1045,1396,1524"
  }
}

Debug Output

https://gist.github.com/jhriggs/bf75c65ebb29c757ee706709cdca087d

Panic Output

Expected Behavior

RDS Cluster Parameter Group should be created.

Actual Behavior

The aws_rds_cluster_parameter_group example provided above has 38 parameters. cluster_parameter_group.go:228 (

terraform-provider-aws/internal/service/rds/cluster_parameter_group.go

Line 228 in e3620ba

paramsToModify, parameters = parameters[:rdsClusterParameterGroupMaxParamsBulkEdit], parameters[rdsClusterParameterGroupMaxParamsBulkEdit:]

) slices/chunks these parameters into groups of rdsClusterParameterGroupMaxParamsBulkEdit (20). Based on the debug output, you can see that the ModifyDBClusterParameterGroup API call fails, because the first chunk of parameters provided includes collation_server but not character_set_server. Thus, setting collation to utf8_* is invalid when the character set is still the default of latin1.

I don't know if there is any way to specify or force the order of these parameters from the TF code side. (I don't think so.) From the golang side, I would hate to have to hardcode things, but perhaps we have to somehow force character_set_* and collation_* parameters into the same batch. 🤷🏻‍♂️

Steps to Reproduce

1. terraform apply

Important Factoids

References

• MariaDB Parameter Group Fails for UTF8 #2304

[Shashwatpathak05]

Found any solution or fix for this?

[jhriggs]

No. I think the workarounds are:

1. create the parameter group manually and then terraform import it
2. if I am remembering correctly…it has been a while since I have done this:
   a. let it fail/error on the first apply
   b. TF then wants to destroy+re-create it, so terraform untaint the parameter group
   c. TF apply will then update it in place properly

It's a pain, but fortunately it is not a super frequent thing (for me at least).

[ewbankkit]

There's information here about which parameters need to be sent together:

    public static final List<Set<String>> PARAMETER_DEPENDENCIES = ImmutableList.of(
            ImmutableSet.of("collation_server", "character_set_server"),
            ImmutableSet.of("gtid-mode", "enforce_gtid_consistency"),
            ImmutableSet.of("password_encryption", "rds.accepted_password_auth_method"),
            ImmutableSet.of("ssl_max_protocol_version", "ssl_min_protocol_version"),
            ImmutableSet.of("rds.change_data_capture_streaming", "binlog_format"),
            ImmutableSet.of("aurora_enhanced_binlog", "binlog_backup", "binlog_replication_globaldb")
    );

and the same for aws_db_parameter_group:

    public static final List<Set<String>> DEPENDENCIES = ImmutableList.of(
            ImmutableSet.of("collation_server", "character_set_server"),
            ImmutableSet.of("gtid-mode", "enforce_gtid_consistency"),
            ImmutableSet.of("password_encryption", "rds.accepted_password_auth_method"),
            ImmutableSet.of("ssl_max_protocol_version", "ssl_min_protocol_version")
    );

[github-actions]

Warning

This Issue has been closed, meaning that any additional comments are much easier for the maintainers to miss. Please assume that the maintainers will not see them.

Ongoing conversations amongst community members are welcome, however, the issue will be locked after 30 days. Moving conversations to another venue, such as the AWS Provider forum, is recommended. If you have additional concerns, please open a new issue, referencing this one where needed.

[github-actions]

This functionality has been released in v5.98.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!