unable to set some parameters in the DbClusterParameterGroup

[luis-alen]

Describe the bug

I'm unable to set parameters such as slow_query_log, long_query_time and log_queries_not_using_indexes in DbClusterParameterGroup objects. I get an unknown parameter error message:

Name:         luis
Namespace:    rc
Labels:       <none>
API Version:  rds.services.k8s.aws/v1alpha1
Kind:         DBClusterParameterGroup
Metadata:
  Creation Timestamp:  2023-07-13T18:40:54Z
  Finalizers:
    finalizers.rds.services.k8s.aws/DBClusterParameterGroup
  Generation:        7
  Resource Version:  593698696
  UID:               e62bcf3f-cbb9-482f-99d3-adb241686b24
Spec:
  Description:  luis aurora-mysql8.0 parameter group managed by ACK
  Family:       aurora-mysql8.0
  Name:         luis
  Parameter Overrides:
    character_set_client:      utf8mb4
    character_set_connection:  utf8mb4
    character_set_database:    utf8mb4
    character_set_results:     utf8mb4
    character_set_server:      utf8mb4
    collation_connection:      utf8mb4_unicode_ci
    collation_server:          utf8mb4_unicode_ci
    slow_query_log:            1
Status:
  Conditions:
    Message:               unknown parameter: slow_query_log
    Status:                True
    Type:                  ACK.Terminal
    Last Transition Time:  2023-07-14T13:02:08Z
    Message:               Resource not synced
    Reason:                resource is in terminal condition
    Status:                False
    Type:                  ACK.ResourceSynced
Events:                    <none>

I'm able to set parameters such as character_set_*, but the not the ones I mentioned above. I confirm that such parameters are available and are modifiable on the db cluster parameter group using the aws console.

I also confirm that it does work for the DbInstance kind. Maybe it's somehow related to #869 (comment)?

Steps to reproduce
Try to create a new DbClusterParameterGroup or try updating an existing one (with family aurora-mysql8.0) overwriting the slow_query_log parameter.

Expected outcome
Parameter should be set accordingly.

Environment

• Kubernetes version 1.23
• Using EKS (yes/no), if so version?
• AWS service targeted (S3, RDS, etc.) RDS
• aws-rds-controller version v.1.1.2

[a-hilaly]

Hi @luis-alen , the rds-controller uses a kinda specific cache implementation inside the controller. Can you try to restart the controller? if the problem is solved them we might have an issue with the cache implementation :)

[luis-alen]

Hey, @a-hilaly. Unfortunately it didn't help. I turned on debug log level for the controller and I see that it captures the change, but is unable to apply it.

2023-07-25T10:18:39.211Z	DEBUG	ackrt	> r.Sync	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "generation": 7}
2023-07-25T10:18:39.211Z	DEBUG	ackrt	>> r.resetConditions	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "generation": 7}
2023-07-25T10:18:39.211Z	DEBUG	ackrt	<< r.resetConditions	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "generation": 7}
2023-07-25T10:18:39.211Z	DEBUG	ackrt	>> rm.ResolveReferences	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.211Z	DEBUG	ackrt	<< rm.ResolveReferences	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.211Z	DEBUG	ackrt	>> rm.EnsureTags	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.211Z	DEBUG	ackrt	<< rm.EnsureTags	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.211Z	DEBUG	ackrt	>> rm.ReadOne	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.211Z	DEBUG	ackrt	>>> rm.sdkFind	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.410Z	DEBUG	ackrt	<<< rm.sdkFind	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.411Z	DEBUG	ackrt	<< rm.ReadOne	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.411Z	DEBUG	ackrt	>> r.updateResource	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.411Z	INFO	ackrt	desired resource state has changed	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7, "diff": [{"Path":{"Parts":["Spec","ParameterOverrides"]},"A":{"character_set_client":"utf8mb4","character_set_connection":"utf8mb4","character_set_database":"utf8mb4","character_set_results":"utf8mb4","character_set_server":"utf8mb4","collation_connection":"utf8mb4_unicode_ci","collation_server":"utf8mb4_unicode_ci","slow_query_log":"1"},"B":{}}]}
2023-07-25T10:18:39.411Z	DEBUG	ackrt	>>> rm.Update	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.411Z	DEBUG	ackrt	>>>> rm.customUpdate	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.411Z	DEBUG	ackrt	>>>>> rm.syncParameters	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.412Z	DEBUG	ackrt	>>>>>> rm.modifyParameters	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.412Z	DEBUG	ackrt	<<<<<< rm.modifyParameters	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7, "error": "unknown parameter: slow_query_log"}
2023-07-25T10:18:39.412Z	DEBUG	ackrt	<<<<< rm.syncParameters	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7, "error": "unknown parameter: slow_query_log"}
2023-07-25T10:18:39.412Z	DEBUG	ackrt	<<<< rm.customUpdate	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7, "error": "unknown parameter: slow_query_log"}
2023-07-25T10:18:39.412Z	DEBUG	ackrt	<<< rm.Update	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7, "latest": {}, "error": "resource is in terminal condition"}
2023-07-25T10:18:39.412Z	DEBUG	ackrt	<< r.updateResource	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7, "error": "resource is in terminal condition"}
2023-07-25T10:18:39.412Z	DEBUG	ackrt	>> r.ensureConditions	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.412Z	DEBUG	ackrt	>>> rm.IsSynced	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.412Z	DEBUG	ackrt	<<< rm.IsSynced	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.412Z	DEBUG	ackrt	<< r.ensureConditions	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.412Z	DEBUG	ackrt	< r.Sync	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7, "error": "resource is in terminal condition"}
2023-07-25T10:18:39.412Z	DEBUG	ackrt	> r.patchResourceStatus	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.412Z	DEBUG	ackrt	>> kc.Patch (status)	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.511Z	DEBUG	ackrt	patched resource status	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7, "json": "{\"metadata\":{\"resourceVersion\":\"601235537\"},\"spec\":{\"tags\":null},\"status\":{\"conditions\":[{\"message\":\"unknown parameter: slow_query_log\",\"status\":\"True\",\"type\":\"ACK.Terminal\"},{\"lastTransitionTime\":\"2023-07-25T10:18:39Z\",\"message\":\"Resource not synced\",\"reason\":\"resource is in terminal condition\",\"status\":\"False\",\"type\":\"ACK.ResourceSynced\"}]}}"}
2023-07-25T10:18:39.511Z	DEBUG	ackrt	<< kc.Patch (status)	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}
2023-07-25T10:18:39.511Z	DEBUG	ackrt	< r.patchResourceStatus	{"account": "xxxxxxxxxx", "role": "", "region": "eu-central-1", "kind": "DBClusterParameterGroup", "namespace": "rc", "name": "rc-aurora-mysql8-0", "is_adopted": false, "generation": 7}

[riddhi2910]

@a-hilaly I also verified the same with debug logs, and see that DbClusterParameterGroup object goes into not synced mode and throws an error

'unknown parameter: long_query_time', We also see the error on the binlog_format, and read_only kind parameters.

[riddhi2910]

Also, Debug logs don't show much information other than what is mentioned in the #1847 (comment)

[riddhi2910]

@a-hilaly further investigating the issue:

the error is only coming for the parameters which are https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Reference.ParameterGroups.html#AuroraMySQL.Reference.Parameters.Instance instance level.

However, I need to check how the status is updated on DBClusterParameterGroup since it shows Instance instance level parameters in the status section.

[cogren1]

We're using controller version 1.1.10. We just encountered this with idle_in_transaction_session_timeout for Aurora Postgres. It worked fine for the instance parameter groups for our non-Aurora databases, but for the cluster parameter groups, we see the following:

    - message: 'unknown parameter: idle_in_transaction_session_timeout'
      status: 'True'
      type: ACK.Terminal

I removed this parameter from the spec, and restarted the controller, but the error still persists.

[riddhi2910]

@cogren1 aws-controllers-k8s/rds-controller#167 will fix the issue that you are facing.

However please make sure, DbClusterParameterGroup should have Cluster Level Parameters https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Reference.ParameterGroups.html#AuroraMySQL.Reference.Parameters.Instance

[cogren1]

@riddhi2910 Thank you for linking that fix. I look forward to it getting in!

idle_in_transaction_session_timeout is a parameter that exists in cluster parameter groups for Aurora Postgres.

[riddhi2910]

@cogren1 Yeah that isn't very clear, the console UI can do much more than the API call to fetch the supported cluster-level parameters.

[salimadr]

Hello, I am experiencing the same problem, any solutions?

  conditions:
    - message: 'unknown parameter: max_connections'
      status: 'True'
      type: ACK.Terminal

[ack-bot]

Issues go stale after 180d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 60d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/aws-controllers-k8s/community.
/lifecycle stale