ActiveMQArtemis CR OutOfSync, mismatched checksum on property file

[john9x]

Describe the bug
The cluster deployed with config-bp secret

apiVersion: v1
kind: Secret
metadata:
  name: config-bp
stringData:
  config.json: |
    {
      "addressSettings": { ... }
    ....
    }

Status of CR always OutOfSync

     Message:               out of sync on pod artemis-broker-ss-0, mismatched checksum on property file config.json, expected: 2594123130, current: 46476296. A delay can occur before a volume mount projection is refreshed.                                                                                                                                                                                                           
│     Reason:                OutOfSync                                                                                                                                                                                 │
│     Status:                False                                                                                                                                                                                     │
│     Type:                  BrokerPropertiesApplied

Tip

Vote this issue reacting with 👍 or 👎

[john9x]

Maybe related to #633 - I have addresses and queues with dots

[gtully]

can you paste in an example of the -bp secret content that causes the issue?

[john9x]

@gtully

apiVersion: v1
kind: Secret
metadata:
  name: config-bp
stringData:
  config.json: |
    {
      "addressSettings": {
        "#": {         
          "maxDeliveryAttempts": 2,
          "redeliveryDelay": 10000,
          "maxRedeliveryDelay": 100000,
          "addressFullPolicy": "PAGE"
        },
        "server.push": {
          "maxDeliveryAttempts": 2
        },
        "server.email": {
          "maxDeliveryAttempts": 5
        }
      },
      "addressConfigurations": {        
        "server.push": {
          "routingTypes": "ANYCAST",
          "queueConfigs": {
            "server.push": {
              "routingType": "ANYCAST"
            }
          }
        },
        "server.email": {
          "routingTypes": "ANYCAST",
          "queueConfigs": {
            "server.email": {
              "routingType": "ANYCAST"
            }
          }
        }
      },
      "securityRoles": {
        "#": {
          "admin": {
            "createNonDurableQueue": true,
            "deleteNonDurableQueue": true,
            "createDurableQueue": true,
            "deleteDurableQueue": true,
            "createAddress": true,
            "deleteAddress": true,
            "consume": true,
            "browse": true,
            "send": true,
            "manage": true
          }
        },
        "emd.#": {
          "admin": {
            "createNonDurableQueue": true,
            "deleteNonDurableQueue": true,
            "createDurableQueue": true,
            "deleteDurableQueue": true,
            "createAddress": true,
            "deleteAddress": true,
            "consume": true,
            "browse": true,
            "send": true,
            "manage": true
          },
          "mqtt_device": {
            "createNonDurableQueue": true,
            "deleteNonDurableQueue": true,
            "createDurableQueue": true,
            "deleteDurableQueue": true,
            "createAddress": true,
            "deleteAddress": true,
            "consume": false,
            "browse": false,
            "send": true,
            "manage": false
          }
        }
      }
    }

Message: out of sync on pod artemis-broker-ss-0, mismatched checksum on property file config.json, expected: 3352354407, current: 160423527.

[john9x]

From my tests the issue appears when configuration contains names with dots.

[john9x]

Also, this issue hides real problem: addressFullPolicy it's a mistake, must be addressFullMessagePolicy
without dotted names I see
Message: some properties from config-bp resulted in error on pod artemis-broker-ss-0 : reasons: map[config.json:[{"value":"addressSettings.#.addressFullPolicy=PAGE","reason":"java.lang.reflect.InvocationTargetException: No accessor method descriptor for: addressFullPolicy on: class org.apache.activemq.artemis.core.settings.impl.AddressSettings"}]]

[gtully]

thanks for the detail. the support for json may be the trigger.
I suspect this problem can be avoided (worked around) by the more verbose nested properties format. ie: build up a config.properties

I will take a look.

[gtully]

it seems the problem relates to the order in which json is mapped to properties and fed through the alder32 hash. The treatment of the dot results in a quoted key value, and that effects order!

[john9x]

@gtully
IMHO

function SortedKeyValuePairsFromMap at activemqartemis_reconciler.go sorts properties after dotted keys were quoted meanwhile
org.apache.activemq.artemis.core.config.impl.ConfigurationImpl.InsertionOrderedProperties#loadJsonObject sorts no quoted keys

[john9x]

yeah. the issue is in sorting.
OK - quoting not changes sorting

stringData:
  config.json: |
    {
      "addressSettings": {
        "a.a": {         
          "maxDeliveryAttempts": 2
        },
        "b.b": {
          "maxDeliveryAttempts": 2
        }
      }
    }

NOT OK - quoting changes sorting and addressSettings."b.b".maxDeliveryAttempts=2 becomes before addressSettings.a.maxDeliveryAttempts=2

stringData:
  config.json: |
    {
      "addressSettings": {
        "a": {         
          "maxDeliveryAttempts": 2
        },
        "b.b": {
          "maxDeliveryAttempts": 2
        }
      }
    }

[gtully]

@john9x that PR has the fix, feel free to review, all feedback welcome. Thanks for the report and analysis

[gtully]

a workaround, not a great one, but one that should work is to put keys with dots in a separate config-with-dots.json key in your secret.