Using profile-specific ConfigMap-entry leads to duplicate key-exception

[stoetti]

Hi!

I am in the process of migrating to kubernetes-config for our Spring Boot applications. We are relying on profile-specific configurations which might override some default values based on activated profiles.
My understanding from the documentation was that I could defined a ConfigMap like this

apiVersion: v1
kind: ConfigMap
metadata:
  name: test-application
data:
  test-application.properties: |
    key=value1
  test-application-profileA.properties: |
    key=value2

If the application with the name test-application is started with only default-profiles active the value for property key should be value1 and if started with the profile profileA active the key should have value2.
The current implementation throws a duplicate key error when trying to combine the two "properties-files" into a single map ignoring the fact that one should, at lease in my opinion, take precedence over the other. The exception is thrown here

spring-cloud-kubernetes/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/config/SourceDataEntriesProcessor.java

Line 89 in a4e2115

return input.entrySet().stream().map(e -> extractProperties(e.getKey(), e.getValue(), fileNames, environment))

For reproduction I created a small sample-project including a unit-test the demonstrates the behavior https://github.com/stoetti/spring-cloud-kubernetes-config-profile

[wind57]

a comment just to let you know that I will take a look at this one a bit closer soon and will reply. thank you for opening it

[wind57]

My understanding from the documentation

can you show where exactly please?

[stoetti]

I re-read the part of the documentation at indeed it was my assumption as the documentation only states that all of the entries are loaded (Bullet points after the first sample configMap).
https://docs.spring.io/spring-cloud-kubernetes/reference/property-source-config/configmap-propertysource.html

I simply assumed that the values from the profile-specific entry would override the others as this is the logic when file-based configuration is used as PropertySources.

I'd suggest to either update the documentation that duplicate keys are not supported multiple items in the configMap or implement the same mechanism for overriding properties via profile-configuration.

[wind57]

I need to look at some other things first, to recall how we handle them, because your use case sounds very valid to me.

[wind57]

yeah, this looks inconsistent with the other support that we have. I've pushed a PR against your project to show you what I mean. We give the option to specify something like this as sources for your application:

spring:
  cloud:
    kubernetes:
      config:
        sources:
          - name: my-source
            namespace: default
            include-profile-specific-sources: true

specifically look at include-profile-specific-sources.

It that example, we read two config maps my-source and because include-profile-specific-sources=true, we also will read my-source-test. Both of these contain the same key: key=defaultValue and key=non-default-value, but the app does not fail this time. Instead the profile based value is overridden, we even have a comment in this regard in our code base:

// this is an ordered stream, and it means that non-profile based sources will be
// processed before profile based sources. This way, we replicate that
// "application-dev.yaml" overrides properties from "application.yaml"

I think this is a bug that we do not do the same for your use case. I can fix it, but I need @ryanjbaxter to agree on this.

[ryanjbaxter]

I agree it should override the value and not throw an exception

[wind57]

@stoetti I have a PR to fix this, here. FYI

[stoetti]

@stoetti I have a PR to fix this, here. FYI

Thanks, that's great news 🤩