Define how controllers / operator should be externally configured

[metacosm]

#228 was created as an experiment to see how things could be done but I think we need to formalize how configuration should be handled so that users can configure their controllers / operator without having to modify / rebuild the code.

[metacosm]

Proposal

• Be able to configure things in an application.properties or application.yml file.
• Configure the client at the operator level
• Name controllers so that they can be configured individually in particular for scope and retry
• Config classes should be POJOs to be extended by frameworks for specific support (i.e. Spring Boot, Quarkus…)

Client

• io.javaoperatorsdk.kubernetes.client prefix
• Properties similar to what's already defined in OperatorProperties
• OpenShift status should probably be automatically inferred and not "hardcoded"

Controllers

• io.javaoperatorsdk.controller.<controller name> prefix
• io.javaoperatorsdk.controller.<controller name>.namespaces: comma-separated list of target namespaces the controller watches, if no value is provided, current namespace (i.e. the one the client is connected to) is assumed. If the value is (or contains) ALL_NAMESPACES, the controller watches all namespaces.
• io.javaoperatorsdk.controller.<controller name>.retry prefix: properties similar to what's already defined in RetryProperties

/cc @adam-sandor @kirek007 @psycho-ir @csviri

[s-soroosh]

Thanks, @metacosm, the proposal looks good to me.
Maybe one thing to add, we may need a way to make the operator cluster-scoped.
Adding a new field like:
io.javaoperatorsdk.controller.<controller name>.scope = namespace|cluster may help in this regard.

[metacosm]

Thanks, @metacosm, the proposal looks good to me.
Maybe one thing to add, we may need a way to make the operator cluster-scoped.
Adding a new field like:
io.javaoperatorsdk.controller.<controller name>.scope = namespace|cluster may help in this regard.

We could make it more explicit but it should be covered using io.javaoperatorsdk.controller.<controller name>.namespaces=ALL_NAMESPACES, no?

[s-soroosh]

Thanks, @metacosm, the proposal looks good to me.
Maybe one thing to add, we may need a way to make the operator cluster-scoped.
Adding a new field like:
io.javaoperatorsdk.controller.<controller name>.scope = namespace|cluster may help in this regard.

We could make it more explicit but it should be covered using io.javaoperatorsdk.controller.<controller name>.namespaces=ALL_NAMESPACES, no?

I don't think so, a CR can be cluster-scoped, that means its resources don't live in none of the namespaces.
e.g. PersistentVolume is a cluster-scoped resource.

[metacosm]

Ah yes, forgot about that. Let's add the scope property then.

[kirek007]

Thanks, @metacosm, the proposal looks good to me.
Maybe one thing to add, we may need a way to make the operator cluster-scoped.
Adding a new field like:
io.javaoperatorsdk.controller.<controller name>.scope = namespace|cluster may help in this regard.

Definitely we should go this way. It's the way objects in kubernetes are scoped.

[metacosm]

One issue that popped out after the work done in #238, is how should we configure the Kubernetes client. The Quarkus extension relies on the Kubernetes Client extension to retrieve a fully configured client. However, this means that the client is configured using quarkus.kubernetes-client. prefixed properties. Moreover, the available properties cover a lot more than what is currently supported in the operator SDK.

The question is therefore whether or not we should strive to expose a simplified / unified k8s client configuration properties for all implementations or are we OK to have Quarkus-based implementations use a different configuration for that aspect? What do you think @adam-sandor @psycho-ir @csviri?