Missing affinity and tolerations customization for log, db, prometheus in helm template for tenant

[vldorl]

Hello!
I have nodes: worker-1, worker-2, worker-3, worker-N
Nodes worker-{1..3} are tainted and labeled for selecting.
I deploy tenant to nodes worker-{1..3} using affinity rules and tolerations with helm chart.

Expected Behavior

All pool pods and service pods (log, log:db, prometheus) running on nodes worker-{1..3}.

Current Behavior

Pool pods are assigned to nodes as expected.
Pods log, log:db, prometheus are assigned randomly, affinity rules and tolerations are ignored due to absent customized section in tenant template.

Possible Solution

To fix template (e.g. https://github.com/minio/operator/blob/master/helm/tenant/templates/tenant.yaml#L249) adding customization for affinity and tolerations.

Steps to Reproduce (for bugs)

1. Deploy operator using helm chart
2. Set affinity and tolerations for pool, log, log:db and prometheus in values.yaml for tenant chart
3. Deploy tenant
4. See running pool pods on nodes worker{1..3} and rest on random nodes

Context

I have 2 node tiers: stateful and stateless. Minio pool pods and rest have to be deployed on stateful, which have storage.

Your Environment

• Version used: 4.4.28
• Environment name and version: k8s v1.23.7
• Server type and version: Virtual Machine
• Operating System and version: Linux worker-1 5.4.17-2136.307.3.1.el8uek.x86_64 #2 SMP Mon May 9 17:29:47 PDT 2022 x86_64 x86_64 x86_64 GNU/Linux

[harshavardhana]

Feel free to send a PR @vldorl

[dnskr]

@vldorl Hi! There is similar PR #674
Do you want to create a PR?

[vldorl]

Hello, @dnskr!
Already #1265

[pjuarezd]

Thank you for the PR @vldorl, It seems that as is will not work, please merge this PR into your branch with some bugfixes

vldorl#1