Starboard deployment managed through ArgoCD not working

[AnaisUrlichs]

I used this guide by ArgoCD to deploy Starboard: https://argo-cd.readthedocs.io/en/stable/user-guide/helm/

Here is my YAML used to deploy:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: starboard
  namespace: argocd
spec:
  project: default
  source:
    chart: starboard-operator
    repoURL: https://aquasecurity.github.io/helm-charts/
    targetRevision: 0.9.1
    helm:
      releaseName: starboard
  destination:
    server: https://kubernetes.default.svc
    namespace: starboard-system

When I deploy starboard through Helm directly without ArgoCD, it works like expected.
However, with ArgoCD, the operator pod would spin up and then Crash after a few minutes, the logs show the following errors:

Any ideas on how I can fix it or further debug?

[chen-keinan]

@AnaisUrlichs its look like the argocd helm deployment doesn't pick up the rbac and service account resources

[AnaisUrlichs]

So here is what I found -- when I use ArgoCD to install the Helm Chart, the ClusterRole from here https://github.com/aquasecurity/starboard/blob/main/deploy/helm/templates/rbac.yaml gets installed as Role -- similar, when you run Helm template, it just shows the role but not the ClusterRole

helm template starboard-operator \                                                                       
  aqua/starboard-operator --namespace starboard-system \
  --version 0.9.1

HOWEVER, if I install the Helm Chart directly through Helm, it will install the ClusterRole normally -- SO I think this is a bug on our end. Here are the screenshots of the

ClusterRole via Helm

The Role via ArgoCD

@chen-keinan @danielpacak why does it only show Role with the Helm template command but not the ClusterRole? and how is the ClusterRole then created when you run Helm install instead of Helm template?

[danielpacak]

Our Helm chart has logic to create Roles or ClusterRoles depending on target namespaces. For example, if you watch a single namespace it will only create Roles. On the other hand, if you configure it to watch the whole cluster it will create ClusterRoles.

What is the target namespace in case you assume our Helm chart does not work as expected?

[danielpacak]

Another look at our Helm chart reveals that indeed it has some problem with so called "same namespace" Install Mode, i.e. when you deploy it in the starboard-system namespace and watch the same starboard-system namespace.

helm install starboard-operator ./deploy/helm \
  --namespace starboard-system \
  --create-namespace

This is probably what ArgoCD does in your case when you don't set the targetNamespaces Helm value.

I can reproduce in my cluster with the following error:

E0304 16:20:35.908432       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.23.4/tools/cache/reflector.go:167:
Failed to watch *v1alpha1.ClusterConfigAuditReport: failed to list *v1alpha1.ClusterConfigAuditReport:
clusterconfigauditreports.aquasecurity.github.io is forbidden:
User "system:serviceaccount:starboard-system:starboard-operator" cannot list resource
"clusterconfigauditreports" in API group "aquasecurity.github.io" at the cluster scope

[danielpacak]

I'm wondering if you can set the targetNamespaces Helm values in your ArgoCD descriptor to confirm that the problem is only with the same namespace install mode, i.e. do with ArgoCD equivalent to:

helm install starboard-operator ./deploy/helm \
  --namespace starboard-system \
  --create-namespace \
  --set="targetNamespaces=default"

[danielpacak]

I created #1001 to fix our Helm chart.