compliance report results percentage give incorrect results

[chen-keinan]

Description

The compliance report results percentage incorrect give results.
The rego checks are basing on the defsec validation engine to run k8s policies on k8s resources.
As for today the defsec validation engine do not know how to execute specific policy on a specific resource.

For example :
A rego policy how validate NetworkPolicy resource should run only on NetworkPolicy however today the engine execute it also on workloads which result with status of PASS

Proposed solutions:
short terms:

• change on trivy : replace Compliance with Status (PASS / FAIL) and show only one of two states on each the control PASS or FAIL

Long term (right solution):

• change on defsec: add additional fields on each rego policy metadata , Group, Version, Name and Namespace (optional) the rego engine will validate if the input match these new metadata fields and if match found the check will no be executed

[chen-keinan]

As immediate solution we will progress with option one , short term solution:
report might look like this

Summary Report for compliance: nsa
Summary Report for compliance: nsa
┌──────┬──────────┬──────────────────────────────────────────────────────────┬────────┬────────┐
│  ID  │ Severity │                       Control Name                       │ Status │ Issues │
├──────┼──────────┼──────────────────────────────────────────────────────────┼────────┼────────┤
│ 1.0  │ MEDIUM   │                   Non-root containers                    │  FAIL  │   11   │
│ 1.1  │ LOW      │             Immutable container file systems             │  FAIL  │   9    │
│ 1.2  │ HIGH     │             Preventing privileged containers             │  FAIL  │   1    │
│ 1.3  │ HIGH     │           Share containers process namespaces            │  PASS  │   0    │
│ 1.4  │ HIGH     │              Share host process namespaces               │  FAIL  │   6    │
│ 1.5  │ HIGH     │                   Use the host network                   │  PASS  │   0    │
│ 1.6  │ LOW      │  Run with root privileges or with root group membership  │  PASS  │   0    │
│ 1.7  │ MEDIUM   │         Restricts escalation to root privileges          │  FAIL  │   11   │
│ 1.8  │ MEDIUM   │        Sets the SELinux context of the container         │  PASS  │   0    │
│ 1.9  │ MEDIUM   │ Restrict a container's access to resources with AppArmor │  FAIL  │   13   │
│ 1.10 │ LOW      │   Sets the seccomp profile used to sandbox containers.   │  FAIL  │   13   │
│ 1.11 │ MEDIUM   │          Protecting Pod service account tokens           │  FAIL  │   1    │
│ 1.12 │ MEDIUM   │    Namespace kube-system should not be used by users     │  FAIL  │   4    │
│ 2.0  │ MEDIUM   │           Pod and/or namespace Selectors usage           │  PASS  │   0    │
│ 3.0  │ CRITICAL │      Use CNI plugin that supports NetworkPolicy API      │        │        │
│ 4.0  │ MEDIUM   │      Use ResourceQuota policies to limit resources       │  PASS  │   0    │
│ 4.1  │ MEDIUM   │        Use LimitRange policies to limit resources        │  PASS  │   0    │
│ 5.0  │ CRITICAL │            Control plan disable insecure port            │        │        │
│ 5.1  │ CRITICAL │                Encrypt etcd communication                │  PASS  │   0    │
│ 6.0  │ CRITICAL │            Ensure kube config file permission            │  PASS  │   0    │
│ 6.1  │ CRITICAL │       Check that encryption resource has been set        │  FAIL  │   1    │
│ 6.2  │ CRITICAL │                Check encryption provider                 │  PASS  │   0    │
│ 7.0  │ CRITICAL │            Make sure anonymous-auth is unset             │  FAIL  │   1    │
│ 7.1  │ CRITICAL │            Make sure -authorization-mode=RBAC            │  PASS  │   0    │
│ 8.0  │ HIGH     │                Audit policy is configure                 │  PASS  │   0    │
│ 8.1  │ MEDIUM   │               Audit log path is configure                │  FAIL  │   1    │
│ 8.2  │ MEDIUM   │                     Audit log aging                      │  FAIL  │   1    │
└──────┴──────────┴──────────────────────────────────────────────────────────┴────────┴────────┘

{
  "ID": "0001",
  "Title": "nsa",
  "SummaryControls": [
    {
      "ID": "1.0",
      "Name": "Non-root containers",
      "Severity": "MEDIUM",
      "TotalFail": 11
    },
    {
      "ID": "1.1",
      "Name": "Immutable container file systems",
      "Severity": "LOW",
      "TotalFail": 9
    },
    {
      "ID": "1.2",
      "Name": "Preventing privileged containers",
      "Severity": "HIGH",
      "TotalFail": 1
    },
    {
      "ID": "1.3",
      "Name": "Share containers process namespaces",
      "Severity": "HIGH",
      "TotalFail": 0
    },
    {
      "ID": "1.4",
      "Name": "Share host process namespaces",
      "Severity": "HIGH",
      "TotalFail": 6
    },
    {
      "ID": "1.5",
      "Name": "Use the host network",
      "Severity": "HIGH",
      "TotalFail": 0
    },
    {
      "ID": "1.6",
      "Name": "Run with root privileges or with root group membership",
      "Severity": "LOW",
      "TotalFail": 0
    },
    {
      "ID": "1.7",
      "Name": "Restricts escalation to root privileges",
      "Severity": "MEDIUM",
      "TotalFail": 11
    },
    {
      "ID": "1.8",
      "Name": "Sets the SELinux context of the container",
      "Severity": "MEDIUM",
      "TotalFail": 0
    },
    {
      "ID": "1.9",
      "Name": "Restrict a container's access to resources with AppArmor",
      "Severity": "MEDIUM",
      "TotalFail": 13
    },
    {
      "ID": "1.10",
      "Name": "Sets the seccomp profile used to sandbox containers.",
      "Severity": "LOW",
      "TotalFail": 13
    },
    {
      "ID": "1.11",
      "Name": "Protecting Pod service account tokens",
      "Severity": "MEDIUM",
      "TotalFail": 1
    },
    {
      "ID": "1.12",
      "Name": "Namespace kube-system should not be used by users",
      "Severity": "MEDIUM",
      "TotalFail": 4
    },
    {
      "ID": "2.0",
      "Name": "Pod and/or namespace Selectors usage",
      "Severity": "MEDIUM",
      "TotalFail": 0
    },
    {
      "ID": "3.0",
      "Name": "Use CNI plugin that supports NetworkPolicy API",
      "Severity": "CRITICAL"
    },
    {
      "ID": "4.0",
      "Name": "Use ResourceQuota policies to limit resources",
      "Severity": "MEDIUM",
      "TotalFail": 0
    },
    {
      "ID": "4.1",
      "Name": "Use LimitRange policies to limit resources",
      "Severity": "MEDIUM",
      "TotalFail": 0
    },
    {
      "ID": "5.0",
      "Name": "Control plan disable insecure port",
      "Severity": "CRITICAL"
    },
    {
      "ID": "5.1",
      "Name": "Encrypt etcd communication",
      "Severity": "CRITICAL",
      "TotalFail": 0
    },
    {
      "ID": "6.0",
      "Name": "Ensure kube config file permission",
      "Severity": "CRITICAL",
      "TotalFail": 0
    },
    {
      "ID": "6.1",
      "Name": "Check that encryption resource has been set",
      "Severity": "CRITICAL",
      "TotalFail": 1
    },
    {
      "ID": "6.2",
      "Name": "Check encryption provider",
      "Severity": "CRITICAL",
      "TotalFail": 0
    },
    {
      "ID": "7.0",
      "Name": "Make sure anonymous-auth is unset",
      "Severity": "CRITICAL",
      "TotalFail": 1
    },
    {
      "ID": "7.1",
      "Name": "Make sure -authorization-mode=RBAC",
      "Severity": "CRITICAL",
      "TotalFail": 0
    },
    {
      "ID": "8.0",
      "Name": "Audit policy is configure",
      "Severity": "HIGH",
      "TotalFail": 0
    },
    {
      "ID": "8.1",
      "Name": "Audit log path is configure",
      "Severity": "MEDIUM",
      "TotalFail": 1
    },
    {
      "ID": "8.2",
      "Name": "Audit log aging",
      "Severity": "MEDIUM",
      "TotalFail": 1
    }
  ]
}