Skip to content

Add Kubernetes CIS Benchmark integration #2930

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 85 commits into from
Apr 11, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
8ee900d
Feature/kube cis package (#1)
DaveSys911 Dec 23, 2021
528f290
working version of adding index template
CohenIdo Jan 26, 2022
2686ccd
add index tempplate per resource
CohenIdo Jan 26, 2022
031e03e
update base fields
CohenIdo Jan 26, 2022
f62ddf9
working version of adding data view
CohenIdo Jan 27, 2022
6c5d9ad
drop unused fields
CohenIdo Jan 27, 2022
6465e56
rename and fix text
Jan 30, 2022
fea5f60
fix deploy name
eyalkraft Jan 31, 2022
54c166f
use elastic-package to create data-stream
eyalkraft Jan 31, 2022
2af5347
pr comments
CohenIdo Jan 31, 2022
5ad3b5e
Description update
CohenIdo Jan 31, 2022
7a8a419
add group
CohenIdo Jan 31, 2022
0e9e168
.
CohenIdo Jan 31, 2022
a410e55
Merge pull request #6 from build-security/integration-update-looks
eyalkraft Feb 1, 2022
05fecb6
set multiple to false
eyalkraft Feb 2, 2022
e17f006
Merge pull request #7 from build-security/single-instance-per-policy
eyalkraft Feb 2, 2022
2a01de0
rename
eyalkraft Feb 8, 2022
912bc1c
Merge pull request #5 from build-security/add_data_view
CohenIdo Feb 15, 2022
fef48f2
Merge pull request #3 from build-security/add_fields_mapping
CohenIdo Feb 15, 2022
f6feb62
update assets path
CohenIdo Feb 15, 2022
1e3e458
replace quotes
CohenIdo Feb 15, 2022
c84fe61
rebane kibana assets filename
CohenIdo Feb 15, 2022
9764185
Merge pull request #10 from build-security/moveAssetsToNewIntegration
CohenIdo Feb 15, 2022
ec8b0fa
format existing assets
CohenIdo Feb 16, 2022
540092f
working version of adding vars
CohenIdo Feb 17, 2022
eb23f3f
clean
CohenIdo Feb 17, 2022
e2525eb
add default policy
CohenIdo Feb 17, 2022
b6fde4e
update index pattern acording to cloudbeat change
CohenIdo Feb 28, 2022
6b14e7d
format
CohenIdo Feb 28, 2022
3d62761
Merge pull request #14 from build-security/updateDataViewIndexPattern
CohenIdo Feb 28, 2022
337a0e2
hide data yml
CohenIdo Feb 28, 2022
a47d2e8
wokring version of index template competible with latest cloudbeat. p…
CohenIdo Mar 1, 2022
c7e0a0c
add cluster id mapping
CohenIdo Mar 1, 2022
b555cc2
Merge pull request #15 from build-security/updateIndexTemplate
CohenIdo Mar 1, 2022
072e0ac
Update CIS Kubernetes benchmark Kibana version (#16)
ari-aviran Mar 1, 2022
ac4eb87
aff latest transform based on resource id and new naming convention
CohenIdo Mar 7, 2022
474e812
added type field to template
Mar 8, 2022
b655364
transforms with index templates
CohenIdo Mar 9, 2022
790d005
Merge branch 'master' into addYamlVar
CohenIdo Mar 9, 2022
05a84b5
Delete .prettierignore
CohenIdo Mar 9, 2022
45374d7
Delete settings.json
CohenIdo Mar 9, 2022
d24cc18
Delete cis_kubernetes_benchmark-9129a080-7f48-11ec-8249-431333f83c5d.…
CohenIdo Mar 9, 2022
d66ba66
make datayaml non required var
CohenIdo Mar 9, 2022
053c78d
Merge branch 'addYamlVar' of https://github.com/build-security/integr…
CohenIdo Mar 9, 2022
4b37ce4
Merge pull request #12 from build-security/addYamlVar
CohenIdo Mar 9, 2022
66baf35
version changes
JordanSh Mar 9, 2022
548229d
Merge pull request #17 from build-security/adding-resource-type-keyword
JordanSh Mar 9, 2022
b9bd783
version changes
JordanSh Mar 9, 2022
4e062ed
Merge pull request #19 from build-security/manifest-ver-update
JordanSh Mar 9, 2022
7dd273c
Merge branch 'master' into addTransforms
CohenIdo Mar 9, 2022
cfdb03a
Merge branch 'master' into addTransforms
CohenIdo Mar 9, 2022
8edddc2
working version of transforms, score index template need to be fix
CohenIdo Mar 10, 2022
3708e24
working version including score index template
CohenIdo Mar 10, 2022
71f7635
remove template
CohenIdo Mar 10, 2022
aa5d7e6
update version
CohenIdo Mar 10, 2022
414f28d
calc score per cluster
CohenIdo Mar 10, 2022
0320121
remove unused parmas from conf
CohenIdo Mar 16, 2022
c05310a
Merge branch 'elastic:main' into master
DaveSys911 Mar 20, 2022
138e62d
Merge branch 'master' into rename-beat
eyalkraft Mar 23, 2022
9b8eff0
Merge pull request #8 from build-security/rename-beat
eyalkraft Mar 24, 2022
c4105ea
change indices names
CohenIdo Mar 24, 2022
a4935fb
Merge branch 'elastic:main' into master
DaveSys911 Mar 29, 2022
dd13dfd
Merge pull request #18 from build-security/addTransforms
CohenIdo Mar 30, 2022
75657e3
Merge branch 'elastic:main' into initial-merge-cis-benchmark
eyalkraft Mar 30, 2022
cae072b
fixes
eyalkraft Mar 30, 2022
aa22e21
Merge branch 'main' into initial-merge-cis-benchmark
eyalkraft Mar 30, 2022
66ba5d8
retrigger checks
eyalkraft Mar 30, 2022
00d1601
fix changelog
eyalkraft Mar 30, 2022
02b46a7
update screenshots
eyalkraft Mar 31, 2022
3e75705
modify screenshots again
eyalkraft Mar 31, 2022
3cd1643
Merge branch 'main' into initial-merge-cis-benchmark
eyalkraft Apr 6, 2022
6ae7dfa
changes to score transform
JordanSh Apr 6, 2022
9f662f2
sort codeowners
eyalkraft Apr 6, 2022
ce6625c
remove codeowners trailing newline
eyalkraft Apr 6, 2022
f5b9442
changelog update
JordanSh Apr 6, 2022
4f75c3b
Fixed all findings index pattern so it won't overlap with latest find…
kfirpeled Apr 7, 2022
20c917f
Merge pull request #20 from build-security/change-score-transform
JordanSh Apr 7, 2022
f080e37
Merge branch 'master' into fix_index_pattern
eyalkraft Apr 7, 2022
ede7667
Merge pull request #21 from build-security/fix_index_pattern
eyalkraft Apr 7, 2022
f23f7e1
Merge branch 'elastic:main' into initial-merge-cis-benchmark
eyalkraft Apr 7, 2022
285c74d
update from build-security/master
eyalkraft Apr 7, 2022
5feeb48
fix codeowners
eyalkraft Apr 7, 2022
1a1bd8f
remove newline
eyalkraft Apr 7, 2022
4674dd6
remove transforms
eyalkraft Apr 7, 2022
da4a3d7
update owner group name
eyalkraft Apr 7, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
/packages/cassandra @elastic/integrations
/packages/cef @elastic/security-external-integrations
/packages/checkpoint @elastic/security-external-integrations
/packages/cis_kubernetes_benchmark @elastic/cloud-security-posture
/packages/cisco_asa @elastic/security-external-integrations
/packages/cisco_duo @elastic/security-external-integrations
/packages/cisco_ftd @elastic/security-external-integrations
Expand Down
Empty file.
6 changes: 6 additions & 0 deletions packages/cis_kubernetes_benchmark/changelog.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# newer versions go on top
- version: "0.0.1"
changes:
- description: Initial draft of the package
type: enhancement
link: https://github.com/elastic/integrations/pull/2930
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
processors:
- add_cluster_id: ~
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
- name: data_stream.type
type: constant_keyword
description: Data stream type.
- name: data_stream.dataset
type: constant_keyword
description: Data stream dataset.
- name: data_stream.namespace
type: constant_keyword
description: Data stream namespace.
- name: '@timestamp'
type: date
description: Event timestamp.
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
- name: cycle_id
type: text
multi_fields:
- name: keyword
type: keyword
ignore_above: 1024
- name: type
type: text
multi_fields:
- name: keyword
type: keyword
ignore_above: 1024
- name: resource_id
type: text
multi_fields:
- name: keyword
type: keyword
ignore_above: 1024
- name: cluster_id
type: text
multi_fields:
- name: keyword
type: keyword
ignore_above: 1024
- name: agent
type: group
fields:
- name: id
type: text
description: Agent ID
multi_fields:
- name: keyword
type: keyword
ignore_above: 1024
- name: resource
type: group
fields:
- name: type
type: text
description: Source type of the resource
multi_fields:
- name: keyword
type: keyword
ignore_above: 1024
- name: filename
type: text
description: Resource filename
multi_fields:
- name: keyword
type: keyword
ignore_above: 1024
- name: rule
type: group
fields:
- name: name
type: keyword
description: Rule name
multi_fields:
- name: keyword
type: keyword
ignore_above: 1024
- name: benchmark
type: group
fields:
- name: name
type: text
description: Benchmark name
multi_fields:
- name: keyword
type: keyword
ignore_above: 1024
- name: result
type: group
fields:
- name: evaluation
type: text
description: Rule result
multi_fields:
- name: keyword
type: keyword
ignore_above: 1024
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
title: "Findings"
type: logs
streams:
- input: cloudbeat
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is cloudbeat already available in the elastic-agent-complete Docker image?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I understand it should be in the elastic-agent-complete Docker image as a result of this PR to the agent (and these artifacts), But it is not yet since the CI job that builds the artifacts for this Docker image has failed for the last 6 days in a row (unrelated to cloudbeat).
@DaveSys911 @oren-zohar If you have anything to add please comment

Copy link
Contributor

@mtojek mtojek Mar 31, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, then most likely you have another blocker until that happens. Probably it isn't serious, it's rather something to be aware of.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure this was resolved.
@oren-zohar please approve

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mtojek @eyalkraft the 8.2.0-snapshot/elastic-agent-complete shouldn't contain cloudbeat, but a path to the artifactory. Unfortunately, we just found a bug in the path, so the agent will not be able to pull cloudbeat in the latest docker image.
We're working on a fix for it rn and I'll post here when it's fixed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the information, Oren!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Release version bump here: #3003

title: K8s CIS Compliance
description: Check CIS Benchmark compliance
28 changes: 28 additions & 0 deletions packages/cis_kubernetes_benchmark/docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# CIS Kubernetes Benchmark

This integration compares [Kubernetes](https://kubernetes.io/) configuration against CIS benchmark checks. It computes a score that ranges between 0 - 100. This integration requires access to node files, node processes, and the Kuberenetes api-server therefore it assumes the agent will be installed as a [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) with the proper [Roles](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole) and [RoleBindings](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding) attached.

See agent [installation instructions](https://www.elastic.co/guide/en/fleet/current/running-on-kubernetes-managed-by-fleet.html).

Additionally, In order for the integration to be installed, The Cloud Security Posture Kibana plugin must be enabled.

This could be done by adding the following configuration line to `kibana.yml`:
```
xpack.cloudSecurityPosture.enabled: true
```

## Leader election

To collect cluster level data (compared to node level information) the integration makes use of the [leader election](https://www.elastic.co/guide/en/fleet/master/kubernetes_leaderelection-provider.html) mechanism.
This mechanism assures that the cluster level data is collected by only one of the agents running as aprt of the DeamonSet and not by all of them.

Cluster level data example: List of the running pods.
Node level data examle: kubelet configuration.

## Compatibility

The Kubernetes package is tested with Kubernetes 1.21.x

## Dashboard

CIS Kubernetes Benchmark integration is shipped including default dashboards and screens to manage the benchmark rules and inspect the compliance score and findings.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"attributes": {
"description": "",
"title": "logs-cis_kubernetes_benchmark.findings-*"
},
"coreMigrationVersion": "8.1.0",
"id": "cis_kubernetes_benchmark-9129a080-7f48-11ec-8249-431333f83c5f",
"migrationVersion": {
"index-pattern": "8.0.0"
},
"type": "index-pattern",
"updated_at": "2022-01-27T08:10:19.277Z",
"version": "WzMwNDY5LDFd"
}
57 changes: 57 additions & 0 deletions packages/cis_kubernetes_benchmark/manifest.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
format_version: 1.0.0
name: cis_kubernetes_benchmark
title: "CIS Kubernetes Benchmark"
version: 0.0.1
license: basic
description: "Check Kubernetes cluster compliance with the Kubernetes CIS benchmark."
type: integration
categories:
- containers
- kubernetes
release: experimental
conditions:
kibana.version: "^8.2.0"
screenshots:
- src: /img/dashboard.png
title: Dashboard page
size: 1293x718
type: image/png
- src: /img/findings.png
title: Findings page
size: 3134x1740
type: image/png
- src: /img/findings-flyout.png
title: Detailed view of a single finding
size: 3176x1748
type: image/png
- src: /img/benchmarks.png
title: Benchmarks page
size: 3168x1752
type: image/png
- src: /img/rules.png
title: Rules page
size: 3160x1708
type: image/png
icons:
- src: /img/cis-kubernetes-benchmark-logo.svg
title: CIS Kubernetes Benchmark logo
size: 32x32
type: image/svg+xml
policy_templates:
- name: findings
title: Compliance findings
description: Collect findings
multiple: false
inputs:
- type: cloudbeat
title: Enable CIS Kubernetes Benchmark
description: Collecting findings
vars:
- name: dataYaml
type: yaml
title: Rules Activation Yaml
multi: false
required: false
show_user: false
owner:
github: elastic/cloud-security-posture