Skip to content

Commit a77632b

Browse files
libanderlibander
committed
xref fix
1 parent 0056361 commit a77632b

36 files changed

+2975
-0
lines changed

‎_topic_maps/_topic_map.yml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2818,6 +2818,23 @@ Topics:
28182818
File: logging-5-8-release-notes
28192819
- Name: Logging 5.7
28202820
File: logging-5-7-release-notes
2821+
- Name: Logging 6.0
2822+
Dir: logging-6.0
2823+
Topics:
2824+
- Name: Release notes
2825+
File: log6x-release-notes
2826+
- Name: About logging 6.0
2827+
File: log6x-about
2828+
- Name: Upgrading to Logging 6.0
2829+
File: log6x-upgrading-to-6
2830+
- Name: Configuring log forwarding
2831+
File: log6x-clf
2832+
- Name: Configuring LokiStack storage
2833+
File: log6x-loki
2834+
- Name: Visualization for logging
2835+
File: log6x-visual
2836+
# - Name: API reference 6.0
2837+
# File: log6x-api-reference
28212838
- Name: Support
28222839
File: cluster-logging-support
28232840
- Name: Troubleshooting logging

‎modules/log6x-audit-log-filtering.adoc

Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * observability/logging/logging-6.0/log6x-clf.adoc
4+
5+
:_mod-docs-content-type: CONCEPT
6+
[id="log6x-audit-filtering_{context}"]
7+
= Overview of API audit filter
8+
OpenShift API servers generate audit events for each API call, detailing the request, response, and the identity of the requester, leading to large volumes of data. The API Audit filter uses rules to enable the exclusion of non-essential events and the reduction of event size, facilitating a more manageable audit trail. Rules are checked in order, and checking stops at the first match. The amount of data that is included in an event is determined by the value of the `level` field:
9+
10+
* `None`: The event is dropped.
11+
* `Metadata`: Audit metadata is included, request and response bodies are removed.
12+
* `Request`: Audit metadata and the request body are included, the response body is removed.
13+
* `RequestResponse`: All data is included: metadata, request body and response body. The response body can be very large. For example, `oc get pods -A` generates a response body containing the YAML description of every pod in the cluster.
14+
15+
The `ClusterLogForwarder` custom resource (CR) uses the same format as the standard link:https://kubernetes.io/docs/tasks/debug/debug-cluster/audit/#audit-policy[Kubernetes audit policy], while providing the following additional functions:
16+
17+
Wildcards:: Names of users, groups, namespaces, and resources can have a leading or trailing `\*` asterisk character. For example, the namespace `openshift-\*` matches `openshift-apiserver` or `openshift-authentication`. Resource `\*/status` matches `Pod/status` or `Deployment/status`.
18+
19+
Default Rules:: Events that do not match any rule in the policy are filtered as follows:
20+
* Read-only system events such as `get`, `list`, and `watch` are dropped.
21+
* Service account write events that occur within the same namespace as the service account are dropped.
22+
* All other events are forwarded, subject to any configured rate limits.
23+
24+
To disable these defaults, either end your rules list with a rule that has only a `level` field or add an empty rule.
25+
26+
Omit Response Codes:: A list of integer status codes to omit. You can drop events based on the HTTP status code in the response by using the `OmitResponseCodes` field, which lists HTTP status codes for which no events are created. The default value is `[404, 409, 422, 429]`. If the value is an empty list, `[]`, then no status codes are omitted.
27+
28+
The `ClusterLogForwarder` CR audit policy acts in addition to the {product-title} audit policy. The `ClusterLogForwarder` CR audit filter changes what the log collector forwards and provides the ability to filter by verb, user, group, namespace, or resource. You can create multiple filters to send different summaries of the same audit stream to different places. For example, you can send a detailed stream to the local cluster log store and a less detailed stream to a remote site.
29+
30+
[NOTE]
31+
====
32+
You must have a cluster role `collect-audit-logs` to collect the audit logs. The following example provided is intended to illustrate the range of rules possible in an audit policy and is not a recommended configuration.
33+
====
34+
35+
.Example audit policy
36+
[source,yaml]
37+
----
38+
apiVersion: observability.openshift.io/v1
39+
kind: ClusterLogForwarder
40+
metadata:
41+
name: <log_forwarder_name>
42+
namespace: <log_forwarder_namespace>
43+
spec:
44+
serviceAccount:
45+
name: <service_account_name>
46+
pipelines:
47+
- name: my-pipeline
48+
inputRefs: audit # <1>
49+
filterRefs: my-policy # <2>
50+
filters:
51+
- name: my-policy
52+
type: kubeAPIAudit
53+
kubeAPIAudit:
54+
# Don't generate audit events for all requests in RequestReceived stage.
55+
omitStages:
56+
- "RequestReceived"
57+
58+
rules:
59+
# Log pod changes at RequestResponse level
60+
- level: RequestResponse
61+
resources:
62+
- group: ""
63+
resources: ["pods"]
64+
65+
# Log "pods/log", "pods/status" at Metadata level
66+
- level: Metadata
67+
resources:
68+
- group: ""
69+
resources: ["pods/log", "pods/status"]
70+
71+
# Don't log requests to a configmap called "controller-leader"
72+
- level: None
73+
resources:
74+
- group: ""
75+
resources: ["configmaps"]
76+
resourceNames: ["controller-leader"]
77+
78+
# Don't log watch requests by the "system:kube-proxy" on endpoints or services
79+
- level: None
80+
users: ["system:kube-proxy"]
81+
verbs: ["watch"]
82+
resources:
83+
- group: "" # core API group
84+
resources: ["endpoints", "services"]
85+
86+
# Don't log authenticated requests to certain non-resource URL paths.
87+
- level: None
88+
userGroups: ["system:authenticated"]
89+
nonResourceURLs:
90+
- "/api*" # Wildcard matching.
91+
- "/version"
92+
93+
# Log the request body of configmap changes in kube-system.
94+
- level: Request
95+
resources:
96+
- group: "" # core API group
97+
resources: ["configmaps"]
98+
# This rule only applies to resources in the "kube-system" namespace.
99+
# The empty string "" can be used to select non-namespaced resources.
100+
namespaces: ["kube-system"]
101+
102+
# Log configmap and secret changes in all other namespaces at the Metadata level.
103+
- level: Metadata
104+
resources:
105+
- group: "" # core API group
106+
resources: ["secrets", "configmaps"]
107+
108+
# Log all other resources in core and extensions at the Request level.
109+
- level: Request
110+
resources:
111+
- group: "" # core API group
112+
- group: "extensions" # Version of group should NOT be included.
113+
114+
# A catch-all rule to log all other requests at the Metadata level.
115+
- level: Metadata
116+
----
117+
<1> The log types that are collected. The value for this field can be `audit` for audit logs, `application` for application logs, `infrastructure` for infrastructure logs, or a named input that has been defined for your application.
118+
<2> The name of your audit policy.

‎modules/log6x-code-ex.adoc

Lines changed: 241 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,241 @@
1+
:_mod-docs-content-type: REFERENCE
2+
[id="log6x-code-ex_{context}"]
3+
= Logging 6.0 Code Examples
4+
5+
Code examples used in the wider Logging 6.0 documentation are hosted here for single sourcing.
6+
7+
////
8+
This file is not intended to be included in whole. Includes from this file should be using tagged regions only.
9+
References:
10+
* https://github.com/openshift/openshift-docs/blob/main/contributing_to_docs/doc_guidelines.adoc#including-by-tags
11+
* https://docs.asciidoctor.org/asciidoc/latest/directives/include-tagged-regions/
12+
////
13+
14+
// Content template within commented out block.
15+
////
16+
// tag::tagname[]
17+
[source,yaml]
18+
----
19+
Content
20+
More Content
21+
----
22+
// end::tagname[]
23+
////
24+
25+
26+
// tag::filters-unchanged[]
27+
[source,yaml]
28+
----
29+
apiVersion: observability.openshift.io/v1
30+
kind: ClusterLogForwarder
31+
metadata:
32+
name: my-forwarder
33+
spec:
34+
serviceAccount:
35+
name: my-account
36+
filters:
37+
- name: my-multiline
38+
type: detectMultilineException
39+
- name: my-parse
40+
type: parse
41+
- name: my-labels
42+
type: openshiftLabels
43+
openshiftLabels:
44+
foo: bar
45+
pipelines:
46+
- name: my-pipeline
47+
inputRefs:
48+
- application
49+
outputRefs:
50+
- my-output
51+
filterRefs:
52+
- my-multiline
53+
- my-parse
54+
- my-labels
55+
outputs:
56+
- name: my-output
57+
type: http
58+
http:
59+
url: http://my-log-output:80
60+
----
61+
// end::filters-unchanged[]
62+
63+
// tag::filters-changed[]
64+
[source,yaml]
65+
----
66+
apiVersion: observability.openshift.io/v1
67+
kind: ClusterLogForwarder
68+
metadata:
69+
name: my-forwarder
70+
spec:
71+
serviceAccount:
72+
name: my-account
73+
filters:
74+
- name: drop-filter
75+
type: drop
76+
drop:
77+
- test:
78+
- field: '.level'
79+
matches: 'debug'
80+
- name: prune-filter
81+
type: prune
82+
prune:
83+
in:
84+
- '.kubernetes.labels.foobar'
85+
notIn:
86+
- '.message'
87+
- '.log_type'
88+
- name: audit-filter
89+
type: kubeAPIAudit
90+
kubeAPIAudit:
91+
omitResponseCodes:
92+
- 404
93+
- 409
94+
pipelines:
95+
- name: my-pipeline
96+
inputRefs:
97+
- application
98+
- audit
99+
outputRefs:
100+
- my-output
101+
filterRefs:
102+
- drop-filter
103+
- prune-filter
104+
- audit-filter
105+
outputs:
106+
- name: my-output
107+
type: http
108+
http:
109+
url: http://my-log-output:80
110+
----
111+
// end::filters-changed[]
112+
113+
// tag::inputs-app-audit-infra[]
114+
[source,yaml]
115+
----
116+
apiVersion: observability.openshift.io/v1
117+
kind: ClusterLogForwarder
118+
metadata:
119+
name: my-forwarder
120+
spec:
121+
serviceAccount:
122+
name: my-account
123+
inputs:
124+
- name: app-logs
125+
type: application
126+
application:
127+
includes:
128+
- namespace: my-ns1
129+
container: my-app1
130+
excludes:
131+
- namespace: my-ns2
132+
container: my-app2
133+
- name: audit-logs
134+
type: audit
135+
audit:
136+
....
137+
- name: infra-logs
138+
type: infrastructure
139+
infrastructure:
140+
....
141+
filters:
142+
- name: my-parse
143+
type: parse
144+
- name: my-app-label
145+
type: openshiftLabels
146+
openshiftLabels:
147+
my-log-index: app
148+
- name: my-infra-label
149+
type: openshiftLabels
150+
openshiftLabels:
151+
my-log-index: infra
152+
outputs:
153+
......
154+
pipelines:
155+
- name: my-app
156+
inputRefs:
157+
- application
158+
filterRefs:
159+
- my-parse
160+
- my-app-label
161+
outputRefs:
162+
- es-output-by-label
163+
- name: my-infra
164+
inputRefs:
165+
- infrastructure
166+
filterRefs:
167+
- my-parse
168+
- my-infra-label
169+
outputRefs:
170+
- es-output-by-label
171+
----
172+
// end::inputs-app-audit-infra[]
173+
174+
// tag::output-cw-token[]
175+
[source,yaml]
176+
----
177+
apiVersion: observability.openshift.io/v1
178+
kind: ClusterLogForwarder
179+
metadata:
180+
name: my-forwarder
181+
spec:
182+
serviceAccount:
183+
name: my-account
184+
outputs:
185+
- name: my-cw
186+
type: cloudwatch
187+
cloudwatch:
188+
groupName: test-cluster_{.log_type||"unknown"}
189+
region: us-east-1
190+
authentication:
191+
type: iamRole
192+
iamRole:
193+
roleARN:
194+
secretName: role-for-sts
195+
key: credentials
196+
token:
197+
from: serviceAccount
198+
pipelines:
199+
- name: my-cw-logs
200+
inputRefs:
201+
- application
202+
- infrastructure
203+
outputRefs:
204+
- my-cw
205+
----
206+
// end::output-cw-token[]
207+
208+
// tag::output-cw-static[]
209+
[source,yaml]
210+
----
211+
apiVersion: observability.openshift.io/v1
212+
kind: ClusterLogForwarder
213+
metadata:
214+
name: my-forwarder
215+
spec:
216+
serviceAccount:
217+
name: my-account
218+
outputs:
219+
- name: my-cw
220+
type: cloudwatch
221+
cloudwatch:
222+
groupName: test-cluster_{.log_type||"unknown"}
223+
region: us-east-1
224+
authentication:
225+
type: awsAccessKey
226+
awsAccessKey:
227+
keyId:
228+
secretName: cw-secret
229+
key: aws_access_key_id
230+
keySecret:
231+
secretName: cw-secret
232+
key: aws_secret_access_key
233+
pipelines:
234+
- name: my-cw-logs
235+
inputRefs:
236+
- application
237+
- infrastructure
238+
outputRefs:
239+
- my-cw
240+
----
241+
// end::output-cw-static[]

0 commit comments

Comments
 (0)