Skip to content

Commit a8134ec

Browse files
kcbimontekcbimonte
committed
docx: working through operation
Signed-off-by: Kevin Bimonte <kbimonte@gmail.com>
1 parent 57ff054 commit a8134ec

File tree

6 files changed

+464
-5
lines changed

6 files changed

+464
-5
lines changed

‎docs/docs/operation/encryption.md‎

Lines changed: 80 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,89 @@
22
title: Encryption
33
---
44

5+
Automating everything means authorizing something to automate many things. This makes CI systems a high-risk target for
6+
security leaks.
7+
8+
Concourse pipelines are loaded with credentials: resources are configured with private keys, tasks are given credentials
9+
to servers they integrate via [credential manager variables](creds/index.md), [`task` step `vars`](../steps/task.md),
10+
or [`task` step `params`](../steps/task.md), etc. If someone gets their hands on your config, they have access to
11+
everything.
12+
13+
To mitigate this, Concourse supports encrypting sensitive information before it reaches the database. This way the
14+
plaintext credentials only exist in memory for as long as they need to, and if someone gains access to your database,
15+
they can't so easily gain the keys to the kingdom.
16+
17+
We strongly encourage anyone running Concourse to configure encryption. Going further, it's best to have Concourse not
18+
store the credentials in the first place, in which case you may want to
19+
configure [credential management](creds/index.md) as well.
20+
521
## What's encrypted?
622

23+
The following values are expected to contain credentials, and so will be encrypted:
24+
25+
* Resource [`resource.sources`](../resources/index.md#resource-schema), as they often contain private keys and other
26+
credentials for writing to (or simply granting access to) the resource.
27+
* Resource type [`resource_type.sources`](../resource-types/index.md#resource_type-schema), for the same reason as
28+
above, though this is probably a less common use case.
29+
* Pipeline [`task` step `vars`](../steps/task.md) and [`task` step `params`](../steps/task.md), in case they contain
30+
sensitive information such as usernames and/or passwords.
31+
* Put step [`put` step `params`](../steps/put.md) and get step [`get` step `params`](../steps/get.md) are also
32+
encrypted, even though they rarely should contain credentials (they're usually in [
33+
`resource.source`](../resources/index.md#resource-schema)).
34+
* Team auth configurations, as they often contain things like GitHub or other oAuth client secrets.
35+
36+
!!! note
37+
38+
The actual implementation encrypts things in a more heavy-handed way than the above list implies. For example,
39+
pipeline configs are actually encrypted as one large blob.
40+
41+
Notably, the following things are NOT encrypted:
42+
43+
* Build logs. If your jobs are outputting credentials, encryption won't help you. We have chosen not to tackle this
44+
initially as it would introduce a performance burden for what is not as much of an obvious win.
45+
* Resource versions. These should never contain credentials, and are often meaningless on their own.
46+
* Resource metadata. These are visible to anyone if your pipeline
47+
is [exposed](https://concourse-ci.org/managing-pipelines.html#fly-expose-pipeline), and should never contain
48+
credentials.
49+
* Pipeline names, job names, etc. - anything else that is not a high-risk target for credential leakage, as opposed to
50+
regular information leaks.
51+
<br/><br/>
52+
Resources and jobs in particular exist in their own tables, with their names in plaintext, and only their config
53+
encrypted. In this way, names are not protected, even though the pipeline config itself is also stored as one big
54+
encrypted blob.
55+
756
## Enabling Encryption
857

58+
To enable encryption, you'll just need to come up with a 16 or 32-byte random character sequence and configure it as
59+
`--encryption-key` flag to the `web` command. For BOSH, this is the [
60+
`encryption_key`](https://bosh.io/jobs/web?source=github.com/concourse/concourse-bosh-release#p=encryption_key)
61+
property.
62+
63+
On startup, the [`web` node](../install/running-web.md) will encrypt all existing plaintext data, and any new data being
64+
written will be encrypted before it's sent over the network to the database.
65+
66+
The initial bulk encryption shouldn't take too long, but it will scale linearly with the amount of data that you have,
67+
and if another ATC is running it'll suddenly not be able to read the data until it's also given the key. So, expect some
68+
downtime.
69+
970
## Rotating the Encryption Key
1071

11-
## Disabling Encryption
72+
To swap out the encryption key, you'll need to pass the previous key as `--old-encryption-key` (or [
73+
`old_encryption_key`](https://bosh.io/jobs/web?source=github.com/concourse/concourse-bosh-release#p=old_encryption_key)),
74+
and the new key as `--encryption-key` (or [
75+
`encryption_key`](https://bosh.io/jobs/web?source=github.com/concourse/concourse-bosh-release#p=encryption_key)).
76+
77+
On startup, the [`web` node](../install/running-web.md) will decrypt all existing data and re-encrypt it with the new
78+
key, in one go. If it encounters a row which is already encrypted with the new key, it will continue on (as may be the
79+
case when restarting with the flags again, or if the ATC died in the middle of rotating).
80+
81+
If the ATC encounters a row which cannot be decrypted with neither the old key nor the new one, it will log loudly and
82+
fail to start, telling you which row it choked on. This data must be dealt with in some way, either by re-configuring
83+
the key the row was encrypted with as the old key, or manually performing database surgery to remove the offending row.
84+
Hopefully this doesn't happen to you!
85+
86+
## Disabling Encryption
87+
88+
To opt out of encryption entirely (I'm sure you have your reasons), simply pass `--old-encryption-key` (or [
89+
`old_encryption_key`](https://bosh.io/jobs/web?source=github.com/concourse/concourse-bosh-release#p=old_encryption_key))
90+
alone. With no new encryption key, the [web node](../install/running-web.md) will decrypt all existing data on start.

‎docs/docs/operation/index.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,10 @@
11
---
22
title: Operation
33
---
4+
5+
The following sections describes operator-focused features and tools that Concourse provides, such as monitoring and
6+
credential management.
7+
8+
These concepts are not required to operate Concourse, but are for users that are looking to extend the capabilities of
9+
managing a Concourse deployment. For users that are new to these concepts, we do recommend learning how to set
10+
up [Credential Management](creds/index.md) and [Encryption](encryption.md).

‎docs/docs/operation/metrics.md‎

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,28 @@
22
title: Metrics
33
---
44

5+
Metrics are essential in understanding how any large system is behaving and performing. Concourse can emit metrics about
6+
both the system health itself and about the builds that it is running. Operators can tap into these metrics in order to
7+
observe the health of the system.
8+
59
## Configuring Metrics
610

7-
## What's emitted?
11+
The [`web` node](../install/running-web.md) can be configured to emit metrics on start.
12+
13+
Currently supported metrics emitters are InfluxDB, NewRelic, Prometheus, and Datadog. There is also a dummy emitter that
14+
will just spit the metrics out in to the logs at `DEBUG` level, which can be enabled with the `--emit-to-logs` flag.
15+
16+
Regardless of your metrics emitter, you can set `CONCOURSE_METRICS_BUFFER_SIZE` to determine how many metrics emissions
17+
are sent at a time. Increasing this number can be helpful if sending metrics is regularly failing (due to rate limiting
18+
or network failures) or if latency is particularly high.
19+
20+
There are various flags for different emitters; run `concourse web --help` and look for "Metric Emitter" to see what's
21+
available.
22+
23+
## What's emitted?
24+
25+
This reference section lists of all the metrics that Concourse emits via the Prometheus emitter.
26+
27+
To make this document easy to maintain, Prometheus is used as the "source of truth" - primarily because it has help text
28+
built-in, making this list easy to generate. Treat this list as a reference when looking for the equivalent metric names
29+
for your emitter of choice.

‎docs/docs/operation/opa-integration.md‎

Lines changed: 218 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,229 @@
22
title: Open Policy Agent Integration
33
---
44

5+
!!! note
6+
7+
The [Open Policy Agent](https://www.openpolicyagent.org/docs/latest/) (OPA, pronounced “oh-pa”) is an open source,
8+
general-purpose policy engine that unifies policy enforcement across the stack.
9+
10+
OPA allows you to create arbitrary rules within Concourse without having to add a new feature to Concourse. You could
11+
even recreate Concourse's [RBAC system](../auth-and-teams/user-roles.md) using OPA.
12+
13+
More likely use-cases are to enforce rules your organization may have, such as not using certain container images or
14+
disallowing the use of privileged workloads. With OPA you can be as general or fine-grained as you want, enforcing these
15+
rules at the team or pipeline level.
16+
17+
The next few sections explain how to configure Concourse to talk to an OPA server and how to write OPA rules for
18+
Concourse.
19+
520
## Configuring Concourse
621

22+
There are four configuration options you need to set on the `concourse web` nodes to have them interact with OPA.
23+
24+
`CONCOURSE_OPA_URL`: The OPA policy check endpoint.
25+
26+
: Should point to a specific `package/rule` that contains all Concourse rules for your cluster.
27+
28+
: _Example_: `http://opa-endpoint.com/v1/data/concourse/decision`
29+
30+
`CONCOURSE_POLICY_CHECK_FILTER_HTTP_METHOD`: API http methods to go through policy check.
31+
32+
: You will need to make sure these match up with an API action in the next two configuration options.
33+
34+
: _Example_: `PUT,POST`
35+
36+
`CONCOURSE_POLICY_CHECK_FILTER_ACTION`: Actions in this list will go through policy check.
37+
38+
: _Example_: `ListWorkers,ListContainers`
39+
40+
`CONCOURSE_POLICY_CHECK_FILTER_ACTION_SKIP`: Actions in this list will not go through policy check
41+
42+
: _Example_: `PausePipeline,UnpausePipeline`
43+
44+
For the last three configuration options you can refer
45+
to [this list of routes](https://github.com/concourse/concourse/blob/master/atc/routes.go) for a list of API actions and
46+
their respective HTTP method. There are also some [Special Actions](#special-actions) not directly in the API.
47+
748
## Writing OPA Rules
849

50+
On the OPA server you'll need to create a package and policy for Concourse. This should match up with the endpoint
51+
provided to Concourse. The [OPA documentation](https://www.openpolicyagent.org/docs/latest/) has a good guide explaining
52+
how to generally write OPA rules and set up an OPA server.
53+
54+
For any actions that Concourse has been configured to filter it will send a JSON request to the OPA server with the
55+
following details. Top-level data directly under the `input` key will be present for most actions. The information under
56+
the `data` key will differ based on the action being checked.
57+
58+
This sample JSON payload is what OPA is sent when a user sets a pipeline. The `data` key contains the pipeline in JSON
59+
format.
60+
61+
```json
62+
{
63+
"input": {
64+
"service": "concourse",
65+
"cluster_name": "dev",
66+
"cluster_version": "7.4.0",
67+
"http_method": "PUT",
68+
"action": "SaveConfig",
69+
"user": "test",
70+
"team": "main",
71+
"pipeline": "check-pipeline",
72+
"data": {
73+
"jobs": [
74+
{
75+
"name": "test",
76+
"plan": [
77+
{
78+
"get": "tiny"
79+
},
80+
{
81+
"config": {
82+
"image_resource": {
83+
"source": {
84+
"repository": "busybox"
85+
},
86+
"type": "registry-image"
87+
},
88+
"platform": "linux",
89+
"run": {
90+
"args": [
91+
"-exc",
92+
"echo hello"
93+
],
94+
"path": "sh"
95+
}
96+
},
97+
"task": "a-task"
98+
}
99+
]
100+
}
101+
]
102+
}
103+
}
104+
}
105+
```
106+
107+
An OPA rule can respond to Concourse with three fields:
108+
109+
<div class="annotate" markdown>
110+
111+
* `allowed` (_required_): Boolean type. Setting to `False` will deny the action unless the `block` field is `False`.
112+
* `block` (_optional_): Boolean type. If set to `False` and `allowed` is `True` this creates a soft-policy enforcement.
113+
The action will be allowed and the `reasons` will still be printed to the web UI like a warning message. (1)
114+
* `reasons` (_optional_): List of string type. If an action is denied based on the `allowed` field then the reason(s)
115+
will be displayed in the UI.
116+
117+
</div>
118+
119+
1. Not setting `block` is the same as setting `"block": true`.
120+
121+
Here is an example OPA policy. By default, it will allow whatever action it has been sent. It will deny the action if
122+
one or more of the three deny rules are true.
123+
124+
```rego title="concourse.rego" linenums="1"
125+
package concourse
126+
127+
default decision = {"allowed": true}
128+
129+
decision = {"allowed": false, "reasons": reasons} {
130+
count(deny) > 0
131+
reasons := deny
132+
}
133+
134+
deny["cannot use docker-image types"] {
135+
input.action == "UseImage"
136+
input.data.image_type == "docker-image"
137+
}
138+
139+
deny["cannot run privileged tasks"] {
140+
input.action == "SaveConfig"
141+
input.data.jobs[_].plan[_].privileged
142+
}
143+
144+
deny["cannot use privileged resource types"] {
145+
input.action == "SaveConfig"
146+
input.data.resource_types[_].privileged
147+
}
148+
```
149+
9150
## Special Actions
10151

152+
Most of the actions you can filter for come directly from the list
153+
of [API actions](../auth-and-teams/user-roles.md#action-matrix). There are currently two special actions you can also
154+
filter on.
155+
11156
### `UseImage`
12157

13-
### `SetPipeline`
158+
Before Concourse starts a container you can check what image it is going to use to create the container. Depending on
159+
the `image_type` the `image_source` field may contain other fields. The JSON payload for this action will look similar
160+
to the following example:
161+
162+
```json
163+
{
164+
"input": {
165+
"service": "concourse",
166+
"cluster_name": "dev",
167+
"cluster_version": "7.4.0",
168+
"action": "UseImage",
169+
"team": "main",
170+
"pipeline": "simple",
171+
"data": {
172+
"image_type": "registry-image",
173+
"privileged": true,
174+
"image_source": {
175+
"repository": "alpine",
176+
"tag": "latest"
177+
}
178+
}
179+
}
180+
}
181+
```
182+
183+
### `SetPipeline`
184+
185+
This action occurs whenever a [`set_pipeline` step](../steps/set-pipeline.md) is run. The JSON payload for this action
186+
will contain the pipeline config in JSON format under the `data` key:
187+
188+
```json
189+
{
190+
"input": {
191+
"service": "concourse",
192+
"cluster_name": "dev",
193+
"cluster_version": "7.4.0",
194+
"action": "SetPipeline",
195+
"team": "main",
196+
"pipeline": "simple",
197+
"data": {
198+
"jobs": [
199+
{
200+
"name": "test",
201+
"plan": [
202+
{
203+
"get": "tiny"
204+
},
205+
{
206+
"config": {
207+
"image_resource": {
208+
"source": {
209+
"repository": "busybox"
210+
},
211+
"type": "registry-image"
212+
},
213+
"platform": "linux",
214+
"run": {
215+
"args": [
216+
"-exc",
217+
"echo hello"
218+
],
219+
"path": "sh"
220+
}
221+
},
222+
"task": "a-task"
223+
}
224+
]
225+
}
226+
]
227+
}
228+
}
229+
}
230+
```

0 commit comments

Comments
 (0)