Skip to content

Listener scope #310

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 29 commits into from
Nov 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
e9d4896
grpc listener integration
nightkr Aug 24, 2023
4626bf0
Read listeners from PodListeners instead of calling listop's CSI
nightkr Aug 24, 2023
6289d71
Drop listop grpc
nightkr Aug 24, 2023
0b16680
Infer node scope for nodeport listeners
nightkr Aug 25, 2023
c94eefc
Add listener scope to k8sSearch
nightkr Aug 28, 2023
f3aeb8e
Use resolved listener name for k8sSearch
nightkr Aug 28, 2023
29507ab
Allow searching for multiple listener scopes
nightkr Aug 28, 2023
c985cfc
Use listener address tags rather than assuming DNS
nightkr Aug 29, 2023
48d043b
Fix warnings
nightkr Aug 30, 2023
bf1ef39
Rename PodListenerInfo to SchedulingPodInfo
nightkr Aug 30, 2023
a05239e
Start Snafuizing listener integration
nightkr Aug 30, 2023
db46cb7
More cleanup
nightkr Aug 31, 2023
d31981b
Stop injecting node secret scope for node-scoped listener scopes
nightkr Aug 31, 2023
0982682
Further cleanup
nightkr Sep 1, 2023
b600443
Drop stale dependencies
nightkr Sep 1, 2023
6d11c7f
Run pre-commit hooks
nightkr Sep 7, 2023
167279c
Update op-rs
nightkr Sep 7, 2023
d712fea
Fix stale reference to podlistenerses
nightkr Sep 7, 2023
bf6445b
Add docs for listener scope
nightkr Sep 8, 2023
9ff7db7
Add test, fail louder if no listener address was found
nightkr Sep 8, 2023
3abbb69
Rename listener scope to listener-volume
nightkr Sep 13, 2023
d71df8c
Update rust/operator-binary/src/backend/pod_info.rs
nightkr Oct 19, 2023
ebcda1a
Drop now-unused listop CSI socket stuff
nightkr Oct 26, 2023
2f3dc5f
Merge branch 'feature/listener' of github.com:stackabletech/secret-op…
nightkr Oct 26, 2023
9ad9cfc
Merge branch 'main' into feature/listener
nightkr Oct 26, 2023
2d7948d
Update docs/modules/secret-operator/pages/scope.adoc
nightkr Oct 26, 2023
e7f11e0
Unbacktick Kube kind names
nightkr Oct 26, 2023
a1a794f
Merge branch 'feature/listener' of github.com:stackabletech/secret-op…
nightkr Oct 26, 2023
664768b
Re-render README
nightkr Oct 30, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

- Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
- Please make sure all these things are done and tick the boxes

```[tasklist]
# Author
- [ ] Changes are OpenShift compatible
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@ crate-hashes.json
result
image.tar

tilt_options.json
tilt_options.json
2 changes: 1 addition & 1 deletion .readme/partials/borrowed/documentation.md.j2
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@ If you are interested in the most recent state of this repository, check out the

The documentation for all Stackable products can be found at [docs.stackable.tech](https://docs.stackable.tech).

If you have a question about the Stackable Data Platform contact us via our [homepage](https://stackable.tech/) or ask a public questions in our [Discussions forum](https://github.com/orgs/stackabletech/discussions).
If you have a question about the Stackable Data Platform contact us via our [homepage](https://stackable.tech/) or ask a public questions in our [Discussions forum](https://github.com/orgs/stackabletech/discussions).
2 changes: 1 addition & 1 deletion .readme/partials/borrowed/header.md.j2
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@
<img width="150" src="./.readme/static/borrowed/Icon_Stackable.svg" alt="Stackable Logo"/>
</p>

<h1 align="center">{{title}}</h1>
<h1 align="center">{{title}}</h1>
2 changes: 1 addition & 1 deletion .readme/partials/borrowed/links.md.j2
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-green.svg)](https://docs.stackable.tech/home/stable/contributor/index.html)
[![License OSL3.0](https://img.shields.io/badge/license-OSL3.0-green)](./LICENSE)

[Documentation](https://docs.stackable.tech/{{operator_docs_slug}}/stable/index.html) {% if quickstart_link %}| [Quickstart]({{quickstart_link}}) {% endif %}| [Stackable Data Platform](https://stackable.tech/) | [Platform Docs](https://docs.stackable.tech/) | [Discussions](https://github.com/orgs/stackabletech/discussions) | [Discord](https://discord.gg/7kZ3BNnCAF)
[Documentation](https://docs.stackable.tech/{{operator_docs_slug}}/stable/index.html) {% if quickstart_link %}| [Quickstart]({{quickstart_link}}) {% endif %}| [Stackable Data Platform](https://stackable.tech/) | [Platform Docs](https://docs.stackable.tech/) | [Discussions](https://github.com/orgs/stackabletech/discussions) | [Discord](https://discord.gg/7kZ3BNnCAF)
2 changes: 1 addition & 1 deletion .readme/partials/borrowed/overview_blurb.md.j2
Original file line number Diff line number Diff line change
@@ -1 +1 @@
It is part of the Stackable Data Platform, a curated selection of the best open source data apps like Apache Kafka, Apache Druid, Trino or Apache Spark, [all](#our-operators) working together seamlessly. Based on Kubernetes, it runs everywhere – [on prem or in the cloud](#supported-platforms).
It is part of the Stackable Data Platform, a curated selection of the best open source data apps like Apache Kafka, Apache Druid, Trino or Apache Spark, [all](#our-operators) working together seamlessly. Based on Kubernetes, it runs everywhere – [on prem or in the cloud](#supported-platforms).
2 changes: 1 addition & 1 deletion .readme/partials/borrowed/related_reading.md.j2
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@
{% for (text, link) in related_reading_links %}
* [{{text}}]({{link}})
{%- endfor %}
{%- endif -%}
{%- endif -%}
2 changes: 1 addition & 1 deletion .readme/partials/main.md.j2
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,4 @@ The Secret Operator is deployed as a DaemonSet and provides a CSI to mount files
<img width="500" src="./.readme/static/secret-operator.drawio.svg" alt="A diagram overview of the workings of the operator"/>

The Secret Operator by [Stackable](https://stackable.tech/).
This is a Kubernetes Operator to provision and inject secrets for Kubernetes Pods. It is part of the [Stackable Data Platform](https://stackable.tech/), a curated selection of the best open source data apps like Kafka, Druid, Trino or Spark, all working together seamlessly. Based on Kubernetes, it runs everywhere – on prem or in the cloud.
This is a Kubernetes Operator to provision and inject secrets for Kubernetes Pods. It is part of the [Stackable Data Platform](https://stackable.tech/), a curated selection of the best open source data apps like Kafka, Druid, Trino or Spark, all working together seamlessly. Based on Kubernetes, it runs everywhere – on prem or in the cloud.
2 changes: 1 addition & 1 deletion .readme/static/secret-operator.drawio.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,9 @@ All notable changes to this project will be documented in this file.
### Added

- Added support for encrypting PKCS#12 keystores ([#314]).
- Added listener scope for provisioned secrets ([#310]).

[#310]: https://github.com/stackabletech/secret-operator/pull/310
[#314]: https://github.com/stackabletech/secret-operator/pull/314

## [23.7.0] - 2023-07-14
Expand Down Expand Up @@ -121,11 +123,13 @@ The easiest way to fix this is to perform a rolling reboot of all nodes after th
This is a one-time migration.

### Changed

- Store secrets on tmpfs ([#37]).
- Locked down secret permissions by default ([#37]).
- Operator-rs: 0.8.0 -> 0.10.0 ([#49]).

### Bugfixes

- Fixed thread starvation and slow shutdowns ([#47]).

[#37]: https://github.com/stackabletech/secret-operator/pull/37
Expand All @@ -135,4 +139,5 @@ This is a one-time migration.
## [0.1.0] - 2022-02-03

### Added

- Initial release
3 changes: 2 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -52,5 +52,6 @@ yasna = "0.5"
# Workaround for https://github.com/hyperium/tonic/issues/243
h2 = { git = "https://github.com/stackabletech/h2.git", branch = "feature/grpc-uds" }

# [patch."https://github.com/stackabletech/operator-rs.git"]
[patch."https://github.com/stackabletech/operator-rs.git"]
# stackable-operator = { path = "../operator-rs" }
# stackable-operator = { git = "https://github.com/stackabletech//operator-rs.git", branch = "main" }
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,19 @@
<h1 align="center">Stackable Secret Operator</h1>



[![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://GitHub.com/stackabletech/secret-operator/graphs/commit-activity)
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-green.svg)](https://docs.stackable.tech/home/stable/contributor/index.html)
[![License OSL3.0](https://img.shields.io/badge/license-OSL3.0-green)](./LICENSE)

[Documentation](https://docs.stackable.tech/secret-operator/stable/index.html) | [Stackable Data Platform](https://stackable.tech/) | [Platform Docs](https://docs.stackable.tech/) | [Discussions](https://github.com/orgs/stackabletech/discussions) | [Discord](https://discord.gg/7kZ3BNnCAF)


This is a Kubernetes Operator to provision and inject secrets for Kubernetes pods. Compared to Kubernetes' native secret mounts, this permits secrets to be selected dynamically based on properties of the `Pod`.

It is part of the Stackable Data Platform, a curated selection of the best open source data apps like Apache Kafka, Apache Druid, Trino or Apache Spark, [all](#our-operators) working together seamlessly. Based on Kubernetes, it runs everywhere – [on prem or in the cloud](#supported-platforms).


## Installation

You can install the operator using [stackablectl or helm](https://docs.stackable.tech/secret-operator/stable/installation.html).
Expand All @@ -34,6 +37,7 @@ The documentation for all Stackable products can be found at [docs.stackable.tec

If you have a question about the Stackable Data Platform contact us via our [homepage](https://stackable.tech/) or ask a public questions in our [Discussions forum](https://github.com/orgs/stackabletech/discussions).


## What does it do?

The Secret Operator provisions and mounts secrets, but unlike the default Kubernetes Secret mechanism, Secrets can be created and mounted dynamically based on Pod properties. The Operator supports:
Expand All @@ -49,6 +53,7 @@ The Secret Operator is deployed as a DaemonSet and provides a CSI to mount files
The Secret Operator by [Stackable](https://stackable.tech/).
This is a Kubernetes Operator to provision and inject secrets for Kubernetes Pods. It is part of the [Stackable Data Platform](https://stackable.tech/), a curated selection of the best open source data apps like Kafka, Druid, Trino or Spark, all working together seamlessly. Based on Kubernetes, it runs everywhere – on prem or in the cloud.


## About The Stackable Data Platform

This operator is written and maintained by [Stackable](https://stackable.tech) and it is part of a larger data platform.
Expand Down
7 changes: 7 additions & 0 deletions deploy/helm/secret-operator/templates/roles.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,13 @@ rules:
- secretclasses
verbs:
- get
- apiGroups:
- listeners.stackable.tech
resources:
- listenerclasses
- podlisteners
verbs:
- get
- apiGroups:
- security.openshift.io
resourceNames:
Expand Down
1 change: 0 additions & 1 deletion docs/modules/secret-operator/pages/commandline_args.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,3 @@

The path to the https://github.com/container-storage-interface/spec/blob/master/spec.md[Container Storage Interface] Unix Domain Socket
that the operator should listen on.

7 changes: 3 additions & 4 deletions docs/modules/secret-operator/pages/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,11 @@

This is an operator for Kubernetes that provisions and injects secrets into Kubernetes Pods.

Kubernetes `Secret` objects contain sensitive payloads such as passwords, tokens or keys. These objects
Kubernetes Secret objects contain sensitive payloads such as passwords, tokens or keys. These objects
are usually self-contained and static in the sense that their contents remain unchanged as long as their owners
do not update them.

The Stackable Secret Operator enhances the functionality of Kubernetes Secrets by introducing the concept of a
`SecretClass`. A `SecretClass` represents a reference to a source of sensitive data. In addition to Kubernetes
`Secret` objects, the operator can provision Pods with TLS Certificates, Kerberos keytabs or authentication
SecretClass. A SecretClass represents a reference to a source of sensitive data. In addition to Kubernetes
Secret objects, the operator can provision Pods with TLS Certificates, Kerberos keytabs or authentication
tokens from external services.

19 changes: 15 additions & 4 deletions docs/modules/secret-operator/pages/scope.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -13,19 +13,30 @@ include this extra context. The exact effect of the scope depends on which xref:
[#node]
=== `node`

The `node` scope is resolved to the name of the Kubernetes `Node` object that the `Pod` is running on. This will typically
The `node` scope is resolved to the name of the Kubernetes Node object that the Pod is running on. This will typically
be the DNS name of the node.

[#pod]
=== `pod`

The `pod` scope is resolved to the name of the Kubernetes `Pod`. This allows the secret to differentiate between `StatefulSet` replicas.
The `pod` scope is resolved to the name of the Kubernetes Pod. This allows the secret to differentiate between StatefulSet replicas.

[#service]
=== `service`

The `service` scope allows `Pod` objects to specify custom scopes. This should typically correspond to `Service` objects that the
`Pod` participate in.
The `service` scope allows Pod objects to specify custom scopes. This should typically correspond to Service objects that the
Pod participate in.

[#listener-volume]
=== `listener-volume`

The `listener-volume` scope allows Pod objects to request secrets corresponding to a xref:listener-operator:volume.adoc[listener volume] that is bound to the same Pod.

The `listener-volume` scope takes the name of the listener volume as a paremeter.

note:: The parameter is the name of the Pod's _volume_, not the name of the PersistentVolumeClaim, PersistentVolume, or xref:listener-operator:listener.adoc[Listener].

note:: The `listener-volume` scope also implies the xref:#node[] scope for xref:listener-operator:listenerclass.adoc#servicetype-nodeport[NodePort] listeners.

== Example

Expand Down
20 changes: 10 additions & 10 deletions docs/modules/secret-operator/pages/troubleshooting.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,17 @@

== My secret-consuming Pods get stuck `Pending`!

. Does the `Pod` have any events relating to scheduling? (`kubectl describe pod/$POD_NAME`)
. Is the `PersistentVolumeClaim` being created? It should have the name `$POD_NAME-$VOLUME_NAME`.
. Is the `PersistentVolumeClaim` bound to a `PersistentVolume`? If not:
.. Does the `PersistentVolumeClaim` have any relevant events? (`kubectl describe pvc/$PVC_NAME`)
.. If the `PersistentVolumeClaim` has no relevant events (or only an event that it is waiting for a volume to be provisioned),
. Does the Pod have any events relating to scheduling? (`kubectl describe pod/$POD_NAME`)
. Is the PersistentVolumeClaim being created? It should have the name `$POD_NAME-$VOLUME_NAME`.
. Is the PersistentVolumeClaim bound to a PersistentVolume? If not:
.. Does the PersistentVolumeClaim have any relevant events? (`kubectl describe pvc/$PVC_NAME`)
.. If the PersistentVolumeClaim has no relevant events (or only an event that it is waiting for a volume to be provisioned),
check the logs of the secret-operator sidecar container named `external-provisioner`.
.. Does the `PersistentVolumeClaim` have a `StorageClass` set?
.. Is the `StorageClass` configured to use the provisioner named `secrets.stackable.tech`?
. Does the `PersistentVolume` have any relevant events? (`kubectl describe pv/$PV_NAME`)
. Is the `CSIDriver` object named `secrets.stackable.tech` configured correctly?
. Is the CSI driver registered on the relevant `CSINode` objects? (`kubectl get csinode/$NODE_NAME -o yaml` should contain the driver `secrets.stackable.tech`)
.. Does the PersistentVolumeClaim have a StorageClass set?
.. Is the StorageClass configured to use the provisioner named `secrets.stackable.tech`?
. Does the PersistentVolume have any relevant events? (`kubectl describe pv/$PV_NAME`)
. Is the CSIDriver object named `secrets.stackable.tech` configured correctly?
. Is the CSI driver registered on the relevant CSINode objects? (`kubectl get csinode/$NODE_NAME -o yaml` should contain the driver `secrets.stackable.tech`)
. Does the secret-operator sidecar container named `node-driver-registrar` have any relevant log entries?
. Does the kubelet have any relevant log entries?
. When running on OpenShift also have a look at the xref:openshift.adoc[OpenShift documentation].
8 changes: 4 additions & 4 deletions docs/modules/secret-operator/pages/usage.adoc
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
= Usage

The operator injects secret data into xref:volume.adoc[] mounts that declare a CSI volume with `driver: secrets.stackable.tech`.
A minimal secret-consuming `Pod` looks like this:

A minimal secret-consuming Pod looks like this:

[source,yaml]
----
include::example$usage-pod.yaml[]
----

xref:secretclass.adoc[] defines where the secrets come from. For example, the following `SecretClass`
issues TLS certificates, storing its CA certificate in the Kubernetes `Secret` object named `secret-provisioner-tls-ca`:
xref:secretclass.adoc[] defines where the secrets come from. For example, the following SecretClass
issues TLS certificates, storing its CA certificate in the Kubernetes Secret object named `secret-provisioner-tls-ca`:

[source,yaml]
----
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/secret-operator/pages/volume.adoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
= Volume

The primary entry point for applications is by mounting a secret into a `Pod` object's `volume` set. This is done by using Kubernetes'
The primary entry point for applications is by mounting a secret into a Pod object's `volume` set. This is done by using Kubernetes'
https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.23/#ephemeralvolumesource-v1-core[`EphemeralVolumeSource`] type.
For example:

Expand Down
1 change: 0 additions & 1 deletion docs/modules/secret-operator/partials/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,3 @@
** xref:secret-operator:volume.adoc[]
* xref:secret-operator:security.adoc[]
* xref:secret-operator:troubleshooting.adoc[]

5 changes: 3 additions & 2 deletions rust/operator-binary/src/backend/dynamic.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ use std::{collections::HashSet, fmt::Display};

use super::{
kerberos_keytab::{self, KerberosProfile},
pod_info::PodInfo,
pod_info::{PodInfo, SchedulingPodInfo},
tls, SecretBackend, SecretBackendError, SecretVolumeSelector,
};
use crate::crd::{self, SecretClass};
Expand Down Expand Up @@ -51,9 +51,10 @@ impl<B: SecretBackend + Send + Sync> SecretBackend for DynamicAdapter<B> {
async fn get_qualified_node_names(
&self,
selector: &SecretVolumeSelector,
pod_info: SchedulingPodInfo,
) -> Result<Option<HashSet<String>>, Self::Error> {
self.0
.get_qualified_node_names(selector)
.get_qualified_node_names(selector, pod_info)
.await
.map_err(|err| DynError(Box::new(err)))
}
Expand Down
Loading