Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configurable object selection #42

Open
smlx opened this issue Feb 5, 2024 · 3 comments
Open

Configurable object selection #42

smlx opened this issue Feb 5, 2024 · 3 comments
Labels
enhancement New feature or request

Comments

@smlx
Copy link

smlx commented Feb 5, 2024

Context

If a cluster has sensitive, but autogenerated or externally managed objects then I don't want those to be dumped by the object-dumper script for backup by k8up. Examples of such objects include:

  • TLS-ACME certificates
  • Service account tokens
  • Helm releases

Since sensitive objects are almost always Secrets, maybe it would make sense to be able to exclude Secret objects by Secret type? Or at least be able to add a label to exclude an object from backup.

Alternatives

I could use an alternative to the object-dumper script to dump my objects. But this project has integration for use with k8up.

Are there any existing alternative object-dumping tools for use with k8up for a "full cluster backup" that has object filtering capabilities?

I guess this is related to k8up-io/k8up#908, because if k8up gained this functionality natively then I could stop using object-dumper.

@smlx smlx added the enhancement New feature or request label Feb 5, 2024
bastjan added a commit that referenced this issue Nov 12, 2024
Differences:
- All APIs are fully qualified in both the options (`--must-exist=certificates.cert-manager.io`, `--ignore=deployment.apps`) and the output files (`objects-Certificate.cert-manager.io.json`).
  This makes it possible to distinguish between objects with the same kind but different groups. See #47.
- Resources without a list endpoint are ignored and do not cause and error or need to be explicitly ignored.
- Ignore and must-exist options are now command line flags instead of files in `/usr/local/share`.

Fixes #47 and #42.
@bastjan bastjan mentioned this issue Nov 12, 2024
3 tasks
@bastjan bastjan linked a pull request Nov 12, 2024 that will close this issue
3 tasks
bastjan added a commit that referenced this issue Nov 12, 2024
Differences:
- All APIs are fully qualified in both the options (`--must-exist=certificates.cert-manager.io`, `--ignore=deployment.apps`) and the output files (`objects-Certificate.cert-manager.io.json`).
  This makes it possible to distinguish between objects with the same kind but different groups. See #47.
- Resources without a list endpoint are ignored and do not cause and error or need to be explicitly ignored.
- Ignore and must-exist options are now command line flags instead of files in `/usr/local/share`.

Fixes #47 and #42.
@bastjan bastjan removed a link to a pull request Nov 12, 2024
3 tasks
@bastjan
Copy link
Contributor

bastjan commented Nov 12, 2024

Would a blanket exclude by resource be enough for you? Or would you need finer grained control?

bastjan added a commit that referenced this issue Nov 12, 2024
Differences:
- All APIs are fully qualified in both the options (`--must-exist=certificates.cert-manager.io`, `--ignore=deployment.apps`) and the output files (`objects-Certificate.cert-manager.io.json`).
  This makes it possible to distinguish between objects with the same kind but different groups. See #47.
- Resources without a list endpoint are ignored and do not cause and error or need to be explicitly ignored.
- Ignore and must-exist options are now command line flags instead of files in `/usr/local/share`.

Fixes #47 and #42.
bastjan added a commit that referenced this issue Nov 12, 2024
Differences:
- All APIs are fully qualified in both the options (`--must-exist=certificates.cert-manager.io`, `--ignore=deployment.apps`) and the output files (`objects-Certificate.cert-manager.io.json`).
  This makes it possible to distinguish between objects with the same kind but different groups. See #47.
- Resources without a list endpoint are ignored and do not cause and error or need to be explicitly ignored.
- Ignore and must-exist options are now command line flags instead of files in `/usr/local/share`.

Fixes #47 and #42.
@smlx
Copy link
Author

smlx commented Nov 12, 2024

I think having both options would be ideal. For example to blanket exclude all secrets you could do something like this:

excludeKinds:
- Secret

But I think it would make sense to also be able to select using standard label selectors or field selectors? That way the types that you specify could be filtered but other types would be unfiltered.

So, for example, to match the behaviour of this kubectl command:

kubectl get pods --selector=app.kubernetes.io/instance=nginx --field-selector=status.phase=Running

... you could use a configuration like this to select pods which are the intersection of the two filter specifications:

filterKinds:
- kind: Pod
  matchLabels:
    app.kubernetes.io/instance: nginx
- kind: Pod
  fieldSelector:
    status.phase: Running

Or you could have a configuration like this to select only secrets that are an intersection of the two filter specifications:

filterKinds:
- kind: Secret
  fieldSelector:
    type: kubernetes.io/tls
- kind: Secret
  matchLabels:
    controller.cert-manager.io/fao: true

bastjan added a commit that referenced this issue Nov 12, 2024
Rewrite dumper in Go

Differences:
- All APIs are fully qualified in both the options (`--must-exist=certificates.cert-manager.io`, `--ignore=deployment.apps`) and the output files (`objects-Certificate.cert-manager.io.json`).
  This makes it possible to distinguish between objects with the same kind but different groups. See #47.
- Resources without a list endpoint are ignored, do not cause an error, and don't need to be explicitly ignored.
- Ignore and must-exist options are now command line flags instead of files in `/usr/local/share`.

Fixes #47 and maybe #42.
@bastjan
Copy link
Contributor

bastjan commented Nov 15, 2024

That's how i imagined it as well, with the per resource matching of field and label selector.

We now brought the project to a testable and maintainable state, but i can't guarantee we have time currently to implement this feature. However blanked ignore is implemented.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants