new(nok8s): json schema validation for tracingpolicies

[FedeDP]

Description

Since in nok8s builds we can't rely upon kubernetes validation, we were lacking any sort of validation for the tracing policy yaml object.
This patch introduces a validation based upon the json schema of the crds that were previously generated and used only in the kubeconform make target.

New dependency: github.com/santhosh-tekuri/jsonschema/v6; only used in nok8s build.
Size change:

• main: 49532
• HEAD: 50752

~800K.

Most of the line changes come from:

• the new pkg/tracingpolicy/schemas/ json schemas
• the new vendored dependency

Changelog

new(nok8s): json schema + CEL validation for tracingpolicies

[FedeDP]

Run static checks / build-nok8s (pull_request)Failing after 1m

So, this is failing because in the tetragon-nok8s target i clean up the pkg/tracingpolicy/schemas/ folder, but then tetra-nok8s fails to build because go:embed can't find the file.
2 possibilities:

• we keep the install/kubernetes tetragon/schemas/ folder alive (and we re-generate it in generate target); then both tetragon-nok8s and tetra-nok8s will rely upon a pkg/tracingpolicy/schemas/ target that just cp -r install/kubernetes/tetragon/schemas/ pkg/tracingpolicy/schemas/; finally, we gitignore pkg/tracingpolicy/schemas/.
• both tetra-nok8s and tetragon-nok8s will depend upon pkg/tracingpolicy/schemas/ target that generates json schemas from crds, and both targets clean pkg/tracingpolicy/schemas/ folder up once they are done

For now, i implement 1, but am open to switch to 2 if requested.

EDIT: in the end, i chose to follow what we already do for crds: make generate will re-generate the files directly in the desired folder (pkg/tracingpolicy/schemas/) and they are then versioned. This allows go test -tags nok8s or just go build -tags nok8s to work like a charm since the required files are correctly there.

[netlify]

✅ Deploy Preview for tetragon ready!

Name	Link
🔨 Latest commit	66da2ba
🔍 Latest deploy log	https://app.netlify.com/projects/tetragon/deploys/6a677bf77e1d9b0007a5fa6d
😎 Deploy Preview	https://deploy-preview-5322--tetragon.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[FedeDP]

Example output from feeding a wrong syntax policy:

level=error msg="Failed to execute tetragon" error="failed to validate TracingPolicy: jsonschema validation failed with 'file:///tracingpolicy#'\n- at '/spec': additional properties 'lsmhoooks' not allowed"

In main, same policy:

level=info msg="Added TracingPolicy with success" TracingPolicy=/home/fdipierr/pol-lsm-wrong.yaml metadata.namespace="" metadata.name=protection-policy-cve-2024-21626

Policy file (note lsmhoooks :D )

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "protection-policy-cve-2024-21626"
spec:
  lsmhoooks:
  - hook: "bprm_check_security"
    args:
    - index: 0 # struct linux_binprm *bprm
      type: "path"
      resolve: "mm.owner.fs.pwd"
    selectors:
    - matchArgs:
      - index: 0
        operator: "Prefix"
        values:
        - "/sys/fs/cgroup"
      matchActions:
      - action: Post
    message: cve-2024-21626

[andrewstrohman]

I confirmed that this came from the vendor's repo, and it's not something that we created by accident. (I guess our CI would catch it, if it came from us).

[andrewstrohman]

It might be worth doing an explicit validation for drift between the JSON schema and the CRDs in CI.

[FedeDP]

It might be worth doing an explicit validation for drift between the JSON schema and the CRDs in CI.

This should never happen since make generate takes care of rebuilding both.

[kkourt]

Note: x-kubernetes-validations cel expressions are not supported; we would need to extend the json schema validation and manually support them; it's not super easy to do, but it would allow us to cleanup some code.

Is this something that we plan to do?

Because if we can't do it, then I think the best option would be to do just duplicate the validation logic in go since we are using CEL expressions extensively in our spec.

[FedeDP]

Is this something that we plan to do?

I think we might want to commit to do that in the future; unfortunately, it is a bit complex and would take quite a bit of time.

Because if we can't do it, then I think the best option would be to do just duplicate the validation logic in go since we are using CEL expressions extensively in our spec.

I am not sure, it quickly becomes risky to rely upon the double validation (sooner or later we would forget to add it) and also why pay the perf penalty (for validation) in k8s builds when k8s already validates for us?

[mtardy]

how much do we trust this new dep?

[FedeDP]

Seems widely used and updated. But can't really say :/

[mtardy]

could you explain a little bit in the git commit log the why of these changes? I'm not super familiar on why we were doing all this before

[FedeDP]

I'm not super familiar on why we were doing all this before

I'm not either :) i just split the schemas generation code from being part of the kubeconform target to its own.
(I just restored the SCHEMAS_CACHE thing because it's still useful).
I expanded the commit message to be more precise :)

Also, note that i tested make kubeconform and it works fine.

[kkourt]

Is this something that we plan to do?

I think we might want to commit to do that in the future; unfortunately, it is a bit complex and would take quite a bit of time.

Because if we can't do it, then I think the best option would be to do just duplicate the validation logic in go since we are using CEL expressions extensively in our spec.

I am not sure, it quickly becomes risky to rely upon the double validation (sooner or later we would forget to add it)

What's the alternative? Note that my statement is qualified by "if we can't do it" (i.e., translate the CEL validation to a json schema).

and also why pay the perf penalty (for validation) in k8s builds when k8s already validates for us?

I would, also, expect that performance overhead of this is small. Indeed, I would expect that it's negligible compared to how long it takes to load a policy.

[FedeDP]

Note that my statement is qualified by "if we can't do it" (i.e., translate the CEL validation to a json schema).

I think we should be able to do it, but it's a bit complex; i can give it a shot and keep this PR on hold until i find the right way to achieve that if you prefer.

[kkourt]

Note that my statement is qualified by "if we can't do it" (i.e., translate the CEL validation to a json schema).

I think we should be able to do it, but it's a bit complex

OK, good enough for me :)

i can give it a shot and keep this PR on hold until i find the right way to achieve that if you prefer.

IMO, no need to put the PR on hold. Let's just go over the review process as normal and merge it (assuming there are on blockers). Once merged, please create an issue to track the followup work.

[FedeDP]

@kkourt ooops i think i am really near to make CEL validation work without many changes 🚀 ; pushed the changes in their own commit to keep them separated with their own commit message.

It was easier than i thought, i just needed to dig a bit in the jsonschema library.

[FedeDP]

Moving to draft to fix CEL stuff.

[FedeDP]

Fixed CEL stuff.