[Fleet] Fix package policy validation

[nchaulet]

Summary

Part of #122679

Improve package policy validation to use variable definition from the package instead of the one provided by the user.
Improve the UI to no crash in case an invalidate package policy is here to allow user to fix it.

In a following PR I will add the package policy validation to the create and update package policy API where we do not validate the user provided info against the package.

How to test

1. Create an invalid package policy (missing required variables)

curl --request POST \
  --url http://localhost:5601/api/fleet/package_policies \
  --header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ==' \
  --header 'Content-Type: application/json' \
  --header 'kbn-xsrf: as' \
  --data '{
  "enabled": true,
  "inputs": [
    {
      "enabled": true,
      "streams": [
        {
          "data_stream": {
            "dataset": "auditd.log",
            "type": "logs"
          },
          "enabled": true,
          "vars": {
          }
        }
      ],
      "type": "logfile"
    }
  ],
  "name": "test-124",
  "namespace": "default",
  "output_id": "fleet-default-output",
  "package": {
    "name": "auditd",
    "title": "Auditd",
    "version": "1.3.0"
  },
  "policy_id": "a14e96b6-ad4c-5e47-aa64-6d3d8c8c6ef5"
}'

2. Visit the UI you should see validation errors and be able to fix them

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[nchaulet]

@elasticmachine merge upstream

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 72f5bfb
• Storybooks Preview

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
fleet	644.1KB	644.3KB	+177.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
fleet	112.7KB	112.8KB	+76.0B

History

• 💚 Build #17796 succeeded 8a4c8f3

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @nchaulet

[juliaElastic]

I think the API should not accept an invalid policy, are you saying that will be fixed in the next pr?

[nchaulet]

I think the API should not accept an invalid policy, are you saying that will be fixed in the next pr?

Yes I want to add the API validation in a separate PR so it's easily revert-able if this cause new bugs.

[juliaElastic]

LGTM

[kibanamachine]

💚 All backports created successfully

Status	Branch	Result
✅	8.0
✅	7.17

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

[dikshachauhan-qasource]

Hi @nchaulet

We have attempted to validate above changes using API shared above on 8.0 selfmanged env.

On hitting the API, we observed below errors shared in screenshot:

Further, when we attempted to add Auditd integration with missing values to default policy, observed only two errors for path and Tags.

However, we didn't observed any error on enabling preserve original event as shared in above screenshot in ticket summary. Could you please let us know if we are missing anything.

We also observed same behavior on cloud while adding Auditd integration with missing values.

Thanks
QAS