Add support for manifest V2 validations

[mgarabed]

What does this PR do?

Adds support for validating Manifest v2 files.

For v2, we have the actual schema validation taking place on the Datadog app side via a new endpoint that's been created. This runs through a JSONSchema serializer, then applies basic validations against the payload. Since we will also be doing this validation within the app, it seemed like a better choice to avoid the schema duplication across different repos and projects with the risk of things getting out of sync.

Motivation

Add support for new manifest versions.

Additional Notes

A valid API and APP key will need to be added to the ddev config when validating the new manifests.

Tests continue to pass for v1, but additional tests are needed for v2.

To test manually, create a folder named testmanifest and add the following file:

manifest.json

{
    "app_id": "datadog-oracle",
    "assets": {
        "dashboards": {
            "oracle": "https://app.datadoghq.com/screen/integration/240/oracle-database---overview"
        },
        "integration": {
            "configuration": {
                "spec": "assets/configuration/spec.yaml"
            },
            "events": {
                "creates_events": true
            },
            "id": "testmanifest",
            "metrics": {
                "auto_install": true,
                "check": "oracle.session_count",
                "metadata_path": "metrics_metadata.csv",
                "prefix": "oracle."
            },
            "service_checks": {
                "metadata_path": "assets/service_checks.json"
            },
            "source_type_name": "Oracle Database"
        },
        "ui extension": {}
    },
    "author": {
        "homepage": "https://www.datadoghq.com",
        "name": "Datadog",
        "sales_email": "help@datadoghq.com",
        "support_email": "help@datadoghq.com"
    },
    "display_on_public_website": true,
    "legal_terms": {},
    "manifest_version": "2.0.0",
    "oauth": {},
    "pricing": [
        {
            "billing_type": "free"
        }
    ],
    "tile": {
        "changelog": "CHANGELOG.md",
        "classifier_tags": [
            "Category::Marketplace",
            "Category::Cloud",
            "Supported OS::Windows",
            "Supported OS::Mac OS",
            "Offering::Integration",
            "Offering::UI Extension"
        ],
        "configuration": "README.md#Setup",
        "description": "Oracle relational database system designed for enterprise grid computing",
        "media": [],
        "overview": "README.md#Overview",
        "title": "Oracle"
    }
}

To test the server side validation, try changing the classifier tag Offering::UI Extension to Offering::UI-Extension or some nonsense name.

Review checklist (to be filled by reviewers)

• Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
• PR title must be written as a CHANGELOG entry (see why)
• Files changes must correspond to the primary purpose of the PR as described in the title (small unrelated changes should have their own PR)
• PR must have changelog/ and integration/ labels attached

[codecov]

Codecov Report

Merging #9968 (0f6021d) into master (7ae23bb) will decrease coverage by 0.05%.
The diff coverage is 45.19%.

Flag	Coverage Δ
datadog_checks_dev	78.73% <45.19%> (-0.69%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

[nmuesch]

This looks great!!! I left some inline questions/suggestions but nothing major.

[nmuesch]

👍 on continuing to list all validations that would be skipped instead of quickly aborting.

[nmuesch]

I think these v2 versions need to be assets/integration/id, ``assets/integration/source_type_name, and app_id`

[mgarabed]

The content_changed function below just returns git blocks which are what are being tested - so we wouldn't be able to fully trace a path like assets/integration/id. It does mean we could end up catching other id fields in this case though ..

[nmuesch]

Thanks, this LGTM

[ofek]

Nice!

[ofek]

Let's move this under tooling/ instead

[mgarabed]

Done, and since we are under that tooling dir, added back the jsonpointer dependency to the cli extras 😝

[ofek]

Let's move this under tooling/ instead
Done, and since we are under that tooling dir, added back the jsonpointer dependency to the cli extras 😝

Oh wait, can we still not do that? It's quite unnecessary