Publish the JSONSchema for CRDs so that the editor can validate the CRDs

[chrisduong]

Summary

I can see that Argo-Workflow publish the JSONSchema at https://raw.githubusercontent.com/argoproj/argo-workflows/master/api/jsonschema/schema.json. It would be great if Argo-CD can do the same thing

Motivation

The editor can use the JSONSchema to validate the CRDs and also other useful jobs in the editor (Hover support, Autocompletion,...)

Proposal

Publish the JSONSchema, the most convenient way is publishing to https://www.schemastore.org/json/ so the popular editor can detect it automatically without any hustles for extra configurations.

[johnhamelink]

I'd love to see official support for JSONSchema - it would provide support to anyone using yaml-language-server under the hood (I use it with Eglot, lots of people use it with VSCode).

I did try this repo out, but it isn't to spec:

• The URI validation on repoURL does not handle git URIs like git@github.com:example/app.git
• syncPolicy is considered required inside spec, but it isn't needed in the minimal example in the docs
• labels is considered required inside metadata, but it isn't needed in the minimal example in the docs
• releaseName is considered required inside helm, but it isn't required by Argo from what I can tell.

So I've made a few minor modifications to make my Argo Application resources work correctly.

Ideally, instead of removing the URI validation of the repoURI, a replacement validation pattern is used which can cover all the formats that valid repo URIs comprise of.

[feldoh]

I'd also love to see this, the autocomplete alone for complex yaml worked on by non-experts is exceptionally useful

[jessebot]

In case anyone else is running into this, I recently discovered the crd-extractor.sh from datreeio/CRDs-catalog. It does exactly what it says in the name: extracts json schemas from CRDs :) They seem to maintain lots of schemas for different k8s project CRDs, and so since argoproj.io was listed there, I submitted some updates that should be up to date as of 3 days ago (so it should work with v2.8.0). Until my PR gets merged, feel free to grab them from here.

For any maintainers that may notice this thread: Would you be interested in a PR to run the CRD extractor as part of your release process? It requires you to install the CRDs, but if you're already doing a step in your ci/cd workflow where you install argo-cd to kind/k3s/k3d/etc, then you could easily have a step that runs the crd_extractor.sh script to generate some official json schemas :)

[crenshaw-dev]

Yes! We do run e2e tests in k3s, so I think it could work quite well.

[jessebot]

Ok, awesome :) I see the ci-build.yml is the one with the k3s install:

argo-cd/.github/workflows/ci-build.yaml

Lines 390 to 463 in 6e7e472

	- name: Install K3S
	env:
	INSTALL_K3S_VERSION: ${{ matrix.k3s-version }}+k3s1
	run: |
	set -x
	curl -sfL https://get.k3s.io | sh -
	sudo chmod -R a+rw /etc/rancher/k3s
	sudo mkdir -p $HOME/.kube && sudo chown -R runner $HOME/.kube
	sudo k3s kubectl config view --raw > $HOME/.kube/config
	sudo chown runner $HOME/.kube/config
	sudo chmod go-r $HOME/.kube/config
	kubectl version
	- name: Restore go build cache
	uses: actions/cache@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8 # v3.3.1
	with:
	path: ~/.cache/go-build
	key: ${{ runner.os }}-go-build-v1-${{ github.run_id }}
	- name: Add ~/go/bin to PATH
	run: |
	echo "/home/runner/go/bin" >> $GITHUB_PATH
	- name: Add /usr/local/bin to PATH
	run: |
	echo "/usr/local/bin" >> $GITHUB_PATH
	- name: Add ./dist to PATH
	run: |
	echo "$(pwd)/dist" >> $GITHUB_PATH
	- name: Download Go dependencies
	run: |
	go mod download
	go install github.com/mattn/goreman@latest
	- name: Install all tools required for building & testing
	run: |
	make install-test-tools-local
	- name: Setup git username and email
	run: |
	git config --global user.name "John Doe"
	git config --global user.email "john.doe@example.com"
	- name: Pull Docker image required for tests
	run: |
	docker pull ghcr.io/dexidp/dex:v2.37.0
	docker pull argoproj/argo-cd-ci-builder:v1.0.0
	docker pull redis:7.0.11-alpine
	- name: Create target directory for binaries in the build-process
	run: |
	mkdir -p dist
	chown runner dist
	- name: Run E2E server and wait for it being available
	timeout-minutes: 30
	run: |
	set -x
	# Something is weird in GH runners -- there's a phantom listener for
	# port 8080 which is not visible in netstat -tulpen, but still there
	# with a HTTP listener. We have API server listening on port 8088
	# instead.
	make start-e2e-local 2>&1 | sed -r "s/[[:cntrl:]]\[[0-9]{1,3}m//g" > /tmp/e2e-server.log &
	count=1
	until curl -f http://127.0.0.1:8088/healthz; do
	sleep 10;
	if test $count -ge 180; then
	echo "Timeout"
	exit 1
	fi
	count=$((count+1))
	done
	- name: Run E2E testsuite
	run: |
	set -x
	make test-e2e-local
	- name: Upload e2e-server logs
	uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # v3.1.2
	with:
	name: e2e-server-k8s${{ matrix.k3s-version }}.log
	path: /tmp/e2e-server.log
	if: ${{ failure() }}

Should I add a step after the end to end tests, or should this go into the Makefile that's referenced there?

To get this out the door, I could just add a couple of steps after Upload e2e-server logs:

      - name: Run CRD JSON Schema Extractor
        run: |
          sudo apt install zip
          curl -sLO https://github.com/datreeio/CRDs-catalog/releases/latest/download/crd-extractor.zip
          unzip crd-extractor.zip
          sh crd-extractor.sh && ls ~/.datree/crdSchemas/argoproj.io
      - name: Upload argproj.io CRD JSON Schemas
        uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # v3.1.2
        with:
          name: json_schemas.zip
          path: |
           ~/.datree/crdSchemas/argoproj.io/appproject_v1alpha1.json
           ~/.datree/crdSchemas/argoproj.io/application_v1alpha1.json
           ~/.datree/crdSchemas/argoproj.io/applicationset_v1alpha1.json

Main issue here is that this is currently part of a matrix strategy job, so this means these two steps would run four times (once for each k3s version you're testing) :/ Also, I'm not sure if the ~ will work, but I also don't know which user this runs as? If it's root, I guess we'd change all the ~ to /root` (and remove the sudo). I can submit this as a proper PR, but wanted to make sure we're on the same page.

Oh, second issue, this is uploaded, but only as an artifact and is part of the ci, but not release, so we'd still need to permalink to that in each release somehow so others were aware of it 🤔

[crenshaw-dev]

All fair. I feel like the nicest approach would be to abstract the start-k3s step and run it in the release workflow to generate/upload the schema. But that's maybe a bit more invasive than you're trying to get into right now?

[ibauersachs]

You can filter the steps in a matrix job with conditions to not run multiple times, see e.g. the dnsjava build.

Actions run as a user, with $GITHUB_WORKSPACE as the default working directory for a step (unless it's changed with cd). So .datree without a prefix should work.

[jessebot]

All fair. I feel like the nicest approach would be to abstract the start-k3s step and run it in the release workflow to generate/upload the schema. But that's maybe a bit more invasive than you're trying to get into right now?

Yeah, it makes way more sense to put into the release directly 😁 I can grab the step that's already in place in the ci-build for the k3s-install, or do you wanna just use a pre-built one for the release step like jupyterhub/action-k3s-helm. Jupyterhub tends to be pretty good about patching vulnerabilities in their code bases.

Either way, the steps should be something like:

    steps:
     - name: Checkout code
        uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3
        with:
          fetch-depth: 0
          token: ${{ secrets.GITHUB_TOKEN }}
      - name: Start a local k8s cluster
        uses: jupyterhub/action-k3s-helm@v3
        with:
          k3s-channel: latest      
      - name: Apply Kubernetes Manifests
        run: |
          kubectl apply -f manifests/crds/appproject-crd.yaml 
          kubectl apply -f manifests/crds/applicationset-crd.yaml 
          kubectl apply -f manifests/crds/appproject-crd.yaml
      - name: Run CRD JSON Schema Extractor
        run: |
          sudo apt install zip
          curl -sLO https://github.com/datreeio/CRDs-catalog/releases/latest/download/crd-extractor.zip
          unzip crd-extractor.zip
          sh crd-extractor.sh
         
      # this step should be replaced with a releasing action/step, however, not sure which
      - name: Upload argproj.io CRD JSON Schemas
        uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # v3.1.2
        with:
          name: e2e-server-k8s${{ matrix.k3s-version }}.log
          path: |
           .datree/crdSchemas/argoproj.io/appproject_v1alpha1.json
           .datree/crdSchemas/argoproj.io/application_v1alpha1.json
           .datree/crdSchemas/argoproj.io/applicationset_v1alpha1.json

open questions:

• Is there a preferred action for releasing?
• Should it be its own release or bundled with the main release of the repo?
• Should this be in its own job after the rest of the jobs in release.yml?

Actions run as a user, with $GITHUB_WORKSPACE as the default working directory for a step (unless it's changed with cd). So .datree without a prefix should work.

thank you! :)

You can filter the steps in a matrix job with conditions to not run multiple times, see e.g. the dnsjava build.

My only objection in specific instance with the matrix of versions is that we'd have to specify a specific k8s version for it run on (in the if: conditional we used), which would then need to be updated each time that version was no longer supported, and really that's only a gripe about maintenance.

[crenshaw-dev]

I can grab the step that's already in place

I'd start there, I'm in favor of small/incremental if possible. Can always switch to the dedicated action as a follow up.

Is there a preferred action for releasing?

I think the new logic would be embedded here and would produce the schema similarly to how other parts produce SBOMs etc.

Should it be its own release or bundled with the main release of the repo?

I'd do main release.

Should this be in its own job after the rest of the jobs in release.yml?

I'm not too good with GitHub Actions, so not sure of the exact structure. But basically I'd piggy-back on how other asset generation (e.g. SBOM) is organized.

[jessebot]

Ok, cool, thanks for answering all my questions! I'll likely get working on that in the next day or so after I clear a few other things from my backlog, but this gives me a good starting point :)

[patelvp]

I am trying to get a bump on this.

[IanMoroney]

Just encountered this issue when wondering why vscode was showing red for me on specifying sources instead of source, and it thought source was a mandatory field.
Went to override the schema for the yaml validator, and found the schema wasn't public 🤦
Can the argocd schema please be published somewhere, preferably to schemastore so we don't need to do any end user configuration? (as per the OP's post)

[crenshaw-dev]

We now have tooling to generate schema files for use with Kustomize, but I'm not sure whether they're valid for IDEs. https://github.com/argoproj/argo-schema-generator/tree/main/schema

[shakefu]

Please publish these schemas it makes managing ArgoCD objects in k8s so much nicer!