Extend the definition of 'analyzer' in Semgrep rules

[mjambon]

This PR extends the Semgrep rule schema in accordance with the work I did on the OCaml side. The PR is https://github.com/semgrep/semgrep-proprietary/pull/1641 and I'm going to push my work to it very soon.

This PR also adds JSON Schema validation tests. They're useful to check that the JSON Schema is valid, that it validates valid Semgrep rules, and rejects invalid Semgrep rules - without having to rely on another git repo to test this.

test plan: make test

This will run check-jsonschema. make setup will ask the user to install it if it's missing.

• I ran make setup && make to update the generated code after editing a .atd file (TODO: have a CI check)
• I made sure we're still backward compatible with old versions of the CLI.
  For example, the Semgrep backend need to still be able to consume data generated
  by Semgrep 1.17.0.
  See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades

[github-actions]

Backwards compatibility summary:

Checking backward compatibility of semgrep_output_v1.atd against past version v1.29.0
Skipping v1.30.0 because commit 78720c795cd5a186f5102c87125ef876c6435a0c has already been checked
Skipping v1.31.0 because commit 78720c795cd5a186f5102c87125ef876c6435a0c has already been checked
Skipping v1.31.1 because commit 78720c795cd5a186f5102c87125ef876c6435a0c has already been checked
Skipping v1.31.2 because commit 78720c795cd5a186f5102c87125ef876c6435a0c has already been checked
Skipping v1.32.0 because commit 78720c795cd5a186f5102c87125ef876c6435a0c has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.33.0
Skipping v1.33.1 because commit 8849e56ddb0977e38a120a6cfbd1c396eb6fa15e has already been checked
Skipping v1.33.2 because commit 8849e56ddb0977e38a120a6cfbd1c396eb6fa15e has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.34.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.37.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.38.0
Skipping v1.38.1 because commit fd294683e7369cabf63738febeaba8a22c925187 has already been checked
Skipping v1.38.2 because commit fd294683e7369cabf63738febeaba8a22c925187 has already been checked
Skipping v1.38.3 because commit fd294683e7369cabf63738febeaba8a22c925187 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.39.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.40.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.41.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.42.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.43.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.44.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.45.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.46.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.47.0
Skipping v1.48.0 because commit 278ed753e0c66b8bfc3f2d805fde53be022dd4b6 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.49.0
Skipping v1.50.0 because commit 857682f41eb09e0b330a247ff1adf3bfeaf9d9ca has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.52.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.53.0
Skipping v1.54.0 because commit 3b72d494260258497e796d094b1a4916501a6df1 has already been checked
Skipping v1.54.1 because commit 3b72d494260258497e796d094b1a4916501a6df1 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.54.2
Skipping v1.54.3 because commit 9f1c50383a9a9969e2fe7a5f9bff9ca0a7c837bb has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.55.0
Skipping v1.55.1 because commit 6dffeaa692153fd33b4f154fddaefde1f2f1ae27 has already been checked
Skipping v1.55.2 because commit 6dffeaa692153fd33b4f154fddaefde1f2f1ae27 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.56.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.57.0
Skipping v1.58.0 because commit 4cc11b00d411c02fc611aa8c78a336520438fb48 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.59.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.59.1
Checking backward compatibility of semgrep_output_v1.atd against past version v1.60.0
Skipping v1.60.1 because commit eed58a091fd7d19e402a6d4cf2d287e137215d03 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.61.0
Skipping v1.61.1 because commit bbfd1c5b91bd411bceffc3de73f5f0b37f04433d has already been checked
Skipping v1.62.0 because commit bbfd1c5b91bd411bceffc3de73f5f0b37f04433d has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.63.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.64.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.65.0
Skipping v1.66.0 because commit 3e7bbafa2b7e722d893303a7fb90a83dab6737a7 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.66.1
Skipping v1.66.2 because commit 215a54782174de84f97188632b4a37e35ba0f827 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.67.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.68.0
Skipping v1.69.0 because commit d5b91fa4f6a03240db31e9bbbc5376a99bc8eeea has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.70.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.71.0
Skipping v1.72.0 because commit 75abf193687b84ab341d8267d865ad68d81a89c9 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.73.0
Skipping v1.74.0 because commit 9f38254957c50c68ea402eebae0f7aa40dd01cbf has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.75.0
Skipping v1.76.0 because commit 9102031608aa4154e1c37f557550ec4eabc8780c has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.77.0
Skipping v1.78.0 because commit dcb5d77b420ddee61f58aadd3c2c7aef38778154 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.79.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.80.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.81.0
Skipping v1.82.0 because commit 9e0f3bec26b07b4fb6753a32cb75277f45f2572c has already been checked
Skipping v1.83.0 because commit 9e0f3bec26b07b4fb6753a32cb75277f45f2572c has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.84.0
Skipping v1.84.1 because commit 3daef49297ada205359cc1d2996354c94b628b0d has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.85.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.86.0

[aryx]

Can you also update rule_schema_v2.atd ?

[mjambon]

Can you also update rule_schema_v2.atd ?

I updated it. It's fairly complicated due an ATD adapter that can't be fully generic. It's explained in the comments along with a possible improvement in ATD that would simplify this. I created an ATD issue here: ahrefs/atd#413 (I don't think we really need this but it's good to start a discussion somewhere)

I still have to check if this OCaml code compiles and works (I don't have the impression that we're testing it here).