ci: clarinet cli revamp

[CharlieC3]

Related: #1078

• Reduces CI build time for clarinet-cli from ~75 minutes to ~10 minutes
• Splits audit and formatting job into separate workflow to reduce duplicate configs later when we have more component-specific workflows
• Improves Rust caching to speed up builds with no depencency changes since last run
• Dists will now be built depending on what's triggering the CI run. x86_64-unknown-linux-gnu dist will always get created, others will only run on pushes to main and develop branches.
  • This is configurable in the ci_clarinet_cli_dist_matrix.json file
  • This cuts down on CI time for PRs considerably
• Unit tests in the dist job will be skipped for the x86_64-unknown-linux-gnu target, since this target should already be covered in the code coverage job.
  • This cuts down on CI time for PRs considerably
• Docker image now reuses the pre-built dist instead of rebuilding from scratch
• Sets up initial scaffolding to build a ARM64 Docker image on pushes to the main and develop branches
  • This is configurable in the ci_clarinet_cli_docker_matrix.json file
  • I have this disabled at the moment because the arm64 binary isn't able to execute in an arm64 docker image for some reason, and it will require a larger time investment to resolve. This can probably be put into a separate task, and may require something like this to work
• Resolves build warnings with rustup

[codecov]

Codecov Report

Merging #1104 (6b93005) into develop (a2a21d8) will decrease coverage by 16.25%.
Report is 1116 commits behind head on develop.
The diff coverage is 81.11%.

❗ Current head 6b93005 differs from pull request most recent head ad479cc. Consider uploading reports for the commit ad479cc to get more accurate results

@@             Coverage Diff              @@
##           develop    #1104       +/-   ##
============================================
- Coverage    34.02%   17.78%   -16.25%     
============================================
  Files          180      145       -35     
  Lines        47486    42650     -4836     
============================================
- Hits         16158     7584     -8574     
- Misses       31328    35066     +3738     

Flag	Coverage Δ
unittests	17.78% <81.11%> (-16.25%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
components/clarinet-cli/src/bin.rs	0.00% <ø> (-25.00%)	⬇️
components/clarinet-cli/src/chainhooks/mod.rs	0.00% <ø> (ø)
components/clarinet-cli/src/deployments/mod.rs	0.00% <ø> (ø)
components/clarinet-cli/src/deployments/types.rs	0.00% <ø> (ø)
components/clarinet-cli/src/deployments/ui/app.rs	0.00% <ø> (ø)
components/clarinet-cli/src/deployments/ui/mod.rs	0.00% <ø> (ø)
components/clarinet-cli/src/deployments/ui/ui.rs	0.00% <ø> (ø)
components/clarinet-cli/src/frontend/cli.rs	0.00% <ø> (ø)
components/clarinet-cli/src/frontend/dap.rs	0.00% <ø> (ø)
components/clarinet-cli/src/frontend/telemetry.rs	0.00% <ø> (ø)
... and 94 more

... and 174 files with indirect coverage changes

[hugocaillard]

This is awesome, thanks @CharlieC3 🙌
If I understand well, this PR doesn't impact our release workflow right?

[CharlieC3]

@hugocaillard Yup, that's the idea! I'm grabbing the latest version of the clarinet-cli component tag here, then comparing it against the version found in components/clarinet-cli/Cargo.toml here. If there's no match, a new tag will be created in the format of clarinet-cli-v<Cargo.toml version>.

So just like before, to create a new clarinet-cli version, you have to bump this line and commit/merge it to main.

However, we likely won't get good results if we merge this CI workflow in while we keep the original CI workflow active. It's up to you on how you'd like to handle this, but I would suggest disabling the new workflow until you have workflows for all components set up and ready before removing the old CI workflow.

[CharlieC3]

@hugocaillard Not sure if you saw my comment above. Any thoughts. How would you like to proceed?

[hugocaillard]

I need to check what was the old CI doing that the new doesn't.
And I don't think there's a big difference so maybe we could switch the new one pretty easily

[CharlieC3]

@hugocaillard There are some things the old CI workflow is building that the new is not, like devnet JS. So I would not suggest removing the old CI until more CI component workflows are created to cover the gap. Maybe the new workflow should be disabled until the others are created?

[hugocaillard]

@CharlieC3 Ah right.
So you suggest merging this PR but disabling the CI?

[CharlieC3]

Since more workflows are needed for a complete picture, yes.

[hugocaillard]

@CharlieC3
I manually disabled the "Audit and Format" and "CI Clarinet CLI" actions 👌

Will the changes on clarinet dockerfile have impacts on the current CI?

[CharlieC3]

@hugocaillard Ah good catch, I've updated the old workflow to use the dev Dockerfile, since it's pretty much the same as the old Dockerfile.

[smcclellan]

@hugocaillard @CharlieC3 What's next on this PR?

[CharlieC3]

I think just an approval! Let me know if there's something I'm missing @hugocaillard

[hugocaillard]

LGTM
This new workflow has been disabled for now.
We'll keep using the old one until the workflows have been adapted for other components (mainly stacks-devnet).
The release of clarinet 2 later this year will be a good time to activate it