ci: cleanup ci

[bmwill]

This PR includes the following commits:

ci: various cleanups

This patch does some various cleanups to our rust ci pipeline. In
particular it runs linting (clippy, rustfmt) as well as tests using the
compiler toolchain specified inside of the rust-toolchain file instead
of overriding it with either stable or nightly either of which could
cause CI to break when a new toolchain is released to the stable or
nightly channels.

It also moves canarying of a new compiler toolchain from the workflow
that runs on every PR to a 'nightly' Github Action workflow. It also
prefers using the 'beta' channel instead of 'nightly' channel in order
to reduce noise.

clippy: correctly disallow tokio unbounded channel

ci: enable 'librocksdb-sys' to be able to be properly cached

Today we leverage the 'Swatinem/rust-cache' github action in order to
cache rust third-party dependencies and avoid needing to rebuild them
every time CI is run. Unfortunately it was identified that despite a
cache hit the 'librocksdb-sys' crate was always retriggering a very
costly build.

The crux of the issue is mostly highlighted by this issue
(rust-rocksdb/rust-rocksdb#574) on the
rust-rocksdb repository. The 'librocksdb-sys' crate uses a build script
to be able to build the c++ rocksdb project (tracked via a submodule in
the rust-rocksdb repo) and makes use of of the cargo directive
"cargo:rerun-if-changed=rocksdb/" to ensure that anytime the submodule
is updated that the c++ code should be recompiled. In particular this
cargo directive only uses the filesystem last-modified "mtime" timestamp
to determine if the file has changed
(https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorerun-if-changedpath).

The 'Swatinem/rust-cache' action explicitly avoids caching the
'.cargo/registry/src' directory since its faster for cargo to repopulate
the unpacked crate src from their archives in '.cargo/registry/cache'
(which is cached by the github action). This leads to the unintended
consiquency that when cargo goes to unpack the 'librocksdb-sys' crate
src, the mtime of the "rocksdb/" directory no longer matches the cached
mtime that was restored from a previous build of the 'librocksdb-sys'
dependency resulting in 'librocksdb-sys' being rebuilt.

In order to fix this issue we can additionally cache the
'.cargo/registry/src//librocksdb-sys-' directory (since the github
caching infrastructure preserves mtimes of cached files) and ensure that
the mtime matches what cargo expects, avoiding a rebuild. In order to do
this I've forked the 'Swatinem/rust-cache' action to 'bmwill/rust-cache'
add added the ability to additionally specify other paths that should be
cached and specified the '.cargo/registry/src//lib-rocksdb-sys-'
directory.

Results

Due to the proper caching of the 'librocksdb-sys' crate CI times have improved:

job	cache-miss	cache-hit pre-patch	cache-hit post-patch	speed up
cargo-test	29m	18m	12m 15s	1.5x
cargo-clippy	13m	8m 14s	1m 17s	6.6x

[huitseeker]

Early feedback to developers on whether they added a needless dependency is extremely valuable. We have many PRs where we include such a needless crate and end up not using it. When it's 24h later and merged on main is not a good time to herd cats and assemble the devs of the prior day to clean up added dependencies they didn't really need.

[huitseeker]

Actually, if we can run that on stable, I'm a happy camper!

[huitseeker]

As far as I'm concerned I still want the code to break early if it doesn't test properly on stable.

[huitseeker]

leaving a note here that we'll need to coordinate on merging, as the branch protection settings need an update

[huitseeker]

Gimme the awesome speedup!

[huitseeker]

Actually, if we can run that on stable, I'm a happy camper!

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (main@fa90884). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1164   +/-   ##
=======================================
  Coverage        ?   79.37%           
=======================================
  Files           ?       94           
  Lines           ?    20357           
  Branches        ?        0           
=======================================
  Hits            ?    16158           
  Misses          ?     4199           
  Partials        ?        0           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fa90884...4735504. Read the comment docs.