sdk: Create new Cargo workspace for sdk/ (solana-labs#4685)

* sdk: Create new Cargo workspace for `sdk/`

#### Problem

We want to move everything under `sdk/` into a standalone repository,
but all of the crates are currently part of the monorepo workspace,
which makes the extraction impossible.

#### Summary of changes

Remove all crates in `sdk/` from the top-level workspace, but keep the
local dependency. Create a new `Cargo.toml` for the sdk workspace along
with all required dependencies and patches.

* Remove unused dependencies

* Update lockfile

* CI: Add sdk scripts to run through GitHub Actions

#### Problem

The monorepo contains a lot of testing steps for all of the packages.
Once the sdk lives in its own repos, it will need to run adequate CI,
just as before.

#### Summary of changes

Add all of the steps run over the sdk in the monorepo as scripts in
`sdk/scripts`, along with a setup step and workflow file.

These steps include:

* checking crate ownership (check-crates.sh)
* running hack checks (check-hack.sh)
* cargo check with nightly toolchain (check-nightly.sh)
* clippy (check-clippy.sh)
* crate sorting in Cargo.toml (check-sort.sh)
* dcou checks (check-dev-context-only-utils.sh)
* rustfmt check (check-fmt.sh)
* auditing crates (check-audit.sh)
* miri tests (test-miri.sh)
* frozen-abi tests (test-frozen-abi.sh)
* tests with stable toolchain (test-stable.sh)
* wasm tests (test-wasm.sh)
* coverage tests (test-coverage.sh)
* shellcheck (check-shell.sh)
* sanity checks (check-porcelain.sh and check-nits.sh)
* publish order (order-crates-for-publishing.py)
* benches (test-bench.sh)

A few differences with the monorepo testing:

* coverage reports are not uploaded
* bench results are not uploaded
* version check is omitted, since the sdk will version crates separately
* ssh key check is omitted, since there's no net scripts

Future work:

* downstream tests into solana-program repos
* semver checks
* spellchecking

* CI: Remove sdk specific commands and tests

.github/actions/setup/action.yml

@@ -0,0 +1,147 @@
+name: Setup environment
+
+inputs:
+  cargo-cache-key:
+    description: The key to cache cargo dependencies. Skips cargo caching if not provided.
+    required: false
+  cargo-cache-fallback-key:
+    description: The fallback key to use when caching cargo dependencies. Default to not using a fallback key.
+    required: false
+  cargo-cache-local-key:
+    description: The key to cache local cargo dependencies. Skips local cargo caching if not provided.
+    required: false
+  stable-toolchain:
+    description: Install stable toolchain if `true`. Defaults to `false`.
+    required: false
+  nightly-toolchain:
+    description: Install nightly toolchain if `true`. Defaults to `false`.
+    required: false
+  clippy:
+    description: Install Clippy if `true`. Defaults to `false`.
+    required: false
+  rustfmt:
+    description: Install Rustfmt if `true`. Defaults to `false`.
+    required: false
+  miri:
+    description: Install Miri if `true`. Defaults to `false`.
+    required: false
+  llvm-tools-preview:
+    description: Install llvm-tools-preview if `true`. Defaults to `false`.
+    required: false
+  purge:
+    description: Purge unused directories if `true`. Defaults to `false`.
+    required: false
+
+runs:
+  using: 'composite'
+  steps:
+    - name: Purge unused ubuntu runner directories
+      if: ${{ inputs.purge == 'true' }}
+      shell: bash
+      run: |
+        # If there are still disk space issues, try to add more packages from
+        # https://github.com/jlumbroso/free-disk-space
+        sudo rm -rf /usr/share/dotnet
+        sudo rm -rf /usr/share/swift
+        sudo rm -rf /usr/share/mysql
+        sudo rm -rf /usr/share/az_*
+        sudo rm -rf /usr/share/postgresql-common
+        sudo rm -rf /opt/ghc
+        sudo rm -rf /opt/az
+        sudo rm -rf /opt/pipx
+        sudo rm -rf /opt/microsoft
+        sudo rm -rf /opt/google
+        sudo rm -rf /opt/hostedtoolcache
+        sudo rm -rf /usr/local/lib/android
+        sudo rm -rf /usr/local/lib/heroku
+        sudo rm -rf /imagegeneration
+        sudo rm -rf "$AGENT_TOOLSDIRECTORY"
+        sudo docker image prune --all --force
+
+    - name: Set Environment Variables
+      shell: bash
+      run: |
+        source ./sdk/scripts/rust-version.sh
+        echo "RUST_STABLE=${rust_stable}" >> $GITHUB_ENV
+        echo "RUST_NIGHTLY=${rust_nightly}" >> $GITHUB_ENV
+
+    - name: Install stable toolchain
+      if: ${{ inputs.stable-toolchain == 'true' }}
+      uses: dtolnay/rust-toolchain@master
+      with:
+        toolchain: ${{ env.RUST_STABLE }}
+
+    - name: Install nightly toolchain
+      if: ${{ inputs.nightly-toolchain == 'true' }}
+      uses: dtolnay/rust-toolchain@master
+      with:
+        toolchain: ${{ env.RUST_NIGHTLY }}
+
+    - name: Install Rustfmt
+      if: ${{ inputs.rustfmt == 'true' }}
+      uses: dtolnay/rust-toolchain@master
+      with:
+        toolchain: ${{ env.RUST_NIGHTLY }}
+        components: rustfmt
+
+    - name: Install Clippy
+      if: ${{ inputs.clippy == 'true' }}
+      uses: dtolnay/rust-toolchain@master
+      with:
+        toolchain: ${{ env.RUST_NIGHTLY }}
+        components: clippy
+
+    - name: Install Miri
+      if: ${{ inputs.miri == 'true' }}
+      uses: dtolnay/rust-toolchain@master
+      with:
+        toolchain: ${{ env.RUST_NIGHTLY }}
+        components: miri
+
+    - name: Install llvm-tools-preview
+      if: ${{ inputs.llvm-tools-preview == 'true' }}
+      uses: dtolnay/rust-toolchain@master
+      with:
+        toolchain: ${{ env.RUST_NIGHTLY }}
+        components: llvm-tools-preview
+
+    - name: Cache Cargo Dependencies
+      if: ${{ inputs.cargo-cache-key && !inputs.cargo-cache-fallback-key }}
+      uses: actions/cache@v4
+      with:
+        path: |
+          ~/.cargo/bin/
+          ~/.cargo/registry/index/
+          ~/.cargo/registry/cache/
+          ~/.cargo/git/db/
+          target/
+        key: ${{ runner.os }}-${{ inputs.cargo-cache-key }}-${{ hashFiles('**/Cargo.lock') }}
+        restore-keys: ${{ runner.os }}-${{ inputs.cargo-cache-key }}
+
+    - name: Cache Cargo Dependencies With Fallback
+      if: ${{ inputs.cargo-cache-key && inputs.cargo-cache-fallback-key }}
+      uses: actions/cache@v4
+      with:
+        path: |
+          ~/.cargo/bin/
+          ~/.cargo/registry/index/
+          ~/.cargo/registry/cache/
+          ~/.cargo/git/db/
+          target/
+        key: ${{ runner.os }}-${{ inputs.cargo-cache-key }}-${{ hashFiles('**/Cargo.lock') }}
+        restore-keys: |
+          ${{ runner.os }}-${{ inputs.cargo-cache-key }}
+          ${{ runner.os }}-${{ inputs.cargo-cache-fallback-key }}-${{ hashFiles('**/Cargo.lock') }}
+          ${{ runner.os }}-${{ inputs.cargo-cache-fallback-key }}
+
+    - name: Cache Local Cargo Dependencies
+      if: ${{ inputs.cargo-cache-local-key }}
+      uses: actions/cache@v4
+      with:
+        path: |
+          .cargo/bin/
+          .cargo/registry/index/
+          .cargo/registry/cache/
+          .cargo/git/db/
+        key: ${{ runner.os }}-${{ inputs.cargo-cache-local-key }}-${{ hashFiles('**/Cargo.lock') }}
+        restore-keys: ${{ runner.os }}-${{ inputs.cargo-cache-local-key }}

.github/workflows/benchmark.yml

@@ -29,10 +29,6 @@ jobs:
         #    you only need to check `name`, `median` and `deviation`
         #
         test:
-          - {
-              name: "solana-sdk",
-              commands: ["cargo +$rust_nightly bench -p solana-sdk --features openssl-vendored"],
-            }
           - {
               name: "solana-runtime",
               commands: ["cargo +$rust_nightly bench -p solana-runtime"],

.github/workflows/client-targets.yml

@@ -9,7 +9,6 @@ on:
       - master
     paths:
       - "client/**"
-      - "sdk/**"
       - ".github/workflows/client-targets.yml"
       - "ci/rust-version.sh"
       - "**/Cargo.toml"