Skip to content

Commit

Permalink
Feature: simulate publishing (zellij-org#2194)
Browse files Browse the repository at this point in the history
* xtask: Add arguments to `publish`

that allow specifying a custom git remote to push to and a custom cargo
registry to publish packages to.

* xtask/publish: Don't release `xtask` subcrate

because it's not meant to be released at all.

* xtask/publish: Add status messages to publish

so we see what crate is currently being published, too.

* xtask/publish: Disable default features on `zellij`

because otherwise it tries to pick up the debug builds of the plugins,
which aren't part of released version of `zellij utils`.

* xtask/publish: Fix handling of custom registry

* docs: Add `RELEASE.md`

which explains how to simulate a zellij release.

* xtask: Apply rustfmt

* xtask: Remove `wasm-opt` from build steps

because recent versions cause havoc in the release process in GitHub
pipelines and it's primary goal is to only reduce binary size. Current
rust versions seem to produce very compact wasm binaries themselves,
though.

* .github: Don't install wasm-opt in workflows
  • Loading branch information
har7an authored and joshheyse committed Mar 11, 2023
1 parent 3b3b6a7 commit 845f673
Show file tree
Hide file tree
Showing 8 changed files with 193 additions and 133 deletions.
2 changes: 0 additions & 2 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,6 @@ jobs:
run: sudo apt-get install -y --no-install-recommends musl-tools
- name: Add musl target
run: rustup target add x86_64-unknown-linux-musl
- name: Install wasm-opt
run: sudo apt-get install -y --no-install-recommends binaryen
#run: cargo install --debug cargo-make
- name: Build asset
run: cargo xtask ci e2e --build
Expand Down
8 changes: 0 additions & 8 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,14 +63,6 @@ jobs:
if: matrix.os == 'ubuntu-latest'
run: sudo apt-get install -y --no-install-recommends musl-tools

- name: Install wasm-opt (macos)
if: runner.os == 'macos'
run: brew install binaryen

- name: Install wasm-opt (ubuntu)
if: matrix.os == 'ubuntu-latest'
run: sudo apt-get install -y --no-install-recommends binaryen

# Workaround for <https://github.com/actions/virtual-environments/issues/2557>
- name: Switch Xcode SDK
if: runner.os == 'macos'
Expand Down
3 changes: 0 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,6 @@ cargo xtask manpage
You can see a list of all commands (with supported arguments) with `cargo xtask
--help`. For convenience, `xtask` may be shortened to `x`: `cargo x build` etc.

To run `install` or `publish`, you'll need the package `binaryen` in the
version `wasm-opt --version` > 97, for it's command `wasm-opt`.

To run `test`, you will need the package `pkg-config` and a version of `openssl`.

## Running the end-to-end tests
Expand Down
144 changes: 144 additions & 0 deletions docs/RELEASE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
# How to release a zellij version

This document is primarily target at zellij maintainers in need to (prepare to)
release a new zellij version.


## Simulating a release

This section explains how to do a "dry-run" of the release process. This is
useful to check if a release is successful beforehand, i.e. before publishing
it to the world. Because there is no "undo"-button for a real release as
described below, it is recommended to perform a simulated release first.


### Requirements

You only need a publicly accessible Git repository to provide a cargo registry.


### High-level concept

The setup explained below will host a third-party cargo registry software
([ktra](https://github.com/moriturus/ktra)) locally on your PC. In order for
`cargo` to pick this up and be able to work with it, we must perform a few
modifications to the zellij repository and other components. Once setup, we
release a zellij version to this private registry and install zellij from there
to make sure it works as expected.


### Step-by-step guide

1. Create a cargo index repository
1. Create a new repo on some git forge (GitHub/GitLab/...)
1. Clone the repo **with HTTPS (not SSH)**, we'll refer to the `https://`
clone-url as `$INDEX_REPO` for the remainder of this text
1. Add a file named `config.json` with the following content in the root:
```json
{"dl":"http://localhost:8000/dl","api":"http://localhost:8000"}
```
1. Generate an access token for full repo access, we'll refer to this as
`$TOKEN` for the remained of this text
1. Create and push a commit with these changes. Provide the following HTTPS
credentials:
1. Username: Your git-forge username
1. Password: `$TOKEN`
1. Prepare the zellij repo
1. `cd` into your local copy of the zellij repository
1. Add a new cargo registry to `.cargo/config.toml` like this:
```toml

[registries]
ktra = { index = "https://$INDEX_REPO" }
```
1. Modify **all** `Cargo.toml` in the zellij repo to retrieve the individual
zellij subcrates from the private registry:
1. Find all dependencies that look like this:
```toml
zellij-utils = { path = "../zellij-utils/", version = "XXX" }
```
1. Change them to look like this
```toml
zellij-utils = { path = "../zellij-utils/", version = "XXX", registry = "ktra" }
```
1. This applies to all zellij subcrates, e.g. `zellij-client`,
`zellij-server`, ... You can ignore the plugins, because these aren't
released as sources.
1. Launch your private registry
1. Create the file `~/.cargo/config.toml` with the following content:
```
[registries.ktra]
index = "https://$INDEX_REPO"
```
1. Install `ktra`, the registry server: `cargo install ktra`
1. In a separate shell/pane/whatever, navigate to some folder where you
want to store all data for the registry
1. Create a config file for `ktra` named `ktra.toml` there with the
following content:
```toml
[index_config]
remote_url = "https://$INDEX_REPO"
https_username = "your-git-username"
https_password = "$TOKEN"
branch = "main" # Or whatever branch name you used
```
1. Launch ktra (with logging to see what happens): `RUST_LOG=debug ktra`
1. Get a registry token for `ktra` (The details don't really matter, unless
you want to reuse this registry):
```bash
curl -X POST -H 'Content-Type: application/json' -d '{"password":"PASSWORD"}' http://localhost:8000/ktra/api/v1/new_user/ALICE
```
1. Login to the registry with the token you received as reply to the
previous command:
```bash
cargo login --registry ktra "KTRA_TOKEN"
```
1. **Install safety measures to prevent accidentally performing a real release**:
1. In your `zellij` repo, remove all configured remotes that allow you to
push/publish directly to the zellij main GitHub repo. Setup a fork of
the main zellij repo instead and configure a remote that allows you to
push/publish to that. Please, this is very important.
1. Comment out the entire `[registry]` section in `~/.cargo/credentials` to
prevent accidentally pushing a new release to `crates.io`.
1. **Simulate a release**
1. Go back to the zellij repo, type:
```bash
cargo x publish --git-remote <YOUR_ZELLIJ_FORK> --cargo-registry ktra
```
1. A prompt will open with the commit message for the release commit. Just
save and close your editor to continue
1. If all goes well, the release will be done in a few minutes and all the
crates are published to the private `ktra` registry!
1. Testing the release binary
1. Install zellij from the registry to some local directory like this:
```bash
$ cargo install --registry ktra --root /tmp zellij
```
1. Execute the binary to see if all went well:
```bash
$ /tmp/bin/zellij
```
1. Cleaning up
1. Uncomment the `[registry]` section in `~/.cargo/config.toml`
1. Restore your original git remotes for the zellij repo
1. Undo your last commit:
```bash
$ git reset --hard HEAD~1
```
1. Undo your last commit in the remote zellij repo:
```bash
$ git push --force <YOUR_ZELLIJ_FORK>
```
1. Delete the release tag:
```bash
$ git tag -d "vX.Y.Z"
```
1. Delete the release tag in the remote zellij repo
```bash
$ git push <YOUR_ZELLIJ_FORK> --force --delete "vX.Y.Z"
```

You're done! :tada:


## Releasing a new version
82 changes: 0 additions & 82 deletions xtask/src/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
//! Currently has the following functions:
//!
//! - [`build`]: Builds general cargo projects (i.e. zellij components) with `cargo build`
//! - [`wasm_opt_plugin`]: Calls `wasm-opt` on all plugins
//! - [`manpage`]: Builds the manpage with `mandown`
use crate::flags;
use anyhow::Context;
Expand Down Expand Up @@ -48,91 +47,10 @@ pub fn build(sh: &Shell, flags: flags::Build) -> anyhow::Result<()> {
base_cmd = base_cmd.arg("--release");
}
base_cmd.run().with_context(err_context)?;

if subcrate.contains("plugins") {
let (_, plugin_name) = subcrate
.rsplit_once('/')
.context("Cannot determine plugin name from '{subcrate}'")?;

if flags.release {
// Perform wasm-opt on plugin
wasm_opt_plugin(sh, plugin_name).with_context(err_context)?;
}
}
}
Ok(())
}

/// Call `wasm-opt` on all plugins.
///
/// Plugins are discovered automatically by scanning the contents of `target/wasm32-wasi/release`
/// for filenames ending with `.wasm`. For this to work the plugins must be built beforehand.
// TODO: Should this panic if there is no plugin found? What should we do when only some plugins
// have been built before?
pub fn wasm_opt_plugin(sh: &Shell, plugin_name: &str) -> anyhow::Result<()> {
let err_context = || format!("failed to run 'wasm-opt' on plugin '{plugin_name}'");

let wasm_opt = wasm_opt(sh).with_context(err_context)?;

let asset_dir = crate::project_root()
.join("zellij-utils")
.join("assets")
.join("plugins");
sh.create_dir(&asset_dir).with_context(err_context)?;
let _pd = sh.push_dir(asset_dir);

let plugin = PathBuf::from(
std::env::var_os("CARGO_TARGET_DIR")
.unwrap_or(crate::project_root().join("target").into_os_string()),
)
.join("wasm32-wasi")
.join("release")
.join(plugin_name)
.with_extension("wasm");

if !plugin.is_file() {
return Err(anyhow::anyhow!("No plugin found at '{}'", plugin.display()))
.with_context(err_context);
}
let name = match plugin.file_name().with_context(err_context)?.to_str() {
Some(name) => name,
None => {
return Err(anyhow::anyhow!(
"couldn't read filename containing invalid unicode"
))
.with_context(err_context)
},
};

// This is a plugin we want to optimize
println!();
let msg = format!(">> Optimizing plugin '{name}'");
crate::status(&msg);
println!("{}", msg);

let input = plugin.as_path();
cmd!(sh, "{wasm_opt} -O {input} -o {name}")
.run()
.with_context(err_context)?;

Ok(())
}

/// Get the path to a `wasm-opt` executable.
///
/// If the executable isn't found, an error is returned instead.
// TODO: Offer the user to install latest wasm-opt on path?
fn wasm_opt(_sh: &Shell) -> anyhow::Result<PathBuf> {
match which::which("wasm-opt") {
Ok(path) => Ok(path),
Err(e) => {
println!("!! 'wasm-opt' wasn't found but is needed for this build step.");
println!("!! Please install it from here: https://github.com/WebAssembly/binaryen");
Err(e).context("couldn't find 'wasm-opt' executable")
},
}
}

/// Build the manpage with `mandown`.
// mkdir -p ${root_dir}/assets/man
// mandown ${root_dir}/docs/MANPAGE.md 1 > ${root_dir}/assets/man/zellij.1
Expand Down
7 changes: 6 additions & 1 deletion xtask/src/flags.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ xflags::xflags! {
cmd publish {
/// Perform a dry-run (don't push/publish anything)
optional --dry-run
/// Push commit to custom git remote
optional --git-remote remote: OsString
/// Publish crates to custom registry
optional --cargo-registry registry: OsString
}

/// Package zellij for distribution (result found in ./target/dist)
Expand Down Expand Up @@ -151,6 +155,8 @@ pub struct Manpage;
#[derive(Debug)]
pub struct Publish {
pub dry_run: bool,
pub git_remote: Option<OsString>,
pub cargo_registry: Option<OsString>,
}

#[derive(Debug)]
Expand All @@ -175,7 +181,6 @@ pub struct Run {
pub args: Vec<OsString>,

pub data_dir: Option<PathBuf>,

pub singlepass: bool,
}

Expand Down
30 changes: 0 additions & 30 deletions xtask/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,36 +5,6 @@
//! xtask install` for installation of rust-analyzer server and client.
//!
//! This binary is integrated into the `cargo` command line by using an alias in `.cargo/config`.
// Current default "flow":
// - format-flow: `cargo fmt`
// - format-toml-conditioned-flow: ??
// - build: `cargo build`
// - test: `cargo test`
// - clippy: `cargo clippy --all-targets --all-features -- --deny warnings $@`
//
// # Install flow:
// - build-plugins-release: `cargo build --release ...`
// - wasm-opt-plugins: `wasm-opt ...`
// - build-release: `cargo build --release`
// - install-mandown: `cargo install mandown`
// - manpage: |
// mkdir -p ${root_dir}/assets/man
// mandown ${root_dir}/docs/MANPAGE.md 1 > ${root_dir}/assets/man/zellij.1
// - install: `cp target/release/zellij "$1"`
//
// # Release flow:
// - workspace: cargo make --profile development -- release
//
// # Publish flow:
// - update-default-config:
// - build-plugins-release: `cargo build --release ...`
// - wasm-opt-plugins: `wasm-opt ...`
// - release-commit:
// - commit-all: `git commit -aem "chore(release): v${CRATE_VERSION}"`
// - tag-release: `git tag --annotate --message "Version ${CRATE_VERSION}"
// "v${CRATE_VERSION}"`
// - `git push --atomic origin main "v${CRATE_VERSION}"`
// - publish-zellij: `cargo publish [tile, client, server, utils, tile-utils, zellij]`

mod build;
mod ci;
Expand Down
Loading

0 comments on commit 845f673

Please sign in to comment.