Skip to content

Commit

Permalink
feat: support for compressed wasm & switch to the runtimes key
Browse files Browse the repository at this point in the history
- add compressed wasm support
- switch to the new "runtimes" keys instead of using the root of the json
- fix tests
- fix doc and add 'md' script
  • Loading branch information
chevdor committed Jun 28, 2021
1 parent 9675fde commit a04f7c6
Show file tree
Hide file tree
Showing 6 changed files with 225 additions and 29 deletions.
60 changes: 60 additions & 0 deletions .github/workflows/polkadot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
name: Build Polkadot Repo
on:
push:
branches:
- "*"

workflow_dispatch:
inputs:
srtool_tag:
description: The SRTOOL tag to use
default: 1.53.0
required: false

schedule:
- cron: "00 04 * * 1" # 4AM weekly on mondays

jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
chain: ["polkadot", "kusama", "westend"]

name: Build ${{ matrix.chain }}
steps:

- uses: actions/checkout@v2
with:
path: action

- uses: actions/checkout@v2
with:
repository: paritytech/polkadot
fetch-depth: 0
path: repo

- name: debug
run: |
ls -al
pwd
# - name: Go in the repo
# working-directory: ./repo
# run: pwd

- name: Srtool build
id: srtool_build
uses: ./action
with:
chain: ${{ matrix.chain }}
tag: ${{ github.event.inputs.srtool_tag }}
# runtime_dir: repo/runtime/${{ matrix.chain }}
workdir: "${{ github.workspace }}/repo"

- name: Summary
run: |
echo '${{ steps.srtool_build.outputs.json }}' | jq . | tee ${{ matrix.chain }}-srtool-digest.json
echo "Compact Runtime location : ${{ steps.srtool_build.outputs.wasm }}"
echo "Compressed Runtime location : ${{ steps.srtool_build.outputs.wasm_compressed }}"
24 changes: 0 additions & 24 deletions .github/workflows/test.yml

This file was deleted.

149 changes: 149 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
# SRTOOL Github Action

## Introduction

[srtool](https://gitlab.com/chevdor/srtool) is a docker image that allows building Substrate WASM Runtimes in a deterministic manner. It ensures that all `srtool` users (and tooling), using the same/latest tag, will produce 100% exactly the same runtime. It enables further trustless verifications.

![srtool gh action 512px](resources/logo/srtool-gh-action_512px.png)

This repository contains a custom [Github Action](https://docs.github.com/en/actions) that will help you integrate `srtool` in your Github CI.

Gitlab users are not left behind and can use the 2 first options mentioned below.

## Srtool helpers

There are now several ways to use `srtool` in your project:

- using `srtool` [via alias](https://gitlab.com/chevdor/srtool#user-content-using-an-alias): powerful but very verbose and prone to errors. This is for developers. This option is being deprecated and not recommended.

- using the [srtool-cli](https://gitlab.com/chevdor/srtool-cli): much easier to use and removes a whole bunch of potential user’s mistakes. This is for developers.

- using the [srtool-app](https://gitlab.com/chevdor/srtool-app): the easiest option of all thanks to its GUI. This is good for non-developers.

- using the Github actions from [this repo](https://github.com/chevdor/srtool-actions). This is for your automatic CI.

## Requirements

In order to use this Github Action, you will need a Github repository with your Substrate based project. You can learn how to get started with a first Github workflow [here](https://docs.github.com/en/actions/quickstart).

Before you setup your new workflow, you should gather the following information:

- name of your chain: ie. polkadot

- runtime package: ie. runtime-polkadot

- location of your runtime: ie. runtime/polkadot

If your project uses standard values (such as your runtime package being named `xyz-runtime` if your chain is `xyz`), you will be able to skip many of the inputs. If you have a different setup, you can override the defaults to make it work for you.

## Sample workflows

Make sure you store the yml files shown below in your repository under `.github/workflows`.

## basic

**basic.yml**

name: Srtool build

on: push

jobs:
srtool:
runs-on: ubuntu-latest
strategy:
matrix:
chain: ["statemine", "westmint"]
steps:
- uses: actions/checkout@v2
- name: Srtool build
id: srtool_build
uses: chevdor/srtool-actions@v0.1.0
with:
chain: ${{ matrix.chain }}
runtime_dir: polkadot-parachains/${{ matrix.chain }}-runtime
- name: Summary
run: |
echo '${{ steps.srtool_build.outputs.json }}' | jq . > ${{ matrix.chain }}-srtool-digest.json
cat ${{ matrix.chain }}-srtool-digest.json
echo "Runtime location: ${{ steps.srtool_build.outputs.wasm }}"

## Report

name: Srtool build

on: push

jobs:
srtool:
runs-on: ubuntu-latest
strategy:
matrix:
chain: ["westend"]
steps:
- uses: actions/checkout@v2
- name: Srtool build
id: srtool_build
uses: chevdor/srtool-actions@v0.1.0
with:
chain: ${{ matrix.chain }}
- name: Summary
run: |
echo Summary:
echo - version: ${{ steps.srtool_build.outputs.version }}
echo - info: ${{ steps.srtool_build.outputs.info }}
echo - prop: ${{ steps.srtool_build.outputs.proposal_hash }}
echo - json: ${{ steps.srtool_build.outputs.json }}

## Artifacts

name: Srtool build

on: push

jobs:
srtool:
runs-on: ubuntu-latest
strategy:
matrix:
chain: ["statemine", "westmint"]
steps:
- uses: actions/checkout@v2
- name: Srtool build
id: srtool_build
uses: chevdor/srtool-actions@v0.1.0
with:
chain: ${{ matrix.chain }}
runtime_dir: polkadot-parachains/${{ matrix.chain }}-runtime
- name: Summary
run: |
echo '${{ steps.srtool_build.outputs.json }}' | jq . > ${{ matrix.chain }}-srtool-digest.json
cat ${{ matrix.chain }}-srtool-digest.json
echo "Runtime location: ${{ steps.srtool_build.outputs.wasm }}"
- name: Archive Runtime
uses: actions/upload-artifact@v2
with:
name: ${{ matrix.chain }}-runtime-${{ github.sha }}
path: |
${{ steps.srtool_build.outputs.wasm }}
${{ matrix.chain }}-srtool-digest.json

## Dev notes

Due to a [bug in act](https://github.com/nektos/act/issues/655), the defaults defined in the action are not applied. That means **must** pass all the inputs while testing with `act`.

You can test locally using [act](https://github.com/nektos/act):

act -W tests

To make it easier, you can also use `just`:

just test <test_name>

For instance:

# Run the default dev workflow: tests/dev.yml
just test

# Run the ipfs test workflow
just test ipfs
File renamed without changes.
16 changes: 11 additions & 5 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,13 @@ outputs:
value: ${{ steps.build.outputs.ipfs }}

wasm:
description: Path of the produced runtime
description: Path of the produced compact runtime
value: ${{ steps.build.outputs.wasm }}

wasm_compressed:
description: Path of the produced compressed runtime
value: ${{ steps.build.outputs.wasm_compressed }}

runs:
using: "composite"
steps:
Expand Down Expand Up @@ -162,15 +166,17 @@ runs:
echo ::set-output name=json::$JSON
echo $JSON | jq .
PROP=`echo $JSON | jq -r .prop`
PROP=`echo $JSON | jq -r .runtimes.compact.prop`
echo ::set-output name=proposal_hash::$PROP
WASM=`echo $JSON | jq -r .wasm`
WASM=`echo $JSON | jq -r .runtimes.compact.wasm`
echo ::set-output name=wasm::$WASM
IPFS=`echo $JSON | jq -r .ipfs`
echo ::set-output name=ipfs::$IPFS
Z_WASM=`echo $JSON | jq -r .runtimes.compressed.wasm`
echo ::set-output name=wasm_compressed::$Z_WASM
IPFS=`echo $JSON | jq -r .runtimes.compact.ipfs`
echo ::set-output name=ipfs::$IPFS
}
echo ::endgroup
5 changes: 5 additions & 0 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,8 @@ graph file="dev":

clean:
rm -rf polkadot workflow _actions

# Generate the readme as .md
md:
#!/usr/bin/env bash
asciidoctor -b docbook -a leveloffset=+1 -o - README_src.adoc | pandoc --markdown-headings=atx --wrap=preserve -t markdown_strict -f docbook - > README.md

0 comments on commit a04f7c6

Please sign in to comment.