From 418cabb0091e8c6707ded04169b327150ff9d1ec Mon Sep 17 00:00:00 2001 From: Dongsu Park Date: Fri, 24 Aug 2018 11:44:42 +0200 Subject: [PATCH] docs: Address review comments Address review comments written by @wking. * Add a blank line between non-manpage refs and manpage refs. * Add backticks for `git status`. * Append a punctuation mark to each list item. * Use `pull request` instead of `PR`. * Specify `console` format to the example commands * Use relative links instead of static URIs * Remove a trailing blank * Remove trailing whitespaces * Run with sudo whenever it's possible * Use correct relative links * Move sections for running runtime validation one level up to the section `Running the runtime validation suite`. * Use `command -v` instead of `which`. * Elaborate why we should specify the TAP variable. Signed-off-by: Dongsu Park --- README.md | 3 +- docs/devel/release.md | 51 +++++++++++++-------------- docs/runtime-compliance-testing.md | 56 ++++++++++++++++-------------- 3 files changed, 56 insertions(+), 54 deletions(-) diff --git a/README.md b/README.md index ffbbdd0e2..da903b7a0 100644 --- a/README.md +++ b/README.md @@ -30,11 +30,12 @@ INFO[0000] Bundle validation succeeded. ## Testing OCI runtimes -Use the [runtime validation suite](doc/runtime-compliance-testing.md). +Use the [runtime validation suite](docs/runtime-compliance-testing.md). [bundle]: https://github.com/opencontainers/runtime-spec/blob/master/bundle.md [config.json]: https://github.com/opencontainers/runtime-spec/blob/master/config.md [runC]: https://github.com/opencontainers/runc [runtime-spec]: https://github.com/opencontainers/runtime-spec + [generate.1]: man/oci-runtime-tool-generate.1.md [validate.1]: man/oci-runtime-tool-validate.1.md diff --git a/docs/devel/release.md b/docs/devel/release.md index 9a3fa8647..81785ee0c 100644 --- a/docs/devel/release.md +++ b/docs/devel/release.md @@ -8,57 +8,56 @@ The following example assumes we're going from version 0.5.0 (`v0.5.0`) to 0.6.0 Let's get started: - Start at the relevant milestone on GitHub (e.g. https://github.com/opencontainers/runtime-tools/milestones/v0.6.0): ensure all referenced issues are closed (or moved elsewhere, if they're not done). Close the milestone. -- runtime-tools does not use a [roadmap file](https://github.com/opencontainers/runtime-tools/issues/465) but GitHub milestones. Update the [other milestones](https://github.com/opencontainers/runtime-tools/milestones), if necessary -- Branch from the latest master, make sure your git status is clean -- Update the [VERSION](https://github.com/opencontainers/runtime-tools/blob/master/VERSION) +- runtime-tools does not use a [roadmap file](https://github.com/opencontainers/runtime-tools/issues/465) but GitHub milestones. Update the [other milestones](https://github.com/opencontainers/runtime-tools/milestones), if necessary. +- Branch from the latest master, make sure your `git status` is clean. +- Update the [VERSION](https://github.com/opencontainers/runtime-tools/blob/master/VERSION). - Update the [release notes][changelog]. Try to capture most of the salient changes since the last release, but don't go into unnecessary detail (better to link/reference the documentation wherever possible). Ensure the branch is correct: - Ensure the build is clean! - - `git clean -ffdx && make && make test` should work -- Integration tests on CI should be green + - `git clean -ffdx && make && make test` should work. +- Integration tests on CI should be green. - Check the version of the binaries: - - Check `./oci-runtime-tool --version` - - Check `./runtimetest --version` + - Check `./oci-runtime-tool --version`. + - Check `./runtimetest --version`. Once everything is fine: -- File a PR -- Ensure the CI on the release PR is green +- File a pull request. +- Ensure the CI on the release PR is green. - Send an email to the [mailing list][mailinglist] ([example for v0.5.0](https://groups.google.com/a/opencontainers.org/forum/#!topic/dev/iuWpWUai4_I)) and get reviews from other [maintainers][maintainers]. -- Once the maintainers agree, merge the PR +- Once the maintainers agree, merge the PR. Sign a tagged release and push it to GitHub: -- Add a signed tag: `git tag -s v0.6.0 -m "release v0.6.0"` -- Push the tag to GitHub: `git push origin v0.6.0` +- Add a signed tag: `git tag -s v0.6.0 -m "release v0.6.0"`. +- Push the tag to GitHub: `git push origin v0.6.0`. Now we switch to the GitHub web UI to conduct the release: -- Start a [new release][gh-new-release] on Github -- Tag "v0.6.0", release title "v0.6.0" -- Copy-paste the release notes you added earlier in [CHANGELOG.md][changelog] +- Start a [new release][gh-new-release] on Github. +- Tag "v0.6.0", release title "v0.6.0". +- Copy-paste the release notes you added earlier in [CHANGELOG.md][changelog]. - Attach the release. This is a simple tarball: -``` -export VER="1.2.0" -export NAME="runtime-tools-v$VER" -mkdir -p $NAME/validation -cp oci-runtime-tool $NAME/ -cp validation/*.t $NAME/validation/ -sudo chown -R root:root $NAME/ -tar czvf $NAME.tar.gz --numeric-owner $NAME/ +```console +$ export VER="1.2.0" +$ export NAME="runtime-tools-v$VER" +$ mkdir -p $NAME/validation +$ cp oci-runtime-tool $NAME/ +$ cp validation/*.t $NAME/validation/ +$ sudo chown -R root:root $NAME/ +$ tar czvf $NAME.tar.gz --numeric-owner $NAME/ ``` - Publish the release! - Clean your git tree: `sudo git clean -ffdx`. -[changelog]: https://github.com/opencontainers/runtime-tools/blob/master/CHANGELOG.md -[maintainers]: https://github.com/opencontainers/runtime-tools/blob/master/MAINTAINERS +[changelog]: ../../CHANGELOG.md +[maintainers]: ../../MAINTAINERS [mailinglist]: https://groups.google.com/a/opencontainers.org/forum/#!forum/dev [gh-new-release]: https://github.com/opencontainers/runtime-tools/releases/new - diff --git a/docs/runtime-compliance-testing.md b/docs/runtime-compliance-testing.md index 1d1c25834..fec73764e 100644 --- a/docs/runtime-compliance-testing.md +++ b/docs/runtime-compliance-testing.md @@ -14,29 +14,15 @@ If your distribution does not package node-tap, you can install [npm][] (for exa ```console $ npm install tap ``` -### From a release - -Check if your release has pre-compiled tests on the [release page][releases] page. - -``` -$ tar xf runtime-tools-v0.6.0.tar.gz -$ RUNTIME=runc tap ./runtime-tools-v0.6.0/validation/*.t -``` - -### From source -Build the validation executables: - -```console -$ make runtimetest validation-executables -``` - -Runtime validation currently [only supports](docs/runtime-compliance-testing.md) the [OCI Runtime Command Line Interface](doc/command-line-interface.md). +Runtime validation currently [only supports](runtime-compliance-testing.md) the [OCI Runtime Command Line Interface](command-line-interface.md). If we add support for alternative APIs in the future, runtime validation will gain an option to select the desired runtime API. -For the command line interface, the `RUNTIME` option selects the runtime command (`funC` in the [OCI Runtime Command Line Interface](doc/command-line-interface.md)). +For the command line interface, the `RUNTIME` option selects the runtime command (`funC` in the [OCI Runtime Command Line Interface](command-line-interface.md)). + +Note that we should specify the `TAP` variable to the normal user's tap command, because usually the normal user had a newer version of `tap` installed locally via npm, for example, under `$HOME/node_modules/.bin`. ``` -$ sudo TAP="$(which tap)" RUNTIME=runc make localvalidation +$ sudo TAP="$(command -v tap)" RUNTIME=runc make localvalidation RUNTIME=runc /home/alban/.nvm/versions/node/v9.7.1/bin/tap validation/pidfile.t validation/linux_cgroups_memory.t validation/linux_rootfs_propagation_shared.t validation/kill.t validation/linux_readonly_paths.t validation/hostname.t validation/hooks_stdin.t validation/create.t validation/poststart.t validation/linux_cgroups_network.t validation/poststop_fail.t validation/prestart_fail.t validation/linux_cgroups_relative_blkio.t validation/default.t validation/poststop.t validation/linux_seccomp.t validation/prestart.t validation/process_rlimits.t validation/linux_masked_paths.t validation/killsig.t validation/process.t validation/linux_cgroups_relative_pids.t validation/hooks.t validation/linux_rootfs_propagation_unbindable.t validation/linux_cgroups_relative_cpus.t validation/misc_props.t validation/linux_sysctl.t validation/process_oom_score_adj.t validation/linux_devices.t validation/process_capabilities_fail.t validation/start.t validation/linux_cgroups_pids.t validation/process_capabilities.t validation/poststart_fail.t validation/linux_cgroups_relative_hugetlb.t validation/mounts.t validation/linux_cgroups_hugetlb.t validation/linux_cgroups_relative_memory.t validation/state.t validation/root_readonly_true.t validation/linux_cgroups_blkio.t validation/delete.t validation/linux_cgroups_relative_network.t validation/process_rlimits_fail.t validation/linux_cgroups_cpus.t validation/linux_uid_mappings.t validation/pidfile.t .................................. 1/1 455ms validation/linux_cgroups_memory.t ..................... 9/9 @@ -101,7 +87,7 @@ validation/linux_seccomp.t ............................ 0/1 not ok validation/linux_seccomp.t error: >- Pre-start hooks MUST be called after the `start` operation is called - + Refer to: https://github.com/opencontainers/runtime-spec/blob/v1.0.0/config.md#prestart @@ -208,10 +194,10 @@ validation/process_capabilities_fail.t .............. 20/27 validation/start.t .....exit status 2 validation/start.t .................................... 6/7 not ok test count !== plan - +++ found - --- wanted - -1 - +6 + +++ found + --- wanted + -1 + +6 results: ok: false count: 6 @@ -266,7 +252,7 @@ validation/linux_cgroups_blkio.t .................... 15/15 validation/delete.t ................................... 3/5 22s not ok attempting to `delete` a container that is not `stopped` MUST generate an error reference: 'https://github.com/opencontainers/runtime-spec/blob/v1.0.0/runtime.md#delete' - + not ok attempting to `delete` a container that is not `stopped` MUST have no effect on the container error: exit status 1 reference: 'https://github.com/opencontainers/runtime-spec/blob/v1.0.0/runtime.md#delete' @@ -294,7 +280,7 @@ validation/linux_uid_mappings.t ..................... 20/26 syscall action SCMP_ACT_ALLOW total ............................................. 420/517 - + 420 passing (1m) 88 pending @@ -340,7 +326,7 @@ ok 26 - gid mappings or with the environment variable `VALIDATION_TESTS`: ```console -$ sudo make TAP=$(which tap) RUNTIME=runc VALIDATION_TESTS=validation/default.t localvalidation +$ sudo make TAP=$(command -v tap) RUNTIME=runc VALIDATION_TESTS=validation/default.t localvalidation RUNTIME=runc /home/alban/.nvm/versions/node/v9.7.1/bin/tap validation/default.t validation/default.t ................................ 20/26 Skipped: 6 @@ -368,6 +354,22 @@ Files=2, Tests=35, 1 wallclock secs ( 0.03 usr 0.00 sys + 0.12 cusr 0.12 csy Result: PASS ``` +### From a release + +Check if your release has pre-compiled tests on the [release page][releases] page. + +```console +$ tar xf runtime-tools-v0.6.0.tar.gz +$ sudo RUNTIME=runc tap ./runtime-tools-v0.6.0/validation/*.t +``` + +### From source + +Build the validation executables: + +```console +$ make runtimetest validation-executables +``` [compliance]: https://github.com/opencontainers/runtime-spec/blob/v1.0.1/spec.md [debian-node-tap]: https://packages.debian.org/stretch/node-tap