Commit 488dfe4
Add initial command line spec
# Commands
## create
The --bundle [start-pr-bundle] and --pid-file options and ID argument
[runc-start-id] match runC's interface.
One benefit of the early-exit 'create' is that the exit code does not
conflate container process exits with "failed to setup the sandbox"
exits. We can take advantage of that and use non-zero 'create' exits
to allow stderr writing (so the runtime can log errors while dying
without having to successfully connect to syslog or some such).
Trevor still likes the long-running 'create' API because it makes
collecting the exit code easier, see the entry under rejected-for-now
avenues at the end of this commit message.
### --pid-file
You can get the PID by calling 'state' [container-pid-from-state], and
container PIDs may not be portable [container-pid-not-portable]. But
it's a common way for interfacing with init systems like systemd
[systemd-pid], and for this first pass at the command line API folks
are ok with some Linux-centrism [linux-centric].
### Document LISTEN_FDS for passing open file descriptors
This landed in runC with [runc-listen-fds], but the bundle-author <->
runtime specs explicitly avoided talking about how this is set (since
the bundle-author didn't care about the runtime-caller <-> runtime
interface) [runtime-spec-caller-api-agnostic]. This commit steps away
from that agnosticism.
Trevor left LISTEN_PID [sd_listen_fds,listen-fds-description] out,
since he doesn't see how the runtime-caller would choose anything
other than 1 for its value. It seems like something that a process
would have to set for itself (because guessing the PID of a child
before spawning it seems racy ;). In any event, the runC
implementation seems to set this to 1 regardless of what systemd
passes to it [listen-fds-description].
We've borrowed Shishir's wording for the example
[listen-fds-description].
## state [state-pr]
Partially catch up with opencontainers/runtime-spec@7117ede7 (Expand
on the definition of our ops, 2015-10-13,
opencontainers#225, v0.4.0). The state example is
adapted from runtime.md, but we defer the actual specification of the
JSON to that file.
The encoding for the output JSON (and all standard-stream activity) is
covered by the "Character encodings" section. In cases where the
runtime ignores the SHOULD (still technically compliant), RFC 7159
makes encoding detection reasonably straightforward [rfc7159-s8.1].
The obsolete RFC 4627 has some hints as well [rfc4627-s3] (although
these were dropped in RFC 7518 [rfc7518-aA], probably as a result of
removing the constraint that "JSON text" be an object or array
[rfc7518-aA]). The hints should still apply to the state output,
because we know it will be an object. If that ends up being too dicey
and we want to certify runtimes that do not respect their
operating-system conventions, we can add an --encoding option later.
## kill
Partially catch up with opencontainers/runtime-spec@be594153 (Split
create and start, 2016-04-01, opencontainers#384). The
interface is based on POSIX [posix-kill], util-linux
[util-linux-kill], and GNU coreutils [coreutils-kill].
The TERM/KILL requirement is a minimum portability requirement for
soft/hard stops. Windows lacks POSIX signals [windows-signals], and
currently supports soft stops in Docker with whatever is behind
hcsshim.ShutdownComputeSystem [docker-hcsshim]. The docs we're
landing here explicitly allow that sort of substitution, because we
need to have soft/hard stop on those platforms but *can't* use POSIX
signals. They borrow wording from
opencontainers/runtime-spec@35b0e9ee (config: Clarify MUST for
platform.os and .arch, 2016-05-19, opencontainers#441) to
recommend runtime authors document the alternative technology so
bundle-authors can prepare (e.g. by installing the equivalent to a
SIGTERM signal handler).
# Command style
Use imperative phrasing for command summaries, to follow the practice
recommended by Python's PEP 257 [pep257-docstring]:
The docstring is a phrase ending in a period. It prescribes the
function or method's effect as a command ("Do this", "Return that"),
not as a description; e.g. don't write "Returns the pathname ...".
The commands have the following layout:
### {command name}
{one-line description}
* *Options:* ...
...
* *Exit code:* ...
{additional notes}
#### Example
{example}
The four-space list indents follow opencontainers/runtime-spec@7795661
(runtime.md: Fix sub-bullet indentation, 2016-06-08,
opencontainers#495). From [markdown-syntax]:
List items may consist of multiple paragraphs. Each subsequent
paragraph in a list item must be indented by either 4 spaces or one
tab...
Trevor expects that's intended to be read with "block element" instead
of "paragraph", in which case it applies to nested lists too.
And while GitHub supports two-space indents [github-lists]:
You can create nested lists by indenting lines by two spaces.
it seems that pandoc does not.
# Global options
This section is intended to allow runtimes to extend the command line
API with additional options and commands as they see fit without
interfering with the commands and options specified in this document.
The last line in this section makes it explicit that any later
specification (e.g. "MUST print the state JSON to its stdout") do not
apply to cases where the caller has included an unspecified option or
command (e.g. --format=protobuf). For extensive discussion on this
point see [extensions-unspecified].
With regard to the statement "Command names MUST NOT start with
hyphens", the rationale behind this decision is to distinguish
unrecognized commands from unrecognized options
[distinguish-unrecognized-commands] because we want to allow (but not
require) runtimes to fail fast when faced with an unrecognized
command [optional-fail-fast].
# Long options
Use GNU-style long options to avoid ambiguous, one-character options
in the spec, while still allowing the runtime to support one-character
options with packing. We don't specify one-character options in this
spec, because portable callers can use the long form, and not
specifying short forms leaves runtimes free to assign those as they
see fit.
# Character encodings
Punt to the operating system for character encodings. Without this,
the character set for the state JSON or other command output seemed
too ambiguous.
Trevor wishes there were cleaner references for the
{language}.{encoding} locales like en_US.UTF-8 and UTF-8. But
[wikipedia-utf-8,wikipedia-posix-locale] seems too glib, and he can't
find a more targetted UTF-8 link than just dropping folks into a
Unicode chapter (which is what [wikipedia-utf-8] does):
The Unicode Standard, Version 6.0, §3.9 D92, §3.10 D95 (2011)
With the current v8.0 (2015-06-17), it's still §3.9 D92 and §3.10 D95.
The TR35 link is for:
In addition, POSIX locales may also specify the character encoding,
which requires the data to be transformed into that target encoding.
and the POSIX §6.2 link is for:
In other locales, the presence, meaning, and representation of any
additional characters are locale-specific.
# Standard streams
The "MUST NOT attempt to read from its stdin" means a generic caller
can safely exec the command with a closed or null stdin, and not have
to worry about the command blocking or crashing because of that. The
stdout spec for start/delete is more lenient, because runtimes are
unlikely to change their behavior because they are unable to write to
stdout. If this assumption proves troublesome, we may have to tighten
it up later.
# Event triggers
The "Callers MAY block..." wording is going to be hard to enforce, but
with the runC model, clients rely on the command exits to trigger
post-create and post-start activity. The longer the runtime hangs
around after completing its action, the laggier those triggers will
be.
For an alternative event trigger approach, see the discussion of an
'event' command in the rejected-for-now avenues at the end of this
commit message.
# Lifecycle notes
These aren't documented in the current runtime-spec, and may no longer
be true. But they were true at one point, and informed the
development of this specification.
## Process cleanup
On IRC on 2015-09-15 (with PDT timestamps):
10:56 < crosbymichael> if the main process dies in the container,
all other process are killed
...
10:58 < julz> crosbymichael: I'm assuming what you mean is you kill
everything in the cgroup -> everything in the container dies?
10:58 < crosbymichael> julz: yes, that is how its implemented
...
10:59 < crosbymichael> julz: we actually freeze first, send the
KILL, then unfreeze so we don't have races
## Container IDs for namespace joiners
You can create a config that adds no isolation vs. the runtime
namespace or completely joins another set of existing namespaces. It
seems odd to call that a new "container", but the ID is really more of
a process ID, and less of a container ID. The "container" phrasing is
just a useful hint that there might be some isolation going on. And
we're always creating a new "container process" with 'create'.
# Other changes
This commit also drops the file-descriptor docs from runtime-linux.
It's unclear how these apply to runtimes APIs that are not based on
the command line / execve, and the functionality is covered by the
more tightly scoped LISTEN_FDS wording in the command-line docs.
# Avenues pursued but rejected (for now)
* Early versions of this specification had 'start' taking '--config'
and '--runtime', but this commit uses '--bundle' [start-pr-bundle].
The single config file change [single-config-proposal] went through,
but Trevor would also like to be able to pipe a config into the
'funC start' command (e.g. via a /dev/fd/3 pseudo-filesystem path)
[runc-config-via-stdin], and he has a working example that supports
this without difficulty [ccon-config-via-stdin]. But since
[runc-bundle-option] landed on 2015-11-16, runC has replaced their
--config-file and --runtime-file flags with --bundle, and the
current goal of this API is "keeping as much similarity with the
existing runC command-line as possible", not "makes sense to Trevor"
;). It looks like runC was reacting [runc-required-config-file] to
strict wording in the spec [runtime-spec-required-config-file], so
we might be able to revisit this if/when we lift that restriction.
* Having 'start' (now 'create') take a --state option to write state
to a file [start-pr-state]. This is my preferred approach to
sharing container state, since it punts a persistent state registry
to higher-level tooling [punt-state-registry]. But runtime-spec
currently requires the runtime to maintain such a registry
[state-registry], and we don't need two ways to do that ;).
On systems like Solaris, the kernel maintains a registry of
container IDs directly, so they don't need an external registry
[solaris-kernel-state].
* Having 'start' (now 'create') take an --id option instead of a
required ID argument, and requiring the runtime to generate a unique
ID if the option was not set. When there is a long-running host
process waiting on the container process to perform cleanup, the
runtime-caller may not need to know the container ID. However, runC
has been requiring a user-specified ID since [runc-start-id], and
this spec follows the early-exit 'create' from [runc-create-start],
so we require one here. We can revisit this if we regain a
long-running 'create' process.
* Having a long-running 'create' process. Trevor is not a big fan of
this early-exit 'create', which requires platform-specific magic to
collect the container process's exit code. The ptrace idea in this
commit is from Mrunal [mrunal-ptrace].
Trevor has a proposal for an 'event' operation [event] which would
provide a convenient created trigger. With 'event' in place, we
don't need the 'create' process exit to serve as that trigger, and
could have a long-running 'create' that collects the container
process exit code using the portable waitid() family. But the
consensus after this week's meeting was to table that while we land
docs for the runC API [mimic-runc].
* Having a 'version' command to make it easy for a caller to report
which runtime they're using. But we don't have a use-case that
makes it strictly necessary for interop, so we're leaving it out for
now [no-version].
* Using 'sh' syntax highlighting [syntax-highlighting] for the fenced
code blocks. The 'sh' keyword comes from [linguist-languages]. But
the new fenced code blocks are shell sessions, not scripts, and we
don't want shell-syntax highlighting in the command output.
[ccon-config-via-stdin]: https://github.com/wking/ccon/tree/v0.4.0#configuration
[container-pid-from-state]: https://github.com/opencontainers/runtime-spec/pull/511/files#r70353376
Subject: Add initial pass at a cmd line spec
[container-pid-not-portable]: opencontainers#459
Subject: [ Runtime ] Allow for excluding pid from state
[coreutils-kill]: http://www.gnu.org/software/coreutils/manual/html_node/kill-invocation.html
[distinguish-unrecognized-commands]: https://github.com/wking/oci-command-line-api/pull/8/files#r46898167
Subject: Clarity for commands vs global options
[docker-hcsshim]: https://github.com/docker/docker/pull/16997/files#diff-5d0b72cccc4809455d52aadc62329817R230
moby/moby@bc503ca8 (Windows: [TP4] docker kill handling,
2015-10-12, moby/moby#16997)
[event]: opencontainers#508
Subject: runtime: Add an 'event' operation for subscribing to pushes
[extensions-unspecified]: http://ircbot.wl.linuxfoundation.org/eavesdrop/%23opencontainers/%23opencontainers.2016-07-27.log.html#t2016-07-27T16:37:56
[github-lists]: https://help.github.com/articles/basic-writing-and-formatting-syntax/#lists
[linguist-languages]: https://github.com/github/linguist/blob/master/lib/linguist/languages.yml
[linux-centric]: http://ircbot.wl.linuxfoundation.org/meetings/opencontainers/2016/opencontainers.2016-07-13-17.03.log.html#l-39
[listen-fds-description]: opencontainers/runc#231 (comment)
Subject: Systemd integration with runc, for on-demand socket
activation
[markdown-syntax]: http://daringfireball.net/projects/markdown/syntax#list
[mimic-runc]: http://ircbot.wl.linuxfoundation.org/meetings/opencontainers/2016/opencontainers.2016-07-13-17.03.log.html#l-15
[mrunal-ptrace]: http://ircbot.wl.linuxfoundation.org/eavesdrop/%23opencontainers/%23opencontainers.2016-07-13.log.html#t2016-07-13T18:58:54
[no-version]: http://ircbot.wl.linuxfoundation.org/meetings/opencontainers/2016/opencontainers.2016-07-20-21.03.log.html#l-75
[optional-fail-fast]: wking/oci-command-line-api@527f3c6#commitcomment-14835617
Subject: Use RFC 2119's keywords (MUST, MAY, ...)
[pep257-docstring]: https://www.python.org/dev/peps/pep-0257/#one-line-docstrings
[posix-kill]: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/kill.html
[punt-state-registry]: http://ircbot.wl.linuxfoundation.org/meetings/opencontainers/2015/opencontainers.2015-12-02-18.01.log.html#l-79
[rfc4627-s3]: https://tools.ietf.org/html/rfc4627#section-3
[rfc7158-aA]: https://tools.ietf.org/html/rfc7158#appendix-A
RFC 7518 is currently identical to 7519.
[rfc7159-s8.1]: https://tools.ietf.org/html/rfc7159#section-8.1
[runc-bundle-option]: opencontainers/runc#373
Subject: adding support for --bundle
[runc-config-via-stdin]: opencontainers/runc#202
Subject: Can runc take its configuration on stdin?
[runc-listen-fds]: opencontainers/runc#231
Subject: Systemd integration with runc, for on-demand socket
activation
[runc-required-config-file]: opencontainers/runc#310 (comment)
Subject: specifying a spec file on cmd line?
[runc-start-id]: opencontainers/runc#541
opencontainers/runc@a7278cad (Require container id as arg1,
2016-02-08, opencontainers/runc#541)
[runtime-spec-caller-api-agnostic]: opencontainers#113 (comment)
Subject: Add fd section for linux container process
[runtime-spec-required-config-file]: https://github.com/opencontainers/runtime-spec/pull/210/files#diff-8b310563f1c6f616aa98e6aeffc4d394R14
106ec2d (Cleanup bundle.md, 2015-10-02,
opencontainers#210)
[sd_listen_fds]: http://www.freedesktop.org/software/systemd/man/sd_listen_fds.html
[single-config-proposal]: https://groups.google.com/a/opencontainers.org/forum/#!topic/dev/0QbyJDM9fWY
Subject: Single, unified config file (i.e. rolling back specs#88)
Date: Wed, 4 Nov 2015 09:53:20 -0800
Message-ID: <20151104175320.GC24652@odin.tremily.us>
[solaris-kernel-state]: wking/oci-command-line-api#3 (comment)
Subject: Drop exec, pause, resume, and signal
[start-pr-bundle]: wking/oci-command-line-api#11
Subject: start: Change --config and --runtime to --bundle
[start-pr-state]: wking/oci-command-line-api#14
Subject: start: Add a --state option
[state-pr]: wking/oci-command-line-api#16
Subject: runtime: Add a 'state' command
[state-registry]: https://github.com/opencontainers/runtime-spec/pull/225/files#diff-b84a8d65d8ed53f4794cd2db7e8ea731R61
7117ede (Expand on the definition of our ops, 2015-10-13,
opencontainers#225)
[syntax-highlighting]: https://help.github.com/articles/github-flavored-markdown/#syntax-highlighting
[systemd-pid]: http://ircbot.wl.linuxfoundation.org/meetings/opencontainers/2016/opencontainers.2016-07-20-21.03.log.html#l-69
[util-linux-kill]: http://man7.org/linux/man-pages/man1/kill.1.html
[wikipedia-utf-8]: https://en.wikipedia.org/wiki/UTF-8
[wikipedia-posix-locale]: https://en.wikipedia.org/wiki/Locale#POSIX_platforms
[windows-singals]: https://groups.google.com/a/opencontainers.org/forum/#!topic/dev/PlGKu7QUwLE
Subject: Fwd: Windows support for OCI stop/signal/kill (runtime-spec#356)
Date: Thu, 26 May 2016 11:03:29 -0700
Message-ID: <20160526180329.GL17496@odin.tremily.us>
Signed-off-by: Julian Friedman <julz.friedman@uk.ibm.com>
Hopefully-Signed-off-by: Mike Brown <brownwm@us.ibm.com>
Signed-off-by: W. Trevor King <wking@tremily.us>
Reviewed-by: Jesse Butler <jeeves.butler@gmail.com>1 parent 2fc8875 commit 488dfe4
File tree
4 files changed
+191
-6
lines changed4 files changed
+191
-6
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
| 35 | + | |
35 | 36 | | |
36 | 37 | | |
37 | 38 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
| 19 | + | |
19 | 20 | | |
20 | 21 | | |
21 | 22 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | | - | |
4 | | - | |
5 | | - | |
6 | | - | |
7 | | - | |
8 | | - | |
9 | 3 | | |
10 | 4 | | |
11 | 5 | | |
| |||
0 commit comments