-
Notifications
You must be signed in to change notification settings - Fork 592
runtime: Add more steps to the lifecycle docs #207
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -12,27 +12,28 @@ Presently there are `Prestart`, `Poststart` and `Poststop`. | |
| Hooks allow one to run code before/after various lifecycle events of the container. | ||
| Hooks MUST be called in the listed order. | ||
| The state of the container is passed to the hooks over stdin, so the hooks could get the information they need to do their work. | ||
| All hooks execute in the host environment (e.g. the same namespace, cgroups, etc. that apply to the host process). | ||
|
|
||
| Hook paths are absolute and are executed from the host's filesystem. | ||
|
|
||
| ### Pre-start | ||
|
|
||
| The pre-start hooks are called after the container process is spawned, but before the user supplied command is executed. | ||
| The pre-start hooks are called [after the container process is spawned, but before the user supplied command is executed](runtime.md#typical-lifecycle). | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we need to mention
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. On Mon, Oct 12, 2015 at 12:47:00PM -0700, Vish Kannan wrote:
The container is complete when the pre-start hooks run. The container
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You are stating runC's behavior. Why do we have to require that behavior in the Spec?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. On Mon, Oct 12, 2015 at 01:20:20PM -0700, Vish Kannan wrote:
This commit is mostly about describing the runC approach so we have a That being said, I do think all runtimes will need to execute |
||
| They are called after the container namespaces are created on Linux, so they provide an opportunity to customize the container. | ||
| In Linux, for e.g., the network namespace could be configured in this hook. | ||
|
|
||
| If a hook returns a non-zero exit code, then an error including the exit code and the stderr is returned to the caller and the container is torn down. | ||
| If a hook returns a non-zero exit code, [then an error including the exit code and the stderr is returned to the caller and the container is torn down](runtime.md#typical-lifecycle). | ||
|
|
||
| ### Post-start | ||
|
|
||
| The post-start hooks are called after the user process is started. | ||
| The post-start hooks are called [after the user process is started](runtime.md#typical-lifecycle). | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How about
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. On Mon, Oct 12, 2015 at 12:48:09PM -0700, Vish Kannan wrote:
The container process (initially running runtime-supplied code) is Host process Container process start So there's a lot going on in the container process before it turns
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. On Mon, Oct 12, 2015 at 12:48:09PM -0700, Vish Kannan wrote:
Also “init process” implies a PID namespace, and those are optional
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. On Mon, Oct 12, 2015 at 12:56:13PM -0700, W. Trevor King wrote:
After launching and before blocking, the host process should also be
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I get the runC behavior @wking. One alternative to the runC behavior is that of unsharing namespaces (excepting pid), bind mounting the namespaces, and letting the hooks run before starting a new process in a new pid namespace (if requested by the user). Similarly, I'm wondering if the new semantics you are introducing are valid in other OSes.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. On Mon, Oct 12, 2015 at 03:48:06PM -0700, Vish Kannan wrote:
In the “user requested a PID namespace” case, that sounds like “the In either case, there is still a container process with
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not really. If you bind-mount the ns files, you don't need a process to On Mon, Oct 12, 2015 at 4:02 PM, W. Trevor King notifications@github.com
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. On Mon, Oct 12, 2015 at 04:03:30PM -0700, Vish Kannan wrote:
Oh, that will make my life a lot easier :). Is that: Host process Temporary child Launch a child process -> In that case, excepting the PID namespace from the pre-start On the other hand, we'll probably need to revisit the state JSON that
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. On Mon, Oct 12, 2015 at 04:14:38PM -0700, W. Trevor King wrote:
And it looks like it is :). Language supporting this in namespaces(7): Bind mounting (see mount(2)) one of the files in this directory to For example: tty1$ unshare --user --uts --map-root-user tty2# touch /tmp/uts-alice tty1$ exit tty2# ps aux | grep 25477 | grep -v grep |
||
| For example this hook can notify user that real process is spawned. | ||
|
|
||
| If a hook returns a non-zero exit code, then an error is logged and the remaining hooks are executed. | ||
|
|
||
| ### Post-stop | ||
|
|
||
| The post-stop hooks are called after the container process is stopped. | ||
| The post-stop hooks are called [after the container process is stopped](runtime.md#typical-lifecycle). | ||
| Cleanup or debugging could be performed in such a hook. | ||
| If a hook returns a non-zero exit code, then an error is logged and the remaining hooks are executed. | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -29,24 +29,60 @@ The root directory to the bundle is provided in the state so that consumers can | |
| } | ||
| ``` | ||
|
|
||
| ## Lifecycle | ||
| ## Typical lifecycle | ||
|
|
||
| A typical lifecyle progresses like this: | ||
|
|
||
| 1. There is no container | ||
| 2. A user tells the runtime to start a container and launch a process inside it | ||
| 3. The runtime [creates the container](#create) | ||
| 4. The runtime executes any [pre-start hooks](runtime-config.md#pre-start) | ||
| 5. The runtime [executes the container process](#start-process) | ||
| 6. The container process is running | ||
| 7. The runtime executes any [post-start hooks](runtime-config.md#post-start) | ||
| 8. A user tells the runtime to send a termination signal to the container process | ||
| 9. The runtime [sends a termination signal to the container process](#stop-process) | ||
| 10. The container process exits | ||
| 11. The runtime [terminates any other processes in the container](#stop-process) | ||
| 12. The runtime executes any [post-stop hooks](runtime-config.md#post-stop) | ||
| 13. The runtime [removes the container](#cleanup) | ||
|
|
||
| With steps 7 and 8, the user is explicitly stopping the container process (via the runtime), but it's also possible that the container process could exit for other reasons. | ||
| In that case we skip directly from 6 to [10](#stop-process), skipping any post-start hooks that hadn't been launched and terminating any in-progress post-start hook. | ||
|
|
||
| Failure in a pre-start hook or other setup task can cause a jump straight to [12](runtime-config.md#post-stop). | ||
|
|
||
| ### Create | ||
|
|
||
| Creates the container: file system, namespaces, cgroups, capabilities. | ||
| Create the container: file system, namespaces, cgroups, capabilities, etc. | ||
| The invoked process forks, with one branch that stays in the host namespace and another that enters the container. | ||
| The host process carries out all container setup actions, and continues running for the life of the container so it can perform teardown after the container process exits. | ||
| The container process changes users and drops privileges in preparation for the container process start. | ||
| At this point, the host process writes the [`state.json`](#state) file with the host-side version of the container-process's PID (the container process may be in a PID namespace). | ||
|
|
||
| ### Start (process) | ||
|
|
||
| Runs a process in a container. | ||
| Can be invoked several times. | ||
| After the pre-start hooks complete, the host process signals the container process to execute the runtime. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This sentence does not make sense.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. On Wed, Oct 07, 2015 at 11:40:34AM -0700, Michael Crosby wrote:
I haven't looked into the runC implementation here, but my guess was
The line you're quoting was supposed to sketch out (6) and (7). Can
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. On Wed, Oct 07, 2015 at 11:51:50AM -0700, W. Trevor King wrote:
It looks like libcontainer actually has a fairly detailed spec for how |
||
| The runtime execs the process defined in `config.json`'s [**`process`** attribute](config.md#process-configuration). | ||
| On Linux hosts, some information for this execution may come from outside the `config.json` and `runtime.json` specifications. | ||
| See the [Linux-specific notes for details](runtime-linux.md#file-descriptors). | ||
|
|
||
| ### Stop (process) | ||
|
|
||
| Not sure we need that from runc cli. | ||
| Process is killed from the outside. | ||
| Send a termination signal to the container process (can optionally send other signals to the container process, e.g. a kill signal). | ||
| When the process exits, the host process collects it's exit status to return as its own exit status. | ||
| If there are any remaining processes in the container's cgroup (and [we only support unified-hierarchies](runtime-config-linux.md#control-groups)), the host process kills and reaps them. | ||
|
|
||
| ### Cleanup | ||
|
|
||
| The host process removes the [`state.json`](#state) file and the container: unmounting file systems, removing namespaces, etc. | ||
| This is the inverse of create. | ||
| The host process then exits with the container processes's exit status. | ||
|
|
||
| This event needs to be captured by runc to run onstop event handlers. | ||
| ## Joining existing containers | ||
|
|
||
| ## Hooks | ||
| Joining an existing container looks just like the usual workflow, except that the container process [joins the target container](runtime-config-linux.md#control-groups) at the beginning of step 3. | ||
| It can then, depending on its configuration, continue to create an additional child cgroup underneath the one it joined. | ||
|
|
||
| See [runtime configuration for hooks](./runtime-config.md) | ||
| When exiting, the reaping logic in the [stop phase](#stop-process) is the same. | ||
| If the container process created a child cgroup, all other processes in that child cgroup are reaped, but no other processes in the joined cgroup (which the container process did not create) are reaped. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is incorrect. They are forked from the runtime process so they inherit from the runtime, not all host processes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On Wed, Oct 07, 2015 at 11:36:22AM -0700, Michael Crosby wrote:
There are at least two runtime processes here. There is a process
launched with (for example) ‘runC start’, which I've been referring to
as the “host process”. That process forks and the child ends up in
the container, eventually running the config.json process binary. I'd
been referring to the latter as the “container process”. Do you
prefer alternative names for those two processes?