runtime: Add more steps to the lifecycle docs

And explain how the handling differs when a new start-call joins an
existing container vs. creating a new container.  This is spun off
from the mailing list thread now that that's settled down [1].

This write-up is just about Linux containers, since I don't understand
the other systems well enough to do them justice.

[1]: https://groups.google.com/a/opencontainers.org/forum/#!topic/dev/frUXLljXy8Y
     Message-ID: <20150915034717.GO18018@odin.tremily.us>

Signed-off-by: W. Trevor King <wking@tremily.us>

Author: wking

runtime-config.md

@@ -18,11 +18,11 @@ Hook paths are absolute and are executed from the host's filesystem.
 
 ### Pre-start
 
-The pre-start hooks are called after the container process is spawned, but before the user supplied command is executed.
+The pre-start hooks are called [after the container process is spawned, but before the user supplied command is executed](runtime.md#typical-lifecycle).
 They are called after the container namespaces are created on Linux, so they provide an opportunity to customize the container.
 In Linux, for e.g., the network namespace could be configured in this hook.
 
-If a hook returns a non-zero exit code, then an error including the exit code and the stderr is returned to the caller and the container is torn down.
+If a hook returns a non-zero exit code, [then an error including the exit code and the stderr is returned to the caller and the container is torn down](runtime.md#typical-lifecycle).
 
 ### Post-start
 
@@ -33,7 +33,7 @@ If a hook returns a non-zero exit code, then an error is logged and the remainin
 
 ### Post-stop
 
-The post-stop hooks are called after the container process is stopped.
+The post-stop hooks are called [after the container process is stopped](runtime.md#typical-lifecycle).
 Cleanup or debugging could be performed in such a hook.
 If a hook returns a non-zero exit code, then an error is logged and the remaining hooks are executed.
 

runtime.md

@@ -29,24 +29,59 @@ The root directory to the bundle is provided in the state so that consumers can
 }
 ```
 
-## Lifecycle
+## Typical lifecycle
+
+A typical lifecyle progresses like this:
+
+1. There is no container
+2. A user tells the runtime to start a container and launch a process inside it
+3. The runtime [creates the container](#create)
+4. The runtime executes any [pre-start hooks](runtime-config.md#pre-start)
+5. The runtime [executes the container process](#start-process)
+6. The container process is running
+7. A user tells the runtime to send a termination signal to the container process
+8. The runtime [sends a termination signal to the container process](#stop-process)
+9. The container process exits
+10. The runtime [terminates any other processes in the container](#stop-process)
+11. The runtime executes any [post-stop hooks](runtime-config.md#post-stop)
+12. The runtime [removes the container](#cleanup)
+
+With steps 7 and 8, the user is explicitly stopping the container process (via the runtime), but it's also possible that the container process could exit for other reasons.
+In that case we skip directly from 6 to [10](#stop-process).
+
+Failure in a pre-start hook or other setup task can cause a jump straight to [11](runtime-config.md#post-stop).
 
 ### Create
 
-Creates the container: file system, namespaces, cgroups, capabilities.
+Create the container: file system, namespaces, cgroups, capabilities, etc.
+The invoked process forks, with one branch that stays in the host namespace and another that enters the container.
+The host process carries out all container setup actions, and continues running for the life of the container so it can perform teardown after the container process exits.
+The container process changes users and drops privileges in preparation for the container process start.
+At this point, the host process writes the [`state.json`](#state) file with the host-side version of the container-process's PID (the container process may be in a PID namespace).
 
 ### Start (process)
 
-Runs a process in a container.
-Can be invoked several times.
+After the pre-start hooks complete, the host process signals the container process to execute the runtime.
+The runtime execs the process defined in `config.json`'s [**`process`** attribute](config.md#process-configuration).
+On Linux hosts, some information for this execution may come from outside the `config.json` and `runtime.json` specifications.
+See the [Linux-specific notes for details](runtime-linux.md#file-descriptors).
 
 ### Stop (process)
 
-Not sure we need that from runc cli.
-Process is killed from the outside.
+Send a termination signal to the container process (can optionally send other signals to the container process, e.g. a kill signal).
+When the process exits, the host process collects it's exit status to return as its own exit status.
+If there are any remaining processes in the container's cgroup (and [we only support unified-hierarchies](runtime-config-linux.md#control-groups)), the host process kills and reaps them.
+
+### Cleanup
+
+The host process removes the [`state.json`](#state) file and the container: unmounting file systems, removing namespaces, etc.
+This is the inverse of create.
+The host process then exits with the container processes's exit status.
 
-This event needs to be captured by runc to run onstop event handlers.
+## Joining existing containers
 
-## Hooks
+Joining an existing container looks just like the usual workflow, except that the container process [joins the target container](runtime-config-linux.md#control-groups) at the beginning of step 3.
+It can then, depending on its configuration, continue to create an additional child cgroup underneath the one it joined.
 
-See [runtime configuration for hooks](./runtime-config.md)
+When exiting, the reaping logic in the [stop phase](#stop-process) is the same.
+If the container process created a child cgroup, all other processes in that child cgroup are reaped, but no other processes in the joined cgroup (which the container process did not create) are reaped.