Skip to content

Commit

Permalink
ccon: Add '--socket PATH' to support a create/start split
Browse files Browse the repository at this point in the history
Along the lines of the create/start split we've been discussing for
the OCI [1,2].  Pull some common functionality into a libccon,
although I don't intend to make that a more than an internal helper.

Post-error cleanup in ccon-cli is pretty slim, since the process is
just about to die anyway.  I'll probably go back through and add
proper cleanup later.

serve_socket gets too deeply nested for my taste, but it's not quite
to the point where I'd pull out the request handling into a
handle_start_request helper or some such.  Mostly because I'd have to
either setup a new buf/iov/msg in the helper's stack or pass that all
through from serve_socket ;).

A few notes on the tests:

I don't have a stand-alone 'wait' on my system (it's built into most
shells [3]), but I've added an explicit check for it because POSIX
doesn't require it to be built in.

The waiting in the create/start tests is a bit awkward, but here's the
reasoning for the various steps:

* Put the socket in a 'sock' subdirectory so we don't have to mess
  with inotifywait's --exclude (when what we really want is an
  --include).  In most cases, the socket would be the first thing
  created in the test directory, but the process.host test will create
  a pivot-root.* before creating the socket.

* Clear a 'wait' file before launching the inotifywait/start subshell
  and append to it from that subshell so grep has something to look at
  (even if it racily starts looking before the subshell processes the
  inotifywait line).

* Block on a busy-wait grep until inotifywait sets up its watcher.
  This ensures we don't call ccon and create the socket before the
  watcher is looking.

  The zero-second sleep while we wait for inotifywait to setup its
  watcher is busy, but POSIX doesn't require 'sleep' to support
  non-integer times [4].

All of these issues could be abstracted out into an 'event' command
[5], but they're fine for a proof-of-concept create/start split.

[1]: https://groups.google.com/a/opencontainers.org/d/msg/dev/qWHoKs8Fsrk/k55FQrBzBgAJ
     Subject: Re: Splitting Start into create() and run()
     Date: Thu, 24 Mar 2016 15:04:14 -0700
     Message-ID: <20160324220414.GC23066@odin.tremily.us>
[2]: opencontainers/runtime-spec#384
     Subject: Split create and start
[3]: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap01.html#tag_17_06
[4]: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sleep.html
[5]: opencontainers/runtime-spec#508
     Subject: runtime: Add an 'event' operation for subscribing to
       pushes
  • Loading branch information
wking committed Feb 26, 2018
1 parent 1077e0d commit b634390
Show file tree
Hide file tree
Showing 12 changed files with 1,166 additions and 293 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
ccon
ccon-cli
*.o
10 changes: 5 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,16 @@ LDLIBS := $(shell pkg-config --libs-only-l jansson libcap-ng)
.PHONY: all clean fmt
.PRECIOUS: %.o

all: ccon
all: ccon ccon-cli

%.o: %.c
$(CC) $(CFLAGS) -c -o "$@" "$<"

ccon: %: %.o
$(CC) $(LDFLAGS) -o "$@" "$<" $(LDLIBS)
ccon ccon-cli: %: %.o libccon.o
$(CC) $(LDFLAGS) -o "$@" $^ $(LDLIBS)

clean:
rm -f *.o ccon
rm -f *.o ccon ccon-cli

fmt:
indent --ignore-profile --linux-style *.c
indent --ignore-profile --linux-style *.h *.c
87 changes: 85 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ less opinionated than [LXC][lxc.container.conf.5]).
## Table of contents

* [Lifecycle](#lifecycle)
* [Socket communication](#socket-communication)
* [Getting the container process's
PID](#getting-the-container-processs-pid)
* [Start request](#start-request)
* [Configuration](#configuration)
* [Version](#version)
* [Namespaces](#namespaces)
Expand Down Expand Up @@ -59,8 +63,11 @@ synchronize the container setup. Here's an outline of the lifecycle:
| | mounts filesystems |
| | ← sends namespaces-complete |
| runs pre-start hooks | blocks on exec-message |
| sends exec-message → | |
| | opens the local ptmx |
| binds to socket path | |
| sends connection socket → | |
| blocks on exec-process message | listens for process JSON |
| | ← sends exec-process message |
| removes socket path | opens the local ptmx |
| | ← sends pseudoterminal master |
| | bind mounts `/dev/console` |
| | ← sends pseudoterminal slave |
Expand All @@ -86,6 +93,77 @@ use [`nsenter`][nsenter.1] or a wrapping ccon invocation to join those
namespaces before the main ccon invocation creates the new mount
namespace.

## Socket communication

With `--socket=PATH`, ccon will bind a [`SOCK_SEQPACKET` Unix
socket][unix.7] to `PATH`. This path is created after namespace-setup
completes, so users can use its presence as a trigger for further
configuration (e.g. network setup) before [starting](#start-request)
the [user-specified code](#process). The path is removed after a
[start request](#start-request) is received or after the container
process exits, whichever comes first.

The [`ccon-cli`](ccon-cli.c) program distributed with this repository
is one client for the ccon socket.

### Getting the container process's PID

An [`SO_PEERCRED`][socket.7] request will return the container
process's PID [in the receiving process's PID
namespace][pid_namespaces.7]. The client can use this to look up the
container process in their local [`/proc`][proc.5]. This request may
be performed as many times as you like.

### Start request

The request is a single [`struct iovec`][recv.2] containing either a
leading null byte or process JSON. Sending a single null-byte message
will trigger the [**`process`**](#process) field present in the
original configuration, while non-empty strings will completely
override that field.

The response is a single [`struct iovec`][recv.2] containing either a
single null-byte message (for success) or an error message encoded in
[ASCII][ascii.7] ([RFC 1345][rfc1345.s5]). In this context, “success”
means “successfully received the start request”, because the container
process sends the response before actually executing the
[user-specified code](#process).

If you set [**`host`**](#host) in your process JSON, `ccon-cli` will
open the referenced path and pass the open file descriptor to the
container over the Unix socket.

### Example

In one shell, launch ccon and have it listen on a socket at
`/tmp/ccon-sock`:

```
$ ccon --socket /tmp/ccon-sock
```

In a second shell, get the container process's PID, but don't trigger
the user-specified code:

```
$ PID=$(ccon-cli --socket /tmp/ccon-sock --pid)
$ echo "${PID}"
2186
```

You can then perform additional configuration using that PID:

```
$ ip link set ccon-ex-veth1 netns "${PID}"
```

And when you're finished setting up the environment, you can trigger
the [user-specified code](#process):

```
$ ccon-cli --socket /tmp/ccon-sock --config-string '{"args": ["busybox", "sh"]}'
```

## Configuration

Ccon is similar to an [Open Container Iniative Runtime
Expand Down Expand Up @@ -858,6 +936,7 @@ be distributed under the GPLv3+.
[setgid.2]: http://man7.org/linux/man-pages/man2/setgid.2.html
[setuid.2]: http://man7.org/linux/man-pages/man2/setuid.2.html
[syscall.2]: http://man7.org/linux/man-pages/man2/syscall.2.html
[recv.2]: http://man7.org/linux/man-pages/man2/recv.2.html
[environ.3p]: https://www.kernel.org/pub/linux/docs/man-pages/man-pages-posix/
[exec.3]: http://man7.org/linux/man-pages/man3/exec.3.html
[getcwd.3]: http://man7.org/linux/man-pages/man3/getcwd.3.html
Expand All @@ -868,11 +947,14 @@ be distributed under the GPLv3+.
[pts.4]: http://man7.org/linux/man-pages/man4/pty.4.html
[filesystems.5]: http://man7.org/linux/man-pages/man5/filesystems.5.html
[lxc.container.conf.5]: https://linuxcontainers.org/lxc/manpages/man5/lxc.container.conf.5.html
[proc.5]: https://linuxcontainers.org/lxc/manpages/man5/proc.5.html
[ascii.7]: http://man7.org/linux/man-pages/man7/ascii.7.html
[capabilities.7]: http://man7.org/linux/man-pages/man7/capabilities.7.html
[namespaces.7]: http://man7.org/linux/man-pages/man7/namespaces.7.html
[pid_namespaces.7]: http://man7.org/linux/man-pages/man7/pid_namespaces.7.html
[pty.7]: http://man7.org/linux/man-pages/man7/pty.7.html
[signal.7]: http://man7.org/linux/man-pages/man7/signal.7.html
[socket.7]: http://man7.org/linux/man-pages/man7/socket.7.html
[unix.7]: http://man7.org/linux/man-pages/man7/unix.7.html
[user_namespaces.7]: http://man7.org/linux/man-pages/man7/user_namespaces.7.html
[mount.8]: http://man7.org/linux/man-pages/man8/pty.8.html
Expand All @@ -881,4 +963,5 @@ be distributed under the GPLv3+.
[cgroups]: https://www.kernel.org/doc/Documentation/cgroup-v1/cgroups.txt
[cgroups-unified]: https://www.kernel.org/doc/Documentation/cgroup-v2.txt
[devpts]: https://www.kernel.org/doc/Documentation/filesystems/devpts.txt
[rfc1345.s5]: https://tools.ietf.org/html/rfc1345#section-5
[sd_listen_fds]: http://www.freedesktop.org/software/systemd/man/sd_listen_fds.html
Loading

0 comments on commit b634390

Please sign in to comment.