Move poststart hook from runc create to runc start

[kolyshkin]

The runtime-spec 1 currently says:

6. Runtime's start command is invoked with the unique identifier of
   the container.
7. The startContainer hooks MUST be invoked by the runtime. If any
   startContainer hook fails, the runtime MUST generate an error, stop
   the container, and continue the lifecycle at step 12.
8. The runtime MUST run the user-specified program, as specified by
   process.
9. The poststart hooks MUST be invoked by the runtime. If any
   poststart hook fails, the runtime MUST generate an error, stop the
   container, and continue the lifecycle at step 12.
   ...
10. Runtime's delete command is invoked with the unique identifier of
    the container.
11. The container MUST be destroyed by undoing the steps performed
    during create phase (step 2).
12. The poststop hooks MUST be invoked by the runtime. If any poststop
    hook fails, the runtime MUST log a warning, but the remaining hooks
    and lifecycle continue as if the hook had succeeded.

Currently, we do 9 before 8 (heck, even before 6), which is clearly
against the spec and results in issues like the one described in 2.

Let's move running poststart hook to after the user-specified process
has started.

NOTE this patch only fixes the order and does not implement removing
the container when the poststart hook failed (as this part of the spec
is controversial -- destroy et al and should probably be, and currently
are, part of "runc delete").

Fixes: #5182
Fixes: #4347
Closes: #4348

[kolyshkin]

@ningmingxiao @eriksjolund PTAL

[rata]

We will have to revert this if it breaks anyone. But I don't think hooks are so used (they were mostly unexported to upper layers), so hopefully we can do this and no one notices?

[cyphar]

I thought we decided to just keep the broken poststart hook as is and push people to use the new createRuntime and friends?

EDIT: I think the issue is that opencontainers/runtime-spec#1169 missed a few changes when changing the explanation of prestart to match what runc does in practice.

[cyphar]

I suspect the issue is we just need to update the lifecycle doc in the spec, since the description of prestart says:

The prestart hooks MUST be called as part of the create operation after the runtime environment has been created (according to the configuration in config.json) but before the pivot_root or any equivalent operation has been executed.

[kolyshkin]

@cyphar this one is about poststart only. Only prestart is deprecated in the spec (in favor of createContainer and friends).

Currently runc implements poststart hook as part of runc create, while both crun and youki implement it as part of start (i.e. per spec).

I feel we need to fix the discrepancy in one way or another.

[eriksjolund]

@ningmingxiao @eriksjolund PTAL

@kolyshkin I built runc from your branch poststart (git commit 2d88d2e).
Then I tried the reproducer in #5182 .
The file /tmp/poststop-created was created.
In other words, it seems this PR fixes #5182 .

[rata]

@kad this is the PR we were talking yesterday, can you please take a look?

I'd like to the @kad input before we merge, hooks are very much used by GPUs.

[ningmingxiao]

can you add Co-authored add me @kolyshkin

[kad]

/cc @elezar : Evan, would it break any of the usages in nvidia container toolkit?

[elezar]

The NVIDIA tooling uses only createContainer hooks in the CDI-based approach and the precreate hook in the legacy stack. Changing poststart behaviour should not affect this.

[halaney]

We use a good bit of OCI hooks, but don't use this one (anymore, and digging thru the history it seems that's actually due to this bug I think).

This seems to make sense to me and matches the spec as I read it.

[kolyshkin]

can you add Co-authored add me @kolyshkin

I haven't actually used any of your code here (although I've reviewed it). In any case, thank you for your contribution! I have acknowledged it by adding the Reported-by: tag to the last commit, hope it's all right.

[rata]

I'm a little scared of this change, does this enables something that was not possible before?

But if the consensus is to do the change, as most big users seem are fine with this, I'm fine. We can always revert. LGTM