container init was failed to kill if signal comes before container init executed

[gnawux]

Let me describe the detail:

(1) init new container
        |
       \/
(2) fork() ----->(3b) fork again for namespace ----> (B) init stdio & exec process
        |
       \/
(3a) return success
        |
       \/
(A) next command (signal container with SIGKILL)

Some failed logs:

The (A) was shown (signal and return normally) in log before (B) (hyper_install_process_stdio ), and the container process was not killed.

Detailed logs:

fork and success:

SB[vm-QGyxKjgyXa] [CNL] hyper send type 8
SB[vm-QGyxKjgyXa] [CNL] prerequisite process pid 332
SB[vm-QGyxKjgyXa] [CNL] create child process pid=333 in the sandbox
SB[vm-QGyxKjgyXa] [CNL] hyper send type 333
SB[vm-QGyxKjgyXa] [CNL] hyper_init_event exec stdin event 0x867eb8, ops 0x61d5c0, fd 7
SB[vm-QGyxKjgyXa] [CNL] hyper_add_event add event fd 7, 0x61d5c0
SB[vm-QGyxKjgyXa] [CNL] hyper_init_event exec stdout event 0x867ef8, ops 0x61d5a0, fd 8
SB[vm-QGyxKjgyXa] [CNL] hyper_add_event add event fd 8, 0x61d5a0
SB[vm-QGyxKjgyXa] [CNL] hyper_init_event exec stderr event 0x867f38, ops 0x61d580, fd 11
SB[vm-QGyxKjgyXa] [CNL] hyper_add_event add event fd 11, 0x61d580
SB[vm-QGyxKjgyXa] [CNL] hyper_run_process process pid 333
SB[vm-QGyxKjgyXa] [CNL] hyper ctl append type 9, len 0
SB[vm-QGyxKjgyXa] [CNL] hyper_handle_event event EPOLLOUT, he 0x61d648, fd 3, 0x61d4c0

then, kill and success:

SB[vm-QGyxKjgyXa] [CNL] hyper_handle_event event EPOLLIN, he 0x61d648, fd 3, 0x61d4c0
SB[vm-QGyxKjgyXa] [CNL] hyper ctl append type 14, len 4
SB[vm-QGyxKjgyXa] [CNL] hyper_modify_event modify event fd 3, 0x61d648, event 8197
SB[vm-QGyxKjgyXa] [CNL] hyper_ctlfd_read: get length 116
SB[vm-QGyxKjgyXa] [CNL] hyper ctl append type 14, len 4
SB[vm-QGyxKjgyXa] [CNL] hyper_ctlmsg_handle, type 24, len 116
SB[vm-QGyxKjgyXa] [CNL] hyper ctl append type 9, len 0
SB[vm-QGyxKjgyXa] [CNL] hyper_handle_event event EPOLLOUT, he 0x61d648, fd 3, 0x61d4c0

type 24 is SIGNALPROCESS, and the command payload is

{"container":"d9f81367f4ad974fb2ba3f60f38917602d28914a15764f01c117be082aad9e39","process":"init","signal":9}

then, initialize container process:

SB[vm-QGyxKjgyXa] [CNL] pid 332 exit normally, status 0
SB[vm-QGyxKjgyXa] [CNL] hyper_install_process_stdio

after test timeout, the daemon and pod was killed (again):

(more than 9 minutes later, timeout)

SB[vm-QGyxKjgyXa] [CNL] hyper ctl append type 14, len 4
SB[vm-QGyxKjgyXa] [CNL] hyper_modify_event modify event fd 3, 0x61d648, event 8197
SB[vm-QGyxKjgyXa] [CNL] hyper_ctlfd_read: get length 117
SB[vm-QGyxKjgyXa] [CNL] hyper ctl append type 14, len 4
SB[vm-QGyxKjgyXa] [CNL] hyper_ctlmsg_handle, type 24, len 117
SB[vm-QGyxKjgyXa] [CNL] hyper ctl append type 9, len 0
SB[vm-QGyxKjgyXa] [CNL] hyper_handle_event event EPOLLOUT, he 0x61d648, fd 3, 0x61d4c0
SB[vm-QGyxKjgyXa] [CNL] hyper_handle_event event EPOLLHUP or EPOLLERR, he 0x867f38, fd 11, 10
SB[vm-QGyxKjgyXa] [CNL] stderr_hup, seq 1, id init
SB[vm-QGyxKjgyXa] [CNL] still have 3 user of exec
SB[vm-QGyxKjgyXa] [CNL] hyper_handle_event event EPOLLHUP or EPOLLERR, he 0x867ef8, fd 8, 10
SB[vm-QGyxKjgyXa] [CNL] stdout_hup, seq 1, id init
SB[vm-QGyxKjgyXa] [CNL] still have 2 user of exec
SB[vm-QGyxKjgyXa] [CNL] hyper_handle_event event EPOLLHUP or EPOLLERR, he 0x867eb8, fd 7, 8
SB[vm-QGyxKjgyXa] [CNL] stdin_hup, seq 1, id init
SB[vm-QGyxKjgyXa] [CNL] still have 1 user of exec
SB[vm-QGyxKjgyXa] [CNL] pid 333 exit by signal, status 15
SB[vm-QGyxKjgyXa] [CNL] hyper_handle_exec_exit exec exit pid 333, seq 1, container d9f81367f4ad974fb2ba3f60f38917602d28914a15764f01c117be082aad9e39
SB[vm-QGyxKjgyXa] [CNL] container init process 333
SB[vm-QGyxKjgyXa] [CNL] last user of exec exit, release

complete logs:

Failed: http://ci.hypercontainer.io:8080/job/hyperd-auto/288/consoleFull
Success: http://ci.hypercontainer.io:8080/job/hyperd-auto/295/consoleFull

[gnawux]

could we move the

			hyper_send_type(pidpipe, ret);

in hyper_enter_sandbox() to later position, such as just before execvp? I know it means we have to get the pid again in the child process, however, this may prevent exec other handlers, such as "siganl process" before the process was initialized well.

[bergwolf]

It seems the root cause for the test hang in hyperhq/hyperd#576 is because of #257 that kills container exec by killing the process group. However, exec->pid is the process id rather than its own pgid. We use fork() to create exec process so it inherits its parent's pgid. Then kill(-exec->pid) sends the signal to no one.

Tested a bit and I was wrong. Only the parent process sees the inherited pgid after fork() in case of new pidns...

[gnawux]

fixed in #286