Misbehaving processes aren't killed

[whymarrh]

When running a program that can/does correctly respond to SIGTERM, Concurrently will successfully terminate the process and exit when using -k/--kill-others:

$ ./node_modules/.bin/concurrently -k './test/programs/a' 'sleep 10'
[1] sleep 10 exited with code 0
--> Sending SIGTERM to other processes..
[0] GRACEFUL
[0] ./test/programs/a exited with code 0
$ echo $?
0

But when a child process does not exit on SIGTERM the process isn't terminated at all:

$ ./node_modules/.bin/concurrently -k './test/programs/a' 'sleep 10' './test/programs/b'
[1] sleep 10 exited with code 0
--> Sending SIGTERM to other processes..
[0] GRACEFUL
[2] IGNORING SIGNAL
[0] ./test/programs/a exited with code 0

Further SIGTERM and SIGINT signals sent to concurrently do nothing. Manually sending SIGKILLto the child process will allow concurrently to exit (with 1).

Example programs

test/programs/a (correctly responds to SIGTERM):

#!/usr/bin/env python -u

from __future__ import print_function
import signal
import sys

def graceful_exit(sig, frame):
    print('GRACEFUL')
    print('GRACEFUL', file=open("was-graceful.log", "wt"))
    sys.exit(0)

signal.signal(signal.SIGINT,  graceful_exit)
signal.signal(signal.SIGTERM, graceful_exit)
signal.pause()

test/programs/b (incorrectly ignores SIGTERM):

#!/usr/bin/env python -u

from __future__ import print_function
import signal
import sys
import time

def ignore_signal(sig, frame):
    print('IGNORING SIGNAL')
    while True:
        time.sleep(42)

signal.signal(signal.SIGINT,  ignore_signal)
signal.signal(signal.SIGTERM, ignore_signal)
signal.pause()

Possible solution

Based on a bit of research, I think a more correct way to kill child processes would be to send SIGTERM to the children (not the the whole trees, just immediate children) and after some timeout send SIGKILL to the tree. This is what a few init systems will do.

Related issues

• feat: add --sigkill option to kill other processes with sigkill inste… #99 suggests adding a flag to default to SIGKILL which seems dangerous. I think it would be better to always send SIGTERM first before sending SIGKILL. I would imagine the impetus for that flag is this same situation.
• Processes not terminating correctly (or at all?) #135 raises a similar solution: "Is it possible to set a timeout for concurrently, e.g. forcing child processes to stop after a set maximum period of waiting?"

[gustavohenke]

Heya, do you mind giving v4 a try?
It fixed a similar case of processes hanging (#105).

[whymarrh]

@gustavohenke thanks, I'll try to take a look this week

[whymarrh]

@gustavohenke I think v4 has the same behaviour exhibited in the OP:

$ ./node_modules/.bin/concurrently --version
4.1.1
$ ./node_modules/.bin/concurrently -k './test/programs/a' 'sleep 10'
[1] sleep 10 exited with code 0
--> Sending SIGTERM to other processes..
[0] GRACEFUL
[0] ./test/programs/a exited with code 0
$ ./node_modules/.bin/concurrently -k './test/programs/a' 'sleep 10' './test/programs/b'
[1] sleep 10 exited with code 0
--> Sending SIGTERM to other processes..
[0] GRACEFUL
[2] IGNORING SIGNAL
[0] ./test/programs/a exited with code 0
--> Sending SIGTERM to other processes..
[2] IGNORING SIGNAL

The last invocation of concurrently shown above hangs indefinitely. Further SIGTERM and SIGINT signals sent to concurrently do nothing. Manually sending SIGKILLto the child process will allow concurrently to exit.

[gustavohenke]

Cool. So it seems we really need a timeout (e.g. --kill-timeout) to wait for before sending SIGKILL.