Don't kill plugin process with signal SIGKILL

[zhanhb]

What version of shadowsocks-libev are you using?

shadowsocks-libev 3.2.0(installed through dnf)

What operating system are you using?

Linux 2.6.32-042stab127.2 # 1 SMP Thu Jan 4 16:41:44 MSK 2018 x86_64 x86_64 x86_64 GNU/Linux
NAME="CentOS Linux"
VERSION="7 (Core)"

What did you do?

I wrote a plugin wrapper to support SIP003 protocol with a plugin kcptun-sip003-wrapper. So ss-server will start my shell and the shell will start the plugin binary. Then I execute the command killall ss-server

What did you expect to see?

ss-server can kill all the child process or ss-server not to kill my wrpper process with signal SIGKILL.

What did you see instead?

Process ss-server got killed and leave the kcptun process still alive.

What is your config in detail (with all sensitive info masked)?

ss-server --plugin kcptun --plugin-opts 'key=some_key_here'

Here is the command I executed. kcptun is a shell script in the repository mentioned above. And binary file server_linux_amd64 in the same directory.

I searched the source of this repository, and found this line

shadowsocks-libev/src/plugin.c

Line 334 in 5e65dc0

cork_subprocess_abort(sub);

IMO, maybe something should changed here.

I have the control of signal, but When my wrapper process got signal SIGKILL, it has no chance to clean child process.

https://libcork.readthedocs.io/en/0.14.0/subprocess/#c.cork_subprocess_wait

[madeye]

I cannot reproduce your issue locally with any official SIP003 plugin, but it's better to wait for subprocess here.

Please check if your wrapper can handle SIGTERM well, as we didn't SIGKILL the subprocess.

[zhanhb]

I'm sure my script can handle signal SIGTERM, here is the log I execute my script directly.

[zhanhb@local ~]$ kcptun &
[1] 8478
[zhanhb@local ~]$ 2019/06/19 09:44:36 version: 20190611
2019/06/19 09:44:36 initiating key derivation
2019/06/19 09:44:36 listening on: [::]:29900
2019/06/19 09:44:36 target: 127.0.0.1:12948
2019/06/19 09:44:36 encryption: aes
... log of kcptun

[zhanhb@local ~]$ ps -ef | grep kcptun
zhanhb    8478  8388  0 09:44 pts/1    00:00:00 /bin/sh /usr/local/bin/kcptun
zhanhb    8494  8478  0 09:44 pts/1    00:00:00 /usr/local/bin/kcptun-server --listen :29900 --target 127.0.0.1:12948
zhanhb    8501  8388  0 09:44 pts/1    00:00:00 grep --color=auto kcptun
[zhanhb@local ~]$ kill -TERM 8478
[zhanhb@local ~]$ 
[1]+  Exit 143                kcptun
[zhanhb@local ~]$ ps -ef | grep kcptun
zhanhb    8503  8388  0 09:45 pts/1    00:00:00 grep --color=auto kcptun
[zhanhb@local ~]$ 

[zhanhb]

Here's the log I executed with ss-server

[zhanhb@local ~]$ ss-server -s 0.0.0.0 -p 1234 -m aes-256-cfb -k 1234 --plugin kcptun --plugin-opts 'key=123' &
[1] 8738
[zhanhb@local ~]$  2019-06-19 10:04:19 INFO: plugin "kcptun" enabled
 2019-06-19 10:04:19 INFO: initializing ciphers... aes-256-cfb
 2019-06-19 10:04:19 INFO: tcp server listening at 127.0.0.1:57570
2019/06/19 10:04:19 version: 20190611
2019/06/19 10:04:19 initiating key derivation
2019/06/19 10:04:19 listening on: [::]:1234
2019/06/19 10:04:19 target: 127.0.0.1:57570
...

[zhanhb@local ~]$ ps -ef|cat
UID        PID  PPID  C STIME TTY          TIME CMD
zhanhb    8738  8388  0 10:04 pts/1    00:00:00 ss-server -s 0.0.0.0 -p 1234 -m aes-256-cfb -k 1234 --plugin kcptun --plugin-opts key=123
zhanhb    8739  8738  0 10:04 pts/1    00:00:00 /bin/sh /usr/local/bin/kcptun
zhanhb    8755  8739  0 10:04 pts/1    00:00:00 /usr/local/bin/kcptun-server --listen 0.0.0.0:1234 --target 127.0.0.1:57570 --key 123
zhanhb    8761  8388  0 10:04 pts/1    00:00:00 ps -ef
zhanhb    8762  8388  0 10:04 pts/1    00:00:00 cat
[zhanhb@local ~]$ killall ss-server
[1]+  Done                    ss-server -s 0.0.0.0 -p 1234 -m aes-256-cfb -k 1234 --plugin kcptun --plugin-opts 'key=123'
[zhanhb@local ~]$ ps -ef|grep kcptun
zhanhb    8755     1  0 10:04 pts/1    00:00:00 /usr/local/bin/kcptun-server --listen 0.0.0.0:1234 --target 127.0.0.1:57570 --key 123
zhanhb    8767  8388  0 10:04 pts/1    00:00:00 grep --color=auto kcptun

[madeye]

To verify if it's an issue of shadowsocks-libev, please try any other plugin first.

For example, https://github.com/shadowsocks/kcptun

[madeye]

Also, please try the latest version of shadowsocks-libev.

[zhanhb]

This wont happen with other plugins, for each of them has only one child plugin process. But with my wrapper, there are two child plugin processes. I don't think it will solve my problem to do with latest version shadowsocks-libev.

[madeye]

If so, you need to handle your child processes correctly.

It means handle the SIGTERM signal and kill the child processes

Subprocess waiting is also added in the latest commit, you can compile the latest code and try again.

[zhanhb]

Unfortunately, the problem is still there when I execute the binary compiled with the source code of the master branch, latest commit id: 37ca683.

[zhanhb@local shadowsocks-libev]$ ss-server --version
 2019-06-19 10:43:43 ERROR: Unrecognized option: (null)

shadowsocks-libev 3.3.0
...

[madeye]

Unfortunately, I think you have to figure out this by yourself.

Any child process of the plugin is not controlled by shadowsocks-libev.

BTW, why would you write a wrapper of kcptun, given we already have an official plugin there.

[zhanhb]

I just want to upgrade to the latest version of kcptun, and make it easy to manager start scripts.

[icpz]

@madeye

I've noticed that libcork waits for child with NOHANG flag immediately after signal the child here.

This will cause waitpid returns zero and thus send a SIGKILL to child in most cases, because the child may have no time to handle the SIGTERM yet.

For a single process sip003 plugin, everything behaves ok but the plugin always has no time to do its own cleanup stuff. Is this behavior by design? Maybe we can add a timeout and then wait with NOHANG flag to see if the plugin hangs.

[madeye]

@lcdtyph LGTM, what about a pull request?