shutdown sequence while using zmq_socket_monitor()

[maxkozlovsky]

I have tried to use zmq_socket_monitor() in my application and encountered some problems in shutdown phase trying to close sockets and terminate zmq context. The problems ranged from hangs in zmq_ctx_term() to crashes with the stack trace included below. It seems that there is some race condition with closing a socket connected to socket monitor. I don't think I can come up with a small test case that demonstrates the problem, but I can reproduce it fairly quickly (on the order of few minutes) with my tests which involve the whole application.

I have been able to work around the problem by unregistering the monitor by calling zmq_socket_monitor() with nullptr endpoint address before starting to close the sockets. The man page for zmq_socket_monitor() does not mention the unregister option and the example on the man page does not include any monitor unregister calls.

This is on Linux Centos 7 with zmq 4.1.5.

(gdb) where
#0  0x00007f6ab644f5f7 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007f6ab6450ce8 in __GI_abort () at abort.c:90
#2  0x00007f6ab7d0b769 in zmq::zmq_abort (errmsg_=errmsg_@entry=0x7f6ab7d3f410 "pfd.revents & POLLIN") at src/err.cpp:83
#3  0x00007f6ab7d231d9 in zmq::signaler_t::wait (this=this@entry=0x17f2540, timeout_=timeout_@entry=-1) at src/signaler.cpp:239
#4  0x00007f6ab7d0ef20 in zmq::mailbox_t::recv (this=this@entry=0x17f24e0, cmd_=cmd_@entry=0x7f6a467fb000, timeout_=timeout_@entry=-1)
at src/mailbox.cpp:80
#5  0x00007f6ab7d24226 in zmq::socket_base_t::process_commands (this=this@entry=0x17f2120, timeout_=timeout_@entry=-1,
throttle_=throttle_@entry=false) at src/socket_base.cpp:1017
#6  0x00007f6ab7d2460f in zmq::socket_base_t::send (this=this@entry=0x17f2120, msg_=msg_@entry=0x7f6a467fb0b0, flags_=flags_@entry=2)
at src/socket_base.cpp:863
#7  0x00007f6ab7d3a0dc in s_sendmsg (s_=0x17f2120, msg_=0x7f6a467fb0b0, flags_=2) at src/zmq.cpp:346
#8  0x00007f6ab7d249c3 in zmq::socket_base_t::monitor_event (this=0x17f0080, event_=event_@entry=128, value_=120,
addr_="tcp://127.0.0.1:5557") at src/socket_base.cpp:1357
#9  0x00007f6ab7d24b0d in zmq::socket_base_t::event_closed (this=<optimized out>, addr_="tcp://127.0.0.1:5557", fd_=<optimized out>)
at src/socket_base.cpp:1332
#10 0x00007f6ab7d34396 in zmq::tcp_listener_t::close (this=this@entry=0x17f08e0) at src/tcp_listener.cpp:137
#11 0x00007f6ab7d34a53 in zmq::tcp_listener_t::process_term (this=0x17f08e0, linger_=0) at src/tcp_listener.cpp:86
#12 0x00007f6ab7d0c3ac in zmq::io_thread_t::in_event (this=0x17ec650) at src/io_thread.cpp:83
#13 0x00007f6ab7d0b3ee in zmq::epoll_t::loop (this=0x17ec8d0) at src/epoll.cpp:176
#14 0x00007f6ab7d34dc0 in thread_routine (arg_=0x17ec950) at src/thread.cpp:96
#15 0x00007f6ab7ad8dc5 in start_thread (arg=0x7f6a467fc700) at pthread_create.c:308
#16 0x00007f6ab6510ced in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

[volmart]

Have same issue. Is there any correct shutdown process? In my case I have monitoring socket for dealer and router sockets

[bluca]

I think this might be similar to #2158 and fixed by #2159
Could you please try again from the latest head of libzmq master and report back?

[somdoron]

I remember ypu must close the monitoring before closing the socket (with
calling null, we might need to add an api method to do this, instead of
this hack), might be fixed with latest fix.

On Oct 23, 2016 17:10, "Luca Boccassi" notifications@github.com wrote:

I think this might be similar to #2158
#2158 and fixed by #2159
#2159
Could you please try again from the latest head of libzmq master and
report back?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#2083 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AClv9lh518bcxyVXR8xsBZXuEB83gmlVks5q22ragaJpZM4Jk4EO
.

[stale]

This issue has been automatically marked as stale because it has not had activity for 365 days. It will be closed if no further activity occurs within 56 days. Thank you for your contributions.