[CRASH] Opensips 3.2.2 is crashing in TLS connection clean with wolf SSL. #2805
Description
Opensips is crashing while clearing the TLS connection on connection time out. We have observed this issue with few number of registrations. After service running for 2-3 days this crash is occurring.
Version details:
version: opensips 3.2.2 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, HP_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll, sigio_rt, select.
svn revision: 1:30M
main.c compiled on 11:32:41 Apr 6 2022 with gcc 8
Operating system:
Oracle 8
Crash back trace details:
#0 0x00007ff1057b8d7a in __strlen_sse2 () from /lib64/libc.so.6
#1 0x00007ff1057710af in vfprintf () from /lib64/libc.so.6
#2 0x00007ff105816228 in __vsyslog_chk () from /lib64/libc.so.6
#3 0x00007ff105816320 in syslog () from /lib64/libc.so.6
#4 0x000000000055502b in fm_free (fm=, p=0x7fee0ad72b28, file=, func=,
line=) at mem/f_malloc_dyn.h:237
#5 0x00007fedffc5f2d4 in _shm_free (file=0x7fedffe22146 "wolfssl.c", function=, line=136, ptr=0x7fee0ad72b28)
at ../../mem/shm_mem.h:448
#6 oss_free (ptr=0x7fee0ad72b28) at wolfssl.c:136
#7 0x00007fedffc7ced5 in wolfSSL_Free (ptr=0x7fee0ad72b28) at wolfcrypt/src/memory.c:200
#8 0x00007fedffddaded in FreeSuites (ssl=0x7fee0a8d4348) at src/internal.c:6402
#9 0x00007fedffddae9f in SSL_ResourceFree (ssl=0x7fee0a8d4348) at src/internal.c:6424
#10 0x00007fedffddbcaa in FreeSSL (ssl=0x7fee0a8d4348, heap=0x0) at src/internal.c:6910
#11 0x00007fedffc96653 in wolfSSL_free (ssl=0x7fee0a8d4348) at src/ssl.c:574
#12 0x00007fedffc70fb0 in _wolfssl_tls_conn_clean (c=0x7fee0ac97010, tls_dom=0x7fffeafc2a40) at wolfssl_conn_ops.c:296
#13 0x00007fee0078b946 in tls_conn_clean (c=, tls_dom=) at tls_mgm.c:1362
#14 0x00007fee00df19ee in proto_tls_conn_clean (c=0x7fee0ac97010) at proto_tls.c:443
#15 0x000000000063bdf4 in _tcpconn_rm (c=0x7fee0ac97010) at net/net_tcp.c:644
#16 0x000000000063bfa9 in tcpconn_destroy (tcpconn=0x7fee0ac97010) at net/net_tcp.c:915
#17 0x000000000063fe9e in handle_tcp_worker (tcp_c=, fd_i=) at net/net_tcp.c:1275
#18 0x0000000000640d37 in handle_io (fm=0x7fef057c7f78, idx=55, event_type=1) at net/net_tcp.c:1466
#19 0x0000000000641035 in io_wait_loop_epoll (repeat=, t=, h=)
at net/../io_wait_loop.h:304
#20 0x0000000000642285 in io_wait_loop_sigio_rt (h=, t=5) at net/../io_wait_loop.h:410
#21 tcp_main_server () at net/net_tcp.c:1643
#22 0x00000000006467fa in tcp_start_listener () at net/net_tcp.c:2062
#23 0x000000000041b177 in main_loop () at main.c:249
#24 main (argc=, argv=) at main.c:934
(gdb) f 7
#7 0x00007fedffc7ced5 in wolfSSL_Free (ptr=0x7fee0ad72b28) at wolfcrypt/src/memory.c:200
200 free_function(ptr);
(gdb) p ptr
$1 = (void *) 0x7fee0ad72b28
(gdb) p *ptr
Attempt to dereference a generic pointer.
(gdb) f 4
#4 0x000000000055502b in fm_free (fm=, p=0x7fee0ad72b28, file=, func=,
line=) at mem/f_malloc_dyn.h:237
237 check_double_free(p, f, fm);
(gdb) p p
$2 = (void *) 0x7fee0ad72b28
(gdb) p f
$3 = (struct fm_frag *) 0x7fee0ad72af8
(gdb) p fm
$4 =
(gdb) p f->file
$5 = 0x72634f6541593658 <error: Cannot access memory at address 0x72634f6541593658>