Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bgp snmp socket issue #5099

Closed
wants to merge 2 commits into from
Closed

Bgp snmp socket issue #5099

wants to merge 2 commits into from

Conversation

sudhanshukumar22
Copy link
Contributor

Problem Description/Summary :
vtysh hangs on first try to enter after a reboot with BGP dynamic peers

Expected Behavior :
VTYSH should not hang

When we debug more into bgpd docker by doing gdb on its threads, we find the below thread of bgpd, which is causing the issue.
Thread 1 (Thread 0x7f1e1ec46d40 (LWP 47)):

#0 0x00007f1e1d762593 in recvfrom () from /lib/x86_64-linux-gnu/libpthread.so.0

#1 0x00007f1e1aadd09b in netsnmp_tcpbase_recv () from /usr/lib/x86_64-linux-gnu/libnetsnmp.so.30

#2 0x00007f1e1aad9617 in netsnmp_transport_recv () from /usr/lib/x86_64-linux-gnu/libnetsnmp.so.30

#3 0x00007f1e1aab2c07 in _sess_read () from /usr/lib/x86_64-linux-gnu/libnetsnmp.so.30

#4 0x00007f1e1aab3a29 in snmp_sess_read2 () from /usr/lib/x86_64-linux-gnu/libnetsnmp.so.30

#5 0x00007f1e1aab3a7b in snmp_read2 () from /usr/lib/x86_64-linux-gnu/libnetsnmp.so.30

#6 0x00007f1e1aab3acf in snmp_read () from /usr/lib/x86_64-linux-gnu/libnetsnmp.so.30

#7 0x00007f1e1b44d7ec in agentx_read (t=0x7fffa75f0080) at lib/agentx.c:63

#8 0x00007f1e1e7d6451 in thread_call (thread=0x7fffa75f0080) at lib/thread.c:1620

#9 0x00007f1e1e770699 in frr_run (master=0x559396ea60f0) at lib/libfrr.c:1011

#10 0x0000559395b4d953 in main (argc=5, argv=0x7fffa75f02b8) at bgpd/bgp_main.c:492

(gdb) bt

#0 0x00007f830c89d210 in __read_nocancel () from /lib/x86_64-linux-gnu/libpthread.so.0

#1 0x000056450e1e8238 in vtysh_client_run (vclient=0x56450e4a8b40 <vtysh_client+24768>, line=0x56450e21add0 "enable", callback=0x0, cbarg=0x0) at vtysh/vtysh.c:216

#2 0x000056450e1e8c6b in vtysh_client_run_all (head_client=0x56450e4a8b40 <vtysh_client+24768>, line=0x56450e21add0 "enable", continue_on_err=0, callback=0x0, cbarg=0x0) at vtysh/vtysh.c:356

#3 0x000056450e1e8ddb in vtysh_client_execute (head_client=0x56450e4a8b40 <vtysh_client+24768>, line=0x56450e21add0 "enable") at vtysh/vtysh.c:393

#4 0x000056450e1e9c82 in vtysh_execute_func (line=0x56450e21add0 "enable", pager=0) at vtysh/vtysh.c:598

#5 0x000056450e1e9dee in vtysh_execute_no_pager (line=0x56450e21add0 "enable") at vtysh/vtysh.c:619

#6 0x000056450e1f7d48 in vtysh_read_file (confp=0x56451000a9d0, top_cfg=1) at vtysh/vtysh_config.c:494

#7 0x000056450e1f7ef2 in vtysh_read_config (config_default_dir=0x56450e4edc20 <frr_config> "/etc/frr/frr.conf", top_cfg=1) at vtysh/vtysh_config.c:522

#8 0x000056450e1e5de4 in vtysh_apply_top_level_config () at vtysh/vtysh_main.c:301

#9 0x000056450e1e7842 in main (argc=2, argv=0x7ffc81e6f598, env=0x7ffc81e6f5b0) at vtysh/vtysh_main.c:692

The fix has been taken from the following link.
https://sourceforge.net/p/net-snmp/patches/1348/

Signed-off-by: Preetham Singh (preetham.singh@broadcom.com)

Description: The changes have been done to make the snmp socket
			 non-blocking before calling snmp_read()

Signed-off-by: Preetham Singh (preetham.singh@broadcom.com)
Description: The changes have been done to make the snmp socket
                         non-blocking before calling snmp_read()
Problem Description/Summary :
vtysh hangs on first try to enter after a reboot with BGP dynamic peers

Expected Behavior :
VTYSH should not hang

When we debug more into bgpd docker by doing gdb on its threads, we find the below thread of bgpd, which is causing the issue.
Thread 1 (Thread 0x7f1e1ec46d40 (LWP 47)):

(gdb) bt

The fix has been taken from the following link.
https://sourceforge.net/p/net-snmp/patches/1348/

Signed-off-by: Preetham Singh (preetham.singh@broadcom.com)
@LabN-CI
Copy link
Collaborator

LabN-CI commented Oct 3, 2019

💚 Basic BGPD CI results: SUCCESS, 0 tests failed

Results table
_ _
Result SUCCESS git merge/5099 aeef84a
Date 10/03/2019
Start 01:20:16
Finish 01:41:59
Run-Time 21:43
Total 1815
Pass 1815
Fail 0
Valgrind-Errors 0
Valgrind-Loss 0
Details vncregress-2019-10-03-01:20:16.txt
Log autoscript-2019-10-03-01:21:06.log.bz2
Memory 429 433 360

For details, please contact louberger

@NetDEF-CI
Copy link
Collaborator

Continuous Integration Result: FAILED

See below for issues.
CI System Testrun URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-9121/

This is a comment from an automated CI system.
For questions and feedback in regards to this CI system, please feel free to email
Martin Winter - mwinter (at) opensourcerouting.org.

Get source / Pull Request: Successful

Building Stage: Successful

Basic Tests: Failed

Topotest tests on Ubuntu 16.04 i386: Failed (click for details)

Topology Test Results are at https://ci1.netdef.org/browse/FRR-FRRPULLREQ-TOPOI386-9121/test

Topology Tests failed for Topotest tests on Ubuntu 16.04 i386:

RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
*** defaultIntf: warning: r1 has no interfaces
2019-10-03 07:54:11,281 ERROR: 'router_json_cmp' failed after 102.51 seconds
2019-10-03 07:54:11,284 ERROR: assert failed at "bgp_vrf_lite_ipv6_rtadv.test_bgp_vrf_lite_ipv6_rtadv/test_protocols_convergence": "r2" JSON output mismatches
assert json value is different (
  --- Expected value
  +++ Current value
  @@ -7 +7 @@
  -        "internalStatus": 32, 
  +        "internalStatus": 34, 
  @@ -21,0 +22 @@
  +        "uptime": "00:01:44", )
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
*** defaultIntf: warning: lm has no interfaces
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument

see full log at https://ci1.netdef.org/browse/FRR-FRRPULLREQ-9121/artifact/TOPOI386/ErrorLog/log_topotests.txt

Successful on other platforms
  • Fedora 29 rpm pkg check
  • IPv4 protocols on Ubuntu 14.04
  • Debian 9 deb pkg check
  • Ubuntu 12.04 deb pkg check
  • Static analyzer (clang)
  • IPv6 protocols on Ubuntu 14.04
  • Topology tests on Ubuntu 18.04 amd64
  • Ubuntu 16.04 deb pkg check
  • Debian 10 deb pkg check
  • CentOS 7 rpm pkg check
  • IPv4 ldp protocol on Ubuntu 16.04
  • Ubuntu 14.04 deb pkg check
  • Addresssanitizer topotest
  • Debian 8 deb pkg check
  • Topology tests on Ubuntu 16.04 amd64

Topology Tests memory analysis: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-9121/artifact/TOPOI386/MemoryLeaks/

CLANG Static Analyzer Summary

  • Github Pull Request 5099, comparing to Git base SHA 6d7b7e4

Fixed warnings:

  • Logic error: Dereference of null pointer in nhrpd.h, function notifier_call, line 79

New warnings:

Static Analysis warning summary compared to base:

  • Fixed warnings: 1
  • New warnings: 13

13 Static Analyzer issues remaining.

See details at
https://ci1.netdef.org/browse/FRR-FRRPULLREQ-9121/artifact/shared/static_analysis/index.html

@sudhanshukumar22
Copy link
Contributor Author

I have closed this PR and raised a new PR. Please refer #5134.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants