Pim possible crash #8671

donaldsharp · 2021-05-14T13:55:31Z

see individual commits for data

polychaeta

Thanks for your contribution to FRR!

One of your commits has a missing or badly formatted Signed-off-by line; we can't accept your contribution until all of your commits have one

If you are a new contributor to FRR, please see our contributing guidelines.

After making changes, you do not need to create a new PR. You should perform an amend or interactive rebase followed by a force push.

blocking comments addressed

LabN-CI · 2021-05-14T14:26:23Z

Outdated results 💚

Basic BGPD CI results: SUCCESS, 0 tests failed

_	_
Result	SUCCESS git merge/8671 `b0bef71`
Date	05/14/2021
Start	10:00:45
Finish	10:26:19
Run-Time	25:34
Total	1815
Pass	1815
Fail	0
Valgrind-Errors
Valgrind-Loss
Details	vncregress-2021-05-14-10:00:45.txt
Log	autoscript-2021-05-14-10:02:02.log.bz2
Memory	515 491 430

For details, please contact louberger

NetDEF-CI · 2021-05-14T14:28:45Z

Continuous Integration Result: FAILED

See below for issues.
CI System Testrun URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-18991/

This is a comment from an automated CI system.
For questions and feedback in regards to this CI system, please feel free to email
Martin Winter - mwinter (at) opensourcerouting.org.

Get source / Pull Request: Successful

Building Stage: Failed

Debian 10 amd64 build: Failed (click for details)

Debian 10 amd64 build: Unknown Log URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-18991/artifact/DEB10BUILD/config.log/config.log.gz Debian 10 amd64 build: Unknown Log URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-18991/artifact/DEB10BUILD/ErrorLog/ Debian 10 amd64 build: config.status output from configure script can be found at https://ci1.netdef.org/browse/FRR-FRRPULLREQ-18991/artifact/DEB10BUILD/config.status/config.status Debian 10 amd64 build: Unknown Log URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-18991/artifact/DEB10BUILD/frr.xref.xz/frr.xref.xz Debian 10 amd64 build: No useful log found

Successful on other platforms/tests

Ubuntu 18.04 arm7 build
Ubuntu 18.04 ppc64le build
Debian 9 amd64 build
Ubuntu 20.04 amd64 build
Ubuntu 18.04 i386 build
Ubuntu 16.04 arm8 build
Ubuntu 16.04 i386 build
Fedora 29 amd64 build
Debian 11 amd64 build
CentOS 7 amd64 build
CentOS 8 amd64 build
Ubuntu 18.04 arm8 build
Ubuntu 16.04 arm7 build
FreeBSD 12 amd64 build
OpenBSD 6 amd64 build
NetBSD 8 amd64 build
Ubuntu 16.04 amd64 build
Ubuntu 18.04 amd64 build
FreeBSD 11 amd64 build

NetDEF-CI · 2021-05-14T19:39:49Z

Continuous Integration Result: FAILED

See below for issues.
CI System Testrun URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-18992/

This is a comment from an automated CI system.
For questions and feedback in regards to this CI system, please feel free to email
Martin Winter - mwinter (at) opensourcerouting.org.

Get source / Pull Request: Successful

Building Stage: Successful

Basic Tests: Failed

Topotests debian 10 amd64 part 9: Failed (click for details)

Topology Test Results are at https://ci1.netdef.org/browse/FRR-FRRPULLREQ-TOPO9DEB10AMD64-18992/test

Topology Tests failed for Topotests debian 10 amd64 part 9:

2021-05-14 14:36:43,515 WARNING: vtysh_cmd: failed to convert json output
2021-05-14 14:37:04,555 WARNING: vtysh_cmd: failed to convert json output
2021-05-14 14:47:09,323 ERROR: Traceback (most recent call last):
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO9DEB10AMD64/topotests/lib/common_config.py", line 1802, in create_interfaces_cfg
    tgen, c_router, interface_data, "interface_config", build=build
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO9DEB10AMD64/topotests/lib/common_config.py", line 344, in create_common_configuration
    load_config_to_router(tgen, router)
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO9DEB10AMD64/topotests/lib/common_config.py", line 605, in load_config_to_router
    raise InvalidCLIError("%s" % output)
InvalidCLIError: line 3: % Unknown command[16]: ip ospf hello-interval 65536 


2021-05-14 14:47:47,877 ERROR: Traceback (most recent call last):
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO9DEB10AMD64/topotests/lib/common_config.py", line 1802, in create_interfaces_cfg
    tgen, c_router, interface_data, "interface_config", build=build
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO9DEB10AMD64/topotests/lib/common_config.py", line 344, in create_common_configuration
    load_config_to_router(tgen, router)
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO9DEB10AMD64/topotests/lib/common_config.py", line 605, in load_config_to_router
    raise InvalidCLIError("%s" % output)
InvalidCLIError: line 3: % Unknown command[16]: ip ospf dead-interval 65536 


2021-05-14 14:52:23,958 ERROR: 'router_json_cmp' failed after 95.37 seconds
2021-05-14 14:52:23,961 ERROR: assert failed at "ospf_sr_topo1.test_ospf_sr_topo1/test_rib_ipv4_step4": "rt6" JSON output mismatches the expected result
assert Generated JSON diff error report:
  
  > $->10.0.8.0/24: d2 has the following element at index 0 which is not present in d1: 
  
  	{
  	    "distance": 110,
  	    "prefix": "10.0.8.0/24",
  	    "protocol": "ospf",
  	    "metric": 10,
  	    "nexthops": [
  	        {
  	            "active": true,
  	            "directlyConnected": true,
  	            "interfaceName": "eth-rt5"
  	        }
  	    ]
  	}
  
  	Closest match in d1 is at index 0 with the following errors: 
  
  	> $->10.0.8.0/24[0]->nexthops: d2 has the following element at index 0 which is not present in d1: 
  	
  		{
  		    "active": true,
  		    "directlyConnected": true,
  		    "interfaceName": "eth-rt5"
  		}
  	
  		Closest match in d1 is at index 0 with the following errors: 
  	
  		> $->10.0.8.0/24[0]->nexthops[0]: d2 has key 'active' which is not present in d1

see full log at https://ci1.netdef.org/browse/FRR-FRRPULLREQ-18992/artifact/TOPO9DEB10AMD64/ErrorLog/log_topotests.txt

Successful on other platforms/tests

Topotests Ubuntu 18.04 i386 part 0
Addresssanitizer topotests part 3
Topotests Ubuntu 18.04 i386 part 5
Addresssanitizer topotests part 1
Topotests Ubuntu 18.04 amd64 part 5
Addresssanitizer topotests part 0
IPv4 ldp protocol on Ubuntu 18.04
Topotests Ubuntu 18.04 arm8 part 6
Topotests Ubuntu 18.04 i386 part 9
Topotests Ubuntu 18.04 arm8 part 1
Topotests Ubuntu 18.04 amd64 part 3
Addresssanitizer topotests part 8
Topotests Ubuntu 18.04 amd64 part 4
Topotests Ubuntu 18.04 amd64 part 0
IPv6 protocols on Ubuntu 18.04
Topotests Ubuntu 18.04 arm8 part 4
Addresssanitizer topotests part 6
Topotests Ubuntu 18.04 arm8 part 9
Debian 10 deb pkg check
Topotests Ubuntu 18.04 amd64 part 7
IPv4 protocols on Ubuntu 18.04
Topotests debian 10 amd64 part 7
Fedora 29 rpm pkg check
Debian 9 deb pkg check
Topotests debian 10 amd64 part 8
Ubuntu 20.04 deb pkg check
Topotests Ubuntu 18.04 amd64 part 1
Addresssanitizer topotests part 2
Topotests Ubuntu 18.04 arm8 part 3
CentOS 7 rpm pkg check
Topotests Ubuntu 18.04 amd64 part 8
Topotests debian 10 amd64 part 1
Topotests Ubuntu 18.04 i386 part 7
Topotests Ubuntu 18.04 amd64 part 9
Topotests Ubuntu 18.04 i386 part 3
Topotests debian 10 amd64 part 5
Topotests Ubuntu 18.04 i386 part 2
Topotests debian 10 amd64 part 6
Topotests debian 10 amd64 part 0
Topotests Ubuntu 18.04 arm8 part 7
Addresssanitizer topotests part 9
Addresssanitizer topotests part 4
Topotests Ubuntu 18.04 arm8 part 2
Topotests Ubuntu 18.04 i386 part 8
Topotests debian 10 amd64 part 2
Topotests Ubuntu 18.04 i386 part 1
Topotests Ubuntu 18.04 i386 part 6
Topotests Ubuntu 18.04 arm8 part 8
Topotests Ubuntu 18.04 amd64 part 2
Addresssanitizer topotests part 7
Topotests Ubuntu 18.04 i386 part 4
Topotests debian 10 amd64 part 4
Topotests Ubuntu 18.04 amd64 part 6
Topotests Ubuntu 18.04 arm8 part 0
Topotests Ubuntu 18.04 arm8 part 5
Topotests debian 10 amd64 part 3
Static analyzer (clang)
Ubuntu 18.04 deb pkg check
Addresssanitizer topotests part 5
Ubuntu 16.04 deb pkg check

LabN-CI · 2021-05-18T21:06:56Z

Outdated results 💚

Basic BGPD CI results: SUCCESS, 0 tests failed

_	_
Result	SUCCESS git merge/8671 `03cb199`
Date	05/18/2021
Start	16:41:25
Finish	17:06:52
Run-Time	25:27
Total	1815
Pass	1815
Fail	0
Valgrind-Errors
Valgrind-Loss
Details	vncregress-2021-05-18-16:41:25.txt
Log	autoscript-2021-05-18-16:42:35.log.bz2
Memory	504 501 429

For details, please contact louberger

NetDEF-CI · 2021-05-19T02:13:18Z

Continuous Integration Result: FAILED

See below for issues.
CI System Testrun URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-19085/

This is a comment from an automated CI system.
For questions and feedback in regards to this CI system, please feel free to email
Martin Winter - mwinter (at) opensourcerouting.org.

Get source / Pull Request: Successful

Building Stage: Successful

Basic Tests: Failed

Topotests Ubuntu 18.04 i386 part 9: Failed (click for details)

Topology Test Results are at https://ci1.netdef.org/browse/FRR-FRRPULLREQ-TOPO9U18I386-19085/test

Topology Tests failed for Topotests Ubuntu 18.04 i386 part 9:

2021-05-18 21:30:30,461 WARNING: vtysh_cmd: failed to convert json output
2021-05-18 21:30:54,613 WARNING: vtysh_cmd: failed to convert json output
2021-05-18 21:34:54,692 ERROR: r3: zebra left a dead pidfile (pid=11863)
2021-05-18 21:42:55,375 ERROR: Traceback (most recent call last):
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO9U18I386/topotests/lib/common_config.py", line 1803, in create_interfaces_cfg
    tgen, c_router, interface_data, "interface_config", build=build
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO9U18I386/topotests/lib/common_config.py", line 344, in create_common_configuration
    load_config_to_router(tgen, router)
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO9U18I386/topotests/lib/common_config.py", line 605, in load_config_to_router
    raise InvalidCLIError("%s" % output)
InvalidCLIError: line 3: % Unknown command[16]: ip ospf hello-interval 65536 


2021-05-18 21:43:38,654 ERROR: Traceback (most recent call last):
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO9U18I386/topotests/lib/common_config.py", line 1803, in create_interfaces_cfg
    tgen, c_router, interface_data, "interface_config", build=build
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO9U18I386/topotests/lib/common_config.py", line 344, in create_common_configuration
    load_config_to_router(tgen, router)
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO9U18I386/topotests/lib/common_config.py", line 605, in load_config_to_router
    raise InvalidCLIError("%s" % output)
InvalidCLIError: line 3: % Unknown command[16]: ip ospf dead-interval 65536 


2021-05-18 21:49:38,298 ERROR: 'router_json_cmp' failed after 97.16 seconds
2021-05-18 21:49:38,300 ERROR: assert failed at "ospf_sr_topo1.test_ospf_sr_topo1/test_rib_ipv4_step4": "rt6" JSON output mismatches the expected result
assert Generated JSON diff error report:
  
  > $->10.0.7.0/24: d2 has the following element at index 0 which is not present in d1: 
  
  	{
  	    "distance": 110,
  	    "prefix": "10.0.7.0/24",
  	    "protocol": "ospf",
  	    "metric": 10,
  	    "nexthops": [
  	        {
  	            "active": true,
  	            "directlyConnected": true,
  	            "interfaceName": "eth-rt4"
  	        }
  	    ]
  	}
  
  	Closest match in d1 is at index 0 with the following errors: 
  
  	> $->10.0.7.0/24[0]->nexthops: d2 has the following element at index 0 which is not present in d1: 
  	
  		{
  		    "active": true,
  		    "directlyConnected": true,
  		    "interfaceName": "eth-rt4"
  		}
  	
  		Closest match in d1 is at index 0 with the following errors: 
  	
  		> $->10.0.7.0/24[0]->nexthops[0]: d2 has key 'active' which is not present in d1

see full log at https://ci1.netdef.org/browse/FRR-FRRPULLREQ-19085/artifact/TOPO9U18I386/ErrorLog/log_topotests.txt

Successful on other platforms/tests

Addresssanitizer topotests part 7
Topotests Ubuntu 18.04 arm8 part 4
Topotests Ubuntu 18.04 arm8 part 9
Topotests Ubuntu 18.04 amd64 part 7
Topotests Ubuntu 18.04 amd64 part 9
Static analyzer (clang)
Fedora 29 rpm pkg check
Topotests debian 10 amd64 part 9
Topotests Ubuntu 18.04 arm8 part 0
Topotests Ubuntu 18.04 arm8 part 5
Addresssanitizer topotests part 5
Addresssanitizer topotests part 4
Topotests Ubuntu 18.04 amd64 part 5
Topotests Ubuntu 18.04 amd64 part 1
Topotests Ubuntu 18.04 arm8 part 3
Topotests debian 10 amd64 part 6
Topotests Ubuntu 18.04 i386 part 2
CentOS 7 rpm pkg check
Topotests Ubuntu 18.04 i386 part 7
Addresssanitizer topotests part 0
Topotests debian 10 amd64 part 4
Topotests Ubuntu 18.04 arm8 part 1
Topotests Ubuntu 18.04 amd64 part 3
Topotests Ubuntu 18.04 i386 part 4
IPv6 protocols on Ubuntu 18.04
Topotests Ubuntu 18.04 amd64 part 4
Topotests Ubuntu 18.04 amd64 part 0
Addresssanitizer topotests part 1
Topotests debian 10 amd64 part 5
Topotests Ubuntu 18.04 i386 part 6
Topotests debian 10 amd64 part 2
Topotests Ubuntu 18.04 arm8 part 8
Topotests Ubuntu 18.04 i386 part 1
Addresssanitizer topotests part 6
Addresssanitizer topotests part 8
Debian 10 deb pkg check
Topotests Ubuntu 18.04 arm8 part 6
Topotests debian 10 amd64 part 7
IPv4 protocols on Ubuntu 18.04
Topotests Ubuntu 18.04 amd64 part 2
Topotests Ubuntu 18.04 amd64 part 6
Topotests debian 10 amd64 part 3
Ubuntu 18.04 deb pkg check
Ubuntu 20.04 deb pkg check
Ubuntu 16.04 deb pkg check
Topotests debian 10 amd64 part 8
Addresssanitizer topotests part 2
Topotests Ubuntu 18.04 i386 part 0
Topotests Ubuntu 18.04 i386 part 5
Addresssanitizer topotests part 3
Topotests debian 10 amd64 part 1
Topotests Ubuntu 18.04 amd64 part 8
IPv4 ldp protocol on Ubuntu 18.04
Debian 9 deb pkg check
Topotests Ubuntu 18.04 arm8 part 7
Addresssanitizer topotests part 9
Topotests debian 10 amd64 part 0
Topotests Ubuntu 18.04 arm8 part 2
Topotests Ubuntu 18.04 i386 part 8
Topotests Ubuntu 18.04 i386 part 3

When bind to a socket fails, let's give some color so we can understand and fix the issue. Signed-off-by: Donald Sharp <sharpd@nvidia.com>

When running pim on an interface and that interface has state and we move that interface into a different vrf there exists a call path where we have not created the pimreg device yet. Prevent a crash in this rare situation. Ticket: #2552763 Signed-off-by: Donald Sharp <sharpd@nvidia.com>

LabN-CI · 2021-05-19T19:21:58Z

Outdated results 🚧

Basic BGPD CI results: Partial FAILURE, 1 tests failed

_	_
Result	SUCCESS git merge/8671 `8c70a46`
Date	05/19/2021
Start	14:56:23
Finish	15:21:54
Run-Time	25:31
Total	1815
Pass	1814
Fail	1
Valgrind-Errors
Valgrind-Loss
Details	vncregress-2021-05-19-14:56:23.txt
Log	autoscript-2021-05-19-14:57:37.log.bz2
Memory	492 515 426

For details, please contact louberger

NetDEF-CI · 2021-05-20T02:00:07Z

Continuous Integration Result: SUCCESSFUL

Congratulations, this patch passed basic tests

Tested-by: NetDEF / OpenSourceRouting.org CI System

CI System Testrun URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-19117/

This is a comment from an automated CI system.
For questions and feedback in regards to this CI system, please feel free to email
Martin Winter - mwinter (at) opensourcerouting.org.

Jafaral · 2021-05-20T04:37:33Z

Looking good @donaldsharp. what is the failure @louberger ?

LabN-CI · 2021-05-20T11:26:27Z

💚 Basic BGPD CI results: SUCCESS, 0 tests failed

Results table

_	_
Result	SUCCESS git merge/8671 `8c70a46`
Date	05/20/2021
Start	07:00:46
Finish	07:26:23
Run-Time	25:37
Total	1815
Pass	1815
Fail	0
Valgrind-Errors
Valgrind-Loss
Details	vncregress-2021-05-20-07:00:46.txt
Log	autoscript-2021-05-20-07:01:59.log.bz2
Memory	493 477 428

For details, please contact louberger

polychaeta added the pim label May 14, 2021

polychaeta previously requested changes May 14, 2021

View reviewed changes

donaldsharp force-pushed the pim_possible_crash branch from 0aea5b8 to b0bef71 Compare May 14, 2021 13:56

donaldsharp force-pushed the pim_possible_crash branch from b0bef71 to 03cb199 Compare May 18, 2021 19:36

donaldsharp added 2 commits May 19, 2021 14:48

pimd: When bind fails give some extra data

262649b

When bind to a socket fails, let's give some color so we can understand and fix the issue. Signed-off-by: Donald Sharp <sharpd@nvidia.com>

donaldsharp force-pushed the pim_possible_crash branch from 03cb199 to 8c70a46 Compare May 19, 2021 18:48

Jafaral merged commit 10d52ac into FRRouting:master May 21, 2021

idryzhov added the backport label May 21, 2021

donaldsharp deleted the pim_possible_crash branch June 23, 2021 11:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pim possible crash #8671

Pim possible crash #8671

donaldsharp commented May 14, 2021

polychaeta left a comment

LabN-CI commented May 14, 2021 •

edited

Loading

Basic BGPD CI results: SUCCESS, 0 tests failed

NetDEF-CI commented May 14, 2021 •

edited

Loading

Continuous Integration Result: FAILED

Get source / Pull Request: Successful

Building Stage: Failed

NetDEF-CI commented May 14, 2021 •

edited

Loading

Continuous Integration Result: FAILED

Get source / Pull Request: Successful

Building Stage: Successful

Basic Tests: Failed

LabN-CI commented May 18, 2021 •

edited

Loading

Basic BGPD CI results: SUCCESS, 0 tests failed

NetDEF-CI commented May 19, 2021 •

edited

Loading

Continuous Integration Result: FAILED

Get source / Pull Request: Successful

Building Stage: Successful

Basic Tests: Failed

LabN-CI commented May 19, 2021 •

edited

Loading

Basic BGPD CI results: Partial FAILURE, 1 tests failed

NetDEF-CI commented May 20, 2021

Jafaral commented May 20, 2021

LabN-CI commented May 20, 2021

Pim possible crash #8671

Pim possible crash #8671

Conversation

donaldsharp commented May 14, 2021

polychaeta left a comment

Choose a reason for hiding this comment

LabN-CI commented May 14, 2021 • edited Loading

Basic BGPD CI results: SUCCESS, 0 tests failed

NetDEF-CI commented May 14, 2021 • edited Loading

Continuous Integration Result: FAILED

Get source / Pull Request: Successful

Building Stage: Failed

NetDEF-CI commented May 14, 2021 • edited Loading

Continuous Integration Result: FAILED

Get source / Pull Request: Successful

Building Stage: Successful

Basic Tests: Failed

LabN-CI commented May 18, 2021 • edited Loading

Basic BGPD CI results: SUCCESS, 0 tests failed

NetDEF-CI commented May 19, 2021 • edited Loading

Continuous Integration Result: FAILED

Get source / Pull Request: Successful

Building Stage: Successful

Basic Tests: Failed

LabN-CI commented May 19, 2021 • edited Loading

Basic BGPD CI results: Partial FAILURE, 1 tests failed

NetDEF-CI commented May 20, 2021

Continuous Integration Result: SUCCESSFUL

Jafaral commented May 20, 2021

LabN-CI commented May 20, 2021

💚 Basic BGPD CI results: SUCCESS, 0 tests failed

LabN-CI commented May 14, 2021 •

edited

Loading

NetDEF-CI commented May 14, 2021 •

edited

Loading

NetDEF-CI commented May 14, 2021 •

edited

Loading

LabN-CI commented May 18, 2021 •

edited

Loading

NetDEF-CI commented May 19, 2021 •

edited

Loading

LabN-CI commented May 19, 2021 •

edited

Loading