Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tests: Disable Redist test with call in route-map as it never worked #8873

Closed
wants to merge 1 commit into from
Closed

tests: Disable Redist test with call in route-map as it never worked #8873

wants to merge 1 commit into from

Conversation

mwinter-osr
Copy link
Member

Disabling OSPF redistribution test with a route-map which uses call function.

The test never worked correctly. It used to pass as it ran too fast and didn't verify if the checked routes were withdrawn first (previous test used same test routes), but started to fail on CI on arm platform.

Test needs to be fixed after Issue #8872 is fixed, but done the correct way. See comment in commit

Signed-off-by: Martin Winter mwinter@opensourcerouting.org

@mwinter-osr
tests: Disable Redist test with call in route-map as it never worked
d81d0ad 
Signed-off-by: Martin Winter <mwinter@opensourcerouting.org>
@polychaeta polychaeta added the tests Topotests, make check, etc label Jun 17, 2021
donaldsharp
donaldsharp requested changes Jun 17, 2021
View reviewed changes
Copy link
Member

@donaldsharp donaldsharp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NAK we are not disablingtests. We fix them

@LabN-CI
Copy link
Collaborator

LabN-CI commented Jun 17, 2021

💚 Basic BGPD CI results: SUCCESS, 0 tests failed

Results table
_ _
Result SUCCESS git merge/8873 d81d0ad
Date 06/17/2021
Start 12:11:34
Finish 12:37:03
Run-Time 25:29
Total 1815
Pass 1815
Fail 0
Valgrind-Errors
Valgrind-Loss
Details vncregress-2021-06-17-12:11:34.txt
Log autoscript-2021-06-17-12:12:46.log.bz2
Memory 502 516 419

For details, please contact louberger

@qlyoung
Copy link
Member

qlyoung commented Jun 17, 2021

I think there are some circumstances when disabling tests is called for. We discussed this in last week's meeting. We should probably repeat the discussion next meeting.

@NetDEF-CI
Copy link
Collaborator

Continuous Integration Result: FAILED

Test incomplete. See below for issues.
CI System Testrun URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-19673/

This is a comment from an automated CI system.
For questions and feedback in regards to this CI system, please feel free to email
Martin Winter - mwinter (at) opensourcerouting.org.

Get source / Pull Request: Successful

Building Stage: Successful

Basic Tests: Incomplete

Topotests Ubuntu 18.04 arm8 part 2: Incomplete (check logs for details)
Topotests Ubuntu 18.04 arm8 part 2: Incomplete (check logs for details)
Topotests Ubuntu 18.04 arm8 part 8: Incomplete (check logs for details)
Topotests Ubuntu 18.04 i386 part 1: Failed (click for details)

Topology Test Results are at https://ci1.netdef.org/browse/FRR-FRRPULLREQ-TOPO1U18I386-19673/test

Topology Tests failed for Topotests Ubuntu 18.04 i386 part 1:

2021-06-17 18:58:48,808 ERROR: 'router_json_cmp' failed after 135.64 seconds
2021-06-17 18:58:48,810 ERROR: assert failed at "bfd_topo2.test_bfd_topo2/test_protocols_convergence": "r2" JSON output mismatches
assert Generated JSON diff error report:
  
  > $: d2 has key '10.254.254.1/32' which is not present in d1
2021-06-17 19:30:09,817 ERROR: Traceback (most recent call last):
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO1U18I386/topotests/lib/bgp.py", line 203, in create_router_bgp
    tgen, router, data_all_bgp, "bgp", build, load_config
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO1U18I386/topotests/lib/common_config.py", line 355, in create_common_configuration
    load_config_to_router(tgen, router)
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO1U18I386/topotests/lib/common_config.py", line 619, in load_config_to_router
    raise InvalidCLIError("%s" % output)
InvalidCLIError: line 6: % Unknown command[27]: neighbor 10.0.0.13 remote-as 0 
% Specify remote-as or peer-group commands first
line 7: Failure to communicate[13] to bgpd, line: neighbor 10.0.0.13 timers 3 10 

line 9: % Unknown command[30]: neighbor fd00:0:0:3::1 remote-as 0 
% Specify remote-as or peer-group commands first
line 11: Failure to communicate[13] to bgpd, line: neighbor fd00:0:0:3::1 activate 

% Specify remote-as or peer-group commands first
line 12: Failure to communicate[13] to bgpd, line: neighbor fd00:0:0:3::1 timers 3 10 

% Specify remote-as or peer-group commands first
line 14: Failure to communicate[13] to bgpd, line: no neighbor fd00:0:0:3::1 activate 



2021-06-17 19:30:10,593 ERROR: Traceback (most recent call last):
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO1U18I386/topotests/lib/bgp.py", line 203, in create_router_bgp
    tgen, router, data_all_bgp, "bgp", build, load_config
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO1U18I386/topotests/lib/common_config.py", line 355, in create_common_configuration
    load_config_to_router(tgen, router)
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO1U18I386/topotests/lib/common_config.py", line 619, in load_config_to_router
    raise InvalidCLIError("%s" % output)
InvalidCLIError: % No BGP process is configured
line 2: Failure to communicate[13] to bgpd, line: no router bgp  



2021-06-17 19:37:39,393 ERROR: r1: bgpd left a dead pidfile (pid=18339)

see full log at https://ci1.netdef.org/browse/FRR-FRRPULLREQ-19673/artifact/TOPO1U18I386/ErrorLog/log_topotests.txt

Topotests Ubuntu 18.04 arm8 part 9: Failed (click for details) Topotests Ubuntu 18.04 arm8 part 9: No useful log found
Topotests Ubuntu 18.04 arm8 part 2: Incomplete (check logs for details)
Topotests Ubuntu 18.04 arm8 part 2: Incomplete (check logs for details)
Topotests Ubuntu 18.04 arm8 part 8: Incomplete (check logs for details)
Topotests Ubuntu 18.04 i386 part 1: Failed (click for details)

Topology Test Results are at https://ci1.netdef.org/browse/FRR-FRRPULLREQ-TOPO1U18I386-19673/test

Topology Tests failed for Topotests Ubuntu 18.04 i386 part 1:

2021-06-17 18:58:48,808 ERROR: 'router_json_cmp' failed after 135.64 seconds
2021-06-17 18:58:48,810 ERROR: assert failed at "bfd_topo2.test_bfd_topo2/test_protocols_convergence": "r2" JSON output mismatches
assert Generated JSON diff error report:
  
  > $: d2 has key '10.254.254.1/32' which is not present in d1
2021-06-17 19:30:09,817 ERROR: Traceback (most recent call last):
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO1U18I386/topotests/lib/bgp.py", line 203, in create_router_bgp
    tgen, router, data_all_bgp, "bgp", build, load_config
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO1U18I386/topotests/lib/common_config.py", line 355, in create_common_configuration
    load_config_to_router(tgen, router)
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO1U18I386/topotests/lib/common_config.py", line 619, in load_config_to_router
    raise InvalidCLIError("%s" % output)
InvalidCLIError: line 6: % Unknown command[27]: neighbor 10.0.0.13 remote-as 0 
% Specify remote-as or peer-group commands first
line 7: Failure to communicate[13] to bgpd, line: neighbor 10.0.0.13 timers 3 10 

line 9: % Unknown command[30]: neighbor fd00:0:0:3::1 remote-as 0 
% Specify remote-as or peer-group commands first
line 11: Failure to communicate[13] to bgpd, line: neighbor fd00:0:0:3::1 activate 

% Specify remote-as or peer-group commands first
line 12: Failure to communicate[13] to bgpd, line: neighbor fd00:0:0:3::1 timers 3 10 

% Specify remote-as or peer-group commands first
line 14: Failure to communicate[13] to bgpd, line: no neighbor fd00:0:0:3::1 activate 



2021-06-17 19:30:10,593 ERROR: Traceback (most recent call last):
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO1U18I386/topotests/lib/bgp.py", line 203, in create_router_bgp
    tgen, router, data_all_bgp, "bgp", build, load_config
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO1U18I386/topotests/lib/common_config.py", line 355, in create_common_configuration
    load_config_to_router(tgen, router)
  File "/root/bamboo-agent-home/xml-data/build-dir/FRR-FRRPULLREQ-TOPO1U18I386/topotests/lib/common_config.py", line 619, in load_config_to_router
    raise InvalidCLIError("%s" % output)
InvalidCLIError: % No BGP process is configured
line 2: Failure to communicate[13] to bgpd, line: no router bgp  



2021-06-17 19:37:39,393 ERROR: r1: bgpd left a dead pidfile (pid=18339)

see full log at https://ci1.netdef.org/browse/FRR-FRRPULLREQ-19673/artifact/TOPO1U18I386/ErrorLog/log_topotests.txt

Topotests Ubuntu 18.04 arm8 part 9: Failed (click for details) Topotests Ubuntu 18.04 arm8 part 9: No useful log found
Topotests Ubuntu 18.04 arm8 part 1: Incomplete (check logs for details)
Successful on other platforms/tests
  • Topotests Ubuntu 18.04 amd64 part 2
  • Topotests Ubuntu 18.04 arm8 part 7
  • Topotests debian 10 amd64 part 2
  • Ubuntu 16.04 deb pkg check
  • Addresssanitizer topotests part 6
  • Topotests Ubuntu 18.04 amd64 part 6
  • Topotests Ubuntu 18.04 i386 part 6
  • Addresssanitizer topotests part 2
  • Ubuntu 18.04 deb pkg check
  • Topotests Ubuntu 18.04 amd64 part 1
  • Ubuntu 20.04 deb pkg check
  • Debian 9 deb pkg check
  • Debian 10 deb pkg check
  • CentOS 7 rpm pkg check
  • Topotests Ubuntu 18.04 amd64 part 8
  • Topotests debian 10 amd64 part 3
  • Addresssanitizer topotests part 9
  • Topotests Ubuntu 18.04 i386 part 3
  • Topotests debian 10 amd64 part 1
  • Addresssanitizer topotests part 7
  • Topotests debian 10 amd64 part 0
  • Topotests Ubuntu 18.04 i386 part 8
  • Addresssanitizer topotests part 3
  • Static analyzer (clang)
  • Addresssanitizer topotests part 5
  • Topotests Ubuntu 18.04 amd64 part 7
  • Topotests Ubuntu 18.04 i386 part 9
  • Topotests Ubuntu 18.04 amd64 part 0
  • Topotests Ubuntu 18.04 i386 part 4
  • Topotests Ubuntu 18.04 arm8 part 5
  • Topotests Ubuntu 18.04 arm8 part 0
  • Addresssanitizer topotests part 0
  • IPv4 ldp protocol on Ubuntu 18.04
  • Topotests Ubuntu 18.04 i386 part 0
  • Topotests debian 10 amd64 part 9
  • Topotests Ubuntu 18.04 i386 part 5
  • Topotests debian 10 amd64 part 4
  • Topotests Ubuntu 18.04 arm8 part 4
  • Topotests debian 10 amd64 part 8
  • Topotests Ubuntu 18.04 amd64 part 9
  • IPv6 protocols on Ubuntu 18.04
  • Fedora 29 rpm pkg check
  • Topotests Ubuntu 18.04 arm8 part 3
  • Topotests Ubuntu 18.04 amd64 part 3
  • Topotests Ubuntu 18.04 amd64 part 5
  • Addresssanitizer topotests part 1
  • Topotests debian 10 amd64 part 5
  • Topotests Ubuntu 18.04 i386 part 7
  • Topotests debian 10 amd64 part 6
  • Addresssanitizer topotests part 4
  • Topotests Ubuntu 18.04 amd64 part 4
  • Topotests Ubuntu 18.04 arm8 part 6
  • Topotests debian 10 amd64 part 7
  • IPv4 protocols on Ubuntu 18.04
  • Addresssanitizer topotests part 8
  • Topotests Ubuntu 18.04 i386 part 2

@idryzhov
Copy link
Contributor

The ospfd bug is fixed in #8874.
The test itself still has issues described by Martin. We can keep this PR but actually fix the test instead of disabling it.

@donaldsharp
Copy link
Member

fixed in a different way

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@donaldsharp donaldsharp donaldsharp requested changes

Assignees
No one assigned
Labels
tests Topotests, make check, etc
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@mwinter-osr @LabN-CI @qlyoung @NetDEF-CI @idryzhov @donaldsharp @polychaeta