Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tests: Topotests are not giving sufficient time to propagate data #4680

Merged
merged 1 commit into from
Jul 17, 2019
Merged

tests: Topotests are not giving sufficient time to propagate data #4680

merged 1 commit into from
Jul 17, 2019

Conversation

donaldsharp
Copy link
Member

A bunch of our current tests setup data and redistribute
it across some bgp connections and then test for it
being there. A delay of 2 seconds that was initially
there to ensure that the data has propagated does not
actually work all the time when you have a loaded
virtualized environment.

Make the sleep 10 seconds. I agree this is not the ideal
solution but I would rather get the damn tests up and running
again.

Signed-off-by: Donald Sharp sharpd@cumulusnetworks.com

@donaldsharp
tests: Topotests are not giving sufficient time to propagate data
1d90ade 
A bunch of our current tests setup data and redistribute
it across some bgp connections and then test for it
being there.  A delay of 2 seconds that was initially
there to ensure that the data has propagated does not
actually work all the time when you have a loaded
virtualized environment.

Make the sleep 10 seconds.  I agree this is not the ideal
solution but I would rather get the damn tests up and running
again.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
@polychaeta polychaeta added the tests Topotests, make check, etc label Jul 11, 2019
@LabN-CI
Copy link
Collaborator

LabN-CI commented Jul 11, 2019

💚 Basic BGPD CI results: SUCCESS, 0 tests failed

Results table
_ _
Result SUCCESS git merge/4680 1d90ade
Date 07/11/2019
Start 14:50:26
Finish 15:12:34
Run-Time 22:08
Total 1813
Pass 1813
Fail 0
Valgrind-Errors 0
Valgrind-Loss 0
Details vncregress-2019-07-11-14:50:26.txt
Log autoscript-2019-07-11-14:51:24.log.bz2
Memory 435 422 360

For details, please contact louberger

@NetDEF-CI
Copy link
Collaborator

Continuous Integration Result: FAILED

See below for issues.
CI System Testrun URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-8352/

This is a comment from an automated CI system.
For questions and feedback in regards to this CI system, please feel free to email
Martin Winter - mwinter (at) opensourcerouting.org.

Get source / Pull Request: Successful

Building Stage: Successful

Basic Tests: Failed

Topology tests on Ubuntu 18.04 amd64: Failed (click for details)

Topology Test Results are at https://ci1.netdef.org/browse/FRR-FRRPULLREQ-TOPOU1804-8352/test

Topology Tests failed for Topology tests on Ubuntu 18.04 amd64:

2019-07-11 19:14:25,086 WARNING: BGP is not yet Converged for router r4
2019-07-11 19:14:41,466 WARNING: BGP is not yet Converged for router r1 after bgp clear
2019-07-11 19:21:45,593 WARNING: BGP is not yet Converged for router r1 after bgp clear
2019-07-11 19:24:16,492 WARNING: BGP is not yet Converged for router r4
*** defaultIntf: warning: r1 has no interfaces
2019-07-11 19:33:52,528 ERROR: ******************************************************************************
2019-07-11 19:33:52,530 ERROR: Test Target Summary                                                  Pass Fail
2019-07-11 19:33:52,530 ERROR: ******************************************************************************
2019-07-11 19:33:52,530 ERROR: FILE: scripts/check_linux_vrf.py
2019-07-11 19:33:52,530 ERROR: FILE: scripts/adjacencies.py
2019-07-11 19:33:52,530 ERROR: FILE: scripts/check_routes.py
2019-07-11 19:33:52,530 ERROR: FILE: scripts/check_linux_mpls.py
2019-07-11 19:33:52,530 ERROR: FILE: scripts/scale_up.py
2019-07-11 19:33:52,531 ERROR: FILE: scripts/scale_down.py
2019-07-11 19:33:52,531 ERROR: 143  ce1    BGP routes removed +12.52 secs                           0    1
2019-07-11 19:33:52,531 ERROR: See /tmp/topotests/bgp_l3vpn_to_bgp_vrf.test_bgp_l3vpn_to_bgp_vrf/output.log for details of errors
2019-07-11 19:33:52,534 ERROR: assert failed at "bgp_l3vpn_to_bgp_vrf.test_bgp_l3vpn_to_bgp_vrf/test_check_scale_down": 1 tests failed
2019-07-11 19:39:22,999 WARNING: BGP is not yet Converged for router r2
2019-07-11 19:39:43,620 WARNING: BGP is not yet Converged for router r2
2019-07-11 19:40:03,963 WARNING: BGP is not yet Converged for router r2
*** defaultIntf: warning: lm has no interfaces

see full log at https://ci1.netdef.org/browse/FRR-FRRPULLREQ-8352/artifact/TOPOU1804/ErrorLog/log_topotests.txt

Successful on other platforms
  • Fedora 29 rpm pkg check
  • Debian 9 deb pkg check
  • Ubuntu 12.04 deb pkg check
  • Static analyzer (clang)
  • Ubuntu 16.04 deb pkg check
  • IPv6 protocols on Ubuntu 14.04
  • Topotest tests on Ubuntu 16.04 i386
  • CentOS 7 rpm pkg check
  • Debian 10 deb pkg check
  • Ubuntu 14.04 deb pkg check
  • IPv4 ldp protocol on Ubuntu 16.04
  • Addresssanitizer topotest
  • Debian 8 deb pkg check
  • IPv4 protocols on Ubuntu 14.04
  • Topology tests on Ubuntu 16.04 amd64

Topology Tests memory analysis: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-8352/artifact/TOPOU1804/MemoryLeaks/

Warnings Generated during build:

Debian 10 amd64 build: Successful with additional warnings

Debian Package lintian failed for Debian 10 amd64 build:
(see full package build log at https://ci1.netdef.org/browse/FRR-FRRPULLREQ-8352/artifact/DEB10BUILD/ErrorLog/log_lintian.txt)

W: frr source: changelog-should-mention-nmu

CLANG Static Analyzer Summary

  • Github Pull Request 4680, comparing to Git base SHA 84f4f72

No Changes in Static Analysis warnings compared to base

1 Static Analyzer issues remaining.

See details at
https://ci1.netdef.org/browse/FRR-FRRPULLREQ-8352/artifact/shared/static_analysis/index.html

@donaldsharp
Copy link
Member Author

ci:rerun

@NetDEF-CI
Copy link
Collaborator

Continuous Integration Result: FAILED

See below for issues.
CI System Testrun URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-8352/

This is a comment from an automated CI system.
For questions and feedback in regards to this CI system, please feel free to email
Martin Winter - mwinter (at) opensourcerouting.org.

Get source / Pull Request: Successful

Building Stage: Successful

Basic Tests: Failed

Topology tests on Ubuntu 18.04 amd64: Failed (click for details)

Topology Test Results are at https://ci1.netdef.org/browse/FRR-FRRPULLREQ-TOPOU1804-8352/test

Topology Tests failed for Topology tests on Ubuntu 18.04 amd64:

2019-07-12 13:20:41,424 WARNING: BGP is not yet Converged for router r4
2019-07-12 13:20:57,716 WARNING: BGP is not yet Converged for router r1 after bgp clear
2019-07-12 13:30:30,482 WARNING: BGP is not yet Converged for router r4
*** defaultIntf: warning: r1 has no interfaces
2019-07-12 13:40:21,400 ERROR: ******************************************************************************
2019-07-12 13:40:21,401 ERROR: Test Target Summary                                                  Pass Fail
2019-07-12 13:40:21,401 ERROR: ******************************************************************************
2019-07-12 13:40:21,401 ERROR: FILE: scripts/check_linux_vrf.py
2019-07-12 13:40:21,401 ERROR: FILE: scripts/adjacencies.py
2019-07-12 13:40:21,401 ERROR: FILE: scripts/check_routes.py
2019-07-12 13:40:21,401 ERROR: FILE: scripts/check_linux_mpls.py
2019-07-12 13:40:21,401 ERROR: FILE: scripts/scale_up.py
2019-07-12 13:40:21,402 ERROR: FILE: scripts/scale_down.py
2019-07-12 13:40:21,402 ERROR: 143  ce1    BGP routes removed +10.36 secs                           0    1
2019-07-12 13:40:21,402 ERROR: See /tmp/topotests/bgp_l3vpn_to_bgp_vrf.test_bgp_l3vpn_to_bgp_vrf/output.log for details of errors
2019-07-12 13:40:21,408 ERROR: assert failed at "bgp_l3vpn_to_bgp_vrf.test_bgp_l3vpn_to_bgp_vrf/test_check_scale_down": 1 tests failed
*** defaultIntf: warning: lm has no interfaces

see full log at https://ci1.netdef.org/browse/FRR-FRRPULLREQ-8352/artifact/TOPOU1804/ErrorLog/log_topotests.txt

Successful on other platforms
  • Fedora 29 rpm pkg check
  • Debian 9 deb pkg check
  • Ubuntu 12.04 deb pkg check
  • Static analyzer (clang)
  • Ubuntu 16.04 deb pkg check
  • IPv6 protocols on Ubuntu 14.04
  • Topotest tests on Ubuntu 16.04 i386
  • CentOS 7 rpm pkg check
  • Debian 10 deb pkg check
  • Ubuntu 14.04 deb pkg check
  • IPv4 ldp protocol on Ubuntu 16.04
  • Addresssanitizer topotest
  • Debian 8 deb pkg check
  • IPv4 protocols on Ubuntu 14.04
  • Topology tests on Ubuntu 16.04 amd64

Topology Tests memory analysis: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-8352/artifact/TOPOU1804/MemoryLeaks/

Warnings Generated during build:

Debian 10 amd64 build: Successful with additional warnings

Debian Package lintian failed for Debian 10 amd64 build:
(see full package build log at https://ci1.netdef.org/browse/FRR-FRRPULLREQ-8352/artifact/DEB10BUILD/ErrorLog/log_lintian.txt)

W: frr source: changelog-should-mention-nmu

CLANG Static Analyzer Summary

  • Github Pull Request 4680, comparing to Git base SHA 84f4f72

No Changes in Static Analysis warnings compared to base

1 Static Analyzer issues remaining.

See details at
https://ci1.netdef.org/browse/FRR-FRRPULLREQ-8352/artifact/shared/static_analysis/index.html

mjstapp
mjstapp approved these changes Jul 17, 2019
View reviewed changes
Copy link
Contributor

@mjstapp mjstapp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving - we're going to try merging this, and see if it helps with some of the CI issues in suites that use this framework

@mjstapp mjstapp merged commit 974c214 into FRRouting:master Jul 17, 2019
@donaldsharp donaldsharp deleted the test_longer branch December 9, 2019 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mjstapp mjstapp mjstapp approved these changes

Assignees
No one assigned
Labels
tests Topotests, make check, etc
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@donaldsharp @LabN-CI @NetDEF-CI @mjstapp @polychaeta