Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

topotests: skip tests when any assert fails #5087

Conversation

rzalamena
Copy link
Member

@rzalamena rzalamena commented Oct 1, 2019

When an assert fails we should skip all other tests on the file. Once
a failure is detected we can't rely on the setup anymore, since most of
the tests assume the previous worked.


This will help tests to end earlier in case of tests failures and will only skip tests with the guard:

    if tgen.routers_have_failure():
        pytest.skip(tgen.errors) 

(this only applies to topologies using topogen)

Reported by @rwestphal

When an `assert` fails we should skip all other tests on the file. Once
a failure is detected we can't rely on the setup anymore, since most of
the tests assume the previous worked.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
@rzalamena rzalamena added the tests Topotests, make check, etc label Oct 1, 2019
@LabN-CI
Copy link
Collaborator

LabN-CI commented Oct 1, 2019

💚 Basic BGPD CI results: SUCCESS, 0 tests failed

Results table
_ _
Result SUCCESS git merge/5087 c8c265f
Date 10/01/2019
Start 08:30:15
Finish 08:51:49
Run-Time 21:34
Total 1815
Pass 1815
Fail 0
Valgrind-Errors 0
Valgrind-Loss 0
Details vncregress-2019-10-01-08:30:15.txt
Log autoscript-2019-10-01-08:31:09.log.bz2
Memory 420 426 360

For details, please contact louberger

Copy link
Member

@rwestphal rwestphal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks Rafael!

@NetDEF-CI
Copy link
Collaborator

Continuous Integration Result: SUCCESSFUL

Congratulations, this patch passed basic tests

Tested-by: NetDEF / OpenSourceRouting.org CI System

CI System Testrun URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-9100/

This is a comment from an automated CI system.
For questions and feedback in regards to this CI system, please feel free to email
Martin Winter - mwinter (at) opensourcerouting.org.


CLANG Static Analyzer Summary

  • Github Pull Request 5087, comparing to Git base SHA 698603c

No Changes in Static Analysis warnings compared to base

1 Static Analyzer issues remaining.

See details at
https://ci1.netdef.org/browse/FRR-FRRPULLREQ-9100/artifact/shared/static_analysis/index.html

@donaldsharp donaldsharp merged commit dda524e into FRRouting:master Oct 1, 2019
@rzalamena rzalamena deleted the topotest-skip-on-assert-failure branch October 9, 2019 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tests Topotests, make check, etc
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants