timeouting test cases #206
Description
usually on pypy, but not always, some hypothesis test cases fail because example generation timesout:
GLOB sdist-make: /home/travis/build/warner/python-ecdsa/setup.py
pypy3 create: /home/travis/build/warner/python-ecdsa/.tox/pypy3
pypy3 installdeps: pytest, hypothesis, coverage==4.5.4
pypy3 inst: /home/travis/build/warner/python-ecdsa/.tox/.tmp/package/1/ecdsa-0.15+38.g46d0f14.zip
pypy3 installed: attrs==19.3.0,cffi==1.10.1,coverage==4.5.4,ecdsa @ file:///home/travis/build/warner/python-ecdsa/.tox/.tmp/package/1/ecdsa-0.15%2B38.g46d0f14.zip,greenlet==0.4.12,hypothesis==5.23.7,importlib-metadata==1.7.0,iniconfig==1.0.0,more-itertools==8.4.0,packaging==20.4,pathlib2==2.3.5,pluggy==0.13.1,py==1.9.0,pyparsing==2.4.7,pytest==6.0.0,readline==6.2.4.1,six==1.15.0,sortedcontainers==2.2.2,toml==0.10.1,zipp==1.2.0
pypy3 run-test-pre: PYTHONHASHSEED='481791472'
pypy3 run-test: commands[0] | coverage run --branch -m pytest src/ecdsa
============================= test session starts ==============================
platform linux -- Python 3.5.3[pypy-5.8.0-beta], pytest-6.0.0, py-1.9.0, pluggy-0.13.1
cachedir: .tox/pypy3/.pytest_cache
rootdir: /home/travis/build/warner/python-ecdsa
plugins: hypothesis-5.23.7
collected 1507 items
src/ecdsa/test_der.py .................................................. [ 3%]
................ [ 4%]
src/ecdsa/test_ecdh.py .................................sssssssssssss [ 7%]
src/ecdsa/test_ecdsa.py ........................................... [ 10%]
src/ecdsa/test_ellipticcurve.py ........................ [ 11%]
src/ecdsa/test_jacobi.py .................................F..... [ 14%]
src/ecdsa/test_keys.py ................................................. [ 17%]
........................................................................ [ 22%]
[ 22%]
src/ecdsa/test_malformed_sigs.py ....................................... [ 25%]
........................................................................ [ 29%]
........................................................................ [ 34%]
................................................................... [ 39%]
src/ecdsa/test_numbertheory.py ......................................... [ 41%]
........................................................................ [ 46%]
........................................................................ [ 51%]
................................................. [ 54%]
src/ecdsa/test_pyecdsa.py .............................................. [ 57%]
........................................................................ [ 62%]
........................................................................ [ 67%]
........................................................................ [ 71%]
........................................................................ [ 76%]
........................................................................ [ 81%]
........................................................................ [ 86%]
........................................................................ [ 91%]
...............................................................sssssss.. [ 95%]
.......sssssss............................................ [ 99%]
src/ecdsa/test_rw_lock.py .... [100%]
=================================== FAILURES ===================================
__________________________ TestJacobi.test_precompute __________________________
self = <ecdsa.test_jacobi.TestJacobi testMethod=test_precompute>
@settings(max_examples=10)
> @given(st.integers(min_value=0, max_value=int(generator_256.order())))
@example(0)
@example(int(generator_256.order()))
def test_precompute(self, mul):
src/ecdsa/test_jacobi.py:195:
.tox/pypy3/site-packages/hypothesis/core.py:343: in execute_explicit_examples
print_example=True,
.tox/pypy3/site-packages/hypothesis/core.py:610: in execute_once
result = self.test_runner(data, run)
.tox/pypy3/site-packages/hypothesis/executors.py:52: in default_new_style_executor
return function(data)
.tox/pypy3/site-packages/hypothesis/core.py:606: in run
return test(*args, **kwargs)
src/ecdsa/test_jacobi.py:195: in test_precompute
@given(st.integers(min_value=0, max_value=int(generator_256.order())))
args = (<ecdsa.test_jacobi.TestJacobi testMethod=test_precompute>, 0)
kwargs = {}, initial_draws = 0, start = 120.235048722, result = None
finish = 120.456533556, internal_draw_time = 0
runtime = datetime.timedelta(0, 0, 221485)
current_deadline = timedelta(milliseconds=200)
@proxies(self.test)
def test(*args, **kwargs):
self.__test_runtime = None
initial_draws = len(data.draw_times)
start = time.perf_counter()
result = self.test(*args, **kwargs)
finish = time.perf_counter()
internal_draw_time = sum(data.draw_times[initial_draws:])
runtime = datetime.timedelta(
seconds=finish - start - internal_draw_time
self.__test_runtime = runtime
current_deadline = self.settings.deadline
if not is_final:
current_deadline = (current_deadline // 4) * 5
if runtime >= current_deadline:
> raise DeadlineExceeded(runtime, self.settings.deadline)
E hypothesis.errors.DeadlineExceeded: Test took 221.48ms, which exceeds the deadline of 200.00ms
.tox/pypy3/site-packages/hypothesis/core.py:540: DeadlineExceeded
---------------------------------- Hypothesis ----------------------------------
Falsifying example: test_precompute(
self=<ecdsa.test_jacobi.TestJacobi testMethod=test_precompute>, mul=0,
=========================== short test summary info ============================
FAILED src/ecdsa/test_jacobi.py::TestJacobi::test_precompute - hypothesis.err...
============ 1 failed, 1479 passed, 27 skipped in 131.83s (0:02:11) ============
ERROR: InvocationError for command /home/travis/build/warner/python-ecdsa/.tox/pypy3/bin/coverage run --branch -m pytest src/ecdsa (exited with code 1)
we should increase the timeout for them