Fails to run unit tests

[jorgedch]

Documentation

openjournals/joss-reviews#4517

• Automated tests:

1. Fails to run unit tests:

(venv) ➜  salmon git:(master) pytest

========== test session starts ==========
platform darwin -- Python 3.9.12, pytest-7.1.2, pluggy-1.0.0 -- /Users/jorge/Desktop/JOSS/salmon/venv/bin/python
cachedir: .pytest_cache
rootdir: /Users/jorge/Desktop/JOSS/salmon, configfile: setup.cfg
collected 0 items / 14 errors

...

========== short test summary info ==========
ERROR salmon/triplets/test_config.py
ERROR salmon/triplets/samplers/adaptive/search/tests/test_gram_utils.py
ERROR salmon/triplets/samplers/adaptive/search/tests/test_search_refactor.py
ERROR salmon/triplets/samplers/adaptive/search/tests/test_uncertainty.py
ERROR salmon/triplets/samplers/tests/test_rr.py
ERROR tests/test_active.py
ERROR tests/test_active_search.py
ERROR tests/test_allowabe_targets.py
ERROR tests/test_backend.py
ERROR tests/test_basic.py
ERROR tests/test_examples.py
ERROR tests/test_offline.py
ERROR tests/test_passive.py
ERROR tests/test_validation.py

[stsievert]

Hm... can't reproduce, at least when I'm running from the salmon environment. I think this a setup issue like "ModuleNotFound" errors (because these tests are run every commit and generally pass). What tracebacks are the failing tests providing?

$ conda activate salmon
(salmon) $ pytest salmon
============================ test session starts =============================
platform darwin -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0 -- /Users/scott/anaconda3/envs/salmon/bin/python
cachedir: .pytest_cache
rootdir: /Users/scott/Downloads/salmon, configfile: setup.cfg
plugins: anyio-3.5.0
collected 57 items

salmon/triplets/test_config.py::test_old_html_keys_warns[debrief] PASSED [  1%]
salmon/triplets/test_config.py::test_old_html_keys_warns[instructions] PASSED [  3%]
# ...

[jorgedch]

Hi @stsievert, yes you are right. My mistake for not having installed the requirements beforehand. Once I install them, I get the following (looking better):

(venv) ➜  salmon git:(master)

========== short test summary info ==========
ERROR tests/test_active.py::test_samplers_per_user - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_active.py::test_active_wrong_proportion - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_active.py::test_active_bad_keys - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_active.py::test_active_queries_generated[ARR] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_active.py::test_active_queries_generated[CKL] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_active.py::test_active_basics - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_allowabe_targets.py::test_targets[RoundRobin] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_allowabe_targets.py::test_targets[Random] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_allowabe_targets.py::test_targets_too_large[RoundRobin] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_allowabe_targets.py::test_targets_too_large[Random] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_allowabe_targets.py::test_targets_wrong_type[RoundRobin] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_allowabe_targets.py::test_targets_wrong_type[Random] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_allowabe_targets.py::test_targets_not_integers[RoundRobin] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_allowabe_targets.py::test_targets_not_integers[Random] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_allowabe_targets.py::test_targets_too_small[RoundRobin] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_allowabe_targets.py::test_targets_too_small[Random] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_backend.py::test_backend_basics - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_backend.py::test_init_errors_propogate - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_backend.py::test_run_errors_logged - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_basic.py::test_basics - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_basic.py::test_bad_file_upload - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_basic.py::test_no_repeats - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_basic.py::test_meta - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_basic.py::test_saves_state - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_basic.py::test_download_restore - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_basic.py::test_logs - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_basic.py::test_zip_upload - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_basic.py::test_get_config - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_basic.py::test_config_defaults_update - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_basic.py::test_config_misplaced_error - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_basic.py::test_no_init_twice - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_basic.py::test_auth_repeated_entries - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_basic.py::test_random_error - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_basic.py::test_html_defaults_rendered - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_basic.py::test_defaults_acceptable_config - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_examples.py::test_defaults_config - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_examples.py::test_basic_examples[customize.yaml] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_examples.py::test_basic_examples[complete.yaml] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_examples.py::test_basic_examples[basic.yaml] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_examples.py::test_basic_examples[default.yaml] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_examples.py::test_directory_examples[alien-eggs] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_examples.py::test_directory_examples[colors] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_examples.py::test_directory_examples[zappos] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_examples.py::test_directory_examples[faces] - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_passive.py::test_init_w_queries_valid_idx - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_passive.py::test_validation_sampling - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_passive.py::test_round_robin - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_passive.py::test_round_robin_per_user - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_validation.py::test_init_w_queries_valid_idx - httpx.ConnectError: [Errno 61] Connection refused
ERROR tests/test_validation.py::test_validation_sampling - httpx.ConnectError: [Errno 61] Connection refused
========== 65 passed, 7 warnings, 50 errors in 29.43s ==========

Basically I was trying to run the tests from a local install as the docker build is not completing in my machine (#135), but if I understand correctly it will only work once the container is launched and connecting to the resources created during deployment.

I'm going through these as they are part of the JOSS checklist: Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?. Let's solve first issue #135 and then probably the tests will be passing for me as well. Otherwise, do you have some guidance of how can I get them passing without Docker?

[stsievert]

All the tests in tests/ require Docker and a Salmon instance at localhost:8421. All the tests in salmon/ do not require Docker.

I think the fact that #135 is resolved will resolve this issue; let me know if this issue is persisting.

[jorgedch]

All good to go!