Fixes #4036 Updates setuptools for ci

[kushaldas]

Status

Ready for Review

Description of Changes

Fixes #4036

With the upgraded setuptools package, the errors
in the issue gets fixed.

Testing

To see only the effected tests: $ BASE_OS=xenial securedrop/bin/dev-shell bin/run-test -v tests/test_i18n_tool.py tests/test_i18n.py

make test-xenial

Deployment

None yet.

Checklist

If you made changes to the server application code:

• Linting (make ci-lint) and tests (make -C securedrop test) pass in the development container

If you made changes to securedrop-admin:

• Linting and tests (make -C admin test) pass in the admin development container

If you made changes to the system configuration:

• Configuration tests pass

If you made non-trivial code changes:

• I have written a test plan and validated it for this PR

If you made changes to documentation:

• Doc linting (make docs-lint) passed locally

[redshiftzero]

Nice - so since we don't use the Dockerfile in prod, how do you think we should get this change onto prod servers?

[redshiftzero]

Hey @kushaldas, see my comment here. I think merging this is the best path forward. Before you merge can you verify and comment to confirm that the 2FA issues are not impacted by this change? (I've updated the PR description to indicate this is the case)

[redshiftzero]

Can you add a comment above explaining that this was added due to #4036?

[redshiftzero]

hey @kushaldas check out my comment above - I think this is ready to go and will close one of the Xenial app tickets 😇

[kushaldas]

@redshiftzero This PR is actually fixing the 2fa tests along with i18n ones :)

$ BASE_OS=xenial securedrop/bin/dev-shell bin/run-test -v tests/test_i18n_tool.py tests/test_i18n.py tests/test_2fa.py
Run with DOCKER_BUILD_VERBOSE=true for more information
Docker image build in progress  done !
      write static/css/journalist.css
      write static/css/journalist.css.map
      write static/css/source.css
      write static/css/source.css.map
==================================================================== test session starts =====================================================================
platform linux2 -- Python 2.7.12, pytest-3.3.2, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python
cachedir: tests/.cache
rootdir: /home/kdas/code/securedrop/securedrop/tests, inifile: pytest.ini
plugins: mock-1.7.1, cov-2.5.1
collected 17 items                                                                                                                                           

tests/test_i18n_tool.py::TestI18NTool::test_main PASSED                                                                                                [  5%]
tests/test_i18n_tool.py::TestI18NTool::test_translate_desktop_l10n PASSED                                                                              [ 11%]
tests/test_i18n_tool.py::TestI18NTool::test_translate_messages_l10n PASSED                                                                             [ 17%]
tests/test_i18n_tool.py::TestI18NTool::test_translate_messages_compile_arg PASSED                                                                      [ 23%]
tests/test_i18n_tool.py::TestI18NTool::test_require_git_email_name PASSED                                                                              [ 29%]
tests/test_i18n_tool.py::TestI18NTool::test_update_docs PASSED                                                                                         [ 35%]
tests/test_i18n_tool.py::TestI18NTool::test_update_from_weblate PASSED                                                                                 [ 41%]
tests/test_i18n.py::test_get_supported_locales PASSED                                                                                                  [ 47%]
tests/test_i18n.py::test_i18n PASSED                                                                                                                   [ 52%]
tests/test_i18n.py::test_verify_default_locale_en_us_if_not_defined_in_config PASSED                                                                   [ 58%]
tests/test_i18n.py::test_locale_to_rfc_5646 PASSED                                                                                                     [ 64%]
tests/test_i18n.py::test_html_en_lang_correct PASSED                                                                                                   [ 70%]
tests/test_i18n.py::test_html_fr_lang_correct PASSED                                                                                                   [ 76%]
tests/test_2fa.py::test_totp_reuse_protections PASSED                                                                                                  [ 82%]
tests/test_2fa.py::test_totp_reuse_protections2 PASSED                                                                                                 [ 88%]
tests/test_2fa.py::test_bad_token_fails_to_verify_on_admin_new_user_two_factor_page PASSED                                                             [ 94%]
tests/test_2fa.py::test_bad_token_fails_to_verify_on_new_user_two_factor_page PASSED                                                                   [100%]

------------------------------------------------------ generated xml file: /tmp/test-results/junit.xml -------------------------------------------------------

---------- coverage: platform linux2, python 2.7.12-final-0 ----------
Name                                   Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------------------
config.py                                 41      3      6      3    87%   51, 54-57, 49->51, 52->54, 58->67
create-dev-data.py                        54     54      6      0     0%   4-100
crypto_util.py                           108     35     38      5    60%   23, 96, 122, 141, 177-179, 189-197, 200-204, 207-211, 215-236, 247-249, 263, 18->23, 95->96, 119->122, 151->153, 262->263
db.py                                      2      0      0      0   100%
i18n.py                                   78      2     38      2    97%   55, 162, 54->55, 161->162
i18n_tool.py                             188      5     54      6    95%   68, 117-118, 122, 358, 67->68, 113->122, 115->117, 202->206, 242->244, 354->358
journalist.py                              3      3      0      0     0%   3-7
journalist_app/__init__.py                87     17     22      6    77%   31, 53, 82-85, 90-94, 105-107, 120-121, 139, 147-148, 26->31, 49->53, 104->105, 119->120, 138->139, 146->147
journalist_app/account.py                 44     27     10      2    35%   17-18, 23-35, 42-44, 54-57, 61-69, 39->50, 41->42
journalist_app/admin.py                  150    109     38      2    23%   25-26, 31-48, 53-97, 109-114, 125-130, 135-144, 149-180, 186-193, 198-216, 221-228, 233-236, 106->120, 108->109
journalist_app/api.py                    196    141     48      0    23%   26-28, 34-45, 50-56, 64-69, 74-84, 88-117, 122-123, 129-135, 140-143, 148-151, 156-160, 165-166, 174-182, 188-191, 197-208, 213-269, 276-283, 288-289, 295-296, 302-303, 308-311
journalist_app/col.py                     49     33      8      0    28%   22-24, 28-30, 34-37, 43-48, 52-67, 73-86
journalist_app/decorators.py              11      2      2      1    77%   15-17, 13->15
journalist_app/forms.py                   23      5      4      0    67%   14-16, 24-25
journalist_app/main.py                    98     59     30      1    34%   50-54, 69-74, 97-132, 136-138, 143-165, 169-184, 188-197, 68->69
journalist_app/utils.py                  168    116     38      5    28%   27, 43-55, 61-65, 90-94, 108-109, 122-145, 159-169, 175-178, 182-189, 193, 200-205, 209-214, 218-222, 226-230, 235-245, 250-258, 263-272, 276-295, 304-315, 320-326, 330-339, 349, 22->27, 85->87, 89->90, 103->111, 348->349
manage.py                                226    226     36      0     0%   4-395
management/__init__.py                     0      0      0      0   100%
management/run.py                         14     14      2      0     0%   1-43
models.py                                306    112     64     17    58%   16-17, 39-48, 73-75, 78, 82-83, 87-97, 103-107, 111, 115, 119, 122-156, 174-177, 181, 184-198, 225-229, 233, 236-249, 259-262, 265-266, 302, 305-308, 350, 353, 360, 381, 386, 394, 398, 402, 407, 417, 423-430, 435, 438-444, 451, 455-458, 505-512, 531, 540-541, 553, 555, 559-561, 565-570, 573-579, 32->35, 349->350, 369->378, 378->381, 385->386, 393->394, 397->398, 401->402, 406->407, 412->417, 422->423, 448->451, 498->505, 530->531, 544->552, 552->553, 554->555
qa_loader.py                             159    159     38      0     0%   4-254
request_that_secures_file_uploads.py      10      4      2      0    50%   20-28, 31
rm.py                                      4      2      0      0    50%   23-24
sdconfig.py                              119     47      2      1    60%   12, 21-22, 27-28, 32-33, 57-58, 62-63, 67-68, 72-73, 77-78, 82-83, 87-88, 92-93, 97-98, 102-103, 107-108, 112-113, 118-119, 124-125, 129-130, 135-136, 140-141, 145-146, 150-151, 160-161, 7->12
secure_tempfile.py                        51     31     10      0    33%   48-54, 64-66, 72-74, 82-89, 109-118, 125-132
source.py                                  3      3      0      0     0%   3-7
source_app/__init__.py                    89     24     14      6    71%   31, 52, 80-83, 94-96, 115, 134-139, 151-164, 168, 172, 26->31, 48->52, 93->94, 114->115, 133->134, 150->151
source_app/api.py                         11      4      0      0    64%   13-18
source_app/decorators.py                  15      4      4      1    63%   10-12, 22, 21->22
source_app/forms.py                        7      0      0      0   100%
source_app/info.py                        14      5      0      0    64%   12, 16, 20-22, 29
source_app/main.py                       144    106     38      2    22%   30-35, 44-48, 52-72, 77-114, 126-205, 215-224, 229-242, 248-255, 261-271, 29->30, 247->248
source_app/utils.py                       57     29      8      2    46%   22-31, 48-52, 64-65, 70-71, 80-95, 104-111, 47->48, 59->36
store.py                                  95     68     30      2    23%   38, 43, 59-75, 81-84, 88-116, 120-153, 158-168, 172-175, 181-196, 37->38, 42->43
template_filters.py                       26     18      6      0    25%   11-17, 21-24, 28-47
version.py                                 1      0      0      0   100%
worker.py                                  7      1      0      0    86%   14
----------------------------------------------------------------------------------
TOTAL                                   2658   1468    596     64    42%
Coverage annotated source written to dir /tmp/test-results/cov_annotate
Coverage HTML written to dir /tmp/test-results/cov_html
Coverage XML written to file /tmp/test-results/cov.xml

================================================================= slowest 10 test durations ==================================================================
39.59s call     test_i18n_tool.py::TestI18NTool::test_update_from_weblate
9.86s call     test_i18n.py::test_i18n
4.44s call     test_i18n_tool.py::TestI18NTool::test_translate_messages_l10n
4.37s call     test_i18n_tool.py::TestI18NTool::test_update_docs
4.16s call     test_i18n_tool.py::TestI18NTool::test_translate_messages_compile_arg
4.02s call     test_i18n_tool.py::TestI18NTool::test_translate_desktop_l10n
1.75s call     test_2fa.py::test_bad_token_fails_to_verify_on_admin_new_user_two_factor_page
1.66s call     test_2fa.py::test_bad_token_fails_to_verify_on_new_user_two_factor_page
1.39s call     test_i18n_tool.py::TestI18NTool::test_require_git_email_name
0.69s setup    test_2fa.py::test_bad_token_fails_to_verify_on_admin_new_user_two_factor_page
====================================================================== warnings summary ======================================================================
test_2fa.py::test_totp_reuse_protections
  /usr/local/lib/python2.7/dist-packages/flask/sessions.py:208: UserWarning: "localhost" is not a valid cookie domain, it must contain a ".". Add an entry to your hosts file, for example "localhost.localdomain", and use that instead.
    ' "{rv}.localdomain", and use that instead.'.format(rv=rv)

-- Docs: http://doc.pytest.org/en/latest/warnings.html
=========================================================== 17 passed, 1 warnings in 79.82 seconds ===========================================================

[redshiftzero]

Hey - check out my comment in #4037 which explains why you see passing tests when you run the 2fa tests via BASE_OS=xenial securedrop/bin/dev-shell bin/run-test -v tests/test_i18n_tool.py tests/test_i18n.py tests/test_2fa.py (i.e. without executing any of the functional tests), but yet are failing in CI for this PR.

[kushaldas]

Hey - check out my comment in #4037 which explains why you see passing tests when you run the 2fa tests via BASE_OS=xenial securedrop/bin/dev-shell bin/run-test -v tests/test_i18n_tool.py tests/test_i18n.py tests/test_2fa.py (i.e. without executing any of the functional tests), but yet are failing in CI for this PR.

Ah, I will update the commit message accordingly.

[redshiftzero]

LGTM now, approving