0.12.2: two test failures

[0-wiz-0]

When running the selftests for the 0.12.2 distfile from pypi on NetBSD 8.99.7/amd64 with python-3.6.3 and werkzeug 0.12.2, I see two failures:

======================================================================================= ERRORS ========================================================================================
____________________________________________________________________ ERROR at teardown of test_session_expiration _____________________________________________________________________

recwarn = WarningsRecorder(record=True)

    @pytest.yield_fixture(autouse=True)
    def catch_deprecation_warnings(recwarn):
        yield
        gc.collect()
>       assert not recwarn.list
E       assert not [<warnings.WarningMessage object at 0x7ede1af11d58>]
E        +  where [<warnings.WarningMessage object at 0x7ede1af11d58>] = WarningsRecorder(record=True).list

conftest.py:134: AssertionError
====================================================================================== FAILURES =======================================================================================
______________________________________________________________________________ test_no_error_swallowing _______________________________________________________________________________

flaskext_broken = None

    def test_no_error_swallowing(flaskext_broken):
        with pytest.raises(ImportError) as excinfo:
            import flask.ext.broken
    
>       assert excinfo.type is ImportError
E       AssertionError: assert <class 'ModuleNotFoundError'> is ImportError
E        +  where <class 'ModuleNotFoundError'> = <ExceptionInfo ModuleNotFoundError tblen=4>.type

test_ext.py:183: AssertionError
============================================================== 1 failed, 314 passed, 10 skipped, 1 error in 7.12 seconds ==============================================================
*** Error code 1

[davidism]

Does this fail on master?

[0-wiz-0]

On master I see some failed tests because python3 (in particular, click) does not recognize the default locale as UTF-8, and:

_____________________________________________________________________________ TestRoutes.test_all_methods _____________________________________________________________________________

self = <test_cli.TestRoutes object at 0x78c1a0905818>
invoke = functools.partial(<bound method CliRunner.invoke of <click.testing.CliRunner object at 0x78c1a0906e58>>, <flask.cli.FlaskGroup object at 0x78c1a0916f18>)

    def test_all_methods(self, invoke):
        output = invoke(['routes']).output
        assert 'GET, HEAD, OPTIONS, POST' not in output
        output = invoke(['routes', '--all-methods']).output
>       assert 'GET, HEAD, OPTIONS, POST' in output
E       AssertionError: assert 'GET, HEAD, OPTIONS, POST' in ''

tests/test_cli.py:434: AssertionError

but not this particular error.
python-2.7 runs fine.

I do use different testing methods though: the 0.12.2 distfile, when using 'tox', has very little output:

GLOB sdist-make: /tmp/a/Flask-0.12.2/setup.py
python inst-nodeps: /tmp/a/Flask-0.12.2/.tox/dist/Flask-0.12.2.zip
python installed: click==6.7,Flask==0.12.2,itsdangerous==0.24,Jinja2==2.10,MarkupSafe==1.0,Werkzeug==0.12.2
python runtests: PYTHONHASHSEED='476525965'
_______________________________________________________________________________________ summary _______________________________________________________________________________________
  python: commands succeeded
  congratulations :)

so I do python setup.py build/install/test. Mentioning this just in case it matters.

[davidism]

You should be testing from source, not sdist.

[0-wiz-0]

When I'm packaging, I always test the pypi distfile, since that's what I'm packaging.
Then I try running the self tests to see if there are any issues.

[davidism]

The only test scenario I'm supporting is creating a virtualenv, installing the source, installing tox or pytest, and running the tests with the appropriate tool. Also, I tried setting up NetBSD but didn't have time to figure it out. If you want this fixed, you're going to need to investigate and contribute a PR.

[0-wiz-0]

When I'm packaging, I always use the pypi distfile.
Then I try running the self tests to see if there are any issues.

[davidism]

setup.py test is definitely not supported. I'm surprised it does anything at all.

[davidism]

tox.ini isn't in MANIFEST.in which is why tox doesn't output anything. I can fix that in master.

make test does work, assuming you have pytest installed, but it will skip tests for missing optional dependencies which tox would install.

[0-wiz-0]

I'm sorry I didn't clarify that earlier: I'm running the tests using py.test. Skipping tests for optional dependencies without installing them is perfectly fine in a packaging scenario.
Thanks for adding the tox.ini file.
What in particular would you want investigated?

[davidism]

I don't have access to a NetBSD setup. All tests pass for me on my setup, and on Travis. So if you're observing a failure that I can't reproduce, you'll need to debug it.

[davidism]

I do get those errors by doing git checkout 0.12.2 and tox -e py. I recall that the warning checks were overly aggressive or something, but that's fixed in master. You're going to need to ignore those.

[davidism]

If click is having an issue detecting the locale, that needs to be reported to click.

[0-wiz-0]

Thanks for confirming that the original two bugs are gone.
The click problem is known. You need to set the LANG (some others work too) environment variable to en_US.UTF-8 (or another UTF-8 locale). I have that in my environment, so I guess tox cleans the environment?

[davidism]

tox.ini is passing LANG, in 0.12.2 and master, so not sure why that's not working.

[0-wiz-0]

That's easy to explain - because I only set LC_CTYPE by default, not LANG, and only LANG is passed by tox. So I think we're good here after all!
Thanks for your help!

[davidism]

An sdist from master should now work with tox, just pushed a fix.

Sorry for the confusion, and thanks for reporting it.