Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace bespoke logging facility with logging module #183

Merged
merged 3 commits into from
Nov 13, 2022
Merged

Conversation

jaraco
Copy link
Member

@jaraco jaraco commented Oct 5, 2022

The distutils.log module has its fingers in nearly every module in distutils and setuptools. By addressing the long-standing TODO of adopting Python's logging module (introduced in Python 2.3 IIRC), most of the callers can decouple from distutils, greatly simplifying the landscape as Setuptools aims to absorb distutils.

@jaraco jaraco force-pushed the debt/logging branch 3 times, most recently from 0a50760 to b6d204b Compare October 5, 2022 08:00
jaraco added a commit to pypa/setuptools that referenced this pull request Oct 5, 2022
jaraco added a commit to pypa/setuptools that referenced this pull request Oct 6, 2022
clrpackages pushed a commit to clearlinux-pkgs/pypi-setuptools that referenced this pull request Oct 17, 2022
…version 65.5.0

Anderson Bravalheri (10):
      Improve warnings in editable install
      Document CWD limitations in editable docs
      Fix RST syntax in changelog
      Add news fragment
      Update validate_pyproject
      Add news fragment
      Docs: fix url
      Handle no-packages projects in editable_wheel
      Add news fragment
      Bump version: 65.4.1 → 65.5.0

Jason R. Coombs (10):
      Adopt furo theme for docs.
      Indicate to use latest Python version (workaround for readthedocs/readthedocs.org/#9623). Requires also specifying the OS version (workaround for readthedocs/readthedocs.org#9635).
      Extract context manager for suppressing logs. Ref pypa/distutils#183.
      Remove hack around distutils.dist.log, no longer relevant with pypa/distutils#183.
      Support caplog and capsys now that logs go through logging.
      Suppress monkeypatch when Log is gone, indicating distutils is using logging natively.
      Skip manifest tests as they're copy pasta and broken under a unified logging framework.
      Restore test for log module, but bypass when irrelevant.
      Restore protection of distutils.log patching.
      Update changelog. Ref #3626.

Jason Veatch (3):
      Fix #3419. Update example versions to be PEP440-compliant
      Add changelog file with towncrier news fragment.
      Update docs/userguide/distribution.rst

Mike Salvatore (2):
      Catch an edge case in expand._assert_local()
      Use abspath() instead of resolve() in expand._assert_local()

Miro Hrončok (1):
      Skip stdlib distutils tests on Python 3.12

Per Jensen (1):
      Update pyproject_config.rst

Stéphane Blondon (1):
      remove python2 cases

alexzorin (1):
      fix small error message typo in editable_wheel.py
@jaraco jaraco closed this Oct 29, 2022
@jaraco jaraco reopened this Oct 29, 2022
@jaraco jaraco force-pushed the debt/logging branch 2 times, most recently from e824025 to dce4098 Compare October 29, 2022 21:08
@jaraco
Copy link
Member Author

jaraco commented Nov 13, 2022

I'm able to replicate the failures locally by injecting this branch of distutils into setuptools and running the tests.

The error message suggests to me there may be something about the 'non-utf8' filename that's failing to serialize properly... presumably in the logging infra. It seems xdist may be implicated as well as the traceback points to execnet.gateway_base called by xdist.remote.

@jaraco
Copy link
Member Author

jaraco commented Nov 13, 2022

The error doesn't occur if xdist is disabled.

@jaraco
Copy link
Member Author

jaraco commented Nov 13, 2022

Here's the full run with the isolated test using xdist:

 setuptools main $ tox -- -p no:cov -k test_write_manifest_skips_non_utf8_filenames
python develop-inst-noop: /Users/jaraco/code/pypa/setuptools
python installed: attrs==22.1.0,autocommand==2.2.1,black==22.10.0,build==0.9.0,click==8.1.3,coverage==6.5.0,distlib==0.3.6,docutils==0.19,execnet==1.9.0,filelock==3.8.0,flake8==4.0.1,flake8-2020==1.6.1,ini2toml==0.11,iniconfig==1.1.1,jaraco.context==4.1.2,jaraco.envs==2.4.0,jaraco.functools==3.5.2,jaraco.path==3.4.0,mccabe==0.6.1,more-itertools==9.0.0,mypy==0.990,mypy-extensions==0.4.3,packaging==21.3,path==16.5.0,pathspec==0.10.2,pep517==0.13.0,pip-run==8.8.1,platformdirs==2.5.4,pluggy==1.0.0,py==1.11.0,pycodestyle==2.8.0,pyflakes==2.4.0,pyobjc==9.0,pyobjc-core==9.0,pyobjc-framework-Accessibility==9.0,pyobjc-framework-Accounts==9.0,pyobjc-framework-AddressBook==9.0,pyobjc-framework-AdServices==9.0,pyobjc-framework-AdSupport==9.0,pyobjc-framework-AppleScriptKit==9.0,pyobjc-framework-AppleScriptObjC==9.0,pyobjc-framework-ApplicationServices==9.0,pyobjc-framework-AppTrackingTransparency==9.0,pyobjc-framework-AudioVideoBridging==9.0,pyobjc-framework-AuthenticationServices==9.0,pyobjc-framework-AutomaticAssessmentConfiguration==9.0,pyobjc-framework-Automator==9.0,pyobjc-framework-AVFoundation==9.0,pyobjc-framework-AVKit==9.0,pyobjc-framework-AVRouting==9.0,pyobjc-framework-BackgroundAssets==9.0,pyobjc-framework-BusinessChat==9.0,pyobjc-framework-CalendarStore==9.0,pyobjc-framework-CallKit==9.0,pyobjc-framework-CFNetwork==9.0,pyobjc-framework-ClassKit==9.0,pyobjc-framework-CloudKit==9.0,pyobjc-framework-Cocoa==9.0,pyobjc-framework-Collaboration==9.0,pyobjc-framework-ColorSync==9.0,pyobjc-framework-Contacts==9.0,pyobjc-framework-ContactsUI==9.0,pyobjc-framework-CoreAudio==9.0,pyobjc-framework-CoreAudioKit==9.0,pyobjc-framework-CoreBluetooth==9.0,pyobjc-framework-CoreData==9.0,pyobjc-framework-CoreHaptics==9.0,pyobjc-framework-CoreLocation==9.0,pyobjc-framework-CoreMedia==9.0,pyobjc-framework-CoreMediaIO==9.0,pyobjc-framework-CoreMIDI==9.0,pyobjc-framework-CoreML==9.0,pyobjc-framework-CoreMotion==9.0,pyobjc-framework-CoreServices==9.0,pyobjc-framework-CoreSpotlight==9.0,pyobjc-framework-CoreText==9.0,pyobjc-framework-CoreWLAN==9.0,pyobjc-framework-CryptoTokenKit==9.0,pyobjc-framework-DataDetection==9.0,pyobjc-framework-DeviceCheck==9.0,pyobjc-framework-DictionaryServices==9.0,pyobjc-framework-DiscRecording==9.0,pyobjc-framework-DiscRecordingUI==9.0,pyobjc-framework-DiskArbitration==9.0,pyobjc-framework-DVDPlayback==9.0,pyobjc-framework-EventKit==9.0,pyobjc-framework-ExceptionHandling==9.0,pyobjc-framework-ExecutionPolicy==9.0,pyobjc-framework-ExtensionKit==9.0,pyobjc-framework-ExternalAccessory==9.0,pyobjc-framework-FileProvider==9.0,pyobjc-framework-FileProviderUI==9.0,pyobjc-framework-FinderSync==9.0,pyobjc-framework-FSEvents==9.0,pyobjc-framework-GameCenter==9.0,pyobjc-framework-GameController==9.0,pyobjc-framework-GameKit==9.0,pyobjc-framework-GameplayKit==9.0,pyobjc-framework-HealthKit==9.0,pyobjc-framework-ImageCaptureCore==9.0,pyobjc-framework-IMServicePlugIn==9.0,pyobjc-framework-InputMethodKit==9.0,pyobjc-framework-InstallerPlugins==9.0,pyobjc-framework-InstantMessage==9.0,pyobjc-framework-Intents==9.0,pyobjc-framework-IntentsUI==9.0,pyobjc-framework-IOSurface==9.0,pyobjc-framework-iTunesLibrary==9.0,pyobjc-framework-KernelManagement==9.0,pyobjc-framework-LatentSemanticMapping==9.0,pyobjc-framework-LaunchServices==9.0,pyobjc-framework-libdispatch==9.0,pyobjc-framework-LinkPresentation==9.0,pyobjc-framework-LocalAuthentication==9.0,pyobjc-framework-LocalAuthenticationEmbeddedUI==9.0,pyobjc-framework-MailKit==9.0,pyobjc-framework-MapKit==9.0,pyobjc-framework-MediaAccessibility==9.0,pyobjc-framework-MediaLibrary==9.0,pyobjc-framework-MediaPlayer==9.0,pyobjc-framework-MediaToolbox==9.0,pyobjc-framework-Metal==9.0,pyobjc-framework-MetalFX==9.0,pyobjc-framework-MetalKit==9.0,pyobjc-framework-MetalPerformanceShaders==9.0,pyobjc-framework-MetalPerformanceShadersGraph==9.0,pyobjc-framework-MetricKit==9.0,pyobjc-framework-MLCompute==9.0,pyobjc-framework-ModelIO==9.0,pyobjc-framework-MultipeerConnectivity==9.0,pyobjc-framework-NaturalLanguage==9.0,pyobjc-framework-NetFS==9.0,pyobjc-framework-Network==9.0,pyobjc-framework-NetworkExtension==9.0,pyobjc-framework-NotificationCenter==9.0,pyobjc-framework-OpenDirectory==9.0,pyobjc-framework-OSAKit==9.0,pyobjc-framework-OSLog==9.0,pyobjc-framework-PassKit==9.0,pyobjc-framework-PencilKit==9.0,pyobjc-framework-Photos==9.0,pyobjc-framework-PhotosUI==9.0,pyobjc-framework-PreferencePanes==9.0,pyobjc-framework-PushKit==9.0,pyobjc-framework-Quartz==9.0,pyobjc-framework-QuickLookThumbnailing==9.0,pyobjc-framework-ReplayKit==9.0,pyobjc-framework-SafariServices==9.0,pyobjc-framework-SafetyKit==9.0,pyobjc-framework-SceneKit==9.0,pyobjc-framework-ScreenCaptureKit==9.0,pyobjc-framework-ScreenSaver==9.0,pyobjc-framework-ScreenTime==9.0,pyobjc-framework-ScriptingBridge==9.0,pyobjc-framework-SearchKit==9.0,pyobjc-framework-Security==9.0,pyobjc-framework-SecurityFoundation==9.0,pyobjc-framework-SecurityInterface==9.0,pyobjc-framework-ServiceManagement==9.0,pyobjc-framework-SharedWithYou==9.0,pyobjc-framework-SharedWithYouCore==9.0,pyobjc-framework-ShazamKit==9.0,pyobjc-framework-Social==9.0,pyobjc-framework-SoundAnalysis==9.0,pyobjc-framework-Speech==9.0,pyobjc-framework-SpriteKit==9.0,pyobjc-framework-StoreKit==9.0,pyobjc-framework-SyncServices==9.0,pyobjc-framework-SystemConfiguration==9.0,pyobjc-framework-SystemExtensions==9.0,pyobjc-framework-ThreadNetwork==9.0,pyobjc-framework-UniformTypeIdentifiers==9.0,pyobjc-framework-UserNotifications==9.0,pyobjc-framework-UserNotificationsUI==9.0,pyobjc-framework-VideoSubscriberAccount==9.0,pyobjc-framework-VideoToolbox==9.0,pyobjc-framework-Virtualization==9.0,pyobjc-framework-Vision==9.0,pyobjc-framework-WebKit==9.0,pyparsing==3.0.9,pytest==7.2.0,pytest-black==0.3.12,pytest-checkdocs==2.9.0,pytest-cov==4.0.0,pytest-enabler==1.3.0,pytest-flake8==1.1.1,pytest-mypy==0.10.1,pytest-perf==0.12.1,pytest-timeout==2.1.0,pytest-xdist==3.0.2,pytz==2022.6,six==1.16.0,tempora==5.0.2,toml==0.10.2,tomli_w==1.0.0,tox==3.27.0,typing_extensions==4.4.0,virtualenv==20.16.7
python run-test-pre: PYTHONHASHSEED='3867764596'
python run-test: commands[0] | pytest -p no:cov -k test_write_manifest_skips_non_utf8_filenames
================================================================= test session starts ==================================================================
platform darwin -- Python 3.11.0, pytest-7.2.0, pluggy-1.0.0
cachedir: .tox/python/.pytest_cache
rootdir: /Users/jaraco/code/pypa/setuptools, configfile: pytest.ini
plugins: black-0.3.12, timeout-2.1.0, xdist-3.0.2, perf-0.12.1, checkdocs-2.9.0, flake8-1.1.1, mypy-0.10.1, enabler-1.3.0
gw0 [1] / gw1 [1] / gw2 [1] / gw3 [1] / gw4 [1] / gw5 [1] / gw6 [1] / gw7 [1]
INTERNALERROR> def worker_internal_error(self, node, formatted_error):
INTERNALERROR>         """
INTERNALERROR>         pytest_internalerror() was called on the worker.
INTERNALERROR>     
INTERNALERROR>         pytest_internalerror() arguments are an excinfo and an excrepr, which can't
INTERNALERROR>         be serialized, so we go with a poor man's solution of raising an exception
INTERNALERROR>         here ourselves using the formatted message.
INTERNALERROR>         """
INTERNALERROR>         self._active_nodes.remove(node)
INTERNALERROR>         try:
INTERNALERROR> >           assert False, formatted_error
INTERNALERROR> E           AssertionError: Traceback (most recent call last):
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 1440, in _write_unicode_string
INTERNALERROR> E                 as_bytes = s.encode("utf-8")
INTERNALERROR> E                            ^^^^^^^^^^^^^^^^^
INTERNALERROR> E             UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf6' in position 44: surrogates not allowed
INTERNALERROR> E             
INTERNALERROR> E             During handling of the above exception, another exception occurred:
INTERNALERROR> E             
INTERNALERROR> E             Traceback (most recent call last):
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/_pytest/main.py", line 270, in wrap_session
INTERNALERROR> E                 session.exitstatus = doit(config, session) or 0
INTERNALERROR> E                                      ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/_pytest/main.py", line 324, in _main
INTERNALERROR> E                 config.hook.pytest_runtestloop(session=session)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR> E                 return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR> E                 return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR> E                 return outcome.get_result()
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR> E                 raise ex[1].with_traceback(ex[2])
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR> E                 res = hook_impl.function(*args)
INTERNALERROR> E                       ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/xdist/remote.py", line 115, in pytest_runtestloop
INTERNALERROR> E                 self.run_one_test(torun)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/xdist/remote.py", line 131, in run_one_test
INTERNALERROR> E                 self.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR> E                 return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR> E                 return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR> E                 return outcome.get_result()
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR> E                 raise ex[1].with_traceback(ex[2])
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR> E                 res = hook_impl.function(*args)
INTERNALERROR> E                       ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/_pytest/runner.py", line 112, in pytest_runtest_protocol
INTERNALERROR> E                 runtestprotocol(item, nextitem=nextitem)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/_pytest/runner.py", line 131, in runtestprotocol
INTERNALERROR> E                 reports.append(call_and_report(item, "call", log))
INTERNALERROR> E                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/_pytest/runner.py", line 224, in call_and_report
INTERNALERROR> E                 hook.pytest_runtest_logreport(report=report)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR> E                 return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR> E                 return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR> E                 return outcome.get_result()
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR> E                 raise ex[1].with_traceback(ex[2])
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR> E                 res = hook_impl.function(*args)
INTERNALERROR> E                       ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/xdist/remote.py", line 184, in pytest_runtest_logreport
INTERNALERROR> E                 self.sendevent("testreport", data=data)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/xdist/remote.py", line 69, in sendevent
INTERNALERROR> E                 self.channel.send((name, kwargs))
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 729, in send
INTERNALERROR> E                 self.gateway._send(Message.CHANNEL_DATA, self.id, dumps_internal(item))
INTERNALERROR> E                                                                   ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 1371, in dumps_internal
INTERNALERROR> E                 return _Serializer().save(obj)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 1389, in save
INTERNALERROR> E                 self._save(obj)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 1407, in _save
INTERNALERROR> E                 dispatch(self, obj)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 1494, in save_tuple
INTERNALERROR> E                 self._save(item)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 1407, in _save
INTERNALERROR> E                 dispatch(self, obj)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 1490, in save_dict
INTERNALERROR> E                 self._write_setitem(key, value)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 1484, in _write_setitem
INTERNALERROR> E                 self._save(value)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 1407, in _save
INTERNALERROR> E                 dispatch(self, obj)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 1490, in save_dict
INTERNALERROR> E                 self._write_setitem(key, value)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 1484, in _write_setitem
INTERNALERROR> E                 self._save(value)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 1407, in _save
INTERNALERROR> E                 dispatch(self, obj)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 1480, in save_list
INTERNALERROR> E                 self._write_setitem(i, item)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 1484, in _write_setitem
INTERNALERROR> E                 self._save(value)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 1407, in _save
INTERNALERROR> E                 dispatch(self, obj)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 1494, in save_tuple
INTERNALERROR> E                 self._save(item)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 1407, in _save
INTERNALERROR> E                 dispatch(self, obj)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 1426, in save_str
INTERNALERROR> E                 self._write_unicode_string(s)
INTERNALERROR> E               File "/Users/jaraco/code/pypa/setuptools/.tox/python/lib/python3.11/site-packages/execnet/gateway_base.py", line 1442, in _write_unicode_string
INTERNALERROR> E                 raise DumpError("strings must be utf-8 encodable")
INTERNALERROR> E             execnet.gateway_base.DumpError: strings must be utf-8 encodable
INTERNALERROR> E           assert False
INTERNALERROR> 
INTERNALERROR> .tox/python/lib/python3.11/site-packages/xdist/dsession.py:190: AssertionError

================================================================ no tests ran in 2.57s =================================================================
_______________________________________________________________________ summary ________________________________________________________________________
  python: commands succeeded
  congratulations :)

@jaraco
Copy link
Member Author

jaraco commented Nov 13, 2022

The failure is triggered on line 307 of the test:

https://github.com/pypa/setuptools/blob/c75928017022ac52464ed65e9fe3a553281372fc/setuptools/tests/test_sdist.py#L304-L309

That's annoying. Github won't render the code, seemingly because it only does that for same-repo references :(.

image

@jaraco
Copy link
Member Author

jaraco commented Nov 13, 2022

The issue occurs when the filelist attempts to log an error that the filename is not utf-8 encodable.

So apparently, distutils old logging infrastructure could handle the value but the new one cannot.

Perhaps the issue was previously suppressed by the quiet context. I can see how that would make sense. While working on this change, I've seen other situations where capsys (capture stdout, stderr) was used and had to be replaced with caplog to capture the log outputs. And since the outputs aren't being captured, they're probably being serialized back to the parent process with xdist, and that's where it chokes.

@jaraco
Copy link
Member Author

jaraco commented Nov 13, 2022

Simply adding the caplog fixture to that test doesn't solve the issue.

@jaraco
Copy link
Member Author

jaraco commented Nov 13, 2022

I'm able to replicate the issue with this test file:

# test.py
import logging


def test_log_non_utf8():
    logging.getLogger().warn(
        'föö'.encode('latin-1').decode('utf-8', errors='surrogateescape')
    )

invoked with pip-run -q pytest pytest-xdist -- -m pytest -n auto test.py.

jaraco added a commit to pypa/setuptools that referenced this pull request Nov 13, 2022
@jaraco jaraco closed this Nov 13, 2022
@jaraco jaraco reopened this Nov 13, 2022
jaraco added a commit to pypa/setuptools that referenced this pull request Nov 13, 2022
jaraco added a commit to pypa/setuptools that referenced this pull request Nov 13, 2022
@jaraco jaraco closed this Nov 13, 2022
@jaraco jaraco reopened this Nov 13, 2022
@jaraco jaraco merged commit e0787fa into main Nov 13, 2022
@jaraco jaraco deleted the debt/logging branch November 13, 2022 19:34
@saimn saimn mentioned this pull request Nov 20, 2022
10 tasks
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Nov 22, 2022
v65.6.0

Changes

Sync with pypa/distutils@e0787fa, including pypa/distutils#183 updating distutils to use the Python logging framework.
clrpackages pushed a commit to clearlinux-pkgs/pypi-setuptools that referenced this pull request Nov 22, 2022
…version 65.6.0

Anderson Bravalheri (6):
      Fix misspelling in docs/userguide/development_mode.rst
      Highlight in docs: 'attr' directive needs MANIFEST.in config / SCM plugin
      Ensure pyproject.toml accepts UTF-8 in maintainers
      Minor stylistic change
      Minor stylistic change
      Minor stylistic change

Jason R. Coombs (39):
      Remove docstring in distutils
      Prefer partition for splitting a string.
      Remove readme
      Prefer relative imports for better portability.
      No need to save 'threshold' on the instance.
      Use monkeypatch to set log and threshold.
      Extract fixture for capturing logs in _util modules.
      Expand 'logs' fixture to support features needed by LoggingSilencer.
      Consolidate fixture for capturing logs. Removes LoggingSilencer.
      Prefer capsys to test.support.captured*
      Fix broken test
      Prefer caplog to mocking the logging interface.
      GHA pretty env (#67)
      or maybe not
      Remove setting of dll_libraries in Mingw32CCompiler. One call superseded the other, and the result was redundant with what happens in the parent call.
      Short circuit when MSC version is not found.
      Prefer partition to find.
      Work with ints uniformally.
      Replace if/else with a lookup.
      Move lookup out of the function
      Prefer regex search to string manipulation.
      Use RangeMap to define the ranges in one place.
      Use try/except when assigning msc_ver.
      👹 Feed the hobgoblins (delint).
      Re-paste RangeMap with newlines restored (unsure what made them disappear).
      Pin pytest to <7.2. Workaround for pypa/distutils#186.
      In TempdirManager, use pathlib and more_itertools to more simply write the text.
      Rewrite init/for/append loop as comprehension and if/else as tertiary statement.
      Pin mypy to '<0.990' due to realpython/pytest-mypy#141
      Remove the hyperlink for the Python versions badge. The PyPI badge is a better anchor for the hyperlink.
      Apply explicit_package_bases for mypy and unpin the version. Ref python/mypy#14057.
      Replace bespoke logging facility with logging module, available since Python 2.3.
      Avoid use of the distutils.log module.
      Log to the root logger, as that's the one Setuptools patches/validates.
      ⚫ Fade to black.
      Skip test under xdist as it fails after migrating to Python logging. Ref pypa/distutils#183. Workaround for pytest/pytest-xdist#843.
      Skip an additional test under xdist. Ref pypa/distutils#183. Expanded workaround for pytest-dev/pytest-xdist#843.
      Update changelog.
      Bump version: 65.5.1 → 65.6.0

Sviatoslav Sydorenko (8):
      Make the `check` job a central gate in the CI
      Allow skipping `integration-test` unconditionally
      Upload accurate coverage flags from unit tests job
      Use job id as a coverage flag
      Report coverage from Cygwin jobs
      Explicitly point Codecov at `coverage.xml`
      Use an absolute path for codecov uploads
      Add a test for maintainers w/ international emails

Zach Burnett (1):
      rename `.readthedocs.yml` to `.readthedocs.yaml` (RTD docs indicate that `.readthedocs.yml` will be deprecated) (#68)

shripadbadithe (1):
      Fix typo in docs/userguide/development_mode.rst
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant