Automated tests failing in main

[richardt-engineb]

Overview

hatch -e ci.py3.10 run test is failing on the current main branch. It seems like a number of dependencies have very recently changed:

black released new version that complains about existing files:

cmd [3] | black frictionless tests --check
would reformat /workspace/frictionless-py/frictionless/types.py
would reformat /workspace/frictionless-py/frictionless/schema/types.py
would reformat /workspace/frictionless-py/tests/conftest.py
would reformat /workspace/frictionless-py/tests/resource/test_security.py

This appears to be because black has released a new version that moves to using their new "2024" style.

moto has released a new version with breaking changes

If you let black reformat the files (and that's all the issue is), then pyright complains:

cmd [4] | pyright frictionless tests
WARNING: there is a new pyright version available (v1.1.317 -> v1.1.349).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`

/workspace/frictionless-py/tests/schemes/aws/loaders/test_s3.py
  /workspace/frictionless-py/tests/schemes/aws/loaders/test_s3.py:6:18 - error: "mock_s3" is unknown import symbol (reportGeneralTypeIssues)
1 error, 0 warnings, 0 informations

This appears to be because moto v5.0.0 was released 3 days ago, including the change:

  General:
    * All decorators have been replaced with a single decorator:
      `mock_aws`

so the use of mock_s3 in some tests needs to change to mock_aws (or the older version needs to be pinned).

pytest has released a new major version with breaking changes

If you update to use mock_aws the tests setup fails with:

========================================================================================= short test summary info =========================================================================================
ERROR tests/detector/test_general.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/dialect/test_general.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/fields/test_any.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/fields/test_array.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/fields/test_boolean.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/fields/test_date.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/fields/test_datetime.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/fields/test_duration.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/fields/test_geojson.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/fields/test_geopoint.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/fields/test_integer.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/fields/test_number.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/fields/test_object.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/fields/test_string.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/fields/test_time.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/fields/test_year.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/fields/test_yearmonth.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/formats/excel/test_mapper.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/formats/html/test_parser.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/formats/sql/databases/duckdb/test_adapter.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/formats/sql/databases/mysql/test_adapter.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/formats/sql/databases/postgresql/test_adapter.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/formats/sql/test_adapter.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/indexer/test_resource.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/package/test_general.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/package/test_profile.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/resource/test_datatype.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/resource/test_general.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/resource/test_profile.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/schema/field/test_constraints.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/schema/field/test_general.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/schema/test_general.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
ERROR tests/test_helpers.py - AttributeError: 'CallSpec2' object has no attribute 'funcargs'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 33 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
===================================================================================== 33 warnings, 33 errors in 9.06s =====================================================================================

Ths appears to be because pytest has released a new major version (8.0.0) that changes how pytest.mark.parameterize works (or how plugins work more generally).

[richardt-engineb]

The last issue seems to actually be an issue with pytest-lazy-fixture being incompatible with PyTest 8.0. pytest-lazy-fixture also seems to be unmaintained unfortunately. See TvoroG/pytest-lazy-fixture#65 for others discussing this specific issue, potential fixes, etc.