unit tests failing on 32 bit platform

[maribu]

Things to check first

• I have searched the existing issues and didn't find my bug already reported there

• I have checked that my bug is still present in the latest release

cbor2 version

5.6.1

Python version

3.11.7

What happened?

The unit tests are failing on 32 bit platforms:

armhf

============================= test session starts ==============================
platform linux -- Python 3.11.7, pytest-8.0.0, pluggy-1.3.0
rootdir: /builds/maribu/aports/testing/py3-cbor2/src/cbor2-5.6.1
configfile: pyproject.toml
testpaths: tests
plugins: cov-4.1.0, hypothesis-6.98.2
collected 662 items
tests/test_decoder.py .................................................. [  7%]
........................................................................ [ 18%]
......................................................FF                 [ 26%]
tests/test_encoder.py .................................................. [ 34%]
........................................................................ [ 45%]
.                                                                        [ 45%]
tests/test_types.py ......................                               [ 48%]
tests/test_decoder.py .................................................. [ 56%]
........................................................................ [ 67%]
......................................................FF                 [ 75%]
tests/test_encoder.py .................................................. [ 83%]
........................................................................ [ 94%]
.                                                                        [ 94%]
tests/test_types.py ......................                               [ 97%]
tests/test_tool.py ...............                                       [ 99%]
tests/test_types.py .                                                    [100%]
=================================== FAILURES ===================================
______________________ test_oversized_read[c-bytestring] _______________________
tests/test_decoder.py:957: in test_oversized_read
    impl.load(f)
E   _cbor2.CBORDecodeValueError: excessive bytestring size 0x7FFFFFFFFFFFFF00
________________________ test_oversized_read[c-unicode] ________________________
tests/test_decoder.py:957: in test_oversized_read
    impl.load(f)
E   _cbor2.CBORDecodeValueError: excessive string size 0x7FFFFFFFFFFFFF00
____________________ test_oversized_read[python-bytestring] ____________________
tests/test_decoder.py:957: in test_oversized_read
    impl.load(f)
cbor2/_decoder.py:855: in load
    ).decode()
cbor2/_decoder.py:234: in decode
    return self._decode()
cbor2/_decoder.py:221: in _decode
    return decoder(self, subtype)
cbor2/_decoder.py:309: in decode_bytestring
    raise CBORDecodeValueError("invalid length for bytestring 0x%x" % length)
E   cbor2._types.CBORDecodeValueError: invalid length for bytestring 0x7fffffffffffff00
_____________________ test_oversized_read[python-unicode] ______________________
tests/test_decoder.py:957: in test_oversized_read
    impl.load(f)
cbor2/_decoder.py:855: in load
    ).decode()
cbor2/_decoder.py:234: in decode
    return self._decode()
cbor2/_decoder.py:221: in _decode
    return decoder(self, subtype)
cbor2/_decoder.py:368: in decode_string
    raise CBORDecodeValueError("invalid length for string 0x%x" % length)
E   cbor2._types.CBORDecodeValueError: invalid length for string 0x7fffffffffffff00
======================== 4 failed, 658 passed in 4.94s =========================

armv7

============================= test session starts ==============================
platform linux -- Python 3.11.7, pytest-8.0.0, pluggy-1.3.0
rootdir: /builds/maribu/aports/testing/py3-cbor2/src/cbor2-5.6.1
configfile: pyproject.toml
testpaths: tests
plugins: hypothesis-6.98.2, cov-4.1.0
collected 662 items
tests/test_decoder.py .................................................. [  7%]
........................................................................ [ 18%]
......................................................FF                 [ 26%]
tests/test_encoder.py .................................................. [ 34%]
........................................................................ [ 45%]
.                                                                        [ 45%]
tests/test_types.py ......................                               [ 48%]
tests/test_decoder.py .................................................. [ 56%]
........................................................................ [ 67%]
......................................................FF                 [ 75%]
tests/test_encoder.py .................................................. [ 83%]
........................................................................ [ 94%]
.                                                                        [ 94%]
tests/test_types.py ......................                               [ 97%]
tests/test_tool.py ...............                                       [ 99%]
tests/test_types.py .                                                    [100%]
=================================== FAILURES ===================================
______________________ test_oversized_read[c-bytestring] _______________________
tests/test_decoder.py:957: in test_oversized_read
    impl.load(f)
E   _cbor2.CBORDecodeValueError: excessive bytestring size 0x7FFFFFFFFFFFFF00
________________________ test_oversized_read[c-unicode] ________________________
tests/test_decoder.py:957: in test_oversized_read
    impl.load(f)
E   _cbor2.CBORDecodeValueError: excessive string size 0x7FFFFFFFFFFFFF00
____________________ test_oversized_read[python-bytestring] ____________________
tests/test_decoder.py:957: in test_oversized_read
    impl.load(f)
cbor2/_decoder.py:855: in load
    ).decode()
cbor2/_decoder.py:234: in decode
    return self._decode()
cbor2/_decoder.py:221: in _decode
    return decoder(self, subtype)
cbor2/_decoder.py:309: in decode_bytestring
    raise CBORDecodeValueError("invalid length for bytestring 0x%x" % length)
E   cbor2._types.CBORDecodeValueError: invalid length for bytestring 0x7fffffffffffff00
_____________________ test_oversized_read[python-unicode] ______________________
tests/test_decoder.py:957: in test_oversized_read
    impl.load(f)
cbor2/_decoder.py:855: in load
    ).decode()
cbor2/_decoder.py:234: in decode
    return self._decode()
cbor2/_decoder.py:221: in _decode
    return decoder(self, subtype)
cbor2/_decoder.py:368: in decode_string
    raise CBORDecodeValueError("invalid length for string 0x%x" % length)
E   cbor2._types.CBORDecodeValueError: invalid length for string 0x7fffffffffffff00
======================== 4 failed, 658 passed in 4.64s =========================

x86

============================= test session starts ==============================
platform linux -- Python 3.11.7, pytest-8.0.0, pluggy-1.3.0
rootdir: /builds/maribu/aports/testing/py3-cbor2/src/cbor2-5.6.1
configfile: pyproject.toml
testpaths: tests
plugins: cov-4.1.0, hypothesis-6.98.2
collected 662 items
tests/test_decoder.py .................................................. [  7%]
........................................................................ [ 18%]
......................................................FF                 [ 26%]
tests/test_encoder.py .................................................. [ 34%]
........................................................................ [ 45%]
.                                                                        [ 45%]
tests/test_types.py ......................                               [ 48%]
tests/test_decoder.py .................................................. [ 56%]
........................................................................ [ 67%]
......................................................FF                 [ 75%]
tests/test_encoder.py .................................................. [ 83%]
........................................................................ [ 94%]
.                                                                        [ 94%]
tests/test_types.py ......................                               [ 97%]
tests/test_tool.py ...............                                       [ 99%]
tests/test_types.py .                                                    [100%]
=================================== FAILURES ===================================
______________________ test_oversized_read[c-bytestring] _______________________
tests/test_decoder.py:957: in test_oversized_read
    impl.load(f)
E   _cbor2.CBORDecodeValueError: excessive bytestring size 0x7FFFFFFFFFFFFF00
________________________ test_oversized_read[c-unicode] ________________________
tests/test_decoder.py:957: in test_oversized_read
    impl.load(f)
E   _cbor2.CBORDecodeValueError: excessive string size 0x7FFFFFFFFFFFFF00
____________________ test_oversized_read[python-bytestring] ____________________
tests/test_decoder.py:957: in test_oversized_read
    impl.load(f)
cbor2/_decoder.py:855: in load
    ).decode()
cbor2/_decoder.py:234: in decode
    return self._decode()
cbor2/_decoder.py:221: in _decode
    return decoder(self, subtype)
cbor2/_decoder.py:309: in decode_bytestring
    raise CBORDecodeValueError("invalid length for bytestring 0x%x" % length)
E   cbor2._types.CBORDecodeValueError: invalid length for bytestring 0x7fffffffffffff00
_____________________ test_oversized_read[python-unicode] ______________________
tests/test_decoder.py:957: in test_oversized_read
    impl.load(f)
cbor2/_decoder.py:855: in load
    ).decode()
cbor2/_decoder.py:234: in decode
    return self._decode()
cbor2/_decoder.py:221: in _decode
    return decoder(self, subtype)
cbor2/_decoder.py:368: in decode_string
    raise CBORDecodeValueError("invalid length for string 0x%x" % length)
E   cbor2._types.CBORDecodeValueError: invalid length for string 0x7fffffffffffff00
======================== 4 failed, 658 passed in 4.12s =========================

Additional Notes

On x86_64, aarch64, ppcle64, and s390x the build succeeds, which are all 64 bit platforms. That is why I assume this bug is triggered on 32 bit platforms only.

For the full build logs see:

• https://gitlab.alpinelinux.org/maribu/aports/-/jobs/1267473
• https://gitlab.alpinelinux.org/maribu/aports/-/jobs/1267472
• https://gitlab.alpinelinux.org/maribu/aports/-/jobs/1267468

How can we reproduce the bug?

gpep517 build-wheel \
    --wheel-dir .dist \
    --output-fd 3 3>&1 >&2
python3 -m venv --clear --without-pip --system-site-packages .testenv
.testenv/bin/python3 -m installer .dist/*.whl
.testenv/bin/python3 -m pytest

[agronholm]

Is there a reasonable way for me to run the test suite on a 32-bit platform on GitHub?

[maribu]

Someone in the internet claims that x86 docker images run on x86_64 hosts and the official python docker image is provided for lots of platforms, including x86. So maybe that would work?

Otherwise with multilib it is possible to build and run 32 bit apps on x86_64 hosts.

[agronholm]

Ok, I'll see if running the 32-bit Python image can reproduce this issue even on a 64-bit platform.

[agronholm]

To be clear, this isn't a bug in the code, it's a problem with the test using too high a string length in an operation that's supposed to fail.