Fix compilation errors on latest 3.11.0

[nedbat]

Another attempt at 3.11.0-latest compatibility. This fully compiles, but gets wrong answers for generators.

[nedbat]

The coverage failures look like in the case of a yield, we are confused about the frames, and record a filename/lineno pair using our lineno and the caller's filename, or something like that.

[nedbat]

@vstinner @markshannon, if you have any ideas :)

[vstinner]

@vstinner @markshannon, if you have any ideas :)

What's your question?

The coverage failures look like in the case of a yield, we are confused about the frames, and record a filename/lineno pair using our lineno and the caller's filename, or something like that.

Which failures?

[nedbat]

I didn't post the test failures because they can be hard to interpret, but here they are:

anypy recreate: /Users/nedbatchelder/coverage/trunk/.tox/anypy
anypy installdeps: -rrequirements/pip.pip, -rrequirements/pytest.pip
anypy develop-inst: /Users/nedbatchelder/coverage/trunk
anypy installed: attrs==21.4.0,backports.functools-lru-cache==1.6.4,-e git+ssh://git@github.com/nedbat/coveragepy.git@bac830c4692f47c6bb26a13b34034399b0c4b191#egg=coverage,decorator==5.1.1,distlib==0.3.4,execnet==1.9.0,filelock==3.6.0,flaky==3.7.0,future==0.18.2,hypothesis==6.45.3,importlib-metadata==4.11.3,iniconfig==1.1.1,packaging==21.3,platformdirs==2.5.2,pluggy==1.0.0,py==1.11.0,PyContracts @ git+https://github.com/slorg1/contracts@c5a6da27d4dc9985f68e574d20d86000880919c3,pyparsing==3.0.8,pytest==7.1.2,pytest-forked==1.4.0,pytest-xdist==2.5.0,qualname==0.1.0,six==1.16.0,sortedcontainers==2.4.0,tomli==2.0.1,typing_extensions==4.2.0,virtualenv==20.14.1,zipp==3.8.0
anypy run-test-pre: PYTHONHASHSEED='3493164639'
anypy run-test: commands[0] | python igor.py zip_mods
anypy run-test: commands[1] | python setup.py --quiet build_ext --inplace
anypy run-test: commands[2] | python -m pip install -q -e .
anypy run-test: commands[3] | python igor.py test_with_tracer c
=== CPython 3.11.0a7+ (rev e91dee87ed) with C tracer (.tox/anypy/bin/python) ===
bringing up nodes...
...............................................................................................................................F..F...............FFF.FFF. [ 13%]
.........F.....F.......F..F..F................................................................................ss.ss........ss..s.ss....................... [ 26%]
..............................................................s........................................................................................... [ 39%]
..................................................................F........................................................................................ [ 53%]
..............................s......................................................................sss.................................s........s....... [ 66%]
..........................................................s............................................................................................... [ 80%]
............................................................................s................s............................................................ [ 93%]
.............................................................................                                                                              [100%]
============================================================================ FAILURES ============================================================================
_____________________________________________________________ LoopArcTest.test_generator_expression ______________________________________________________________
[gw0] darwin -- Python 3.11.0 /Users/nedbatchelder/coverage/trunk/.tox/anypy/bin/python

self = <tests.test_arcs.LoopArcTest object at 0x111e4e850>

    def test_generator_expression(self):
        # Generator expression:
>       self.check_coverage("""\
            o = ((1,2), (3,4))
            o = (a for a in o)
            for tup in o:
                x = tup[0]
                y = tup[1]
            """,
            arcz=".1 -22 2-2 12 23 34 45 53 3.",
        )
E       AssertionError:
E         Unpredicted arcs differ: minus is expected, plus is actual
E         -
E         + (2, 4) # 24
E         + (3, 2) # 32
E
E       assert False

/Users/nedbatchelder/coverage/trunk/tests/test_arcs.py:610: AssertionError
_______________________________________________________ LoopArcTest.test_generator_expression_another_way ________________________________________________________
[gw1] darwin -- Python 3.11.0 /Users/nedbatchelder/coverage/trunk/.tox/anypy/bin/python

self = <tests.test_arcs.LoopArcTest object at 0x10931c710>

    def test_generator_expression_another_way(self):
        # https://bugs.python.org/issue44450
        # Generator expression:
>       self.check_coverage("""\
            o = ((1,2), (3,4))
            o = (a for
                 a in
                 o)
            for tup in o:
                x = tup[0]
                y = tup[1]
            """,
            arcz=".1 -22 2-2 12 25 56 67 75 5.",
        )
E       AssertionError:
E         Unpredicted arcs differ: minus is expected, plus is actual
E         -
E         + (2, 6) # 26
E         + (5, 2) # 52
E
E       assert False

/Users/nedbatchelder/coverage/trunk/tests/test_arcs.py:623: AssertionError
_____________________________________________________________________ YieldTest.test_bug_308 _____________________________________________________________________
[gw6] darwin -- Python 3.11.0 /Users/nedbatchelder/coverage/trunk/.tox/anypy/bin/python

self = <tests.test_arcs.YieldTest object at 0x107a1f290>

    @pytest.mark.xfail(
        env.PYVERSION[:5] == (3, 11, 0, 'alpha', 3),
        reason="avoid 3.11 bug: bpo46225",
    )
    def test_bug_308(self):
>       self.check_coverage("""\
            def run():
                for i in range(10):
                    yield lambda: i

            for f in run():
                print(f())
            """,
            arcz=".1 15 56 65 5.  .2 23 32 2.  -33 3-3",
        )
E       AssertionError:
E         Unpredicted arcs differ: minus is expected, plus is actual
E         -
E         + (1, 6) # 16
E         + (5, 1) # 51
E
E       assert False

/Users/nedbatchelder/coverage/trunk/tests/test_arcs.py:1204: AssertionError
---------------------------------------------------------------------- Captured stdout call ----------------------------------------------------------------------
0
1
2
3
4
5
6
7
8
9
___________________________________________________________________ YieldTest.test_coroutines ____________________________________________________________________
[gw7] darwin -- Python 3.11.0 /Users/nedbatchelder/coverage/trunk/.tox/anypy/bin/python

self = <tests.test_arcs.YieldTest object at 0x103eebf10>

    def test_coroutines(self):
>       self.check_coverage("""\
            def double_inputs():
                while len([1]):     # avoid compiler differences
                    x = yield
                    x *= 2
                    yield x

            gen = double_inputs()
            next(gen)
            print(gen.send(10))
            next(gen)
            print(gen.send(6))
            """,
            arcz=".1 17 78 89 9A AB B. .2 23 34 45 52 2.",
            arcz_missing="2.",
        )
E       AssertionError:
E         Missing arcs differ: minus is expected, plus is actual
E           (2, -1) # 2.
E         + (8, 9) # 89
E
E         Unpredicted arcs differ: minus is expected, plus is actual
E         -
E         + (1, 9) # 19
E         + (8, 1) # 81
E
E       assert False

/Users/nedbatchelder/coverage/trunk/tests/test_arcs.py:1256: AssertionError
---------------------------------------------------------------------- Captured stdout call ----------------------------------------------------------------------
20
12
_____________________________________________________________________ YieldTest.test_bug_324 _____________________________________________________________________
[gw4] darwin -- Python 3.11.0 /Users/nedbatchelder/coverage/trunk/.tox/anypy/bin/python

self = <tests.test_arcs.YieldTest object at 0x1118978d0>

    def test_bug_324(self):
        # This code is tricky: the list() call pulls all the values from gen(),
        # but each of them is a generator itself that is never iterated.  As a
        # result, the generator expression on line 3 is never entered or run.
>       self.check_coverage("""\
            def gen(inp):
                for n in inp:
                    yield (i * 2 for i in range(n))

            list(gen([1,2,3]))
            """,
            arcz=
                ".1 15 5. "     # The module level
                ".2 23 32 2. "  # The gen() function
                "-33 3-3",      # The generator expression
            arcz_missing="-33 3-3",
        )
E       AssertionError:
E         Missing arcs differ: minus is expected, plus is actual
E           (-3, 3) # -33
E           (3, -3) # 3-3
E         + (5, -1) # 5.
E
E         Unpredicted arcs differ: minus is expected, plus is actual
E         -
E         + (1, -1) # 1.
E         + (5, 1) # 51
E
E       assert False

/Users/nedbatchelder/coverage/trunk/tests/test_arcs.py:1241: AssertionError
__________________________________________________________________ YieldTest.test_yield_in_loop __________________________________________________________________
[gw5] darwin -- Python 3.11.0 /Users/nedbatchelder/coverage/trunk/.tox/anypy/bin/python

self = <tests.test_arcs.YieldTest object at 0x1111fa610>

    def test_yield_in_loop(self):
>       self.check_coverage("""\
            def gen(inp):
                for n in inp:
                    yield n

            list(gen([1,2,3]))
            """,
            arcz=".1 .2 23 2. 32 15 5.",
        )
E       AssertionError:
E         Missing arcs differ: minus is expected, plus is actual
E         -
E         + (5, -1) # 5.
E
E         Unpredicted arcs differ: minus is expected, plus is actual
E         -
E         + (1, -1) # 1.
E         + (5, 1) # 51
E
E       assert False

/Users/nedbatchelder/coverage/trunk/tests/test_arcs.py:1173: AssertionError
______________________________________________________________ YieldTest.test_padded_yield_in_loop _______________________________________________________________
[gw3] darwin -- Python 3.11.0 /Users/nedbatchelder/coverage/trunk/.tox/anypy/bin/python

self = <tests.test_arcs.YieldTest object at 0x1089aed50>

    def test_padded_yield_in_loop(self):
>       self.check_coverage("""\
            def gen(inp):
                i = 2
                for n in inp:
                    i = 4
                    yield n
                    i = 6
                i = 7

            list(gen([1,2,3]))
            """,
            arcz=".1 19 9.  .2 23 34 45 56 63 37 7.",
        )
E       AssertionError:
E         Missing arcs differ: minus is expected, plus is actual
E         -
E         + (9, -1) # 9.
E
E         Unpredicted arcs differ: minus is expected, plus is actual
E         -
E         + (1, -1) # 1.
E         + (9, 1) # 91
E
E       assert False

/Users/nedbatchelder/coverage/trunk/tests/test_arcs.py:1184: AssertionError
_________________________________________________________________ YieldTest.test_abandoned_yield _________________________________________________________________
[gw1] darwin -- Python 3.11.0 /Users/nedbatchelder/coverage/trunk/.tox/anypy/bin/python

self = <tests.test_arcs.YieldTest object at 0x1092e29d0>

    def test_abandoned_yield(self):
        # https://github.com/nedbat/coveragepy/issues/440
>       self.check_coverage("""\
            def gen():
                print("yup")
                yield "yielded"
                print("nope")

            print(next(gen()))
            """,
            lines=[1, 2, 3, 4, 6],
            missing="4",
            arcz=".1 16 6.  .2 23 34 4.",
            arcz_missing="34 4.",
        )
E       AssertionError:
E         Missing arcs differ: minus is expected, plus is actual
E           (3, 4) # 34
E           (4, -1) # 4.
E         + (6, -1) # 6.
E
E         Unpredicted arcs differ: minus is expected, plus is actual
E         -
E         + (1, -1) # 1.
E         + (6, 1) # 61
E
E       assert False

/Users/nedbatchelder/coverage/trunk/tests/test_arcs.py:1291: AssertionError
---------------------------------------------------------------------- Captured stdout call ----------------------------------------------------------------------
yup
yielded
___________________________________________________________________ YieldTest.test_yield_from ____________________________________________________________________
[gw0] darwin -- Python 3.11.0 /Users/nedbatchelder/coverage/trunk/.tox/anypy/bin/python

self = <tests.test_arcs.YieldTest object at 0x111d79350>

    def test_yield_from(self):
>       self.check_coverage("""\
            def gen(inp):
                i = 2
                for n in inp:
                    i = 4
                    yield from range(3)
                    i = 6
                i = 7

            list(gen([1,2,3]))
            """,
            arcz=".1 19 9.  .2 23 34 45 56 63 37 7.",
        )
E       AssertionError:
E         Missing arcs differ: minus is expected, plus is actual
E         -
E         + (9, -1) # 9.
E
E         Unpredicted arcs differ: minus is expected, plus is actual
E         -
E         + (1, -1) # 1.
E         + (9, 1) # 91
E
E       assert False

/Users/nedbatchelder/coverage/trunk/tests/test_arcs.py:1275: AssertionError
______________________________________________________________ MiscArcTest.test_partial_generators _______________________________________________________________
[gw5] darwin -- Python 3.11.0 /Users/nedbatchelder/coverage/trunk/.tox/anypy/bin/python

self = <tests.test_arcs.MiscArcTest object at 0x1112154d0>

    def test_partial_generators(self):
        # https://github.com/nedbat/coveragepy/issues/475
        # Line 2 is executed completely.
        # Line 3 is started but not finished, because zip ends before it finishes.
        # Line 4 is never started.
>       cov = self.check_coverage("""\
            def f(a, b):
                c = (i for i in a)          # 2
                d = (j for j in b)          # 3
                e = (k for k in b)          # 4
                return dict(zip(c, d))

            f(['a', 'b'], [1, 2, 3])
            """,
            arcz=".1 17 7.  .2 23 34 45 5.  -22 2-2  -33 3-3  -44 4-4",
            arcz_missing="3-3 -44 4-4",
        )
E       AssertionError:
E         Missing arcs differ: minus is expected, plus is actual
E           (-4, 4) # -44
E           (3, -3) # 3-3
E           (4, -4) # 4-4
E         + (5, -1) # 5.
E
E         Unpredicted arcs differ: minus is expected, plus is actual
E         -
E         + (3, -1) # 3.
E         + (5, 2) # 52
E
E       assert False

/Users/nedbatchelder/coverage/trunk/tests/test_arcs.py:1622: AssertionError
____________________________________________________________________ AsyncTest.test_async_for ____________________________________________________________________
[gw2] darwin -- Python 3.11.0 /Users/nedbatchelder/coverage/trunk/.tox/anypy/bin/python

self = <tests.test_arcs.AsyncTest object at 0x105494950>

    @xfail_eventlet_670
    def test_async_for(self):
>       self.check_coverage("""\
            import asyncio

            class AsyncIteratorWrapper:                 # 3
                def __init__(self, obj):                # 4
                    self._it = iter(obj)

                def __aiter__(self):                    # 7
                    return self

                async def __anext__(self):              # A
                    try:
                        return next(self._it)
                    except StopIteration:
                        raise StopAsyncIteration

            async def doit():                           # G
                async for letter in AsyncIteratorWrapper("abc"):
                    print(letter)
                print(".")

            loop = asyncio.new_event_loop()             # L
            loop.run_until_complete(doit())
            loop.close()
            """,
            arcz=
                ".1 13 3G GL LM MN N. "     # module main line
                "-33 34 47 7A A-3 "         # class definition
                "-GH HI IH HJ J-G "         # doit
                "-45 5-4 "                  # __init__
                "-78 8-7 "                  # __aiter__
                "-AB BC C-A DE E-A ",       # __anext__
            arcz_unpredicted="CD",
        )
E       AssertionError:
E         Missing arcs differ: minus is expected, plus is actual
E         -
E         + (17, 18) # HI
E         + (17, 19) # HJ
E
E         Unpredicted arcs differ: minus is expected, plus is actual
E         + (10, 18) # AI
E         + (10, 19) # AJ
E           (12, 13) # CD
E         + (17, 10) # HA
E
E       assert False

/Users/nedbatchelder/coverage/trunk/tests/test_arcs.py:1861: AssertionError
---------------------------------------------------------------------- Captured stdout call ----------------------------------------------------------------------
a
b
c
.
____________________________________________________________________ AsyncTest.test_bug_1176 _____________________________________________________________________
[gw1] darwin -- Python 3.11.0 /Users/nedbatchelder/coverage/trunk/.tox/anypy/bin/python

self = <tests.test_arcs.AsyncTest object at 0x10929ea10>

    @pytest.mark.xfail(
        (3, 10, 0, "alpha", 0, 0) <= env.PYVERSION <= (3, 10, 0, "beta", 4, 0),
        reason="avoid a 3.10 bug fixed after beta 4: 44622"
    )
    @xfail_eventlet_670
    @pytest.mark.skipif(env.PYVERSION < (3, 7), reason="need asyncio.run")
    def test_bug_1176(self):
>       self.check_coverage("""\
            import asyncio

            async def async_gen():
                yield 4

            async def async_test():
                async for i in async_gen():
                    print(i + 8)

            asyncio.run(async_test())
            """,
            arcz=".1 13 36 6A A.  -34 4-3  -67 78 87 7-6",
        )
E       AssertionError:
E         Missing arcs differ: minus is expected, plus is actual
E         -
E         + (7, 8) # 78
E
E         Unpredicted arcs differ: minus is expected, plus is actual
E         -
E         + (3, 8) # 38
E         + (7, 3) # 73
E
E       assert False

/Users/nedbatchelder/coverage/trunk/tests/test_arcs.py:1966: AssertionError
---------------------------------------------------------------------- Captured stdout call ----------------------------------------------------------------------
12
____________________________________________________________________ AsyncTest.test_bug_1158 _____________________________________________________________________
[gw2] darwin -- Python 3.11.0 /Users/nedbatchelder/coverage/trunk/.tox/anypy/bin/python

self = <tests.test_arcs.AsyncTest object at 0x105495c50>

    @pytest.mark.skipif(env.PYVERSION[:2] == (3, 9), reason="avoid a 3.9 bug: 44621")
    @pytest.mark.skipif(env.PYVERSION < (3, 7), reason="need asyncio.run")
    def test_bug_1158(self):
>       self.check_coverage("""\
            import asyncio

            async def async_gen():
                yield 4

            async def async_test():
                global a
                a = 8
                async for i in async_gen():
                    print(i + 10)
                else:
                    a = 12

            asyncio.run(async_test())
            assert a == 12
            """,
            arcz=".1 13 36 6E EF F.  -34 4-3  -68 89 9A 9C A9 C-6",
        )
E       AssertionError:
E         Missing arcs differ: minus is expected, plus is actual
E         -
E         + (9, 10) # 9A
E
E         Unpredicted arcs differ: minus is expected, plus is actual
E         -
E         + (3, 10) # 3A
E         + (9, 3) # 93
E
E       assert False

/Users/nedbatchelder/coverage/trunk/tests/test_arcs.py:1936: AssertionError
---------------------------------------------------------------------- Captured stdout call ----------------------------------------------------------------------
14
______________________________________________________________________ AsyncTest.test_async ______________________________________________________________________
[gw4] darwin -- Python 3.11.0 /Users/nedbatchelder/coverage/trunk/.tox/anypy/bin/python

self = <tests.test_arcs.AsyncTest object at 0x1118b80d0>

    @xfail_eventlet_670
    def test_async(self):
>       self.check_coverage("""\
            import asyncio

            async def compute(x, y):                        # 3
                print(f"Compute {x} + {y} ...")
                await asyncio.sleep(0.001)
                return x + y                                # 6

            async def print_sum(x, y):                      # 8
                result = (0 +
                            await compute(x, y)             # A
                )
                print(f"{x} + {y} = {result}")

            loop = asyncio.new_event_loop()                 # E
            loop.run_until_complete(print_sum(1, 2))
            loop.close()                                    # G
            """,
            arcz=
                ".1 13 38 8E EF FG G. " +
                "-34 45 56 6-3 " +
                "-89 9C C-8",
        )
E       AssertionError:
E         Unpredicted arcs differ: minus is expected, plus is actual
E         -
E         + (5, 636) # 5?
E         + (9, 3) # 93
E
E       assert False

/Users/nedbatchelder/coverage/trunk/tests/test_arcs.py:1834: AssertionError
---------------------------------------------------------------------- Captured stdout call ----------------------------------------------------------------------
Compute 1 + 2 ...
1 + 2 = 3
======================================================================== warnings summary ========================================================================
tests/test_setup.py::SetupPyTest::test_more_metadata
  /Users/nedbatchelder/coverage/trunk/.tox/anypy/lib/python3.11/site-packages/pkg_resources/_vendor/pyparsing.py:87: DeprecationWarning: module 'sre_constants' is deprecated
    import sre_constants

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
==================================================================== short test summary info =====================================================================
FAILED tests/test_arcs.py::LoopArcTest::test_generator_expression - AssertionError:
FAILED tests/test_arcs.py::LoopArcTest::test_generator_expression_another_way - AssertionError:
FAILED tests/test_arcs.py::YieldTest::test_bug_308 - AssertionError:
FAILED tests/test_arcs.py::YieldTest::test_coroutines - AssertionError:
FAILED tests/test_arcs.py::YieldTest::test_bug_324 - AssertionError:
FAILED tests/test_arcs.py::YieldTest::test_yield_in_loop - AssertionError:
FAILED tests/test_arcs.py::YieldTest::test_padded_yield_in_loop - AssertionError:
FAILED tests/test_arcs.py::YieldTest::test_abandoned_yield - AssertionError:
FAILED tests/test_arcs.py::YieldTest::test_yield_from - AssertionError:
FAILED tests/test_arcs.py::MiscArcTest::test_partial_generators - AssertionError:
FAILED tests/test_arcs.py::AsyncTest::test_async_for - AssertionError:
FAILED tests/test_arcs.py::AsyncTest::test_bug_1176 - AssertionError:
FAILED tests/test_arcs.py::AsyncTest::test_bug_1158 - AssertionError:
FAILED tests/test_arcs.py::AsyncTest::test_async - AssertionError:
14 failed, 1123 passed, 19 skipped, 1 warning in 30.06s
ERROR: InvocationError for command /Users/nedbatchelder/coverage/trunk/.tox/anypy/bin/python igor.py test_with_tracer c (exited with code 1)
____________________________________________________________________________ summary _____________________________________________________________________________
ERROR:   anypy: commands failed

Search there in particular for 636:

______________________________________________________________________ AsyncTest.test_async ______________________________________________________________________
[gw4] darwin -- Python 3.11.0 /Users/nedbatchelder/coverage/trunk/.tox/anypy/bin/python

self = <tests.test_arcs.AsyncTest object at 0x1118b80d0>

    @xfail_eventlet_670
    def test_async(self):
>       self.check_coverage("""\
            import asyncio

            async def compute(x, y):                        # 3
                print(f"Compute {x} + {y} ...")
                await asyncio.sleep(0.001)
                return x + y                                # 6

            async def print_sum(x, y):                      # 8
                result = (0 +
                            await compute(x, y)             # A
                )
                print(f"{x} + {y} = {result}")

            loop = asyncio.new_event_loop()                 # E
            loop.run_until_complete(print_sum(1, 2))
            loop.close()                                    # G
            """,
            arcz=
                ".1 13 38 8E EF FG G. " +
                "-34 45 56 6-3 " +
                "-89 9C C-8",
        )
E       AssertionError:
E         Unpredicted arcs differ: minus is expected, plus is actual
E         -
E         + (5, 636) # 5?
E         + (9, 3) # 93
E
E       assert False

/Users/nedbatchelder/coverage/trunk/tests/test_arcs.py:1834: AssertionError
---------------------------------------------------------------------- Captured stdout call ----------------------------------------------------------------------
Compute 1 + 2 ...
1 + 2 = 3

There is definitely not a line 636 in the code under test, which makes me think about frame-level confusion.

[nedbat]

@markshannon: some more information. I defined WHAT_LOG in coverage.py's C code, to log the events seen by the trace function. For the test_async code shown above, I see this with the tip of CPython (just a few lines from the middle of a long output):

b8def70 trace: f:c0303c0 CALL @ /usr/local/cpython/lib/python3.11/asyncio/events.py 78
b8def70 trace: f:c0303c0 LINE @ /usr/local/cpython/lib/python3.11/asyncio/events.py 79
b8def70 trace: f:c0303c0 LINE @ /usr/local/cpython/lib/python3.11/asyncio/events.py 80
b8def70 trace: f:c0304a0 LINE @ /Users/nedbatchelder/coverage/trunk/badasync.py 8
b8def70 trace: f:c0304a0 CALL @ /Users/nedbatchelder/coverage/trunk/badasync.py 8
b8def70 trace: f:c0304a0 LINE @ /Users/nedbatchelder/coverage/trunk/badasync.py 9
b8def70 trace: f:c0304a0 LINE @ /Users/nedbatchelder/coverage/trunk/badasync.py 10
b8def70 trace: f:bfa0a00 LINE @ /Users/nedbatchelder/coverage/trunk/badasync.py 3
b8def70 trace: f:bfa0a00 CALL @ /Users/nedbatchelder/coverage/trunk/badasync.py 3
b8def70 trace: f:bfa0a00 LINE @ /Users/nedbatchelder/coverage/trunk/badasync.py 4
Compute 1 + 2 ...
b8def70 trace: f:bfa0a00 LINE @ /Users/nedbatchelder/coverage/trunk/badasync.py 5                  ***
b8def70 trace: f:c030580 LINE @ /usr/local/cpython/lib/python3.11/asyncio/tasks.py 636             ***
b8def70 trace: f:c030580 CALL @ /usr/local/cpython/lib/python3.11/asyncio/tasks.py 636
b8def70 trace: f:c030580 LINE @ /usr/local/cpython/lib/python3.11/asyncio/tasks.py 638
b8def70 trace: f:c030580 LINE @ /usr/local/cpython/lib/python3.11/asyncio/tasks.py 642
b8def70 trace: f:c030580 LINE @ /usr/local/cpython/lib/python3.11/asyncio/tasks.py 643
b8def70 trace: f:c00e1f0 CALL @ /usr/local/cpython/lib/python3.11/asyncio/base_events.py 425
b8def70 trace: f:c00e1f0 LINE @ /usr/local/cpython/lib/python3.11/asyncio/base_events.py 427
b8def70 trace: f:bc93e20 CALL @ /usr/local/cpython/lib/python3.11/asyncio/base_events.py 1927
b8def70 trace: f:bc93e20 LINE @ /usr/local/cpython/lib/python3.11/asyncio/base_events.py 1928
b8def70 trace: f:bc93e20 RET  @ /usr/local/cpython/lib/python3.11/asyncio/base_events.py 1928
b8def70 trace: f:c00e1f0 RET  @ /usr/local/cpython/lib/python3.11/asyncio/base_events.py 427

The same part of the output when running under 3.10 is:

3f30030 trace: f:3fc2ec0 CALL @ /usr/local/pyenv/pyenv/versions/3.10.4/lib/python3.10/asyncio/events.py 78
3f30030 trace: f:3fc2ec0 LINE @ /usr/local/pyenv/pyenv/versions/3.10.4/lib/python3.10/asyncio/events.py 79
3f30030 trace: f:3fc2ec0 LINE @ /usr/local/pyenv/pyenv/versions/3.10.4/lib/python3.10/asyncio/events.py 80
3f30030 trace: f:41e4900 CALL @ /Users/nedbatchelder/coverage/trunk/badasync.py 8
3f30030 trace: f:41e4900 LINE @ /Users/nedbatchelder/coverage/trunk/badasync.py 9
3f30030 trace: f:41e4900 LINE @ /Users/nedbatchelder/coverage/trunk/badasync.py 10
3f30030 trace: f:41e4c80 CALL @ /Users/nedbatchelder/coverage/trunk/badasync.py 3
3f30030 trace: f:41e4c80 LINE @ /Users/nedbatchelder/coverage/trunk/badasync.py 4
Compute 1 + 2 ...
3f30030 trace: f:41e4c80 LINE @ /Users/nedbatchelder/coverage/trunk/badasync.py 5
3f30030 trace: f:4214220 CALL @ /usr/local/pyenv/pyenv/versions/3.10.4/lib/python3.10/asyncio/tasks.py 593
3f30030 trace: f:4214220 LINE @ /usr/local/pyenv/pyenv/versions/3.10.4/lib/python3.10/asyncio/tasks.py 595
3f30030 trace: f:4214220 LINE @ /usr/local/pyenv/pyenv/versions/3.10.4/lib/python3.10/asyncio/tasks.py 599
3f30030 trace: f:4214220 LINE @ /usr/local/pyenv/pyenv/versions/3.10.4/lib/python3.10/asyncio/tasks.py 600
3f30030 trace: f:40cfac0 CALL @ /usr/local/pyenv/pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py 427
3f30030 trace: f:40cfac0 LINE @ /usr/local/pyenv/pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py 429
3f30030 trace: f:40cb6f0 CALL @ /usr/local/pyenv/pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py 1914
3f30030 trace: f:40cb6f0 LINE @ /usr/local/pyenv/pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py 1915
3f30030 trace: f:40cb6f0 RET  @ /usr/local/pyenv/pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py 1915
3f30030 trace: f:40cfac0 RET  @ /usr/local/pyenv/pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py 429

3.11 seems to be sending two consecutive LINE events (marked with stars), but in different frames.

[markshannon]

It looks like there is an extra LINE event before the CALL event. I've opened python/cpython#92236

[markshannon]

Is there anything special about the code in /Users/nedbatchelder/coverage/trunk/badasync.py?

[nedbat]

Is there anything special about the code in /Users/nedbatchelder/coverage/trunk/badasync.py?

It's just the code from the test, as a separate file:

import asyncio

async def compute(x, y):                        # 3
    print(f"Compute {x} + {y} ...")
    await asyncio.sleep(0.001)
    return x + y                                # 6

async def print_sum(x, y):                      # 8
    result = (0 +
                await compute(x, y)             # A
    )
    print(f"{x} + {y} = {result}")

loop = asyncio.new_event_loop()                 # E
loop.run_until_complete(print_sum(1, 2))
loop.close()

[markshannon]

Thanks.

[nedbat]

@markshannon: for the generator tests, it seems like there's a change in the initial events:

Tip of main shows:

b382f70 trace: f:b383040 CALL @ /Users/nedbatchelder/coverage/trunk/gen.py -1   ***
b382f70 trace: f:b383040 LINE @ /Users/nedbatchelder/coverage/trunk/gen.py 1
b382f70 trace: f:b383040 LINE @ /Users/nedbatchelder/coverage/trunk/gen.py 5
b382f70 trace: f:b779750 LINE @ /Users/nedbatchelder/coverage/trunk/gen.py 1    ***
b382f70 trace: f:b779750 CALL @ /Users/nedbatchelder/coverage/trunk/gen.py 1
b382f70 trace: f:b779750 LINE @ /Users/nedbatchelder/coverage/trunk/gen.py 2
b382f70 trace: f:b779750 RET  @ /Users/nedbatchelder/coverage/trunk/gen.py 2
b382f70 trace: f:b779750 CALL @ /Users/nedbatchelder/coverage/trunk/gen.py 2
b382f70 trace: f:b779750 LINE @ /Users/nedbatchelder/coverage/trunk/gen.py 3
b382f70 trace: f:b779750 RET  @ /Users/nedbatchelder/coverage/trunk/gen.py 3
b382f70 trace: f:b779750 CALL @ /Users/nedbatchelder/coverage/trunk/gen.py 3
b382f70 trace: f:b779750 RET  @ /Users/nedbatchelder/coverage/trunk/gen.py 3
[1, 2]
b382f70 trace: f:b383040 RET  @ /Users/nedbatchelder/coverage/trunk/gen.py 5

while 3.10 shows:

b3fc030 trace: f:b419170 CALL @ /Users/nedbatchelder/coverage/trunk/gen.py 1    ***
b3fc030 trace: f:b419170 LINE @ /Users/nedbatchelder/coverage/trunk/gen.py 1
b3fc030 trace: f:b419170 LINE @ /Users/nedbatchelder/coverage/trunk/gen.py 5
b3fc030 trace: f:b066140 CALL @ /Users/nedbatchelder/coverage/trunk/gen.py 1
b3fc030 trace: f:b066140 LINE @ /Users/nedbatchelder/coverage/trunk/gen.py 2
b3fc030 trace: f:b066140 RET  @ /Users/nedbatchelder/coverage/trunk/gen.py 2
b3fc030 trace: f:b066140 CALL @ /Users/nedbatchelder/coverage/trunk/gen.py 2
b3fc030 trace: f:b066140 LINE @ /Users/nedbatchelder/coverage/trunk/gen.py 3
b3fc030 trace: f:b066140 RET  @ /Users/nedbatchelder/coverage/trunk/gen.py 3
b3fc030 trace: f:b066140 CALL @ /Users/nedbatchelder/coverage/trunk/gen.py 3
b3fc030 trace: f:b066140 RET  @ /Users/nedbatchelder/coverage/trunk/gen.py 3
[1, 2]
b3fc030 trace: f:b419170 RET  @ /Users/nedbatchelder/coverage/trunk/gen.py 5

The starred lines are different.

gen.py is:

def gen():
    yield 1
    yield 2

print(list(gen()))

[markshannon]

I'm struggling to reproduce this.
This test (for test.test_sys_settrace) passes for both 3.10 and 3.11

    def test_gen_in_list(self):

        def func():
            def gen():
                yield 1
                yield 2
            list(gen())

        self.run_and_compare(func,
            [(0, 'call'),
             (1, 'line'),
             (4, 'line'),
             (1, 'call'),
             (2, 'line'),
             (2, 'return'),
             (2, 'call'),
             (3, 'line'),
             (3, 'return'),
             (3, 'call'),
             (3, 'return'),
             (4, 'return')])

[nedbat]

I'm struggling to reproduce this.
This test (for test.test_sys_settrace) passes for both 3.10 and 3.11

Yes, Python trace functions get the same sequence of events on 3.10 and 3.11. My C trace function does not.

[hrnciar]

Hello, I've tried this patch and I can confirm that it works. It fixed about 50 Python packages that were previously broken by coverage. Thank you!

[nedbat]

This is merged into master.