Expose CTracer's trace function

[blueyed]

I am experimenting with monkeypatching sys.settrace to always call coverage.py's trace function, i.e. after pdb.set_trace, and especially with it using sys.settrace(None) etc.

This works good already using the PyTracer, but for the CTracer it seems that its trace function needs to be made available.

I've tried the following quickly, but it crashes, i.e. it likely needs changes to the handling of args - maybe an intermediate CTracer_trace_public C function?

@ coverage/ctracer/tracer.c:1145 @ CTracer_methods[] = {
     { "reset_activity", (PyCFunction) CTracer_reset_activity, METH_VARARGS,
             PyDoc_STR("Reset the activity flag") },

+    { "trace",      (PyCFunction) CTracer_trace,        METH_VARARGS,
+            PyDoc_STR("The internal trace function") },
+
     { NULL }
 };

[nedbat]

Why not use sys.gettrace() to get the trace function?

[blueyed]

Tried that, but it did not work.

Looking at it again, it appears to be due to it setting itself up again as the trace function itself.
So a workaround would be to call sys.settrace() with the wrapper again:

import sys

origtrace = sys.gettrace()


def w(*args):
    print("trace", args)
    print("calling", origtrace)
    origtrace(*args)
    print(sys.gettrace())
    # sys.settrace(w)
    print("called")
    return w


def f():
    pass


f()
sys.settrace(w)
f()
f()

trace (<frame at 0x7f4090d17818, file 't_settrace_ctracer.py', line 16, code f>, 'call', None)
calling <coverage.CTracer object at 0x7f4090d36e70>
<coverage.CTracer object at 0x7f4090d36e70>
called

[blueyed]

So, I think it would be nice if the sys.gettrace() return value would be callable, without setting itself up again - better than exposing trace itself.

[nedbat]

Hmm, what am I doing wrong in my tests that are designed to check that sys.settrace(sys.gettrace()) will work? https://github.com/nedbat/coveragepy/blob/master/tests/test_oddball.py#L424

[blueyed]

You should assert that calling the trace function does not change the existing one..

def w(*args):
    assert sys.gettrace() == w
    origtrace(*args)
    assert sys.gettrace() == w

import sys

origtrace = sys.gettrace()


def w(*args):
    print("trace", args)

    assert sys.gettrace() == w
    origtrace(*args)
    assert sys.gettrace() == w
    return w


def f():
    pass


f()
sys.settrace(w)
f()
f()

[blueyed]

Just for clarification: I was using sys.gettrace() already to get the CTracer object already in the first place.

[blueyed]

Looks like this gets done here:

coveragepy/coverage/ctracer/tracer.c

Lines 1015 to 1017 in 820b255

	if (what == PyTrace_CALL) {
	PyEval_SetTrace((Py_tracefunc)CTracer_trace, (PyObject*)self);
	}

[nedbat]

You're ahead of me here... I'm not sure what the original problem was. You've proposed a solution, but can you show me what you were originally attempting that went wrong? The lines you found in tracer.c were designed to make getting and setting the trace function work better, which sounds like what you are doing, but you say I should get rid of those lines.

Let's start from the beginning so I can understand the whole situation.

[blueyed]

I am not saying to get rid of them - I don't know if they are necessary, but only meant that this is what's causing what I am seeing.

So, when coverage is active, I want to monkeypatch sys.settrace, to always call a custom wrapper function (set via the real sys.settrace).
This would then first call coverage.py's tracer, and then the "real" / other trace function (if any; e.g. the one that pdb/bdb sets). If sys.settrace is used with None in the code, the monkeypatched wrap_sys_settrace would only unset the "real" trace function, but the custom trace function would still keep calling coverage.py's trace function.

The idea is to have coverage also for code while/after sys.settrace is being used, which currently would just remove coverage.py's trace function.

[blueyed]

I have not tried it yet, but the workaround to install my custom settrace function again after calling CTracer's trace function would likely work (only for the "call" case anyway it seems).
It is not a problem with the PyTracer, and there my prototype worked already.

[blueyed]

Related issue: #729

[blueyed]

Additionally, I think the call to the trace function should return itself, so that it is idendical (via "is").

Some more info:

import sys


t1 = sys.gettrace()
t2 = sys.gettrace()
assert t1 is t2

sys.settrace(t1)
assert t1 is t2


def w(*args):
    print(args)
    ret = t1(*args)
    assert ret is t1, (ret, t1)


sys.settrace(w)

Results in:

(<frame at 0x7f4293484ba8, file '…/Vcs/coveragepy/coverage/control.py', line 450, code stop>, 'call', None)
Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "…/Vcs/coveragepy/coverage/__main__.py", line 8, in <module>
    sys.exit(main())
  File "…/Vcs/coveragepy/coverage/cmdline.py", line 762, in main
    status = CoverageScript().command_line(argv)
  File "…/Vcs/coveragepy/coverage/cmdline.py", line 506, in command_line
    return self.do_run(options, args)
  File "…/Vcs/coveragepy/coverage/cmdline.py", line 647, in do_run
    self.coverage.stop()
  File "…/Vcs/coveragepy/coverage/control.py", line 450, in stop
    def stop(self):
  File "t_same.py", line 15, in w
    assert ret is t1, (ret, t1)
AssertionError: (<bound method PyTracer._trace of <PyTracer at 139923915244376: 8 lines in 1 files>>, <bound method PyTracer._trace of <PyTracer at 139923915244376: 8 lines in 1 files>>)
Coverage.py warning: Trace function changed, measurement is likely wrong: None (trace-changed)

Edit: ok, "is" does not work with normal functions set via sys.settrace also, but only "==".