Multiple Static Contexts?

[s0undt3ch]

Consider the following workflow.

We have project which is tested under several python versions, several OS's, and several versions of said OS's.
For example, we test(short list):

• Python 2.7:
  • CentOS 7
  • Debian 9
• Python 3.5:
  • CentOS 7
  • Debian 9
• Python 3.6:
  • CentOS 7
  • Debian 9

For each of the above we define the following contexts:

• py2
• py2.7
• py3
• py3.5
• py3.6
• centos-7
• debian-9

I recently took your new context feature for a spin and what I wanted to get was to know which chunks of code run under py2 or py3, which chunks run for CentOS 7, which run for Debian 9, etc...
See where I'm getting at?

Now, the way, we're setting the contexts is more or less like:

if MAYBE_RUN_COVERAGE:
    try:
        import coverage
        if 'COVERAGE_CONTEXTS' in os.environ:
            coverage_contexts = os.environ['COVERAGE_CONTEXTS'].split('||')
            current = coverage.Coverage.current()
            while coverage_contexts:
                context = coverage_contexts.pop(0)
                if not context:
                    continue
                current.switch_context(context)
    except ImportError:
        pass

Of course, as you might have guessed by now, the context feature doesn't work as I envisioned it. Only the last current.switch_context(context) will be used because I'm switching context on the same "frame"?

Looking at the code, it looks like switch_context(context) sets some attributes which, when covered lines are to be recorded, is what's stored in the database.

Would you be against switching/enhancing the feature to use a "stack" of contexts?
This would mean that a covered line could be under several contexts(and the html report seems to suggest that multiple contexts could be shown for a line, although I can only get it to show 1, even if I pass --context at the CLI because you concat that "static" context with the dynamic one.

Is this an approach to contexts that you didn't foresee?
Is this an approach that you actually didn't want to take in the first place?
Would you be open to improving the contexts feature to support this kind of workflow?

[nedbat]

All of the contexts you are describing sound like static contexts. They are fixed for an entire test suite run. You shouldn't need dynamic contexts at all. I'm not sure how you are using Coverage. You've shown part of some code using the Coverage API, but you haven't shown all of it.

You should be able to use the Coverage command line, and set --context, then combine all of the data files, and generate an HTML report showing the contexts for each line.

If you give more details, we can make it work.

[s0undt3ch]

Yes, these can be considered static contexts, but you can only pass a single static context right?

If the answer is no, why do we see, for example, py2|centos-7 as 1 context instead of py2 and centos-7?
Or you can actually pass several --context to coverage run? But again, the HTML report will concat those ...

What I'm after is a way to know what code runs under Py3, or Py2, or/and centos-7 or/and Debian-9, etc ...

Ie, after I merge all platforms, how can I know what was executed under Py2 alone? Or only on CentOS, regardless of the python version, etc...

This doesn't show my code usage(let's ignore that for now, my code can change) but I hope explains the goal in a better way.

[nedbat]

I see what you are saying. Right now, you can only have a single static context. You could set the context to py2|centos-7, then extract the information you need from the database.

I can see the logic of setting more than one static context, and recording lines once for each static context specified. I think that would give you what you need, but it isn't possible at the moment.