pytest_html_results_table_html hook is not passed any data

[VegarS]

I am testing the 4.0.0rc4 release and have had some issues porting code that modifies the captured log output. (I want to hide the setup output from the log, which I have achieved without problems in the 3.x.x releases)

The pytest_html_results_table_html hook is always passed an empty data object, so it is impossible to modify the output. Printing data.html argument inside the hook always yields the same value: {'html': []}

Additionally, adding elements to data.html has no effect unless I delete something from data first (which sets data._replace_log = True).

How is this intended to work?

[BeyondEvil]

Interesting.

Mind telling me more about your use-case and giving me a concrete example of what you're doing?

@VegarS

[VegarS]

This is the full code that works in 3.1.1. It replaces the last html element in the data argument (the output log) with a more elaborate html code with checkboxes to show/hide setup and call logs separately.

def pytest_html_results_table_html(report, data):
    """ Split log into setup and call """
    if len(data) and len(data[-1]):
        # Find start-index for setup and call logs
        setup_idx = [i for i,elem in enumerate(data[-1]) if type(elem) == str and 'Captured stdout setup' in elem]
        call_idx = [i for i,elem in enumerate(data[-1]) if type(elem) == str and 'Captured stdout call' in elem]
        if not len(setup_idx):
            return
        setup_idx = setup_idx[0]
        other_data = data[-1][:setup_idx]
        new_data = html.div()
        if len(other_data):
            new_data.append(html.div(other_data, class_='log'))

        # Retrieve setup log
        setup_txt = data[-1][setup_idx+2]
        last_processed_idx = setup_idx+2
        # Create checkbox to show/hide setup log
        node_id = uuid.uuid4() # random id
        new_data.append(html.input(id=node_id, class_='toggle', type='checkbox'))
        new_data.append(html.label('Show setup output', for_=node_id, class_='lbl-toggle'))
        new_data.append(html.div(setup_txt, class_='log collapsible-content'))
        if len(call_idx):
            call_idx = call_idx[0]
            call_txt = data[-1][call_idx+2]
            last_processed_idx = call_idx+2
            # Checkbox for call log
            node_id = uuid.uuid4() # random id
            new_data.append(html.input(id=node_id, class_='toggle', type='checkbox', checked='checked'))
            new_data.append(html.label('Show test output', for_=node_id, class_='lbl-toggle'))
            new_data.append(html.div(call_txt, class_='log collapsible-content'))
        elif not len(other_data):
            new_data[0].attr.checked = 'checked' # show setup if no call log and no other data
        if last_processed_idx+1 < len(data[-1]):
            # Keep any other data that is appended
            new_data.append(html.div(data[-1][last_processed_idx+1:], class_='log'))
        data[-1] = new_data

The result looks like this:

@BeyondEvil

[BeyondEvil]

Ooh... that's pretty nice actually. 🤔

[BeyondEvil]

Does it not work in 3.2.0?

[BeyondEvil]

Damn, this is a bit tricky after the refactor...

I'm wondering if the easiest way is to wrap all the headers (Captured stdout call, Captured stdout setup, etc) in a DOM class/id.

[VegarS]

Does it not work in 3.2.0?

I believe it works in 3.2.0 too, but we are having issues with UnicodeEncodeErrors similar to #559 which keeps us at 3.1.1. The fix for the errors is to upgrade to 4.0.0, but then we hit this issue.

[BeyondEvil]

Ok.

I won't be able to fix this in a non-breaking way unfortunately. At least not in a non-brittle way.

What I can do is to make the data passed to the hook more structured.

In legacy, data contains two things:

• "Additional HTML" which is basically all extras.HTML
• Logs

So I'll make sure that the data object now contains those attributes.

• additional_html will be a list (just like before)
• the logs will be a dictionary where the headers ("Captured stdout setup" etc.) will be the keys.

Then in the JS, each header will be wrapped in a div with either some class or id that makes sense.

Thoughts @VegarS ?

[VegarS]

Hmm.. interesting. Will this allow me to achieve a similar setup as in legacy, i.e. add html for checkboxes in the hook?

[BeyondEvil]

I'm not sure, haha.

But I think you'll be able to add your logic to the additional_html list and have it work. Basically attach the checkboxes logic to the class/id of the headers/logs.

Another option is to basically relinquish all control of the log to you (the user). That way you can provide whatever HTML you want, and the frontend will render it.

Not really a fan of that type of omnipotent user-power, but might go that way.

If you're up for it, we can investigate the first option and see if we can solve it?

[VegarS]

Yes, lets give the first proposal a chance. Let me know what and when to test :)

[BeyondEvil]

Will do!

Not sure what TZ you're in. My day just started and I have to focus on my day job. But hopefully I'll have something you can test tomorrow my TZ (Stockholm).

[VegarS]

Excellent! I am in the same timezone (Oslo) and can test as part of my day job or later in the evening.

[VegarS]

Managed to work around it by retrieving the log data from report.sections and manually setting data._replace_log = True (which I believe should not be necessary and is a bug):

Code:

def pytest_html_results_table_html(report, data):
    if report.when in ('call', 'teardown'):
        setup_text = None
        call_text = None
        other_text = ''
        for title,text in report.sections:
            if 'Captured stdout setup' in title:
                setup_text = text
            elif 'Captured stdout call' in title:
                call_text = text
            else:
                other_text += text
        if setup_text is None or len(setup_text) == 0:
            return #??

        html = []
        node_id = uuid.uuid4() # random id
        html.append(f'<input id="{node_id}" class="toggle" type="checkbox">') # Setup checkbox
        html.append(f'<label for="{node_id}" class="lbl-toggle">Show setup output</label>') # Label
        html.append(f'<div class="log collapsible-content">{setup_text}</div>') # Setup log
        if call_text is not None and len(call_text):
            node_id = uuid.uuid4() # random id
            html.append(f'<input id="{node_id}" class="toggle" type="checkbox" checked>') # Call checkbox
            html.append(f'<label for="{node_id}" class="lbl-toggle">Show test output</label>') # Label
            html.append(f'<div class="log collapsible-content">{call_text}</div>') # Call log
        elif not len(other_text):
            # show setup if no call log and no other data
            html[0].replace('<input ', '<input checked ')

        if len(other_text):
            html.append('<div class="log">'+other_text+'</div>')

        data._replace_log = True
        data.append('<div>'+''.join(html)+'</div>')

[BeyondEvil]

I tried reproducing the replace _log thing, but failed.

Would you mind providing a minimal repro case?

Basically what you have above, but also test and setup to help me repro?

I still think creating a proper structure of the logs is a good thing.

[BeyondEvil]

Also, that code "pytest_html_results_table_html" is so much cleaner. 👍