pytest 3.0.0 captures all or nothing

[fenchu]

With pytest 2.1.1 I could finegrain my logs and only capture stdout etc. But when I upgraded to v3.0.0 there was "No log output captured." allover, and I'm only able switch between all or nothing the -s and --capture settings:

-s --capture=sys
 ------------------------------Captured stderr call------------------------------ 
20201120142916.606|INFO|/builds/id-pro/access/access-scim-test/tests/test_03_scim_insert.py:53|lookup 100 users took 6.34sec
20201120142916.607|INFO|/builds/id-pro/access/access-scim-test/tests/test_03_scim_insert.py:55|users already preregistered on username:[]
20201120142926.660|INFO|/builds/id-pro/access/access-scim-test/tests/test_03_scim_insert.py:79|insert 100 users took 10.02sec

 -------------------------------Captured log call-------------------------------- 
20201120142916|INFO|lookup 100 users took 6.34sec
20201120142916|INFO|users already preregistered on username:[]
20201120142926|INFO|insert 100 users took 10.02sec

How can I finegrain this in the new version, or is the a way I can hack this in a fixture.

[gnikonorov]

Hi @fenchu, thank you for opening this issue!

I had a question before I dig any deeper though. Is your issue with pytest or pytest-html?

Also, could you please paste the output of pip list and the full command you used for your reproducer?

[fenchu]

Hi

Thanks for looking into this. Would be nice to minimize test output. Or a workaround via conftest.py and fixtures.

(base) root@65005c93c5bd:/test/access-scim-test# pip freeze
ansi2html==1.6.0
astroid==2.4.2
atomicwrites==1.4.0
attrs==20.3.0
certifi==2020.11.8
cffi==1.14.3
chardet==3.0.4
colorama==0.4.4
cryptography==3.2.1
idna==2.10
iniconfig==1.1.1
isort==5.6.4
lazy-object-proxy==1.4.3
mccabe==0.6.1
packaging==20.4
pep8==1.7.1
pluggy==0.13.1
py==1.9.0
pycparser==2.20
PyJWT==1.7.1
pylint==2.6.0
pyparsing==2.4.7
pytest==6.1.2
pytest-html==3.0.0
pytest-metadata==1.10.0
requests==2.24.0
six==1.15.0
toml==0.10.2
urllib3==1.25.11
wrapt==1.12.1

pytest seem to be fine, this is what I like to log in the report: (below is ubuntu 16.04LTS container, it is exactly the same output in windows):

(base) root@65005c93c5bd:/test/access-scim-test# pytest -s --capture=sys --log-cli-level=INFO --tb=short tests/test_05_scim_filter.py -k 03 --webhost=test1
==================================================================================== test session starts =====================================================================================
platform linux -- Python 3.9.0, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /test/access-scim-test, configfile: pytest.ini
plugins: metadata-1.10.0, html-3.0.0
collected 8 items / 7 deselected / 1 selected

tests/test_05_scim_filter.py::TestClass::test03_get_users_filter_username_eq[test1]
--------------------------------------------------------------------------------------- live log call ----------------------------------------------------------------------------------------
20201124103104|INFO|loaded 100 users from file:'mockusers.json'
20201124103104|INFO|filter: 'username eQ "10019029110"'
20201124103105|INFO|totalResults returned: 1
20201124103105|INFO|itemsPerPage returned: 100
20201124103105|INFO|results counted:1
20201124103105|INFO|['109824433']
PASSED                                                                                                                                                                                 [100%]
============================================================================== 1 passed, 7 deselected in 0.66s ===============================================================================

If I use pytest-html:

(base) root@65005c93c5bd:/test/access-scim-test# pytest -s --capture=sys --log-cli-level=INFO --tb=short tests/test_05_scim_filter.py -k 03 --webhost=test1 --html=test1.html
==================================================================================== test session starts =====================================================================================
platform linux -- Python 3.9.0, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /test/access-scim-test, configfile: pytest.ini
plugins: metadata-1.10.0, html-3.0.0
collected 8 items / 7 deselected / 1 selected

tests/test_05_scim_filter.py::TestClass::test03_get_users_filter_username_eq[test1]
--------------------------------------------------------------------------------------- live log setup ---------------------------------------------------------------------------------------
20201124103220|INFO|creating directory:test1
--------------------------------------------------------------------------------------- live log call ----------------------------------------------------------------------------------------
20201124103220|INFO|loaded 100 users from file:'mockusers.json'
20201124103220|INFO|filter: 'username eQ "10019029110"'
20201124103221|INFO|totalResults returned: 1
20201124103221|INFO|itemsPerPage returned: 100
20201124103221|INFO|results counted:1
20201124103221|INFO|['109824433']
PASSED                                                                                                                                                                                 [100%]

--------------------------------------------------------------- generated html file: file:///test/access-scim-test/test1.html ----------------------------------------------------------------
============================================================================== 1 passed, 7 deselected in 1.51s ===============================================================================

and if I dig into the html file, it logs anything (lynx output of test1.html in container)

Results

   Result          Time                                                  Test                                             Description    Duration
                                                    No results found. Try to check the filters
   Passed 2020-11-24 10:32:21.286 tests/test_05_scim_filter.py::TestClass::test03_get_users_filter_username_eq[test1] filter userName eq 0.50
   -----------------------------Captured stdout setup------------------------------
   using ext fixture:screenshots:None,urls:None,_dir_:None
   -----------------------------Captured stderr setup------------------------------
   20201124103220.762|INFO|/test/access-scim-test/conftest.py:128|creating directory:test1
   -------------------------------Captured log setup-------------------------------
   20201124103220|INFO|creating directory:test1
   ------------------------------Captured stderr call------------------------------
   20201124103220.765|INFO|/test/access-scim-test/tests/test_05_scim_filter.py:36|loaded 100 users from file:'mockusers.json'
   20201124103220.766|INFO|/test/access-scim-test/tests/test_05_scim_filter.py:68|filter: 'username eQ "10019029110"'
   20201124103221.248|INFO|/test/access-scim-test/tools/scim.py:308|totalResults returned: 1
  20201124103221.249|INFO|/test/access-scim-test/tools/scim.py:310|itemsPerPage returned: 100
  20201124103221.249|INFO|/test/access-scim-test/tools/scim.py:314|results counted:1
  20201124103221.250|INFO|/test/access-scim-test/tests/test_05_scim_filter.py:70|['109824433']
   -------------------------------Captured log call--------------------------------
   20201124103220|INFO|loaded 100 users from file:'mockusers.json' 20201124103220|INFO|filter: 'username eQ "10019029110"'
  20201124103221|INFO|totalResults returned: 1
  20201124103221|INFO|itemsPerPage returned: 100 20201124103221|INFO|results counted:1
  20201124103221|INFO|['109824433']

[fenchu]

it is the same with pytest-html3..1.1:

pytest==6.2.1
pytest-base-url==1.4.2
pytest-clarity==0.3.0a0
pytest-html==3.1.1
pytest-metadata==1.10.0
pytest-mock==3.3.1
pytest-variables==1.9.0
pytest-watch==4.2.0

pytest 6.2.1 and pytest-html 2.1.1 works fine, so I'm pretty sure it is in pytest-html

[jayasimha3000]

Probably broken by changes against #171 (PR #353 and #359).

[fenchu]

I forked the v3.1.1 and added a filter so it only captures the stderr or file, also sets screenshots on a new line after log at a fixed size also clicking on screenshots open in new window and some more robust decoding of files
I have 30+ tests using this version now. the diff enclosed for the hack:

git diff 86cfadc43cb0e9068be1561ffdb7ec447f5a85d5 805e3b6b84fb50520d3e4a7c90e64639f2525173 .\src\pytest_html\result.py  
diff --git a/src/pytest_html/result.py b/src/pytest_html/result.py
index da0d240..3069057 100644
--- a/src/pytest_html/result.py
+++ b/src/pytest_html/result.py
@@ -9,8 +9,7 @@ from html import escape
 from os.path import isfile

 from _pytest.logging import _remove_ansi_escape_sequences
-from py.xml import html
-from py.xml import raw
+from py.xml import html, raw

 from . import extras
 from .util import ansi_support
@@ -80,6 +79,8 @@ class TestResult:
         return order.index(self.outcome) < order.index(other.outcome)

     def create_asset(self, content, extra_index, test_index, file_extension, mode="w"):
+        if not content:
+            return None
         asset_file_name = "{}_{}_{}.{}".format(
             re.sub(r"[^\w\.]", "_", self.test_id),
             str(extra_index),
@@ -175,14 +176,17 @@ class TestResult:
             duration_as_gmtime = time.gmtime(report.duration)
             return time.strftime(duration_formatter, duration_as_gmtime)

-    def _populate_html_log_div(self, log, report):
+    def _populate_html_log_div(self, log, report, filter:list=['stderr']):
         if report.longrepr:
             # longreprtext is only filled out on failure by pytest
             #    otherwise will be None.
             #  Use full_text if longreprtext is None-ish
             #   we added full_text elsewhere in this file.
+            # filter is either stderr or log or both in a list
+
             text = report.longreprtext or report.full_text
             for line in text.splitlines():
+
                 separator = line.startswith("_ " * 10)
                 if separator:
                     log.append(line[:80])
@@ -196,9 +200,7 @@ class TestResult:

         for section in report.sections:
             header, content = map(escape, section)
-            log.append(f" {header:-^80} ")
-            log.append(html.br())
-
+            #print(f"report.section header:\"{header}\"")
             if ansi_support():
                 converter = ansi_support().Ansi2HTMLConverter(
                     inline=False, escaped=False
@@ -207,8 +209,13 @@ class TestResult:
             else:
                 content = _remove_ansi_escape_sequences(content)

-            log.append(raw(content))
-            log.append(html.br())
+            if filter:
+                for f in filter:
+                    if re.search(f"Captured {f} ", header):
+                        log.append("-" * 10 + header + "-" * 10)
+                        log.append(html.br())
+                        log.append(raw(content))
+                        #log.append(html.br())

     def append_log_html(
         self,
@@ -250,22 +257,27 @@ class TestResult:
                 )

             html_div = html.a(
-                raw(base_extra_string.format(extra.get("content"))), href=content
+                raw(base_extra_string.format(extra.get("content"))), href=content, target="_blank"
             )
         elif self.self_contained:
             src = f"data:{extra.get('mime_type')};base64,{content}"
             html_div = raw(base_extra_string.format(src))
         else:
-            content = b64decode(content.encode("utf-8"))
-            href = src = self.create_asset(
-                content, extra_index, test_index, extra.get("extension"), "wb"
-            )
-            html_div = html.a(
-                raw(base_extra_string.format(src)),
-                class_=base_extra_class,
-                target="_blank",
-                href=href,
-            )
+            try:
+                content = b64decode(content.encode("utf-8"))
+            except Exception as e:
+                print("Error: unable to decode data")
+                return None
+            else:
+                href = src = self.create_asset(
+                    content, extra_index, test_index, extra.get("extension"), "wb"
+                )
+                html_div = html.a(
+                    raw(base_extra_string.format(src)),
+                    class_=base_extra_class,
+                    target="_blank",
+                    href=href,
+                )
         return html_div

     def _append_image(self, extra, extra_index, test_index):

[pavk88]

Hi ,

is there anything about this issue ?
it blocks me for upgrading to pytest-html v3.0.0 .

I want to capture only the log calls, instead its all or nothing indeed.

[jxcangel]

my packages are these but got same issue:

Package                       Version
----------------------------- -----------
alabaster                     0.7.12
ansi2html                     1.7.0
astroid                       2.11.6
asttokens                     2.0.5
......
pytest                        7.1.2
pytest-cov                    3.0.0
pytest-forked                 1.4.0
pytest-html                   3.1.1
pytest-icdiff                 0.5
pytest-metadata               2.0.1
pytest-mock                   3.7.0
pytest-sugar                  0.9.4
pytest-tldr                   0.2.4
pytest-xdist                  2.5.0
python-dateutil               2.8.2
python-docx                   0.8.11
python-gitlab                 3.5.0

[BeyondEvil]

Sorry that you're having issues and that this has lingered for so long.

I will make sure to revisit this for v4.x which is going to be in beta in a week or two.

Hopefully, with the complete rebuild we're doing for v4, this won't be an issue.