Skip to content

tap2junit fails on test input when tests time out #1847

Closed
Closed
tap2junit fails on test input when tests time out#1847
Labels
good first issuestale

Description

@sam-github
sam-github
opened on Jun 20, 2019
2:03:21 mkdir out/junit || true
12:03:21 tap2junit -i test.tap -o out/junit/test.xml
12:03:21 tap2junit -i cctest.tap -o out/junit/cctest.xml || true
12:03:21 rm -Rf out/Release/obj.target
12:03:21 [aix61-ppc64] $ /bin/sh -xe /tmp/jenkins5548162882024815593.sh
12:03:22 + mkdir out/junit
12:03:22 mkdir: 0653-358 Cannot create out/junit.
12:03:22 out/junit: Do not specify an existing file.
12:03:22 + true
12:03:22 + tap2junit -i test.tap -o out/junit/test.xml
12:03:27 Traceback (most recent call last):
12:03:27   File "/usr/bin/tap2junit", line 11, in <module>
12:03:27     sys.exit(main())
12:03:27   File "/opt/freeware/lib/python2.7/site-packages/tap2junit/__main__.py", line 44, in main
12:03:27     result = parse(os.path.splitext(args.input.name)[0], data)
12:03:27   File "/opt/freeware/lib/python2.7/site-packages/tap2junit/__main__.py", line 15, in parse
12:03:27     t = TestCase(test.description, None, test.yaml['duration_ms'])
12:03:27 TypeError: 'NoneType' object has no attribute '__getitem__'

I sshed in and reproduced it on AIX, then copied the file to my laptop, ubuntu, and reproduced here. tap file is https://gist.github.com/c15ba6bd9ff7b529a7aa04fa176f068a

core/build (centos7-python-3.7 $% u=) % scp  test-osuosl-aix61-ppc64_be-1:/home/iojs/issue.tap .                                                                             
issue.tap                                           100%  185KB   1.3MB/s   00:00
core/build (centos7-python-3.7 $% u=) % pip install tap2junit
Collecting tap2junit
  Downloading https://files.pythonhosted.org/packages/af/22/f8436b6d750114dd5047970ad878259edeafa89fac72b8dced476965c6d6/tap2junit-0.1.4.tar.gz                              
Collecting junit-xml (from tap2junit)
  Downloading https://files.pythonhosted.org/packages/a6/2a/f8d5aab80bb31fcc789d0f2b34b49f08bd6121cd8798d2e37f416df2b9f8/junit-xml-1.8.tar.gz                                
Collecting yamlish (from tap2junit)
  Downloading https://files.pythonhosted.org/packages/29/fd/d607e225d757aa2950abff9ba19145be3c72b74266fc48f3266ec7ba3087/yamlish-0.18.1-py2-none-any.whl                     
Requirement already satisfied: six in /home/sam/.local/lib/python2.7/site-packages (from junit-xml->tap2junit) (1.12.0)                                                      
Requirement already satisfied: PyYAML>=3.09 in /usr/lib/python2.7/dist-packages (from yamlish->tap2junit) (3.13)                                                             
Building wheels for collected packages: tap2junit, junit-xml
  Running setup.py bdist_wheel for tap2junit ... done
  Stored in directory: /home/sam/.cache/pip/wheels/12/a9/54/a5f8ed9d0adfc510303204543f743bac3fc3a1b236d4545b10                                                               
  Running setup.py bdist_wheel for junit-xml ... done
  Stored in directory: /home/sam/.cache/pip/wheels/c6/c5/65/5b166afb6eac87dc300368ec4d92f39502dd41cdc73056d49e                                                               
Successfully built tap2junit junit-xml
Installing collected packages: junit-xml, yamlish, tap2junit
  The script tap2junit is installed in '/home/sam/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.                                                          
Successfully installed junit-xml-1.8 tap2junit-0.1.4 yamlish-0.18.1
core/build (centos7-python-3.7 $% u=) % /home/sam/.local/bin/tap2junit -i issue.tap -o test.xml                                                                              
Traceback (most recent call last):
  File "/home/sam/.local/bin/tap2junit", line 10, in <module>
    sys.exit(main())
  File "/home/sam/.local/lib/python2.7/site-packages/tap2junit/__main__.py", line 44, in main                                                                                
    result = parse(os.path.splitext(args.input.name)[0], data)
  File "/home/sam/.local/lib/python2.7/site-packages/tap2junit/__main__.py", line 15, in parse                                                                               
    t = TestCase(test.description, None, test.yaml['duration_ms'])
TypeError: 'NoneType' object has no attribute '__getitem__'

Activity

  1. sam-github commented on Jun 20, 2019

    @sam-github
    sam-github
    on Jun 20, 2019
    ContributorAuthor

    Did a pip install tap.py and it processed the tap file OK, so I don't think its malformed. Also couldn't see anything about file the format visually, it looks pretty boring.

  2. sam-github commented on Jun 20, 2019

    @sam-github
    sam-github
    on Jun 20, 2019
    ContributorAuthor

    Not a one-off, also happened on https://ci.nodejs.org/job/node-test-commit-aix/nodes=aix61-ppc64/23969/console

  3. sam-github commented on Jun 20, 2019

    @sam-github
    sam-github
    on Jun 20, 2019
    ContributorAuthor

    What's happening is that if the tap file starts with a plan, like 1..2701, but then only has 2410 tests in it, tap2junit tries to append tests for each missing one, but there is a type in the comment field, and the tests lack the yaml property, so the main blows up.

    Doesn't answer "why is the tap file sometimes truncated", does anybody have an idea about that?

  4. sam-github commented on Jun 20, 2019

    @sam-github
    sam-github
    on Jun 20, 2019
    ContributorAuthor

    I think that the job is being killed, see [test-ci] Killed, possibly by jenkins because of a timeout. Does anyone have any ideas of what might be killing the test runner before it is done?

    Possibly this is another symptom of #1820, not sure.

    12:00:26 ok 2410 addons/stringbytes-external-exceed-max/test-stringbytes-external-at-max
12:00:26   ---
12:00:26   duration_ms: 113.396
12:00:26   ...
12:03:21 gmake: *** Waiting for unfinished jobs....
12:03:21 gmake[1]: *** Waiting for unfinished jobs....
12:03:21 gmake[1]: *** [test-ci] Killed
12:03:21 Build step 'Execute shell' marked build as failure

  5. addaleax commented on Jun 20, 2019

    @addaleax
    addaleax
    on Jun 20, 2019
    Member

    Does anyone have any ideas of what might be killing the test runner before it is done?

    Since it’s failing at the stringbytes-external-exceed-max tests, maybe it’s the operating system’s OOM killer? At least for Linux that would seem plausible, I obviously don’t know how that works on AIX…

  6. sam-github commented on Jun 21, 2019

    @sam-github
    sam-github
    on Jun 21, 2019
    ContributorAuthor
    • 23975: 3
    18:54:05 ok 2410 addons/stringbytes-external-exceed-max/test-stringbytes-external-at-max
18:54:05   ---
18:54:05   duration_ms: 2.951
18:54:05   ...
18:56:35 gmake: *** Waiting for unfinished jobs....
18:56:35 gmake[1]: *** Waiting for unfinished jobs....
18:56:36 Build step 'Execute shell' marked build as failure
18:56:41 Performing Post build task...
18:56:41 Match found for : : True
18:56:41 Logical operation result is TRUE
18:56:41 Running script  : #/bin/bash -x +e
18:56:41

    So, this issue looks highly correlated with that specific test. I'll look into this.

  7. sam-github commented on Jun 21, 2019

    @sam-github
    sam-github
    on Jun 21, 2019
    ContributorAuthor

    However, I'm also seeing it on other tests, so leaving this note here, too:

    • 23969: 3
    13:00:06 ok 2581 sequential/test-async-wrap-getasyncid
13:00:06   ---
13:00:06   duration_ms: 1.946
13:00:06   ...
13:02:10 gmake: *** [run-ci] Killed
13:02:10 gmake: *** wait: There are no child processes..  Stop.
13:02:11 Build step 'Execute shell' marked build as failure
    • 23968: 1 ditto

  8. sam-github commented on Jun 25, 2019

    @sam-github
    sam-github
    on Jun 25, 2019
    ContributorAuthor

    Stress test for addons/stringbytes-external-exceed-max/test-stringbytes-external-at-max: https://ci.nodejs.org/job/node-stress-single-test/2235/

  9. danbev
    added a commit that references this issue on Jul 2, 2019

    test: skip tests related to CI failures on AIX

    3c9c89d
  10. targos
    added 2 commits that reference this issue on Jul 2, 2019

    test: skip tests related to CI failures on AIX

    ec06ff4

    test: skip tests related to CI failures on AIX

    3f6f968

  11. 1 remaining item

  12. BethGriggs
    added a commit that references this issue on Jul 25, 2019

    test: skip tests related to CI failures on AIX

    39637cb
  13. sam-github
    changed the title [-]tap2junit fails on test input when test timesout[/-] [+]tap2junit fails on test input when tests time out[/+] on Aug 9, 2019
  14. sam-github
    added
    good first issue
    on Aug 9, 2019

  15. sam-github commented on Aug 9, 2019

    @sam-github
    sam-github
    on Aug 9, 2019
    ContributorAuthor

    tap2junit just needs reworking. Its crud python code, but the problem is easy to reproduce if anyone wants to take a shot at, and doesn't require access to ci, or even to node!

    Its not a priority though, because it is only a follow-on problem, caused by a previous failure.

  16. sam-github commented on Aug 9, 2019

    @sam-github
    sam-github
    on Aug 9, 2019
    ContributorAuthor

    It also may need updating for python3, see #1830 (comment)

  17. BethGriggs
    added a commit that references this issue on Sep 19, 2019

    test: skip tests related to CI failures on AIX

    930647d

  18. Trott commented on Dec 14, 2019

    @Trott
    Trott
    on Dec 14, 2019
    Member

    It also may need updating for python3, see #1830 (comment)

    tap2junit 0.1.5 is compatible with Python 3 and would seem to fix other bugs. Refs: nodejs/node#30955 (comment)

    Can we update to tap2junit 0.1.5?

  19. davidstanke commented on Feb 25, 2020

    @davidstanke
    davidstanke
    on Feb 25, 2020

    Hey folks, I'm starting to investigate this... currently setting up a local environment to reproduce. I'll report back as I learn things.

  20. github-actions commented on Dec 22, 2020

    @github-actions
    github-actionsbot
    on Dec 22, 2020

    This issue is stale because it has been open many days with no activity. It will be closed soon unless the stale label is removed or a comment is made.

  21. github-actions
    added
    stale
    on Dec 22, 2020
  22. github-actions
    closed this as completedon Jan 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    good first issuestale

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions