Severe test performance degradation when using karma-phantomjs-launcher

[NateRedding]

I am converting a set of about 561 unit tests that were previously running on JSTD to run on Karma 0.12.0 and PhantomJS 1.9.7. I'll freely admit that these unit tests are probably doing all sorts of crazy stuff and not cleaning up properly after themselves and such (I didn't write them).

The tests run pretty quick when using karma-chrome-launcher - ~23 seconds. When I switch that to karma-phantom-launcher, it takes ~20 minutes!

I saw a similar issue (about a year old) on the Karma issues page talking about a 10x slowdown mentioning tweaks such as removing console.log output (ie. console.log = function() {}) and such. I can shave off a couple of minutes if I remove that and also turn off reporting of slow tests.

Thinking maybe this was an issue with PhantomJS, I found a shell script test runner to run my tests directly on PhantomJS. It is able to run them in a little under 2 minutes with no problem. So, this leads me to believe it must be a problem with karma-phantom-launcher.

Interestingly, when I was re-running the tests to get the numbers above, I somehow had one run where everything ran in about 2.5 minutes... not sure how that happened. I immediately re-ran it and it took 20+ minutes again.

My config:

    {
        basePath: '.',
        frameworks: ['jasmine'],
        singleRun: true,
        reporters: ['dots', 'beep'],
        colors: true,
        browsers: ['PhantomJS'],
        reportSlowerThan: 500,
        plugins: [
            'karma-jasmine',
            'karma-phantomjs-launcher',
            'karma-beep-reporter'
        ],
        files: files
    }

[NateRedding]

This issue appears to be mostly environmental, but I don't know what the specific factors are. Initially, I thought it was lack of memory, but I have witnessed some test runs that ran poorly even when there should have been plenty of free physical memory. As best as I can tell, running Karma/PhantomJS when my IDE is running (IntelliJ) causes it to run extremely poorly.

[tavisrudd]

Also seeing this with phantom karma-jasmin on a very fast box with almost 30 gigs of free ram. Chrome runs the tests in 30 seconds, Phantom in over 3 minutes.

[dtabuenc]

Also seeing extreme slowdown in one of our projects. The slow down seems to happen when loading json fixtures or html templates from local file using ajax request. The ajax request is instant in chrome but a few seconds in phantom.

[chrylis]

I suspect PhantomJS. I'm seeing equivalent behavior running Java Selenium tests with PhantomJS, Chrome, and Firefox.

[dtabuenc]

My issue was solved. It was slow dns lookup when trying to lookup localhost. Make sure you have an entry for localhost pointing to 127.0.0.1 in your hosts file. This did not manifest itself while running the same tests under chrome, but did under phantom.

[chrylis]

Not the problem for me, unless also having an entry for ::1 will trigger it.

[adammiles10]

I am getting the same problem. It looks that phantomjs is taking about a second per file to serve up (from a xhr request).

+1 for fixing :)

[dignifiedquire]

Could you try with phantomjs 2.0? I could imagine it being faster :)

[DavidKDeutsch]

@dtabuenc - thanks for posting; adding localhost to my hosts file solved the issue for me as well.

[eudisd]

+1

[meg2208]

+1 (hi @eudisd)

anyone still having this problem? It is still slow for me with 2.0 and seems to get slower with each test case.

[rvignacio]

Same here, we have different set ups and tried both PhantomJS 1 and 2. Don't know if it's a launcher or PhantomJS problem.

[patrickmichalina]

I have the same issue, especially slow because of how many files I am loading using SystemJS (all XHR).

[mailmrmanoj]

+1

[santhi23]

+1

[pedrotst]

+1

[eudisd]

The solution is not to use karma, switch over to jest + webpack + React.

[ChrisProlls]

+1

[vfeinman]

How can we all be having this issue and no solution yet?

Karma + Webpack + Jasmine + PhantomJS

147 tests
PhantomJS 4:31
Chrome: 0:13

[andregs]

I replaced PhantomJS by Nightmare thanks to this guy and after that many of my problems are gone.
Also, take a loot at this recommendation:
https://github.com/angular/protractor/blob/master/docs/browser-setup.md#setting-up-phantomjs

[vfeinman]

@andregs Thank you so much for your suggestion! I'd really like to understand why PhantomJS was running so slowly,

147 tests:

• PhantomJS 4:31
• Chrome: 0:13
• Nightmare: 0:17

[patrickmichalina]

Yea - Just move away from Phantom :) seems to be the right answer

[pedrotst]

@patrickmichalina, there is no other options that runs in virtual memory. We are trying to avoid having to setup xvfb in our CI machine just for running our tests.

We are runnning out of options though since every now and then our tests will crash due to random phantomjs timeout 😢

[ckedwards]

@pedrotst I'm not sure why the better solution of headless chrome has not been mentioned yet. I don't know what this Nightmare nonsense is.

New launcher:
karma-runner/karma#2489

Example config:
karma-runner/karma#2603

I haven't tried headless chrome myself yet, but it looks promising.

[pselden]

#30 (comment) -- nightmare just runs electron.

[ArcanisCz]

@andregs very thanks for that Nightmare suggestion - its a bit of workaround but a good one!