Async test fails with timeout instead of assertion error

[gurdiga]

This test:

    it('returns the correct value', function(done) {
      var returnValue = 5;

      aPromise.then(function() {
        expect(returnValue).to.equal(42);
        done();
      });

    });

This test fails with timeout of 2000ms exceeded instead of assertion error. I guess that’s because expect() call throws an error, and the done() never gets executed, and I’m wondering if there is a better way to test this kind of code.

[NickHeiner]

Does aPromise ever resolve? If not, there's not much choice but to throw a timeout.

[gurdiga]

@NickHeiner Yes, it resolves; and then expect() finds returnValue not being equal(42) and throws.

[hallas]

@gurdiga how do you know it throws if you get a timeout and not an assertion error?

[gurdiga]

@hallas @NickHeiner Here is the running thing: http://jsfiddle.net/gurdiga/p9vmj/.

[hallas]

@gurdiga it seems to me that your promise has its own error catching. Try to add a .catch(done) to your promise and I think it'll work as intended.

[gurdiga]

@hallas Wow: that was the answer! :) aPromise.finally() seems to be the perfect fit to put done into: it also needs to be called when the promise is resolved. ;)

Thank you!

[gurdiga]

I feel stupid.

I think I finally got it: when anything throws in any of the promise’s handler functions, be it the one passed to .then(), to .catch() or to .finally(), the error is handled by the promise library. This way, the test runner never sees the real error, the done() is never called (because something before it threw an assertion error), and so the time out error is all you get from the test runner.

To get out of the promise, I use setTimeout():

    it('returns the correct value', function(done) {
      var returnValue = 5;

      aPromise.then(function() {
        setTimeout(function() {
          expect(returnValue).to.equal(42);
          done();
        });
      });
    });

I found this is the only way to get proper error messages and test runner behavior.

With done passed to .catch() or .finally() the test is considered passed in any case, so if there are assertion errors, you’ll never see them.

[DSheffield]

I ran into a similar problem, and eventually realized you shouldn't use done when testing asynchronous functions with promises, instead, just return the promise. So you should be able to do this without the timeout, e.g.:

it('returns the correct value', function() {
    var returnValue = 5;

    return aPromise.then(function() {
        expect(returnValue).to.equal(42);
    });
});

[stevenvachon]

I think that this issue still exists. I'm getting a timeout issue that cannot be resolved by returning a promise because my module does not use promises.

it("works", function(done) {
    new Something()
    .on("eventA", function(result) {
        expect(result).to.be.true;
    })
    .on("eventB", function(result) {
        expect(result).to.be.false;
        done();
    });
});

• Wrapping the instance in a Promise seems excessive.
• Wrapping each assertion in a try/catch also seems excessive and, more importantly, results in Error: done() called multiple times.

Ideas:
http://staxmanade.com/2015/11/testing-asyncronous-code-with-mochajs-and-es7-async-await/

[ScottFreeCode]

As regards the blog post's recommendations, I don't know about async function error handling, but for plain promises the try-catch recommendation is weird: the original promise attempt with no try-catch was nearly correct, it only needed to use .catch(done) instead of using the second parameter to then as done. (Granted, since Mocha has direct support for promises, you can also just return the promise, but for what it's worth...) The problem in the initial promise example wasn't absence of try-catch, it was that the second handler to then is not called with exceptions thrown by the first handler, whereas a following catch is; I don't know what the rationale was for promises to be designed that way, but it's how promises are. Additionally, if there had been multiple thens for some reason, only a single final .catch(done) would be needed, which is a point of superiority over try-catch inside the handlers (on top of the fact that .catch(done) is less boilerplatey to begin with).

As for your API:

1. Are you sure both events are being called, and in the right order? If they were not, how would your test turn that into a normal failure?
2. What normally happens to exceptions that are thrown from your event handlers? If they don't propagate out the way they do in synchronous code and instead are meant to be listened for on the API (e.g. with .on("error", function(error) {...})), then they'll never reach Mocha unless you listen for them and have the listener call done with the error (or just use done with the listener if the listener is passed the error as its first parameter, e.g. .on("error", done). Presumably that would also only need to be written once per test, rather than once per event handler, like .catch(done) in a promise.

[stevenvachon]

1. Yes, and I use an "end"/"drain" event to check if booleans in the other events were set.
2. The timeout happens. I'm trying to find a lean and clean alternative.

[ScottFreeCode]

Sorry, I still don't know how your API is supposed to report errors.