describe.only matches more that one describe test suite

[DinisCruz]

If I have these two test suites (just showing the first one)

describe 'Boolean',->

  it 'is_True',->
    expect(true.is_True    ).to.be.an('Function')
    expect(false.is_True   ).to.be.an('Function')
    expect(true.is_True()  ).to.be.true
    expect(false.is_True() ).to.be.false
    expect((1==1).is_True()).to.be.true
    expect((1==2).is_True()).to.be.false
    expect((1!=1).is_True()).to.be.false

and

describe 'Assert | Boolean',->
  it 'assert_Is_True', ->
    true.assert_Is_True.assert_Is_Function()
    true.assert_Is_True().assert_Is_True()
    (-> false.assert_Is_True()).assert_Throws()

If I add .only to the first one :

describe.only 'Boolean',->

both will now be executed:

and I would expect only the first one to execute

I assuming that mocha behind the scenes is not keeping a pointer to the actual tests to execute, it is just keeping the name/string, which is then searched on all tests/describes loaded (is that correct?)

[dasilvacontin]

Are the two suites in the same file or in different files?

[DinisCruz]

different files

[dasilvacontin]

describe.only current only works per file. Not sure if it's intended design or a bug.

If you had tons of test files, making only work across all files would mean having to go through all of them to see if any of them tries to register a suite or a test-case using only. And with delayed suite creation, that could mean quite a lot of time.

What about using --grep?

[DinisCruz]

my workflow is actually to have all tests in a runnable state and then only add the .only for the method or test suite I want to code (which makes it very efficient)

And it is easy to spot if there is an .only left behind because not all tests will run

[dasilvacontin]

What we could do is:
• go over each file. For each file, register suite/tests, check for only, destroy. (probably some people cant afford having all tests in memory at the same time, and we can use threads for checking the files)
• apply the only grep to all files, and proceed as normal

[DinisCruz]

While you are doing that, another behaviour that happens which is breaks the test workflow is the fact that the .only on the describe overrides the .only on the it

Ideally the It should 'win over', in fact maybe even capture 'all its' that are flagged and only run these. This will be the equivalent of 'just run theses tests'.

My workflow is usually:

1. have WebStorm to run tests on all changes (with a 2 sec delay)
2. add an .only to an 'it' (i.e. test) that I want to work on
3. code that test, in a really nice environment with just about realtime code execution
4. when I'm done with my test, I usually want to work on another test, BUT I can't have the .only on the describe, so as soon as take the .only from the test I'm working on, I have 2 seconds to add the .only to another test of the describe (before the full test execution runs)

Does that make sense?

[dasilvacontin]

Yeah, the inner it should win over the describe.

What happens if you don't add .only in 2 seconds?

Wouldn't it be easier to add [wip] to the title and use --grep? Or use the new tag API? (if that made it into the v2.1 already)

[DinisCruz]

well the whole test suite starts to runs which is not a massive issue since it is quick (about 15secs (including on UI driven tests, like the ones you can see here), and I can also stop it. But it would be nicer to not have that triggered.

The reason I don't like the --grep option is that I prefer not to change the full test execution config (on WebStorm and on package.config)

Regarding your comment on:

Wouldn't it be easier to add [wip] to the title and use --grep? Or use the new tag API? (if that made it
into the v2.1 already)

I'm not sure if I am fully aware of all those options and capabilities. Can you point me to a doc/article/blog about it?

Thanks

[dasilvacontin]

Okay, sorry about the tags thing, the PR hasn't been finished yet.

Can't you make the tests run only when you save the file?

We'll discuss .only behaviour, thanks for submitting the issue!

[boneskull]

The reason I don't like the --grep option is that I prefer not to change the full test execution config (on WebStorm and on package.config)

If --grep is a valid workaround, then that's the solution you should go with for now.

The forthcoming tagging API will help you "tag" specs and suites so you can more efficiently run a subset of tests.

[boneskull]

FWIW I don't really like the .only behavior as it's something of a misnomer. I think most people would prefer it to run one thing and one thing only.

[dasilvacontin]

I think most people would prefer it to run one thing and one thing only.

That's what makes sense. I think more than a few people must have thought 'wth' when they realised current behaviour.

[boneskull]

I'm going to tag this as Bug. I can't think of an efficient, elegant solution off of the top of my head, so I'd be willing to look at PRs for this.