Make easier to generate coverage (And support sourceMaps)

[jhm-ciberman]

Actually, it's pretty hard to generate coverage with alsatian. It would be nice to have a --coverage CLI option like Jest test runner.
Also, the coverage should support typescript files and use the tsc generated sourceMaps.
It could use instanbul internally instead of depending as an external tool

[jamesadarich]

Hey @jhm-ciberman we've got a page on the wiki about coverage. Does this work for you?

[jhm-ciberman]

Yes. I actually use this method (sigtly modified as I use a .nycrc.json file to store the nyc config). But what I mean is to have a integrated comand line option to do the coverage automatically without having to setup nyc manually.
For example, the test runner Jest do it automatically if you pass the --coverage CLI option.
The basic idea would be that if you call alsatian **/**.spec.js --coverage, alsatian would call internally nyc alsatian **/**.spec.js. This will create less conflict and faster setup times for new projects.

[jamesadarich]

@jhm-ciberman sure makes sense let's investigate this! Thanks for your input and support so far!! :)

[cdibbs]

I agree. I am similarly struggling with ts-node + code coverage (nyc).

Anything that can be done to simplify TypeScript testing and code coverage would be a big boon. Even just a wiki page for each of the handful of common tools would save a lot of time.

[cdibbs]

Oy. Four hours later, and I'm cycling between two poles. First pole, the coverage seems to work fine, but not all files are included (despite trying a variety of nyc config incantations and dependency versions). Second, coverage includes all files, but the source maps seem all scrambled.

Has anyone gotten coverage working with NYC + Alsatian? Thanks in advance.

[jamesadarich]

Hey @cdibbs looks like someone has had a similar problem that has a solution here Izhaki/Typescript-Mocha-Istanbul-Boilerplate#2 (comment)

Obviously replace the mocha with your Alsatian call but is this of any use?

[cdibbs]

@jamesrichford Thanks for the quick reply. That was roughly what I was doing. But, if you look closely at the mocha command, you see that it's registering ts-node as a compiler. This is magical to me; in the past, it's just worked. Just now, I went and took a look at the ts-node source, and indeed, the exported register method does lots of things I don't want to try to understand right now. :-)

So, I am unsure whether what I want to do is currently possible. The closest I've come is to omit the ts-node registration from nyc's config, and simply run nyc with this configuration in my package.json:

  "scripts": {
    "precover": "rimraf ./coverage/*",
    "cover": "nyc npm run unittest",
    "unittest": "ts-node --project tsconfig-xunit.json src/spec-lib/runner.ts"    
  },
  "nyc": {
    "include": [
      "src/lib/**"
    ],
    "extension": [
      ".ts"
    ],
    "exclude": [
      "typings",
      "**/*.d.ts",
      "**/*.test.ts",
      "**/*.spec.ts"
    ],
    "reporter": [
      "json",
      "html"
    ],
    "all": true
  }

When I do that, the source maps highlight the right blocks, rather than nonsense byte ranges. However, when using the ts-node Alsatian wrapper I grabbed verbatim from the wiki, and without a ts-node require definition in NYC's config, I can never get it to cover all files, but only those my test files import, if that makes sense.

Anyway, I'll settle for not covering all files, but covering them accurately, for now. I'll drop a line over at nyc, too, to see if this is something they think they should fix with "all": true in their config definition.

Thanks, again.

[cdibbs]

Ohh FYI here's the issue I created: istanbuljs/nyc#776. There's a link to a sample project in the first comment.

[jamesadarich]

Awesome, thanks @cdibbs please let us know if you need any more help :)

[cdibbs]

Thanks, @jamesrichford. BTW, I really enjoy how Alsatian, like most xUnit frameworks, just feels so much more clean and crisp, so I hope it continues to grow. The only thing I find lacking, atm, are the built-in Expects methods.

I do appreciate the simplicity of the core Expects framework, however, as this eases the learning curve. So, perhaps this warrants an extensions package? I would be happy to start one, as I work on some of my own projects. What I find missing are things like Expect(dictionary).toSuperset(inputDict). Is there an interest in this, or perhaps something like that out there already?

Sorry if this is a bit off-topic. Should I create a new issue? Thanks again for all your work on Alsatian!

[jamesadarich]

Awesome @cdibbs very glad to hear it!!

Sounds like a great idea, I suppose it would depend on the particular matchers as to whether they'd go in the core or as an extension package but happy to discuss in their own issue or issues! :)

Thanks for your input so far and looking forward to even more great suggestions so we can evolve Alsatian to be even more useful :)

[biro456]

Hey there, I was checking out Alsatian and came across this using alsatian v3, nyc v15 and ts-node v8.

Did anyone manage to get nyc's coverage reports to list all files?

[jamesadarich]

@biro456 sorry about the late reply on this, I think there is an all flag for nyc that should cover this but we are also currently investigating alternate coverage options and libraries that may get baked into the framework or a supporting library. If you are having trouble still please let me know and will be glad to help

[Gr33nbl00d]

Currently im investigating how to enable coverage run in my intellij webstorm plugin. Did you have any news on this? Tried any other option? Any expirience with your visulcode plugin?

[Gr33nbl00d]

Got it working via NYC, that means the project needs to have NYC as dev dependency but i guess thats ok.