Template tests

[Rich-Harris]

If the app template doesn't include tests, then one of the first questions people will ask is 'how do we add tests'? We should probably have some way of answering this question

[benmccann]

The template lives here: https://github.com/sveltejs/kit/tree/master/packages/create-svelte/template

[benmccann]

A few questions we'll need to answer:

• Do we want to have tests in the default template? It's somewhat opinionated, but will help beginners get started more easily
• What should we use as a test runner? jest and mocha are maybe the most popular, but a lot of our tests use uvu
• Do we want integration tests as well? Cypress was controversial in the sapper-template partially because of its large download size. We're now using Playwright for our own tests and it's smaller. Do we want to use it in the template as well?

[Rich-Harris]

I won't touch jest with a barge pole, personally, and I don't see any advantage to mocha over uvu. uvu is lean enough that we could get away with including it by default. Playwright is a different matter though. I'm inclined to think it should be an optional add-on for that reason. (At any rate, Playwright by itself probably isn't sufficient for integration testing anyway, it's too low-level.)

But it's hard to know what testing setup to suggest when collectively we haven't really figured out what sort of things should live in a test suite

[pngwn]

If we can figure a relatively bullet-proof way to get uvu + JSDOM to work without too much boilerplate we should for it. It isn't perfect but the trade-offs are reasonable for most projects. Luke would be the best person to comment here although I'll be trying to make that work soon myself.

[benmccann]

Here's an example of Svelte + uvu + JSDOM: https://github.com/lukeed/uvu/tree/master/examples/svelte

There's also Svelte Testing Library: https://testing-library.com/docs/svelte-testing-library/example/

[lukeed]

The jsdom + svelte example works well. I have clients running that today in production.

There's also a PR in uvu repo about adding preprocessor support to the example. It's on my todo list to fix that up a bit, but it'll be another good example to point at.

For the record, uvu browser testing story will be its own package, which auto invokes playwright for you. I'm playing with it still to make it as lean as possible, and set it up in a way such that you can auto-rerun tests when related source files change. That may be a bit too lofty of a goal, but was something I was shooting for regardless. Might also add a uvu.jsdom library that, when -rd, will setup the necessary globals so that the uvu example is even less boilerplate-y.

Also, not certain, but I've heard people adding svelte-testing-library to the example and having it just work. Haven't tried myself since the helpers included in example effectively supplant STL entirely.

[pngwn]

The main value of testing library is the neat semantic helpers, it has a few benefits. But being able to do getByRole('button') has slightly better ergonomics than using query selectors, it also prevents you from relying on classes or IDs (although that fragility could be addressed by using data-test ids in any system).

However, it would be easy for users to modify the env file in the example have pretty much the same ergs as testing-library if that is what users wanted.

[ehrencrona]

I spent a few hours trying the various testing approaches.

svelte-testing-library works, but to support preprocessing it needs to spin up a new node instance for each source file which seems like it should give terrible performance.

The uvu pull request that adds preprocessor support borrows the same code. I think this is because the transformation APIs they rely on are synchronous, whereas the preprocessor is async, and they use execSync to work around this. Something like deasync might be an alternative?

The uvu solution also relies on require.extensions, which has been deprecated since Node 0.10.6. But maybe this doesn't cause any actual problems.

I also tried the approach of https://dev.to/d_ir/introduction-4cep that uses Rollup to build the tests. He seems to have a lot of problems with ES6 vs CJS modules and indeed his repository no longer seems to work. I get an error related to ES6 modules that I could not diagnose.

[dummdidumm]

Possible deasync alternative: https://github.com/sindresorhus/make-synchronous

[pngwn]

That looks similar to an approach I’ve taken before, probably similar to what like is going to dig up: https://github.com/pngwn/svelte-test/blob/master/transform.js

[benmccann]

svelte-preprocess only needs to be async for postcss and less. I wonder if for the tests we could disable css processing and just call the preprocessor syncronously? (also requested in svelteness/svelte-jester#20 (comment))

For jest, it looks like there's an issue jestjs/jest#9504 and PR jestjs/jest#9889 to support async transformers. I wonder if that's something that could be added to uvu as well?

[dummdidumm]

svelte-preprocess also asynchronously loads the required dependencies (like await("typescript"))

[ehrencrona]

deasync just hangs for me; not sure why.

make-synchronous doesn't seem like it would work for our use case:

no other code will execute (not even async code) until the given async function is done.
The given function is executed in a subprocess, so you cannot use any variables/imports from outside the scope of the function.

(I get some error about it trying to serialize the function and failing)

Anyway, all these async-to-sync solutions are kludges; I think @benmccann 's suggestion of looking into making preprocessing synchronous seems like the best way forward.

I wonder if that's something that could be added to uvu as well?

uvu uses the Node API require.extensions, which is not well documented, but I think it's synchronous. After all, require is normally synchronous; that's probably the reason why all these transformations need to be.