Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

migrate unit tests from karma to vitest #10452

Closed
wants to merge 3 commits into from
Closed

migrate unit tests from karma to vitest #10452

wants to merge 3 commits into from

Conversation

k-yle
Copy link
Collaborator

@k-yle k-yle commented Sep 3, 2024

this is blocked by #10431, and needs to be rebased after that PR is merged


Closes #10408, Closes #9805, Closes #10382

This PR migrates all our unit tests from karma to vitest. See issue #10408 for more details about why this beneficial. In short: the development cycle is now 10-20x faster, which makes it significantly easier and more pleasant to write tests.

This was suprisingly easy, and only a few test files had to be modified - these changes have been split into a separate commit.

@k-yle k-yle added the chore Improvements to the iD development experience or codebase label Sep 3, 2024
@k-yle k-yle requested a review from tyrasd September 3, 2024 09:51
Copy link
Collaborator Author

@k-yle k-yle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've left some code review comments explaining why certain test cases had to be changed

modules/index.js Show resolved Hide resolved
modules/services/vegbilder.js Show resolved Hide resolved
test/spec/actions/delete_way.js Show resolved Hide resolved
test/spec/services/maprules.js Show resolved Hide resolved
@k-yle k-yle linked an issue Sep 3, 2024 that may be closed by this pull request
@tyrasd
Copy link
Member

tyrasd commented Sep 3, 2024

#10431 is merged now. There are no merge conflicts, but can you please double-check that everything works as expected?

Base automatically changed from kh/dep-cycle to develop September 3, 2024 11:27
@k-yle
Copy link
Collaborator Author

k-yle commented Sep 3, 2024

thanks @tyrasd :) it looks like github automatically changed the target branch which is a neat feature. I've rebased it anyway so that the history is cleaner, and I can confirm that it still works

Copy link
Member

@tyrasd tyrasd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect. 💯

Only one comment: I noticed that on my system, a full run of the new test suite is significantly slower then previously:

Duration 158.62s (transform 3.53s, setup 204.50s, collect 5.84s, tests 9.83s, environment 63.09s, prepare 13.09s)

For comparison, the old code runs npm test:spec in approximately a minute here.

Also the CI on github seems to be a bit slower now: new: 1m 51s, old 1m 8s. 🤔

Is that expected because of jsdom overhead?

modules/globals.d.ts Outdated Show resolved Hide resolved
@k-yle
Copy link
Collaborator Author

k-yle commented Sep 3, 2024

Only one comment: I noticed that on my system, a full run of the new test suite is significantly slower then previously: [...]

interesting, I hadn't checked the overall execution time since it felt similar, but it is indeed slower :(

For me, the more important metric is the time between saving a file, and seeing the test results. That's now reduced to 0-1 seconds, previously it was 10-20 seconds or longer. (since we don't need to run npm run build after every change, and there is a 'watch' mode)

But it's a shame that this is a trade-off... We could try to use the --no-isolate flag, but this will require tweaking a few more tests. In GitHub Actions, it only takes 75 seconds with --no-isolate!

@tyrasd
Copy link
Member

tyrasd commented Sep 3, 2024

For me, the more important metric is the time between saving a file, and seeing the test results.

Agree. 👍

We could try to use the --no-isolate flag

Didn't the test previously essentially run in something equivalent to that? 😉

Also, as the iD modules should not interfere with each other via global states anyway, it might be even preferred to run them like this?!

I just found out that running it with --no-isolate --maxConcurrency 1 seems to work. Without limiting it to a single thread, there are odd errors like ReferenceError: window is not defined, which I don't immediately see where they might originate from. 🤷 But as it's also a lot faster than the isolated approach, it might be worth considering…

//edit: turns out this was unfortunately just a fluke. It now also fails for me even with --maxConcurrency 1. Strange.

This required fixing deferred code that was running after the unit test had unmounted
@k-yle
Copy link
Collaborator Author

k-yle commented Sep 9, 2024

I figured out what was causing the flaky tests and enabled --no-isolate in 156f6a8, which has slightly sped up the process

@tyrasd
Copy link
Member

tyrasd commented Oct 16, 2024

Merged as e839b63 + 108893c (where I activated --no-isolate also for the default npm test command) and b4e55c2 + e7f56cf because I was stupid and accidentally used an outdated branch while merging this PR. Sorry!

@tyrasd tyrasd closed this Oct 16, 2024
@tyrasd tyrasd deleted the kh/vitest branch October 16, 2024 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chore Improvements to the iD development experience or codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Speed up unit tests
3 participants