-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Implement standalone mocha runner and loader #1
Conversation
8578e3c
to
93fd534
Compare
This is great! Really need this 👍 looking forward to the release. Perhaps it make sense to do a alpha release so people can report issues early on? |
@marcus-pousette Highly depends on the feedback at mochajs/mocha#5039 if the Mocha project will adopt this extension or if I publish it under my name 😁 |
dbd3ff6
to
33f0174
Compare
@Danielku15 great! I started trying out this branch. I am not sure if you are getting this but with this simple test repo. The line numbers are wrong. I am using Node v20.12.0 |
@marcus-pousette Yeah, that's something I'm still tackling. Internally I use ESBuild to transpile TypeScript to JavaScript (including Source Maps) and then I need to translate back the positions correctly. For plain JS the positoning is already fine, for TS (and JavaScript with source maps) the bits are still missing. |
Testing monorepos. I have two issues, see image
Node v20.12.0 https://github.com/marcus-pousette/mocha-vscode-monorepo-test |
Ah I see! I kind of spent some time testing different Node versions and noticed how later ones handle whitespace differently. Not sure if that is helpful here since this was also related to ts-node and the other vscode extension that used stack traces for finding the locations of functions. I tried both 18.12.0 and 20.12.0 and get the same result so not related to this |
Not sure if helpful but the jest-vscode extension uses https://github.com/jridgewell/trace-mapping behind the scenes through the |
@marcus-pousette Thanks for the helpful test repos. I quickly fixed the TypeScript source map stuff, line numbering seem fine in my the projects I use for testing. Regarding the mono-repo: Can you update the repository to a state where mocha itself (e.g. running Currently the repo seems not yet in a functional state to work with mocha standalone. I currently strive for consistency between the CLI and the VS Code extension. If you can fix that I can check easier where the inconsistency in the globbing occurs. |
Great! I will try the code out again if you have pushed all changes. I updated https://github.com/marcus-pousette/mocha-vscode-monorepo-test.git now so that
|
The line number issues still seem to persist sadly in both my test repos (after I pulled your changes from 9c25be6 ) |
@marcus-pousette Be sure to rebuild and reload your vs code. I tested with your repos, everything is fine now. 😉 |
My bad, I forgot to rebuild the extension 🙈 Yes, the line locations seem to work now. This is awesome! When you have the monorepo open, do you see a "node_modules" folder in the test explorer sidebar? I kind of see duplicated tests here and the icons for running the tests only turn green if I run the node_modules one. Will try out this extension now on a big repo I am working on |
Yes I see the Mocha filters out duplicates based on the suite and test names but the Plugin currently structures the tests in folders and files. This is where the difference becomes visible. They have default ignore patterns when using watch but not when normally executing:
I just added support for the {
"require": "ts-node/register",
"loader": "ts-node/esm",
"extensions": ["ts", "tsx"],
"spec": [
"**/test/**/*.spec.ts"
],
+ "ignore": [
+ "**/node_modules/**"
+ ],
"watch-files": [
"src"
]
} Not perfect from an end user perspective, I agree. But in-line with the Mocha behavior. Might be worth discussing over at https://github.com/mochajs/mocha that it should ship a default ignore for the node_modules. |
Ah, thanks for the explanation, that makes sense! Yes this update solves that problem. This is great! I now tried it on a larger repository and these are the issues I am observing right now
|
Will look into that. Mocha does not have directly some "unique ids" we could use to reference tests so it rather works with passing in suite and test names to the
I think this is not related to this extension but Mocha in general. If you have concerns regarding this you might reach out to the Mocha devs to discuss potential improvements. It's just a gut feeling but it could be caused by 'ts-node' you are using. I personally prefer Here an example mocharc from one of my projects: https://github.com/CoderLine/alphaTab/blob/develop/.mocharc.json
What specific problems are you facing? Is it the test discovery (not seeing tests, getting errors? or the the test execution. Remember: the plugin just calls If you have discovery problems it might be simply the fact that I do not load any tsconfig yet. Even though ultimately I want to add this bit, it is currently within my expectations. https://github.com/Danielku15/mocha-vscode/blob/feature/mocha-standalone/src/extract/syntax.ts#L29C29-L29C45 I'm grateful for any examples you can provide for testingwith. I still have to dive in how ESBuild might auto-discover configs and then pass on the right tsconfig. This PR is just the "starting point" and certainly quite some follow up work items are still needed to get it out of alpha/beta stage. The developments here effectively just a few days old 😁 |
Thanks!
I see. Well, in my head I was just imagining that it should not matter if I open the project at the root or at specific package in vscode. Running the tests should go equally fast. Now if I put the .mocharc.json in a package and open that folder as the root I can run the same test almost instantly (that previously took seconds to launch). But might be a mocha thing as you said... I also need perhaps read up on whats possible with vscode extension but it feeeels like it should be possible to scan for package.json in all subfolders and use the mocha config file that is closest, and only force esbuild to interact with that specific folder when running the tests.
Using
I see, well I tested that to make sure this was not the case. Also updated the monorepo now to showcase that this only occurs for the test explorer which is interesting.
Aaah that would explain this behaviour I think 🙂 Thanks for the links to the code, I willl play around with the esbuild transform function a bit to see if I get something simple working but the
Totally understand. I am amazed how fast you accomplish things! 👍👍 |
Might have to use https://nodejs.org/api/vm.html#class-vmmodule instead of vm.runInNewContext to get the esm parts to work. Looks like top level await does not work with cjs output format (evanw/esbuild#253) |
I am just writing this here for memory but it also seems like when using |
Seems like it is impossible to override the import crypto from 'crypto';
import { tmpdir } from 'os';
// @ts-ignore
import { loadConfig } from 'mocha/lib/cli/config'
...
async getMochaSpawnArgs(customArgs: readonly string[]): Promise<string[]> {
this._pathToMocha ??= await this._resolveLocalMochaPath('/bin/mocha.js');
// if custom args contains one or more "--run" take each argument and store in array
let runFiles:string[] = [];
const path = require('path');
let newCustomArgs: string[] = [];
for(let i = 0; i < customArgs.length; i++){
if(customArgs[i] === "--run"){
runFiles.push(path.resolve(customArgs[i+1]));
i++;
}
else{
newCustomArgs.push(customArgs[i]);
}
}
// create a tmp .mocarc.json file with the run files inside of the spec and merge other settings found in the .mocarc.json file at path this.uri.fsPath
let currentConfig = loadConfig(this.uri.fsPath)
// create a tmp file with a fixed name so that we don't unecessarely leak memory every test run
// by hashing this.uri.fsPath
const fileName = crypto.createHash('sha256').update(this.uri.fsPath).digest('hex') + "-mocharc.json";
const tempPath = path.join(tmpdir(), fileName)
const data = {
...currentConfig,
spec: runFiles,
};
fs.writeFileSync(tempPath, JSON.stringify(data));
return [
await this.getPathToNode(),
this._pathToMocha,
'--config',
tempPath,
...newCustomArgs,
];
} This change also seem to fix all the performance problems ! The other vscode extension (https://github.com/hbenl/vscode-mocha-test-adapter) seem to spawn a child process and interact with the 'mocha' lib directly to add all tests |
I tapped into using ESM instead of CSJ in the beginning but it currently seems not possible to use it. Beside the fact that it is an experimental feature, you need to know in advance the named exports of a module which are imported by the test file. Returning a dynamic proxy as default export still leads to errors.
The problem seems to be known: privatenumber/tsx#506
An interesting finding. Its true that Mocha combines the Writing a config file for each test run seems an overkill and I would have also performance and synchronization concerns. According to this logic it might be best to
I wanted to avoid this path. I somehow have security and performance concerns evaluating all test files in full extend as part of the test discovery. Its more a gut feeling than something proven. Using a custom UI combined with --dry-run should technically work but it will also require to either spawn and maintain a long running process with mocha for executing this part, or we would spawn processes for every test file. The test discovery is already configurable. It could become an opt-in for devs to choose this mode in their project. |
8701325
to
3f1ee6a
Compare
I merged this PR now as a foundation work. I organized a bit the work in https://github.com/orgs/CoderLine/projects/15/views/1 |
Yeap. But with Node 18.12 I can get the locations to work with
I tried the |
Sounds good! |
This PR adapts the https://github.com/microsoft/vscode-extension-test-runner/ fork codebase to a standalone mocha test runner extension.
Publish extension.