Skip to content

Test runner: recursive or globbed loading of non-JS test files #44023

Description

@meyfa

What is the problem this feature will solve?

As per https://nodejs.org/dist/latest-v18.x/docs/api/test.html#test-runner-execution-model, the test runner can be started as node --test dirname/ to recursively run all test files (ending in .js, .cjs or .mjs) contained in the directory dirname. This unfortunately means that non-JS test files, specifically test files written in TypeScript in my case, will not be found since they end in .ts instead of .js. The docs say to provide each test file separately, e.g. node --loader=ts-node/esm --test dirname/foo.test.ts dirname/bar.test.ts but this gets messy very quickly.

What is the feature you are proposing to solve the problem?

I see a few options (in order of my preference, starting with most preferred):

  • Support globbing (e.g., node --loader ts-node/esm --test "dirname/**/*.ts"). Note that this does work on Linux, to some extent, since the shell will expand asterisks. This doesn't work on Windows, though, and doesn't work when the string is quoted. Tools like mocha will perform globbing independently of the operating system or shell.
  • Provide a CLI arg for additional test file extensions (e.g., node --loader ts-node/esm --test --test-extensions=.ts,.cts,.mts dirname/).
  • Have some sort of configuration file for the test runner where this can be configured.
  • Let loaders tell Node about additional file extensions (e.g., node --loader ts-node/esm --test dirname/ and ts-node could hook into the directory traversal).

What alternatives have you considered?

  • Listing test files manually (hard to maintain)
  • Having a script do the globbing and construct the command line with all files explicitly listed
  • Running node --test pointed to an index file that performs the globbing and dynamically imports all actual test files (very ugly; loses process separation between test files)

I also opened an issue at ts-node (TypeStrong/ts-node#1853) specifically for TypeScript, to see if they are interested in better test runner support.

Activity

  1. benjamingr commented on Jul 28, 2022

    @benjamingr
    Member

    @nodejs/test_runner

  2. benjamingr commented on Jul 28, 2022

    @benjamingr
    Member

    Honestly, I would just add .ts support explicitly (and probably .tsx/.jsx) given how common it is compared to the alternative with a helpful message in case people didn't pass a loader/require hook.

  3. added
    test_runnerIssues and PRs related to the test runner subsystem.
    on Jul 28, 2022
  4. MoLow commented on Jul 28, 2022

    @MoLow
    Member

    probably for starters, as @benjamingr support /.tsx?$/ as well,
    eventually allow configuration of custom file filtering

  5. ljharb commented on Jul 29, 2022

    @ljharb
    SponsorMember

    I don’t think supporting ts (or jsx) by default makes any sense if TS/JSX content isn’t auto loaded.

  6. MoLow commented on Jul 29, 2022

    @MoLow
    Member

    Perhaps we should implement a solution for #43895 (that will allow passing a custom function to filter files)

    I am willing to implement some --test-config flag if there is consensus on it.
    CC @nodejs/test_runner

  7. aduh95 commented on Jul 29, 2022

    @aduh95
    Contributor

    Maybe a better solution would be to have an option for the loader to have an option to tell which file patten it supports for tests? /cc @nodejs/loaders

  8. MoLow commented on Jul 29, 2022

    @MoLow
    Member

    Maybe a better solution would be to have an option for the loader to have an option to tell which file patten it supports for tests?

    why will it be better? I think a configuration file will cover more use-cases (real usecases from here), for example:

    • within a file, filter which tests should run
    • support glob (without relation to ts support)

    loaders should process test files regardless of this discussion, there is a seperate issue about that

  9. aduh95 commented on Jul 29, 2022

    @aduh95
    Contributor

    Running the test runner on a .ts file only makes sense if there's a loader able to understand it, so it's convenient if the loader itself can provide this info. If the info comes from somewhere else, it's more likely to de-sync at some point. Anyway that was just a thought.

  10. ljharb commented on Jul 29, 2022

    @ljharb
    SponsorMember

    I completely agree that that’s the only way to handle it. Nothing should be implicitly included unless node understands it;cams adding a loader should also implicitly include anything the loader handles. Users should only have to configure file extensions if they’re deviating from their loaders’ defaults.

  11. MoLow commented on Jul 29, 2022

    @MoLow
    Member

    I might not fully understand how loaders work, so correct me in case I am wrong - a loader does not expose what file extensions it supports,
    So there is no way for --test to know what to extensions to filter
    If we are talking about a new flag/option declaring what extensions a loader supports in the context of running tests - I don't really see a reason that should be part of loaders and not just part of the test runner config?

  12. meyfa commented on Jul 29, 2022

    @meyfa
    ContributorAuthor

    Personally, I see a huge advantage in letting users specify the file pattern precisely: I often find myself putting test fixtures, shared code etc. in the test/ directory (e.g. test/fixtures.ts), along with test files (such as test/foo.test.ts, test/bar.test.ts). In these cases, I would want to use a glob like test/**/*.test.ts instead of basing it off of the .ts extension.

  13. cspotcode commented on Jul 29, 2022

    @cspotcode

    I see mts and cts file extensions were not mentioned above. This is evidence in favor of allowing users and customization hooks to specify.

  14. 42 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    feature requestIssues requesting new Node.js features.loadersIssues and PRs related to ES module loaders.test_runnerIssues and PRs related to the test runner subsystem.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions