--watch compilation with no change takes 3 seconds

[zen0wu]

I have a project that's fairly small (167 project TS files, ~8000 lines of code, and 987 total TS files including libraries).

When I work on one of the top modules (I use referenced projects and have 5 modules, the top one is a webserver module that depends on a bunch of heavy TS libraries like nestjs), even though I just save the file as it is (or touch webserver/x.ts), the triggered watch compilation takes about 4 seconds to finish, which is not very efficient to work with.

Results of tsc -b tsconfig.json -w --diagnostics showing most of the time is in parsing and binding.

[8:23:06] File change detected. Starting incremental compilation...

Files:           987
Lines:        422816
Nodes:       1248135
Identifiers:  453635
Symbols:      273809
Types:            73
Memory used: 783590K
I/O read:      0.44s
I/O write:     0.01s
Parse time:    2.95s
Bind time:     0.55s
Check time:    0.00s
Emit time:     0.00s
Total time:    3.50s
[8:23:10] Found 0 errors. Watching for file changes.

I did some simple profiling of tsc.js with the process above, and it appears the compiler takes significant amount of time scanning all the modules, but my gut feeling is all these files/dependencies can be cached.

I wonder if there's any obvious mistakes I'm making, or it's an item on the roadmap since it'd be a big improvement if this can be better.

TypeScript Version: 3.5.2

Search Terms:

Expected behavior:
Since no program change happens, it should be nearly instant to finish the compilation.

Actual behavior:
It takes 3-4 seconds to finish the watch compilation.

Playground Link:

Related Issues:

[fatcerberus]

Are you using --incremental? That was introduced in 3.4 and improved in 3.5 so I'm thinking it might help here.

[zen0wu]

Oh, I forgot to mention. Since I'm using project reference, so all my modules have composite: true enabled, which infers incremental.

[zen0wu]

Did another experiment, where I converted the project into a singular project, without project reference. Now touching a file without any change and trigger compilation under -w is faster, like under 1 second.

Update: With singular project, it has to be -p tsconfig.json rather than -b tsconfig.json, otherwise it won't speed up. But of course the issue is, without -b, the startup of compilation is pretty slow.

[sheetalkamat]

@ZenoZen For scalability we do not yet cache module resolution across compilations in watch mode with --build mode. (consider having too many projects and running out of memory because of that). We do want to investigate about storing such information for the currently edited/changed programs and that might help your situation.
cc: @RyanCavanaugh @DanielRosenwasser

[zen0wu]

@sheetalkamat Thanks for replying!

So it means that recompilation with either -w or --incremental would scan all the files, that's intentional, right? It seems that there is info about scanned modules/files in .tsbuildinfo file, we're just not using them?

Is there anything I can do for now to make it better, other than switching back to non-referenced project structure?

[RyanCavanaugh]

@sheetalkamat let's think long-term about possible improvements here

[chyzwar]

It seems that with incremental tsc still parse all files? I would think that if hash is the same there should be no need to parse files ?

In my case incremental makes compilation slower:

{
    "compilerOptions": {
        "module": "commonjs",
        "moduleResolution": "node",
        "target": "es2018",
        "strict": false,
        "sourceMap": true,
        "declaration": true,
        "emitDeclarationOnly": true,
        "allowJs": false,
        "incremental": true,
        "composite": true,
        "esModuleInterop": true,
        "jsx": "react",
        "rootDir": "src",
        "outDir": "types",
        "skipLibCheck": true,
        "tsBuildInfoFile": "types/tsBuildInfo.json",
        "types": [
            "react",
            "jest",
            "prop-types",
        ],
        "lib": [
            "esnext",
            "dom"
        ]
    },
    "include": [
        "src/**/*.ts",
        "src/**/*.tsx",
        "src/**/*.json"
    ],
    "exclude": [
        "src/**/*.test.ts",
        "src/**/*.test.tsx"
    ]
}

Result in

$ tsc --build --diagnostics
Files:           86
Lines:        77126
Nodes:       242645
Identifiers:  90151
Symbols:      48461
Types:         2376
Memory used: 90497K
I/O read:     0.02s
I/O write:    0.00s
Parse time:   1.22s
Bind time:    0.68s
Check time:   0.38s
Emit time:    0.02s
Total time:   2.30s
Done in 2.66s.

After nuking outDir and disabling incremental

{
    "compilerOptions": {
        "module": "commonjs",
        "moduleResolution": "node",
        "target": "es2018",
        "strict": false,
        "sourceMap": true,
        "declaration": true,
        "emitDeclarationOnly": true,
        "allowJs": false,
        "incremental": false,
        "composite": false,
        "esModuleInterop": true,
        "jsx": "react",
        "rootDir": "src",
        "outDir": "types",
        "skipLibCheck": true,
        "types": [
            "react",
            "jest",
            "prop-types"
        ],
        "lib": [
            "esnext",
            "dom"
        ]
    },
    "include": [
        "src/**/*.ts",
        "src/**/*.tsx",
        "src/**/*.json"
    ],
    "exclude": [
        "src/**/*.test.ts",
        "src/**/*.test.tsx"
    ]
}

$ tsc --build --diagnostics
Files:           86
Lines:        77126
Nodes:       242645
Identifiers:  90151
Symbols:      48461
Types:         2382
Memory used: 94655K
I/O read:     0.01s
I/O write:    0.00s
Parse time:   1.04s
Bind time:    0.58s
Check time:   0.32s
Emit time:    0.08s
Total time:   2.03s
Done in 2.36s.

I am using latest version of tsc.

[zen0wu]

@chyzwar Yeah, IMU, --incremental works pretty much the same way as --watch, so they both parse all the files with --build mode.

Right now I actually moved to lerna/yarn workspaces sort of project structure as the codebase grows, but because of this issue, I have to hack how yarn links projects, to have a single tsconfig.json (to avoid using --build mode) and re-link all node_modules/@my-project/<module> to the outDir/<module>. It's extremely hacky and has a few downside...