TypeScript --watch initial compilation several times slower for the same codebase since 2.8

[devuo]

TypeScript Version: 2.9.2, >= 2.8

Search Terms: Slow compilation, Slow watch

Code
The initial watch compilation time issue is not directly reproducible under OSX, and is only manifested when executed under a Docker container. Thus for easier debugging and demonstration of the issue, I've created a repo with all the required code that reproduces the issue and contains additional supporting documentation such as v8 profiling. See the following repository for a reproducible step-by-step instructions and code.

https://github.com/devuo/typescript-slow-watch-bug

Expected behavior:
Initial --watch compilation times should take about the same time the same time as they did in 2.7. Here's how --watch looks like in 2.7 for the codebase referred above:

node_modules/.bin/tsc --watch --extendedDiagnostics

08:31:33 - Starting compilation in watch mode...
Files:                   61
Lines:                62744
Nodes:               273576
Identifiers:          98075
Symbols:              80280
Types:                29130
Memory used:        125488K
I/O Read time:        0.19s
Parse time:           0.77s
Program time:         1.97s
Bind time:            0.48s
Check time:           1.62s
transformTime time:   0.02s
printTime time:       0.02s
Emit time:            0.04s
commentTime time:     0.00s
Source Map time:      0.00s
I/O Write time:       0.01s
Total time:           4.12s
08:31:37 - Compilation complete. Watching for file changes.

Actual behavior:
Initial --watch compilation times for the same codebase take many times under >= 2.8 than they did in 2.7. Exact initial compilation times since 2.8 appears to depend on the number of files being scanned under the root project directory. Here's how --watch looks like in 2.9 for the same codebase above:

node_modules/.bin/tsc --watch --extendedDiagnostics

08:27:10 - Starting compilation in watch mode..
08:27:43 - Found 0 errors. Watching for file changes.
Files:                   61
Lines:                63371
Nodes:               279942
Identifiers:          99371
Symbols:              79865
Types:                30108
Memory used:        140212K
I/O Read time:        0.18s
Parse time:           0.70s
Program time:        30.17s
Bind time:            0.45s
Check time:           1.80s
transformTime time:   0.02s
commentTime time:     0.00s
Source Map time:      0.00s
I/O Write time:       0.01s
printTime time:       0.05s
Emit time:            0.05s
Total time:          32.47s

Here's where 2.9 appears to spend most of its time during a profiling session:

   ticks parent  name
  22067   67.1%  /lib/ld-musl-x86_64.so.1
  15181   68.8%    LazyCompile: *getAccessibleFileSystemEntries /app/node_modules/typescript/lib/tsc.js:3502:52
  15161   99.9%      LazyCompile: ~getAccessibleSortedChildDirectories /app/node_modules/typescript/lib/tsc.js:3303:67
  15161  100.0%        LazyCompile: ~watchChildDirectories /app/node_modules/typescript/lib/tsc.js:3120:39
  15161  100.0%          LazyCompile: ~updateChildWatches /app/node_modules/typescript/lib/tsc.js:3115:36
  15152   99.9%            LazyCompile: ~createDirectoryWatcher /app/node_modules/typescript/lib/tsc.js:3098:40

Compare with 2.7:

   ticks parent  name
   1831   35.4%  /usr/local/bin/node
   1667   91.0%    /usr/local/bin/node
    253   15.2%      LazyCompile: ~directoryExists /app/node_modules/typescript/lib/tsc.js:2766:37
    249   98.4%        LazyCompile: ~directoryExists /app/node_modules/typescript/lib/tsc.js:61964:39
    249  100.0%          LazyCompile: ~directoryExists /app/node_modules/typescript/lib/tsc.js:58876:33
    249  100.0%            LazyCompile: ~compilerHost.directoryExists /app/node_modules/typescript/lib/tsc.js:62085:82

Related Issues: #22953

[nelsonfamorim]

Can confirm, getting this behaviour as well

[sheetalkamat]

This seems like a correct change as part of #21243 since we added support for recursive directory watching on OS that werent supported by node. Can you please use tsc --w --extendedDiagnostics with typescript@next and share the result. We can investigate further to decide how to optimize this further from those result.

[devuo]

Will do, however I cannot comprehend how you can define this as a "correct change", as it made --watch non-useable/broken for development purposes since 2.8 whereas it was until 2.7.

[sheetalkamat]

It was correct change because thats what will give you right behavior. The issue has various options to try out different kinds of watches on different. There is always scope to do optimize based on user experience.

[devuo]

typescript 3.0.0-dev.20180616

21:15:10 - Starting compilation in watch mode...

21:15:42 - Found 0 errors. Watching for file changes.
Files:                   67
Lines:                64772
Nodes:               284175
Identifiers:         100872
Symbols:              81598
Types:                30924
Memory used:        147366K
I/O Read time:        0.19s
Parse time:           0.74s
Program time:        29.77s
Bind time:            0.48s
Check time:           1.77s
transformTime time:   0.03s
commentTime time:     0.00s
Source Map time:      0.00s
I/O Write time:       0.01s
printTime time:       0.07s
Emit time:            0.07s
Total time:          32.09s

20847   66.9%  /lib/ld-musl-x86_64.so.1
  14557   69.8%    LazyCompile: *getAccessibleFileSystemEntries /app/node_modules/typescript/lib/tsc.js:2438:52
  14534   99.8%      LazyCompile: ~getAccessibleSortedChildDirectories /app/node_modules/typescript/lib/tsc.js:2238:67
  14534  100.0%        LazyCompile: ~watchChildDirectories /app/node_modules/typescript/lib/tsc.js:2053:39
  14534  100.0%          LazyCompile: ~updateChildWatches /app/node_modules/typescript/lib/tsc.js:2048:36
  14496   99.7%            LazyCompile: ~createDirectoryWatcher /app/node_modules/typescript/lib/tsc.js:2031:40

No difference whatsoever. Updated the repository https://github.com/devuo/typescript-slow-watch-bug with this information. You can use it to reproduce the issue.

[devuo]

I've also seen it scanning recursively folders like .git, go to the depths of node_modules/ and other folders not specified in include or exclude, I can't see the point of recursively scanning under directories not specified in tsconfig by the authors.

[devuo]

I also tried to use all the different environment variable values for TSC_WATCHFILE and TSC_WATCHDIRECTORY to no avail and made no difference whatsoever, e.g:

export TSC_WATCHFILE="UseFsEventsOnParentDirectory"
export TSC_WATCHDIRECTORY="RecursiveDirectoryUsingFsWatchFile"

node_modules/.bin/tsc --watch --extendedDiagnostics

[21:38:29] Starting compilation in watch mode...
[21:39:03] Found 0 errors. Watching for file changes.

Files:                   62
Lines:                63379
Nodes:               279962
Identifiers:          99376
Symbols:              79869
Types:                30111
Memory used:        140182K
I/O Read time:        0.19s
Parse time:           0.71s
Program time:        30.35s
Bind time:            0.50s
Check time:           1.79s
transformTime time:   0.04s
commentTime time:     0.01s
Source Map time:      0.00s
I/O Write time:       0.01s
printTime time:       0.09s
Emit time:            0.10s
Total time:          32.74s