On linux or editor with canUseEvents to prefer immediate directory if its not in root or node_modules #58866

sheetalkamat · 2024-06-14T21:43:14Z

Experiment with what to watch with preferNonRecursiveWatch option on the host

Helps with #58319

typescript-bot · 2024-06-14T21:43:26Z

Looks like you're introducing a change to the public API surface area. If this includes breaking changes, please document them on our wiki's API Breaking Changes page.

Also, please make sure @DanielRosenwasser and @RyanCavanaugh are aware of the changes, just as a heads up.

sheetalkamat · 2024-06-14T21:47:48Z

cc: @dmichon-msft

sheetalkamat · 2024-06-14T21:48:06Z

@typescript-bot pack this

typescript-bot · 2024-06-14T21:48:10Z

Starting jobs; this comment will be updated as builds start and complete.

Command	Status	Results
`pack this`	✅ Started	✅ Results

typescript-bot · 2024-06-14T21:49:40Z

Hey @sheetalkamat, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/162257/artifacts?artifactName=tgz&fileId=AD496CF156DCB5278061745E27AFE41A2D675BC21A387FF89B00275293DBA4D002&fileName=/typescript-5.6.0-insiders.20240614.tgz"
    }
}

and then running npm install.

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@5.6.0-pr-58866-4".;

dmichon-msft · 2024-06-17T22:07:22Z

I'm working on pulling this down to test in my environment. I'll note that I find it useful to also have a toggle to disable watching any path that contains a node_modules element, since modifying the set of installed packages during a watch mode session is already not allowed by the processes in my repository (and I think this is fairly common for monorepos in general). I can approximate the latter with the new ${configDir} functionality though it gets tedious since the relative path depends on the project depth.

dmichon-msft · 2024-06-17T22:32:14Z

Ah, right, I can't test this directly in my target repository, need to experiment with consuming it from heft-typescript-plugin and passing the necessary config option first. That'll take a bit longer to set up.

dmichon-msft · 2024-06-20T23:12:24Z

Happy to see this, but it looks like I need to do some work to be compatible with the various changes that shipped in TypeScript 5.5 before I'll be able to test it E2E.

…refers it

sheetalkamat · 2024-06-25T17:40:24Z

Ran a experiment with watching packageDir instead of node_modules but that doesnt work out well because of all the node_modules lookups we do for resolution in the folders. It will end up creating more polling watches because say src/node_modules doesnt exist. If i add check of "whether it exists or not" to determine to watch node_modules or package dir, it goes into issue with just having one unresolved module will watch whole node_modules folder and then remove all those watches right after the module is installed. So this isnt performing well so this is the best of both worlds.

Users who dont want to watch node_modules can always ignore them through watchOptions or host.watchDirectory implementation whichever way the user is using tsserver or watch without API or not.

sheetalkamat · 2024-06-25T22:35:18Z

@typescript-bot test it

typescript-bot · 2024-06-25T22:35:21Z

Starting jobs; this comment will be updated as builds start and complete.

Command	Status	Results
`test top400`	✅ Started	✅ Results
`user test this`	✅ Started	✅ Results
`run dt`	✅ Started	✅ Results
`perf test this faster`	✅ Started	👀 Results

typescript-bot · 2024-06-25T22:56:08Z

Hey @sheetalkamat, the results of running the DT tests are ready.

Everything looks the same!

You can check the log here.

typescript-bot · 2024-06-25T23:01:58Z

@sheetalkamat Here are the results of running the user tests with tsc comparing main and refs/pull/58866/merge:

Everything looks good!

typescript-bot · 2024-06-25T23:11:25Z

@sheetalkamat
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Compiler-Unions - node (v18.15.0, x64)}
_Errors	₃₀	₃₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_62,153	_62,153	_~	_~	_~	_{p=1.000 n=6}
_Types	_50,242	_50,242	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{193,456k (± 0.92%)}	_{192,186k (± 0.08%)}	_~	_192,079k	_192,482k	_{p=0.149 n=6}
_{Parse Time}	_{1.32s (± 0.42%)}	_{1.31s (± 0.62%)}	_~	_1.29s	_1.31s	_{p=0.054 n=6}
_{Bind Time}	_0.71s	_0.71s	_~	_~	_~	_{p=1.000 n=6}
_{Check Time}	_{9.43s (± 0.28%)}	_{9.42s (± 0.23%)}	_~	_9.40s	_9.45s	_{p=0.570 n=6}
_{Emit Time}	_{2.75s (± 0.65%)}	_{2.74s (± 0.38%)}	_~	_2.73s	_2.76s	_{p=0.366 n=6}
_{Total Time}	_{14.20s (± 0.25%)}	_{14.18s (± 0.15%)}	_~	_14.16s	_14.21s	_{p=0.288 n=6}
_{angular-1 - node (v18.15.0, x64)}
_Errors	₅	₅	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_944,114	_944,114	_~	_~	_~	_{p=1.000 n=6}
_Types	_407,050	_407,050	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{1,218,310k (± 0.00%)}	_{1,218,379k (± 0.00%)}	_{+69k (+ 0.01%)}	_1,218,362k	_1,218,391k	_{p=0.008 n=6}
_{Parse Time}	_{6.65s (± 0.61%)}	_{6.66s (± 0.32%)}	_~	_6.63s	_6.68s	_{p=0.622 n=6}
_{Bind Time}	_{1.87s (± 0.28%)}	_{1.86s (± 0.48%)}	_~	_1.85s	_1.87s	_{p=0.190 n=6}
_{Check Time}	_{30.58s (± 0.35%)}	_{30.59s (± 0.36%)}	_~	_30.46s	_30.70s	_{p=1.000 n=6}
_{Emit Time}	_{13.58s (± 0.23%)}	_{13.51s (± 0.59%)}	_~	_13.38s	_13.60s	_{p=0.060 n=6}
_{Total Time}	_{52.68s (± 0.24%)}	_{52.61s (± 0.29%)}	_~	_52.48s	_52.84s	_{p=0.422 n=6}
_{mui-docs - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_2,135,096	_2,135,096	_~	_~	_~	_{p=1.000 n=6}
_Types	_927,168	_927,168	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{2,117,275k (± 0.00%)}	_{2,117,276k (± 0.00%)}	_~	_2,117,196k	_2,117,485k	_{p=0.810 n=6}
_{Parse Time}	_{6.64s (± 0.53%)}	_{6.64s (± 0.22%)}	_~	_6.63s	_6.66s	_{p=0.620 n=6}
_{Bind Time}	_{2.33s (± 1.05%)}	_{2.33s (± 0.97%)}	_~	_2.29s	_2.35s	_{p=0.340 n=6}
_{Check Time}	_{70.59s (± 0.37%)}	_{70.84s (± 0.47%)}	_~	_70.53s	_71.30s	_{p=0.230 n=6}
_{Emit Time}	_{0.13s (± 3.87%)}	_{0.14s (± 5.44%)}	_~	_0.13s	_0.15s	_{p=0.247 n=6}
_{Total Time}	_{79.69s (± 0.38%)}	_{79.96s (± 0.44%)}	_~	_79.63s	_80.44s	_{p=0.230 n=6}
_{self-build-src - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_1,231,459	_1,231,988	_{+529 (+ 0.04%)}	_~	_~	_{p=0.001 n=6}
_Types	_261,178	_261,417	_{+239 (+ 0.09%)}	_~	_~	_{p=0.001 n=6}
_{Memory used}	_{2,346,776k (± 0.02%)}	_{2,347,610k (± 0.05%)}	_~	_2,345,742k	_2,348,583k	_{p=0.128 n=6}
_{Parse Time}	_{5.05s (± 1.13%)}	_{5.02s (± 0.43%)}	_~	_4.99s	_5.05s	_{p=0.521 n=6}
_{Bind Time}	_{1.92s (± 0.39%)}	_{1.93s (± 0.71%)}	_~	_1.91s	_1.95s	_{p=0.099 n=6}
_{Check Time}	_{34.15s (± 0.54%)}	_{34.21s (± 0.65%)}	_~	_34.05s	_34.64s	_{p=0.471 n=6}
_{Emit Time}	_{2.75s (± 3.33%)}	_{2.72s (± 3.17%)}	_~	_2.61s	_2.81s	_{p=0.630 n=6}
_{Total Time}	_{43.87s (± 0.54%)}	_{43.91s (± 0.56%)}	_~	_43.71s	_44.39s	_{p=0.630 n=6}
_{self-build-src-public-api - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_1,231,459	_1,231,988	_{+529 (+ 0.04%)}	_~	_~	_{p=0.001 n=6}
_Types	_261,178	_261,417	_{+239 (+ 0.09%)}	_~	_~	_{p=0.001 n=6}
_{Memory used}	_{2,424,836k (± 0.03%)}	_{2,435,103k (± 0.99%)}	_~	_2,424,249k	_2,484,319k	_{p=0.230 n=6}
_{Parse Time}	_{7.77s (± 0.89%)}	_{7.81s (± 0.80%)}	_~	_7.71s	_7.90s	_{p=0.470 n=6}
_{Bind Time}	_{2.52s (± 0.69%)}	_{2.51s (± 0.54%)}	_~	_2.49s	_2.52s	_{p=0.105 n=6}
_{Check Time}	_{50.05s (± 0.32%)}	_{50.01s (± 0.26%)}	_~	_49.85s	_50.22s	_{p=0.689 n=6}
_{Emit Time}	_{3.89s (± 2.37%)}	_{3.94s (± 2.25%)}	_~	_3.85s	_4.06s	_{p=0.378 n=6}
_{Total Time}	_{64.25s (± 0.27%)}	_{64.28s (± 0.11%)}	_~	_64.18s	_64.40s	_{p=1.000 n=6}
_{self-compiler - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_258,577	_258,600	_{+23 (+ 0.01%)}	_~	_~	_{p=0.001 n=6}
_Types	_104,827	_104,828	_{+1 (+ 0.00%)}	_~	_~	_{p=0.001 n=6}
_{Memory used}	_{428,241k (± 0.02%)}	_{428,236k (± 0.01%)}	_~	_428,192k	_428,267k	_{p=0.810 n=6}
_{Parse Time}	_{3.31s (± 0.59%)}	_{3.31s (± 0.91%)}	_~	_3.28s	_3.35s	_{p=0.871 n=6}
_{Bind Time}	_{1.30s (± 0.80%)}	_{1.32s (± 1.97%)}	_~	_1.27s	_1.34s	_{p=0.089 n=6}
_{Check Time}	_{17.74s (± 0.36%)}	_{17.78s (± 0.76%)}	_~	_17.61s	_18.01s	_{p=0.748 n=6}
_{Emit Time}	_{1.38s (± 1.82%)}	_{1.38s (± 0.85%)}	_~	_1.36s	_1.39s	_{p=0.870 n=6}
_{Total Time}	_{23.73s (± 0.28%)}	_{23.78s (± 0.59%)}	_~	_23.61s	_24.03s	_{p=0.629 n=6}
_{ts-pre-modules - node (v18.15.0, x64)}
_Errors	₃₅	₃₅	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_224,565	_224,565	_~	_~	_~	_{p=1.000 n=6}
_Types	_93,734	_93,734	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{369,459k (± 0.04%)}	_{369,518k (± 0.03%)}	_~	_369,342k	_369,665k	_{p=0.471 n=6}
_{Parse Time}	_{2.78s (± 0.89%)}	_{2.77s (± 1.15%)}	_~	_2.73s	_2.81s	_{p=0.810 n=6}
_{Bind Time}	_{1.59s (± 1.11%)}	_{1.58s (± 0.95%)}	_~	_1.57s	_1.61s	_{p=1.000 n=6}
_{Check Time}	_{15.49s (± 0.13%)}	_{15.46s (± 0.38%)}	_~	_15.38s	_15.55s	_{p=0.146 n=6}
_{Emit Time}	_0.00s	_0.00s	_~	_~	_~	_{p=1.000 n=6}
_{Total Time}	_{19.85s (± 0.17%)}	_{19.82s (± 0.38%)}	_~	_19.73s	_19.91s	_{p=0.332 n=6}
_{vscode - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_2,878,705	_2,878,705	_~	_~	_~	_{p=1.000 n=6}
_Types	_975,213	_975,213	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{3,041,996k (± 0.00%)}	_{3,042,016k (± 0.00%)}	_~	_3,041,950k	_3,042,061k	_{p=0.630 n=6}
_{Parse Time}	_{13.54s (± 0.26%)}	_{13.51s (± 0.15%)}	_~	_13.49s	_13.54s	_{p=0.147 n=6}
_{Bind Time}	_{4.19s (± 0.18%)}	_{4.19s (± 0.43%)}	_~	_4.17s	_4.22s	_{p=0.620 n=6}
_{Check Time}	_{73.34s (± 0.33%)}	_{73.97s (± 2.35%)}	_~	_72.96s	_77.48s	_{p=1.000 n=6}
_{Emit Time}	_{24.04s (± 0.30%)}	_{23.42s (± 6.51%)}	_~	_20.32s	_24.33s	_{p=0.423 n=6}
_{Total Time}	_{115.11s (± 0.21%)}	_{115.09s (± 0.25%)}	_~	_114.79s	_115.55s	_{p=0.810 n=6}
_{webpack - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_267,117	_267,117	_~	_~	_~	_{p=1.000 n=6}
_Types	_108,775	_108,775	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{411,549k (± 0.01%)}	_{411,552k (± 0.03%)}	_~	_411,486k	_411,779k	_{p=0.230 n=6}
_{Parse Time}	_{3.83s (± 0.38%)}	_{3.81s (± 0.77%)}	_~	_3.78s	_3.86s	_{p=0.462 n=6}
_{Bind Time}	_{1.69s (± 0.37%)}	_{1.69s (± 0.31%)}	_~	_1.68s	_1.69s	_{p=0.386 n=6}
_{Check Time}	_{16.70s (± 0.40%)}	_{16.71s (± 0.57%)}	_~	_16.57s	_16.85s	_{p=1.000 n=6}
_{Emit Time}	_0.00s	_0.00s	_~	_~	_~	_{p=1.000 n=6}
_{Total Time}	_{22.21s (± 0.31%)}	_{22.21s (± 0.34%)}	_~	_22.11s	_22.34s	_{p=1.000 n=6}
_{xstate-main - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_525,251	_525,251	_~	_~	_~	_{p=1.000 n=6}
_Types	_178,574	_178,574	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{462,844k (± 0.05%)}	_{462,879k (± 0.07%)}	_~	_462,460k	_463,270k	_{p=0.936 n=6}
_{Parse Time}	_{3.92s (± 0.70%)}	_{3.93s (± 0.70%)}	_~	_3.89s	_3.96s	_{p=0.419 n=6}
_{Bind Time}	_{1.45s (± 0.73%)}	_{1.45s (± 1.14%)}	_~	_1.43s	_1.47s	_{p=0.869 n=6}
_{Check Time}	_{22.24s (± 0.55%)}	_{22.22s (± 0.32%)}	_~	_22.13s	_22.33s	_{p=0.873 n=6}
_{Emit Time}	_0.00s	_0.00s	_~	_~	_~	_{p=1.000 n=6}
_{Total Time}	_{27.60s (± 0.38%)}	_{27.60s (± 0.28%)}	_~	_27.49s	_27.72s	_{p=0.936 n=6}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

Compiler-Unions - node (v18.15.0, x64)
angular-1 - node (v18.15.0, x64)
mui-docs - node (v18.15.0, x64)
self-build-src - node (v18.15.0, x64)
self-build-src-public-api - node (v18.15.0, x64)
self-compiler - node (v18.15.0, x64)
ts-pre-modules - node (v18.15.0, x64)
vscode - node (v18.15.0, x64)
webpack - node (v18.15.0, x64)
xstate-main - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

Developer Information:

Download Benchmarks

typescript-bot · 2024-06-26T00:32:05Z

@sheetalkamat Here are the results of running the top 400 repos with tsc comparing main and refs/pull/58866/merge:

Everything looks good!

jakebailey · 2024-06-27T17:06:33Z

tests/baselines/reference/api/typescript.d.ts

@@ -2651,6 +2651,7 @@ declare namespace ts {
        interface ServerHost extends System {
            watchFile(path: string, callback: FileWatcherCallback, pollingInterval?: number, options?: WatchOptions): FileWatcher;
            watchDirectory(path: string, callback: DirectoryWatcherCallback, recursive?: boolean, options?: WatchOptions): FileWatcher;
+            preferNonRecursiveWatch?: boolean;


Is the intent that downstream users configure this? Do we need to wait for David to check that this helps?

I was not sure whether to leave this public on ServerHost or not but TscWatch yes because i think last i checked david was using the API to patch tsc watch.

I am looking to get this in early and revert or reiterate sooner than later as we dont get more feedback without this being in nightly esp since this depends on OS and vscode setting to use their watchers

i can make these internal on tsc-watch and tsserver internal and make them public in next PR if they are needed if that approach is better

Understood; just checking; I think you have more context than me on this.

typescript-bot assigned sheetalkamat Jun 14, 2024

typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Jun 14, 2024

dmichon-msft mentioned this pull request Jun 18, 2024

Allow consumers of Compiler API to override getDirectoryToWatchFailedLookupLocation #58856

Closed

6 tasks

sheetalkamat force-pushed the recursiveWatchOptimization branch from 560ac16 to a820f56 Compare June 20, 2024 22:49

sheetalkamat force-pushed the recursiveWatchOptimization branch from a820f56 to 827866a Compare June 21, 2024 20:36

sheetalkamat added 2 commits June 25, 2024 10:23

Baseline before change

ff24415

Prefer non recursive watch outside rootDir and node_modules if host p…

651bfe2

…refers it

sheetalkamat force-pushed the recursiveWatchOptimization branch from 827866a to 651bfe2 Compare June 25, 2024 17:25

sheetalkamat requested review from andrewbranch and jakebailey June 25, 2024 17:41

sheetalkamat marked this pull request as ready for review June 25, 2024 17:41

sheetalkamat changed the title ~~[WIP] On linux or editor with canUseEvents to prefer immediate directory if its not in root or node_modules~~ On linux or editor with canUseEvents to prefer immediate directory if its not in root or node_modules Jun 25, 2024

jakebailey reviewed Jun 27, 2024

View reviewed changes

jakebailey approved these changes Jun 27, 2024

View reviewed changes

sheetalkamat merged commit f7833b2 into main Jun 27, 2024
28 checks passed

sheetalkamat deleted the recursiveWatchOptimization branch June 27, 2024 18:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

On linux or editor with canUseEvents to prefer immediate directory if its not in root or node_modules #58866

On linux or editor with canUseEvents to prefer immediate directory if its not in root or node_modules #58866

sheetalkamat commented Jun 14, 2024 •

edited

Loading

typescript-bot commented Jun 14, 2024

sheetalkamat commented Jun 14, 2024

sheetalkamat commented Jun 14, 2024

typescript-bot commented Jun 14, 2024 •

edited

Loading

typescript-bot commented Jun 14, 2024 •

edited

Loading

dmichon-msft commented Jun 17, 2024

dmichon-msft commented Jun 17, 2024

dmichon-msft commented Jun 20, 2024

sheetalkamat commented Jun 25, 2024

sheetalkamat commented Jun 25, 2024

typescript-bot commented Jun 25, 2024 •

edited

Loading

typescript-bot commented Jun 25, 2024

typescript-bot commented Jun 25, 2024

typescript-bot commented Jun 25, 2024

tsc

typescript-bot commented Jun 26, 2024

jakebailey Jun 27, 2024

sheetalkamat Jun 27, 2024

sheetalkamat Jun 27, 2024

jakebailey Jun 27, 2024

On linux or editor with canUseEvents to prefer immediate directory if its not in root or node_modules #58866

On linux or editor with canUseEvents to prefer immediate directory if its not in root or node_modules #58866

Conversation

sheetalkamat commented Jun 14, 2024 • edited Loading

typescript-bot commented Jun 14, 2024

sheetalkamat commented Jun 14, 2024

sheetalkamat commented Jun 14, 2024

typescript-bot commented Jun 14, 2024 • edited Loading

typescript-bot commented Jun 14, 2024 • edited Loading

dmichon-msft commented Jun 17, 2024

dmichon-msft commented Jun 17, 2024

dmichon-msft commented Jun 20, 2024

sheetalkamat commented Jun 25, 2024

sheetalkamat commented Jun 25, 2024

typescript-bot commented Jun 25, 2024 • edited Loading

typescript-bot commented Jun 25, 2024

typescript-bot commented Jun 25, 2024

typescript-bot commented Jun 25, 2024

tsc

typescript-bot commented Jun 26, 2024

jakebailey Jun 27, 2024

Choose a reason for hiding this comment

sheetalkamat Jun 27, 2024

Choose a reason for hiding this comment

sheetalkamat Jun 27, 2024

Choose a reason for hiding this comment

jakebailey Jun 27, 2024

Choose a reason for hiding this comment

sheetalkamat commented Jun 14, 2024 •

edited

Loading

typescript-bot commented Jun 14, 2024 •

edited

Loading

typescript-bot commented Jun 14, 2024 •

edited

Loading

typescript-bot commented Jun 25, 2024 •

edited

Loading