Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Defer creation of barebonesLibSourceFile #58931

Merged
merged 1 commit into from
Jun 19, 2024

Conversation

jakebailey
Copy link
Member

@jakebailey jakebailey commented Jun 19, 2024

Pulled out of #58928.

Locally, this looks like a big win for typescript.js startup:

Benchmark 1 (695 runs): node ./built/local-old/typescript.js
  measurement          mean ± σ            min … max           outliers         delta
  wall_time           144ms ± 1.76ms     140ms …  152ms         15 ( 2%)        0%
  peak_rss           89.6MB ±  541KB    88.3MB … 94.3MB          7 ( 1%)        0%
  cpu_cycles          638M  ± 5.60M      629M  …  675M          18 ( 3%)        0%
  instructions       1.07G  ±  453K     1.07G  … 1.07G           2 ( 0%)        0%
  cache_references   16.8M  ±  362K     15.7M  … 18.0M          13 ( 2%)        0%
  cache_misses       2.31M  ±  201K     2.06M  … 3.33M          34 ( 5%)        0%
  branch_misses      4.74M  ± 14.8K     4.70M  … 4.79M           6 ( 1%)        0%
Benchmark 2 (726 runs): node ./built/local/typescript.js
  measurement          mean ± σ            min … max           outliers         delta
  wall_time           138ms ± 1.86ms     134ms …  154ms         21 ( 3%)        ⚡-  4.3% ±  0.1%
  peak_rss           89.4MB ±  661KB    88.0MB … 95.1MB         38 ( 5%)          -  0.2% ±  0.1%
  cpu_cycles          607M  ± 6.24M      598M  …  669M          24 ( 3%)        ⚡-  4.8% ±  0.1%
  instructions       1.03G  ±  457K     1.03G  … 1.04G           3 ( 0%)        ⚡-  3.4% ±  0.0%
  cache_references   15.1M  ±  334K     14.2M  … 16.0M          16 ( 2%)        ⚡- 10.1% ±  0.2%
  cache_misses       2.29M  ±  206K     1.99M  … 3.85M          28 ( 4%)          -  1.2% ±  0.9%
  branch_misses      4.48M  ± 14.4K     4.44M  … 4.54M          18 ( 2%)        ⚡-  5.5% ±  0.0%

Also, this apparently runs before services swaps out the object allocators, so it causes a decrease in monomorphism.

Co-authored-by: Titian Cernicova-Dragomir <tcernicovad1@bloomberg.net>
@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Jun 19, 2024
@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 19, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 192,790k (± 0.73%) 194,027k (± 0.99%) ~ 192,167k 195,837k p=0.066 n=6
Parse Time 1.57s (± 0.84%) 1.58s (± 0.65%) ~ 1.56s 1.59s p=0.506 n=6
Bind Time 0.85s (± 1.21%) 0.86s (± 1.14%) ~ 0.85s 0.87s p=0.203 n=6
Check Time 11.19s (± 0.40%) 11.19s (± 0.39%) ~ 11.13s 11.25s p=0.747 n=6
Emit Time 3.28s (± 0.83%) 3.30s (± 0.45%) ~ 3.27s 3.31s p=0.166 n=6
Total Time 16.90s (± 0.41%) 16.92s (± 0.26%) ~ 16.88s 16.98s p=0.571 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,051 407,051 ~ ~ ~ p=1.000 n=6
Memory used 1,218,361k (± 0.00%) 1,218,295k (± 0.00%) ~ 1,218,238k 1,218,351k p=0.066 n=6
Parse Time 7.97s (± 0.53%) 7.94s (± 0.69%) ~ 7.86s 8.00s p=0.572 n=6
Bind Time 2.24s (± 0.62%) 2.22s (± 0.44%) -0.02s (- 0.75%) 2.21s 2.23s p=0.047 n=6
Check Time 35.67s (± 0.23%) 35.70s (± 0.24%) ~ 35.62s 35.82s p=0.809 n=6
Emit Time 16.14s (± 0.50%) 16.15s (± 0.30%) ~ 16.07s 16.19s p=0.630 n=6
Total Time 62.02s (± 0.27%) 62.01s (± 0.08%) ~ 61.94s 62.07s p=0.630 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,133,539 2,133,539 ~ ~ ~ p=1.000 n=6
Types 926,551 926,551 ~ ~ ~ p=1.000 n=6
Memory used 2,115,712k (± 0.00%) 2,115,723k (± 0.01%) ~ 2,115,595k 2,115,949k p=0.936 n=6
Parse Time 7.87s (± 0.22%) 7.86s (± 0.19%) ~ 7.85s 7.88s p=0.567 n=6
Bind Time 2.76s (± 0.55%) 2.77s (± 0.55%) ~ 2.74s 2.78s p=0.282 n=6
Check Time 83.86s (± 0.38%) 83.81s (± 0.36%) ~ 83.46s 84.23s p=0.936 n=6
Emit Time 0.16s (± 2.52%) 0.16s (± 2.58%) ~ 0.15s 0.16s p=0.218 n=6
Total Time 94.64s (± 0.33%) 94.60s (± 0.32%) ~ 94.25s 95.02s p=0.873 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,761 1,230,762 +1 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,189 261,187 -2 (- 0.00%) ~ ~ p=0.001 n=6
Memory used 2,346,502k (± 0.03%) 2,345,895k (± 0.03%) ~ 2,344,838k 2,346,901k p=0.230 n=6
Parse Time 5.95s (± 0.96%) 5.99s (± 0.85%) ~ 5.92s 6.07s p=0.261 n=6
Bind Time 2.26s (± 0.33%) 2.24s (± 0.73%) ~ 2.22s 2.26s p=0.165 n=6
Check Time 39.71s (± 0.31%) 39.84s (± 0.42%) ~ 39.63s 40.02s p=0.230 n=6
Emit Time 3.10s (± 1.31%) 3.20s (± 1.19%) +0.10s (+ 3.23%) 3.14s 3.24s p=0.008 n=6
Total Time 51.04s (± 0.35%) 51.29s (± 0.29%) +0.25s (+ 0.48%) 51.04s 51.43s p=0.031 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,761 1,230,762 +1 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,189 261,187 -2 (- 0.00%) ~ ~ p=0.001 n=6
Memory used 2,423,509k (± 0.01%) 2,423,424k (± 0.02%) ~ 2,422,810k 2,424,157k p=0.810 n=6
Parse Time 7.75s (± 0.65%) 7.73s (± 0.56%) ~ 7.68s 7.79s p=0.810 n=6
Bind Time 2.53s (± 0.85%) 2.49s (± 1.06%) -0.05s (- 1.78%) 2.45s 2.52s p=0.016 n=6
Check Time 49.43s (± 0.19%) 49.48s (± 0.18%) ~ 49.32s 49.57s p=0.298 n=6
Emit Time 3.92s (± 2.02%) 3.86s (± 0.93%) ~ 3.81s 3.90s p=0.126 n=6
Total Time 63.65s (± 0.09%) 63.57s (± 0.18%) ~ 63.38s 63.70s p=0.261 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,573 258,573 ~ ~ ~ p=1.000 n=6
Types 104,819 104,819 ~ ~ ~ p=1.000 n=6
Memory used 428,198k (± 0.01%) 428,173k (± 0.01%) ~ 428,123k 428,250k p=0.230 n=6
Parse Time 3.33s (± 0.96%) 3.32s (± 1.35%) ~ 3.29s 3.39s p=0.568 n=6
Bind Time 1.31s (± 1.50%) 1.32s (± 1.41%) ~ 1.30s 1.34s p=0.324 n=6
Check Time 17.75s (± 0.25%) 17.73s (± 0.32%) ~ 17.64s 17.80s p=0.470 n=6
Emit Time 1.36s (± 0.93%) 1.37s (± 1.07%) ~ 1.35s 1.39s p=0.392 n=6
Total Time 23.75s (± 0.19%) 23.73s (± 0.38%) ~ 23.60s 23.82s p=1.000 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,477k (± 0.02%) 369,415k (± 0.02%) ~ 369,346k 369,541k p=0.066 n=6
Parse Time 2.77s (± 0.97%) 2.76s (± 0.87%) ~ 2.73s 2.80s p=0.747 n=6
Bind Time 1.58s (± 0.95%) 1.58s (± 0.74%) ~ 1.57s 1.60s p=1.000 n=6
Check Time 15.45s (± 0.57%) 15.47s (± 0.37%) ~ 15.38s 15.54s p=0.748 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.80s (± 0.45%) 19.81s (± 0.34%) ~ 19.73s 19.88s p=1.000 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=NaN n=0
Symbols 0 0 ~ ~ ~ p=NaN n=0
Types 0 0 ~ ~ ~ p=NaN n=0
Memory used 0k 0k ~ ~ ~ p=NaN n=0
Parse Time 0s 0s ~ ~ ~ p=NaN n=0
Bind Time 0s 0s ~ ~ ~ p=NaN n=0
Check Time 0s 0s ~ ~ ~ p=NaN n=0
Emit Time 0s 0s ~ ~ ~ p=NaN n=0
Total Time 0s 0s ~ ~ ~ p=NaN n=0
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 266,926 266,926 ~ ~ ~ p=1.000 n=6
Types 108,691 108,691 ~ ~ ~ p=1.000 n=6
Memory used 411,270k (± 0.02%) 411,323k (± 0.01%) ~ 411,269k 411,369k p=0.230 n=6
Parse Time 3.16s (± 0.33%) 3.15s (± 0.28%) ~ 3.14s 3.16s p=0.273 n=6
Bind Time 1.41s 1.41s (± 0.29%) ~ 1.41s 1.42s p=0.405 n=6
Check Time 14.25s (± 0.23%) 14.24s (± 0.25%) ~ 14.19s 14.30s p=0.935 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.82s (± 0.17%) 18.81s (± 0.20%) ~ 18.76s 18.87s p=0.748 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 523,765 523,765 ~ ~ ~ p=1.000 n=6
Types 178,055 178,055 ~ ~ ~ p=1.000 n=6
Memory used 462,123k (± 0.03%) 461,673k (± 0.07%) -450k (- 0.10%) 461,352k 462,119k p=0.045 n=6
Parse Time 3.16s (± 0.62%) 3.16s (± 0.99%) ~ 3.13s 3.21s p=0.558 n=6
Bind Time 1.17s (± 0.44%) 1.18s (± 1.03%) ~ 1.17s 1.20s p=0.923 n=6
Check Time 17.97s (± 0.53%) 17.93s (± 0.25%) ~ 17.85s 17.97s p=0.336 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.31s (± 0.45%) 22.27s (± 0.23%) ~ 22.17s 22.32s p=0.146 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,277ms (± 0.43%) 2,296ms (± 0.60%) ~ 2,279ms 2,310ms p=0.065 n=6
Req 2 - geterr 5,081ms (± 0.61%) 5,080ms (± 0.30%) ~ 5,065ms 5,101ms p=0.810 n=6
Req 3 - references 265ms (± 0.40%) 264ms (± 0.41%) ~ 262ms 265ms p=0.554 n=6
Req 4 - navto 228ms (± 0.18%) 227ms (± 1.40%) ~ 223ms 231ms p=0.445 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 88ms (± 0.58%) 89ms (± 7.13%) ~ 77ms 94ms p=0.276 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,393ms (± 0.49%) 2,418ms (± 1.14%) ~ 2,369ms 2,450ms p=0.093 n=6
Req 2 - geterr 3,787ms (± 0.16%) 3,795ms (± 0.24%) ~ 3,783ms 3,807ms p=0.172 n=6
Req 3 - references 276ms (± 0.30%) 277ms (± 0.32%) +2ms (+ 0.54%) 276ms 278ms p=0.021 n=6
Req 4 - navto 226ms (± 0.23%) 226ms (± 0.33%) ~ 225ms 227ms p=0.784 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 69ms (± 0.75%) 73ms (± 1.11%) 🔻+5ms (+ 6.80%) 72ms 74ms p=0.004 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 5,192ms (± 0.73%) 5,197ms (± 0.86%) ~ 5,168ms 5,287ms p=0.936 n=6
Req 2 - geterr 1,121ms (± 0.35%) 1,127ms (± 1.87%) ~ 1,111ms 1,168ms p=0.936 n=6
Req 3 - references 76ms 77ms (± 3.28%) ~ 75ms 82ms p=0.599 n=6
Req 4 - navto 450ms (± 0.70%) 455ms (± 0.72%) +5ms (+ 1.00%) 448ms 457ms p=0.036 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 828ms (± 2.25%) 837ms (± 1.61%) ~ 816ms 856ms p=0.470 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 157.02ms (± 0.17%) 156.97ms (± 0.19%) -0.05ms (- 0.03%) 156.03ms 162.11ms p=0.004 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 241.66ms (± 0.15%) 232.55ms (± 0.14%) 🟩-9.11ms (- 3.77%) 231.22ms 237.16ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 236.71ms (± 0.13%) 227.62ms (± 0.14%) 🟩-9.08ms (- 3.84%) 226.05ms 231.44ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 236.43ms (± 0.14%) 227.29ms (± 0.15%) 🟩-9.14ms (- 3.86%) 225.50ms 232.15ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member Author

Yeah that's 4% startup time.

@DanielRosenwasser
Copy link
Member

@typescript-bot cherry-pick this to release-5.5

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 19, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
cherry-pick this to release-5.5 ✅ Started ✅ Results

@typescript-bot
Copy link
Collaborator

Hey, @DanielRosenwasser! I've created #58933 for you.

@jakebailey jakebailey merged commit 47bb090 into microsoft:main Jun 19, 2024
28 checks passed
@jakebailey jakebailey deleted the defer-barebones-lib branch June 19, 2024 18:34
DanielRosenwasser pushed a commit that referenced this pull request Jun 19, 2024
…e-5.5 (#58933)

Co-authored-by: Jake Bailey <5341706+jakebailey@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

4 participants