Skip to content

Fix perf regression from #42556 #43949

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 7, 2021

Conversation

amcasey
Copy link
Member

@amcasey amcasey commented May 4, 2021

PR #42556 was a nice optimization that dramatically sped up comparisons of discriminated unions. Unfortunately, the cost of determining whether a union is discriminated can be prohibitively high. In particular, an internal team with a very large repo saw their type count double and their memory usage increase from 6GB to 9GB, breaking their build. This changes splits the difference by not trying to compute the property types of intersection types - a notoriously slow operation.

Fixes #

PR microsoft#42556 was a nice optimization that dramatically sped up comparisons of discriminated unions.  Unfortunately, the cost of determining whether a union is discriminated can be prohibitively high.  In particular, an internal team with a very large repo saw their type count double and their memory usage increase from 6GB to 9GB, breaking their build.  This changes splits the difference by not trying to compute the property types of intersection types - a notoriously slow operation.
@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels May 4, 2021
Copy link
Member

@weswigham weswigham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, the fail case is not using this fast path for identifying which union member to compare to, so this seems fine; though it's maybe worth noting that changes to the input program (namely some more comparisons between types) could make it potentially bloom to this size anyway, since it's not like we were manifesting anything that might not have been made manifest for other checks.

@amcasey
Copy link
Member Author

amcasey commented May 4, 2021

@weswigham I think you're pointing out that this will slow down some compilations (those that would have tugged on the properties anyway and were using discriminated unions), but only to the performance they were at before #42556. Did I get that right?

@weswigham
Copy link
Member

Yeah.

@DanielRosenwasser
Copy link
Member

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 5, 2021

Heya @DanielRosenwasser, I've started to run the perf test suite on this PR at abfa620. You can monitor the build here.

Update: The results are in!

@DanielRosenwasser
Copy link
Member

@ahejlsberg we'd like to land this for the RC in case you want to take a quick look.

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..43949

Metric master 43949 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 344,723k (± 0.03%) 344,723k (± 0.02%) -1k (- 0.00%) 344,567k 344,876k
Parse Time 1.91s (± 0.37%) 1.92s (± 0.69%) +0.01s (+ 0.47%) 1.90s 1.95s
Bind Time 0.84s (± 0.66%) 0.84s (± 0.83%) -0.00s (- 0.12%) 0.83s 0.86s
Check Time 5.26s (± 0.44%) 5.27s (± 0.56%) +0.02s (+ 0.29%) 5.20s 5.34s
Emit Time 5.92s (± 0.45%) 5.92s (± 0.79%) +0.00s (+ 0.07%) 5.85s 6.07s
Total Time 13.92s (± 0.26%) 13.95s (± 0.53%) +0.03s (+ 0.21%) 13.82s 14.20s
Compiler-Unions - node (v10.16.3, x64)
Memory used 200,687k (± 0.10%) 200,735k (± 0.03%) +49k (+ 0.02%) 200,613k 200,899k
Parse Time 0.78s (± 0.83%) 0.79s (± 1.39%) +0.01s (+ 0.77%) 0.77s 0.81s
Bind Time 0.53s (± 1.29%) 0.53s (± 1.10%) +0.00s (+ 0.19%) 0.51s 0.54s
Check Time 7.53s (± 0.56%) 7.54s (± 0.84%) +0.01s (+ 0.09%) 7.45s 7.73s
Emit Time 2.51s (± 1.23%) 2.52s (± 1.38%) +0.00s (+ 0.12%) 2.42s 2.57s
Total Time 11.35s (± 0.49%) 11.37s (± 0.72%) +0.01s (+ 0.12%) 11.18s 11.61s
Monaco - node (v10.16.3, x64)
Memory used 341,671k (± 0.02%) 341,658k (± 0.02%) -13k (- 0.00%) 341,548k 341,811k
Parse Time 1.56s (± 0.59%) 1.55s (± 0.88%) -0.01s (- 0.58%) 1.53s 1.59s
Bind Time 0.74s (± 0.64%) 0.74s (± 1.08%) -0.00s (- 0.13%) 0.73s 0.77s
Check Time 5.37s (± 0.31%) 5.39s (± 0.44%) +0.02s (+ 0.39%) 5.36s 5.47s
Emit Time 3.01s (± 0.74%) 3.03s (± 0.62%) +0.02s (+ 0.76%) 2.99s 3.07s
Total Time 10.68s (± 0.43%) 10.72s (± 0.33%) +0.04s (+ 0.33%) 10.64s 10.80s
TFS - node (v10.16.3, x64)
Memory used 304,306k (± 0.02%) 304,244k (± 0.04%) -62k (- 0.02%) 304,077k 304,682k
Parse Time 1.22s (± 0.39%) 1.21s (± 0.67%) -0.01s (- 0.58%) 1.19s 1.22s
Bind Time 0.70s (± 0.71%) 0.70s (± 0.70%) -0.00s (- 0.14%) 0.69s 0.71s
Check Time 4.82s (± 0.57%) 4.81s (± 0.53%) -0.01s (- 0.23%) 4.74s 4.87s
Emit Time 3.18s (± 1.01%) 3.20s (± 2.08%) +0.02s (+ 0.63%) 3.10s 3.37s
Total Time 9.92s (± 0.34%) 9.93s (± 0.92%) +0.00s (+ 0.04%) 9.80s 10.16s
material-ui - node (v10.16.3, x64)
Memory used 474,615k (± 0.01%) 473,859k (± 0.01%) -756k (- 0.16%) 473,783k 473,949k
Parse Time 1.94s (± 0.75%) 1.95s (± 0.51%) +0.01s (+ 0.62%) 1.92s 1.97s
Bind Time 0.65s (± 1.04%) 0.65s (± 1.39%) -0.00s (- 0.46%) 0.64s 0.67s
Check Time 14.80s (± 0.89%) 14.88s (± 1.04%) +0.08s (+ 0.56%) 14.57s 15.19s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 17.39s (± 0.80%) 17.48s (± 0.92%) +0.09s (+ 0.53%) 17.17s 17.81s
Angular - node (v12.1.0, x64)
Memory used 322,335k (± 0.02%) 322,309k (± 0.03%) -26k (- 0.01%) 322,120k 322,496k
Parse Time 1.91s (± 0.67%) 1.91s (± 0.52%) -0.00s (- 0.10%) 1.88s 1.92s
Bind Time 0.83s (± 1.38%) 0.83s (± 1.22%) -0.01s (- 0.84%) 0.81s 0.85s
Check Time 5.15s (± 0.47%) 5.17s (± 0.49%) +0.02s (+ 0.41%) 5.11s 5.22s
Emit Time 6.12s (± 0.32%) 6.17s (± 0.63%) +0.04s (+ 0.69%) 6.10s 6.29s
Total Time 14.01s (± 0.23%) 14.07s (± 0.32%) +0.06s (+ 0.42%) 13.96s 14.17s
Compiler-Unions - node (v12.1.0, x64)
Memory used 187,889k (± 0.22%) 188,060k (± 0.16%) +171k (+ 0.09%) 187,303k 188,558k
Parse Time 0.77s (± 0.80%) 0.77s (± 0.80%) +0.00s (+ 0.26%) 0.76s 0.79s
Bind Time 0.53s (± 0.89%) 0.54s (± 1.28%) +0.01s (+ 0.94%) 0.53s 0.56s
Check Time 7.06s (± 1.16%) 7.05s (± 0.57%) -0.01s (- 0.17%) 6.95s 7.13s
Emit Time 2.50s (± 1.89%) 2.49s (± 1.54%) -0.02s (- 0.64%) 2.42s 2.59s
Total Time 10.86s (± 0.92%) 10.84s (± 0.50%) -0.02s (- 0.18%) 10.73s 10.97s
Monaco - node (v12.1.0, x64)
Memory used 324,057k (± 0.01%) 324,046k (± 0.02%) -12k (- 0.00%) 323,956k 324,166k
Parse Time 1.54s (± 0.81%) 1.53s (± 0.49%) -0.01s (- 0.39%) 1.52s 1.55s
Bind Time 0.72s (± 0.51%) 0.72s (± 0.62%) +0.00s (+ 0.28%) 0.71s 0.73s
Check Time 5.20s (± 0.46%) 5.21s (± 0.44%) +0.01s (+ 0.19%) 5.17s 5.27s
Emit Time 3.07s (± 0.66%) 3.05s (± 0.50%) -0.02s (- 0.62%) 3.02s 3.08s
Total Time 10.52s (± 0.36%) 10.51s (± 0.33%) -0.01s (- 0.10%) 10.43s 10.61s
TFS - node (v12.1.0, x64)
Memory used 288,731k (± 0.02%) 288,731k (± 0.02%) -0k (- 0.00%) 288,635k 288,847k
Parse Time 1.21s (± 0.69%) 1.22s (± 0.46%) +0.01s (+ 1.08%) 1.21s 1.23s
Bind Time 0.70s (± 1.27%) 0.70s (± 0.74%) +0.00s (+ 0.29%) 0.69s 0.71s
Check Time 4.70s (± 0.40%) 4.69s (± 0.33%) -0.01s (- 0.17%) 4.65s 4.73s
Emit Time 3.19s (± 0.54%) 3.20s (± 0.43%) +0.01s (+ 0.19%) 3.18s 3.23s
Total Time 9.79s (± 0.37%) 9.80s (± 0.28%) +0.01s (+ 0.13%) 9.76s 9.89s
material-ui - node (v12.1.0, x64)
Memory used 452,406k (± 0.06%) 451,584k (± 0.06%) -822k (- 0.18%) 450,585k 451,800k
Parse Time 1.95s (± 0.55%) 1.94s (± 0.48%) -0.01s (- 0.67%) 1.92s 1.96s
Bind Time 0.64s (± 0.78%) 0.64s (± 0.91%) +0.00s (+ 0.31%) 0.62s 0.65s
Check Time 13.35s (± 0.55%) 13.29s (± 0.54%) -0.06s (- 0.47%) 13.12s 13.51s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.94s (± 0.50%) 15.86s (± 0.42%) -0.07s (- 0.45%) 15.73s 16.08s
Angular - node (v14.15.1, x64)
Memory used 321,033k (± 0.01%) 321,036k (± 0.01%) +4k (+ 0.00%) 320,995k 321,062k
Parse Time 1.91s (± 0.53%) 1.91s (± 0.61%) +0.01s (+ 0.47%) 1.89s 1.94s
Bind Time 0.87s (± 0.64%) 0.88s (± 0.57%) +0.01s (+ 0.81%) 0.87s 0.89s
Check Time 5.16s (± 0.43%) 5.17s (± 0.27%) +0.02s (+ 0.33%) 5.15s 5.22s
Emit Time 6.21s (± 0.34%) 6.23s (± 0.61%) +0.02s (+ 0.29%) 6.13s 6.33s
Total Time 14.14s (± 0.32%) 14.19s (± 0.33%) +0.05s (+ 0.35%) 14.07s 14.31s
Compiler-Unions - node (v14.15.1, x64)
Memory used 188,484k (± 0.63%) 188,843k (± 0.61%) +359k (+ 0.19%) 186,872k 190,139k
Parse Time 0.80s (± 0.83%) 0.80s (± 0.59%) +0.00s (+ 0.37%) 0.79s 0.81s
Bind Time 0.56s (± 0.90%) 0.56s (± 0.67%) 0.00s ( 0.00%) 0.55s 0.56s
Check Time 7.12s (± 0.50%) 7.16s (± 0.82%) +0.05s (+ 0.65%) 7.08s 7.30s
Emit Time 2.51s (± 1.07%) 2.50s (± 1.23%) -0.01s (- 0.32%) 2.45s 2.61s
Total Time 10.98s (± 0.46%) 11.03s (± 0.71%) +0.04s (+ 0.39%) 10.91s 11.28s
Monaco - node (v14.15.1, x64)
Memory used 323,149k (± 0.00%) 323,164k (± 0.00%) +15k (+ 0.00%) 323,111k 323,190k
Parse Time 1.57s (± 0.44%) 1.57s (± 0.79%) +0.00s (+ 0.32%) 1.55s 1.60s
Bind Time 0.75s (± 0.49%) 0.75s (± 0.77%) +0.00s (+ 0.54%) 0.74s 0.76s
Check Time 5.19s (± 0.40%) 5.22s (± 0.37%) +0.03s (+ 0.64%) 5.17s 5.27s
Emit Time 3.13s (± 0.60%) 3.14s (± 0.99%) +0.01s (+ 0.42%) 3.09s 3.24s
Total Time 10.63s (± 0.35%) 10.69s (± 0.28%) +0.06s (+ 0.54%) 10.61s 10.75s
TFS - node (v14.15.1, x64)
Memory used 287,658k (± 0.01%) 287,637k (± 0.01%) -21k (- 0.01%) 287,595k 287,664k
Parse Time 1.26s (± 0.85%) 1.28s (± 1.98%) +0.02s (+ 1.35%) 1.23s 1.34s
Bind Time 0.71s (± 0.67%) 0.72s (± 0.90%) +0.01s (+ 0.70%) 0.71s 0.74s
Check Time 4.73s (± 0.48%) 4.74s (± 0.47%) +0.01s (+ 0.19%) 4.70s 4.81s
Emit Time 3.29s (± 0.90%) 3.29s (± 0.61%) -0.00s (- 0.12%) 3.22s 3.33s
Total Time 10.00s (± 0.42%) 10.03s (± 0.44%) +0.03s (+ 0.25%) 9.93s 10.17s
material-ui - node (v14.15.1, x64)
Memory used 450,721k (± 0.01%) 449,807k (± 0.07%) -914k (- 0.20%) 448,764k 450,107k
Parse Time 1.98s (± 0.59%) 1.98s (± 0.54%) -0.00s (- 0.00%) 1.96s 2.01s
Bind Time 0.70s (± 0.64%) 0.70s (± 0.52%) -0.00s (- 0.29%) 0.69s 0.70s
Check Time 13.52s (± 0.58%) 13.55s (± 1.10%) +0.03s (+ 0.23%) 13.34s 14.03s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 16.20s (± 0.49%) 16.23s (± 0.95%) +0.03s (+ 0.20%) 16.01s 16.72s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-206-generic
Architecturex64
Available Memory16 GB
Available Memory7 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v10.16.3, x64)
  • Compiler-Unions - node (v12.1.0, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v14.15.1, x64)
Benchmark Name Iterations
Current 43949 10
Baseline master 10

Developer Information:

Download Benchmark

@amcasey amcasey merged commit 1ce15a4 into microsoft:master May 7, 2021
@amcasey amcasey deleted the DiscriminatedUnionRegression branch May 7, 2021 00:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants