Skip to content

Do not erase signature constraints when calculating variance #55864

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

weswigham
Copy link
Member

Fixes #55671

As noted in the linked issue, this is, unfortunately, probably in-use to mask a bunch of real issues (or as a workaround for other issues) in the wild, since it was an easy to make a type that would "accept" any assignments between instances of the same alias, even though once you did a structural comparison it only accepted near-exact matches. Even in our test suite it comes up a couple times in some examples we've pulled from the wild - I expect DT and top100 to have more.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Sep 25, 2023
@typescript-bot
Copy link
Collaborator

Looks like you're introducing a change to the public API surface area. If this includes breaking changes, please document them on our wiki's API Breaking Changes page.

Also, please make sure @DanielRosenwasser and @RyanCavanaugh are aware of the changes, just as a heads up.

@weswigham
Copy link
Member Author

@typescript-bot pack this
@typescript-bot user test this
@typescript-bot test this
@typescript-bot run dt
@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 25, 2023

Heya @weswigham, I've started to run the tarball bundle task on this PR at bc9d806. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 25, 2023

Heya @weswigham, I've started to run the parallelized Definitely Typed test suite on this PR at bc9d806. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 25, 2023

Heya @weswigham, I've started to run the regular perf test suite on this PR at bc9d806. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 25, 2023

Heya @weswigham, I've started to run the diff-based user code test suite on this PR at bc9d806. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

Hey @weswigham, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/157930/artifacts?artifactName=tgz&fileId=756BC6A23A8214E69D96856325D91C698C04FE8174B1A180F2E40F1DE5EF112902&fileName=/typescript-5.3.0-insiders.20230925.tgz"
    }
}

and then running npm install.

@typescript-bot
Copy link
Collaborator

@weswigham Here are the results of running the user test suite comparing main and refs/pull/55864/merge:

There were infrastructure failures potentially unrelated to your change:

  • 1 instance of "Unknown failure"
  • 2 instances of "Package install failed"

Otherwise...

Something interesting changed - please have a look.

Details

fp-ts

dtslint/ts3.5/tsconfig.json

examples/tsconfig.json

  • [NEW] error TS2322: Type '<A>(first: NonEmptyArray<A | B>) => (B | A)[]' is not assignable to type '<A>(first: NonEmptyArray<A>) => (B | A)[]'.

tsconfig.build-es6.json

  • [NEW] error TS2322: Type '<A>(first: NonEmptyArray<A | B>) => (B | A)[]' is not assignable to type '<A>(first: NonEmptyArray<A>) => (B | A)[]'.

tsconfig.eslint.json

  • [NEW] error TS2345: Argument of type '(as: NonEmptyArray<unknown>) => Option<NonEmptyArray<number>>' is not assignable to parameter of type '(a: NonEmptyArray<string | number>) => Option<NonEmptyArray<number>>'.
  • [NEW] error TS2345: Argument of type 'NonEmptyArray<string | number>' is not assignable to parameter of type 'NonEmptyArray<unknown>'.
  • [NEW] error TS2322: Type '<A>(first: NonEmptyArray<A | B>) => (B | A)[]' is not assignable to type '<A>(first: NonEmptyArray<A>) => (B | A)[]'.

tsconfig.json

@typescript-bot
Copy link
Collaborator

@weswigham
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 294,923k (± 0.02%) 295,002k (± 0.02%) +79k (+ 0.03%) 294,944k 295,072k p=0.020 n=6
Parse Time 2.61s (± 0.59%) 2.61s (± 0.78%) ~ 2.59s 2.63s p=0.868 n=6
Bind Time 0.83s (± 0.00%) 0.83s (± 1.18%) ~ 0.82s 0.85s p=1.000 n=6
Check Time 8.06s (± 0.24%) 8.07s (± 0.24%) ~ 8.04s 8.09s p=0.744 n=6
Emit Time 7.06s (± 0.19%) 7.04s (± 0.28%) ~ 7.01s 7.07s p=0.122 n=6
Total Time 18.56s (± 0.12%) 18.55s (± 0.25%) ~ 18.47s 18.60s p=1.000 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 192,653k (± 1.54%) 191,186k (± 0.02%) ~ 191,111k 191,241k p=0.378 n=6
Parse Time 1.34s (± 1.13%) 1.36s (± 1.47%) ~ 1.33s 1.38s p=0.216 n=6
Bind Time 0.73s (± 0.00%) 0.73s (± 0.00%) ~ 0.73s 0.73s p=1.000 n=6
Check Time 9.11s (± 0.39%) 9.16s (± 0.43%) ~ 9.10s 9.21s p=0.064 n=6
Emit Time 2.64s (± 0.52%) 2.65s (± 0.67%) ~ 2.63s 2.68s p=0.192 n=6
Total Time 13.82s (± 0.22%) 13.90s (± 0.33%) +0.08s (+ 0.58%) 13.83s 13.96s p=0.013 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,223k (± 0.01%) 347,211k (± 0.01%) ~ 347,194k 347,244k p=0.335 n=6
Parse Time 2.46s (± 0.36%) 2.45s (± 0.33%) ~ 2.44s 2.46s p=0.270 n=6
Bind Time 0.94s (± 0.67%) 0.94s (± 0.00%) ~ 0.94s 0.94s p=1.000 n=6
Check Time 6.87s (± 0.26%) 6.86s (± 0.28%) ~ 6.84s 6.89s p=0.369 n=6
Emit Time 4.01s (± 0.34%) 4.02s (± 0.30%) ~ 4.00s 4.03s p=0.452 n=6
Total Time 14.28s (± 0.19%) 14.27s (± 0.10%) ~ 14.25s 14.29s p=0.806 n=6
TFS - node (v18.15.0, x64)
Memory used 302,474k (± 0.00%) 302,487k (± 0.00%) ~ 302,473k 302,497k p=0.128 n=6
Parse Time 2.00s (± 0.42%) 2.01s (± 0.92%) ~ 1.99s 2.04s p=0.452 n=6
Bind Time 1.00s (± 0.41%) 1.00s (± 0.83%) ~ 1.00s 1.02s p=0.115 n=6
Check Time 6.25s (± 0.31%) 6.23s (± 0.29%) ~ 6.21s 6.25s p=0.226 n=6
Emit Time 3.53s (± 1.01%) 3.53s (± 0.79%) ~ 3.50s 3.58s p=0.935 n=6
Total Time 12.78s (± 0.16%) 12.78s (± 0.22%) ~ 12.73s 12.80s p=0.871 n=6
material-ui - node (v18.15.0, x64)
Memory used 470,440k (± 0.01%) 470,535k (± 0.00%) +94k (+ 0.02%) 470,515k 470,554k p=0.005 n=6
Parse Time 2.58s (± 0.87%) 2.57s (± 0.60%) ~ 2.56s 2.59s p=0.666 n=6
Bind Time 1.00s (± 1.04%) 1.00s (± 0.55%) ~ 0.99s 1.00s p=0.663 n=6
Check Time 16.56s (± 0.11%) 16.53s (± 0.31%) ~ 16.48s 16.63s p=0.086 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.14s (± 0.18%) 20.10s (± 0.23%) ~ 20.06s 20.19s p=0.063 n=6
xstate - node (v18.15.0, x64)
Memory used 512,531k (± 0.02%) 513,275k (± 0.01%) +745k (+ 0.15%) 513,214k 513,391k p=0.005 n=6
Parse Time 3.27s (± 0.26%) 3.27s (± 0.19%) ~ 3.26s 3.28s p=0.226 n=6
Bind Time 1.55s (± 0.48%) 1.54s (± 1.04%) ~ 1.51s 1.55s p=0.209 n=6
Check Time 2.82s (± 0.35%) 2.91s (± 0.68%) +0.09s (+ 3.13%) 2.89s 2.94s p=0.004 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 0.00%) ~ 0.08s 0.08s p=1.000 n=6
Total Time 7.71s (± 0.11%) 7.79s (± 0.39%) +0.09s (+ 1.12%) 7.75s 7.84s p=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,348ms (± 0.79%) 2,342ms (± 1.05%) ~ 2,322ms 2,389ms p=0.575 n=6
Req 2 - geterr 5,314ms (± 1.29%) 5,473ms (± 1.25%) +160ms (+ 3.00%) 5,340ms 5,526ms p=0.013 n=6
Req 3 - references 327ms (± 0.97%) 329ms (± 1.18%) ~ 325ms 334ms p=0.466 n=6
Req 4 - navto 278ms (± 1.26%) 275ms (± 0.74%) ~ 274ms 279ms p=0.128 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 81ms (± 8.68%) 87ms (± 6.38%) ~ 76ms 90ms p=0.276 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,466ms (± 1.13%) 2,456ms (± 1.35%) ~ 2,409ms 2,491ms p=0.575 n=6
Req 2 - geterr 4,173ms (± 1.87%) 4,175ms (± 1.85%) ~ 4,065ms 4,236ms p=0.810 n=6
Req 3 - references 337ms (± 1.51%) 336ms (± 1.30%) ~ 332ms 344ms p=0.745 n=6
Req 4 - navto 283ms (± 0.27%) 283ms (± 0.41%) ~ 281ms 284ms p=0.734 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 79ms (± 7.54%) 78ms (± 6.41%) ~ 75ms 88ms p=0.929 n=6
xstateTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,595ms (± 0.40%) 2,595ms (± 0.51%) ~ 2,580ms 2,615ms p=1.000 n=6
Req 2 - geterr 1,711ms (± 0.82%) 1,735ms (± 1.89%) ~ 1,694ms 1,776ms p=0.229 n=6
Req 3 - references 112ms (± 8.06%) 115ms (± 8.17%) ~ 106ms 126ms p=0.628 n=6
Req 4 - navto 365ms (± 1.65%) 360ms (± 0.64%) ~ 357ms 363ms p=0.060 n=6
Req 5 - completionInfo count 2,071 (± 0.00%) 2,071 (± 0.00%) ~ 2,071 2,071 p=1.000 n=6
Req 5 - completionInfo 302ms (± 2.38%) 309ms (± 1.74%) ~ 301ms 315ms p=0.078 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstateTSServer - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 151.67ms (± 0.16%) 151.67ms (± 0.15%) ~ 150.58ms 153.67ms p=0.761 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 227.90ms (± 0.15%) 227.86ms (± 0.15%) ~ 226.32ms 230.82ms p=0.306 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 229.13ms (± 0.18%) 229.13ms (± 0.18%) ~ 227.15ms 233.94ms p=0.894 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 229.32ms (± 0.20%) 229.37ms (± 0.18%) ~ 227.65ms 236.72ms p=0.083 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@@ -339,6 +347,15 @@ immutable.ts(391,22): error TS2430: Interface 'Set<T>' incorrectly extends inter
export function Set<T>(): Seq.Set<T>;
export function Set<T>(collection: Iterable<T>): Seq.Set<T>;
export interface Set<T> extends Seq<never, T>, Collection.Set<T> {
~~~
!!! error TS2430: Interface 'Set<T>' incorrectly extends interface 'Seq<never, T>'.
!!! error TS2430: The types of 'map(...).filter(...).concat(...).toMap().toSeq().toOrderedMap().keySeq().toMap().mapKeys(...).toSeq().toOrderedMap().toSet().union' are incompatible between these types.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😵

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We go hard at finding the incompatibility, don't we. 😆

Guess there's an any somewhere in this type definition that should be a never (or vice-versa, depending on intent).

@typescript-bot
Copy link
Collaborator

Hey @weswigham, the results of running the DT tests are ready.
There were interesting changes:
Changes are too big to display here, please check the log.
You can check the log here.

@weswigham
Copy link
Member Author

DT reported as "a lot of problems", but it's just a lot of packages depending on immutable (which has the same extends error as in the test here - so I should probably figure out where an any needs to be a never (or vice-versa) and fix it), and dependencies on fp-ts (which, just like with the user suite, is an array subtype we now measure as invariant instead of covariant (which you can work around with an out annotation on the type parameter, as I've done in our own API). There's also a few instances of the propTypes circularity that's now found in the tests (usually via styled-components, which is where the test comes from), too. Not really sure what the workaround for that one should be yet, since the types are quite complicated. And lastly, there's some dependencies on a package called clownface, which also is detected as having an incorrect extends now (albeit much less deeply nested than immutable!). There's a few others, too (which are usually array or collection subtyping issues caused by more accurate variance calculation - zod and bluebird come to mind), but those are the ones that infect a lot of packages on DT by proxy.

I think this is a change we can take, once we get those big offenders fixed. Though the array subtype issue might warrant some special-casing in code, even though you can work around the issue with an annotation; arrays are forced to be covariant, whereas if we structurally inspect them (like we do their subtypes), they're measured as invariant (which is more correct! just.... harder to use). It's probably worth detecting array subtypes and short-circuiting their variance calculations, so subtypes have the same hardcoded-covariant behavior array itself does. That change can probably stand on its' own, too, since it's likely performance-positive w.r.t. variance calculation for array subtypes (unlike this change, which can only be bad for perf, since it forces us to make more comparisons with a stricter variance result).

@fatcerberus
Copy link

It’s interesting to me that method bivariance alone isn’t enough to keep arrays covariant (though it does help with user-defined arraylikes, as pointed out by Ryan in the array subtype covariance PR)

@weswigham weswigham added Breaking Change Would introduce errors in existing code Experiment A fork with an experimental idea which might not make it into master labels Aug 13, 2024
@weswigham weswigham marked this pull request as draft August 13, 2024 22:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team Breaking Change Would introduce errors in existing code Experiment A fork with an experimental idea which might not make it into master For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
Status: Waiting on author
Development

Successfully merging this pull request may close these issues.

Instantiations of constrained generic signatures allow all assignments
4 participants