Tests: wierd compilation errors

[majocha]

Recently started showing up.
For example:

  Failed EmittedIL.Structure.ModuleArity01_fs(compilation: File: ModuleArity01.fs) [1 s]
  Error Message:
   System.Exception : Mismatch in Errors, expected '0', got '1'.
All errors:
[(Error 193, Line 136, Col 7, Line 136, Col 33, "Type constraint mismatch. The type 
    'Type'    
is not compatible with type
    'MemberInfo'    
")]
  Stack Trace:
     at Microsoft.FSharp.Core.PrintfModule.PrintFormatToStringThenFail@1448.Invoke(String message)
   at Microsoft.FSharp.Core.PrintfModule.gprintf[a,TState,TResidue,TResult,TPrinter](FSharpFunc`2 envf, PrintfFormat`4 format) in D:\a\_work\1\s\src\FSharp.Core\printf.fs:line 1398
   at FSharp.Test.CompilerAssertHelpers.assertErrors(Int32 libAdjust, Boolean ignoreWarnings, FSharpDiagnostic[] errors, Tuple`4[] expectedErrors) in D:\a\_work\1\s\tests\FSharp.Test.Utilities\CompilerAssert.fs:line 503
   at FSharp.Test.CompilerAssertHelpers.evaluateReferences@574.Invoke(Tuple`2 tupledArg) in D:\a\_work\1\s\tests\FSharp.Test.Utilities\CompilerAssert.fs:line 576
   at Microsoft.FSharp.Primitives.Basics.List.map[T,TResult](FSharpFunc`2 mapping, FSharpList`1 x) in D:\a\_work\1\s\src\FSharp.Core\local.fs:line 247
   at FSharp.Test.CompilerAssertHelpers.evaluateReferences(DirectoryInfo outputDir, Boolean ignoreWarnings, Compilation cmpl) in D:\a\_work\1\s\tests\FSharp.Test.Utilities\CompilerAssert.fs:line 573
   at FSharp.Test.CompilerAssertHelpers.compileCompilationAux(DirectoryInfo outputDirectory, Boolean ignoreWarnings, Compilation cmpl) in D:\a\_work\1\s\tests\FSharp.Test.Utilities\CompilerAssert.fs:line 534
   at FSharp.Test.Compiler.compileFSharpCompilation(Compilation compilation, Boolean ignoreWarnings, CompilationUnit cUnit) in D:\a\_work\1\s\tests\FSharp.Test.Utilities\Compiler.fs:line 828
   at FSharp.Test.Compiler.compileAndRun@1115.Invoke(CompilationUnit x)
   at EmittedIL.Structure.verifyExecution(CompilationUnit compilation) in D:\a\_work\1\s\tests\FSharp.Compiler.ComponentTests\EmittedIL\Structure\Structure.fs:line 31
   at EmittedIL.Structure.ModuleArity01_fs(CompilationHelper compilation) in D:\a\_work\1\s\tests\FSharp.Compiler.ComponentTests\EmittedIL\Structure\Structure.fs:line 182

Possibly it is because of reusing the checker while the same generated filename "test.fs" shows up in multiple tests?

[T-Gro]

On main history, its green most of the time, but I do see flukes as well.
https://github.com/dotnet/fsharp/commits/main/

I have seen it at some PR already, and if I am not mistaken, it has always been Type constraint mismatch. The type.. ; wasn't it?

If I had to guess, type subsumption cache would be a candidate - possibly affecting this via a non trivial chain.

(type inference -> method overloading selection -> only then producing an error)

The error mentioned here comes from a block which uses helpers from https://github.com/dotnet/fsharp/blob/b3bcf48df551a84db25bcfde12598d3525e0e9ff/tests/FSharp.Compiler.ComponentTests/EmittedIL/Structure/CodeGenHelper.fs , could it be that sharing of this file is what causes the issue?

It likely is the same file used across more tests?

[majocha]

It makes sense! I'll try to keep an eye on it. With more examples maybe a solid repro would emerge.

[majocha]

Another one from yesterday:

  Failed EmittedIL.RealInternalSignature.ModuleInitialization.Simple TypeOne and TypeTwo and TypeThree in module(realSig: True, recursiveModule: False) [877 ms]
  Error Message:
   System.Exception : Compilation should be successful in order to run.
 Errors: [{ Error = Warning 760
   Range = { StartLine = 13
             StartColumn = 31
             EndLine = 13
             EndColumn = 46 }
   NativeRange = (13,31--13,46)
   Message =
    "It is recommended that objects supporting the IDisposable interface are created using the syntax 'new Type(args)', rather than 'Type(args)' or 'Type' as a function value representing the constructor, to indicate that resources may be owned by the generated value"
   SubCategory = "typecheck" }]
  Stack Trace:
     at FSharp.Test.Compiler.run(CompilationResult result) in D:\a\_work\1\s\tests\FSharp.Test.Utilities\Compiler.fs:line 1100
   at EmittedIL.RealInternalSignature.ModuleInitialization.Simple TypeOne and TypeTwo and TypeThree in module(Boolean realSig, Boolean recursiveModule) in D:\a\_work\1\s\tests\FSharp.Compiler.ComponentTests\EmittedIL\RealInternalSignature\ModuleInitialization.fs:line 401

and also #18786 (comment)

[majocha]

Ok, I have a repro just by "running until failure" the EmittedIL.Structure net472 tests in VS.

[majocha]

So I was able to confirm this is 100% TypeSubsumptionCache related.
I can repro this locally, and I can make it go away by switching off the feature.

The cache itself does not do anything fancy. Unfortunately we're probably hitting some limitation of stampEquals when comparing keys.

fsharp/src/Compiler/Checking/import.fs

Line 78 in c853846

HashStamps.stampEquals this.ty1 other.ty1

[majocha]

The cache currently is a truly global singleton and this happens when there are many compilations going on at thesame time.
With one cache instance per tcGlobals instead, the bug no longer manifests.

[majocha]

I'm now more convinced this is a concurrency issue within the Cache.

[majocha]

This is subtly racey.
This can happen:

fsharp/src/Compiler/Utilities/Caches.fs

Lines 50 to 53 in 4410428

	member this.ReUse(key, value) =
	this.key <- key
	this.value <- value
	this

While we read the value on another thread:

fsharp/src/Compiler/Utilities/Caches.fs

Lines 251 to 257 in 4410428

	member _.TryGetValue(key: 'Key, value: outref<'Value>) =
	match store.TryGetValue(key) with
	| true, entity ->
	hits.Add 1L
	evictionProcessor.Post(EvictionQueueMessage.Update entity)
	value <- entity.Value
	true

That's the problem when some things are governed by mailboxprocessor but not all. Things quickly become very tricky to get right.

[majocha]

I think #18786 fixes this now.