Skip to content

Refactoring of IncrementalBuilder #14903

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 83 commits into from
Apr 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
83 commits
Select commit Hold shift + click to select a range
250230e
one pass mapFold, cache parse tasks
majocha Mar 15, 2023
4c2438a
Merge branch 'main' into parse-background
majocha Mar 15, 2023
1558d02
refactor a bit
majocha Mar 15, 2023
7b1e81b
ah, ok
majocha Mar 15, 2023
54a4683
works again?
majocha Mar 15, 2023
8c85688
Merge branch 'main' into parse-background
majocha Mar 15, 2023
7f14437
some refactoring
majocha Mar 16, 2023
2daf886
Merge branch 'main' into parse-background
majocha Mar 16, 2023
0fecfde
fix test for now
majocha Mar 16, 2023
d1a0b52
lock it
majocha Mar 16, 2023
a4938d5
better
majocha Mar 16, 2023
7c39bad
Merge branch 'main' into parse-background
majocha Mar 16, 2023
b60e3ae
cleanup
majocha Mar 16, 2023
7a30846
more correct?
majocha Mar 16, 2023
bd43464
not in use
majocha Mar 16, 2023
92f5183
restore test
majocha Mar 16, 2023
70cba99
Merge branch 'main' into parse-background
majocha Mar 16, 2023
419f456
wip
majocha Mar 16, 2023
b396602
merge
majocha Mar 20, 2023
0b4d952
refactor away partialCheck flag where possible
majocha Mar 21, 2023
00a2c47
Merge branch 'main' into parse-background
majocha Mar 21, 2023
2fbee03
fantomas
majocha Mar 21, 2023
6d7de17
Merge branch 'parse-background' of https://github.com/majocha/fsharp …
majocha Mar 21, 2023
f3acc7e
fix
majocha Mar 21, 2023
2934375
ok
majocha Mar 21, 2023
2411974
add this back
majocha Mar 21, 2023
28a8972
format
majocha Mar 21, 2023
70872da
fmt
majocha Mar 21, 2023
6470b96
Merge branch 'main' into parse-background
majocha Mar 21, 2023
025f8b2
Merge branch 'parse-background' of https://github.com/majocha/fsharp …
majocha Mar 21, 2023
fb4b46c
wip
majocha Mar 21, 2023
5e51581
Merge branch 'main' into parse-background
majocha Mar 22, 2023
915c693
restore enablePartialTypeChecking
majocha Mar 22, 2023
11db113
restore enablePartialTypeChecking
majocha Mar 22, 2023
cfc9874
fix conflict
majocha Mar 22, 2023
73043ea
Merge branch 'main' into parse-background
majocha Mar 22, 2023
1575559
naming stuff
majocha Mar 23, 2023
a9b18fe
fix test
majocha Mar 23, 2023
9b69486
task run
majocha Mar 29, 2023
e8ead2c
update docs
majocha Mar 29, 2023
5d25c48
Merge branch 'main' into parse-background
majocha Mar 29, 2023
78a0d20
use SemaphoreSlim
majocha Mar 29, 2023
a536721
Merge branch 'main' into parse-background
majocha Mar 29, 2023
1acaa82
add a comment
majocha Mar 29, 2023
e64795d
Merge branch 'parse-background' of https://github.com/majocha/fsharp …
majocha Mar 29, 2023
2ae1e75
fix
majocha Mar 29, 2023
493b3b2
revert to lock
majocha Mar 30, 2023
0a492db
Merge branch 'main' into parse-background
majocha Mar 31, 2023
632b3ba
make full type check more parallel
majocha Mar 31, 2023
81a5f39
Revert "make full type check more parallel"
majocha Mar 31, 2023
d7d34ee
Merge branch 'main' into parse-background
majocha Apr 3, 2023
65d4f81
refactor BoundModel, parallel typecheck when possible
majocha Apr 3, 2023
067d2a2
Merge branch 'main' into parse-background
majocha Apr 3, 2023
f0b1a53
Merge branch 'main' into parse-background
majocha Apr 3, 2023
59c3407
use GraphNode to keep parsing result
majocha Apr 3, 2023
eb635f9
Merge branch 'parse-background' of https://github.com/majocha/fsharp …
majocha Apr 3, 2023
e1f5c6e
fantomas
majocha Apr 3, 2023
3dbe7bc
Merge branch 'main' into parse-background
majocha Apr 3, 2023
5af0d9d
commited too much, fix
majocha Apr 3, 2023
75cb390
Merge branch 'parse-background' of https://github.com/majocha/fsharp …
majocha Apr 3, 2023
2b04ab5
fix bug, fix tests
majocha Apr 3, 2023
aeeb9e3
eager parsing
majocha Apr 4, 2023
3c47913
Merge branch 'main' into parse-background
majocha Apr 4, 2023
637e549
not useful and not deterministic
majocha Apr 4, 2023
e5480b5
remove eager parsing
majocha Apr 4, 2023
15a3d94
oh, well
majocha Apr 4, 2023
77ca7db
cleanup
majocha Apr 4, 2023
dd94afd
Merge branch 'main' into parse-background
majocha Apr 5, 2023
b0aff61
Merge branch 'main' into parse-background
majocha Apr 6, 2023
f064930
Merge branch 'main' into parse-background
majocha Apr 6, 2023
f14c952
Merge branch 'main' into parse-background
T-Gro Apr 7, 2023
144c26f
do not compute temp states
majocha Apr 11, 2023
a393ca4
Merge branch 'main' into parse-background
majocha Apr 11, 2023
18ce94a
Merge branch 'parse-background' of https://github.com/majocha/fsharp …
majocha Apr 11, 2023
902e2c4
remove private
majocha Apr 11, 2023
0a4ed38
Merge branch 'main' into parse-background
majocha Apr 11, 2023
d9a8def
reclaim type check memory faster
majocha Apr 12, 2023
2eb34eb
Merge branch 'main' into parse-background
majocha Apr 12, 2023
1a2ca9e
dont check when state is given
majocha Apr 12, 2023
7c755b8
Merge branch 'main' into parse-background
majocha Apr 12, 2023
0119b13
less ugly
majocha Apr 13, 2023
14241bd
Merge branch 'main' into parse-background
majocha Apr 13, 2023
2c8c7c2
Merge branch 'main' into parse-background
majocha Apr 14, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
177 changes: 84 additions & 93 deletions src/Compiler/Driver/ParseAndCheckInputs.fs
Original file line number Diff line number Diff line change
Expand Up @@ -1156,10 +1156,6 @@ let GetInitialTcState (m, ccuName, tcConfig: TcConfig, tcGlobals, tcImports: TcI
tcsImplicitOpenDeclarations = openDecls0
}

/// Dummy typed impl file that contains no definitions and is not used for emitting any kind of assembly.
let CreateEmptyDummyImplFile qualNameOfFile sigTy =
CheckedImplFile(qualNameOfFile, [], sigTy, ModuleOrNamespaceContents.TMDefs [], false, false, StampMap [], Map.empty)

let AddCheckResultsToTcState
(tcGlobals, amap, hadSig, prefixPathOpt, tcSink, tcImplEnv, qualNameOfFile, implFileSigType)
(tcState: TcState)
Expand Down Expand Up @@ -1208,28 +1204,68 @@ let AddCheckResultsToTcState

type PartialResult = TcEnv * TopAttribs * CheckedImplFile option * ModuleOrNamespaceType

/// Typecheck a single file (or interactive entry into F# Interactive)
let CheckOneInputAux
/// Returns partial type check result for skipped implementation files.
let SkippedImplFilePlaceholder (tcConfig: TcConfig, tcImports: TcImports, tcGlobals, tcState, input: ParsedInput) =
use _ =
Activity.start "ParseAndCheckInputs.SkippedImplFilePlaceholder" [| Activity.Tags.fileName, input.FileName |]

CheckSimulateException tcConfig

match input with
| ParsedInput.ImplFile file ->
let qualNameOfFile = file.QualifiedName

// Check if we've got an interface for this fragment
let rootSigOpt = tcState.tcsRootSigs.TryFind qualNameOfFile

// Check if we've already seen an implementation for this fragment
if Zset.contains qualNameOfFile tcState.tcsRootImpls then
errorR (Error(FSComp.SR.buildImplementationAlreadyGiven (qualNameOfFile.Text), input.Range))

let hadSig = rootSigOpt.IsSome

match rootSigOpt with
| Some rootSigTy ->
// Delay the typecheck the implementation file until the second phase of parallel processing.
// Adjust the TcState as if it has been checked, which makes the signature for the file available later
// in the compilation order.
let tcStateForImplFile = tcState
let amap = tcImports.GetImportMap()

let ccuSigForFile, tcState =
AddCheckResultsToTcState
(tcGlobals, amap, hadSig, None, TcResultsSink.NoSink, tcState.tcsTcImplEnv, qualNameOfFile, rootSigTy)
tcState

let emptyImplFile =
CheckedImplFile(qualNameOfFile, [], rootSigTy, ModuleOrNamespaceContents.TMDefs [], false, false, StampMap [], Map.empty)

let tcEnvAtEnd = tcStateForImplFile.TcEnvFromImpls
Some((tcEnvAtEnd, EmptyTopAttrs, Some emptyImplFile, ccuSigForFile), tcState)

| _ -> None
| _ -> None

/// Typecheck a single file (or interactive entry into F# Interactive).
let CheckOneInput
(
checkForErrors,
tcConfig: TcConfig,
tcImports: TcImports,
tcGlobals,
prefixPathOpt,
tcSink,
tcGlobals: TcGlobals,
prefixPathOpt: LongIdent option,
tcSink: TcResultsSink,
tcState: TcState,
inp: ParsedInput,
skipImplIfSigExists: bool
) =

input: ParsedInput
) : Cancellable<PartialResult * TcState> =
cancellable {
try
use _ =
Activity.start "ParseAndCheckInputs.CheckOneInput" [| Activity.Tags.fileName, inp.FileName |]
Activity.start "ParseAndCheckInputs.CheckOneInput" [| Activity.Tags.fileName, input.FileName |]

CheckSimulateException tcConfig

let m = inp.Range
let m = input.Range
let amap = tcImports.GetImportMap()

let conditionalDefines =
Expand All @@ -1238,7 +1274,7 @@ let CheckOneInputAux
else
Some tcConfig.conditionalDefines

match inp with
match input with
| ParsedInput.SigFile file ->
let qualNameOfFile = file.QualifiedName

Expand Down Expand Up @@ -1285,7 +1321,7 @@ let CheckOneInputAux
tcsCreatesGeneratedProvidedTypes = tcState.tcsCreatesGeneratedProvidedTypes || createsGeneratedProvidedTypes
}

return Choice1Of2(tcEnv, EmptyTopAttrs, None, ccuSigForFile), tcState
return (tcEnv, EmptyTopAttrs, None, ccuSigForFile), tcState

| ParsedInput.ImplFile file ->
let qualNameOfFile = file.QualifiedName
Expand All @@ -1299,92 +1335,47 @@ let CheckOneInputAux

let hadSig = rootSigOpt.IsSome

match rootSigOpt with
| Some rootSig when skipImplIfSigExists ->
// Delay the typecheck the implementation file until the second phase of parallel processing.
// Adjust the TcState as if it has been checked, which makes the signature for the file available later
// in the compilation order.
let tcStateForImplFile = tcState
let qualNameOfFile = file.QualifiedName
let priorErrors = checkForErrors ()

let ccuSigForFile, tcState =
AddCheckResultsToTcState
(tcGlobals, amap, hadSig, prefixPathOpt, tcSink, tcState.tcsTcImplEnv, qualNameOfFile, rootSig)
tcState
// Typecheck the implementation file
let! topAttrs, implFile, tcEnvAtEnd, createsGeneratedProvidedTypes =
CheckOneImplFile(
tcGlobals,
amap,
tcState.tcsCcu,
tcState.tcsImplicitOpenDeclarations,
checkForErrors,
conditionalDefines,
tcSink,
tcConfig.internalTestSpanStackReferring,
tcState.tcsTcImplEnv,
rootSigOpt,
file,
tcConfig.diagnosticsOptions
)

let partialResult =
(amap, conditionalDefines, rootSig, priorErrors, file, tcStateForImplFile, ccuSigForFile)
let tcState =
{ tcState with
tcsCreatesGeneratedProvidedTypes = tcState.tcsCreatesGeneratedProvidedTypes || createsGeneratedProvidedTypes
}

return Choice2Of2 partialResult, tcState
let ccuSigForFile, tcState =
AddCheckResultsToTcState
(tcGlobals, amap, hadSig, prefixPathOpt, tcSink, tcState.tcsTcImplEnv, qualNameOfFile, implFile.Signature)
tcState

| _ ->
// Typecheck the implementation file
let! topAttrs, implFile, tcEnvAtEnd, createsGeneratedProvidedTypes =
CheckOneImplFile(
tcGlobals,
amap,
tcState.tcsCcu,
tcState.tcsImplicitOpenDeclarations,
checkForErrors,
conditionalDefines,
tcSink,
tcConfig.internalTestSpanStackReferring,
tcState.tcsTcImplEnv,
rootSigOpt,
file,
tcConfig.diagnosticsOptions
)

let tcState =
{ tcState with
tcsCreatesGeneratedProvidedTypes = tcState.tcsCreatesGeneratedProvidedTypes || createsGeneratedProvidedTypes
}

let ccuSigForFile, tcState =
AddCheckResultsToTcState
(tcGlobals, amap, hadSig, prefixPathOpt, tcSink, tcState.tcsTcImplEnv, qualNameOfFile, implFile.Signature)
tcState

let result = (tcEnvAtEnd, topAttrs, Some implFile, ccuSigForFile)
return Choice1Of2 result, tcState
let result = (tcEnvAtEnd, topAttrs, Some implFile, ccuSigForFile)
return result, tcState

with e ->
errorRecovery e range0
return Choice1Of2(tcState.TcEnvFromSignatures, EmptyTopAttrs, None, tcState.tcsCcuSig), tcState
}

/// Typecheck a single file (or interactive entry into F# Interactive). If skipImplIfSigExists is set to true
/// then implementations with signature files give empty results.
let CheckOneInput
((checkForErrors,
tcConfig: TcConfig,
tcImports: TcImports,
tcGlobals,
prefixPathOpt,
tcSink,
tcState: TcState,
input: ParsedInput,
skipImplIfSigExists: bool): (unit -> bool) * TcConfig * TcImports * TcGlobals * LongIdent option * TcResultsSink * TcState * ParsedInput * bool)
: Cancellable<PartialResult * TcState> =
cancellable {
let! partialResult, tcState =
CheckOneInputAux(checkForErrors, tcConfig, tcImports, tcGlobals, prefixPathOpt, tcSink, tcState, input, skipImplIfSigExists)

match partialResult with
| Choice1Of2 result -> return result, tcState
| Choice2Of2 (_amap, _conditionalDefines, rootSig, _priorErrors, file, tcStateForImplFile, ccuSigForFile) ->
let emptyImplFile = CreateEmptyDummyImplFile file.QualifiedName rootSig
let tcEnvAtEnd = tcStateForImplFile.TcEnvFromImpls
return (tcEnvAtEnd, EmptyTopAttrs, Some emptyImplFile, ccuSigForFile), tcState
return (tcState.TcEnvFromSignatures, EmptyTopAttrs, None, tcState.tcsCcuSig), tcState
}

// Within a file, equip loggers to locally filter w.r.t. scope pragmas in each input
let DiagnosticsLoggerForInput (tcConfig: TcConfig, input: ParsedInput, oldLogger) =
GetDiagnosticsLoggerFilteringByScopedPragmas(false, input.ScopedPragmas, tcConfig.diagnosticsOptions, oldLogger)

/// Typecheck a single file (or interactive entry into F# Interactive)
let CheckOneInputEntry (ctok, checkForErrors, tcConfig: TcConfig, tcImports, tcGlobals, prefixPathOpt, skipImplIfSigExists) tcState input =
let CheckOneInputEntry (ctok, checkForErrors, tcConfig: TcConfig, tcImports, tcGlobals, prefixPathOpt) tcState input =
// Equip loggers to locally filter w.r.t. scope pragmas in each input
use _ =
UseTransformedDiagnosticsLogger(fun oldLogger -> DiagnosticsLoggerForInput(tcConfig, input, oldLogger))
Expand All @@ -1393,7 +1384,7 @@ let CheckOneInputEntry (ctok, checkForErrors, tcConfig: TcConfig, tcImports, tcG

RequireCompilationThread ctok

CheckOneInput(checkForErrors, tcConfig, tcImports, tcGlobals, prefixPathOpt, TcResultsSink.NoSink, tcState, input, skipImplIfSigExists)
CheckOneInput(checkForErrors, tcConfig, tcImports, tcGlobals, prefixPathOpt, TcResultsSink.NoSink, tcState, input)
|> Cancellable.runWithoutCancellation

/// Finish checking multiple files (or one interactive entry into F# Interactive)
Expand All @@ -1411,7 +1402,7 @@ let CheckMultipleInputsFinish (results, tcState: TcState) =

let CheckOneInputAndFinish (checkForErrors, tcConfig: TcConfig, tcImports, tcGlobals, prefixPathOpt, tcSink, tcState, input) =
cancellable {
let! result, tcState = CheckOneInput(checkForErrors, tcConfig, tcImports, tcGlobals, prefixPathOpt, tcSink, tcState, input, false)
let! result, tcState = CheckOneInput(checkForErrors, tcConfig, tcImports, tcGlobals, prefixPathOpt, tcSink, tcState, input)
let finishedResult = CheckMultipleInputsFinish([ result ], tcState)
return finishedResult
}
Expand All @@ -1431,7 +1422,7 @@ let CheckClosedInputSetFinish (declaredImpls: CheckedImplFile list, tcState) =

let CheckMultipleInputsSequential (ctok, checkForErrors, tcConfig, tcImports, tcGlobals, prefixPathOpt, tcState, inputs) =
(tcState, inputs)
||> List.mapFold (CheckOneInputEntry(ctok, checkForErrors, tcConfig, tcImports, tcGlobals, prefixPathOpt, false))
||> List.mapFold (CheckOneInputEntry(ctok, checkForErrors, tcConfig, tcImports, tcGlobals, prefixPathOpt))

open FSharp.Compiler.GraphChecking

Expand Down
8 changes: 6 additions & 2 deletions src/Compiler/Driver/ParseAndCheckInputs.fsi
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,11 @@ type TcState =
/// Get the initial type checking state for a set of inputs
val GetInitialTcState: range * string * TcConfig * TcGlobals * TcImports * TcEnv * OpenDeclaration list -> TcState

/// Returns partial type check result for skipped implementation files.
val SkippedImplFilePlaceholder:
tcConfig: TcConfig * tcImports: TcImports * tcGlobals: TcGlobals * tcState: TcState * input: ParsedInput ->
((TcEnv * TopAttribs * CheckedImplFile option * ModuleOrNamespaceType) * TcState) option

/// Check one input, returned as an Eventually computation
val CheckOneInput:
checkForErrors: (unit -> bool) *
Expand All @@ -143,8 +148,7 @@ val CheckOneInput:
prefixPathOpt: LongIdent option *
tcSink: NameResolution.TcResultsSink *
tcState: TcState *
input: ParsedInput *
skipImplIfSigExists: bool ->
input: ParsedInput ->
Cancellable<(TcEnv * TopAttribs * CheckedImplFile option * ModuleOrNamespaceType) * TcState>

/// Finish the checking of multiple inputs
Expand Down
Loading