tsc instance reuse in composite project causes out of memory crash

[leemhenson]

TypeScript Version: 3.4.5
Search Terms: Composite, project references, out of memory

Expected behavior:
Building a large composite project where each component builds successfully should not run out of memory.

Actual behavior:
It looks like the same tsc instance is being used to compile each phase, and the memory grows until it trips the node memory limit. Would it instead make sense to fork off a new tsc process for each phase, or force some sort of garbage collection?

[leemhenson]

@RyanCavanaugh what info do you want me to provide?

I assume you have a composite project that you test with, you should be able to see that memory usage grows non-stop as each component project is compiled. The project I'm working on at the moment has nearly 50 components projects which each depend on a single large shared project. When the codebase has no compiler errors it will OOM the node process after 5 or 6 projects. I then restart with tsc -b --verbose --incremental again and this time it will build the project it crashed on, but then crash again 5 or 6 projects later. Rinse and repeat until all projects are compiled successfully.

With such a large project it's important that I can run the watch mode with --incremental, but if the node process is going to OOM every handful of file saves because of this issue then I'm not going to have a good experience.

[ericanderson]

@RyanCavanaugh @DanielRosenwasser We are seeing the OOM but with tsc --build --watch. Also on a really big project. Not right away but after a few recompiles it dies. We can repro on 3.5 and 3.4 but not on 3.3.

[ericanderson]

Some more information:

• 1mm lines of typescript across 100 referenced projects.
• Deleting tsbuildinfo seems to prevent the problem. This ties with doing a full --build (sans watch), deleting an output directory, and running again causing the issue.
• Most devs see the failure after around 45 minutes of activity
• There are some reports of zero activity (as in pull in master, run tsc --build --watch, go get coffee, return to OOM, no local changes)
• We are on node 8.11.4
• Most of our devs are configured to 4gb of old-space and we are going to start trying with 6gb to see if that changes anything

If there are theories about certain code patterns or usage patterns or whatever, we would be more happy to run some bespoke builds / nightlies to provide more information. I might be able to get a source code release to ya'll with an NDA.

I'll send you my cell phone number over twitter @DanielRosenwasser, in case @sheetalkamat wants to contact us directly.