Super-linear compilation times if source file contains thousands of top-level LOC

[VAndrJ]

Description

Since the number of lines in a file increases, the compilation time per line of code also increases, I suggest reducing the number of lines of code in the files and breaking them into smaller ones.

Here is an example of a build with a timing summary (M1 Pro):

it takes too much time to build some parts.

This is the result of one benchmark, showing the dependence of compilation time per line of code depending on the number of lines in the Swift file:

[ahoppen]

Tracked in Apple’s issue tracker as rdar://125489574

[ahoppen]

How did you generate the graph that you attached? Compilation time should scale linearly with the number of lines in the source file in general.

[VAndrJ]

@ahoppen , these are the results of a slightly modified benchmark from this topic:

https://forums.swift.org/t/regarding-swift-type-inference-compile-time-performance

The absolute results will of course be different for regular code, but the dynamics are the same, after a certain point, when adding new lines, the compile time per line grows exponentially.
Here is the code of the benchmark itself, the results of which are on the graph:
https://gist.github.com/VAndrJ/c0bdbaebb803fe8db4fb0c7c05ce01ee

[ahoppen]

The primary performance issue with your test case appears to be that you’re putting all source lines as top-level code. When adding them as statements (the parameter to range in your test) to a function, the compilation time scales linearly. As swift-syntax doesn’t contain any top-level code your measurements thus don’t apply to this repo.

Let’s move it to the compiler repo to track the compilation performance of source files that contain thousands of top-level lines of code.

Statements|a.swift                  | b.swift
----------+-------------------------+-----------------------
10'000    | 5.006s (0.5006ms / LOC) | 1.976s (0.1976ms / LOC)
 5'000    | 2.537s (0.5074ms / LOC) | 1.040s (0.208ms / LOC)
 2'000    | 1.076s (0.5375ms / LOC) | 0.489s (0.244ms / LOC)

Updated performance test script

#!/usr/bin/env python3
import os

filenames = ["a", "b"]
codeMixed = [
  'someFunc(data: SomeClass(data: SomeStruct(data: SomeStruct.Nested1.Nested2(point: CGPoint(x: {}, y: {})))))',
  'someFunc(data: .init(data: .init(data: .init(point: .init(x: {}, y: {})))))'
]

for (i, filename) in enumerate(filenames):
    with open(filename + ".swift", "w") as f:
        s = """import CoreGraphics
struct SomeStruct { struct Nested1 { struct Nested2 { let point: CGPoint } }; let data: Nested1.Nested2 }
class SomeClass { let data: SomeStruct; init(data: SomeStruct) { self.data = data } }
func someFunc(data: SomeClass) { _ = data.data.data.point }
func test() {
"""
        for j in range(2000):
            s += (codeMixed[i] + '\n').format(j, j)
        s += "}"
        f.write(s)
        os.system("hyperfine 'xcrun swiftc -typecheck {}'".format(filename + ".swift"))