Too many `await` statements in one method triggers exponential build time

[gwynne]

Describe the bug
Since Swift 5.6 and continuing through the present, when a function containing more than ~100-150 independent uses of await is compiled in Xcode, the runtime for the swift-frontend invocation which includes the appropriate file increases extremely rapidly for each additional use of await - for a method with ~200 awaits, the build time reached over 20 minutes.

The method exhibiting this behavior in my direct experience was a test method in an XCTestCase, belonging to a test target in an SPM package. I have not had a chance to verify whether the problem is specific to test methods, nor to narrow down a repro case. From the investigations I have had a chance to make, the problem appears to be during the coroutine splitting pass and seems related to a considerable amount of repeated copying of instances of an LLVM container type.

Moving the implementation of the method in question into three private methods invoked sequentially was sufficient to reduce the build time to a bit under 20 seconds. I have not yet been able to look into it further.

Steps To Reproduce
Steps to reproduce the behavior:

1. N/A

Expected behavior
Reasonable build time.

Environment (please fill out the following information)

• OS: macOS Monterey
• Xcode Version/Tag/Branch: Xcode 14.0 - 14.1 and some versions of 13.x.

[kavon]

Can you please provide details about the test method? I'm curious about what kinds of async calls or property accesses it is awaiting. And also the control-flow. By itself the presence of an await does not give me much to go on in terms of the underlying code generation, because the kind of code we generate around each of those awaited call-sites depends on the thing being called. For example, awaiting an async-imported ObjC method has requires different code generation than for a native async Swift function.

I tried to reproduce this with 700 awaited call-sites to an actor-isolated method appearing in one function and compilation time is in the tens of seconds on an M1 Mac. That's is still not great, but an order of magnitude different than what you're experiencing, so I don't think I've captured this issue yet.

[DougGregor]

Mirrored as rdar://102054547

[gwynne]

@kavon The context is a method of an XCTestCase having the following form:

class FooTests: XCTestCase {
    override func setUp() async throws {
        // boilerplate for setting up testing of a Vapor server

        self.testStub1 = try await createATestStub(on: self.application.db) // opens connection to MySQL database specified in test scheme environment - in my case it was running via Homebrew
        // another dozen-odd stubs get built here
    }
    override func tearDown() async throws {
        // boilerplate for teardown of test Vapor instance
    }

    func testRouteRequiringExtremelyComplexDataPreparation() async throws {
        // create 20 variants of a model
        // use the IDs of those variants to build most of the possible permutations of each of those variants with some of the earlier stubs
        // basically just repeat `try await doADatabaseOperationAndReturnResult(almost always params depend on previous results)` a _lot_
        
        // 220-some-odd awaits worth of test data later...
        let response = try await self.application.test(calling the route this is all being done to test)
        // About 120 XCTAssertEqual()s happen here, validating the response's details; no additional `await` statements appear
    }
}

As far as I'm aware, there are no actors in use at all, either in the app being tested or in any dependencies. The async database operations are all thin shims over the old ELF-based calls with a .get() suffix, and the utility methods used to create the test data (which use the database ops as well) are imported from a test utility module in the same project.

[gwynne]

Note: This issue continues to exist on Monterey 12.6.2 with Xcode 14.2 and Swift 5.7.2.

[gwynne]

I should belatedly add that I also discovered a few weeks ago that the usage of async let in any given file often has the same effect of significantly slowing down compilation - but while it takes well over 100 uses of await to cause real issues, just three or four async lets in a single file, even in different scopes, can slow individual compilation commands down by multiples of seconds. The issue is definitely not one of memory exhaustion, CPU time starvation, or I/O bandwidth contention in the compiler, at least not according to any of the stats I was looking at.

Note: This has all still been with Xcode 14.2 - I haven't had a chance to check if the problems occur in Linux builds, although I can say with at least some confidence that it's not as severe there. My hunch was that it might have something to do with Xcode's tendency to invoke the driver with batches of ten or so files per compile command.