Adding FirebaseFirestore pod dependency adds minutes to build time

[jmagman]

• Xcode version: 11.2
• Firebase Component: FirebaseFirestore
• Component version: 1.7.0

Dependency on FirebaseFirestore adds minutes to build time.

Steps to reproduce:

Create new Xcode project (I called it TestFirebase)
New Podfile:

target 'TestFirebase' do
  pod 'FirebaseFirestore', '1.7.0'
end

$ pod install
$ /usr/bin/xcodebuild -workspace TestFirebase.xcworkspace -scheme TestFirebase

Build TestFirebase_2019-11-11T14-17-13.txt

I ran with a build timing summary on my 2018 15" MacBook Pro:

Build Timing Summary

CompileC (1114 tasks) | 1642.840 seconds
...

(I'm guessing that 1642.840 seconds is divided over my 6 cores).

Impact

Investigating this on behalf of Flutter's cloud_firestore plugin iOS users (though I can reproduce the slow build times without Flutter or cloud_firestore). See context at flutter/flutter#37582 (55 👍 ) and firebase/flutterfire#349 (46 👍 ).

[google-oss-bot]

I found a few problems with this issue:

• I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.
• This issue does not seem to follow the issue template. Make sure you provide all the required information.

[jmagman]

\cc @collinjackson

[wilhuff]

This is expected:

• Firestore itself is 60k LOC
• BoringSSL is 194k LOC
• gRPC (core and C++) is 210k LOC
• LevelDB is 16k LOC
• Abseil in 40k LOC

Half a million lines of mostly C++ isn't going to compile quickly on a single host. ~400k of this comes from gRPC and its dependencies, so even if you reduced code actually in Firestore to zero you'd still have a huge build.

At least when building for iOS directly (using Xcode) you generally pay this cost once, not on every build.

[paulb777]

Some options to investigate:

• cocoapods-packager
• cocoapods-binary
• cocoapods-rome

Also, we are considering adapting the Swift-built Firebase zip builder to a general purpose Podfile->zip distribution tool.

[morganchen12]

I was able to reproduce the results here, where adding Firestore increased the build time significantly. I'll see if I can find out what's causing that and if there's an easy fix there.

In the meantime, I also proposed ccache as a way to keep a cache separate of DerivedData and CocoaPods cache that wouldn't be as easily invalidated but also was responsive in reflecting new code changes.

[morganchen12]

I was able to build Firestore with the -ftime-trace and wasn't able to identify any significant compile time bottlenecks that would be easy to fix. The traces are attached below and can be opened in chrome at chrome://tracing. See this blog post for more details.

I think we should focus on ways to better cache the Firestore build so flutter users don't have to do full rebuilds as frequently.

firestore_traces.zip

[morganchen12]

I'll see if I can get a flutter project to integrate with Firestore via cocoapods-rome next.

[mitchross]

Same issue here

[morganchen12]

Note that this issue would exist for any sufficiently large dependency, not just Firestore. It's only been associated with Firestore because Firestore is, at the moment, the most popular large flutter package.

Rome is a not-simple solution because it requires adding new behavior to flutter and opens up a lot of possibilities for new framework linking build failures. Thus far the Firebase team (myself included) has not found the time to propose this change to Flutter, and if we did, there would likely be a good amount of back and forth on the implementation details.

This is a non-issue for native iOS developers because the build cache is not frequently cleaned, so rebuilds from source are relatively rare.

Since significantly reducing the build-from-scratch time is not feasible, here are some other options:

• Submit an API proposal to Flutter add an option to prebuild dependencies, maybe during flutter pub get.
• Or, find a way to clean the Xcode build cache less frequently.