Move (back) to a Monorepo #3612
Description
TL;DR: All the google-cloud-cpp-* GitHub repos will be combined into this repo as a single "monorepo" in Q2-2020. We apologize for any confusion this may cause our customers, but we think this will be much simpler for our customers going forward. This should not result in any build breaks for our customers, but in order for customers to pick up the newest monorepo versions of our libraries, they will need to make minimal changes to their build scripts.
Background
This GitHub repo is currently a "monorepo" in the sense that it contains the code for several client libraries. It contains the GCS client code, the Bigtable client code, and the beginnings of a Firestore client. However, our plan has been to change to a multi-repo approach where each client library lives in its own GitHub respository. This plan was outlined in #3350.
We started doing the multi-repo approach about a year ago with with our -spanner repo. At this point we now have the following GitHub repos:
- https://github.com/googleapis/google-cloud-cpp (this "mono" repo)
- https://github.com/googleapis/google-cloud-cpp-spanner
- https://github.com/googleapis/google-cloud-cpp-common
- https://github.com/googleapis/google-cloud-cpp-pubsub
- https://github.com/googleapis/google-cloud-cpp-bigquery
These separate repos are becoming exceedingly difficult to manage, there's lots of duplication across repos, and redundant work to keep things in sync, and the separate repos make it difficult for customers to explore all the C++ clients that we have available. This latter point will become an even bigger problem as the number of supported client libraries increase.
Given the new data we have from the experience of managing multiple repos for about a year, we have decided to reverse this decision and to instead move back to a monorepo.
The Plan
Our plan is to combine all of the "google-cloud-cpp-*" projects into a single GitHub repo, which will be this repo. The code, samples, and documentation for each client library will live in its own directory at google/cloud/$library, for example, google/cloud/bigtable, google/cloud/storage, and google/cloud/spanner.
Our plan is to begin and complete this work in Q2-2020 (roughly April - June, 2020). The work to do this is captured in the following milestones:
- Optimize CI Builds
- Prepare to be a monorepo
- Prepare builds for incoming repos
- Move google-cloud-cpp-pubsub
- Move google-cloud-cpp-bigquery
- Move google-cloud-cpp-spanner
- Move google-cloud-cpp-common
Expected User Impact
We apologize for any confusion or difficulty this change may cause our customers. We are committed to minimizing this friction. We also believe that the end result will be simpler, clearer, and better for our current and new customers going forward.
We know this change impacts existing users, but we believe users will be able to migrate to this new scheme with small amounts of effort. We anticipate that no changes to the C++ code will be needed. We expect that some changes to the build and/or packaging scripts will be needed.
Users Downloading from GitHub
We expect that a number of our customers download the source from GitHub and then incorporate the code into their build scripts. Existing releases of google-cloud-cpp and the other google-cloud-cpp-* repos will continue to work, and will have exactly the same content as before. Therefore, customers who have pinned their scripts to use a release will have no impact until such time as they decide to upgrade to the new version in the new "monorepo". Users who download master will break immediately, but we hope this is a small number, and that they are cognizant of the risks of depending on a non-release branch.
Changes for CMake Users depending on pre-installed libraries
No changes to their CMakeLists.txt files should be needed for these customers. Some changes to their build / installation scripts would be needed at the time they decide to upgrade.
Changes for CMake Users with Super Builds
These customers will need to change their top-level CMakeLists.txt to add new external projects.
Bazel Users
Bazel users will need to change their WORKSPACE file to introduce google-cloud-cpp as a new dependency.
Package Maintainers
Package maintainers who use google-cloud-cpp-spanner will need to create a new package for google-cloud-cpp, if one does not already exist.