Write the monorepo RFC

rfcs/20221124-monorepo.md

@@ -0,0 +1,116 @@
+↖️ Table of Contents
+
+# RFC: Monorepo
+
+**Status:** 🚧 WIP, comments are welcome nonetheless
+
+## Reviewers
+
+- [ ] <your name here>
+- [ ] <your name here>
+
+## Rationale
+
+For a comprehensive discussion about the pros, the cons and the counterpoints to each see [discussion](https://github.com/WordPress/openverse/issues/192). This is not the purpose of this RFC.
+
+This RFC summarily lists the benefits and then, with the twin assumptions of a monorepo being ultimately beneficial and the decision to migrate being finalised in the above discussion, proceeds to go into the implementation details.
+
+### Exclusive benefits of monorepo
+
+This only includes things that cannot be accomplished without the use of a monorepo.
+
+1. Single place to go for issues, PRs and all activity. Currently tickets are scattered across several repos, and any tickets that could benefit more than a single layer must in opened in each of the different repos.
+
+1. Singular copy (different from synced independent copies) of scaffolding code such as Git hooks, lint rules and common workflows. This is distinctly better than elaborate sync workflows.
+
+1. Central place for all technical documentation, enabling documentation for different parts of the stack to cross-reference other pieces and stay current with changes in other places.
+
+1. Enables the infra to deploy the code to coexist with the code itself. Apart from the private secrets that will still need to be encrypted, the IaC files could be organised identical to the code.
+
+1. Milestones that can span across multiple layers of the stack are only possible in GitHub. This is a limitation imposed by GitHub and there is no workaround for this.
+
+## Migration path
+
+First we will merge the API and the frontend. This decision was made for the following reasons.
+
+1. API and frontend are tightly linked. The frontend is a direct consumer of what the API produces.
+
+1. The API and frontend form the "service" side of Openverse that directly faces the users (both API consumers and Search engine users).
+
+1. The frontend uses ECS deployments and the API is well on the same track. This makes it possible for them to share some deployment code.
+
+1. To the RFC author, the API and frontend are very familiar so merging them would be easier. Adding a third component would make the task daunting.
+
+1. Merging incurs a productivity hit for the initial transition. So merging everything in one swoop is not ideal.
+
+1. The API’s comprehensive tooling for developer documentation can benefit frontend devs and create a unified docs site for contributors.
+
+1. The API is already organised by stack folders so the `frontend/` directory will fit right in with the others like `api/` and `ingestion_server/`.
+
+1. The API and frontend share identical tooling for Git hooks, linting and formatting. We will fight our tools less and encounter minimal friction.
+
+   - In fact, we employ a number of hacks to install and configure pre-commit for the frontend. Merging it with the API eliminates the need for such hacks.
+
+1. The entire system can be integration tested during releases. The real API, populated with test data, can even replace the Talkback server.
+
+The `WordPress/openverse-api` repo will absorb the `WordPress/openverse-frontend` repo. The `WordPress/openverse-catalog` will also be merged, _later_.
+
+### Reference
+
+I'm following the steps listed below in a fork at [@dhruvkb/monopenverse](https://github.com/dhruvkb/monopenverse/). You can refer to the fork, but note that it is a comes from a place of haste and has not been treated with the same level of love and care that the final treatment will receive.
+
+### Step 0. Get the timing right
+
+The first step will be to release the frontend, call a code freeze and pause work on it. This is to prevent the frontend repo from continuing to drift as we merge a snapshot of it with the API.
+
+This can prove difficult given how productive our team is, so we will need to channel this productivity towards the catalog in the meantime. I can foresee the end-to-end migration taking one fortnight.
+
+### Step 1: Merge the repos
+
+This is a quick process.
+
+1. Move the entire content of frontend inside a `frontend/` directory, except the following top-level files and folders. Please comment if you can add to this list.
+
+   - `.github/`
+   - `.editorconfig`
+   - `.eslintignore`
+   - `.eslintrc.js`
+   - `.gitignore`
+   - `.npmrc`
+   - `.pnpmfile.cjs`
+   - `.pre-commit-config.yaml`
+   - `.prettierignore`
+   - `justfile`
+
+1. Create the final commit on the `WordPress/frontend` repo. After the merge we might want to add a notice about the migration to the `README.md` file but GitHub's built-in archival process could suffice here.
+
+1. Merge this repo's `main` branch into the `WordPress/openverse-api` repo's `main` branch (see Git docs for `--allow-unrelated-histories`).
+
+1. Turn off the frontend workflows by renaming the directory.
+
+1. Create "stack: \*" labels to help with issue and PR management.
+
+1. Migrate issues, refer to @obulat's prior work in this department when we migrated from CC Search to Openverse. Apply the "stack: frontend" label to them.
+
+With this done, we can archive the frontend repo.
+
+### Step 2. Restore functionality
+
+1. Stopped frontend workflows will need some refactoring to start again.
+
+1. The API's workflows will need refactoring to only run the API is modified.
+
+1. CODEOWNERS will need to be updated to scope by directory.
+
+1. The action `banyan/auto-label` will need to be configured (`auto-label.json`) to add the "stack: \*" labels based on the modified directory.
+
+With this done, the development on the frontend can continue inside the subdirectory.
+
+### Step 3. Buff the rough edges
+
+There will be a few rough edges that I cannot foresee and we can continuously fix those as we spot them. But up to this point we should be in a position where
+we can continue to build the API and the frontend independently but from one repo.
+
+### Step 4. Integration
+
+I need ideas here.