A Google Drive source that a person connects and that syncs the folders they choose #1223

What you are trying to do / 想做成什么事

We run Utopia for a consultancy, with one knowledge base per client. Most of what a client engagement produces lives in Google Drive: proposals, meeting notes, assessments, contracts and slide decks, both in personal drives and in shared drives. Right now we push meeting transcripts from Pocket through the api source. Drive documents have no good way in.

• Exporting by hand and uploading is a snapshot. It is out of date the next day, and nobody re-uploads.
• Copying into a bucket and syncing it with the GCS or S3 source loses each file's identity (a rename or move becomes a new document), and doc_time becomes the copy's time, not the document's own.
• Notion-style connectors take a key an administrator pastes in. Drive has no such key for a person's own files. A service account sees only what is explicitly shared with it, and domain-wide delegation lets one key read everyone's Drive, which is far more than anyone chose to include.

Drive passes the four tests of 0013:

• modifiedTime is the content's own clock.
• Documents are rewritten in place.
• A file id survives renames and moves.
• The knowledge actually lives there.

What would help / 希望怎样

A google_drive source kind that a person connects with their own Google account and that syncs only the folders they choose:

1. An editor creates the source and clicks Connect Google Drive. A Google consent popup asks for read-only Drive access (drive.readonly, plus openid email so the settings can show which account is connected). The refresh token is stored as a sealed source secret, and the browser never holds a Google token.
2. A folder browser (My Drive, Shared with me, and shared drives) lets them tick folders. Subfolders are included, and nothing outside the chosen folders is read.
3. Sync works like this:
   • Docs, Sheets and Slides are exported to DOCX, XLSX and PPTX, so the existing parsers keep headings, tables and every sheet. Other files are downloaded as they are.
   • doc_time is modifiedTime, and the identity is the file id.
   • Files that disappear are marked "not in source", but only after a complete walk.
4. The OAuth client belongs to the deployment and is set by three environment variables. When they are not set, the kind is not offered.

One trap that shapes the design: a Google export is not byte-stable. Exporting an unchanged Doc twice gives two different zips. If changes were detected by content hash, as the other connectors do, every Doc would be re-extracted on every sync. The sync therefore keeps Drive's own change marker (md5Checksum, or modifiedTime for Google formats) and skips files whose marker hasn't changed.

The full reasoning, including what was rejected (a service account, the Google Picker, the changes feed), is written up as ADR 0068 in the PR.

What you do today instead / 目前的替代做法

We export to PDF by hand and upload. It goes stale, and the documents lose their identity and dates.

What we did

We built it and opened #1222 against dev:

• An ADR, a migration (0112), the connector, OAuth routes and the web panel, in English and Chinese.
• cargo fmt, clippy -D warnings and cargo test --workspace pass, with 12 new tests against mocked Drive and OAuth responses. pnpm build and pnpm test also pass.
• We also tried it by hand against a real Google Drive. We built an image from the branch and ran it on GKE in a throwaway namespace with its own Postgres 16 and pgvector. All migrations applied, and connecting an account, choosing folders and syncing worked.

The PR is large (about 2,100 lines, roughly half of it comments and tests). We kept it as one change because neither half is usable on its own, but we're happy to split it into the backend (ADR, migration, connector and routes) and the web panel if that's easier to review.

Open questions, also listed in the ADR:

• Drive revision history. Drive keeps old revisions. Which ones to sample, if any, is a separate decision.
• When the connected person leaves. Their grant stops working, and the sync fails with "reconnect". An alert for that would make it visible.
• Google review. drive.readonly is a restricted scope. An Internal OAuth client in a Workspace organisation needs no Google review; an External one does.