feat(email): REST Gmail/Graph mail, OAuth SMTP retry, and migration API updates

[amiteshwarp]

Summary

This change adds an optional REST mail mode for Google and Microsoft email channels: inbound mail can be fetched via the Gmail API and Microsoft Graph instead of IMAP, and outbound mail can use matching REST delivery when that mode is active. It also hardens OAuth SMTP sending when access tokens are stale, fixes OAuth token refresh persistence on JSONB provider_config, and ensures scheduled inbound fetch runs for channels that rely on REST (or OAuth mail APIs) without IMAP being enabled.

Together, these updates support migrated or narrow-scope OAuth tokens where full IMAP/SMTP-style scopes are not available, while preserving existing behavior for channels that still use IMAP inbound + SMTP outbound.

Background and intent

Narrow OAuth scopes: Some integrations only have REST-oriented scopes (e.g. Gmail modify / Graph mail read-write-send) rather than broad legacy mail scopes. For those channels, the application must use provider REST APIs for read and send instead of assuming IMAP/SMTP.

Compatibility: Channels created through the standard in-app Google/Microsoft flows that already use IMAP + SMTP should behave as before when REST mode is not enabled.

Problems addressed

1. Stale token SMTP failures — Outbound mail using XOAUTH2 over SMTP could fail with authentication errors when provider_config['access_token'] was expired, eventually pushing the channel toward reauthorization/disconnect.
2. Refresh + hash key mismatch — After refreshing tokens, code could read the new access token with symbol keys from a string-keyed JSONB hash, effectively returning nil in some paths.
3. Concurrent writers — The mailer path and inbound fetch path could both refresh and overwrite provider_config, causing races and dropped metadata.
4. REST-only channels never polled — Scheduled jobs gated inbound work on imap_enabled, so channels that only fetch via REST (or OAuth mail without IMAP) were skipped.

What changed (by area)

Channel::Email (app/models/channel/email.rb)

• rest_api_mode? — provider_config['api_mode'] == 'rest'.
• gmail_api_inbound? / microsoft_graph_inbound? — Select REST inbound fetchers for Google/Microsoft when REST is set or when OAuth mail APIs apply without IMAP.
• inbound_fetch_enabled? — IMAP enabled OR REST mode OR OAuth API inbound without IMAP, so the scheduler and fetch job actually run for REST-only setups.

Inbound jobs

• Inboxes::FetchImapEmailInboxesJob — Uses inbound_fetch_enabled? instead of only imap_enabled.
• Inboxes::FetchImapEmailsJob — Dispatches to Gmail::FetchEmailService / MicrosoftGraph::FetchEmailService when the channel is in REST inbound mode; otherwise keeps the existing IMAP service chain.

REST inbound implementations apply provider-specific idempotency (e.g. label/category) so messages are not fetched repeatedly.

Outbound mail

• ConversationReplyMailerHelper — For REST mode, sets delivery_method to :gmail_api or :microsoft_graph with { channel: } options. For classic OAuth SMTP, uses oauth_access_token via the refresh services as a read-only consumer of cached tokens (no destructive merge from the mailer).
• config/initializers/api_email_delivery.rb — Registers custom Action Mailer delivery methods.
• lib/mail/gmail_api_delivery.rb and lib/mail/microsoft_graph_delivery.rb — Send via provider REST APIs.

OAuth refresh and SMTP retry

• BaseRefreshOauthTokenService — update_channel_provider_config merges new tokens into existing provider_config (string keys) so flags like api_mode survive refresh; clears cached indifferent access after refresh so access_token returns the updated value.
• Email::OauthSmtpRetrier — On Net::SMTPAuthenticationError for Google/Microsoft email channels: refresh tokens, retry the send block once, clear stale reauthorization UI state when a send proves credentials work, and use authorization_error! on persistent failure.
• Email::SendOnEmailService — Wraps deliver_now in Email::OauthSmtpRetrier.with_retry.

Note for this codebase: Some upstream trees enqueue outgoing mail through a dedicated Sidekiq worker class; this branch delivers the relevant path through Email::SendOnEmailService, so the retrier is wired there (not duplicated in a separate worker that does not exist on this line).

Platform API — email channel migrations

• Platform::Api::V1::EmailChannelMigrationsController — On create/update for Google/Microsoft, ensures api_mode: 'rest' is stamped into provider_config where applicable. Adds update support for migrated inboxes/channels (including optional reauthorized! when credentials are rotated).
• config/routes.rb — Exposes update on email_channel_migrations with param: :inbox_id.

Tests and factories

Request specs for the migration API (create + update), job specs, model specs for fetch eligibility, Gmail/Graph fetch and mail delivery specs, refresh-token service coverage, and factory traits for REST-shaped channels.

Files touched (reference)

Area	Paths
Model	app/models/channel/email.rb
Jobs	app/jobs/inboxes/fetch_imap_emails_job.rb, app/jobs/inboxes/fetch_imap_email_inboxes_job.rb
Mail	app/mailers/conversation_reply_mailer_helper.rb, lib/mail/gmail_api_delivery.rb, lib/mail/microsoft_graph_delivery.rb, config/initializers/api_email_delivery.rb
Services	app/services/gmail/fetch_email_service.rb, app/services/microsoft_graph/fetch_email_service.rb, app/services/email/oauth_smtp_retrier.rb, app/services/email/send_on_email_service.rb, app/services/base_refresh_oauth_token_service.rb
Platform API	app/controllers/platform/api/v1/email_channel_migrations_controller.rb, config/routes.rb
Specs / factories	Under spec/ and spec/factories/channel/channel_email.rb

Deploy / infra: If your environment previously relied on separate Helm or env-only changes for OAuth client configuration, confirm those values in your deployment pipeline; this application PR does not add Helm value files on this branch.

How to test

1. Migration API — With EMAIL_CHANNEL_MIGRATION enabled, create channels via the platform migrations endpoint; confirm provider_config['api_mode'] is rest for Google/Microsoft where designed.
2. Inbound REST — Run or wait for the scheduled fetch job; confirm new mail becomes conversations and is not repeatedly re-imported.
3. Outbound REST — Send a reply from the UI; confirm it appears in the provider’s sent folder/mailbox.
4. Inbound without IMAP — With imap_enabled false but REST (or OAuth mail API) configuration present, confirm the inbox is still picked up by FetchImapEmailInboxesJob and processed.
5. OAuth SMTP path — With classic IMAP+OAuth SMTP (no REST), send mail; optionally force an expired token and confirm refresh + single retry behavior and stable provider_config (including api_mode when set).
6. PATCH migration — Update a migrated inbox/channel via PATCH .../email_channel_migrations/:inbox_id and verify attributes and reauthorization state when rotating provider_config.

Do not paste real tokens or client secrets into tickets or pull requests.

Base branch note for reviewers

This work sits on a branch that includes platform email channel migrations and related baseline not present on every long-lived develop tip. If the diff against develop is noisy, switch the PR base to the integration or release branch that already contains that baseline so the review focuses on the mail/OAuth/migration changes above.

[scmmishra]

I couldn't find the changes to auth flow to add the new scopes. Unless you wish to migrate users to the mail.google.com scope when re-authenticating it would be worth changing that

[scmmishra]

This label will be applied to the actual inbox of the customers as "CHATWOOT FETCHED". Is that acceptable? This will effectively expose chatwoot name to them

[scmmishra]

It's best to abstract out a GMail Client, that can be used elsewhere

[scmmishra]

There's an issue here, add_label runs before Imap::ImapMailbox creates the Chatwoot message. If processing fails afterward, Gmail excludes the email from future syncs via -label:CHATWOOT_FETCHED, so the email can be silently lost.

This can be considered an acceptable risk, but worth documenting it

[scmmishra]

In general this shares a lot of it's boiler plate with Microsoft fetching service, would be worth abstracting it for long term maintainability

[scmmishra]

mark_fetched runs before the mailbox processing succeeds. A later processing failure leaves the Outlook message categorized as fetched, so future syncs skip it.

[scmmishra]

The new model/job path explicitly supports REST API inbound with imap_enabled false, but this delivery branch is still gated by @channel.imap_enabled. In that configuration email_reply also fails its email_imap_enabled? && email_oauth_enabled? guard, so outbound replies return nil or fall back to global SMTP instead of using Gmail/Graph delivery.