cloud/photo-request-errors Fix: reject photo request Promise immediately on error instead of 30s timeout by isaiahb · Pull Request #2070 · Mentra-Community/MentraOS

isaiahb · 2026-02-17T20:29:45Z

Spec: Photo Error REST Endpoint

Overview

What this doc covers: Adding a REST endpoint (POST /api/client/photo/response) so the phone can report photo errors (and successes) to the cloud over HTTP instead of WebSocket.

Why this doc exists: When a photo request fails on the phone (camera error, BLE timeout, upload failure), the error currently needs to travel back over the glasses WebSocket. But the whole point of error reporting is that things are going wrong — the WebSocket might be the thing that's flaky. REST is fire-and-forget reliable. The client team lead also prefers REST for this.

What you need to know first: The original WebSocket-based spec is in cloud/docs/photo-error-reporting.md. This spec supersedes the transport mechanism (WS → REST) but keeps the same payload shape and error codes.

Who should read this: Cloud engineers implementing the endpoint, client/mobile engineers calling it.

The Problem in 30 Seconds

App requests a photo → cloud tells glasses → glasses/phone try to capture/transfer/upload → something fails → phone swallows the error silently → SDK hangs for 30s then times out. We need the phone to POST the error back immediately over REST so the cloud can reject the pending photo promise and notify the requesting app.

Spec

Endpoint

POST /api/client/photo/response
Authorization: Bearer <token>
Content-Type: application/json

Middleware: clientAuth → requireUserSession (same as device-state, notifications, etc.)

Request Body — Error Case

{
  "requestId": "uuid-from-original-take_photo",
  "success": false,
  "errorCode": "PHONE_UPLOAD_FAILED",
  "errorMessage": "Upload returned HTTP 502"
}

Request Body — Success Case

{
  "requestId": "uuid-from-original-take_photo",
  "success": true,
  "photoUrl": "https://...",
  "savedToGallery": true
}

Error Codes

From existing PhotoErrorCode enum in sdk/src/types/messages/glasses-to-cloud.ts. No new codes needed.

Code	When
`CAMERA_INIT_FAILED`	Camera couldn't initialize
`CAMERA_CAPTURE_FAILED`	Capture failed (glasses reported error)
`CAMERA_BUSY`	Another capture in progress
`CAMERA_TIMEOUT`	Glasses didn't respond in time
`BLE_TRANSFER_FAILED`	BLE photo transfer failed
`BLE_TRANSFER_TIMEOUT`	BLE transfer timed out
`PHONE_UPLOAD_FAILED`	Phone got the photo but HTTP upload to webhook failed
`PHONE_TIMEOUT`	Phone-side timeout waiting for glasses
`COMPRESSION_FAILED`	Image compression failed
`UNKNOWN_ERROR`	Catch-all

Response

200 — processed:

{
  "success": true,
  "message": "Photo response processed",
  "timestamp": "2026-02-17T..."
}

400 — missing requestId:

{
  "success": false,
  "message": "requestId is required",
  "timestamp": "2026-02-17T..."
}

503 — no active session (from requireUserSession middleware, already handled)

Cloud-Side Processing

The handler calls userSession.photoManager.handlePhotoResponse(body) — the exact same method that currently handles WebSocket photo responses from glasses. PhotoManager already normalizes the flat {errorCode, errorMessage} format:

PhotoManager.handlePhotoResponse()
  → normalizes {errorCode, errorMessage} into PhotoResponse.error shape
  → looks up pendingPhotoRequests by requestId
  → if success: sends photo result to the requesting app
  → if error: sends photo error to the requesting app

Zero changes needed in PhotoManager. It already handles both formats.

Data Flow

Before (WebSocket):
  Phone → glasses WS → bun-websocket → glasses-message-handler → PhotoManager

After (REST):
  Phone → POST /api/client/photo/response → photo.api.ts → PhotoManager

Same destination, more reliable transport.

Changes Summary

File	Change
`src/api/hono/client/photo.api.ts`	New file — endpoint handler (~50 lines)
`src/api/hono/client/index.ts`	Add 1 export line
`src/hono-app.ts`	Add 1 route mount line
`src/api/index.ts`	Add 1 Express mount line (compat)

… timeout

github-actions · 2026-02-17T20:29:56Z

📋 PR Review Helper

📱 Mobile App Build

⏳ Waiting for build...

🕶️ ASG Client Build

⏳ Waiting for build...

🔀 Test Locally

gh pr checkout 2070

coderabbitai · 2026-02-17T20:30:01Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch cloud/photo-request-errors

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Add POST /api/client/photo/response so the phone can report photo capture errors (and successes) over HTTP instead of WebSocket. Why REST over WS: the whole point is error reporting when things go wrong — the WebSocket might be the broken thing. REST is independent and reliable. Follows the migration pattern (device state, notifications, calendar, location all moved from WS to REST). Changes: - New endpoint: POST /api/client/photo/response (photo.api.ts) - Wired into Hono router at /api/client/photo - Delegates to existing PhotoManager.handlePhotoResponse() — zero changes needed in PhotoManager, it already handles both flat {errorCode, errorMessage} and nested {error: {code, message}} shapes - Removed dead PHOTO_RESPONSE case from glasses-message-handler.ts (phone was never sending photo_response over WS — that's the bug) - Moved old WS spec from cloud/docs/ to cloud/issues/038-*/ - Added proper spec.md following issue doc format - Updated porter-debug.yml to deploy this branch See: cloud/issues/038-photo-error-rest-endpoint/spec.md

Discovered during 038 — two deprecated photo service files (photo-request.service.ts, photo-taken.service.ts) still have active importers in hardware routes, photos routes, and glasses-auth middleware. PhotoManager superseded them but they were never fully removed. Tracked as separate issue since it touches hardware routes, upload validation, and Express compat — different blast radius from 038.

aisraelov · 2026-02-18T01:50:46Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a2900e989b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-02-18T01:54:42Z

cloud/packages/cloud/src/services/session/handlers/glasses-message-handler.ts

+      // Photo — PHOTO_RESPONSE is handled via REST at POST /api/client/photo/response
+      // See: cloud/issues/038-photo-error-rest-endpoint/spec.md


Keep PHOTO_RESPONSE WebSocket handling during migration

Removing the PHOTO_RESPONSE case means WebSocket photo replies no longer go through PhotoManager.handlePhotoResponse; they now fall into the default relay path, which wraps them as DATA_STREAM messages for subscribed apps only. In the current repo state there is still no client posting to /api/client/photo/response (rg "api/client/photo/response" only matches this new route/spec), while mobile still emits type: "photo_response" over WebSocket (mobile/src/services/SocketComms.ts:194-203), so photo errors/successes can bypass request-ID routing and leave pending photo requests unresolved until timeout.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-02-18T01:54:43Z

cloud/packages/cloud/src/api/hono/client/photo.api.ts

+      `Photo response received via REST: ${body.success ? "success" : `error (${body.errorCode})`}`,
+    );
+
+    await userSession.photoManager.handlePhotoResponse(body);


Normalize REST photo success payload before forwarding

This handler forwards the REST body as-is to photoManager, but success payloads documented for this endpoint do not include type. PhotoManager.handlePhotoResponse only injects type: PHOTO_RESPONSE for the flat error shape (errorCode + errorMessage), so successful REST responses can be sent to apps without a type field and fail SDK photo-response type guards (cloud/packages/sdk/src/types/messages/cloud-to-app.ts:425-427), preventing consumers from handling the success callback correctly.

Useful? React with 👍 / 👎.

cloudflare-workers-and-pages · 2026-02-18T23:27:40Z

Deploying mentra-live-ota-site with Cloudflare Pages

Latest commit:	`5478385`
Status:	✅ Deploy successful!
Preview URL:	https://3cb1c2b4.mentra-live-ota-site.pages.dev
Branch Preview URL:	https://cloud-photo-request-errors.mentra-live-ota-site.pages.dev

View logs

aisraelov · 2026-02-18T23:27:44Z

@codex review

cloudflare-workers-and-pages · 2026-02-18T23:29:07Z

Deploying mentra-store-dev with Cloudflare Pages

Latest commit:	`5478385`
Status:	✅ Deploy successful!
Preview URL:	https://b1042e0d.augmentos-appstore-2.pages.dev
Branch Preview URL:	https://cloud-photo-request-errors.augmentos-appstore-2.pages.dev

View logs

cloudflare-workers-and-pages · 2026-02-18T23:30:36Z

Deploying dev-augmentos-console with Cloudflare Pages

Latest commit:	`5478385`
Status:	✅ Deploy successful!
Preview URL:	https://b6923cc1.dev-augmentos-console.pages.dev
Branch Preview URL:	https://cloud-photo-request-errors.dev-augmentos-console.pages.dev

View logs

cloudflare-workers-and-pages · 2026-02-18T23:31:54Z

Deploying prod-augmentos-account with Cloudflare Pages

Latest commit:	`5478385`
Status:	✅ Deploy successful!
Preview URL:	https://3d0d348f.augmentos-e84.pages.dev
Branch Preview URL:	https://cloud-photo-request-errors.augmentos-e84.pages.dev

View logs

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5478385d9e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-02-18T23:33:36Z

cloud/packages/cloud/src/api/hono/client/photo.api.ts

+      `Photo response received via REST: ${body.success ? "success" : `error (${body.errorCode})`}`,
+    );
+
+    await userSession.photoManager.handlePhotoResponse(body);


Normalize REST success payloads before PhotoManager dispatch

The new POST /api/client/photo/response path forwards the raw body directly to PhotoManager.handlePhotoResponse, but the documented REST success payload does not include a type field. PhotoManager only normalizes error-shaped payloads, so success responses are sent to apps without type, which AppSession.validateMessage treats as invalid rather than a photo response. When clients use this new success format, photo requests can still time out (and emit spurious SDK error events) instead of completing via the cloud response path.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-02-18T23:33:36Z

cloud/packages/cloud/src/services/session/handlers/glasses-message-handler.ts

+      // Photo — PHOTO_RESPONSE is handled via REST at POST /api/client/photo/response
+      // See: cloud/issues/038-photo-error-rest-endpoint/spec.md


Retain PHOTO_RESPONSE handling until Express route is added

This removes the websocket PHOTO_RESPONSE case entirely, but the commit only mounts the new photo response endpoint in Hono routes. In the still-supported Express server path (index-express.ts via registerApi), /api/client/photo/response is not mounted, so Express deployments lose both ingestion paths: websocket responses are no longer consumed by PhotoManager, and there is no REST replacement, leaving requests pending until timeout in that environment.

Useful? React with 👍 / 👎.

- REST handler validates requestId, success, photoUrl (on success) - Builds typed PhotoResponse before passing to PhotoManager - Error payloads lenient (crash path), fall back to UNKNOWN_ERROR - PhotoManager.handlePhotoResponse now takes PhotoResponse, not any - No unvalidated data flows downstream to apps

aisraelov · 2026-02-19T00:06:42Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 257647bccf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-02-19T00:13:09Z

cloud/packages/cloud/src/services/session/handlers/glasses-message-handler.ts

+      // Photo — PHOTO_RESPONSE is handled via REST at POST /api/client/photo/response
+      // See: cloud/issues/038-photo-error-rest-endpoint/spec.md


Add explicit PHOTO_RESPONSE case before default relay

Removing the PHOTO_RESPONSE switch case makes those websocket messages fall through to the default branch, which calls userSession.relayMessageToApps(...) instead of PhotoManager. If any phone/glasses client still emits photo_response on WS (e.g., during rollout or retry/fallback paths), this bypasses the requestId→originating-app mapping and can fan out another app’s photo response to any app subscribed to photo_response/*, while the originating app’s pending photo request no longer gets completed through the intended path.

Useful? React with 👍 / 👎.

ws is depricated

chatgpt-codex-connector · 2026-02-19T00:13:09Z

cloud/packages/cloud/src/hono-app.ts

 app.route("/api/client/calendar", calendarApi);
 app.route("/api/client/location", locationApi);
 app.route("/api/client/notifications", notificationsApi);
+app.route("/api/client/photo", photoApi);


Mount /api/client/photo in Express fallback too

This commit only wires the new photo-response route into the Hono app, but the repo still supports an Express runtime (bun run dev:express via index-express.ts/registerApi). In that supported mode, POST /api/client/photo/response is not registered, so photo error callbacks 404 and requests still sit until timeout. Please add the equivalent Express mount (or explicitly retire that runtime) to avoid behavior differences across supported server entrypoints.

Useful? React with 👍 / 👎.

express is depricated

fix: reject photo request Promise immediately on error instead of 30s…

c9384ff

… timeout

isaiahb self-assigned this Feb 17, 2026

isaiahb requested a review from a team as a code owner February 17, 2026 20:29

isaiahb added 2 commits February 17, 2026 14:49

chatgpt-codex-connector bot reviewed Feb 18, 2026

View reviewed changes

Merge branch 'dev' into cloud/photo-request-errors

5478385

chatgpt-codex-connector bot reviewed Feb 18, 2026

View reviewed changes

chatgpt-codex-connector bot reviewed Feb 19, 2026

View reviewed changes

Merge branch 'dev' into cloud/photo-request-errors

526761d

aisraelov merged commit a81c8c8 into dev Feb 19, 2026
8 checks passed

		// Photo — PHOTO_RESPONSE is handled via REST at POST /api/client/photo/response
		// See: cloud/issues/038-photo-error-rest-endpoint/spec.md

Comments

Conversation

isaiahb commented Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Spec: Photo Error REST Endpoint

Overview

The Problem in 30 Seconds

Spec

Endpoint

Request Body — Error Case

Request Body — Success Case

Error Codes

Response

Cloud-Side Processing

Data Flow

Changes Summary

Uh oh!

github-actions bot commented Feb 17, 2026

📋 PR Review Helper

📱 Mobile App Build

🕶️ ASG Client Build

🔀 Test Locally

Uh oh!

coderabbitai bot commented Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

aisraelov commented Feb 18, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

cloudflare-workers-and-pages bot commented Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying mentra-live-ota-site with Cloudflare Pages

Uh oh!

aisraelov commented Feb 18, 2026

Uh oh!

cloudflare-workers-and-pages bot commented Feb 18, 2026

Deploying mentra-store-dev with Cloudflare Pages

Uh oh!

cloudflare-workers-and-pages bot commented Feb 18, 2026

Deploying dev-augmentos-console with Cloudflare Pages

Uh oh!

cloudflare-workers-and-pages bot commented Feb 18, 2026

Deploying prod-augmentos-account with Cloudflare Pages

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

aisraelov commented Feb 19, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

isaiahb Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

isaiahb Feb 19, 2026

isaiahb commented Feb 17, 2026 •

edited

Loading

coderabbitai bot commented Feb 17, 2026 •

edited

Loading

cloudflare-workers-and-pages bot commented Feb 18, 2026 •

edited

Loading