Measure size of string fields in UTF-8 bytes for maxMessageSize accounting by sacOO7 · Pull Request #2289 · ably/ably-js

sacOO7 · 2026-08-12T12:41:26Z

Problem

The client-side maxMessageSize publish gate sums per-field sizes and rejects an over-limit publish before it hits the wire. ably-js measured string fields (clientId, LiveObjects map/operation keys) by their UTF-16 code-unit count (.length), but the server accounts for these fields in UTF-8 bytes. For ASCII these coincide; for non-ASCII they diverge (你 = 3 UTF-8 bytes / 1 UTF-16 code unit; 😊 = 4 UTF-8 bytes / 2 UTF-16 code units), so js undercounted.

Ably's published accounting — How is maximum message size measured? — is explicit:

name and clientId: "calculated as the size in bytes of their UTF-8 representation"
data: "calculated as the size in bytes if it is in binary, or its UTF-8 byte length if it is a string"
extras: "calculated as the string length of its JSON representation"

Consequences of the undercount:

False-accepts. A boundary message whose UTF-16 count sat under the limit but whose UTF-8 byte count is over would pass the local gate and get asynchronously NACKed by the server (40009/40006) — or drop the connection — instead of being cleanly rejected locally.
Cross-SDK gate divergence. The same clientId/key was accepted by js but rejected by ably-java / ably-cocoa (which already counted UTF-8) — one publish, two verdicts.

Spec disambiguation (anchor PR): ably/specification#516. History / earlier attempt: ably/specification#331.

Solution

Measure string fields in UTF-8 bytes via the existing Utils.dataSizeBytes (TextEncoder / Buffer.byteLength), matching the server. extras is deliberately unchanged — the published accounting has an explicit distinct rule for it ("string length of its JSON representation", i.e. UTF-16 code units), and measuring extras in UTF-8 would over-count and false-reject documented-valid messages.

Sites changed:

src/plugins/liveobjects/objectmessage.ts — 5 sites ?.length → dataSizeBytes(...): clientId (OM3f), OMP4a1 (map-state key), MCR3a1, MST3c, MRM3a (operation keys). extras stays JSON.stringify(...).length; ObjectData (OD3b–g) already used dataSizeBytes.
src/common/lib/types/message.ts (getMessageSize) — name and clientId .length → Utils.dataSizeBytes(...) (UTF-8). extras stays JSON.stringify(...).length.

Tests

test/realtime/liveobjects.test.js — +6 non-ASCII ObjectMessage size scenarios (clientId = '你😊' etc.) locking the convention: OM3f/MCR3a1/MST3c/MRM3a/OMP4a1 asserted against Utils.dataSizeBytes(...) (UTF-8), OM3d (extras) asserted against JSON.stringify(...).length (UTF-16).

Verification

npm run build:node — clean.
npx tsc --noEmit -p tsconfig.json — 0 errors.
mocha test/realtime/liveobjects.test.js --grep "message size" — 37 passing (incl. the 6 new non-ASCII scenarios).

Notes

Follow-up: the name/clientId UTF-8 change in getMessageSize (regular Message) has no dedicated non-ASCII unit test yet — the new scenarios cover the LiveObjects ObjectMessage path only. Worth adding one alongside the regular-Message size tests.

Summary by CodeRabbit

Bug Fixes
- Corrected message size calculations to accurately account for UTF-8 byte lengths in client IDs, map keys, and object entries.
- Improved size reporting for messages containing non-ASCII characters.
Tests
- Added coverage for Unicode client IDs, map keys, object entries, and extra data to verify accurate size calculations.

The client-side maxMessageSize gate undercounted non-ASCII clientIds and LiveObjects map keys by measuring them in UTF-16 code units (.length), while the server accounts for them in UTF-8 bytes. This false-accepted boundary messages the server would reject, and diverged from ably-java/ably-cocoa (which already counted UTF-8). - src/plugins/liveobjects/objectmessage.ts: clientId (OM3f), OMP4a1, MCR3a1, MST3c, MRM3a now use this._utils.dataSizeBytes(...) (UTF-8). extras stays JSON.stringify(...).length (UTF-16); ObjectData (OD3b-g) already used dataSizeBytes. - src/common/lib/types/message.ts (getMessageSize): name and clientId now use Utils.dataSizeBytes(...) (UTF-8). extras stays JSON.stringify(...).length. - test/realtime/liveobjects.test.js: +6 non-ASCII ObjectMessage size scenarios (OM3f/MCR3a1/MST3c/MRM3a/OMP4a1 UTF-8; OM3d UTF-16). Aligns with ably/specification (UTF-8 byte length default, extras as UTF-16 string length of its JSON representation).

coderabbitai · 2026-08-12T12:44:50Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b3adac3c-9974-4971-a5be-54bd9e0d6130

📥 Commits

Reviewing files that changed from the base of the PR and between c3d0605 and 1ee7100.

📒 Files selected for processing (3)

src/common/lib/types/message.ts
src/plugins/liveobjects/objectmessage.ts
test/realtime/liveobjects.test.js

Walkthrough

The change updates message and LiveObjects size calculations to use UTF-8 byte lengths for names, client IDs, and map keys. Realtime tests add non-ASCII scenarios and preserve UTF-16 JSON length handling for extras.

Changes

Message size accounting

Layer / File(s)	Summary
UTF-8 byte-length calculations `src/common/lib/types/message.ts`, `src/plugins/liveobjects/objectmessage.ts`	Message names, client IDs, and map keys now use UTF-8 byte lengths instead of JavaScript string lengths.
Non-ASCII size validation `test/realtime/liveobjects.test.js`	Tests cover non-ASCII client IDs, map keys, object-entry keys, and extras. Extras continues to use the UTF-16 length of its JSON representation.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: lawrence-forooghian

Poem

A rabbit counts bytes in the moonlit night,
UTF-8 keys hop into size just right.
Client IDs twinkle, maps align,
Extras keep their JSON line.
“Nibble-tested!” the rabbit sings,
As accurate totals grow little wings.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change to UTF-8 byte accounting for message-size calculations.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch size-accounting-utf8-byte-length

Warning

Tools execution failed with the following error:

Failed to run tools: 14 UNAVAILABLE: read ECONNRESET

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

Copilot

Pull request overview

Aligns client-side maxMessageSize publish-gate accounting with Ably’s documented/server behavior by measuring relevant string fields using their UTF-8 byte length (instead of JS UTF-16 code unit length), preventing false-accepts for non-ASCII content and reducing cross-SDK divergence.

Changes:

Update regular Message size calculation to measure name and clientId using Utils.dataSizeBytes (UTF-8 bytes).
Update LiveObjects WireObjectMessage size calculation to measure clientId and map/operation keys using dataSizeBytes (UTF-8 bytes), keeping extras measured via JSON.stringify(...).length.
Add LiveObjects tests covering non-ASCII scenarios to lock in the intended accounting rules.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
`src/common/lib/types/message.ts`	Switch `name`/`clientId` sizing to UTF-8 byte length for regular `Message` maxMessageSize accounting.
`src/plugins/liveobjects/objectmessage.ts`	Switch LiveObjects `clientId` and key sizing to UTF-8 byte length for object message maxMessageSize accounting.
`test/realtime/liveobjects.test.js`	Add non-ASCII ObjectMessage size scenarios validating UTF-8 sizing for strings/keys and UTF-16 JSON string length for `extras`.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

  let size = 0;
  if (msg.name) {
-    size += msg.name.length;
+    size += Utils.dataSizeBytes(msg.name); // UTF-8 byte length


Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/common/lib/types/message.ts:46

Add non-ASCII coverage for the regular Message path changed here. The new scenarios only exercise WireObjectMessage.getMessageSize(), while the existing regular-message maxMessageSize tests use ASCII payloads, so neither the name nor clientId UTF-8 accounting is verified at the publish boundary. A boundary test with 你😊 should assert the local 40009 decision for these fields.

    size += Utils.dataSizeBytes(msg.name); // UTF-8 byte length
  }
  if (msg.clientId) {
    size += Utils.dataSizeBytes(msg.clientId); // UTF-8 byte length

sacOO7 mentioned this pull request Aug 12, 2026

Define message-size string accounting as UTF-8 byte length (with explicit string-length exceptions) ably/specification#516

Open

github-actions Bot deployed to staging/pull/2289/features August 12, 2026 12:42 View deployment

github-actions Bot deployed to staging/pull/2289/bundle-report August 12, 2026 12:42 View deployment

github-actions Bot deployed to staging/pull/2289/typedoc August 12, 2026 12:42 View deployment

sacOO7 requested a lite review from Copilot August 12, 2026 12:45

Copilot started reviewing on behalf of sacOO7 August 12, 2026 12:48 View session

Copilot AI reviewed Aug 12, 2026

View reviewed changes

Comment thread src/common/lib/types/message.ts

let size = 0;

if (msg.name) {

size += msg.name.length;

size += Utils.dataSizeBytes(msg.name); // UTF-8 byte length

sacOO7 requested a balanced review from Copilot August 21, 2026 11:01

Copilot started reviewing on behalf of sacOO7 August 21, 2026 11:01 View session

Copilot AI reviewed Aug 21, 2026

View reviewed changes

Navigation Menu

Uh oh!

Measure size of string fields in UTF-8 bytes for maxMessageSize accounting - #2289

Measure size of string fields in UTF-8 bytes for maxMessageSize accounting#2289
sacOO7 wants to merge 1 commit into
mainfrom
size-accounting-utf8-byte-length

sacOO7 commented Aug 12, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Aug 12, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

sacOO7 commented Aug 12, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Solution

Tests

Verification

Notes

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

sacOO7 commented Aug 12, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Aug 12, 2026 •

edited

Loading