Skip to content

EFF-808 Fix Channel.decodeText UTF-8 chunk boundaries - #2209

Merged
tim-smart merged 2 commits into
mainfrom
lalph/EFF-808
May 18, 2026
Merged

EFF-808 Fix Channel.decodeText UTF-8 chunk boundaries#2209
tim-smart merged 2 commits into
mainfrom
lalph/EFF-808

Conversation

@tim-smart

Copy link
Copy Markdown
Collaborator

Summary

  • Fix Channel.decodeText to use streaming TextDecoder state across Uint8Array and upstream chunk boundaries.
  • Flush the decoder when upstream completes so pending bytes are emitted or finalized.
  • Add a regression test for a UTF-8 multi-byte character split across chunk boundaries.

Validation

  • pnpm lint-fix
  • pnpm test Channel.test.ts
  • pnpm check:tsgo
  • cd packages/effect && pnpm docgen

@changeset-bot

changeset-bot Bot commented May 18, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 746fea5

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 27 packages
Name Type
effect Patch
@effect/opentelemetry Patch
@effect/platform-browser Patch
@effect/platform-bun Patch
@effect/platform-node-shared Patch
@effect/platform-node Patch
@effect/vitest Patch
@effect/ai-anthropic Patch
@effect/ai-openai-compat Patch
@effect/ai-openai Patch
@effect/ai-openrouter Patch
@effect/atom-react Patch
@effect/atom-solid Patch
@effect/atom-vue Patch
@effect/sql-clickhouse Patch
@effect/sql-d1 Patch
@effect/sql-libsql Patch
@effect/sql-mssql Patch
@effect/sql-mysql2 Patch
@effect/sql-pg Patch
@effect/sql-pglite Patch
@effect/sql-sqlite-bun Patch
@effect/sql-sqlite-do Patch
@effect/sql-sqlite-node Patch
@effect/sql-sqlite-react-native Patch
@effect/sql-sqlite-wasm Patch
@effect/openapi-generator Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actions Bot commented May 18, 2026

Copy link
Copy Markdown
Contributor

📊 JSDoc Documentation Analysis

📈 Current Analysis Results
Analyzing 136 TypeScript files in packages/effect/src/ (including schema and config subdirectories)...

============================================================
         EFFECT JSDOC ANALYSIS REPORT
============================================================

📊 SUMMARY STATISTICS
------------------------------
Total files analyzed: 136
Total exported members: 4320
Missing @example: 4314 (99.9%)
Missing @category: 397 (9.2%)

🎯 TOP FILES NEEDING ATTENTION
----------------------------------------
1. Schema.ts
   📝 520 missing examples, 🏷️  203 missing categories
   📦 520 total exports
2. Effect.ts
   📝 253 missing examples, 🏷️  0 missing categories
   📦 253 total exports
3. Stream.ts
   📝 239 missing examples, 🏷️  0 missing categories
   📦 239 total exports
4. Channel.ts
   📝 152 missing examples, 🏷️  0 missing categories
   📦 152 total exports
5. Array.ts
   📝 140 missing examples, 🏷️  0 missing categories
   📦 140 total exports
6. DateTime.ts
   📝 108 missing examples, 🏷️  0 missing categories
   📦 108 total exports
7. SchemaRepresentation.ts
   📝 96 missing examples, 🏷️  12 missing categories
   📦 96 total exports
8. SchemaAST.ts
   📝 77 missing examples, 🏷️  21 missing categories
   📦 77 total exports
9. Chunk.ts
   📝 86 missing examples, 🏷️  0 missing categories
   📦 86 total exports
10. Sink.ts
   📝 81 missing examples, 🏷️  2 missing categories
   📦 81 total exports
11. Cause.ts
   📝 79 missing examples, 🏷️  2 missing categories
   📦 79 total exports
12. Graph.ts
   📝 80 missing examples, 🏷️  0 missing categories
   📦 80 total exports
13. SchemaTransformation.ts
   📝 43 missing examples, 🏷️  31 missing categories
   📦 43 total exports
14. Option.ts
   📝 67 missing examples, 🏷️  1 missing categories
   📦 67 total exports
15. String.ts
   📝 63 missing examples, 🏷️  1 missing categories
   📦 63 total exports

✅ PERFECTLY DOCUMENTED FILES
-----------------------------------
   index.ts (0 exports)

🔍 SAMPLE MISSING ITEMS FROM Schema.ts
-----------------------------------
   Optionality (type, line 150): missing example, category
   Mutability (type, line 159): missing example, category
   ConstructorDefault (type, line 169): missing example, category
   MakeOptions (interface, line 183): missing example, category
   Bottom (interface, line 211): missing example, category
   declareConstructor (interface, line 271): missing example
   declareConstructor (function, line 335): missing example
   declare (interface, line 362): missing example
   declare (function, line 395): missing example
   revealBottom (function, line 435): missing example, category

📋 BREAKDOWN BY EXPORT TYPE
-----------------------------------
const: 2982 missing examples, 114 missing categories
interface: 482 missing examples, 91 missing categories
type: 353 missing examples, 78 missing categories
function: 359 missing examples, 88 missing categories
namespace: 72 missing examples, 26 missing categories
class: 66 missing examples, 0 missing categories

📈 DOCUMENTATION PROGRESS
------------------------------
Examples: 6/4320 (0.1% complete)
Categories: 3923/4320 (90.8% complete)

============================================================
Analysis complete! 4711 items need attention.
============================================================

📄 Detailed results saved to: jsdoc-analysis-results.json

This comment is automatically updated on each push. View the analysis script for details.

@github-actions

Copy link
Copy Markdown
Contributor

Bundle Size Analysis

File Name Current Size Previous Size Difference
basic.ts 6.72 KB 6.72 KB 0.00 KB (0.00%)
batching.ts 9.22 KB 9.22 KB 0.00 KB (0.00%)
brand.ts 6.20 KB 6.20 KB 0.00 KB (0.00%)
cache.ts 9.70 KB 9.70 KB 0.00 KB (0.00%)
config.ts 18.40 KB 18.40 KB 0.00 KB (0.00%)
differ.ts 16.85 KB 16.85 KB 0.00 KB (0.00%)
http-client.ts 20.20 KB 20.20 KB 0.00 KB (0.00%)
logger.ts 10.18 KB 10.18 KB 0.00 KB (0.00%)
metric.ts 8.56 KB 8.56 KB 0.00 KB (0.00%)
optic.ts 7.37 KB 7.37 KB 0.00 KB (0.00%)
pubsub.ts 13.80 KB 13.80 KB 0.00 KB (0.00%)
queue.ts 11.08 KB 11.08 KB 0.00 KB (0.00%)
schedule.ts 10.08 KB 10.08 KB 0.00 KB (0.00%)
schema-representation-roundtrip.ts 26.17 KB 26.17 KB 0.00 KB (0.00%)
schema-string-transformation.ts 12.59 KB 12.59 KB 0.00 KB (0.00%)
schema-string.ts 10.34 KB 10.34 KB 0.00 KB (0.00%)
schema-template-literal.ts 13.94 KB 13.94 KB 0.00 KB (0.00%)
schema-toArbitraryLazy.ts 18.44 KB 18.44 KB 0.00 KB (0.00%)
schema-toCodeDocument.ts 21.12 KB 21.12 KB 0.00 KB (0.00%)
schema-toCodecJson.ts 17.66 KB 17.66 KB 0.00 KB (0.00%)
schema-toEquivalence.ts 17.61 KB 17.61 KB 0.00 KB (0.00%)
schema-toFormatter.ts 17.44 KB 17.44 KB 0.00 KB (0.00%)
schema-toJsonSchemaDocument.ts 19.96 KB 19.96 KB 0.00 KB (0.00%)
schema-toRepresentation.ts 18.00 KB 18.00 KB 0.00 KB (0.00%)
schema.ts 17.05 KB 17.05 KB 0.00 KB (0.00%)
stm.ts 11.95 KB 11.95 KB 0.00 KB (0.00%)
stream.ts 9.25 KB 9.25 KB 0.00 KB (0.00%)

@tim-smart tim-smart linked an issue May 18, 2026 that may be closed by this pull request
@tim-smart
tim-smart enabled auto-merge (squash) May 18, 2026 02:58
@tim-smart
tim-smart merged commit 6f38f07 into main May 18, 2026
15 checks passed
@tim-smart
tim-smart deleted the lalph/EFF-808 branch May 18, 2026 03:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Channel.decodeText corrupts UTF-8 across chunk boundaries

1 participant