Skip to content

feat: add Uint8Array support for zero-copy worker_threads transfer#3183

Open
umeshmore45 wants to merge 1 commit intoredis:masterfrom
umeshmore45:feature/uint8array-support
Open

feat: add Uint8Array support for zero-copy worker_threads transfer#3183
umeshmore45 wants to merge 1 commit intoredis:masterfrom
umeshmore45:feature/uint8array-support

Conversation

@umeshmore45
Copy link

Description

This PR adds comprehensive Uint8Array support to enable zero-copy data transfer to worker_threads, significantly improving performance for multi-threaded applications handling large payloads.

Motivation

Currently, Redis values are returned as Buffer, which requires copying when transferring to worker threads. With Uint8Array support, the underlying ArrayBuffer can be transferred with zero-copy semantics.

Performance Benefits:

  • Zero memory copying when transferring to workers
  • Reduced memory usage for large payloads
  • Better performance in multi-threaded applications

For a 10MB payload, this eliminates 10MB of memory copying when transferring between threads.

Usage Example

import { createClient, RESP_TYPES } from 'redis';
import { Worker } from 'worker_threads';

const client = await createClient().connect();

// Get data as Uint8Array
const data = await client
  .withTypeMapping({
    [RESP_TYPES.BLOB_STRING]: Uint8Array
  })
  .get('large-file');

// Zero-copy transfer to worker
const worker = new Worker('./worker.js');
worker.postMessage(data, [data.buffer]);

// After transfer, data.byteLength === 0 (ownership transferred)

Changes Made

Type System (types.ts):

  • Extended BlobStringReply, SimpleStringReply, VerbatimStringReply to support Uint8Array
  • Updated UnwrapConstructor and ReplyWithTypeMapping to handle Uint8Array
  • Added Uint8Array to RedisArgument type

Decoder (decoder.ts):

  • Added Uint8Array conversion in 4 string decoding methods (single/multi-chunk)
  • Handles both simple strings and blob strings correctly
  • Maintains backward compatibility with Buffer

Encoder (encoder.ts):

  • Added Uint8Array handling in command encoding
  • Properly ordered Buffer check before Uint8Array (Buffer extends Uint8Array)
  • Updated error messages to include Uint8Array

Command Support:

  • Updated type guards in HSET, GEOSEARCH, SEARCH, AGGREGATE to accept Uint8Array arguments
  • Fixed Buffer/Uint8Array handling in cluster-slots.ts and time-series/helpers.ts

Testing:

  • Added 4 decoder tests (simple/blob/verbatim strings)
  • Added 1 encoder test
  • Added 1 integration test for type mapping
  • All existing tests continue to pass

Breaking Changes

None! This is fully backward compatible:

  • Buffer remains the default type
  • Existing code continues to work unchanged
  • Uint8Array is opt-in via withTypeMapping()

Files Changed

  • 12 files modified
  • 95 lines added, 19 lines removed
  • Net addition: 76 lines

Closes #3106


Checklist

  • Does npm test pass with this change (including linting)?
  • Is the new or changed code fully tested?
  • Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?

Note: Documentation can be added in a follow-up PR if needed. The code includes inline comments and the PR description provides usage examples.

- Add Uint8Array to type system (BlobStringReply, SimpleStringReply, VerbatimStringReply)
- Implement Uint8Array encoding/decoding in RESP protocol handlers
- Support Uint8Array as command arguments across all commands
- Add comprehensive test coverage for Uint8Array operations
- Fix type guards in HSET, GEOSEARCH, AGGREGATE, SEARCH commands
- Fix Buffer/Uint8Array handling in time-series helpers and cluster-slots
- Maintain full backward compatibility with Buffer

Enables zero-copy transfer to worker_threads via ArrayBuffer transfer.

Closes redis#3106
@jit-ci
Copy link

jit-ci bot commented Feb 21, 2026

Hi, I’m Jit, a friendly security platform designed to help developers build secure applications from day zero with an MVS (Minimal viable security) mindset.

In case there are security findings, they will be communicated to you as a comment inside the PR.

Hope you’ll enjoy using Jit.

Questions? Comments? Want to learn more? Get in touch with us.

@nkaradzhov
Copy link
Collaborator

Thanks @umeshmore45, i will take a look

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support Uint8Array values for zero-copy transfer to worker_threads

2 participants