Skip to content

[BUG] Array field with pipe characters incorrectly parsed when using HASH data structure #270

@hygorchristian

Description

@hygorchristian

Summary

When using a string[] field type with the HASH data structure, arrays containing strings with pipe (|) characters are incorrectly split during retrieval, causing data corruption.

Environment

  • redis-node version: "redis": "^5.8.2"
  • redis-om version: "redis-om": "^0.4.7"
  • Node.js version: v20.14.0
  • Redis version: 7.4.5
  • Data structure: HASH

Expected Behavior

Array fields should preserve their original structure and content when stored and retrieved, regardless of the characters contained within the strings.

Actual Behavior

Strings containing pipe (|) characters within array fields are incorrectly split at the pipe character during retrieval, corrupting the data structure.

Steps to Reproduce

  1. Create a schema with a string[] field
  2. Store data containing a string with a pipe character (e.g., "auth0|vVFy3-1234567")
  3. Use HASH as the data structure
  4. Retrieve the data

Minimal Code Example

import { Entity, Schema, Repository } from 'redis-om';

// Define schema with string array field
const userSchema = new Schema('User', {
  metadata: { type: 'string[]' }
}, {
  dataStructure: 'HASH'
});

// Create repository
const userRepository = new Repository(userSchema, client);

// Test data with pipe character
const testData = {
  metadata: ['{"meta":{"userId":"auth0|vVFy3-1234567"}}']
};

// Store and retrieve
const user = userRepository.createEntity(testData);
await userRepository.save(user);
const retrieved = await userRepository.fetch(user.entityId);

console.log('Original:', testData.metadata);
console.log('Retrieved:', retrieved.metadata);

Actual vs Expected Output

Data stored in Redis:

{"meta":{"userId":"auth0|vVFy3-1234567"}}

Expected retrieval:

["{\"meta\":{\"userId\":\"auth0|vVFy3-1234567\"}}"]

Actual retrieval (corrupted):

["{\"meta\":{\"userId\":\"auth0","vVFy3-1234567\"}}"]

Root Cause Analysis

The issue appears to be related to how redis-om parses array fields when using the HASH data structure. It seems to be using the pipe character (|) as a delimiter for array elements, which conflicts with pipe characters that are part of the actual data content.

Workaround

The issue does not occur when using the JSON data structure:

const userSchema = new Schema('User', {
  metadata: { type: 'string[]' }
}, {
  dataStructure: 'JSON'  // Works correctly
});

However, this workaround is not viable for environments where the Redis JSON module is not available.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions