Skip to content

fix(dynamodb-seeder): Correct S3 file stream handling with transformToString #218

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

cmcneilniu
Copy link

@cmcneilniu cmcneilniu commented May 1, 2025

Resolve the issue: #214

Summary

Fixes JSON parsing errors in the DynamoDB seeder by:

  • Replacing body.toString() with transformToString('utf-8') for proper S3 stream handling

Changes

  • Updated getSeedsFromS3 to use AWS SDK v3's transformToString:
    • Ensures correct UTF-8 decoding of S3 files
    • Prevents malformed JSON errors from stream corruption

Impact

  • Fixes Unexpected token errors when parsing seed files
  • Maintains backward compatibility with existing S3 buckets/keys

Error Context

Fixes cases where body.toString() corrupted S3 files, causing errors like:
Unexpected token 'o' in JSON at position 1

Fixes JSON parsing errors in the DynamoDB seeder by:
- Replacing `body.toString()` with `transformToString('utf-8')` for proper S3 stream handling
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant