Skip to content

Added support for DynamoDB data jobs - #597

Merged
juanpardo merged 8 commits into
masterfrom
jp/dynamodb-import-support
Mar 19, 2026
Merged

Added support for DynamoDB data jobs#597
juanpardo merged 8 commits into
masterfrom
jp/dynamodb-import-support

Conversation

@juanpardo

@juanpardo juanpardo commented Mar 9, 2026

Copy link
Copy Markdown
Contributor

Summary of the changes / Why this is an improvement

Added support for DynamoDB data jobs

Checklist

@juanpardo
juanpardo requested review from plaharanne and tomach March 9, 2026 10:03
@coderabbitai

coderabbitai Bot commented Mar 9, 2026 •
edited
Loading

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f2bb8a64-de0f-4919-84d2-3e3b08c59135

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch jp/dynamodb-import-support
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Tip

You can validate your CodeRabbit configuration file in your editor.

If your editor has YAML language server, you can enable auto-completion and validation by adding # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json at the top of your CodeRabbit configuration file.

@amotl amotl mentioned this pull request Mar 9, 2026
Open
@amotl

amotl commented Mar 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

I would also like to add data loading support for DynamoDB to the SDK. This patch makes it easy to learn about relevant ingredients. Thank you! I am looking forward to the upcoming release.

tomach
tomach reviewed Mar 10, 2026
View reviewed changes
Comment thread croud/clusters/commands.py Outdated
}
if args.endpoint:
extra_body["dynamodb"]["endpoint"] = args.endpoint
if args.kinesis_stream_name:

@tomach tomach Mar 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should there be a validation that kinesis_stream_name is required if ingestion_type is IMPORT_AND_CDC or CDC_ONLY?

@juanpardo juanpardo Mar 10, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I just added a check

Comment thread tests/commands/test_clusters.py Outdated
@juanpardo
juanpardo force-pushed the jp/dynamodb-import-support branch from f501061 to 10571bb Compare March 10, 2026 09:51
@juanpardo
juanpardo requested a review from tomach March 10, 2026 10:41
tomach
tomach reviewed Mar 10, 2026
View reviewed changes
Comment thread croud/clusters/commands.py Outdated
extra_body["ingestion_type"] = args.ingestion_type
if "CDC" in extra_body.get("ingestion_type", ""):
if not args.kinesis_stream_name:
raise Exception(

@tomach tomach Mar 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Raising a plain Exception here will print a full stack trace I think? maybe raise a ValueError so the error looks a bit better? sth like

raise ValueError(
    "--kinesis-stream-name must be set when using CDC ingestion types."
)

@juanpardo juanpardo Mar 13, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Raising any exception leads to a stack trace, so I had to end up using a controlled error exit

plaharanne
plaharanne reviewed Mar 11, 2026
View reviewed changes
Comment thread docs/commands/data-import-export.rst
@juanpardo
juanpardo requested review from plaharanne and tomach March 13, 2026 10:53
plaharanne
plaharanne reviewed Mar 13, 2026
View reviewed changes
Comment thread docs/commands/data-import-export.rst
@juanpardo
juanpardo requested a review from plaharanne March 16, 2026 08:52
plaharanne
plaharanne reviewed Mar 16, 2026
View reviewed changes
Comment thread croud/__main__.py Outdated
plaharanne
plaharanne reviewed Mar 16, 2026
View reviewed changes
Comment thread docs/commands/data-import-export.rst
@juanpardo
juanpardo requested a review from plaharanne March 17, 2026 10:17
plaharanne
plaharanne approved these changes Mar 19, 2026
View reviewed changes

@plaharanne plaharanne left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@juanpardo
juanpardo merged commit 415661a into master Mar 19, 2026
10 checks passed
@juanpardo
juanpardo deleted the jp/dynamodb-import-support branch March 19, 2026 09:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@plaharanne plaharanne plaharanne approved these changes
@tomach tomach Awaiting requested review from tomach

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@juanpardo @amotl @plaharanne @tomach