Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug fix: restrict lines=True to JSON format in Kafka read_gdf method #17333

Merged

Conversation

a-hirota
Copy link
Contributor

This change modifies the read_gdf method in kafka.py to include the lines=True parameter only when the message format is JSON. For other formats, the lines parameter is omitted to avoid potential errors with unsupported formats.

Description

This pull request modifies the read_gdf method in kafka.py to pass the lines parameter only when the message_format is "json". This prevents lines from being passed to other formats (e.g., CSV, Avro, ORC, Parquet), which do not support this parameter.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

Additional Information

  • Problem: Previously, lines=True was passed regardless of the format, causing issues with formats that do not support lines.
  • Solution: Updated read_gdf in kafka.py to conditionally add the lines=True parameter only for JSON format.
  • Additional Context: This change improves compatibility with different file formats and prevents errors when reading non-JSON formats.

This change modifies the read_gdf method in kafka.py to include
the lines=True parameter only when the message format is JSON.
For other formats, the lines parameter is omitted to avoid
potential errors with unsupported formats.
@a-hirota a-hirota requested a review from a team as a code owner November 15, 2024 01:48
@a-hirota a-hirota requested review from bdice and mroeschke November 15, 2024 01:48
Copy link

copy-pr-bot bot commented Nov 15, 2024

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions bot added the Python Affects Python cuDF API. label Nov 15, 2024
@vyasr vyasr added bug Something isn't working non-breaking Non-breaking change labels Nov 19, 2024
@vyasr
Copy link
Contributor

vyasr commented Nov 19, 2024

/ok to test

Copy link
Contributor

@vyasr vyasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, thanks! And thank you for the contribution!

@vyasr
Copy link
Contributor

vyasr commented Nov 20, 2024

/merge

@rapids-bot rapids-bot bot merged commit 05365af into rapidsai:branch-24.12 Nov 20, 2024
104 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working non-breaking Non-breaking change Python Affects Python cuDF API.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants