Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-50967][SS] Add option to skip emitting initial state keys within the FMGWS operator #49632

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

anishshri-db
Copy link
Contributor

What changes were proposed in this pull request?

Add option to skip emitting initial state keys within the FMGWS operator

Why are the changes needed?

Without this change, user does not have an easy way of filtering initial state rows emitted as part of the first batch.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Added unit tests

[info] Run completed in 15 seconds, 349 milliseconds.
[info] Total number of tests run: 4
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 4, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 51 s, completed Jan 23, 2025, 4:26:03 PM

Was this patch authored or co-authored using generative AI tooling?

No

@anishshri-db anishshri-db changed the title [SPARK-50967] Add option to skip emitting initial state keys within the FMGWS operator [SPARK-50967][SS] Add option to skip emitting initial state keys within the FMGWS operator Jan 24, 2025
@anishshri-db
Copy link
Contributor Author

cc - @HeartSaVioR - PTAL, thx !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant