-
Notifications
You must be signed in to change notification settings - Fork 8.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HDFS-10930.07.patch #160
HDFS-10930.07.patch #160
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @xiaoyuyao. The patch looks good overall. Findbugs flagged a few warnings and the TestWriteToReplica failure looks related.
There's a few remaining static methods like Storage#rename, and DatanodeUtil#createTmpFile that aren't as easy to factor out. We can look at them in a subsequent Jira.
In a subsequent change we'll want to move the refactored methods into interfaces for testability and optional instrumentation.
ae8b273
to
1cd7ae8
Compare
Update PR to fix the findbugs and unit test failure. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 LGTM, pending Jenkins.
We will compute triggering duration as follows: - If user configures `task.window.ms` we will honor it as the triggering duration - If not, we will use the `GCD(windowTriggerDurations, joinTTLs)` as the triggering duration. Changes in this PR: - Common Interface for all time based triggers - Additional APIs in `StreamGraphImpl` to recursively traverse all `OperatorSpec`s - Recursive computation of `triggerInterval` for each `WindowOperatorSpec` - Tests for all the above Author: vjagadish1989 <jvenkatr@linkedin.com> Reviewers: Prateek Maheshwari <pmaheshw@linkedin.com>, Jacob Maes <jmaes@linkedin.com>, Xinyu Liu <xinyu@apache.org> Closes apache#160 from vjagadish1989/samza-1155
Create a PR for easy review.