Skip to content

Commit

Permalink
Update register stream timestamp more often
Browse files Browse the repository at this point in the history
### What changes are proposed in this pull request?

During a worker registration stream, we identify stale stream and
`DefaultBlockMaster$WorkerRegisterStreamGCExecutor` recycles them based
on the last-message-received timestamp. This change updates the
timestamp more often.

### Why are the changes needed?

There will be fewer false positives for stale streams. The false
positives are expensive.

### Does this PR introduce any user facing changes?

NA

pr-link: Alluxio#14566
change-id: cid-04a8cc61d7a00933949aed40b2903cdc244e7d64
  • Loading branch information
jiacheliu3 authored Nov 23, 2021
1 parent 635cebb commit 62f980a
Showing 1 changed file with 6 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,9 @@ public RegisterWorkerPResponse call() throws Exception {
+ "Probably %s was exceeded!",
PropertyKey.MASTER_WORKER_REGISTER_STREAM_RESPONSE_TIMEOUT.toString());

// Update the TS before and after processing the request, so that when a message
// takes long to process, the stream does not time out.
mContext.updateTs();
mBlockMaster.workerRegisterStream(mContext, chunk, isHead);
mContext.updateTs();
// Return an ACK to the worker so it sends the next batch
Expand Down Expand Up @@ -151,6 +154,9 @@ public RegisterWorkerPResponse call() throws Exception {
+ "Probably %s was exceeded!",
PropertyKey.MASTER_WORKER_REGISTER_STREAM_RESPONSE_TIMEOUT.toString());

// Update the TS before and after processing the request, so that when a message
// takes long to process, the stream does not time out.
mContext.updateTs();
mBlockMaster.workerRegisterFinish(mContext);
mContext.updateTs();

Expand Down

0 comments on commit 62f980a

Please sign in to comment.