Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(client): read events will be blocked when switching to a new segment #235

Merged
merged 11 commits into from
Sep 5, 2022

Conversation

wenfengwang
Copy link
Contributor

@wenfengwang wenfengwang commented Sep 5, 2022

What problem does this PR solve?

Issue Number: NO

Problem Summary

In now version, it will be sucked to get events from segments that maximum offset less than the target offset, which was caused by the client updating route info mechanism: it maintains the last segment's endOffset as MAXIMUM_INT64, so exceeded offset doesn't trigger route info updating, only each periodical(30s) task will update route info and then consuming back to normal.

What is changed and how does it work?

Add force flag to force update route info when client meets Overflow error.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)

it can be seen the blocking time reduced to 1-2s from 10-30s

  • No code

@wenfengwang wenfengwang changed the title fix: get event long blocking when segment switch happended fix: get event is blocked by segment switch Sep 5, 2022
@wenfengwang wenfengwang changed the title fix: get event is blocked by segment switch fix: get event is blocked by segment switching Sep 5, 2022
@codecov
Copy link

codecov bot commented Sep 5, 2022

Codecov Report

Merging #235 (cda684a) into main (2eddcb6) will decrease coverage by 0.00%.
The diff coverage is 64.28%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #235      +/-   ##
==========================================
- Coverage   64.38%   64.37%   -0.01%     
==========================================
  Files         115      115              
  Lines       11269    11275       +6     
==========================================
+ Hits         7255     7258       +3     
- Misses       3462     3464       +2     
- Partials      552      553       +1     
Impacted Files Coverage Δ
internal/controller/eventbus/eventlog/eventlog.go 68.09% <0.00%> (ø)
internal/controller/eventbus/block/block.go 56.09% <70.00%> (-0.32%) ⬇️
internal/controller/eventbus/controller.go 21.42% <100.00%> (ø)
internal/timer/timingwheel/timingwheel.go 87.01% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update af0805a...cda684a. Read the comment docs.

@ifplusor ifplusor changed the title fix: get event is blocked by segment switching fix: read events will be blocked when switching to a new segment Sep 5, 2022
Copy link
Collaborator

@ifplusor ifplusor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wenfengwang wenfengwang merged commit 714e228 into main Sep 5, 2022
@ifplusor ifplusor changed the title fix: read events will be blocked when switching to a new segment fix(client): read events will be blocked when switching to a new segment Sep 6, 2022
@ifplusor ifplusor added this to the v0.3 milestone Sep 7, 2022
@ifplusor ifplusor added the bug Something isn't working label Sep 7, 2022
@wenfengwang wenfengwang deleted the fix-comsuming-shake branch September 10, 2022 07:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants