Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle long messages in event parser #2

Merged
merged 2 commits into from
Jul 8, 2023

Conversation

aldld
Copy link
Contributor

@aldld aldld commented Jul 7, 2023

This PR fixes a bug that causes the go-sse client to drop messages that are longer than 4096 bytes with an Unexpected EOF error.

The issue here is that the Scanner that gets used to fill the input buffer for the byte parser (which gets used to extract fields) may not call the SplitFunc with a buffer containing a full token. In that case, the split function needs to request more data, signalling to the Scanner to call the SplitFunc again with a larger buffer.

This PR includes a new test case in TestReaderParser that fails if this change is not present, along with a new test in internal/parser/split_func_test.go.

Copy link
Owner

@tmaxmax tmaxmax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for raising the issue and the PR! The requested changes address only some style preferences. It's also great you've added tests for this edge case – seems like I missed that part of bufio.Scanner's behavior. I appreciate you taking time to find and fix this issue. Was it easy navigating the codebase? If you have any naming/layout/other suggestions, please let me know.

internal/parser/reader_parser.go Outdated Show resolved Hide resolved
internal/parser/reader_parser.go Outdated Show resolved Hide resolved
@tmaxmax
Copy link
Owner

tmaxmax commented Jul 7, 2023

Also make sure to rebase on master – I've updated and fixed the actions.

@aldld aldld force-pushed the aldld/split-func-long-strings branch from de55b92 to fb68785 Compare July 8, 2023 02:18
@aldld
Copy link
Contributor Author

aldld commented Jul 8, 2023

Thank you for raising the issue and the PR! The requested changes address only some style preferences. It's also great you've added tests for this edge case – seems like I missed that part of bufio.Scanner's behavior. I appreciate you taking time to find and fix this issue. Was it easy navigating the codebase? If you have any naming/layout/other suggestions, please let me know.

Thanks for the feedback! I've rebased and updated the PR based on your comments. I found most of the codebase quite easy to follow, particularly the top-level package and the readme documentation.

It did take me some time to wrap my head around the internal/parser package, mainly in terms of understanding the relationship between the different functions named Scan that get called by ReaderParser.Scan(). It was a bit difficult to keep in my head the difference between e.g. r.p.Scan() and r.sc.Scan(), so perhaps naming those fields a bit differently (maybe eventScanner and fieldScanner?) would help clarify what's happening at each step.

Other than that, though, I had no trouble understanding and navigating the codebase.

@tmaxmax
Copy link
Owner

tmaxmax commented Jul 8, 2023

Thank you for raising the issue and the PR! The requested changes address only some style preferences. It's also great you've added tests for this edge case – seems like I missed that part of bufio.Scanner's behavior. I appreciate you taking time to find and fix this issue. Was it easy navigating the codebase? If you have any naming/layout/other suggestions, please let me know.

Thanks for the feedback! I've rebased and updated the PR based on your comments. I found most of the codebase quite easy to follow, particularly the top-level package and the readme documentation.

It did take me some time to wrap my head around the internal/parser package, mainly in terms of understanding the relationship between the different functions named Scan that get called by ReaderParser.Scan(). It was a bit difficult to keep in my head the difference between e.g. r.p.Scan() and r.sc.Scan(), so perhaps naming those fields a bit differently (maybe eventScanner and fieldScanner?) would help clarify what's happening at each step.

Other than that, though, I had no trouble understanding and navigating the codebase.

That's a very useful suggestion, thank you! I'll implement it in another PR. Will merge your PR and release a new version soon, as I've found another bug and an opportunity to remove some code from the codebase.

@codecov-commenter
Copy link

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.02 🎉

Comparison is base (3baa34d) 96.39% compared to head (c235a39) 96.41%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files
@@            Coverage Diff             @@
##           master       #2      +/-   ##
==========================================
+ Coverage   96.39%   96.41%   +0.02%     
==========================================
  Files          14       14              
  Lines        1054     1060       +6     
==========================================
+ Hits         1016     1022       +6     
  Misses         30       30              
  Partials        8        8              
Impacted Files Coverage Δ
internal/parser/reader_parser.go 100.00% <100.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@tmaxmax tmaxmax merged commit 4ad53c4 into tmaxmax:master Jul 8, 2023
@tmaxmax
Copy link
Owner

tmaxmax commented Jul 8, 2023

See latest release. I've also made the suggested refactor.

This was referenced Aug 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants