-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ext_proc filter crash when sending trailers when request has no body #27430
ext_proc filter crash when sending trailers when request has no body #27430
Conversation
Signed-off-by: Yanjun Xiang <yanjunxiang@google.com>
/assign @adisuissa @yanavlasov |
The detail fuzz report could be found here: https://clusterfuzz.corp.google.com/testcase-detail/5547889199546368 The crash traceback decode is: [2023-05-16 15:50:05.109][2540875][info][misc] [test/test_common/test_random_generator.cc:20] TestRandomGenerator running with seed -38107400 [2023-05-16 15:50:05.171][2540875][trace][ext_proc] [source/extensions/filters/http/ext_proc/ext_proc.cc:183] decodeHeaders: end_stream = false [2023-05-16 15:50:05.172][2540875][trace][ext_proc] [source/extensions/filters/http/ext_proc/ext_proc.cc:467] decodeTrailers
|
The test case to reproduce the crash is: cat test/extensions/filters/http/ext_proc/unit_test_fuzz/ext_proc_corpus/clusterfuzz-testcase-minimized-ext_proc_unit_test_fuzz-5547889199546368.test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the fix!
I don't really know the inner details of ext-proc, so I just left some high-level observations.
@mpwarres can you PTAL?
@@ -195,7 +195,9 @@ class Filter : public Logger::Loggable<Logger::Id::ext_proc>, | |||
|
|||
void sendBufferedData(ProcessorState& state, ProcessorState::CallbackState new_state, | |||
bool end_stream) { | |||
sendBodyChunk(state, *state.bufferedData(), new_state, end_stream); | |||
if (state.hasBufferedData()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not an expert on ext-proc, so high-level observation.
IIUC the current fix prevents sending a body of length 0 with end_stream. If this is the desired behavior, then I think it is ok.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the comments!
Actually, ext_proc filter supports sending a body of length 0:
TEST_P(ExtProcIntegrationTest, BufferBodyOverridePostWithEmptyBody) { |
This change is causing test failures. So, we changed it into if the no buffered data, then sends an empty body.
After that, we seeing another issue with stream_ pointer as null, so the new change also fixed it. Please check the root cause comments for details.
@@ -195,7 +195,9 @@ class Filter : public Logger::Loggable<Logger::Id::ext_proc>, | |||
|
|||
void sendBufferedData(ProcessorState& state, ProcessorState::CallbackState new_state, | |||
bool end_stream) { | |||
sendBodyChunk(state, *state.bufferedData(), new_state, end_stream); | |||
if (state.hasBufferedData()) { | |||
sendBodyChunk(state, *state.bufferedData(), new_state, end_stream); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Another observation: the call in case of an empty body, will not change the state to new_state
. Is this the desired outcome?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, that's a good catch. We should send an empty body if no buffered data as in the new diffs.
@stevenzzzz as an additional pair of eyes |
Signed-off-by: Yanjun Xiang <yanjunxiang@google.com>
This fuzzer test cases exposed two issues in the ext_proc filter. If the filter processing mode is set into request header: SKIP and request body :BUFFERED, at same time, when the request only contains headers and trailers then it hit two distinct crash:
|
/assign @htuch |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it would be good to add this scenario as a test case in one of the suites (filter or integration). As I think it is hard to read what is going form just the corpus file.
/wait
Signed-off-by: Yanjun Xiang <yanjunxiang@google.com>
Done! |
…nvoyproxy#27430) * ext_proc filter crash when sending trailers without sending body. Signed-off-by: Yanjun Xiang <yanjunxiang@google.com> Signed-off-by: Ryan Eskin <ryan.eskin89@protonmail.com>
ext_proc filter crash when sending trailers when request has no body.
The code is assuming the request always has body available and stored in the ext_proc buffer state when processing trailers, which leads into the crash.
Commit Message:
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]