grpc: fix receiving empty messages when compression is enabled and maxReceiveMessageSize is MaxInt #7753 #7918

vinothkumarr227 · 2024-12-10T14:11:51Z

RELEASE NOTES:

grpc: fix receiving empty messages when compression is enabled and maxReceiveMessageSize is maxInt

codecov · 2024-12-12T07:04:41Z

Codecov Report

Attention: Patch coverage is 72.97297% with 10 lines in your changes missing coverage. Please review.

Project coverage is 82.04%. Comparing base (e8055ea) to head (da82d30).
Report is 17 commits behind head on master.

Files with missing lines	Patch %	Lines
rpc_util.go	72.97%	7 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7918      +/-   ##
==========================================
- Coverage   82.08%   82.04%   -0.04%     
==========================================
  Files         379      381       +2     
  Lines       38261    38547     +286     
==========================================
+ Hits        31406    31626     +220     
- Misses       5551     5600      +49     
- Partials     1304     1321      +17

Files with missing lines	Coverage Δ
rpc_util.go	`79.48% <72.97%> (+0.16%)`	⬆️

... and 25 files with indirect coverage changes

arjan-bal · 2024-12-12T08:06:20Z

rpc_util.go

 	if err != nil {
 		out.Free()
 		return nil, 0, err
 	}
+	if err = checkReceiveMessageOverflow(int64(out.Len()), int64(maxReceiveMessageSize), dcReader); err != nil {
+		return nil, out.Len() + 1, err


Adding 1 to out.Len() can result in an overflow on 32 bit systems. The following part of the function seems strange to me:

Optionally, if data will be over maxReceiveMessageSize, just return the size

I suggest making the following change to avoid this:

Declare a global error for indicating that the max receive size is exceeded:

var errMaxMessageSizeExceeded = errors.New("max message size exceeded")

When the check here fails, nil, 0, errMaxMessageSizeExceeded. Update the godoc to mention the same.

In the caller, i.e. recvAndDecompress, instead of checking if size > maxReceiveMessageSize, check if err == errMaxMessageSizeExceeded. Also update the error message to not mention the actual size, because we didn't read the entire message anyways: grpc: received message after decompression larger than max %d.

This ensures we're using the returned error to indicate the failure instead of relying on special values of the bytes read count.

Please let the reviewer resolve comments.

We should not attempt to add out.Len() + 1, instead return errMaxMessageSizeExceeded to signal the same.

wait, in case of overflow i think we are still reading the partial data so we need to return the length upto which we have read?

so, the logic should be read till maxMessageSize -1 first and then check if we still have bytes to read by reading one byte and if we overflow, throw the error along with lenght of how much we have read

rpc_util.go

arjan-bal · 2024-12-13T07:19:09Z

rpc_util.go

 	if err != nil {
 		out.Free()
 		return nil, 0, err
 	}
+	if err = checkReceiveMessageOverflow(int64(out.Len()), int64(maxReceiveMessageSize), dcReader); err != nil {
+		return nil, out.Len() + 1, err


Please let the reviewer resolve comments.

arjan-bal · 2024-12-13T07:19:27Z

rpc_util.go

 	if err != nil {
 		out.Free()
 		return nil, 0, err
 	}
+	if err = checkReceiveMessageOverflow(int64(out.Len()), int64(maxReceiveMessageSize), dcReader); err != nil {
+		return nil, out.Len() + 1, err


We should not attempt to add out.Len() + 1, instead return errMaxMessageSizeExceeded to signal the same.

rpc_util.go

rpc_util_test.go

This reverts commit a21e192.

…c#7956)" This reverts commit 09132bf.

This reverts commit 46faf72.

…age size (grpc#7938)" This reverts commit bd4989a.

grpc#7908)" This reverts commit 985965a.

rpc_util_test.go

arjan-bal · 2024-12-23T11:00:13Z

rpc_util_test.go

+			wantErr:               nil,
+		},
+		{
+			name:                  "Handles large buffer size (MaxInt)",


Please don't use special characters (like (, )) in the subtest names. Subtest names are used in test filters and go special characters will be escaped. It is not immediately clear what the escaped name would be in such cases.

arjan-bal · 2024-12-23T11:02:35Z

rpc_util_test.go

+// premature data end. It ensures that the function behaves correctly with different
+// inputs, buffer sizes, and error conditions, using the "gzip" compressor for testing.
+func (s) TestDecompress(t *testing.T) {
+	c := encoding.GetCompressor("gzip")


Please use a longer name, e.g: compressor. Then length of a variable name should be proportional to it's scope. If a variable is accessed after 20 lines or so, it should have a longer name to avoid confusions.

arjan-bal · 2024-12-23T11:03:44Z

rpc_util_test.go

+			}
+
+			if !cmp.Equal(tc.want, output.Materialize()) {
+				t.Fatalf("decompress() output mismatch: Got = %v, Want = %v", output.Materialize(), tc.want)


Small G in got, small Winwant`.

arjan-bal · 2024-12-23T11:06:23Z

rpc_util.go

+}
+
+// doesReceiveMessageOverflow checks if the number of bytes read from the stream
+// exceeds the maximum receive message size allowed by the client. If the `readBytes`


nit: exceed instead of exceeds.

allowed by the client should be allowed by the receiver, the receiver could be the client or the server.

rpc_util.go

purnesh42H · 2024-12-24T06:26:19Z

rpc_util_test.go

+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			output, err := decompress(compressor, tc.input, tc.maxReceiveMessageSize, mem.DefaultBufferPool())
+


nix new line

purnesh42H · 2024-12-24T06:26:25Z

rpc_util_test.go

+			if !cmp.Equal(err, tc.wantErr, cmpopts.EquateErrors()) {
+				t.Fatalf("decompress() error = %v, wantErr = %v", err, tc.wantErr)
+			}
+


nix new line

purnesh42H · 2024-12-24T06:27:32Z

rpc_util_test.go

+			output, err := decompress(compressor, tc.input, tc.maxReceiveMessageSize, mem.DefaultBufferPool())
+
+			if !cmp.Equal(err, tc.wantErr, cmpopts.EquateErrors()) {
+				t.Fatalf("decompress() error = %v, wantErr = %v", err, tc.wantErr)


t.Fatalf("decompress() err = %v, wantErr = %v", err, tc.wantErr) or .Fatalf("decompress() error = %v, want error = %v", err, tc.wantErr)

purnesh42H · 2024-12-24T10:13:28Z

rpc_util_test.go

+		},
+		{
+			name:                  "Handles large buffer size MaxInt",
+			input:                 mustCompress(t, []byte("large message")),


large message is confusing. The bug which we are solving returns nil irrespective of what message size is if maxReceiveMessageSize is math.MaxInt. The test can be just Decompresses successfully with maxReceiveMessageSize MaxInt. Basically it can be similar to first test case except the maxReceiveMessageSize is MaxInt.

The other test case where you are actually going to overflow by providing a message larger than MaxInt is hard to write as unit test

purnesh42H · 2024-12-24T10:16:23Z

rpc_util_test.go

+	return mem.BufferSlice{mem.NewBuffer(&compressedData, nil)}
+}
+
+// TestDecompress tests the decompress function with various scenarios, including


TestDecompress tests the decompress function behaves correctly for following scenarios

decompress successfully when message is <= maxReceiveMessageSize

errors when message > maxReceiveMessageSize

decompress successfully when maxReceiveMessageSize is MaxInt

purnesh42H · 2024-12-24T10:17:34Z

rpc_util_test.go

@@ -294,3 +298,75 @@ func BenchmarkGZIPCompressor512KiB(b *testing.B) {
 func BenchmarkGZIPCompressor1MiB(b *testing.B) {
 	bmCompressor(b, 1024*1024, NewGZIPCompressor())
 }
+
+// mustCompress compresses the input data and returns a BufferSlice.


i think this function is more like compressWithDeterministicError so that we can do comparison in our test cases?

rpc_util.go

arjan-bal

LGTM! Adding a second reviewer.

dfawley · 2025-01-09T18:55:30Z

rpc_util.go

@@ -808,6 +809,9 @@ func (p *payloadInfo) free() {
 	}
 }

+// errMaxMessageSizeExceeded  represents an error due to exceeding the maximum message size limit.


This comment is unnecessary.

dfawley · 2025-01-09T19:30:55Z

rpc_util.go

+				if size := len(uncompressedBuf); size > maxReceiveMessageSize {
+					out.Free()
+					// TODO: Revisit the error code. Currently keep it consistent with java
+					// implementation.
+					return nil, status.Errorf(codes.ResourceExhausted, "grpc: received message after decompression larger than max (%d vs. %d)", size, maxReceiveMessageSize)
+				}


This feels like a good amount of duplication now.

Maybe we can move more logic into decompress (e.g. pass in dc) in order to avoid that?

dfawley · 2025-01-09T19:31:54Z

rpc_util.go

+
+	if doesReceiveMessageOverflow(out.Len(), maxReceiveMessageSize, dcReader) {
+		out.Free()
+		return nil, errMaxMessageSizeExceeded


Can this just return the status.Errorf(...) instead of returning a magic error that gets converted into that?

(We'd want to make sure this method always returns an error that is appropriate to just return directly from recvAndDecompress.)

vinothkumarr227 · 2025-01-10T17:37:03Z

@purnesh42H I mistakenly clicked on 'Unassign me' from the issue. Can you please reassign the issue to me?

arjan-bal self-requested a review December 11, 2024 11:09

arjan-bal self-assigned this Dec 11, 2024

arjan-bal added Area: RPC Features Includes Compression, Encoding, Attributes/Metadata, Interceptors. Type: Bug and removed Area: RPC Features Includes Compression, Encoding, Attributes/Metadata, Interceptors. labels Dec 11, 2024

arjan-bal added this to the 1.70 Release milestone Dec 11, 2024

arjan-bal reviewed Dec 12, 2024

View reviewed changes

arjan-bal assigned vinothkumarr227 and arjan-bal and unassigned arjan-bal and vinothkumarr227 Dec 12, 2024

arjan-bal reviewed Dec 13, 2024

View reviewed changes

arjan-bal assigned vinothkumarr227 and arjan-bal and unassigned arjan-bal and vinothkumarr227 Dec 13, 2024

arjan-bal mentioned this pull request Dec 17, 2024

test: Add a test for decompression exceeding max receive message size #7938

Merged

arjan-bal reviewed Dec 17, 2024

View reviewed changes

rpc_util.go Outdated Show resolved Hide resolved

rpc_util_test.go Show resolved Hide resolved

rpc_util_test.go Outdated Show resolved Hide resolved

rpc_util_test.go Outdated Show resolved Hide resolved

rpc_util_test.go Outdated Show resolved Hide resolved

arjan-bal reviewed Dec 17, 2024

View reviewed changes

rpc_util_test.go Outdated Show resolved Hide resolved

arjan-bal assigned vinothkumarr227 and unassigned arjan-bal Dec 17, 2024

vinothkumarr227 added 7 commits December 18, 2024 07:40

Fix test cases and update code to latest branch

b08845a

small tweaks

a02e58c

small tweaks

54da70d

update code based on review feedback

4efd6fe

fixed the changes based on review feedback

851d13a

Added test cases for Handles maxReceiveMessageSize as MaxInt64

26bf731

Fixed the review changes

3cf9054

vinothkumarr227 force-pushed the 4552-max-receive-message-size-decompression branch from 9d80990 to 3cf9054 Compare December 18, 2024 07:44

vinothkumarr227 added 7 commits December 23, 2024 06:13

Revert "cleanup: replace dial with newclient (grpc#7920)"

cf12ace

This reverts commit a21e192.

Revert "deps: update crypto dependency to resolve CVE-2024-45337 (grp…

0e927fb

…c#7956)" This reverts commit 09132bf.

Revert "envconfig: enable xDS client fallback by default (grpc#7949)"

ee88fc0

This reverts commit 46faf72.

Revert "test: Add a test for decompression exceeding max receive mess…

4ef6aab

…age size (grpc#7938)" This reverts commit bd4989a.

Revert "outlierdetection: Support health listener for ejection updates (

7ef57fd

grpc#7908)" This reverts commit 985965a.

Revert the changes

20d4dc6

Revert the changes for test cases

0907f9d

arjan-bal reviewed Dec 23, 2024

View reviewed changes

Fix decompression size check and refactor based on review feedback

84e8a4f

vinothkumarr227 requested a review from arjan-bal December 23, 2024 12:02

purnesh42H reviewed Dec 24, 2024

View reviewed changes

vinothkumarr227 added 2 commits December 24, 2024 06:58

small tweaks

77ea230

Fixed the space formatting issues

6ff9321

purnesh42H reviewed Dec 24, 2024

View reviewed changes

arjan-bal changed the title ~~grpc: fix receiving empty messages when compression is enabled and maxReceiveMessageSize is maxInt64 #7753~~ grpc: fix receiving empty messages when compression is enabled and maxReceiveMessageSize is maxInt #7753 Dec 24, 2024

arjan-bal changed the title ~~grpc: fix receiving empty messages when compression is enabled and maxReceiveMessageSize is maxInt #7753~~ grpc: fix receiving empty messages when compression is enabled and maxReceiveMessageSize is MaxInt #7753 Dec 24, 2024

arjan-bal approved these changes Dec 24, 2024

View reviewed changes

arjan-bal assigned dfawley Dec 24, 2024

Rename the test cases and refactor some of them

82124a4

dfawley reviewed Jan 9, 2025

View reviewed changes

dfawley removed their assignment Jan 9, 2025

Fix reviewed changes

4e9c665

vinothkumarr227 requested a review from dfawley January 10, 2025 12:45

small tweaks

da82d30

vinothkumarr227 removed their assignment Jan 10, 2025

vinothkumarr227 added 3 commits January 10, 2025 16:49

Fixed test cases for decompress function

33962c2

Fixed test cases for dock block

23baa53

small tweaks

7664e75

dfawley assigned vinothkumarr227 Jan 10, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

grpc: fix receiving empty messages when compression is enabled and maxReceiveMessageSize is MaxInt #7753 #7918

grpc: fix receiving empty messages when compression is enabled and maxReceiveMessageSize is MaxInt #7753 #7918

vinothkumarr227 commented Dec 10, 2024 •

edited by arjan-bal

Loading

codecov bot commented Dec 12, 2024 •

edited

Loading

arjan-bal Dec 12, 2024

arjan-bal Dec 13, 2024

arjan-bal Dec 13, 2024

purnesh42H Dec 17, 2024

purnesh42H Dec 17, 2024

arjan-bal Dec 13, 2024

arjan-bal Dec 13, 2024

arjan-bal Dec 23, 2024

arjan-bal Dec 23, 2024

arjan-bal Dec 23, 2024

arjan-bal Dec 23, 2024

purnesh42H Dec 24, 2024

purnesh42H Dec 24, 2024

purnesh42H Dec 24, 2024

purnesh42H Dec 24, 2024

purnesh42H Dec 24, 2024

purnesh42H Dec 24, 2024

arjan-bal left a comment

dfawley Jan 9, 2025

dfawley Jan 9, 2025

dfawley Jan 9, 2025

vinothkumarr227 commented Jan 10, 2025

grpc: fix receiving empty messages when compression is enabled and maxReceiveMessageSize is MaxInt #7753 #7918

Are you sure you want to change the base?

grpc: fix receiving empty messages when compression is enabled and maxReceiveMessageSize is MaxInt #7753 #7918

Conversation

vinothkumarr227 commented Dec 10, 2024 • edited by arjan-bal Loading

codecov bot commented Dec 12, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arjan-bal left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vinothkumarr227 commented Jan 10, 2025

vinothkumarr227 commented Dec 10, 2024 •

edited by arjan-bal

Loading

codecov bot commented Dec 12, 2024 •

edited

Loading