upgrade grpc to v1.73

[joechenrh]

Why upgrade grpc

Because we are using some expermential API that was already renamed. This will block those libraries that depend on this to use newer version of grpc. (And I found someone has already want to do this: #1672)

What has be changed

• Change WithRecvBufferPool with WithBufferPool.
• Use old codec in grpc 1.63 to make sharedBytes works.

sysbench test

# range scan
sysbench oltp_read_only \
  --db-driver=mysql --mysql-host=172.16.4.180 --mysql-port=32003 \
  --mysql-user=root --mysql-db=test \
  --tables=1 --table-size=100000000 \
  --threads=16 --time=1800 --report-interval=10 \
  --rand-type=uniform --range_size=4096 --range_selects=on \
  --point_selects=0 \
  --db-ps-mode=disable run

The QPS/latency are almost same for both version. The memory for 1.73 is slightly reduced (4.09G->3.98G), while the gc rate increases (0.25->0.3), which is an expected result by introducing sync.Pool.

# point lookup
sysbench oltp_read_only \
  --db-driver=mysql --mysql-host=172.16.4.180 --mysql-port=32003 \
  --mysql-user=root --mysql-db=test \
  --tables=1 --table-size=100000000 \
  --threads=16 --time=1800 --report-interval=10 \
  --rand-type=uniform --range_size=4096 --range_selects=on \
  --db-ps-mode=disable run

For point lookup, both version show almost no difference in QPS/lantence/memory.

Summary by CodeRabbit

• Chores
  • Upgraded gRPC and Protocol Buffer dependencies for improved stability, performance, and compatibility.
  • Reworked internal buffer pooling to a tiered/no-op approach, reducing memory overhead and improving throughput.
  • Added a legacy Protocol Buffer codec to preserve compatibility with older gRPC/protobuf usage and ease migrations.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[ti-chi-bot]

Welcome @joechenrh!

It looks like this is your first PR to tikv/client-go 🎉.

I'm the bot to help you request reviewers, add labels and more, See available commands.

We want to make sure your contribution gets all the attention it needs!



Thank you, and welcome to tikv/client-go. 😃

[joechenrh]

/hold

[joechenrh]

/ok-to-test

[ti-chi-bot]

@joechenrh: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

Details

In response to this:

/ok-to-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[joechenrh]

/unhold

[joechenrh]

/run-all-tests

[joechenrh]

Previously, grpc was upgraded to 1.64 and reverted (by #1369) due to errors in TiFlash regression tests. So I keep using old API DialContext here.

I've ran a test internally with this branch. The plan ID is 7972306, and you can check the result.

---

rpc error: code = DeadlineExceeded desc = received context error while waiting for new LB policy update: context deadline exceeded
rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp 10.200.86.26:3930: i/o timeout

(Although I suspect that error is not related to grpc API change because it seems like we couldn't connect to 10.200.86.26:3930. That is, the first mistake was the result, and the second mistake was the cause, as resolver didn't provide any address to LB.)

[dveeden]

This would fix #1789, right?

[coderabbitai]

📝 Walkthrough

Walkthrough

Upgrades gRPC/protobuf dependencies, adds a legacy protobuf codec, and changes client buffer-pool and batch-encoding logic to use tiered or nop buffer pools and pointer-backed encoded buffers (no exported API changes).

Changes

Cohort / File(s)	Summary
Dependency Upgrades go.mod	Bumped google.golang.org/grpc v1.63.2 → v1.73.0; updated indirect google.golang.org/genproto/... and google.golang.org/protobuf versions.
Connection pool / Dial internal/client/conn_pool.go	Added import google.golang.org/grpc/mem; added //nolint:SA1019 on grpc.DialContext; when shared buffer pool is disabled, inject legacyCodec{} via ForceCodec and use mem.NopBufferPool{} to bypass shared recv buffer pool.
Batch encoding & pooling internal/client/client_batch.go	Replaced global shared buffer pool with mem.NewTieredBufferPool(...); encodeRequestCmd marshals into pointer-backed slice (*data), adjusted Put/Get to use pointer-held slices, updated length checks and assignment so req.Cmd references pointer-held encoded data.
Legacy codec internal/client/legacy_codec.go	New unexported legacyCodec implementing Marshal, Unmarshal, Name, and messageV2Of to adapt proto v1→v2 messages for pre-1.66 grpc-go behavior.

Sequence Diagram(s)

(Skipped — changes are dependency and internal buffer/codec plumbing without a new multi-component runtime control flow needing visualization.)

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

• client: pre-encode request before sending to batch-send-loop #1841 — Related changes to internal/client/client_batch.go around batch encoding and buffer pool refactoring.

Suggested labels

lgtm, size/XL

Suggested reviewers

• ekexium
• cfzjywxk

Poem

🐰 I hopped through modules, buffers in tow,
Legacy codec hummed where old bytes go,
Tiered pools stacked, nop ones stood by,
Pointer slices danced and marshaled high,
🥕 A little hop — the code runs so!

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'upgrade grpc to v1.73' directly and specifically describes the main change in the pull request, which is updating the gRPC dependency from v1.63.2 to v1.73.0 as shown in go.mod.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@internal/resourcecontrol/resource_control.go`:
- Around line 184-190: The code reads r.Data for a *tikvrpc.CopStreamResponse
even when r.Response can be nil, which can panic; update the CopStreamResponse
handling so the readBytes assignment is guarded by the same nil-check that sets
detailsV2/details (i.e., move readBytes = uint64(len(...)) inside the if
r.Response != nil block or use r.Response.Data instead of r.Data), ensuring
detailsV2, details and readBytes are only accessed when r.Response is non-nil.

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@internal/client/legacy_codec.go`:
- Around line 55-56: The lagecyCodec.Name() implementation currently returns an
empty string which violates gRPC codec requirements; update the lagecyCodec.Name
method to return a stable non-empty codec name (for protobuf use "proto") so
Content-Type and registration logic work correctly; locate the lagecyCodec type
and its Name() method and change the return value from "" to "proto".

🧹 Nitpick comments (3)

internal/client/legacy_codec.go (1)

23-24: Fix typo in codec naming (lagecy → legacy).
The misspelling makes searchability and future maintenance harder. Consider renaming the type/comment and updating its usage in internal/client/conn_pool.go.

internal/client/conn_pool.go (2)

80-81: Add a brief rationale for the SA1019 suppression.
A short comment (e.g., “DialContext kept due to prior regression on upgrade”) will prevent accidental cleanup later.

---

125-152: Avoid mutating opts across loop iterations.
opts is re-assigned inside the loop and then appended again when GrpcSharedBufferPool is false, causing duplicate dial options to accumulate across connections. Use a per-iteration slice to keep options stable and predictable.

♻️ Proposed refactor

-		opts = append([]grpc.DialOption{
+		dialOpts := append([]grpc.DialOption{
 			opt,
 			grpc.WithInitialWindowSize(cfg.TiKVClient.GrpcInitialWindowSize),
 			grpc.WithInitialConnWindowSize(cfg.TiKVClient.GrpcInitialConnWindowSize),
 			grpc.WithUnaryInterceptor(unaryInterceptor),
 			grpc.WithStreamInterceptor(streamInterceptor),
 			grpc.WithDefaultCallOptions(callOptions...),
 			grpc.WithConnectParams(grpc.ConnectParams{
 				Backoff: backoff.Config{
 					BaseDelay:  100 * time.Millisecond, // Default was 1s.
 					Multiplier: 1.6,                    // Default
 					Jitter:     0.2,                    // Default
 					MaxDelay:   3 * time.Second,        // Default was 120s.
 				},
 				MinConnectTimeout: a.dialTimeout,
 			}),
 			grpc.WithKeepaliveParams(keepalive.ClientParameters{
 				Time:    time.Duration(keepAlive) * time.Second,
 				Timeout: cfg.TiKVClient.GetGrpcKeepAliveTimeout(),
 			}),
-		}, opts...)
+		}, opts...)
 		if !cfg.TiKVClient.GrpcSharedBufferPool {
-			opts = append(opts, experimental.WithBufferPool(mem.NopBufferPool{}))
+			dialOpts = append(dialOpts, experimental.WithBufferPool(mem.NopBufferPool{}))
 		}
 		conn, err := a.monitoredDial(
 			ctx,
 			fmt.Sprintf("%s-%d", a.target, i),
 			addr,
-			opts...,
+			dialOpts...,
 		)

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign ekexium for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS
• internal/client/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[joechenrh]

NewSharedBufferPool is replaced with NewTieredBufferPool.

• Previous size tier: 16B, 256B, 4KB, 64KB, 1MB
• New size tier: 256B, 4KB, 16KB, 32KB, 1MB

[joechenrh]

After grpc-go 1.66, the default pool was changed from noop pool to defaultBufferPool
https://github.com/grpc/grpc-go/blob/c7ec4d9ae3281bc57a8adce59b572e56965fb728/dialoptions.go#L705-L718
Here we just align with the old behavior: use noop pool by default.

[joechenrh]

Keep using the old (deprecated) API to avoid potential TiFlash problems.
Ref: pingcap/tiflash#9159

[bufferflies]

What about other go.mod in examples?

[joechenrh]

I think it's OK to keep them, but we can upgrade them as well.

[bufferflies]

I think keep same is better