Limit the number of concurrent gRPC requests

[yah01]

/kind improvement

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: yah01
To complete the pull request process, please assign yanliang567 after the PR has been reviewed.
You can assign the PR to them by writing /assign @yanliang567 in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• internal/distributed/OWNERS
• pkg/util/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mergify]

@yah01 E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

[codecov]

Codecov Report

Merging #27718 (781084c) into master (333a0c8) will decrease coverage by 0.03%.
Report is 5 commits behind head on master.
The diff coverage is 66.66%.

❗ Current head 781084c differs from pull request most recent head df64482. Consider uploading reports for the commit df64482 to get more accurate results

@@            Coverage Diff             @@
##           master   #27718      +/-   ##
==========================================
- Coverage   81.52%   81.49%   -0.03%     
==========================================
  Files         806      806              
  Lines      114272   114294      +22     
==========================================
- Hits        93158    93149       -9     
- Misses      17973    18003      +30     
- Partials     3141     3142       +1     

Files	Coverage Δ
internal/distributed/datacoord/service.go	87.96% <100.00%> (+0.05%)	⬆️
internal/distributed/datanode/service.go	82.02% <100.00%> (+0.08%)	⬆️
internal/distributed/indexnode/service.go	70.98% <100.00%> (+0.18%)	⬆️
internal/distributed/proxy/service.go	79.54% <100.00%> (+0.05%)	⬆️
internal/distributed/querycoord/service.go	73.75% <100.00%> (+0.11%)	⬆️
internal/distributed/querynode/service.go	81.28% <100.00%> (+0.09%)	⬆️
internal/distributed/rootcoord/service.go	80.00% <100.00%> (+0.07%)	⬆️
pkg/util/paramtable/grpc_param.go	79.88% <52.63%> (-1.60%)	⬇️

... and 29 files with indirect coverage changes