Fix DataNode processes event out of order

[XuanYang-cn]

The probability is low so very unlikly to reproduce

See also: #15966

Signed-off-by: yangxuan xuan.yang@zilliz.com

/kind bug

[codecov]

Codecov Report

Merging #17440 (622ae37) into master (6426e3d) will increase coverage by 0.06%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master   #17440      +/-   ##
==========================================
+ Coverage   81.07%   81.14%   +0.06%     
==========================================
  Files         466      466              
  Lines       73458    73459       +1     
==========================================
+ Hits        59556    59605      +49     
+ Misses      11283    11239      -44     
+ Partials     2619     2615       -4     

Impacted Files	Coverage Δ
internal/datanode/data_node.go	81.13% <100.00%> (+0.02%)	⬆️
internal/rootcoord/root_coord.go	79.47% <0.00%> (+0.08%)	⬆️
internal/querycoord/task.go	77.26% <0.00%> (+0.33%)	⬆️
internal/proxy/impl.go	90.06% <0.00%> (+0.49%)	⬆️
internal/querycoord/cluster.go	79.84% <0.00%> (+0.58%)	⬆️
internal/indexcoord/index_coord.go	78.72% <0.00%> (+0.79%)	⬆️
internal/querycoord/querynode.go	72.03% <0.00%> (+1.27%)	⬆️
internal/distributed/querycoord/client/client.go	72.44% <0.00%> (+2.04%)	⬆️
internal/proxy/channels_time_ticker.go	80.59% <0.00%> (+2.23%)	⬆️
internal/mq/mqimpl/rocksmq/client/client_impl.go	80.50% <0.00%> (+2.54%)	⬆️

[XuanYang-cn]

/assign @congqixia

[congqixia]

/lgtm

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: congqixia, XuanYang-cn

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• internal/datanode/OWNERS [congqixia]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment