Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rpcMergeMessageSend_TMROLE_1、rpcMergeMessageSend_RMROLE_1的cpu使用率过高 #6041

Open
1 task
gushuyong opened this issue Nov 17, 2023 · 4 comments
Open
1 task
Labels
good first issue Good for newcomers task: help-wanted Extra attention is needed type: optimize

Comments

@gushuyong
Copy link

  • I have searched the issues of this repository and believe that this is not a duplicate.

Ⅰ. Issue Description

seata版本:1.4.2
服务配置:-Xms512m -Xmx1024m -XX:MaxNewSize=128m
因一直收到线上某服务cpu使用率过高的告警,就先nacos进行下线,准备重启大法,发现还是一直告警。然后arthas排查,最后发现无流量情况下,
rpcMergeMessageSend_TMROLE_1、rpcMergeMessageSend_RMROLE_1这2线程的cpu使用率过高

Ⅱ. Describe what happened

"rpcMergeMessageSend_TMROLE_1" Id=31 cpuUsage=31% TIMED_WAITING on java.lang.Object@1c0c220d
at java.lang.Object.wait(Native Method)
- waiting on java.lang.Object@1c0c220d
at io.seata.core.rpc.netty.AbstractRpcRemotingClient$MergedSendRunnable.run(AbstractRpcRemotingClient.java:301)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)

Number of locked synchronizers = 1
- java.util.concurrent.ThreadPoolExecutor$Worker@4998e74b

"rpcMergeMessageSend_RMROLE_1" Id=34 cpuUsage=30% TIMED_WAITING on java.lang.Object@25b70c1e
at java.lang.Object.wait(Native Method)
- waiting on java.lang.Object@25b70c1e
at io.seata.core.rpc.netty.AbstractRpcRemotingClient$MergedSendRunnable.run(AbstractRpcRemotingClient.java:301)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)

Number of locked synchronizers = 1
- java.util.concurrent.ThreadPoolExecutor$Worker@2b08772d
- 

If there is an exception, please attach the exception trace:

Just paste your stack trace here!

Ⅲ. Describe what you expected to happen

Ⅳ. How to reproduce it (as minimally and precisely as possible)

  1. xxx
  2. xxx
  3. xxx

Minimal yet complete reproducer code (or URL to code):

Ⅴ. Anything else we need to know?

Ⅵ. Environment:

  • JDK version(e.g. java -version):1.8
  • Seata client/server version: 1.0.0/1.4.2
  • Database version:
  • OS(e.g. uname -a):
  • Others:
@funky-eyes
Copy link
Contributor

我有三个想法:
1.不再每1ms 进行wait,直接以10ms或者更大的值进行wait,反正sendSyncRequest进来将消息放入队列的时候一定会唤醒线程
2.tm和rm的线程合并掉,不需要单独一个线程
3.直接改造,用批量write后再一次性flush达到批量的效果,不再设置单独的线程
I have three ideas:

  1. no longer wait every 1ms, directly wait with 10ms or greater, anyway, sendSyncRequest come in to put the message into the queue will wake up the thread
  2. tm and rm threads are merged off, do not need a separate thread
  3. directly remodeling, with a batch of write and then a one-time flush to achieve the effect of batch of effect, no longer set up a separate thread

@PleaseGiveMeTheCoke
Copy link
Contributor

@funky-eyes 关于你的前两个想法,我提交了一个pr,能请你帮忙code review 一下吗

@funky-eyes
Copy link
Contributor

@funky-eyes 关于你的前两个想法,我提交了一个pr,能请你帮忙code review 一下吗

非常欢迎你的贡献,在2.0发布完成后,我将进行review
Thank you very much for your contribution. After the completion of version 2.0, I will review it

@leizhiyuan leizhiyuan added this to the 2.1.0 milestone Nov 25, 2023
@wt-better
Copy link
Contributor

Using seata version 1.4.2, empty traffic, I have not reproduced your problem, can you provide a demo code to replay.

@slievrly slievrly removed this from the 2.1.0 milestone Feb 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers task: help-wanted Extra attention is needed type: optimize
Projects
None yet
Development

No branches or pull requests

6 participants