Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

watch命令报错class redefinition failed: attempted to change the schema (add/remove fields) #1847

Closed
1 task
smallchips opened this issue Jul 7, 2021 · 11 comments

Comments

@smallchips
Copy link

  • 我已经在 issues 里搜索,没有重复的issue。

环境信息

  • arthas-boot.jar 或者 as.sh 的版本: xxx

  • Arthas 版本: 3.5.2

  • 操作系统版本: Linux version 3.10.0-1127.13.1.el7.x86_64 (mockbuild@kbuilder.bsys.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) ) jad 命令增加选项支持显示行号 #1 SMP Tue Jun 23 15:46:38 UTC 2020

  • 目标进程的JVM版本:
    java version "1.8.0_251"
    Java(TM) SE Runtime Environment (build 1.8.0_251-b08)
    Java HotSpot(TM) 64-Bit Server VM (build 25.251-b08, mixed mode)

  • 执行arthas-boot的版本: xxx

重现问题的步骤

watch查看方法的返回参数,报错
[arthas@16943]$ watch Controller receive '{params,returnObj,throwExp}' -v -n 5 -x 2
Affect(class count: 1 , method count: 1) cost in 47 ms, listenerId: 3
Enhance error! exception: java.lang.UnsupportedOperationException: class redefinition failed: attempted to change the schema (add/remove fields)
error happens when enhancing class: class redefinition failed: attempted to change the schema (add/remove fields), check arthas log: /home/apprun/logs/arthas/arthas.log

arthas日志显示
[arthas-command-execute] ERROR c.t.arthas.core.advisor.Enhancer -Enhancer error, matchingClasses: [class xx.xx.controller]
java.lang.UnsupportedOperationException: class redefinition failed: attempted to change the schema (add/remove fields)
at sun.instrument.InstrumentationImpl.retransformClasses0(Native Method)
at sun.instrument.InstrumentationImpl.retransformClasses(InstrumentationImpl.java:144)
at com.taobao.arthas.core.advisor.Enhancer.enhance(Enhancer.java:396)
at com.taobao.arthas.core.command.monitor200.EnhancerCommand.enhance(EnhancerCommand.java:168)
at com.taobao.arthas.core.command.monitor200.EnhancerCommand.process(EnhancerCommand.java:115)
at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl.process(AnnotatedCommandImpl.java:82)
at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl.access$100(AnnotatedCommandImpl.java:18)
at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl$ProcessHandler.handle(AnnotatedCommandImpl.java:111)
at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl$ProcessHandler.handle(AnnotatedCommandImpl.java:108)
at com.taobao.arthas.core.shell.system.impl.ProcessImpl$CommandProcessTask.run(ProcessImpl.java:385)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

期望的结果

watch命令能正常执行

实际运行的结果

如上arthas日志和watch执行命令的报错

把异常信息贴到这里
[arthas-command-execute] ERROR c.t.arthas.core.advisor.Enhancer -Enhancer error, matchingClasses: [class xx.xx.controller]
java.lang.UnsupportedOperationException: class redefinition failed: attempted to change the schema (add/remove fields)
 at sun.instrument.InstrumentationImpl.retransformClasses0(Native Method)
 at sun.instrument.InstrumentationImpl.retransformClasses(InstrumentationImpl.java:144)
 at com.taobao.arthas.core.advisor.Enhancer.enhance(Enhancer.java:396)
 at com.taobao.arthas.core.command.monitor200.EnhancerCommand.enhance(EnhancerCommand.java:168)
 at com.taobao.arthas.core.command.monitor200.EnhancerCommand.process(EnhancerCommand.java:115)
 at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl.process(AnnotatedCommandImpl.java:82)
 at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl.access$100(AnnotatedCommandImpl.java:18)
 at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl$ProcessHandler.handle(AnnotatedCommandImpl.java:111)
 at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl$ProcessHandler.handle(AnnotatedCommandImpl.java:108)
 at com.taobao.arthas.core.shell.system.impl.ProcessImpl$CommandProcessTask.run(ProcessImpl.java:385)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)
@smallchips
Copy link
Author

执行watch命令为:watch xx.xx.controller receive '{params,returnObj,throwExp}' -v -n 5 -x 2
linux系统为:Red Hat 4.8.5-39

@hengyunabc
Copy link
Collaborator

参考这个: #969 ,也可能有其它的agent,比如统计代码覆盖率的,修改了.class,增加了一些field。

@smallchips
Copy link
Author

你好,我不太明白,是需要在复制一份java文件,然后用retransform重新加载吗

@smallchips
Copy link
Author

参考这个: #969 ,也可能有其它的agent,比如统计代码覆盖率的,修改了.class,增加了一些field。

你好,我不太明白,是需要在复制一份java文件,然后用retransform重新加载吗

@hengyunabc
Copy link
Collaborator

参考这个: #969 ,也可能有其它的agent,比如统计代码覆盖率的,修改了.class,增加了一些field。

你好,我不太明白,是需要在复制一份java文件,然后用retransform重新加载吗

可以用 dump 命令把 .class dump下来,然后本地再用其它的反编译软件查看是不是多了其它的 field。

如果 dump 命令执行失败,则可以试下用 https://github.com/hengyunabc/dumpclass 这个工具来 dump class,再做对比。


错误提示的原因是很明确的,就是字节码里 多了/删了 field 。所以要找到具体的字节码来查看原因。


另外,在arthas里执行options dump true ,再执行 watch,在日志里可以看到会把 watch命令处理过的.class保存到本地。也可以再拿出来对比。

@smallchips
Copy link
Author

你好,我用options dump true,再执行 watch,下载下来.class文件,查看了下,类实现了EnhancedInstance接口,并且 方法中有SpyAPI.atEnter,SpyAPI.atExit,SpyAPI.atExceptionExit,现在需要怎么处理才能执行watch不报错,能够查看方法的返回参数

@hengyunabc
Copy link
Collaborator

EnhancedInstance接口

这个不是arthas实现的,如果不是应用代码自己带的,就是有其它的类增强软件生成的。检查对比,有没有新增了field,有没有 lambda的新类名不一致等。

@smallchips
Copy link
Author

你好,我们项目中有依赖skywalking,但是我不确定是不是因为skywalking的原因,因为有两套环境都依赖了skywalking,只有其中一个环境的watch命令不行

org.apache.skywalking
apm-toolkit-logback-1.x
8.4.0

@smallchips
Copy link
Author

那个EnhancedInstance接口是来着skywalking的引用

@hengyunabc
Copy link
Collaborator

那个EnhancedInstance接口是来着skywalking的引用

#1141

@wolfiesonfire
Copy link

Skywalking 给的解决方案
Compatibility with other Java agent bytecode processes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants