无法trace 和watch加载skywalking7 的jar中的类方法 #1141

tz826 · 2020-04-24T06:34:03Z

我已经在 issues 里搜索，没有重复的issue。

环境信息

arthas-boot.jar 或者 as.sh 的版本： 3.2.0
Arthas 版本: 3.2.0
操作系统版本: centos7 x64 1810
目标进程的JVM版本: 1.8
执行arthas-boot的版本: 3.2

重现问题的步骤

trace 类方法报错

期望的结果

What do you expected from the above steps？

展示调用栈及耗时

实际运行的结果

实际运行结果，最好有详细的日志，异常栈。尽量贴文本。

Error during processing the command: null

class类被skywalking做了增强.

jad class结果:

@RestController
public class ItemController
implements ItemApi,
EnhancedInstance,
EnhancedInstance {
@Autowired
private IHolidayItemFacade holidayItemFacade;
private volatile Object _$EnhancedClassField_ws;
public static volatile /* synthetic / InstMethodsInter delegate$7gn76d1;
public static volatile / synthetic / InstMethodsInter delegate$lv6d7p1;
public static volatile / synthetic / ConstructorInter delegate$27t3dk1;
private static final / synthetic / Method cachedValue$BGsZ60W7$h0kumm0;
private static final / synthetic / Method cachedValue$BGsZ60W7$mheqqj2;
private static final / synthetic / Method cachedValue$BGsZ60W7$7b1jhl2;
private static final / synthetic / Method cachedValue$BGsZ60W7$h90el41;
private static final / synthetic / Method cachedValue$BGsZ60W7$o2lomh2;
private static final / synthetic / Method cachedValue$BGsZ60W7$2cp92d3;
private static final / synthetic / Method cachedValue$BGsZ60W7$trthg52;
private static final / synthetic / Method cachedValue$BGsZ60W7$etp7162;
private static final / synthetic / Method cachedValue$BGsZ60W7$20eg8u0;
private static final / synthetic / Method cachedValue$BGsZ60W7$5ftnfq2;
private static final / synthetic / Method cachedValue$BGsZ60W7$pg6p5u1;
private static final / synthetic / Method cachedValue$BGsZ60W7$p6onuj0;
private static final / synthetic / Method cachedValue$BGsZ60W7$avq2ph1;
private static final / synthetic / Method cachedValue$BGsZ60W7$qkdlei1;
private static final / synthetic / Method cachedValue$BGsZ60W7$jb5tsd2;
private static final / synthetic / Method cachedValue$BGsZ60W7$5tcatp1;
private static final / synthetic / Method cachedValue$BGsZ60W7$nird923;
private static final / synthetic / Method cachedValue$BGsZ60W7$n3hau21;
private static final / synthetic / Method cachedValue$BGsZ60W7$hu9rep3;
private static final / synthetic / Method cachedValue$BGsZ60W7$kootqq0;
private static final / synthetic / Method cachedValue$BGsZ60W7$png6703;
private static final / synthetic / Method cachedValue$BGsZ60W7$ijv26b1;
private static final / synthetic / Method cachedValue$BGsZ60W7$trbqar3;
private static final / synthetic / Method cachedValue$BGsZ60W7$j92umg1;
private static final / synthetic / Method cachedValue$BGsZ60W7$itutci2;
private static final / synthetic / Method cachedValue$BGsZ60W7$r72ggd2;
public static volatile / synthetic / InstMethodsInter delegate$8v6lb91;
public static volatile / synthetic / InstMethodsInter delegate$a9tapa0;
public static volatile / synthetic / ConstructorInter delegate$i1erl51;
private static final / synthetic / Method cachedValue$v34oxCGE$h0kumm0;
private static final / synthetic / Method cachedValue$v34oxCGE$mheqqj2;
private static final / synthetic / Method cachedValue$v34oxCGE$7b1jhl2;
private static final / synthetic / Method cachedValue$v34oxCGE$h90el41;
private static final / synthetic / Method cachedValue$v34oxCGE$o2lomh2;
private static final / synthetic / Method cachedValue$v34oxCGE$2cp92d3;
private static final / synthetic / Method cachedValue$v34oxCGE$trthg52;
private static final / synthetic / Method cachedValue$v34oxCGE$etp7162;
private static final / synthetic / Method cachedValue$v34oxCGE$20eg8u0;
private static final / synthetic / Method cachedValue$v34oxCGE$5ftnfq2;
private static final / synthetic / Method cachedValue$v34oxCGE$pg6p5u1;
private static final / synthetic / Method cachedValue$v34oxCGE$p6onuj0;
private static final / synthetic / Method cachedValue$v34oxCGE$avq2ph1;
private static final / synthetic / Method cachedValue$v34oxCGE$qkdlei1;
private static final / synthetic / Method cachedValue$v34oxCGE$jb5tsd2;
private static final / synthetic / Method cachedValue$v34oxCGE$5tcatp1;
private static final / synthetic / Method cachedValue$v34oxCGE$nird923;
private static final / synthetic / Method cachedValue$v34oxCGE$n3hau21;
private static final / synthetic / Method cachedValue$v34oxCGE$hu9rep3;
private static final / synthetic / Method cachedValue$v34oxCGE$kootqq0;
private static final / synthetic / Method cachedValue$v34oxCGE$png6703;
private static final / synthetic / Method cachedValue$v34oxCGE$ijv26b1;
private static final / synthetic / Method cachedValue$v34oxCGE$trbqar3;
private static final / synthetic / Method cachedValue$v34oxCGE$j92umg1;
private static final / synthetic / Method cachedValue$v34oxCGE$itutci2;
private static final / synthetic */ Method cachedValue$v34oxCGE$r72ggd2;

public ItemController() {
    this(null);
    delegate$i1erl51.intercept(this, new Object[0]);
}

private /* synthetic */ ItemController(auxiliary.u2IDERyW u2IDERyW2) {
}

}

把异常信息贴到这里

2020-04-22 15:55:51 [arthas-binding-thread] INFO  c.t.arthas.core.util.ArthasBanner -arthas version: 3.2.0
2020-04-22 15:55:52 [nioEventLoopGroup-2-1] INFO  i.n.handler.logging.LoggingHandler -[id: 0x7adbd703] REGISTERED
2020-04-22 15:55:52 [nioEventLoopGroup-2-1] INFO  i.n.handler.logging.LoggingHandler -[id: 0x7adbd703] BIND: /127.0.0.1:3658
2020-04-22 15:55:52 [nioEventLoopGroup-2-1] INFO  i.n.handler.logging.LoggingHandler -[id: 0x7adbd703, L:/127.0.0.1:3658] ACTIVE
2020-04-22 15:55:52 [nioEventLoopGroup-3-1] INFO  i.n.handler.logging.LoggingHandler -[id: 0x0304f924] REGISTERED
2020-04-22 15:55:52 [nioEventLoopGroup-3-1] INFO  i.n.handler.logging.LoggingHandler -[id: 0x0304f924] BIND: /127.0.0.1:8563
2020-04-22 15:55:52 [nioEventLoopGroup-3-1] INFO  i.n.handler.logging.LoggingHandler -[id: 0x0304f924, L:/127.0.0.1:8563] ACTIVE
2020-04-22 15:55:52 [arthas-binding-thread] INFO  c.t.a.core.server.ArthasBootstrap -as-server listening on network=127.0.0.1;telnet=3658;http=8563;timeout=6000;
2020-04-22 15:55:52 [arthas-binding-thread] INFO  c.t.a.core.server.ArthasBootstrap -as-server started in 1044 ms
2020-04-22 15:55:52 [nioEventLoopGroup-2-1] INFO  i.n.handler.logging.LoggingHandler -[id: 0x7adbd703, L:/127.0.0.1:3658] READ: [id: 0xb03c0d6d, L:/127.0.0.1:3658 - R:/127.0.0.1:49094]
2020-04-22 15:55:52 [nioEventLoopGroup-2-1] INFO  i.n.handler.logging.LoggingHandler -[id: 0x7adbd703, L:/127.0.0.1:3658] READ COMPLETE
2020-04-22 15:55:52 [nioEventLoopGroup-2-2] INFO  c.t.a.core.shell.term.impl.Helper -Loaded arthas keymap file from com/taobao/arthas/core/shell/term/readline/inputrc
2020-04-22 15:56:27 [as-command-execute-daemon] ERROR c.t.a.c.s.system.impl.ProcessImpl -Error during processing the command:
java.lang.ClassFormatError: null
	at sun.instrument.InstrumentationImpl.retransformClasses0(Native Method)
	at sun.instrument.InstrumentationImpl.retransformClasses(InstrumentationImpl.java:144)
	at com.taobao.arthas.core.advisor.Enhancer.enhance(Enhancer.java:303)
	at com.taobao.arthas.core.command.monitor200.EnhancerCommand.enhance(EnhancerCommand.java:112)
	at com.taobao.arthas.core.command.monitor200.EnhancerCommand.process(EnhancerCommand.java:67)
	at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl.process(AnnotatedCommandImpl.java:82)
	at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl.access$100(AnnotatedCommandImpl.java:18)
	at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl$ProcessHandler.handle(AnnotatedCommandImpl.java:111)
	at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl$ProcessHandler.handle(AnnotatedCommandImpl.java:108)
	at com.taobao.arthas.core.shell.system.impl.ProcessImpl$CommandProcessTask.run(ProcessImpl.java:371)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

The text was updated successfully, but these errors were encountered:

hengyunabc · 2020-04-24T07:07:33Z

skywalking 应该是给类增加了 field，导致的问题。skywalking实现机制的问题。

RainElohim · 2020-05-22T07:25:26Z

skywalking 应该是给类增加了 field，导致的问题。skywalking实现机制的问题。

该如何解决呢，才能正常使用controller

hengyunabc · 2020-05-22T09:29:30Z

@RainElohim 试下这个版本: #1184

RainElohim · 2020-05-22T10:09:21Z

@RainElohim 试下这个版本: #1184

尝试了一下，还是不可以，报错跟上面楼主的一样，并且用了#1184这里的包之后，执行trace命令后，再执行其他命令会提示Timer already cancelled. 然后无法退出了就，再开个终端用arthas进入同一个进程后执行命令也是这个，不能正常使用了 @hengyunabc 辛苦

hengyunabc · 2020-05-22T10:37:16Z

@RainElohim 具体的Timer already cancelled日志是怎样的？是从哪里打印出来的？可以稳定重现不？

RainElohim · 2020-05-22T10:40:25Z

@RainElohim 具体的Timer already cancelled日志是怎样的？是从哪里打印出来的？可以稳定重现不？

arthas错误日志里没有关于这个的，是从arthas控制台打印出来的，就是输入arthas命令后出现的，在我这，按我上面的操作稳定复现，我换了两个进程都是
@hengyunabc

hengyunabc · 2020-05-23T17:55:28Z

可能和 skywalking 生成的字节码里，用了一些类似lambda的代码有关。

对于下面的controller，skywalking在每次reTransform之后，字节码里的函数名都不一样。

@Controller
public class TestController {
	@RequestMapping("/test")
	@ResponseBody
	public String test() {
		return "test";
	}
}

第一次jad结果：

@Controller
public class TestController
implements EnhancedInstance,
EnhancedInstance {
    private volatile Object _$EnhancedClassField_ws;
    public static volatile /* synthetic */ InstMethodsInter delegate$dnkuls0;
    public static volatile /* synthetic */ InstMethodsInter delegate$4sa87m1;
    public static volatile /* synthetic */ ConstructorInter delegate$qe910b1;
    private static final /* synthetic */ Method cachedValue$0cEL6pOE$lt7gtc1;
    public static volatile /* synthetic */ InstMethodsInter delegate$mniv240;
    public static volatile /* synthetic */ InstMethodsInter delegate$2tpiec0;
    public static volatile /* synthetic */ ConstructorInter delegate$qemr0h1;
    private static final /* synthetic */ Method cachedValue$WIXtGH4B$lt7gtc1;

    public TestController() {
        this(null);
        delegate$qemr0h1.intercept((Object)this, new Object[0]);
    }

    private /* synthetic */ TestController(auxiliary.0X7gQAYw x7gQAYw) {
    }

    @RequestMapping(value={"/test"})
    @ResponseBody
    public String test() {
        return (String)delegate$2tpiec0.intercept((Object)this, new Object[0], (Callable)new auxiliary.dAQJyyYq(this), cachedValue$WIXtGH4B$lt7gtc1);
    }

    private /* synthetic */ String test$original$OZfDOP8V() {
        return "test";
    }

    static {
        ClassLoader.getSystemClassLoader().loadClass("org.apache.skywalking.apm.dependencies.net.bytebuddy.dynamic.Nexus").getMethod("initialize", Class.class, Integer.TYPE).invoke(null, TestController.class, -708451193);
        cachedValue$WIXtGH4B$lt7gtc1 = TestController.class.getMethod("test", new Class[0]);
    }

    final /* synthetic */ String test$original$OZfDOP8V$accessor$WIXtGH4B() {
        return this.test$original$OZfDOP8V();
    }
}

第二次：

@Controller
public class TestController
implements EnhancedInstance,
EnhancedInstance {
    private volatile Object _$EnhancedClassField_ws;
    public static volatile /* synthetic */ InstMethodsInter delegate$dnkuls0;
    public static volatile /* synthetic */ InstMethodsInter delegate$4sa87m1;
    public static volatile /* synthetic */ ConstructorInter delegate$qe910b1;
    private static final /* synthetic */ Method cachedValue$0cEL6pOE$lt7gtc1;
    public static volatile /* synthetic */ InstMethodsInter delegate$8mqi1i0;
    public static volatile /* synthetic */ InstMethodsInter delegate$igef700;
    public static volatile /* synthetic */ ConstructorInter delegate$hqtp9d0;
    private static final /* synthetic */ Method cachedValue$6iYRJM1N$lt7gtc1;

    public TestController() {
        this(null);
        delegate$hqtp9d0.intercept((Object)this, new Object[0]);
    }

    private /* synthetic */ TestController(auxiliary.YL8stOUn yL8stOUn) {
    }

    @RequestMapping(value={"/test"})
    @ResponseBody
    public String test() {
        return (String)delegate$igef700.intercept((Object)this, new Object[0], (Callable)new auxiliary.5212MTs4(this), cachedValue$6iYRJM1N$lt7gtc1);
    }

    private /* synthetic */ String test$original$FiEFBU6M() {
        return "test";
    }

    static {
        ClassLoader.getSystemClassLoader().loadClass("org.apache.skywalking.apm.dependencies.net.bytebuddy.dynamic.Nexus").getMethod("initialize", Class.class, Integer.TYPE).invoke(null, TestController.class, -1004178659);
        cachedValue$6iYRJM1N$lt7gtc1 = TestController.class.getMethod("test", new Class[0]);
    }

    final /* synthetic */ String test$original$FiEFBU6M$accessor$6iYRJM1N() {
        return this.test$original$FiEFBU6M();
    }
}

可以看到，(Callable)new auxiliary.5212MTs4(this) 这里总是变化，这里可能是jvm内部verify失败的原因。

参考： lambda表达式导致arthas无法redefine的问题 https://zhuanlan.zhihu.com/p/91756092

RainElohim · 2020-05-25T02:19:18Z

可能和 skywalking 生成的字节码里，用了一些类似lambda的代码有关。

对于下面的controller，skywalking在每次reTransform之后，字节码里的函数名都不一样。

@Controller
public class TestController {
	@RequestMapping("/test")
	@ResponseBody
	public String test() {
		return "test";
	}
}

第一次jad结果：

@Controller
public class TestController
implements EnhancedInstance,
EnhancedInstance {
    private volatile Object _$EnhancedClassField_ws;
    public static volatile /* synthetic */ InstMethodsInter delegate$dnkuls0;
    public static volatile /* synthetic */ InstMethodsInter delegate$4sa87m1;
    public static volatile /* synthetic */ ConstructorInter delegate$qe910b1;
    private static final /* synthetic */ Method cachedValue$0cEL6pOE$lt7gtc1;
    public static volatile /* synthetic */ InstMethodsInter delegate$mniv240;
    public static volatile /* synthetic */ InstMethodsInter delegate$2tpiec0;
    public static volatile /* synthetic */ ConstructorInter delegate$qemr0h1;
    private static final /* synthetic */ Method cachedValue$WIXtGH4B$lt7gtc1;

    public TestController() {
        this(null);
        delegate$qemr0h1.intercept((Object)this, new Object[0]);
    }

    private /* synthetic */ TestController(auxiliary.0X7gQAYw x7gQAYw) {
    }

    @RequestMapping(value={"/test"})
    @ResponseBody
    public String test() {
        return (String)delegate$2tpiec0.intercept((Object)this, new Object[0], (Callable)new auxiliary.dAQJyyYq(this), cachedValue$WIXtGH4B$lt7gtc1);
    }

    private /* synthetic */ String test$original$OZfDOP8V() {
        return "test";
    }

    static {
        ClassLoader.getSystemClassLoader().loadClass("org.apache.skywalking.apm.dependencies.net.bytebuddy.dynamic.Nexus").getMethod("initialize", Class.class, Integer.TYPE).invoke(null, TestController.class, -708451193);
        cachedValue$WIXtGH4B$lt7gtc1 = TestController.class.getMethod("test", new Class[0]);
    }

    final /* synthetic */ String test$original$OZfDOP8V$accessor$WIXtGH4B() {
        return this.test$original$OZfDOP8V();
    }
}

第二次：

@Controller
public class TestController
implements EnhancedInstance,
EnhancedInstance {
    private volatile Object _$EnhancedClassField_ws;
    public static volatile /* synthetic */ InstMethodsInter delegate$dnkuls0;
    public static volatile /* synthetic */ InstMethodsInter delegate$4sa87m1;
    public static volatile /* synthetic */ ConstructorInter delegate$qe910b1;
    private static final /* synthetic */ Method cachedValue$0cEL6pOE$lt7gtc1;
    public static volatile /* synthetic */ InstMethodsInter delegate$8mqi1i0;
    public static volatile /* synthetic */ InstMethodsInter delegate$igef700;
    public static volatile /* synthetic */ ConstructorInter delegate$hqtp9d0;
    private static final /* synthetic */ Method cachedValue$6iYRJM1N$lt7gtc1;

    public TestController() {
        this(null);
        delegate$hqtp9d0.intercept((Object)this, new Object[0]);
    }

    private /* synthetic */ TestController(auxiliary.YL8stOUn yL8stOUn) {
    }

    @RequestMapping(value={"/test"})
    @ResponseBody
    public String test() {
        return (String)delegate$igef700.intercept((Object)this, new Object[0], (Callable)new auxiliary.5212MTs4(this), cachedValue$6iYRJM1N$lt7gtc1);
    }

    private /* synthetic */ String test$original$FiEFBU6M() {
        return "test";
    }

    static {
        ClassLoader.getSystemClassLoader().loadClass("org.apache.skywalking.apm.dependencies.net.bytebuddy.dynamic.Nexus").getMethod("initialize", Class.class, Integer.TYPE).invoke(null, TestController.class, -1004178659);
        cachedValue$6iYRJM1N$lt7gtc1 = TestController.class.getMethod("test", new Class[0]);
    }

    final /* synthetic */ String test$original$FiEFBU6M$accessor$6iYRJM1N() {
        return this.test$original$FiEFBU6M();
    }
}

可以看到，(Callable)new auxiliary.5212MTs4(this) 这里总是变化，这里可能是jvm内部verify失败的原因。

参考： lambda表达式导致arthas无法redefine的问题 https://zhuanlan.zhihu.com/p/91756092

我的jdk是：
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)

但是还出现这个问题，那么有没有什么更具体的解决方案

hengyunabc · 2020-05-25T07:03:14Z

@RainElohim 我觉得这个是 skywalking的实现的问题。我测试用 btrace也无办法正常工作。测试脚本：

import org.openjdk.btrace.core.annotations.BTrace;
import org.openjdk.btrace.core.annotations.Injected;
import org.openjdk.btrace.core.annotations.OnMethod;
import org.openjdk.btrace.core.annotations.ProbeClassName;
import org.openjdk.btrace.core.annotations.ProbeMethodName;
import org.openjdk.btrace.core.annotations.Self;
import org.openjdk.btrace.core.annotations.ServiceType;
import org.openjdk.btrace.services.impl.Printer;

@BTrace
public class AllMethods {
    @Injected(ServiceType.RUNTIME)
    private static Printer printer;

    @OnMethod(
            clazz = "com.example.demo.TestController",
            method = "test"
    )
    public static void m(@Self Object o, @ProbeClassName String probeClass, @ProbeMethodName String probeMethod) {
        printer.println("this = " + o);
        printer.print("entered " + probeClass);
        printer.println("." + probeMethod);
    }
}

不清楚为什么 skywalking 的实现里每次函数名都变化。

hengyunabc · 2020-05-25T07:42:50Z

@RainElohim 另外，每执行一次 jad，即触发一次 java.lang.instrument.Instrumentation.retransformClasses(Class<?>...) ，就会产生一个新的 auxiliary类，显然是有问题的。

执行多次jad之后，用sc命令可以查找到。

[arthas@82549]$ sc com.example.demo.TestController*
com.example.demo.TestController
com.example.demo.TestController$auxiliary$4ZbpDXX9
com.example.demo.TestController$auxiliary$5RcFgXtT
com.example.demo.TestController$auxiliary$DcNiUd0D
com.example.demo.TestController$auxiliary$EgzoyDwo
com.example.demo.TestController$auxiliary$FgWo5SOK
com.example.demo.TestController$auxiliary$J7p46QSU
com.example.demo.TestController$auxiliary$PYsPbDsI
com.example.demo.TestController$auxiliary$Pc3KSnn2
com.example.demo.TestController$auxiliary$PnenKrBq
com.example.demo.TestController$auxiliary$Y6Hg5yqk
com.example.demo.TestController$auxiliary$YS5oCqQ3
com.example.demo.TestController$auxiliary$aNrdhsqk
com.example.demo.TestController$auxiliary$bEhoPh9t
com.example.demo.TestController$auxiliary$eMmayhvM

RainElohim · 2020-05-25T09:14:04Z

@RainElohim 另外，每执行一次 jad，即触发一次 java.lang.instrument.Instrumentation.retransformClasses(Class<?>...) ，就会产生一个新的 auxiliary类，显然是有问题的。

执行多次jad之后，用sc命令可以查找到。

[arthas@82549]$ sc com.example.demo.TestController*
com.example.demo.TestController
com.example.demo.TestController$auxiliary$4ZbpDXX9
com.example.demo.TestController$auxiliary$5RcFgXtT
com.example.demo.TestController$auxiliary$DcNiUd0D
com.example.demo.TestController$auxiliary$EgzoyDwo
com.example.demo.TestController$auxiliary$FgWo5SOK
com.example.demo.TestController$auxiliary$J7p46QSU
com.example.demo.TestController$auxiliary$PYsPbDsI
com.example.demo.TestController$auxiliary$Pc3KSnn2
com.example.demo.TestController$auxiliary$PnenKrBq
com.example.demo.TestController$auxiliary$Y6Hg5yqk
com.example.demo.TestController$auxiliary$YS5oCqQ3
com.example.demo.TestController$auxiliary$aNrdhsqk
com.example.demo.TestController$auxiliary$bEhoPh9t
com.example.demo.TestController$auxiliary$eMmayhvM

感谢解答，辛苦了。我去skywalking下面提问下，看看有没有解决办法，apache/skywalking#4820

RainElohim · 2020-05-25T09:47:09Z

@RainElohim 另外，每执行一次 jad，即触发一次 java.lang.instrument.Instrumentation.retransformClasses(Class<?>...) ，就会产生一个新的 auxiliary类，显然是有问题的。

执行多次jad之后，用sc命令可以查找到。

[arthas@82549]$ sc com.example.demo.TestController*
com.example.demo.TestController
com.example.demo.TestController$auxiliary$4ZbpDXX9
com.example.demo.TestController$auxiliary$5RcFgXtT
com.example.demo.TestController$auxiliary$DcNiUd0D
com.example.demo.TestController$auxiliary$EgzoyDwo
com.example.demo.TestController$auxiliary$FgWo5SOK
com.example.demo.TestController$auxiliary$J7p46QSU
com.example.demo.TestController$auxiliary$PYsPbDsI
com.example.demo.TestController$auxiliary$Pc3KSnn2
com.example.demo.TestController$auxiliary$PnenKrBq
com.example.demo.TestController$auxiliary$Y6Hg5yqk
com.example.demo.TestController$auxiliary$YS5oCqQ3
com.example.demo.TestController$auxiliary$aNrdhsqk
com.example.demo.TestController$auxiliary$bEhoPh9t
com.example.demo.TestController$auxiliary$eMmayhvM

那边说是arthas运行时的问题。。。具体看apache/skywalking#4820

hengyunabc · 2020-05-25T11:59:17Z

和arthas无关，提供一个中文 only 的demo： https://github.com/hengyunabc/skywalking-error-demo

kylixs · 2020-06-02T03:48:35Z

raphw/byte-buddy#829
可能是与ByteBuddy的 MethodDelegation方式有关系

参考这篇
https://www.infoq.com/articles/Easily-Create-Java-Agents-with-ByteBuddy/

kylixs · 2020-06-04T08:42:12Z

提交了一个PR，增加ByteBuddy class cache特性，解决retransform失败的问题。
apache/skywalking#4858

https://github.com/hengyunabc/skywalking-error-demo 这个原型验证通过，可以使用Arthas注入并执行trace命令。

[arthas@42451]$ trace *TestController *
trace *TestController *
Press Q or Ctrl+C to abort.
Affect(class count: 1 , method count: 7) cost in 164 ms, listenerId: 1
`---ts=2020-06-04 15:38:58;thread_name=http-nio-8080-exec-1;id=8e;is_daemon=true;priority=5;TCCL=org.springframework.boot.web.embedded.tomcat.TomcatEmbeddedWebappClassLoader@c190cfc
    `---[3.650681ms] com.example.demo.TestController:test()
        +---[0.069111ms] com.example.demo.TestController$auxiliary$VUHAiX6C:<init>() #-1
        `---[3.096252ms] org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter:intercept() #-1
            +---[0.019431ms] com.example.demo.TestController:getSkyWalkingDynamicField()
            `---[0.105957ms] com.example.demo.TestController:test$original$83418DTC$accessor$dyYqLLCC()
                `---[0.053324ms] com.example.demo.TestController:test$original$83418DTC() #-1
                    `---[0.010328ms] com.example.demo.TestController:test$original$83418DTC()

kylixs · 2020-09-30T06:26:49Z

SkyWalking: Compatible with other javaagent bytecode processing
https://github.com/apache/skywalking/blob/master/docs/en/FAQ/Compatible-with-other-javaagent-bytecode-processing.md

zhegexiaohuozi · 2021-11-17T09:00:21Z

和arthas无关，提供一个中文 only 的demo： https://github.com/hengyunabc/skywalking-error-demo

中文Only，干的漂亮！

acmug · 2022-12-26T10:25:10Z

你好，我正好也遇到这个问题了，trace失败，skywalking agent版本是8.11，看skywalking.jar中java版本是1.8.0_312，服务器上java 版本是1.8.0_292，这个可能是什么问题呢
@kylixs @RainElohim 帮忙看下这个问题吧，感谢

kiwi-field · 2024-06-17T08:39:41Z

最终怎么解决的，我也遇到了同样问题，项目中同时使用了skywalking和arthas

tz826 changed the title ~~无法trace 和watch加载skywalking 的jar中的类方法~~ 无法trace 和watch加载skywalking7 的jar中的类方法 Apr 24, 2020

hengyunabc mentioned this issue May 23, 2020

基于bytekit的arthas版本测试 #1184

Closed

RainElohim mentioned this issue May 25, 2020

when use skywalking agent ,arthas is can‘t work well apache/skywalking#4820

Closed

hengyunabc added the not-a-bug label May 25, 2020

hengyunabc closed this as completed May 25, 2020

This was referenced May 25, 2020

skywalking 6.4.0 版本兼容性 #898

Closed

【trace使用】对于springboot的control类无法使用 #1147

Closed

hengyunabc added the question-answered label May 25, 2020

flyluv mentioned this issue Sep 15, 2020

使用trace命令报错UnsupportedOperationException #1502

Closed

hengyunabc mentioned this issue Sep 17, 2020

执行trace命令抛异常，Enhancer error，java.lang.ClassFormatError: null #1506

Closed

1 task

hengyunabc mentioned this issue Jul 10, 2021

watch命令报错class redefinition failed: attempted to change the schema (add/remove fields) #1847

Closed

1 task

zhuxiaokang0223 mentioned this issue Nov 15, 2021

mc 程序包auxiliary不存在 #2027

Open

1 task

dreaming-alive mentioned this issue Dec 14, 2021

引入 skywalking 后无法 retransform 被 skywalking增强的类 #2051

Closed

1 task

hengyunabc mentioned this issue Apr 26, 2022

class redefinition failed: attempted to change the schema (add/remove fields) 分析排查 #2165

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

无法trace 和watch加载skywalking7 的jar中的类方法 #1141

无法trace 和watch加载skywalking7 的jar中的类方法 #1141

tz826 commented Apr 24, 2020

hengyunabc commented Apr 24, 2020

RainElohim commented May 22, 2020

hengyunabc commented May 22, 2020

RainElohim commented May 22, 2020 •

edited

Loading

hengyunabc commented May 22, 2020

RainElohim commented May 22, 2020

hengyunabc commented May 23, 2020

RainElohim commented May 25, 2020

hengyunabc commented May 25, 2020

hengyunabc commented May 25, 2020

RainElohim commented May 25, 2020

RainElohim commented May 25, 2020

hengyunabc commented May 25, 2020

kylixs commented Jun 2, 2020

kylixs commented Jun 4, 2020

kylixs commented Sep 30, 2020 •

edited

Loading

zhegexiaohuozi commented Nov 17, 2021

acmug commented Dec 26, 2022 •

edited

Loading

kiwi-field commented Jun 17, 2024

无法trace 和watch加载skywalking7 的jar中的类方法 #1141

无法trace 和watch加载skywalking7 的jar中的类方法 #1141

Comments

tz826 commented Apr 24, 2020

环境信息

重现问题的步骤

期望的结果

实际运行的结果

hengyunabc commented Apr 24, 2020

RainElohim commented May 22, 2020

hengyunabc commented May 22, 2020

RainElohim commented May 22, 2020 • edited Loading

hengyunabc commented May 22, 2020

RainElohim commented May 22, 2020

hengyunabc commented May 23, 2020

RainElohim commented May 25, 2020

hengyunabc commented May 25, 2020

hengyunabc commented May 25, 2020

RainElohim commented May 25, 2020

RainElohim commented May 25, 2020

hengyunabc commented May 25, 2020

kylixs commented Jun 2, 2020

kylixs commented Jun 4, 2020

kylixs commented Sep 30, 2020 • edited Loading

zhegexiaohuozi commented Nov 17, 2021

acmug commented Dec 26, 2022 • edited Loading

kiwi-field commented Jun 17, 2024

RainElohim commented May 22, 2020 •

edited

Loading

kylixs commented Sep 30, 2020 •

edited

Loading

acmug commented Dec 26, 2022 •

edited

Loading