[Proposal] Introduce auto-instrument Golang Agent

[mrproliu]

Hello, the community,

I would like to introduce the automatic instrument Golang Agent for distributed tracing, which aims to simplify the process of integrating tracing capabilities into Golang applications with minimal code changes.
This agent will make it easier for developers to adopt distributed tracing in their projects, improving observability and making it easier to diagnose and resolve performance issues.

To provide more details on this proposal, I have created a document that outlines the motivation, design, and implementation of the automatic enhancement Golang Agent. You can find the document here: https://docs.google.com/document/d/1jMzCrr1vIEWvv-PJG9XiqJ6BGIB1wq4tvwyP4XL9lOI/edit?usp=sharing

Welcome feedback, suggestions, and questions from the community.

[wu-sheng]

I have created the repository for the go agent, https://github.com/apache/skywalking-go

[wu-sheng]

The first online discussion about this in CN, https://www.bilibili.com/video/BV1a54y1F7Qd/?vd_source=1c75fbed43a98ded83f96de5f6ff66e8

For English readers, the docs have covered all cases. If you have any questions, welcome to ask here.

[jcchavezs]

I checked the doc and it looks very interesting. I have some concerns tho which maybe are out of the scope of the autoinstrumenter.

As far as I understand we will be infer the context of the execution path so we don't rely on the context being propagated by the user code. I wonder if that conceives the idea that people could expect to access tracing data somewhere else, for example: this agent will provide a set of libraries for instrumentation but what if I want to use my own with my own tags, I should be able to still plug a middleware and do the job if the data is in the context. Also I should be able to mutate the context if needed (see https://github.com/openzipkin/zipkin-go/blob/master/middleware/http/server.go#L145) or imagine I want to restart the trace because I will go in a async process (e.g. messaging pipeline), will I be able to do so in the context or should I wait for the repo to include all those frameworks out there? will the plugin architecture allow me to do so?

I guess my other question would be how will all this be tests, I would expect the document to explain the testing harness to validate my instrumentation code but that can come later. Also, wonder how would fallback mechanism would work? Like if instrumentation code break the normal flow of the app, will I be able to recover and continue functioning like usual?

Finally I wonder what would be the impact of instrumenting the calls always. For example, in usual code instrumentation you can decide to early return as soon as possible whenever the instrumentation is disabled (see https://github.com/openzipkin/zipkin-go/blob/master/middleware/http/server.go#L143), will we be able to do the same here since that decision is made at runtime?

Anyways great insights here.

[mrproliu]

imagine I want to restart the trace because I will go in a async process (e.g. messaging pipeline), will I be able to do so in the context or should I wait for the repo to include all those frameworks out there? will the plugin architecture allow me to do so?

For messaging pipeline(channel concept in the Golang), we can enhance the data structures within the messages by Enhanced Instance concept. When storing messages in the queue, we capture a snapshot of the current Tracing Context and save the data. Upon retrieval from the data queue, we get the Tracing Context snapshot from the message. However, this method is not applicable to basic data structures, such as int, float64, etc. The next consideration is whether to continue using the current Segment or to create a new Segment.

I would expect the document to explain the testing harness to validate my instrumentation code but that can come later.

I will employ the E2E (end-to-end) functionality to monitor the execution status of plugins, similar to a Java agent. This requires creating a project using the framework and writing the expected file for completion. At the moment, I haven't come up with an effective debugging method for plugin development.

Like if instrumentation code break the normal flow of the app, will I be able to recover and continue functioning like usual?

Yes, It will recover. You can see the bridge method in the method intercepter section.

will we be able to do the same here since that decision is made at runtime?

Yes, we can do the same thing. In the go2sky, if they create a noop span, then it could be early return as soon as possible.

[wu-sheng]

Docs for this new project, https://skywalking.apache.org/docs/skywalking-go/next/readme/

We plan to release the first preview version at the end of this month

[wu-sheng]

I am going to unpin and lock this discussion. As it is going to official release soon.